智能神经假体：心理与技术问题

近年来,微系统技术、神经计算和生物系统等高新技术的发展,为新型植入式智能神经假体装置的发展、生产和应用提供了巨大的可能。其中一项特别引人关注的进展就是智能视觉假体。虽然通过中枢视觉系统到知觉域的机制还未完全了解,但视觉假体是通过电刺激视觉通路的不同部位来产生有用的视觉感受。许多年前,哲学和心理物理学研究都强调视觉感受可能是由不同的原因所引起。为了研制视觉假体,我们使用两张映射图来描述中枢视觉系统的功能是如何实现的。我们提出一种具有自学习功能且包括几个阶段的视网膜编码器（retina encoder,RE）。视网膜模块的RE模拟视网膜的功能并假定其实现了模式P1从物理域到神经域的映射操作M1。与此相对应的,中枢视觉系统模块则实现另一个映射操作M2,即M1在神经域的输出信号被转换到知觉域而产生视觉感受P2。在设定的迭代和基于感受的学习过程中,一名拥有正常视力的被试参与实验来提供由P1而产生P2的感性的相似性估计作为学习算法的输入,该算法反过来调整RE的参数矢量使得P2足够接近P1。更具体的说,全部RE时空滤波器可以借助遗传算法与被试交互式的实现调节。另一种视网膜编码器RE^＊使用特殊的时空滤波器组、决策树算法以及仿真的微小眼动算法组合构成。这种新编码器可以显著提高调节的结果。近期RE的研究集中在试图增强RE本身基于P1模式预处理的特性、模式分割、时间模式表达的选择性调整以及刺激电极簇的选择性控制。随着神经假体技术研究的不断深入和成熟,将会有越来越多的人受益。     

Olfaction spontaneously highlights visual saliency map

Attention is intrinsic to our perceptual representations of sensory inputs. Best characterized in the visual domain, it is typically depicted as a spotlight moving over a saliency map that topographically encodes strengths of visual features and feedback modulations over the visual scene. By introducing smells to two well-established attentional paradigms, the dot-probe and the visual-search paradigms, we find that a smell reflexively directs attention to the congruent visual image and facilitates visual search of that image without the mediation of visual imagery. Furthermore, such effect is independent of, and can override, top-down bias. We thus propose that smell quality acts as an object feature whose presence enhances the perceptual saliency of that object, thereby guiding the spotlight of visual attention. Our discoveries provide robust empirical evidence for a multimodal saliency map that weighs not only visual but also olfactory inputs.     

Orientation enhancement in early visual processing can explain time course of brightness contrast and White’s illusion

Dynamics of orientation tuning in V1 indicates that computational model of V1 should not only comprise of bank of static spatially oriented filters but also include the contribution for dynamical response facilitation or suppression along orientation. Time evolution of orientation response in V1 can emerge due to time- dependent excitation and lateral inhibition in the orientation domain. Lateral inhibition in the orientation domain suggests that Ernst Mach’s proposition can be applied for the enhancement of initial orientation distribution that is generated due to interaction of visual stimulus with spatially oriented filters and subcortical temporal filter. Oriented spatial filtering that appears much early ( $<$ 70 ms) in the sequence of visual information processing can account for many of the brightness illusions observed at steady state. It is therefore expected that time evolution of orientation response might be reflecting in the brightness percept over time. Our numerical study suggests that only spatio-temporal filtering at early phase can explain experimentally observed temporal dynamics of brightness contrast illusion. But, enhancement of orientation response at early phase of visual processing is the key mechanism that can guide visual system to predict the brightness by “Max-rule” or “Winner Takes All” (WTA) estimation and thus producing White’s illusions at any exposure.     

Geometry and the visual brain.

Receptive fields structure of neurons in primary visual cortex suggests that they process visual stimuli in the frequency domain, in a way similar to the frequency analysis performed in the auditory system. As a consequence, both psychophysicists and electrophysiologists have long probed the visual system using extended sine wave gratings that are well localized in the frequency domain but poorly defined in visual space. Meanwhile, how the brain processes the geometrical properties and the spatial and temporal relationships between stimulus parts has received less attention. Recent progress in visual neuroscience that uncovered long-range horizontal connections between cortical neurons and revealed the complex architecture of primary visual cortex and feedback connectivity led to new insights concerned with the processing of geometrical properties of visual stimuli in V1. This paper presents a short historical perspective of the emergence of new issues related to the cortical architecture and its functional consequences on the processing of geometrical properties.     

Fate mapping of Drosophila embryonic mitotic domain 20 reveals that the larval visual system is derived from a subdomain of a few cells.

In an attempt to study the fates of cells in the dorsal head region of Drosophila embryos at gastrulation, we used the photoactivated gene expression system to mark small numbers of cells in selected mitotic domains. We found that mitotic domain 20, which is a cluster of approximately 30 cells on the dorsal posterior surface, gives rise to various ectodermal cell types in the head, including dorsal pouch epithelium, the optic lobe, and head sensory organs, including Bolwig's organ, the larval photoreceptor organ. We found that the optic lobe and larval photoreceptors share the same origin of a few adjacent cells near the center of mitotic domain 20, suggesting that within the mitotic domain, there is a subdomain from which the larval visual system is specified. In addition to the components of the larval visual system, this central region of mitotic domain 20 also generates a part of the eye-antennal disc placode; cells that gives rise to the adult visual system. We also observed that a significant amount of cell death occurred within this domain and used cell ablation experiments to determine the ability of the embryo to compensate for cell loss.     

Removing high contrast artifacts via digital inpainting in cryo-electron tomography: an application of compressed sensing

To cope with poor quality in cryo-electron tomography images, electron-dense markers, such as colloidal goldbeads, are often used to assist image registration and analysis algorithms. However, these markers can create artifacts that occlude a specimen due to their high contrast, which can also cause failure of some image processing algorithms. One way of reducing these artifacts is to replace high contrast objects with pixel densities that blend into the surroundings in the projection domain before volume reconstruction. In this paper, we propose digital inpainting via compressed sensing (CS) as a new method to achieve this goal. We show that cryo-ET projections are sparse in the discrete cosine transform (DCT) domain, and, by finding the sparsest DCT domain decompositions given uncorrupted pixels, we can fill in the missing pixel values that are occluded by high contrast objects without discontinuities. Our method reduces visual artifacts both in projections and in tomograms better than conventional algorithms, such as polynomial interpolation and random noise inpainting.     

The Temporal Dynamics of Early Visual Cortex Involvement in Behavioral Priming

Transcranial magnetic stimulation (TMS) allows for non-invasive interference with ongoing neural processing. Applied in a chronometric design over early visual cortex (EVC), TMS has proved valuable in indicating at which particular time point EVC must remain unperturbed for (conscious) vision to be established. In the current study, we set out to examine the effect of EVC TMS across a broad range of time points, both before (pre-stimulus) and after (post-stimulus) the onset of symbolic visual stimuli. Behavioral priming studies have shown that the behavioral impact of a visual stimulus can be independent from its conscious perception, suggesting two independent neural signatures. To assess whether TMS-induced suppression of visual awareness can be dissociated from behavioral priming in the temporal domain, we thus implemented three different measures of visual processing, namely performance on a standard visual discrimination task, a subjective rating of stimulus visibility, and a visual priming task. To control for non-neural TMS effects, we performed electrooculographical recordings, placebo TMS (sham), and control site TMS (vertex). Our results suggest that, when considering the appropriate control data, the temporal pattern of EVC TMS disruption on visual discrimination, subjective awareness and behavioral priming are not dissociable. Instead, TMS to EVC disrupts visual perception holistically, both when applied before and after the onset of a visual stimulus. The current findings are discussed in light of their implications on models of visual awareness and (subliminal) priming.     

Shape Similarity,Better than Semantic Membership,Accounts for the Structure of Visual Object Representations in a Population of Monkey Inferotemporal Neurons

The anterior inferotemporal cortex (IT) is the highest stage along the hierarchy of visual areas that, in primates, processes visual objects. Although several lines of evidence suggest that IT primarily represents visual shape information, some recent studies have argued that neuronal ensembles in IT code the semantic membership of visual objects (i.e., represent conceptual classes such as animate and inanimate objects). In this study, we investigated to what extent semantic, rather than purely visual information, is represented in IT by performing a multivariate analysis of IT responses to a set of visual objects. By relying on a variety of machine-learning approaches (including a cutting-edge clustering algorithm that has been recently developed in the domain of statistical physics), we found that, in most instances, IT representation of visual objects is accounted for by their similarity at the level of shape or, more surprisingly, low-level visual properties. Only in a few cases we observed IT representations of semantic classes that were not explainable by the visual similarity of their members. Overall, these findings reassert the primary function of IT as a conveyor of explicit visual shape information, and reveal that low-level visual properties are represented in IT to a greater extent than previously appreciated. In addition, our work demonstrates how combining a variety of state-of-the-art multivariate approaches, and carefully estimating the contribution of shape similarity to the representation of object categories, can substantially advance our understanding of neuronal coding of visual objects in cortex.     

Testing Neuronal Accounts of Anisotropic Motion Perception with Computational Modelling

There is an over-representation of neurons in early visual cortical areas that respond most strongly to cardinal (horizontal and vertical) orientations and directions of visual stimuli, and cardinal- and oblique-preferring neurons are reported to have different tuning curves. Collectively, these neuronal anisotropies can explain two commonly-reported phenomena of motion perception – the oblique effect and reference repulsion – but it remains unclear whether neuronal anisotropies can simultaneously account for both perceptual effects. We show in psychophysical experiments that reference repulsion and the oblique effect do not depend on the duration of a moving stimulus, and that brief adaptation to a single direction simultaneously causes a reference repulsion in the orientation domain, and the inverse of the oblique effect in the direction domain. We attempted to link these results to underlying neuronal anisotropies by implementing a large family of neuronal decoding models with parametrically varied levels of anisotropy in neuronal direction-tuning preferences, tuning bandwidths and spiking rates. Surprisingly, no model instantiation was able to satisfactorily explain our perceptual data. We argue that the oblique effect arises from the anisotropic distribution of preferred directions evident in V1 and MT, but that reference repulsion occurs separately, perhaps reflecting a process of categorisation occurring in higher-order cortical areas.     

Music and video iconicity: theory and experimental design

Experimental studies on the relationship between quasi-musical patterns and visual movement have largely focused on either referential, associative aspects or syntactical, accent-oriented alignments. Both of these are very important, however, between the referential and areferential lays a domain where visual pattern perceptually connects to musical pattern; this is iconicity. The temporal syntax of accent structures in iconicity is hypothesized to be important. Beyond that, a multidimensional visual space connects to musical patterning through mapping of visual time/space to musical time/magnitudes. Experimental visual and musical correlates are presented and comparisons to previous research provided.     

Ascidian arrestin (Ci-arr), the origin of the visual and nonvisual arrestins of vertebrate.

Arrestin is one of the key proteins for the termination of G protein signaling. Activated G protein-coupled receptors (GPCRs) are specifically phosphorylated by G protein-coupled receptor kinases (GRKs) and then bind to arrestins to preclude the receptor/G protein interaction, resulting in quenching of the following signal transduction. Vertebrates possess two types of arrestin; visual arrestin expressed exclusively in photoreceptor cells in retinae and pineal organs, and beta-arrestin, which is expressed ubiquitously. Unlike visual arrestin, beta-arrestin contains the clathrin-binding domain at the C-terminus, responsible for the agonist-induced internalization of GPCRs. Here, we isolated a novel arrestin gene (Ci-arr) from the primitive chordate, the ascidian Ciona intestinalis larvae. The deduced amino acid sequence suggests that Ci-Arr be closely related to vertebrate arrestins. Interestingly, this arrestin has the feature of both visual and beta-arrestin. Whereas the expression of Ci-arr was restricted to the photoreceptors in the larvae similarly to visual arrestin, the gene product, containing the clathrin-binding domain, promoted the GPCR internalization in HEK293tsA201 cells similarly to beta-arrestin. The phylogenetic tree shows that Ci-Arr is branched from a common root of visual and beta-arrestins. Southern analysis suggests that the Ciona genome contains only one gene for the arrestin family. These results suggest that the visual and beta-arrestin genes were generated by the duplication of the prototypical arrestin gene like Ci-arr in the early evolution of vertebrates.     

Teneurin 2 is expressed by the neurons of the thalamofugal visual system in situ and promotes homophilic cell-cell adhesion in vitro

The transmembrane glycoprotein teneurin 2 is expressed by neurons in the developing avian thalamofugal visual system at periods that correspond with target recognition and synaptogenesis. Partial and full-length teneurin 2 constructs were expressed in cell lines in vitro. Expression of the cytoplasmic domain is required for the induction of filopodia, the transport of teneurin 2 into neurites and the co-localization of teneurin 2 with the cortical actin cytoskeleton. In addition, expression of the extracellular domain of teneurin 2 by HT1080 cells induced cell aggregation, and the extracellular domain of teneurin 2 became concentrated at sites of cell-cell contact in neuroblastoma cells. These observations indicate that the homophilic binding of teneurin 2 may play a role in the development of specific neuronal circuits in the developing visual system.     

Spectral Characteristics of Phase Sensitivity and Discharge Rate of Neurons in the Ascending Tectofugal Visual System

Drifting gratings can modulate the activity of visual neurons at the temporal frequency of the stimulus. In order to characterize the temporal frequency modulation in the cat’s ascending tectofugal visual system, we recorded the activity of single neurons in the superior colliculus, the suprageniculate nucleus, and the anterior ectosylvian cortex during visual stimulation with drifting sine-wave gratings. In response to such stimuli, neurons in each structure showed an increase in firing rate and/or oscillatory modulated firing at the temporal frequency of the stimulus (phase sensitivity). To obtain a more complete characterization of the neural responses in spatiotemporal frequency domain, we analyzed the mean firing rate and the strength of the oscillatory modulations measured by the standardized Fourier component of the response at the temporal frequency of the stimulus. We show that the spatiotemporal stimulus parameters that elicit maximal oscillations often differ from those that elicit a maximal discharge rate. Furthermore, the temporal modulation and discharge-rate spectral receptive fields often do not overlap, suggesting that the detection range for visual stimuli provided jointly by modulated and unmodulated response components is larger than the range provided by a one response component.     

Visual, auditory, and bimodal activity in the banks of the lateral suprasylvian sulcus in the cat

In addition to visually driven cells we found within the lateral suprasylvian visual cortex of cats a considerable number of auditory and/or bimodal cells. Most of the visually driven cells were direction and orientation selective with responses that were neither highly stimulus time locked nor very stable. Most of the auditory responses were also not very stable, had relatively high thresholds and were readily habituated. Previous studies have suggested that populations of cells within the lateral suprasylvian area are specialized for the analysis of optic flow fields. Given that a remarkable proportion of cells within this area can be also driven by auditory stimuli we hypothesize that the "optic flow" model may be extended to the bimodal domain rather than restricted to visual clues only. This, however, remains to be corroborated experimentally.     

Cybernetic models of pattern sensitive units in the visual system

Summary Pattern-specific response in the visual system is represented in terms of a mathematical model and as the response of corresponding sequential circuits. The mathematical model employs standard relations of propositional calculus, with extension to the time domain. It is shown that pattern sensitive units of many types can be generated by successive applications of a generalised contrast operator together with a spatial summation operator. The implications of the models are interpreted in neural terms.     

Saliency computation via whitened frequency band selection

Many saliency computational models have been proposed to simulate bottom-up visual attention mechanism of human visual system. However, most of them only deal with certain kinds of images or aim at specific applications. In fact, human beings have the ability to correctly select attentive focuses of objects with arbitrary sizes within any scenes. This paper proposes a new bottom-up computational model from the perspective of frequency domain based on the biological discovery of non-Classical Receptive Field (nCRF) in the retina. A saliency map can be obtained according to the idea of Extended Classical Receptive Field. The model is composed of three major steps: firstly decompose the input image into several feature maps representing different frequency bands that cover the whole frequency domain by utilizing Gabor wavelet. Secondly, whiten the feature maps to highlight the embedded saliency information. Thirdly, select some optimal maps, simulating the response of receptive field especially nCRF, to generate the saliency map. Experimental results show that the proposed algorithm is able to work with stable effect and outstanding performance in a variety of situations as human beings do and is adaptive to both psychological patterns and natural images. Beyond that, biological plausibility of nCRF and Gabor wavelet transform make this approach reliable.     

Random Wiring,Ganglion Cell Mosaics,and the Functional Architecture of the Visual Cortex

The architecture of iso-orientation domains in the primary visual cortex (V1) of placental carnivores and primates apparently follows species invariant quantitative laws. Dynamical optimization models assuming that neurons coordinate their stimulus preferences throughout cortical circuits linking millions of cells specifically predict these invariants. This might indicate that V1’s intrinsic connectome and its functional architecture adhere to a single optimization principle with high precision and robustness. To validate this hypothesis, it is critical to closely examine the quantitative predictions of alternative candidate theories. Random feedforward wiring within the retino-cortical pathway represents a conceptually appealing alternative to dynamical circuit optimization because random dimension-expanding projections are believed to generically exhibit computationally favorable properties for stimulus representations. Here, we ask whether the quantitative invariants of V1 architecture can be explained as a generic emergent property of random wiring. We generalize and examine the stochastic wiring model proposed by Ringach and coworkers, in which iso-orientation domains in the visual cortex arise through random feedforward connections between semi-regular mosaics of retinal ganglion cells (RGCs) and visual cortical neurons. We derive closed-form expressions for cortical receptive fields and domain layouts predicted by the model for perfectly hexagonal RGC mosaics. Including spatial disorder in the RGC positions considerably changes the domain layout properties as a function of disorder parameters such as position scatter and its correlations across the retina. However, independent of parameter choice, we find that the model predictions substantially deviate from the layout laws of iso-orientation domains observed experimentally. Considering random wiring with the currently most realistic model of RGC mosaic layouts, a pairwise interacting point process, the predicted layouts remain distinct from experimental observations and resemble Gaussian random fields. We conclude that V1 layout invariants are specific quantitative signatures of visual cortical optimization, which cannot be explained by generic random feedforward-wiring models.     

Decoding mental states from brain activity in humans

Recent advances in human neuroimaging have shown that it is possible to accurately decode a person's conscious experience based only on non-invasive measurements of their brain activity. Such 'brain reading' has mostly been studied in the domain of visual perception, where it helps reveal the way in which individual experiences are encoded in the human brain. The same approach can also be extended to other types of mental state, such as covert attitudes and lie detection. Such applications raise important ethical issues concerning the privacy of personal thought.     

Rapid dynamics of contrast responses in the cat primary visual cortex

The visual information we receive during natural vision changes rapidly and continuously. The visual system must adapt to the spatiotemporal contents of the environment in order to efficiently process the dynamic signals. However, neuronal responses to luminance contrast are usually measured using drifting or stationary gratings presented for a prolonged duration. Since motion in our visual field is continuous, the signals received by the visual system contain an abundance of transient components in the contrast domain. Here using a modified reverse correlation method, we studied the properties of responses of neurons in the cat primary visual cortex to different contrasts of grating stimuli presented statically and transiently for 40 ms, and showed that neurons can effectively discriminate the rapidly changing contrasts. The change in the contrast response function (CRF) over time mainly consisted of an increment in contrast gain (CRF shifts to left) in the developing phase of temporal responses and a decrement in response gain (CRF shifts downward) in the decay phase. When the distribution range of stimulus contrasts was increased, neurons demonstrated decrement in contrast gain and response gain. Our results suggest that contrast gain control (contrast adaptation) and response gain control mechanisms are well established during the first tens of milliseconds after stimulus onset and may cooperatively mediate the rapid dynamic responses of visual cortical neurons to the continuously changing contrast. This fast contrast adaptation may play a role in detecting contrast contours in the context of visual scenes that are varying rapidly.