Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity

The principles by which networks of neurons compute, and how spike-timing dependent plasticity (STDP) of synaptic weights generates and maintains their computational function, are unknown. Preceding work has shown that soft winner-take-all (WTA) circuits, where pyramidal neurons inhibit each other via interneurons, are a common motif of cortical microcircuits. We show through theoretical analysis and computer simulations that Bayesian computation is induced in these network motifs through STDP in combination with activity-dependent changes in the excitability of neurons. The fundamental components of this emergent Bayesian computation are priors that result from adaptation of neuronal excitability and implicit generative models for hidden causes that are created in the synaptic weights through STDP. In fact, a surprising result is that STDP is able to approximate a powerful principle for fitting such implicit generative models to high-dimensional spike inputs: Expectation Maximization. Our results suggest that the experimentally observed spontaneous activity and trial-to-trial variability of cortical neurons are essential features of their information processing capability, since their functional role is to represent probability distributions rather than static neural codes. Furthermore it suggests networks of Bayesian computation modules as a new model for distributed information processing in the cortex.     

Integrated mechanisms of anticipation and rate-of-change computations in cortical circuits

Local neocortical circuits are characterized by stereotypical physiological and structural features that subserve generic computational operations. These basic computations of the cortical microcircuit emerge through the interplay of neuronal connectivity, cellular intrinsic properties, and synaptic plasticity dynamics. How these interacting mechanisms generate specific computational operations in the cortical circuit remains largely unknown. Here, we identify the neurophysiological basis of both the rate of change and anticipation computations on synaptic inputs in a cortical circuit. Through biophysically realistic computer simulations and neuronal recordings, we show that the rate-of-change computation is operated robustly in cortical networks through the combination of two ubiquitous brain mechanisms: short-term synaptic depression and spike-frequency adaptation. We then show how this rate-of-change circuit can be embedded in a convergently connected network to anticipate temporally incoming synaptic inputs, in quantitative agreement with experimental findings on anticipatory responses to moving stimuli in the primary visual cortex. Given the robustness of the mechanism and the widespread nature of the physiological machinery involved, we suggest that rate-of-change computation and temporal anticipation are principal, hard-wired functions of neural information processing in the cortical microcircuit.     

Cortical state and attention

The brain continuously adapts its processing machinery to behavioural demands. To achieve this, it rapidly modulates the operating mode of cortical circuits, controlling the way that information is transformed and routed. This article will focus on two experimental approaches by which the control of cortical information processing has been investigated: the study of state-dependent cortical processing in rodents and attention in the primate visual system. Both processes involve a modulation of low-frequency activity fluctuations and spiking correlation, and are mediated by common receptor systems. We suggest that selective attention involves processes that are similar to state change, and that operate at a local columnar level to enhance the representation of otherwise non-salient features while suppressing internally generated activity patterns.     

Specificity and randomness: structure-function relationships in neural circuits

A fundamental but unsolved problem in neuroscience is how connections between neurons might underlie information processing in central circuits. Building wiring diagrams of neural networks may accelerate our understanding of how they compute. But even if we had wiring diagrams, it is critical to know what neurons in a circuit are doing: their physiology. In both the retina and cerebral cortex, a great deal is known about topographic specificity, such as lamination and cell-type specificity of connections. Little, however, is known about connections as they relate to function. Here, we review how advances in functional imaging and electron microscopy have recently allowed the examination of relationships between sensory physiology and synaptic connections in cortical and retinal circuits.     

Mapping the matrix: the ways of neocortex

While we know that the neocortex occupies 85% of our brains and that its circuits allow an enormous flexibility and repertoire of behavior (not to mention unexplained phenomena like consciousness), a century after Cajal we have very little knowledge of the details of the cortical circuits or their mode of function. One simplifying hypothesis that has existed since Cajal is that the neocortex consists of repeated copies of the same fundamental circuit. However, finding that fundamental circuit has proved elusive, although partial drafts of a "canonical circuit" appear in many different guises of structure and function. Here, we review some critical stages in the history of this quest. In doing so, we consider the style of cortical computation in relation to the neuronal machinery that supports it. We conclude that the structure and function of cortex honors two major computational principles: "just-enough" and "just-in-time."     

Septal networks: relevance to theta rhythm, epilepsy and Alzheimer's disease

Information processing and storing by brain networks requires a highly coordinated operation of multiple neuronal groups. The function of septal neurons is to modulate the activity of archicortical (e.g. hippocampal) and neocortical circuits. This modulation is necessary for the development and normal occurrence of rhythmical cortical activities that control the processing of sensory information and memory functions. Damage or degeneration of septal neurons results in abnormal information processing in cortical circuits and consequent brain dysfunction. Septal neurons not only provide the optimal levels of excitatory background to cortical structures, but they may also inhibit the occurrence of abnormal excitability states.     

Optogenetic approaches to restoring visual function in retinitis pigmentosa

Retinitis pigmentosa is a hereditary eye disease that affects photoreceptors and leads to blindness. The discovery of a microbial light-gated channel and the subsequent development of similar 'optogenetic' sensors have opened the door to creating artificial photoreceptors in the remaining retinal circuits of retinitis pigmentosa retinas via gene therapy. Here we review recent studies in animal models of retinitis pigmentosa that have combined knowledge of retinal cell types, circuits and computations with the ability to equip cell types with optogenetic sensors in order to restore visual activity. We also discuss the translational potential of this therapy.     

Local cortical circuit model inferred from power-law distributed neuronal avalanches

How cortical neurons process information crucially depends on how their local circuits are organized. Spontaneous synchronous neuronal activity propagating through neocortical slices displays highly diverse, yet repeatable, activity patterns called “neuronal avalanches”. They obey power-law distributions of the event sizes and lifetimes, presumably reflecting the structure of local circuits developed in slice cultures. However, the explicit network structure underlying the power-law statistics remains unclear. Here, we present a neuronal network model of pyramidal and inhibitory neurons that enables stable propagation of avalanche-like spiking activity. We demonstrate a neuronal wiring rule that governs the formation of mutually overlapping cell assemblies during the development of this network. The resultant network comprises a mixture of feedforward chains and recurrent circuits, in which neuronal avalanches are stable if the former structure is predominant. Interestingly, the recurrent synaptic connections formed by this wiring rule limit the number of cell assemblies embeddable in a neuron pool of given size. We investigate how the resultant power laws depend on the details of the cell-assembly formation as well as on the inhibitory feedback. Our model suggests that local cortical circuits may have a more complex topological design than has previously been thought. Competing financial interests: The authors declare that they have no competing financial interests. Action Editor: Peter Latham     

Bayesian inference in populations of cortical neurons: a model of motion integration and segmentation in area MT

A major issue in cortical physiology and computational neuroscience is understanding the interaction between extrinsic signals from feedforward connections and intracortical signals from lateral connections. We propose here a computational model for motion perception based on the assumption that the local cortical circuits in the medio-temporal area (area MT) implement a Bayesian inference principle. This approach establishes a functional balance between feedforward and lateral, excitatory and inhibitory, inputs. The model reproduces most of the known properties of the neurons in area MT in response to moving stimuli. It accounts for important motion perception phenomena including motion transparency, spatial and temporal integration/segmentation. While integrating several properties of previously proposed models, it makes specific testable predictions concerning, in particular, temporal properties of neurons and the architecture of lateral connections in area MT. In addition, the proposed mechanism is consistent with the known properties of local cortical circuits in area V1. This suggests that Bayesian inference may be a general feature of information processing in cortical neuron populations. Received: 3 December 1997 / Accepted in revised form: 21 July 1998     

Effects of cortical damage on binocular depth perception

Stereoscopic depth perception requires considerable neural computation, including the initial correspondence of the two retinal images, comparison across the local regions of the visual field and integration with other cues to depth. The most common cause for loss of stereoscopic vision is amblyopia, in which one eye has failed to form an adequate input to the visual cortex, usually due to strabismus (deviating eye) or anisometropia. However, the significant cortical processing required to produce the percept of depth means that, even when the retinal input is intact from both eyes, brain damage or dysfunction can interfere with stereoscopic vision. In this review, I examine the evidence for impairment of binocular vision and depth perception that can result from insults to the brain, including both discrete damage, temporal lobectomy and more systemic diseases such as posterior cortical atrophy.This article is part of the themed issue ‘Vision in our three-dimensional world’.     

Computational aspects of feedback in neural circuits

It has previously been shown that generic cortical microcircuit models can perform complex real-time computations on continuous input streams, provided that these computations can be carried out with a rapidly fading memory. We investigate the computational capability of such circuits in the more realistic case where not only readout neurons, but in addition a few neurons within the circuit, have been trained for specific tasks. This is essentially equivalent to the case where the output of trained readout neurons is fed back into the circuit. We show that this new model overcomes the limitation of a rapidly fading memory. In fact, we prove that in the idealized case without noise it can carry out any conceivable digital or analog computation on time-varying inputs. But even with noise, the resulting computational model can perform a large class of biologically relevant real-time computations that require a nonfading memory. We demonstrate these computational implications of feedback both theoretically, and through computer simulations of detailed cortical microcircuit models that are subject to noise and have complex inherent dynamics. We show that the application of simple learning procedures (such as linear regression or perceptron learning) to a few neurons enables such circuits to represent time over behaviorally relevant long time spans, to integrate evidence from incoming spike trains over longer periods of time, and to process new information contained in such spike trains in diverse ways according to the current internal state of the circuit. In particular we show that such generic cortical microcircuits with feedback provide a new model for working memory that is consistent with a large set of biological constraints. Although this article examines primarily the computational role of feedback in circuits of neurons, the mathematical principles on which its analysis is based apply to a variety of dynamical systems. Hence they may also throw new light on the computational role of feedback in other complex biological dynamical systems, such as, for example, genetic regulatory networks.     

Random Wiring,Ganglion Cell Mosaics,and the Functional Architecture of the Visual Cortex

The architecture of iso-orientation domains in the primary visual cortex (V1) of placental carnivores and primates apparently follows species invariant quantitative laws. Dynamical optimization models assuming that neurons coordinate their stimulus preferences throughout cortical circuits linking millions of cells specifically predict these invariants. This might indicate that V1’s intrinsic connectome and its functional architecture adhere to a single optimization principle with high precision and robustness. To validate this hypothesis, it is critical to closely examine the quantitative predictions of alternative candidate theories. Random feedforward wiring within the retino-cortical pathway represents a conceptually appealing alternative to dynamical circuit optimization because random dimension-expanding projections are believed to generically exhibit computationally favorable properties for stimulus representations. Here, we ask whether the quantitative invariants of V1 architecture can be explained as a generic emergent property of random wiring. We generalize and examine the stochastic wiring model proposed by Ringach and coworkers, in which iso-orientation domains in the visual cortex arise through random feedforward connections between semi-regular mosaics of retinal ganglion cells (RGCs) and visual cortical neurons. We derive closed-form expressions for cortical receptive fields and domain layouts predicted by the model for perfectly hexagonal RGC mosaics. Including spatial disorder in the RGC positions considerably changes the domain layout properties as a function of disorder parameters such as position scatter and its correlations across the retina. However, independent of parameter choice, we find that the model predictions substantially deviate from the layout laws of iso-orientation domains observed experimentally. Considering random wiring with the currently most realistic model of RGC mosaic layouts, a pairwise interacting point process, the predicted layouts remain distinct from experimental observations and resemble Gaussian random fields. We conclude that V1 layout invariants are specific quantitative signatures of visual cortical optimization, which cannot be explained by generic random feedforward-wiring models.     

Towards a Mathematical Theory of Cortical Micro-circuits

The theoretical setting of hierarchical Bayesian inference is gaining acceptance as a framework for understanding cortical computation. In this paper, we describe how Bayesian belief propagation in a spatio-temporal hierarchical model, called Hierarchical Temporal Memory (HTM), can lead to a mathematical model for cortical circuits. An HTM node is abstracted using a coincidence detector and a mixture of Markov chains. Bayesian belief propagation equations for such an HTM node define a set of functional constraints for a neuronal implementation. Anatomical data provide a contrasting set of organizational constraints. The combination of these two constraints suggests a theoretically derived interpretation for many anatomical and physiological features and predicts several others. We describe the pattern recognition capabilities of HTM networks and demonstrate the application of the derived circuits for modeling the subjective contour effect. We also discuss how the theory and the circuit can be extended to explain cortical features that are not explained by the current model and describe testable predictions that can be derived from the model.     

VSDI: a new era in functional imaging of cortical dynamics

During the last few decades, neuroscientists have benefited from the emergence of many powerful functional imaging techniques that cover broad spatial and temporal scales. We can now image single molecules controlling cell differentiation, growth and death; single cells and their neurites processing electrical inputs and sending outputs; neuronal circuits performing neural computations in vitro; and the intact brain. At present, imaging based on voltage-sensitive dyes (VSDI) offers the highest spatial and temporal resolution for imaging neocortical functions in the living brain, and has paved the way for a new era in the functional imaging of cortical dynamics. It has facilitated the exploration of fundamental mechanisms that underlie neocortical development, function and plasticity at the fundamental level of the cortical column.     

Interplay between Short- and Long-Term Plasticity in Cell-Assembly Formation

Various hippocampal and neocortical synapses of mammalian brain show both short-term plasticity and long-term plasticity, which are considered to underlie learning and memory by the brain. According to Hebb’s postulate, synaptic plasticity encodes memory traces of past experiences into cell assemblies in cortical circuits. However, it remains unclear how the various forms of long-term and short-term synaptic plasticity cooperatively create and reorganize such cell assemblies. Here, we investigate the mechanism in which the three forms of synaptic plasticity known in cortical circuits, i.e., spike-timing-dependent plasticity (STDP), short-term depression (STD) and homeostatic plasticity, cooperatively generate, retain and reorganize cell assemblies in a recurrent neuronal network model. We show that multiple cell assemblies generated by external stimuli can survive noisy spontaneous network activity for an adequate range of the strength of STD. Furthermore, our model predicts that a symmetric temporal window of STDP, such as observed in dopaminergic modulations on hippocampal neurons, is crucial for the retention and integration of multiple cell assemblies. These results may have implications for the understanding of cortical memory processes.     

Computational design of synthetic regulatory networks from a genetic library to characterize the designability of dynamical behaviors

The engineering of synthetic gene networks has mostly relied on the assembly of few characterized regulatory elements using rational design principles. It is of outmost importance to analyze the scalability and limits of such a design workflow. To analyze the design capabilities of libraries of regulatory elements, we have developed the first automated design approach that combines such elements to search the genotype space associated to a given phenotypic behavior. Herein, we calculated the designability of dynamical functions obtained from circuits assembled with a given genetic library. By designing circuits working as amplitude filters, pulse counters and oscillators, we could infer new mechanisms for such behaviors. We also highlighted the hierarchical design and the optimization of the interface between devices. We dissected the functional diversity of a constrained library and we found that even such libraries can provide a rich variety of behaviors. We also found that intrinsic noise slightly reduces the designability of digital circuits, but it increases the designability of oscillators. Finally, we analyzed the robust design as a strategy to counteract the evolvability and noise in gene expression of the engineered circuits within a cellular background, obtaining mechanisms for robustness through non-linear negative feedback loops.