Spatio-temporal credit assignment in neuronal population learning

In learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.     

Theoretical analysis of a synaptotropic dendrite growth mechanism

It is generally believed that the genome cannot encode explicit instructions to form each synaptic connection in the nervous system, but may provide general neurite growth mechanisms which will result in proper connectivity. Recent in vivo imaging has provided evidence for a synaptotropic growth mechanism, wherein synapses could influence dendrite growth by selectively stabilizing filopodia upon which they form. We undertook a theoretical investigation into the consequences of such a growth process. Discrete stochastic simulations demonstrate that the synaptotropic mechanism can result in decreased dendritic wiring length, is capable of searching for regions of high density pre-synaptic partners, and can recapitulate specific patterns of dendrite growth and connectivity. A mean-field analysis shows that growth by selective stabilization of filopodia can be approximated as a reaction-diffusion system, with a spatially varying diffusion constant that depends on the probability of synapse formation. Thus, growth will occur faster in regions of appropriate synaptic connections, and the net growth can be shown to climb a gradient of synaptic partner density. Synaptotropic growth thus presents a mechanism for the emergent development of connectivity based on local properties of the circuit elements, rather than explicit dependence on global guidance molecules or innate predetermined branching programs.     

Top-Down Modulation on Perceptual Decision with Balanced Inhibition through Feedforward and Feedback Inhibitory Neurons

Recent physiological studies have shown that neurons in various regions of the central nervous systems continuously receive noisy excitatory and inhibitory synaptic inputs in a balanced and covaried fashion. While this balanced synaptic input (BSI) is typically described in terms of maintaining the stability of neural circuits, a number of experimental and theoretical studies have suggested that BSI plays a proactive role in brain functions such as top-down modulation for executive control. Two issues have remained unclear in this picture. First, given the noisy nature of neuronal activities in neural circuits, how do the modulatory effects change if the top-down control implements BSI with different ratios between inhibition and excitation? Second, how is a top-down BSI realized via only excitatory long-range projections in the neocortex? To address the first issue, we systematically tested how the inhibition/excitation ratio affects the accuracy and reaction times of a spiking neural circuit model of perceptual decision. We defined an energy function to characterize the network dynamics, and found that different ratios modulate the energy function of the circuit differently and form two distinct functional modes. To address the second issue, we tested BSI with long-distance projection to inhibitory neurons that are either feedforward or feedback, depending on whether these inhibitory neurons do or do not receive inputs from local excitatory cells, respectively. We found that BSI occurs in both cases. Furthermore, when relying on feedback inhibitory neurons, through the recurrent interactions inside the circuit, BSI dynamically and automatically speeds up the decision by gradually reducing its inhibitory component in the course of a trial when a decision process takes too long.     

Spike suppression in a local cortical circuit induced by transcranial magnetic stimulation

Transcranial magnetic stimulation (TMS) noninvasively interferes with human cortical function, and is widely used as an effective technique for probing causal links between neural activity and cognitive function. However, the physiological mechanisms underlying TMS-induced effects on neural activity remain unclear. We examined the mechanism by which TMS disrupts neural activity in a local circuit in early visual cortex using a computational model consisting of conductance-based spiking neurons with excitatory and inhibitory synaptic connections. We found that single-pulse TMS suppressed spiking activity in a local circuit model, disrupting the population response. Spike suppression was observed when TMS was applied to the local circuit within a limited time window after the local circuit received sensory afferent input, as observed in experiments investigating suppression of visual perception with TMS targeting early visual cortex. Quantitative analyses revealed that the magnitude of suppression was significantly larger for synaptically-connected neurons than for isolated individual neurons, suggesting that intracortical inhibitory synaptic coupling also plays an important role in TMS-induced suppression. A conventional local circuit model of early visual cortex explained only the early period of visual suppression observed in experiments. However, models either involving strong recurrent excitatory synaptic connections or sustained excitatory input were able to reproduce the late period of visual suppression. These results suggest that TMS targeting early visual cortex disrupts functionally distinct neural signals, possibly corresponding to feedforward and recurrent information processing, by imposing inhibitory effects through intracortical inhibitory synaptic connections.     

Probabilistic decision making by slow reverberation in cortical circuits

Recent physiological studies of alert primates have revealed cortical neural correlates of key steps in a perceptual decision-making process. To elucidate synaptic mechanisms of decision making, I investigated a biophysically realistic cortical network model for a visual discrimination experiment. In the model, slow recurrent excitation and feedback inhibition produce attractor dynamics that amplify the difference between conflicting inputs and generates a binary choice. The model is shown to account for salient characteristics of the observed decision-correlated neural activity, as well as the animal's psychometric function and reaction times. These results suggest that recurrent excitation mediated by NMDA receptors provides a candidate cellular mechanism for the slow time integration of sensory stimuli and the formation of categorical choices in a decision-making neocortical network.     

The mechanisms and molecules that connect photoreceptor axons to their targets in Drosophila

The development of the Drosophila visual system provides a framework for investigating how circuits assemble. A sequence of reciprocal interactions amongst photoreceptors, target neurons and glia creates a precise pattern of connections while reducing the complexity of the targeting process. Both afferent-afferent and afferent-target interactions are required for photoreceptor (R cell) axons to select appropriate synaptic partners. With the identification of some critical cell adhesion and signaling molecules, the logic by which axons make choices amongst alternate synaptic partners is becoming clear. These studies also provide an opportunity to examine the molecular basis of neural circuit evolution.     

Neuronal microcircuits for decision making in C. elegans

The simplicity and genetic tractability of the nervous system of the nematode Caenorhabditis elegans make it an attractive system in which to seek biological mechanisms of decision making. Although work in this area remains at an early stage, four basic types paradigms of behavioral choice, a simple form of decision making, have now been demonstrated in C. elegans. A recent series of pioneering studies, combining genetics and molecular biology with new techniques such as microfluidics and calcium imaging in freely moving animals, has begun to elucidate the neuronal mechanisms underlying behavioral choice. The new research has focussed on choice behaviors in the context of habitat and resource localization, for which the neuronal circuit has been identified. Three main circuit motifs for behavioral choice have been identified. One motif is based mainly on changes in the strength of synaptic connections whereas the other two motifs are based on changes in the basal activity of an interneuron and the sensory neuron to which it is electrically coupled. Peptide signaling seems to play a prominent role in all three motifs, and it may be a general rule that concentrations of various peptides encode the internal states that influence behavioral decisions in C. elegans.     

A fluctuation-driven mechanism for slow decision processes in reverberant networks

The spike activity of cells in some cortical areas has been found to be correlated with reaction times and behavioral responses during two-choice decision tasks. These experimental findings have motivated the study of biologically plausible winner-take-all network models, in which strong recurrent excitation and feedback inhibition allow the network to form a categorical choice upon stimulation. Choice formation corresponds in these models to the transition from the spontaneous state of the network to a state where neurons selective for one of the choices fire at a high rate and inhibit the activity of the other neurons. This transition has been traditionally induced by an increase in the external input that destabilizes the spontaneous state of the network and forces its relaxation to a decision state. Here we explore a different mechanism by which the system can undergo such transitions while keeping the spontaneous state stable, based on an escape induced by finite-size noise from the spontaneous state. This decision mechanism naturally arises for low stimulus strengths and leads to exponentially distributed decision times when the amount of noise in the system is small. Furthermore, we show using numerical simulations that mean decision times follow in this regime an exponential dependence on the amplitude of noise. The escape mechanism provides thus a dynamical basis for the wide range and variability of decision times observed experimentally.     

Reversible plasticity of fear memory-encoding amygdala synaptic circuits even after fear memory consolidation

It is generally believed that after memory consolidation, memory-encoding synaptic circuits are persistently modified and become less plastic. This, however, may hinder the remaining capacity of information storage in a given neural circuit. Here we consider the hypothesis that memory-encoding synaptic circuits still retain reversible plasticity even after memory consolidation. To test this, we employed a protocol of auditory fear conditioning which recruited the vast majority of the thalamic input synaptic circuit to the lateral amygdala (T-LA synaptic circuit; a storage site for fear memory) with fear conditioning-induced synaptic plasticity. Subsequently the fear memory-encoding synaptic circuits were challenged with fear extinction and re-conditioning to determine whether these circuits exhibit reversible plasticity. We found that fear memory-encoding T-LA synaptic circuit exhibited dynamic efficacy changes in tight correlation with fear memory strength even after fear memory consolidation. Initial conditioning or re-conditioning brought T-LA synaptic circuit near the ceiling of their modification range (occluding LTP and enhancing depotentiation in brain slices prepared from conditioned or re-conditioned rats), while extinction reversed this change (reinstating LTP and occluding depotentiation in brain slices prepared from extinguished rats). Consistently, fear conditioning-induced synaptic potentiation at T-LA synapses was functionally reversed by extinction and reinstated by subsequent re-conditioning. These results suggest reversible plasticity of fear memory-encoding circuits even after fear memory consolidation. This reversible plasticity of memory-encoding synapses may be involved in updating the contents of original memory even after memory consolidation.     

Spike timing precision of neuronal circuits

Spike timing is believed to be a key factor in sensory information encoding and computations performed by the neurons and neuronal circuits. However, the considerable noise and variability, arising from the inherently stochastic mechanisms that exist in the neurons and the synapses, degrade spike timing precision. Computational modeling can help decipher the mechanisms utilized by the neuronal circuits in order to regulate timing precision. In this paper, we utilize semi-analytical techniques, which were adapted from previously developed methods for electronic circuits, for the stochastic characterization of neuronal circuits. These techniques, which are orders of magnitude faster than traditional Monte Carlo type simulations, can be used to directly compute the spike timing jitter variance, power spectral densities, correlation functions, and other stochastic characterizations of neuronal circuit operation. We consider three distinct neuronal circuit motifs: Feedback inhibition, synaptic integration, and synaptic coupling. First, we show that both the spike timing precision and the energy efficiency of a spiking neuron are improved with feedback inhibition. We unveil the underlying mechanism through which this is achieved. Then, we demonstrate that a neuron can improve on the timing precision of its synaptic inputs, coming from multiple sources, via synaptic integration: The phase of the output spikes of the integrator neuron has the same variance as that of the sample average of the phases of its inputs. Finally, we reveal that weak synaptic coupling among neurons, in a fully connected network, enables them to behave like a single neuron with a larger membrane area, resulting in an improvement in the timing precision through cooperation.     

Decision Rules and Group Rationality: Cognitive Gain or Standstill?

Recent research in group cognition points towards the existence of collective cognitive competencies that transcend individual group members’ cognitive competencies. Since rationality is a key cognitive competence for group decision making, and group cognition emerges from the coordination of individual cognition during social interactions, this study tests the extent to which collaborative and consultative decision rules impact the emergence of group rationality. Using a set of decision tasks adapted from the heuristics and biases literature, we evaluate rationality as the extent to which individual choices are aligned with a normative ideal. We further operationalize group rationality as cognitive synergy (the extent to which collective rationality exceeds average or best individual rationality in the group), and we test the effect of collaborative and consultative decision rules in a sample of 176 groups. Our results show that the collaborative decision rule has superior synergic effects as compared to the consultative decision rule. The ninety one groups working in a collaborative fashion made more rational choices (above and beyond the average rationality of their members) than the eighty five groups working in a consultative fashion. Moreover, the groups using a collaborative decision rule were closer to the rationality of their best member than groups using consultative decision rules. Nevertheless, on average groups did not outperformed their best member. Therefore, our results reveal how decision rules prescribing interpersonal interactions impact on the emergence of collective cognitive competencies. They also open potential venues for further research on the emergence of collective rationality in human decision-making groups.     

Convergent synaptic and circuit substrates underlying autism genetic risks

There has been a surge of diagnosis of autism spectrum disorders （ASD） over the past decade. While large, high powered genome screening studies of children with ASD have identified numerous genetic risk factors, research efforts to understanding how each of these risk factors contributes to the development autism has met with limited success. Revealing the mechanisms by which these genetic risk factors affect brain development and predispose a child to autism requires mechanistic understanding of the neurobiological changes underlying this devastating group of developmental disorders at multifaceted molecular, cellular and system levels. It has been increasingly clear that the normal trajectory of neurodevelopment is compromised in autism, in multiple domains as much as aberrant neuronal production, growth, functional maturation, patterned connectivity, and balanced excitation and inhibition of brain networks. Many autism risk factors identified in humans have been now reconstituted in experimental mouse models to allow mechanistic interrogation of the biological role of the risk gene. Studies utilizing these mouse models have revealed that underlying the enormous heterogeneity of perturbed cellular events, mechanisms directing synaptic and circuit assembly may provide a unifying explanation for the pathophysiological changes and behavioral endophenotypes seen in autism, although synaptic perturbations are far from being the only alterations relevant for ASD. In this review, we discuss synaptic and circuit abnormalities obtained from several prevalent mouse models, particularly those reflecting syndromic forms of ASD that are caused by single gene perturbations. These compiled results reveal that ASD risk genes contribute to proper signaling of the developing gene networks that maintain synaptic and circuit homeostasis, which is fundamental to normal brain development.     

SynPAnal: Software for Rapid Quantification of the Density and Intensity of Protein Puncta from Fluorescence Microscopy Images of Neurons

Continuous modification of the protein composition at synapses is a driving force for the plastic changes of synaptic strength, and provides the fundamental molecular mechanism of synaptic plasticity and information storage in the brain. Studying synaptic protein turnover is not only important for understanding learning and memory, but also has direct implication for understanding pathological conditions like aging, neurodegenerative diseases, and psychiatric disorders. Proteins involved in synaptic transmission and synaptic plasticity are typically concentrated at synapses of neurons and thus appear as puncta (clusters) in immunofluorescence microscopy images. Quantitative measurement of the changes in puncta density, intensity, and sizes of specific proteins provide valuable information on their function in synaptic transmission, circuit development, synaptic plasticity, and synaptopathy. Unfortunately, puncta quantification is very labor intensive and time consuming. In this article, we describe a software tool designed for the rapid semi-automatic detection and quantification of synaptic protein puncta from 2D immunofluorescence images generated by confocal laser scanning microscopy. The software, dubbed as SynPAnal (for Synaptic Puncta Analysis), streamlines data quantification for puncta density and average intensity, thereby increases data analysis throughput compared to a manual method. SynPAnal is stand-alone software written using the JAVA programming language, and thus is portable and platform-free.     

The dynamical stability of reverberatory neural circuits

The concept of reverberation proposed by Lorente de Nó and Hebb is key to understanding strongly recurrent cortical networks. In particular, synaptic reverberation is now viewed as a likely mechanism for the active maintenance of working memory in the prefrontal cortex. Theoretically, this has spurred a debate as to how such a potentially explosive mechanism can provide stable working-memory function given the synaptic and cellular mechanisms at play in the cerebral cortex. We present here new evidence for the participation of NMDA receptors in the stabilization of persistent delay activity in a biophysical network model of conductance-based neurons. We show that the stability of working-memory function, and the required NMDA/AMPA ratio at recurrent excitatory synapses, depend on physiological properties of neurons and synaptic interactions, such as the time constants of excitation and inhibition, mutual inhibition between interneurons, differential NMDA receptor participation at excitatory projections to pyramidal neurons and interneurons, or the presence of slow intrinsic ion currents in pyramidal neurons. We review other mechanisms proposed to enhance the dynamical stability of synaptically generated attractor states of a reverberatory circuit. This recent work represents a necessary and significant step towards testing attractor network models by cortical electrophysiology.     

Responses of a bursting pacemaker to excitation reveal spatial segregation between bursting and spiking mechanisms

Central pattern generators (CPGs) frequently include bursting neurons that serve as pacemakers for rhythm generation. Phase resetting curves (PRCs) can provide insight into mechanisms underlying phase locking in such circuits. PRCs were constructed for a pacemaker bursting complex in the pyloric circuit in the stomatogastric ganglion of the lobster and crab. This complex is comprised of the Anterior Burster (AB) neuron and two Pyloric Dilator (PD) neurons that are all electrically coupled. Artificial excitatory synaptic conductance pulses of different strengths and durations were injected into one of the AB or PD somata using the Dynamic Clamp. Previously, we characterized the inhibitory PRCs by assuming a single slow process that enabled synaptic inputs to trigger switches between an up state in which spiking occurs and a down state in which it does not. Excitation produced five different PRC shapes, which could not be explained with such a simple model. A separate dendritic compartment was required to separate the mechanism that generates the up and down phases of the bursting envelope (1) from synaptic inputs applied at the soma, (2) from axonal spike generation and (3) from a slow process with a slower time scale than burst generation. This study reveals that due to the nonlinear properties and compartmentalization of ionic channels, the response to excitation is more complex than inhibition.     

Rate of quantal transmitter release at the mammalian rod synapse.

Under scotopic conditions, the mammalian rod encodes either one photon or none within its integration time. Consequently the signal presented to its synaptic terminal is binary. The synapse has a single active zone that releases neurotransmitter quanta tonically in darkness and pauses briefly in response to a rhodopsin isomerization by a photon. We asked: what minimum tonic rate would allow the postsynaptic bipolar cell to distinguish this pause from an extra-long interval between quanta due to the stochastic timing of release? The answer required a model of the circuit that included the rod convergence onto the bipolar cell and the bipolar cell''s signal-to-noise ratio. Calculations from the model suggest that tonic release must be at least 40 quanta/s. This tonic rate is much higher than at conventional synapses where reliability is achieved by employing multiple active zones. The rod''s synaptic mechanism makes efficient use of space, which in the retina is at a premium.     

Robust spatial working memory through homeostatic synaptic scaling in heterogeneous cortical networks

The concept of bell-shaped persistent neural activity represents a cornerstone of the theory for the internal representation of analog quantities, such as spatial location or head direction. Previous models, however, relied on the unrealistic assumption of network homogeneity. We investigate this issue in a network model where fine tuning of parameters is destroyed by heterogeneities in cellular and synaptic properties. Heterogeneities result in the loss of stored spatial information in a few seconds. Accurate encoding is recovered when a homeostatic mechanism scales the excitatory synapses to each cell to compensate for the heterogeneity in cellular excitability and synaptic inputs. Moreover, the more realistic model produces a wide diversity of tuning curves, as commonly observed in recordings from prefrontal neurons. We conclude that recurrent attractor networks in conjunction with appropriate homeostatic mechanisms provide a robust, biologically plausible theoretical framework for understanding the neural circuit basis of spatial working memory.