An auditory neural correlate suggests a mechanism underlying holistic pitch perception

Current theories of auditory pitch perception propose that cochlear place (spectral) and activity timing pattern (temporal) information are somehow combined within the brain to produce holistic pitch percepts, yet the neural mechanisms for integrating these two kinds of information remain obscure. To examine this process in more detail, stimuli made up of three pure tones whose components are individually resolved by the peripheral auditory system, but that nonetheless elicit a holistic, "missing fundamental" pitch percept, were played to human listeners. A technique was used to separate neural timing activity related to individual components of the tone complexes from timing activity related to an emergent feature of the complex (the envelope), and the region of the tonotopic map where information could originate from was simultaneously restricted by masking noise. Pitch percepts were mirrored to a very high degree by a simple combination of component-related and envelope-related neural responses with similar timing that originate within higher-frequency regions of the tonotopic map where stimulus components interact. These results suggest a coding scheme for holistic pitches whereby limited regions of the tonotopic map (spectral places) carrying envelope- and component-related activity with similar timing patterns selectively provide a key source of neural pitch information. A similar mechanism of integration between local and emergent object properties may contribute to holistic percepts in a variety of sensory systems.     

Timing Precision in Population Coding of Natural Scenes in the Early Visual System

The timing of spiking activity across neurons is a fundamental aspect of the neural population code. Individual neurons in the retina, thalamus, and cortex can have very precise and repeatable responses but exhibit degraded temporal precision in response to suboptimal stimuli. To investigate the functional implications for neural populations in natural conditions, we recorded in vivo the simultaneous responses, to movies of natural scenes, of multiple thalamic neurons likely converging to a common neuronal target in primary visual cortex. We show that the response of individual neurons is less precise at lower contrast, but that spike timing precision across neurons is relatively insensitive to global changes in visual contrast. Overall, spike timing precision within and across cells is on the order of 10 ms. Since closely timed spikes are more efficient in inducing a spike in downstream cortical neurons, and since fine temporal precision is necessary to represent the more slowly varying natural environment, we argue that preserving relative spike timing at a ~10-ms resolution is a crucial property of the neural code entering cortex.     

Spiking neurons that keep the rhythm

Detecting the temporal relationship among events in the environment is a fundamental goal of the brain. Following pulses of rhythmic stimuli, neurons of the retina and cortex produce activity that closely approximates the timing of an omitted pulse. This omitted stimulus response (OSR) is generally interpreted as a transient response to rhythmic input and is thought to form a basis of short-term perceptual memories. Despite its ubiquity across species and experimental protocols, the mechanisms underlying OSRs remain poorly understood. In particular, the highly transient nature of OSRs, typically limited to a single cycle after stimulation, cannot be explained by a simple mechanism that would remain locked to the frequency of stimulation. Here, we describe a set of realistic simulations that capture OSRs over a range of stimulation frequencies matching experimental work. The model does not require an explicit mechanism for learning temporal sequences. Instead, it relies on spike timing-dependent plasticity (STDP), a form of synaptic modification that is sensitive to the timing of pre- and post-synaptic action potentials. In the model, the transient nature of OSRs is attributed to the heterogeneous nature of neural properties and connections, creating intricate forms of activity that are continuously changing over time. Combined with STDP, neural heterogeneity enabled OSRs to complex rhythmic patterns as well as OSRs following a delay period. These results link the response of neurons to rhythmic patterns with the capacity of heterogeneous circuits to produce transient and highly flexible forms of neural activity.     

Timing as an intrinsic property of neural networks: evidence from in vivo and in vitro experiments

The discrimination and production of temporal patterns on the scale of hundreds of milliseconds are critical to sensory and motor processing. Indeed, most complex behaviours, such as speech comprehension and production, would be impossible in the absence of sophisticated timing mechanisms. Despite the importance of timing to human learning and cognition, little is known about the underlying mechanisms, in particular whether timing relies on specialized dedicated circuits and mechanisms or on general and intrinsic properties of neurons and neural circuits. Here, we review experimental data describing timing and interval-selective neurons in vivo and in vitro. We also review theoretical models of timing, focusing primarily on the state-dependent network model, which proposes that timing in the subsecond range relies on the inherent time-dependent properties of neurons and the active neural dynamics within recurrent circuits. Within this framework, time is naturally encoded in populations of neurons whose pattern of activity is dynamically changing in time. Together, we argue that current experimental and theoretical studies provide sufficient evidence to conclude that at least some forms of temporal processing reflect intrinsic computations based on local neural network dynamics.     

Ocular dominance development revisited

New approaches to the study of ocular dominance development, a model system for the development of neural architecture, indicate that eye-specific columns in primary visual cortex emerge substantially before the onset of the critical period, during which neural connections can be altered by visual experience. The timing, speed and specificity of column emergence implicate molecular patterning mechanisms, along with patterns of neural activity, in the generation of this columnar architecture.     

The neural representation of time

This review summarizes recent investigations of temporal processing. We focus on motor and perceptual tasks in which crucial events span hundreds of milliseconds. One key question concerns whether the representation of temporal information is dependent on a specialized system, distributed across a network of neural regions, or computed in a local task-dependent manner. Consistent with the specialized system framework, the cerebellum is associated with various tasks that require precise timing. Computational models of timing mechanisms within the cerebellar cortex are beginning to motivate physiological studies. Emphasis has also been placed on the basal ganglia as a specialized timing system, particularly for longer intervals. We outline an alternative hypothesis in which this structure is associated with decision processes.     

Immediate diaphragmatic electromyogram responses to imperceptible mechanical loads in conscious humans.

We used an esophageal electrode to measure the amplitude and neural inspiratory and expiratory (N TE) timing responses of crural diaphragmatic electrical activity in response to flow-resistive (R) and elastic (E) loads at or below the threshold for conscious detection, applied pseudorandomly to the oral airway of eight normal subjects. We observed a rapid first-breath neural reflex that modified respiratory timing such that N TE lengthened significantly in response to R loads in six of eight subjects and shortened in response to E loading in six of seven subjects. The prolongation of N TE with R loading resulted primarily from lengthening the portion of N TE during which phasic activity in the diaphragm is absent (TE NDIA), whereas E loading shortened N TE mainly by reducing TE NDIA. Most subjects responded to both types of loading by decreasing mean tonic diaphragmatic activity, the average level of muscle activity that exists when no phasic changes are occurring, as well as its variability. The observed timing responses are consistent in direction with optimally adaptive pattern regulation, whereas the modulation of tonic activity may be useful in neural regulation of end-expiratory lung volume.     

Neural muscle activation detection: A deep learning approach using surface electromyography

The timing of muscles activation which is a key parameter in determining plenty of medical conditions can be greatly assessed by the surface EMG signal which inherently carries an immense amount of information. Many techniques for measuring muscle activity detection exist in the literature. However, due to the complex nature of the EMG signal as well as the interference from other muscles that is observed during the measurement of the EMG signal, the accuracy of these techniques is compromised. In this paper, we introduce the neural muscle activation detection (NMAD) framework that detects the muscle activation based on deep learning. The main motivation behind using deep learning is to allow the neural network to detect based on the appropriate signal features instead of depending on certain assumptions. Not only the presented approach significantly improves the accuracy of timing detection, but because of the training nature, it can adapt to operate under different levels of interference and signal-to-noise ratio.     

Asynchrony adaptation reveals neural population code for audio-visual timing

The relative timing of auditory and visual stimuli is a critical cue for determining whether sensory signals relate to a common source and for making inferences about causality. However, the way in which the brain represents temporal relationships remains poorly understood. Recent studies indicate that our perception of multisensory timing is flexible--adaptation to a regular inter-modal delay alters the point at which subsequent stimuli are judged to be simultaneous. Here, we measure the effect of audio-visual asynchrony adaptation on the perception of a wide range of sub-second temporal relationships. We find distinctive patterns of induced biases that are inconsistent with the previous explanations based on changes in perceptual latency. Instead, our results can be well accounted for by a neural population coding model in which: (i) relative audio-visual timing is represented by the distributed activity across a relatively small number of neurons tuned to different delays; (ii) the algorithm for reading out this population code is efficient, but subject to biases owing to under-sampling; and (iii) the effect of adaptation is to modify neuronal response gain. These results suggest that multisensory timing information is represented by a dedicated population code and that shifts in perceived simultaneity following asynchrony adaptation arise from analogous neural processes to well-known perceptual after-effects.     

A Neural Network Model to Translate Brain Developmental Events across Mammalian Species

Translating the timing of brain developmental events across mammalian species using suitable models has provided unprecedented insights into neural development and evolution. More importantly, these models can prove to be useful abstractions and predict unknown events across species from known empirical event timing data retrieved from published literature. Such predictions can be especially useful since the distribution of the event timing data is skewed with a majority of events documented only across a few selected species. The present study investigates the choice of single hidden layer feed-forward neural networks (FFNN) for predicting the unknown events from the empirical data. A leave-one-out cross-validation approach is used to determine the optimal number of units in the hidden layer and the decay parameter for the FFNN. It is shown that unlike the present Finlay-Darlington (FD) model, FFNN does not impose any constraints on the functional form of the model and falls under the class of semiparametric regression models that can approximate any continuous function. The results from FFNN as well as FD model also indicate that a majority of events with large absolute prediction errors correspond to those of primates and late events comprising the tail of event timing data distribution with minimal representation in the empirical data. These results also indicate that accurate prediction of primate events may be challenging.     

Time representing cortical activities: two models inspired by prefrontal persistent activity

 Timing information in the range of seconds is significantly correlated with our behavior. There is growing interest in the cognitive behaviors that rely on perception, comparison, or generation of timing. However, little is known about the neural mechanisms underlying such behaviors. Here we model two different neural mechanisms to represent timing information in the range of seconds. In one model, a recurrent network of bistable spiking neurons shows a quasistable state that is initiated by a brief input and typically lasts for a few to several seconds. The duration of this quasistable activity may be regarded as the neural representation of internal time obeying a psychophysical law of time recognition. Another model uses synfire chains to provide the timing information necessary for predicting the times of anticipated events. In this model, the neurons projected to by multiple synfire chains are conditioned to fire synchronously at the times when an external event (GO signal) is expected. The conditioning is accomplished by spike-timing-dependent plasticity. The two models are inspired by the prefrontal activities of the monkeys engaging in different timing-information-related tasks. Thus, this cortical region may provide the timing information required for organizing various behaviors. Received: 12 March 2002 / Accepted in revised form: 26 November 2002 / Published online: 28 March 2003 Correspondence to: T. Fukai (e-mail: tfukai@eng.tamagawa.ac.jp, Tel.: +81-42-7398434, Fax: +81-42-7397135) Acknowledgements. K. Kitano was supported by Japan Society for the Promotion of Science.     

Adaptation supports short-term memory in a visual change detection task

The maintenance of short-term memories is critical for survival in a dynamically changing world. Previous studies suggest that this memory can be stored in the form of persistent neural activity or using a synaptic mechanism, such as with short-term plasticity. Here, we compare the predictions of these two mechanisms to neural and behavioral measurements in a visual change detection task. Mice were trained to respond to changes in a repeated sequence of natural images while neural activity was recorded using two-photon calcium imaging. We also trained two types of artificial neural networks on the same change detection task as the mice. Following fixed pre-processing using a pretrained convolutional neural network, either a recurrent neural network (RNN) or a feedforward neural network with short-term synaptic depression (STPNet) was trained to the same level of performance as the mice. While both networks are able to learn the task, the STPNet model contains units whose activity are more similar to the in vivo data and produces errors which are more similar to the mice. When images are omitted, an unexpected perturbation which was absent during training, mice often do not respond to the omission but are more likely to respond to the subsequent image. Unlike the RNN model, STPNet produces a similar pattern of behavior. These results suggest that simple neural adaptation mechanisms may serve as an important bottom-up memory signal in this task, which can be used by downstream areas in the decision-making process.     

Portrait of visual cortical circuits for generating neural oscillation dynamics

The mouse primary visual cortex (V1) has emerged as a classical system to study neural circuit mechanisms underlying visual function and plasticity. A variety of efferent-afferent neuronal connections exists within the V1 and between the V1 and higher visual cortical areas or thalamic nuclei, indicating that the V1 system is more than a mere receiver in information processing. Sensory representations in the V1 are dynamically correlated with neural activity oscillations that are distributed across different cortical layers in an input-dependent manner. Circuits consisting of excitatory pyramidal cells (PCs) and inhibitory interneurons (INs) are the basis for generating neural oscillations. In general, INs are clustered with their adjacent PCs to form specific microcircuits that gate or filter the neural information. The interaction between these two cell populations has to be coordinated within a local circuit in order to preserve neural coding schemes and maintain excitation–inhibition (E–I) balance. Phasic alternations of the E–I balance can dynamically regulate temporal rhythms of neural oscillation. Accumulating experimental evidence suggests that the two major sub-types of INs, parvalbumin-expressing (PV⁺) cells and somatostatin-expressing (SOM⁺) INs, are active in controlling slow and fast oscillations, respectively, in the mouse V1. The review summarizes recent experimental findings on elucidating cellular or circuitry mechanisms for the generation of neural oscillations with distinct rhythms in either developing or matured mouse V1, mainly focusing on visual relaying circuits and distinct local inhibitory circuits.     

Evolution and Optimality of Similar Neural Mechanisms for Perception and Action during Search

A prevailing theory proposes that the brain''s two visual pathways, the ventral and dorsal, lead to differing visual processing and world representations for conscious perception than those for action. Others have claimed that perception and action share much of their visual processing. But which of these two neural architectures is favored by evolution? Successful visual search is life-critical and here we investigate the evolution and optimality of neural mechanisms mediating perception and eye movement actions for visual search in natural images. We implement an approximation to the ideal Bayesian searcher with two separate processing streams, one controlling the eye movements and the other stream determining the perceptual search decisions. We virtually evolved the neural mechanisms of the searchers'' two separate pathways built from linear combinations of primary visual cortex receptive fields (V1) by making the simulated individuals'' probability of survival depend on the perceptual accuracy finding targets in cluttered backgrounds. We find that for a variety of targets, backgrounds, and dependence of target detectability on retinal eccentricity, the mechanisms of the searchers'' two processing streams converge to similar representations showing that mismatches in the mechanisms for perception and eye movements lead to suboptimal search. Three exceptions which resulted in partial or no convergence were a case of an organism for which the targets are equally detectable across the retina, an organism with sufficient time to foveate all possible target locations, and a strict two-pathway model with no interconnections and differential pre-filtering based on parvocellular and magnocellular lateral geniculate cell properties. Thus, similar neural mechanisms for perception and eye movement actions during search are optimal and should be expected from the effects of natural selection on an organism with limited time to search for food that is not equi-detectable across its retina and interconnected perception and action neural pathways.     

The Attention Modulation on Timing: An Event-Related Potential Study

The present study examined the neural mechanisms of attention modulation on timing using ERP and sLORETA measurements in a dual-task paradigm. We parametrically varied the attention to the durations of a 1000-Hz pure tone and further localized the cortical regions that were sensitive to the attention modulation on timing. Results demonstrated that the attention modulation might happen at early stage, approximately 200 ms after stimulus presentation. The P2 component at frontal area served as an early neural correlate of attention effects on timing. More importantly, the contingent negative variation (CNV) appeared at fronto-central area was sensitive to the attention effect. In addition, the supplementary motor area (SMA) was assumed to be one of the key regions for selectively attending to and estimating time. These findings provide temporal and spatial correlates of attention-modulated time processing and potentially help to investigate the neural mechanisms of patients with time perception deficits.     

The heart and control of renal excretion: neural and endocrine mechanisms.

There has been a great deal of research concerning the heart being an important regulator of renal fluid and electrolyte excretion. This cardiac-renal connection involves two different types of major mechanisms, both of which are covered in this review. The first of these to be discovered was neural reflex regulation. This type of control is due to the fact that the heart possesses nerve receptors whose activity is altered by changes in the degree of cardiac stretch that occur as a result of changes in blood volume. These receptors affect various humoral, neural, and perhaps hemodynamic mechanisms that modify renal excretion. A second, more recently discovered type of regulation is based on the concept that the heart is also an endocrine gland. Similar to neural receptor activity, cardiac hormone secretion is also linked to the degree of cardiac stretch or filling. These cardiac peptides have been shown to have a variety of physiologic effects, most of which directly or indirectly affect renal excretion. Both of the above cardiorenal control mechanisms, one neural and one humoral, may be important not only in maintaining normal fluid-electrolyte balance but may also have pathophysiologic relevance.     

Induction and propagation of transient synchronous activity in neural networks endowed with short-term plasticity

Transient, task related synchronous activity within neural populations has been recognized as the substrate of temporal coding in the brain. The mechanisms underlying inducing and propagation of transient synchronous activity are still unknown, and we propose that short-term plasticity (STP) of neural circuits may serve as a supplemental mechanism therein. By computational modeling, we showed that short-term facilitation greatly increases the reactivation rate of population spikes and decreases the latency of response to reactivation stimuli in local recurrent neural networks. Meanwhile, the timing of population spike reactivation is controlled by the memory effect of STP, and it is mediated primarily by the facilitation time constant. Furthermore, we demonstrated that synaptic facilitation dramatically enhances synchrony propagation in feedforward neural networks and that response timing mediated by synaptic facilitation offers a scheme for information routing. In addition, we verified that synaptic strengthening of intralayer or interlayer coupling enhances synchrony propagation, and we verified that other factors such as the delay of synaptic transmission and the mode of synaptic connectivity are also involved in regulating synchronous activity propagation. Overall, our results highlight the functional role of STP in regulating the inducing and propagation of transient synchronous activity, and they may inspire testable hypotheses for future experimental studies.     

A generative model for measuring latent timing structure in motor sequences

Motor variability often reflects a mixture of different neural and peripheral sources operating over a range of timescales. We present a statistical model of sequence timing that can be used to measure three distinct components of timing variability: global tempo changes that are spread across the sequence, such as might stem from neuromodulatory sources with widespread influence; fast, uncorrelated timing noise, stemming from noisy components within the neural system; and timing jitter that does not alter the timing of subsequent elements, such as might be caused by variation in the motor periphery or by measurement error. In addition to quantifying the variability contributed by each of these latent factors in the data, the approach assigns maximum likelihood estimates of each factor on a trial-to-trial basis. We applied the model to adult zebra finch song, a temporally complex behavior with rich structure on multiple timescales. We find that individual song vocalizations (syllables) contain roughly equal amounts of variability in each of the three components while overall song length is dominated by global tempo changes. Across our sample of syllables, both global and independent variability scale with average length while timing jitter does not, a pattern consistent with the Wing and Kristofferson (1973) model of sequence timing. We also find significant day-to-day drift in all three timing sources, but a circadian pattern in tempo only. In tests using artificially generated data, the model successfully separates out the different components with small error. The approach provides a general framework for extracting distinct sources of timing variability within action sequences, and can be applied to neural and behavioral data from a wide array of systems.     

Understanding the neural control of ingestive behaviors: helping to separate cause from effect with dehydration-associated anorexia

Eating and drinking are motivated behaviors that are made up of coordinated sets of neuroendocrine, autonomic, and behavioral motor events. Although the spinal cord, hindbrain, and hypothalamus contain the motor neurons and circuitry sufficient to maintain the reflex parts of these motor events, inputs from the telencephalon are required to furnish the behavioral components with a motivated (goal-directed) character. Each of these motor events derives from the complex interaction of a variety of sensory inputs with groups of neural networks whose components are distributed throughout the brain and collectively support motor expression and coordination. At a first approximation based on a variety of data, these networks can be divided into three groups: networks that stimulate, those that inhibit, and those that disinhibit motor functions. A fourth contributor is the circadian timing signal that originates in the hypothalamic suprachiasmatic nucleus and provides the temporal anchor for the expression of all behaviors. This article discusses the nature of these networks using neuroanatomical (tract-tracing and neuropeptide in situ hybridization), endocrine, and behavioral evidence from a variety of experimental models. A persistent problem when studying the control of food intake from a neural systems perspective has been the difficulty in separating those neuronal changes that result in hunger from those that are as a consequence of eating. To address this problem, dehydration-associated anorexia is presented as a particularly useful experimental model because it can be used to distinguish between neural mechanisms underlying anorexia and those changes that occur as a consequence of anorexia. The article concludes by highlighting the potential role of neuropeptidergic action in the operation of these networks, using forebrain neuropeptidergic innervation of the parabrachial nucleus as an example.     

Adaptive Robotic Control Driven by a Versatile Spiking Cerebellar Network

The cerebellum is involved in a large number of different neural processes, especially in associative learning and in fine motor control. To develop a comprehensive theory of sensorimotor learning and control, it is crucial to determine the neural basis of coding and plasticity embedded into the cerebellar neural circuit and how they are translated into behavioral outcomes in learning paradigms. Learning has to be inferred from the interaction of an embodied system with its real environment, and the same cerebellar principles derived from cell physiology have to be able to drive a variety of tasks of different nature, calling for complex timing and movement patterns. We have coupled a realistic cerebellar spiking neural network (SNN) with a real robot and challenged it in multiple diverse sensorimotor tasks. Encoding and decoding strategies based on neuronal firing rates were applied. Adaptive motor control protocols with acquisition and extinction phases have been designed and tested, including an associative Pavlovian task (Eye blinking classical conditioning), a vestibulo-ocular task and a perturbed arm reaching task operating in closed-loop. The SNN processed in real-time mossy fiber inputs as arbitrary contextual signals, irrespective of whether they conveyed a tone, a vestibular stimulus or the position of a limb. A bidirectional long-term plasticity rule implemented at parallel fibers-Purkinje cell synapses modulated the output activity in the deep cerebellar nuclei. In all tasks, the neurorobot learned to adjust timing and gain of the motor responses by tuning its output discharge. It succeeded in reproducing how human biological systems acquire, extinguish and express knowledge of a noisy and changing world. By varying stimuli and perturbations patterns, real-time control robustness and generalizability were validated. The implicit spiking dynamics of the cerebellar model fulfill timing, prediction and learning functions.