Use of Frontal Lobe Hemodynamics as Reinforcement Signals to an Adaptive Controller

Decision-making ability in the frontal lobe (among other brain structures) relies on the assignment of value to states of the animal and its environment. Then higher valued states can be pursued and lower (or negative) valued states avoided. The same principle forms the basis for computational reinforcement learning controllers, which have been fruitfully applied both as models of value estimation in the brain, and as artificial controllers in their own right. This work shows how state desirability signals decoded from frontal lobe hemodynamics, as measured with near-infrared spectroscopy (NIRS), can be applied as reinforcers to an adaptable artificial learning agent in order to guide its acquisition of skills. A set of experiments carried out on an alert macaque demonstrate that both oxy- and deoxyhemoglobin concentrations in the frontal lobe show differences in response to both primarily and secondarily desirable (versus undesirable) stimuli. This difference allows a NIRS signal classifier to serve successfully as a reinforcer for an adaptive controller performing a virtual tool-retrieval task. The agent''s adaptability allows its performance to exceed the limits of the NIRS classifier decoding accuracy. We also show that decoding state desirabilities is more accurate when using relative concentrations of both oxyhemoglobin and deoxyhemoglobin, rather than either species alone.     

Decoding Vigilance with NIRS

Sustained, long-term cognitive workload is associated with variations and decrements in performance. Such fluctuations in vigilance can be a risk factor especially during dangerous attention demanding activities. Functional MRI studies have shown that attentional performance is correlated with BOLD-signals, especially in parietal and prefrontal cortical regions. An interesting question is whether these BOLD-signals could be measured in real-world scenarios, say to warn in a dangerous workplace whenever a subjects'' vigilance is low. Because fMRI lacks the mobility needed for such applications, we tested whether the monitoring of vigilance might be possible using Near-Infrared Spectroscopy (NIRS). NIRS is a highly mobile technique that measures hemodynamics in the surface of the brain. We demonstrate that non-invasive NIRS signals correlate with vigilance. These signals carry enough information to decode subjects'' reaction times at a single trial level.     

A Longitudinal Functional Neuroimaging Study in Medication-Na?ve Depression after Antidepressant Treatment

Recent studies have indicated the potential clinical use of near infrared spectroscopy (NIRS) as a tool in assisting the diagnosis of major depressive disorder (MDD); however, it is still unclear whether NIRS signal changes during cognitive task are state- or trait-dependent, and whether NIRS could be a neural predictor of treatment response. Therefore, we conducted a longitudinal study to explore frontal haemodynamic changes following antidepressant treatment in medication-naïve MDD using 52-channel NIRS. This study included 25 medication-naïve individuals with MDD and 62 healthy controls (HC). We performed NIRS scans before and after antidepressant treatment and measured changes of [oxy-Hb] activation during a verbal fluency task (VFT) following treatment. Individuals with MDD showed significantly decreased [oxy-Hb] values during a VFT compared with HC in the bilateral frontal and temporal cortices at baseline. There were no [oxy-Hb] changes between pre- and post-antidepressant treatment time points in the MDD cohort despite significant improvement in depressive symptoms. There was a significant association between mean [oxy-Hb] values during a VFT at baseline and improvement in depressive symptoms following treatment in the bilateral inferior frontal and middle temporal gyri in MDD. These findings suggest that hypofrontality response to a VFT may represent a potential trait marker for depression rather than a state marker. Moreover, the correlation analysis indicates that the NIRS signals before the initiation of treatment may be a biological marker to predict patient’s clinical response to antidepressant treatment. The present study provides further evidence to support a potential application of NIRS for the diagnosis and treatment of depression.     

A Neural Network Approach to fMRI Binocular Visual Rivalry Task Analysis

The purpose of this study was to investigate whether artificial neural networks (ANN) are able to decode participants’ conscious experience perception from brain activity alone, using complex and ecological stimuli. To reach the aim we conducted pattern recognition data analysis on fMRI data acquired during the execution of a binocular visual rivalry paradigm (BR). Twelve healthy participants were submitted to fMRI during the execution of a binocular non-rivalry (BNR) and a BR paradigm in which two classes of stimuli (faces and houses) were presented. During the binocular rivalry paradigm, behavioral responses related to the switching between consciously perceived stimuli were also collected. First, we used the BNR paradigm as a functional localizer to identify the brain areas involved the processing of the stimuli. Second, we trained the ANN on the BNR fMRI data restricted to these regions of interest. Third, we applied the trained ANN to the BR data as a ‘brain reading’ tool to discriminate the pattern of neural activity between the two stimuli. Fourth, we verified the consistency of the ANN outputs with the collected behavioral indicators of which stimulus was consciously perceived by the participants. Our main results showed that the trained ANN was able to generalize across the two different tasks (i.e. BNR and BR) and to identify with high accuracy the cognitive state of the participants (i.e. which stimulus was consciously perceived) during the BR condition. The behavioral response, employed as control parameter, was compared with the network output and a statistically significant percentage of correspondences (p-value <0.05) were obtained for all subjects. In conclusion the present study provides a method based on multivariate pattern analysis to investigate the neural basis of visual consciousness during the BR phenomenon when behavioral indicators lack or are inconsistent, like in disorders of consciousness or sedated patients.     

Song Competition Affects Monoamine Levels in Sensory and Motor Forebrain Regions of Male Lincoln's Sparrows (Melospiza lincolnii)

Male animals often change their behavior in response to the level of competition for mates. Male Lincoln''s sparrows (Melospiza lincolnii) modulate their competitive singing over the period of a week as a function of the level of challenge associated with competitors'' songs. Differences in song challenge and associated shifts in competitive state should be accompanied by neural changes, potentially in regions that regulate perception and song production. The monoamines mediate neural plasticity in response to environmental cues to achieve shifts in behavioral state. Therefore, using high pressure liquid chromatography with electrochemical detection, we compared levels of monoamines and their metabolites from male Lincoln''s sparrows exposed to songs categorized as more or less challenging. We compared levels of norepinephrine and its principal metabolite in two perceptual regions of the auditory telencephalon, the caudomedial nidopallium and the caudomedial mesopallium (CMM), because this chemical is implicated in modulating auditory sensitivity to song. We also measured the levels of dopamine and its principal metabolite in two song control nuclei, area X and the robust nucleus of the arcopallium (RA), because dopamine is implicated in regulating song output. We measured the levels of serotonin and its principal metabolite in all four brain regions because this monoamine is implicated in perception and behavioral output and is found throughout the avian forebrain. After controlling for recent singing, we found that males exposed to more challenging song had higher levels of norepinephrine metabolite in the CMM and lower levels of serotonin in the RA. Collectively, these findings are consistent with norepinephrine in perceptual brain regions and serotonin in song control regions contributing to neuroplasticity that underlies socially-induced changes in behavioral state.     

Temporal Dynamics of Visual Attention Measured with Event-Related Potentials

How attentional modulation on brain activities determines behavioral performance has been one of the most important issues in cognitive neuroscience. This issue has been addressed by comparing the temporal relationship between attentional modulations on neural activities and behavior. Our previous study measured the time course of attention with amplitude and phase coherence of steady-state visual evoked potential (SSVEP) and found that the modulation latency of phase coherence rather than that of amplitude was consistent with the latency of behavioral performance. In this study, as a complementary report, we compared the time course of visual attention shift measured by event-related potentials (ERPs) with that by target detection task. We developed a novel technique to compare ERPs with behavioral results and analyzed the EEG data in our previous study. Two sets of flickering stimulus at different frequencies were presented in the left and right visual hemifields, and a target or distracter pattern was presented randomly at various moments after an attention-cue presentation. The observers were asked to detect targets on the attended stimulus after the cue. We found that two ERP components, P300 and N2pc, were elicited by the target presented at the attended location. Time-course analyses revealed that attentional modulation of the P300 and N2pc amplitudes increased gradually until reaching a maximum and lasted at least 1.5 s after the cue onset, which is similar to the temporal dynamics of behavioral performance. However, attentional modulation of these ERP components started later than that of behavioral performance. Rather, the time course of attentional modulation of behavioral performance was more closely associated with that of the concurrently recorded SSVEPs analyzed. These results suggest that neural activities reflected not by either the P300 or N2pc, but by the SSVEPs, are the source of attentional modulation of behavioral performance.     

The temporal resolution of neural codes: does response latency have a unique role?

This article reviews the nature of the neural code in non-human primate cortex and assesses the potential for neurons to carry two or more signals simultaneously. Neurophysiological recordings from visual and motor systems indicate that the evidence for a role for precisely timed spikes relative to other spike times (ca. 1-10 ms resolution) is inconclusive. This indicates that the visual system does not carry a signal that identifies whether the responses were elicited when the stimulus was attended or not. Simulations show that the absence of such a signal reduces, but does not eliminate, the increased discrimination between stimuli that are attended compared with when the stimuli are unattended. The increased accuracy asymptotes with increased gain control, indicating limited benefit from increasing attention. The absence of a signal identifying the attentional state under which stimuli were viewed can produce the greatest discrimination between attended and unattended stimuli. Furthermore, the greatest reduction in discrimination errors occurs for a limited range of gain control, again indicating that attention effects are limited. By contrast to precisely timed patterns of spikes where the timing is relative to other spikes, response latency provides a fine temporal resolution signal (ca. 10 ms resolution) that carries information that is unavailable from coarse temporal response measures. Changes in response latency and changes in response magnitude can give rise to different predictions for the patterns of reaction times. The predictions are verified, and it is shown that the standard method for distinguishing executive and slave processes is only valid if the representations of interest, as evidenced by the neural code, are known. Overall, the data indicate that the signalling evident in neural signals is restricted to the spike count and the precise times of spikes relative to stimulus onset (response latency). These coding issues have implications for our understanding of cognitive models of attention and the roles of executive and slave systems.     

Reading spike timing without a clock: intrinsic decoding of spike trains

The precise timing of action potentials of sensory neurons relative to the time of stimulus presentation carries substantial sensory information that is lost or degraded when these responses are summed over longer time windows. However, it is unclear whether and how downstream networks can access information in precise time-varying neural responses. Here, we review approaches to test the hypothesis that the activity of neural populations provides the temporal reference frames needed to decode temporal spike patterns. These approaches are based on comparing the single-trial stimulus discriminability obtained from neural codes defined with respect to network-intrinsic reference frames to the discriminability obtained from codes defined relative to the experimenter''s computer clock. Application of this formalism to auditory, visual and somatosensory data shows that information carried by millisecond-scale spike times can be decoded robustly even with little or no independent external knowledge of stimulus time. In cortex, key components of such intrinsic temporal reference frames include dedicated neural populations that signal stimulus onset with reliable and precise latencies, and low-frequency oscillations that can serve as reference for partitioning extended neuronal responses into informative spike patterns.     

The relationship between female urinary estrogen excretion and mating behavior in cotton-topped tamarins,Saguinus oedipus oedipus

Daily urinary estrogen excretion in six singly housed adult female cottontopped tamarins was measured by radioimmunoassay. Each female was paired with a male for 1 hr every day, during which behavioral observations were conducted on the pair from behind a oneway glass screen. During each observation period the duration or frequency of occurrence of various behaviors was recorded. In this paper, the daily variations of (1) total frequency of male mounting, (2) total frequency of male mounting with thrusting, and (3) latency to first male mount are presented in relation to the measured pattern of female estrogen excretion. The results reveal a high degree of behavioral variation between pairs, although a relationship between rhythmic changes in sexual activity and female estrogen excretion in individual pairs is apparent. Mating occurs throughout the female cycle and also during at least the first 2–3 weeks of pregnancy. A gestation period of 170–172 days is estimated in respect of one female, in agreement with a previous observation of one gestation of at least 166 days. The high frequency and intensity of mating recorded in this controlledintroduction study give support to the proposition that mating is important in pairbonding in this monogamous species. It is concluded that under the experimental procedure followed, mating behavior did not reliably reflect the female’s reproductive hormonal state.     

Monotony and the Information Content of Richardson's Ground Squirrel (Spermophilus richardsonii) Repeated calls: Tonic Communication or Signal Certainty?

The repetition of elements within an alarm signal is commonly thought to ensure that receivers have detected that signal, or to promote residual vigilance in light of the dangerous circumstances prompting the signal's initial production (tonic communication). Beyond alerting others and maintaining that state of alertness, however, repetitive signal elements may be parsed so as to encode information about the nature of potential threats. To determine how call length and variation in intersyllable latency might prove informative in the repetitive alarm vocalizations of Richardson's ground squirrels (Spermophilus richardsonii), we conducted a field‐based playback experiment quantifying antipredator responses to manipulated alarm calls. Free‐living juvenile squirrels were exposed to playbacks of 12 syllable (long) and six syllable (short) calls with a constant (monotonous) or changing (variable) call rate. The length of calls had no significant effect on the behaviour of call recipients during and immediately after call production; however, call recipients showed greater vigilance after the playback of monotonous calls than after variable calls. The absence of a call length effect is not consistent with tonic communication in the strict sense; rather, enhanced responsiveness to monotonous relative to variable calls suggests that multiple syllables, and the emergent patterns of intersyllable latency, communicate information about response urgency or the distance to a predatory threat. Only monotonous calls convey those aspects with any certainty on the part of the signaller and hence are selectively attended to by receivers.     

Signal acquisition and analysis for cortical control of neuroprosthetics

Work in cortically controlled neuroprosthetic systems has concentrated on decoding natural behaviors from neural activity, with the idea that if the behavior could be fully decoded it could be duplicated using an artificial system. Initial estimates from this approach suggested that a high-fidelity signal comprised of many hundreds of neurons would be required to control a neuroprosthetic system successfully. However, recent studies are showing hints that these systems can be controlled effectively using only a few tens of neurons. Attempting to decode the pre-existing relationship between neural activity and natural behavior is not nearly as important as choosing a decoding scheme that can be more readily deployed and trained to generate the desired actions of the artificial system. These artificial systems need not resemble or behave similarly to any natural biological system. Effective matching of discrete and continuous neural command signals to appropriately configured device functions will enable effective control of both natural and abstract artificial systems using compatible thought processes.     

Hemodynamic response of the frontal cortex elicited by intravenous thiamine propyldisulphide administration

The intravenous olfaction (IVO) test is a unique type of clinical olfactometry and is widely used in Japan. However, it is difficult to distinguish actual olfactory disturbance from feigned disturbance because the IVO test is a psychophysical test. To resolve this problem, we investigated the possibility of an objective IVO test assisted with near infrared spectroscopy (NIRS). IVO testing was performed according to the usual protocol with thiamine propyldisulphide (alinamin) administration. The relative oxy- and deoxyhemoglobin levels of the orbitofrontal area during olfactory stimulation by IVO test were measured by NIRS. Pairs of NIRS emitters and detectors were positioned on the bilateral frontal scalp. After administration of alinamin, oxyhemoglobin levels increased, though deoxyhemoglobin levels did not change. An increase in oxyhemoglobin levels was observed bilaterally. Administration of saline did not elicit any change in the oxy- or deoxyhemoglobin levels and concentration of the administered alinamin related increasing of the oxyhemoglobin level was observed. Oxyhemoglobin remained unchanged in anosmic subjects despite administration of alinamin. The latency of oxyhemoglobin increase on each side and smelling latency showed significant correlation. Latencies of oxyhemoglobin increases between the right and left sides also showed significant correlation. Oxyhemoglobin response appears to be linked to olfactory related response. NIRS is a useful technique for the development of an objective form of IVO testing.     

Scale‐dependence of phylogenetic signal in ecological traits of ectoparasites

The non‐independence of traits among closely related species is a well‐documented phenomenon underpinning modern methods for comparative analyses or prediction of trait values in new species. Surprisingly such studies have mainly focused on life‐history or morphological traits of free‐living organisms, ignoring ecological attributes of parasite species in spite of the fact that they are critical for conservation and human health. We tested for a phylogenetic signal acting on two ecological traits, abundance and host specificity, using data for 218 flea species parasitic on small mammals in 19 regions of the Palaearctic and Nearctic, and a phylogenetic tree for these species. We tested for the presence of a phylogenetic signal at both regional and continental scales using three measures (Abouheif/Moran's I, Pagel's λ, and Blomberg et al.'s K). Our results show 1) a consistent positive phylogenetic signal for flea abundance, but only a weaker and erratic signal for host specificity, and 2) a clear dependence on scale, with the signals being stronger at the continental scale and relatively weaker or inconsistent at the regional scale. Whenever values of Blomberg et al.'s K were found significant, they were <1 suggesting that the effects of phylogeny on the evolution of abundance and host specificity in fleas are weaker than expected from a Brownian motion model. The most striking finding is that, within a continental fauna, closely‐related flea species are characterized by similar levels of abundance, though this pattern is weaker within local assemblages, possibly eroded by local biotic or abiotic conditions. We discuss the link between history (represented by phylogeny) and pattern of variation among species in morphological and ecological traits, and use comparisons between the Palaearctic and Nearctic to infer a role of historical events in the probability of detecting phylogenetic signals.     

Ethanol-Induced Effects on Sting Extension Response and Punishment Learning in the Western Honey Bee (Apis mellifera)

Acute ethanol administration is associated with sedation and analgesia as well as behavioral disinhibition and memory loss but the mechanisms underlying these effects remain to be elucidated. During the past decade, insects have emerged as important model systems to understand the neural and genetic bases of alcohol effects. However, novel assays to assess ethanol''s effects on complex behaviors in social or isolated contexts are necessary. Here we used the honey bee as an especially relevant model system since bees are typically exposed to ethanol in nature when collecting standing nectar crop of flowers, and there is recent evidence for independent biological significance of this exposure for social behavior. Bee''s inhibitory control of the sting extension response (SER) and a conditioned-place aversion assay were used to study ethanol effects on analgesia, behavioral disinhibition, and associative learning. Our findings indicate that although ethanol, in a dose-dependent manner, increases SER thresholds (analgesic effects), it disrupts the ability of honey bees to inhibit SER and to associate aversive stimuli with their environment. These results suggest that ethanol''s effects on analgesia, behavioral disinhibition and associative learning are common across vertebrates and invertebrates. These results add to the use of honey bees as an ethanol model to understand ethanol''s effects on complex, socially relevant behaviors.     

Encoder-Decoder Optimization for Brain-Computer Interfaces

Neuroprosthetic brain-computer interfaces are systems that decode neural activity into useful control signals for effectors, such as a cursor on a computer screen. It has long been recognized that both the user and decoding system can adapt to increase the accuracy of the end effector. Co-adaptation is the process whereby a user learns to control the system in conjunction with the decoder adapting to learn the user''s neural patterns. We provide a mathematical framework for co-adaptation and relate co-adaptation to the joint optimization of the user''s control scheme ("encoding model") and the decoding algorithm''s parameters. When the assumptions of that framework are respected, co-adaptation cannot yield better performance than that obtainable by an optimal initial choice of fixed decoder, coupled with optimal user learning. For a specific case, we provide numerical methods to obtain such an optimized decoder. We demonstrate our approach in a model brain-computer interface system using an online prosthesis simulator, a simple human-in-the-loop pyschophysics setup which provides a non-invasive simulation of the BCI setting. These experiments support two claims: that users can learn encoders matched to fixed, optimal decoders and that, once learned, our approach yields expected performance advantages.     

Decoding Accuracy in Supplementary Motor Cortex Correlates with Perceptual Sensitivity to Tactile Roughness

Perceptual sensitivity to tactile roughness varies across individuals for the same degree of roughness. A number of neurophysiological studies have investigated the neural substrates of tactile roughness perception, but the neural processing underlying the strong individual differences in perceptual roughness sensitivity remains unknown. In this study, we explored the human brain activation patterns associated with the behavioral discriminability of surface texture roughness using functional magnetic resonance imaging (fMRI). First, a whole-brain searchlight multi-voxel pattern analysis (MVPA) was used to find brain regions from which we could decode roughness information. The searchlight MVPA revealed four brain regions showing significant decoding results: the supplementary motor area (SMA), contralateral postcentral gyrus (S1), and superior portion of the bilateral temporal pole (STP). Next, we evaluated the behavioral roughness discrimination sensitivity of each individual using the just-noticeable difference (JND) and correlated this with the decoding accuracy in each of the four regions. We found that only the SMA showed a significant correlation between neuronal decoding accuracy and JND across individuals; Participants with a smaller JND (i.e., better discrimination ability) exhibited higher decoding accuracy from their voxel response patterns in the SMA. Our findings suggest that multivariate voxel response patterns presented in the SMA represent individual perceptual sensitivity to tactile roughness and people with greater perceptual sensitivity to tactile roughness are likely to have more distinct neural representations of different roughness levels in their SMA.     

Neuroimaging: seeing the trees for the forest

New functional imaging studies demonstrate that it is possible to decode a sensory visual pattern, and even an internal perceptual state, by combining seemingly insignificant feature selective signal biases present in a large number of voxels.     

Spatiotemporal dynamics of human high gamma discriminate naturalistic behavioral states

In analyzing the neural correlates of naturalistic and unstructured behaviors, features of neural activity that are ignored in a trial-based experimental paradigm can be more fully studied and investigated. Here, we analyze neural activity from two patients using electrocorticography (ECoG) and stereo-electroencephalography (sEEG) recordings, and reveal that multiple neural signal characteristics exist that discriminate between unstructured and naturalistic behavioral states such as “engaging in dialogue” and “using electronics”. Using the high gamma amplitude as an estimate of neuronal firing rate, we demonstrate that behavioral states in a naturalistic setting are discriminable based on long-term mean shifts, variance shifts, and differences in the specific neural activity’s covariance structure. Both the rapid and slow changes in high gamma band activity separate unstructured behavioral states. We also use Gaussian process factor analysis (GPFA) to show the existence of salient spatiotemporal features with variable smoothness in time. Further, we demonstrate that both temporally smooth and stochastic spatiotemporal activity can be used to differentiate unstructured behavioral states. This is the first attempt to elucidate how different neural signal features contain information about behavioral states collected outside the conventional experimental paradigm.     

Cross-Modal Multivariate Pattern Analysis

Multivariate pattern analysis (MVPA) is an increasingly popular method of analyzing functional magnetic resonance imaging (fMRI) data^1-4. Typically, the method is used to identify a subject''s perceptual experience from neural activity in certain regions of the brain. For instance, it has been employed to predict the orientation of visual gratings a subject perceives from activity in early visual cortices⁵ or, analogously, the content of speech from activity in early auditory cortices⁶.Here, we present an extension of the classical MVPA paradigm, according to which perceptual stimuli are not predicted within, but across sensory systems. Specifically, the method we describe addresses the question of whether stimuli that evoke memory associations in modalities other than the one through which they are presented induce content-specific activity patterns in the sensory cortices of those other modalities. For instance, seeing a muted video clip of a glass vase shattering on the ground automatically triggers in most observers an auditory image of the associated sound; is the experience of this image in the "mind''s ear" correlated with a specific neural activity pattern in early auditory cortices? Furthermore, is this activity pattern distinct from the pattern that could be observed if the subject were, instead, watching a video clip of a howling dog?In two previous studies^7,8, we were able to predict sound- and touch-implying video clips based on neural activity in early auditory and somatosensory cortices, respectively. Our results are in line with a neuroarchitectural framework proposed by Damasio^9,10, according to which the experience of mental images that are based on memories - such as hearing the shattering sound of a vase in the "mind''s ear" upon seeing the corresponding video clip - is supported by the re-construction of content-specific neural activity patterns in early sensory cortices.     

Cortical local field potential encodes movement intentions in the posterior parietal cortex

The cortical local field potential (LFP) is a summation signal of excitatory and inhibitory dendritic potentials that has recently become of increasing interest. We report that LFP signals in the parietal reach region (PRR) of the posterior parietal cortex of macaque monkeys have temporal structure that varies with the type of planned or executed motor behavior. LFP signals from PRR provide better decode performance for reaches compared to saccades and have stronger coherency with simultaneously recorded spiking activity during the planning of reach movements than during saccade planning. LFP signals predict the animal's behavioral state (e.g., planning a reach or saccade) and the direction of the currently planned movement from single-trial information. This new evidence provides further support for a role of the parietal cortex in movement planning and the potential application of LFP signals for a brain-machine interface.