Mental imagery and visual working memory

Visual working memory provides an essential link between past and future events. Despite recent efforts, capacity limits, their genesis and the underlying neural structures of visual working memory remain unclear. Here we show that performance in visual working memory--but not iconic visual memory--can be predicted by the strength of mental imagery as assessed with binocular rivalry in a given individual. In addition, for individuals with strong imagery, modulating the background luminance diminished performance on visual working memory and imagery tasks, but not working memory for number strings. This suggests that luminance signals were disrupting sensory-based imagery mechanisms and not a general working memory system. Individuals with poor imagery still performed above chance in the visual working memory task, but their performance was not affected by the background luminance, suggesting a dichotomy in strategies for visual working memory: individuals with strong mental imagery rely on sensory-based imagery to support mnemonic performance, while those with poor imagery rely on different strategies. These findings could help reconcile current controversy regarding the mechanism and location of visual mnemonic storage.     

Mapping Genetically Controlled Neural Circuits of Social Behavior and Visuo-Motor Integration by a Preliminary Examination of Atypical Deletions with Williams Syndrome

In this study of eight rare atypical deletion cases with Williams-Beuren syndrome (WS; also known as 7q11.23 deletion syndrome) consisting of three different patterns of deletions, compared to typical WS and typically developing (TD) individuals, we show preliminary evidence of dissociable genetic contributions to brain structure and human cognition. Univariate and multivariate pattern classification results of morphometric brain patterns complemented by behavior implicate a possible role for the chromosomal region that includes: 1) GTF2I/GTF2IRD1 in visuo-spatial/motor integration, intraparietal as well as overall gray matter structures, 2) the region spanning ABHD11 through RFC2 including LIMK1, in social cognition, in particular approachability, as well as orbitofrontal, amygdala and fusiform anatomy, and 3) the regions including STX1A, and/or CYLN2 in overall white matter structure. This knowledge contributes to our understanding of the role of genetics on human brain structure, cognition and pathophysiology of altered cognition in WS. The current study builds on ongoing research designed to characterize the impact of multiple genes, gene-gene interactions and changes in gene expression on the human brain.     

Reduced Crowding and Poor Contour Detection in Schizophrenia Are Consistent with Weak Surround Inhibition

Background

Detection of visual contours (strings of small oriented elements) is markedly poor in schizophrenia. This has previously been attributed to an inability to group local information across space into a global percept. Here, we show that this failure actually originates from a combination of poor encoding of local orientation and abnormal processing of visual context.

Methods

We measured the ability of observers with schizophrenia to localise contours embedded in backgrounds of differently oriented elements (either randomly oriented, near-parallel or near-perpendicular to the contour). In addition, we measured patients’ ability to process local orientation information (i.e., report the orientation of an individual element) for both isolated and crowded elements (i.e., presented with nearby distractors).

Results

While patients are poor at detecting contours amongst randomly oriented elements, they are proportionally less disrupted (compared to unaffected controls) when contour and surrounding elements have similar orientations (near-parallel condition). In addition, patients are poor at reporting the orientation of an individual element but, again, are less prone to interference from nearby distractors, a phenomenon known as visual crowding.

Conclusions

We suggest that patients’ poor performance at contour perception arises not as a consequence of an “integration deficit” but from a combination of reduced sensitivity to local orientation and abnormalities in contextual processing. We propose that this is a consequence of abnormal gain control, a phenomenon that has been implicated in orientation-selectivity as well as surround suppression.     

Personality effects on spatial learning: Comparisons between visual conditions in a weakly electric fish

Recent research has explored links between cognition and personality, with prominent hypotheses proposing that personality drives consistent individual differences in cognitive function. These hypotheses particularly expect bolder individuals to be faster, but less accurate, as a trade‐off in cognitive function. However, cognitive processes are typically interconnected and defined in more complex terms than simply speed and accuracy. Here, we present evidence that personality‐based differences in learning rates are a result of differences in decision‐making during training in a two‐alternative forced‐choice spatial memory task. This was examined in the mormyrid fish Gnathonemus petersii in the presence of light, where both vision and the electric sense are available, and in the dark, where visibility is limited and fish rely mostly on electrosensing. The species is adapted for the dark to avoid visual predators; thus, the presence of light can induce high‐risk and the dark low‐risk. We show that light conditions had little effect on learning, with bolder fish learning faster both in the light and in the dark conditions. Yet the relationship between learning rates and error rates indicates that the effect on learning is indirectly influenced by accuracy during training. Speed‐accuracy trade‐offs were not found in decision‐making, with bolder individuals deciding faster and more accurately both in the light and in the dark. Only learning strategy was affected by light conditions, with significantly more fish preferring response to place learning in the dark than in the light, where distal cues were not visible. We conclude that other than effects from the integration of visual information, bolder individuals show a consistently greater tendency to explore and find food rewards during training. This affects their decision‐making and in turn their learning performance. We highlight the complexity by which personality‐based effects are exhibited in spatial associative learning.     

Abnormal Contextual Modulation of Visual Contour Detection in Patients with Schizophrenia

Schizophrenia patients demonstrate perceptual deficits consistent with broad dysfunction in visual context processing. These include poor integration of segments forming visual contours, and reduced visual contrast effects (e.g. weaker orientation-dependent surround suppression, ODSS). Background image context can influence contour perception, as stimuli near the contour affect detection accuracy. Because of ODSS, this contextual modulation depends on the relative orientation between the contour and flanking elements, with parallel flankers impairing contour perception. However in schizophrenia, the impact of abnormal ODSS during contour perception is not clear. It is also unknown whether deficient contour perception marks genetic liability for schizophrenia, or is strictly associated with clinical expression of this disorder. We examined contour detection in 25 adults with schizophrenia, 13 unaffected first-degree biological relatives of schizophrenia patients, and 28 healthy controls. Subjects performed a psychophysics experiment designed to quantify the effect of flanker orientation during contour detection. Overall, patients with schizophrenia showed poorer contour detection performance than relatives or controls. Parallel flankers suppressed and orthogonal flankers enhanced contour detection performance for all groups, but parallel suppression was relatively weaker for schizophrenia patients than healthy controls. Relatives of patients showed equivalent performance with controls. Computational modeling suggested that abnormal contextual modulation in schizophrenia may be explained by suppression that is more broadly tuned for orientation. Abnormal flanker suppression in schizophrenia is consistent with weaker ODSS and/or broader orientation tuning. This work provides the first evidence that such perceptual abnormalities may not be associated with a genetic liability for schizophrenia.     

Learning to link visual contours

In complex visual scenes, linking related contour elements is important for object recognition. This process, thought to be stimulus driven and hard wired, has substrates in primary visual cortex (V1). Here, however, we find contour integration in V1 to depend strongly on perceptual learning and top-down influences that are specific to contour detection. In naive monkeys, the information about contours embedded in complex backgrounds is absent in V1 neuronal responses and is independent of the locus of spatial attention. Training animals to find embedded contours induces strong contour-related responses specific to the trained retinotopic region. These responses are most robust when animals perform the contour detection task but disappear under anesthesia. Our findings suggest that top-down influences dynamically adapt neural circuits according to specific perceptual tasks. This may serve as a general neuronal mechanism of perceptual learning and reflect top-down mediated changes in cortical states.     

Contour integration and segmentation with self-organized lateral connections

Contour integration in low-level vision is believed to occur based on lateral interaction between neurons with similar orientation tuning. How such interactions could arise in the brain has been an open question. Our model suggests that the interactions can be learned through input-driven self-organization, i.e., through the same mechanism that underlies many other developmental and functional phenomena in the visual cortex. The model also shows how synchronized firing mediated by these lateral connections can represent the percept of a contour, resulting in performance similar to that of human contour integration. The model further demonstrates that contour integration performance can differ in different parts of the visual field, depending on what kinds of input distributions they receive during development. The model thus grounds an important perceptual phenomenon onto detailed neural mechanisms so that various structural and functional properties can be measured and predictions can be made to guide future experiments.     

High visual working memory capacity in trait social anxiety

Working memory capacity is one of the most important cognitive functions influencing individual traits, such as attentional control, fluid intelligence, and also psychopathological traits. Previous research suggests that anxiety is associated with impaired cognitive function, and studies have shown low verbal working memory capacity in individuals with high trait anxiety. However, the relationship between trait anxiety and visual working memory capacity is still unclear. Considering that people allocate visual attention more widely to detect danger under threat, visual working memory capacity might be higher in anxious people. In the present study, we show that visual working memory capacity increases as trait social anxiety increases by using a change detection task. When the demand to inhibit distractors increased, however, high visual working memory capacity diminished in individuals with social anxiety, and instead, impaired filtering of distractors was predicted by trait social anxiety. State anxiety was not correlated with visual working memory capacity. These results indicate that socially anxious people could potentially hold a large amount of information in working memory. However, because of an impaired cognitive function, they could not inhibit goal-irrelevant distractors and their performance decreased under highly demanding conditions.     

Regional brain differences in cortical thickness, surface area and subcortical volume in individuals with Williams syndrome

Williams syndrome (WS) is a rare genetic neurodevelopmental disorder characterized by increased non-social anxiety, sensitivity to sounds and hypersociability. Previous studies have reported contradictory findings with regard to regional brain variation in WS, relying on only one type of morphological measure (usually volume) in each study. The present study aims to contribute to this body of literature and perhaps elucidate some of these discrepancies by examining concurrent measures of cortical thickness, surface area and subcortical volume between WS subjects and typically-developing (TD) controls. High resolution MRI scans were obtained on 31 WS subjects and 50 typically developing control subjects. We derived quantitative regional estimates of cortical thickness, cortical surface area, and subcortical volume using FreeSurfer software. We evaluated between-group ROI differences while controlling for total intracranial volume. In post-hoc exploratory analyses within the WS group, we tested for correlations between regional brain variation and Beck Anxiety Inventory scores. Consistent with our hypothesis, we detected complex patterns of between-group cortical variation, which included lower surface area in combination with greater thickness in the following cortical regions: post central gyrus, cuneus, lateral orbitofrontal cortex and lingual gyrus. Additional cortical regions showed between-group differences in one (but not both) morphological measures. Subcortical volume was lower in the basal ganglia and the hippocampus in WS versus TD controls. Exploratory correlations revealed that anxiety scores were negatively correlated with gray matter surface area in insula, OFC, rostral middle frontal, superior temporal and lingual gyrus. Our results were consistent with previous reports showing structural alterations in regions supporting the socio-affective and visuospatial impairments in WS. However, we also were able to effectively capture novel and complex patterns of cortical differences using both surface area and thickness. In addition, correlation results implicate specific brain regions in levels of anxiety in WS, consistent with previous reports investigating general anxiety disorders in the general population.     

Long-term Retention of Many Visual Patterns by Pigeons

Using a simultaneous discrimination procedure it was shown that pigeons were capable of learning to discriminate 100 different black and white visual patterns from a further 625 similar stimuli, where responses to the former were rewarded and responses to the latter were not rewarded. Tests in which novel stimuli replaced either the rewarded or nonrewarded stimuli showed that the pigeons had not only learned about the 100 positive stimuli but also about the 625 negative stimuli. The fact that novel stimuli enhanced discrimination performance when they replaced the many negative stimuli indicated that the pigeons had categorized the stimuli into two classes, familiar and less familiar. Long-term retention was examined after a 6-month interval. To begin with it seemed poor but a recognition test performed after the subjects had been retrained with a subset of the stimuli after an interval of 7 months confirmed that pigeons are capable of retaining in memory several 100 visual items over an extended period. It is proposed that the initial retrieval weakness was due to a forgetting of the contingencies between stimulus categories and response outcomes. Further tests involving variously modified stimuli indicated that while stimulus size variations had a negative effect on performance, orientation changes did not interfere with recognition, supporting the view that small visual stimuli are memorized by pigeons largely free of orientation labels. The experiment generally confirms that pigeons have the capacity of storing information about a large number of visual stimuli over long periods of time.     

Computational modeling and exploration of contour integration for visual saliency

We propose a computational model of contour integration for visual saliency. The model uses biologically plausible devices to simulate how the representations of elements aligned collinearly along a contour in an image are enhanced. Our model adds such devices as a dopamine-like fast plasticity, local GABAergic inhibition and multi-scale processing of images. The fast plasticity addresses the problem of how neurons in visual cortex seem to be able to influence neurons they are not directly connected to, for instance, as observed in contour closure effect. Local GABAergic inhibition is used to control gain in the system without using global mechanisms which may be non-plausible given the limited reach of axonal arbors in visual cortex. The model is then used to explore not only its validity in real and artificial images, but to discover some of the mechanisms involved in processing of complex visual features such as junctions and end-stops as well as contours. We present evidence for the validity of our model in several phases, starting with local enhancement of only a few collinear elements. We then test our model on more complex contour integration images with a large number of Gabor elements. Sections of the model are also extracted and used to discover how the model might relate contour integration neurons to neurons that process end-stops and junctions. Finally, we present results from real world images. Results from the model suggest that it is a good current approximation of contour integration in human vision. As well, it suggests that contour integration mechanisms may be strongly related to mechanisms for detecting end-stops and junction points. Additionally, a contour integration mechanism may be involved in finding features for objects such as faces. This suggests that visual cortex may be more information efficient and that neural regions may have multiple roles.     

Contour saliency in primary visual cortex

Contour integration is an important intermediate stage of object recognition, in which line segments belonging to an object boundary are perceptually linked and segmented from complex backgrounds. Contextual influences observed in primary visual cortex (V1) suggest the involvement of V1 in contour integration. Here, we provide direct evidence that, in monkeys performing a contour detection task, there was a close correlation between the responses of V1 neurons and the perceptual saliency of contours. Receiver operating characteristic analysis showed that single neuronal responses encode the presence or absence of a contour as reliably as the animal's behavioral responses. We also show that the same visual contours elicited significantly weaker neuronal responses when they were not detected in the detection task, or when they were unattended. Our results demonstrate that contextual interactions in V1 play a pivotal role in contour integration and saliency.     

Oxytocin and vasopressin are dysregulated in Williams Syndrome, a genetic disorder affecting social behavior

The molecular and neural mechanisms regulating human social-emotional behaviors are fundamentally important but largely unknown; unraveling these requires a genetic systems neuroscience analysis of human models. Williams Syndrome (WS), a condition caused by deletion of ~28 genes, is associated with a gregarious personality, strong drive to approach strangers, difficult peer interactions, and attraction to music. WS provides a unique opportunity to identify endogenous human gene-behavior mechanisms. Social neuropeptides including oxytocin (OT) and arginine vasopressin (AVP) regulate reproductive and social behaviors in mammals, and we reasoned that these might mediate the features of WS. Here we established blood levels of OT and AVP in WS and controls at baseline, and at multiple timepoints following a positive emotional intervention (music), and a negative physical stressor (cold). We also related these levels to standardized indices of social behavior. Results revealed significantly higher median levels of OT in WS versus controls at baseline, with a less marked increase in AVP. Further, in WS, OT and AVP increased in response to music and to cold, with greater variability and an amplified peak release compared to controls. In WS, baseline OT but not AVP, was correlated positively with approach, but negatively with adaptive social behaviors. These results indicate that WS deleted genes perturb hypothalamic-pituitary release not only of OT but also of AVP, implicating more complex neuropeptide circuitry for WS features and providing evidence for their roles in endogenous regulation of human social behavior. The data suggest a possible biological basis for amygdalar involvement, for increased anxiety, and for the paradox of increased approach but poor social relationships in WS. They also offer insight for translating genetic and neuroendocrine knowledge into treatments for disorders of social behavior.     

Adult visual cortical plasticity

The visual cortex has the capacity for experience-dependent change, or cortical plasticity, that is retained throughout life. Plasticity is invoked for encoding information during perceptual learning, by internally representing the regularities of the visual environment, which is useful for facilitating intermediate-level vision-contour integration and surface segmentation. The same mechanisms have adaptive value for functional recovery after CNS damage, such as that associated with stroke or neurodegenerative disease. A common feature to plasticity in primary visual cortex (V1) is an association field that links contour elements across the visual field. The circuitry underlying the association field includes a plexus of long-range horizontal connections formed by cortical pyramidal cells. These connections undergo rapid and exuberant sprouting and pruning in response to removal of sensory input, which can account for the topographic reorganization following retinal lesions. Similar alterations in cortical circuitry may be involved in perceptual learning, and the changes observed in V1 may be representative of how learned information is encoded throughout the cerebral cortex.     

Visual and auditory conditional position discrimination in chimpanzees (Pan troglodytes)

Chimpanzee cognition has been studied predominantly through the visual modality, and much less through the auditory modality. The aim of this study was to explore possible differences in chimpanzees’ processing of visual and auditory stimuli. We developed a new conditional position discrimination (CPD) task requiring the association between a stimulus (from either the auditory or the visual modality), and a spatial position (left or right). The stimuli consisted of the face and voice of two individuals well known to the subjects (one chimpanzee and one human). Six chimpanzees participated in both the visual and the auditory conditions. We found contrasting results between the two conditions: the subjects acquired the CPD more easily in the visual than in the auditory condition. This supports previous findings on the difficulties encountered by chimpanzees in learning tasks involving auditory stimuli. Our experiments also revealed individual differences: the chimpanzee with the most extensive experience in symbolic visual matching tasks showed good performance in both conditions. In contrast, the chimpanzee expert in an auditory-visual intermodal matching task showed no sign of learning in either condition. Future work should focus on finding the most appropriate procedure for exploring chimpanzees’ auditory-visual cognitive skills.     

Concurrent Relations between Face Scanning and Language: A Cross-Syndrome Infant Study

Typically developing (TD) infants enhance their learning of spoken language by observing speakers’ mouth movements. Given the fact that word learning is seriously delayed in most children with neurodevelopmental disorders, we hypothesized that this delay partly results from differences in visual face scanning, e.g., focusing attention away from the mouth. To test this hypothesis, we used an eye tracker to measure visual attention in 95 infants and toddlers with Down syndrome (DS), fragile X syndrome (FXS), and Williams syndrome (WS), and compared their data to 25 chronological- and mental-age matched 16-month-old TD controls. We presented participants with two talking faces (one on each side of the screen) and a sound (/ga/). One face (the congruent face) mouthed the syllable that the participants could hear (i.e., /ga/), while the other face (the incongruent face) mouthed a different syllable (/ba/) from the one they could hear. As expected, we found that TD children with a relatively large vocabulary made more fixations to the mouth region of the incongruent face than elsewhere. However, toddlers with FXS or WS who had a relatively large receptive vocabulary made more fixations to the eyes (rather than the mouth) of the incongruent face. In DS, by contrast, fixations to the speaker’s overall face (rather than to her eyes or mouth) predicted vocabulary size. These findings suggest that, at some point in development, different processes or strategies relating to visual attention are involved in language acquisition in DS, FXS, and WS. This knowledge may help further explain why language is delayed in children with neurodevelopmental disorders. It also raises the possibility that syndrome-specific interventions should include an early focus on efficient face-scanning behaviour.     

EEG Findings of Reduced Neural Synchronization during Visual Integration in Schizophrenia

Schizophrenia patients exhibit well-documented visual processing deficits. One area of disruption is visual integration, the ability to form global objects from local elements. However, most studies of visual integration in schizophrenia have been conducted in the context of an active attention task, which may influence the findings. In this study we examined visual integration using electroencephalography (EEG) in a passive task to elucidate neural mechanisms associated with poor visual integration. Forty-six schizophrenia patients and 30 healthy controls had EEG recorded while passively viewing figures comprised of real, illusory, or no contours. We examined visual P100, N100, and P200 event-related potential (ERP) components, as well as neural synchronization in the gamma (30-60 Hz) band assessed by the EEG phase locking factor (PLF). The N100 was significantly larger to illusory vs. no contour, and illusory vs. real contour stimuli while the P200 was larger only to real vs. illusory stimuli; there were no significant interactions with group. Compared to controls, patients failed to show increased phase locking to illusory versus no contours between 40-60 Hz. Also, controls, but not patients, had larger PLF between 30-40 Hz when viewing real vs. illusory contours. Finally, the positive symptom factor of the BPRS was negatively correlated with PLF values between 40-60 Hz to illusory stimuli, and with PLF between 30-40 Hz to real contour stimuli. These results suggest that the pattern of results across visual processing conditions is similar in patients and controls. However, patients have deficits in neural synchronization in the gamma range during basic processing of illusory contours when attentional demand is limited.     

Individuals with autism spectrum disorder have altered visual encoding capacity

Perceptual anomalies in individuals with autism spectrum disorder (ASD) have been attributed to an imbalance in weighting incoming sensory evidence with prior knowledge when interpreting sensory information. Here, we show that sensory encoding and how it adapts to changing stimulus statistics during feedback also characteristically differs between neurotypical and ASD groups. In a visual orientation estimation task, we extracted the accuracy of sensory encoding from psychophysical data by using an information theoretic measure. Initially, sensory representations in both groups reflected the statistics of visual orientations in natural scenes, but encoding capacity was overall lower in the ASD group. Exposure to an artificial (i.e., uniform) distribution of visual orientations coupled with performance feedback altered the sensory representations of the neurotypical group toward the novel experimental statistics, while also increasing their total encoding capacity. In contrast, neither total encoding capacity nor its allocation significantly changed in the ASD group. Across both groups, the degree of adaptation was correlated with participants’ initial encoding capacity. These findings highlight substantial deficits in sensory encoding—independent from and potentially in addition to deficits in decoding—in individuals with ASD.

It is increasingly recognized that individuals with Autism Spectrum Disorder (ASD) show anomalies in perception, and these have been recently attributed to altered decoding (i.e. interpretation of sensory signals). This study reveals that independent of these changes, individuals with ASD show upstream deficits in sensory encoding (i.e., how samples are drawn from the environment).     

Collaborative Filtering for Brain-Computer Interaction Using Transfer Learning and Active Class Selection

Brain-computer interaction (BCI) and physiological computing are terms that refer to using processed neural or physiological signals to influence human interaction with computers, environment, and each other. A major challenge in developing these systems arises from the large individual differences typically seen in the neural/physiological responses. As a result, many researchers use individually-trained recognition algorithms to process this data. In order to minimize time, cost, and barriers to use, there is a need to minimize the amount of individual training data required, or equivalently, to increase the recognition accuracy without increasing the number of user-specific training samples. One promising method for achieving this is collaborative filtering, which combines training data from the individual subject with additional training data from other, similar subjects. This paper describes a successful application of a collaborative filtering approach intended for a BCI system. This approach is based on transfer learning (TL), active class selection (ACS), and a mean squared difference user-similarity heuristic. The resulting BCI system uses neural and physiological signals for automatic task difficulty recognition. TL improves the learning performance by combining a small number of user-specific training samples with a large number of auxiliary training samples from other similar subjects. ACS optimally selects the classes to generate user-specific training samples. Experimental results on 18 subjects, using both nearest neighbors and support vector machine classifiers, demonstrate that the proposed approach can significantly reduce the number of user-specific training data samples. This collaborative filtering approach will also be generalizable to handling individual differences in many other applications that involve human neural or physiological data, such as affective computing.