Mental imagery and visual working memory

Visual working memory provides an essential link between past and future events. Despite recent efforts, capacity limits, their genesis and the underlying neural structures of visual working memory remain unclear. Here we show that performance in visual working memory--but not iconic visual memory--can be predicted by the strength of mental imagery as assessed with binocular rivalry in a given individual. In addition, for individuals with strong imagery, modulating the background luminance diminished performance on visual working memory and imagery tasks, but not working memory for number strings. This suggests that luminance signals were disrupting sensory-based imagery mechanisms and not a general working memory system. Individuals with poor imagery still performed above chance in the visual working memory task, but their performance was not affected by the background luminance, suggesting a dichotomy in strategies for visual working memory: individuals with strong mental imagery rely on sensory-based imagery to support mnemonic performance, while those with poor imagery rely on different strategies. These findings could help reconcile current controversy regarding the mechanism and location of visual mnemonic storage.     

Contrast and Strength of Visual Memory and Imagery Differentially Affect Visual Perception

Visual short-term memory (VSTM) and visual imagery have been shown to modulate visual perception. However, how the subjective experience of VSTM/imagery and its contrast modulate this process has not been investigated. We addressed this issue by asking participants to detect brief masked targets while they were engaged either in VSTM or visual imagery. Subjective experience of memory/imagery (strength scale), and the visual contrast of the memory/mental image (contrast scale) were assessed on a trial-by-trial basis. For both VSTM and imagery, contrast of the memory/mental image was positively associated with reporting target presence. Consequently, at the sensory level, both VSTM and imagery facilitated visual perception. However, subjective strength of VSTM was positively associated with visual detection whereas the opposite pattern was found for imagery. Thus the relationship between subjective strength of memory/imagery and visual detection are qualitatively different for VSTM and visual imagery, although their impact at the sensory level appears similar. Our results furthermore demonstrate that imagery and VSTM are partly dissociable processes.     

综述: 初级视皮层的亮度信息加工

亮度(luminance)是最基本的视觉信息.与其他视觉特征相比,由于视神经元对亮度刺激的反应较弱,并且许多神经元对均匀亮度无反应,对亮度信息编码的神经机制知之甚少.初级视皮层部分神经元对亮度的反应要慢于对比度反应,被认为是由边界对比度诱导的亮度知觉(brightness)的神经基础.我们的研究表明,初级视皮层许多神经元的亮度反应要快于对比度反应,并且这些神经元偏好低的空间频率、高的时间频率和高的运动速度,提示皮层下具有低空间频率和高运动速度通路的信息输入对产生初级视皮层神经元的亮度反应有贡献.已经知道初级视皮层神经元对空间频率反应的时间过程是从低空间频率到高空间频率,我们发现的早期亮度反应是对极低空间频率的反应,与这一时间过程是一致的,是这一从粗到细的视觉信息加工过程的第一步,揭示了处理最早的粗的视觉信息的神经基础.另外,初级视皮层含有偏好亮度下降和高运动速度的神经元,这群神经元的活动有助于在光照差的环境中检测高速运动的低亮度物体.     

Iterative Fragmentation of Cognitive Maps in a Visual Imagery Task

It remains unclear whether spontaneous eye movements during visual imagery reflect the mental generation of a visual image (i.e. the arrangement of the component parts of a mental representation). To address this specificity, we recorded eye movements in an imagery task and in a phonological fluency (non-imagery) task, both consisting in naming French towns from long-term memory. Only in the condition of visual imagery the spontaneous eye positions reflected the geographic position of the towns evoked by the subjects. This demonstrates that eye positions closely reflect the mapping of mental images. Advanced analysis of gaze positions using the bi-dimensional regression model confirmed the spatial correlation of gaze and towns’ locations in every single individual in the visual imagery task and in none of the individuals when no imagery accompanied memory retrieval. In addition, the evolution of the bi-dimensional regression’s coefficient of determination revealed, in each individual, a process of generating several iterative series of a limited number of towns mapped with the same spatial distortion, despite different individual order of towns’ evocation and different individual mappings. Such consistency across subjects revealed by gaze (the mind’s eye) gives empirical support to theories postulating that visual imagery, like visual sampling, is an iterative fragmented processing.     

Motor Imagery Learning Modulates Functional Connectivity of Multiple Brain Systems in Resting State

Background

Learning motor skills involves subsequent modulation of resting-state functional connectivity in the sensory-motor system. This idea was mostly derived from the investigations on motor execution learning which mainly recruits the processing of sensory-motor information. Behavioral evidences demonstrated that motor skills in our daily lives could be learned through imagery procedures. However, it remains unclear whether the modulation of resting-state functional connectivity also exists in the sensory-motor system after motor imagery learning.

Methodology/Principal Findings

We performed a fMRI investigation on motor imagery learning from resting state. Based on previous studies, we identified eight sensory and cognitive resting-state networks (RSNs) corresponding to the brain systems and further explored the functional connectivity of these RSNs through the assessments, connectivity and network strengths before and after the two-week consecutive learning. Two intriguing results were revealed: (1) The sensory RSNs, specifically sensory-motor and lateral visual networks exhibited greater connectivity strengths in precuneus and fusiform gyrus after learning; (2) Decreased network strength induced by learning was proved in the default mode network, a cognitive RSN.

Conclusions/Significance

These results indicated that resting-state functional connectivity could be modulated by motor imagery learning in multiple brain systems, and such modulation displayed in the sensory-motor, visual and default brain systems may be associated with the establishment of motor schema and the regulation of introspective thought. These findings further revealed the neural substrates underlying motor skill learning and potentially provided new insights into the therapeutic benefits of motor imagery learning.     

Contribution of human melanopsin retinal ganglion cells to steady-state pupil responses

The recent discovery of melanopsin-containing retinal ganglion cells (mRGCs) has led to a fundamental reassessment of non-image forming processing, such as circadian photoentrainment and the pupillary light reflex. In the conventional view of retinal physiology, rods and cones were assumed to be the only photoreceptors in the eye and were, therefore, considered responsible for non-image processing. However, signals from mRGCs contribute to this non-image forming processing along with cone-mediated luminance signals; although both signals contribute, it is unclear how these signals are summed. We designed and built a novel multi-primary stimulation system to stimulate mRGCs independently of other photoreceptors using a silent-substitution technique within a bright steady background. The system allows direct measurements of pupillary functions for mRGCs and cones. We observed a significant change in steady-state pupil diameter when we varied the excitation of mRGC alone, with no change in luminance and colour. Furthermore, the change in pupil diameter induced by mRGCs was larger than that induced by a variation in luminance alone: that is, for a bright steady background, the mRGC signals contribute to the pupillary pathway by a factor of three times more than the L- and M-cone signals.     

Imagine that: elevated sensory strength of mental imagery in individuals with Parkinson's disease and visual hallucinations

Visual hallucinations occur when our conscious experience does not accurately reflect external reality. However, these dissociations also regularly occur when we imagine the world around us in the absence of visual stimulation. We used two novel behavioural paradigms to objectively measure visual hallucinations and voluntary mental imagery in 19 individuals with Parkinson''s disease (ten with visual hallucinations; nine without) and ten healthy, age-matched controls. We then used this behavioural overlap to interrogate the connectivity both within and between the major attentional control networks using resting-state functional magnetic resonance imaging. Patients with visual hallucinations had elevated mental imagery strength compared with patients without hallucinations and controls. Specifically, the sensory strength of imagery predicted the frequency of visual hallucinations. Together, hallucinations and mental imagery predicted multiple abnormalities in functional connectivity both within and between the attentional control networks, as measured with resting-state functional magnetic resonance imaging. However, the two phenomena were also dissociable at the neural level, with both mental imagery and visual misperceptions associated with specific abnormalities in attentional network connectivity. Our results provide the first evidence of both the shared and unique neural correlates of these two similar, yet distinct phenomena.     

Neural adjustments to chromatic blur

The perception of blur in images can be strongly affected by prior adaptation to blurry images or by spatial induction from blurred surrounds. These contextual effects may play a role in calibrating visual responses for the spatial structure of luminance variations in images. We asked whether similar adjustments might also calibrate the visual system for spatial variations in color. Observers adjusted the amplitude spectra of luminance or chromatic images until they appeared correctly focused, and repeated these measurements either before or after adaptation to blurred or sharpened images or in the presence of blurred or sharpened surrounds. Prior adaptation induced large and distinct changes in perceived focus for both luminance and chromatic patterns, suggesting that luminance and chromatic mechanisms are both able to adjust to changes in the level of blur. However, judgments of focus were more variable for color, and unlike luminance there was little effect of surrounding spatial context on perceived blur. In additional measurements we explored the effects of adaptation on threshold contrast sensitivity for luminance and color. Adaptation to filtered noise with a 1/f spectrum characteristic of natural images strongly and selectively elevated thresholds at low spatial frequencies for both luminance and color, thus transforming the chromatic contrast sensitivity function from lowpass to nearly bandpass. These threshold changes were found to reflect interactions between different spatial scales that bias sensitivity against the lowest spatial grain in the image, and may reflect adaptation to different stimulus attributes than the attributes underlying judgments of image focus. Our results suggest that spatial sensitivity for variations in color can be strongly shaped by adaptation to the spatial structure of the stimulus, but point to dissociations in these visual adjustments both between luminance and color and different measures of spatial sensitivity.     

Tracking the mind's image in the brain I: time-resolved fMRI during visuospatial mental imagery

Mental imagery, the generation and manipulation of mental representations in the absence of sensory stimulation, is a core element of numerous cognitive processes. We investigate the cortical mechanisms underlying imagery and spatial analysis in the visual domain using event-related functional magnetic resonance imaging during the mental clock task. The time-resolved analysis of cortical activation from auditory perception to motor response reveals a sequential activation of the left and right posterior parietal cortex, suggesting that these regions perform distinct functions in this imagery task. This is confirmed by a trial-by-trial analysis of correlations between reaction time and onset, width, and amplitude of the hemodynamic response. These findings pose neurophysiological constraints on cognitive models of mental imagery.     

The functional impact of mental imagery on conscious perception

Mental imagery has been proposed to contribute to a variety of high-level cognitive functions, including memory encoding and retrieval, navigation, spatial planning, and even social communication and language comprehension. However, it is debated whether mental imagery relies on the same sensory representations as perception, and if so, what functional consequences such an overlap might have on perception itself. We report novel evidence that single instances of imagery can have a pronounced facilitatory influence on subsequent conscious perception. Either seeing or imagining a specific pattern could strongly bias which of two competing stimuli reach awareness during binocular rivalry. Effects of imagery and perception were location and orientation specific, accumulated in strength over time, and survived an intervening visual task lasting several seconds prior to presentation of the rivalry display. Interestingly, effects of imagery differed from those of feature-based attention. The results demonstrate that imagery, in the absence of any incoming visual signals, leads to the formation of a short-term sensory trace that can bias future perception, suggesting a means by which high-level processes that support imagination and memory retrieval may shape low-level sensory representations.     

Simultaneous processing of spatial and chromatic components of patterned stimuli by the human visual system

We report measurements on discrimination of orientation and magnification made for elements differentiated in colour and/or luminance from their background. By performing measurements at a series of background luminances and for fixed luminance of the elements, we show that with colour contrast, discrimination for both spatial parameters is unimpaired when the background is at isoluminance with the elements. Under simple luminance contrast, however, these discriminations become poorer when the background luminance is within some +/- 5% of that of the elements, and are completely absent when the two values are the same. A deuteranomalous subject is unable to make the spatial discrimination around the isoluminance point for colour contrasts which are too small for him to distinguish, but for which subjects with normal colour vision maintain spatial discriminations at isoluminance. This observation establishes that the physiological mechanisms of normal colour vision, rather than stimulus artefacts, mediate the observed spatial discriminations. We conclude that the visual processing of colour and spatial parameters such as orientation and magnification are intrinsically related to each other.     

A Computational Model of Afterimage Rotation in the Peripheral Drift Illusion Based on Retinal ON/OFF Responses

Human observers perceive illusory rotations after the disappearance of circularly repeating patches containing dark-to-light luminance. This afterimage rotation is a very powerful phenomenon, but little is known about the mechanisms underlying it. Here, we use a computational model to show that the afterimage rotation can be explained by a combination of fast light adaptation and the physiological architecture of the early visual system, consisting of ON- and OFF-type visual pathways. In this retinal ON/OFF model, the afterimage rotation appeared as a rotation of focus lines of retinal ON/OFF responses. Focus lines rotated clockwise on a light background, but counterclockwise on a dark background. These findings were consistent with the results of psychophysical experiments, which were also performed by us. Additionally, the velocity of the afterimage rotation was comparable with that observed in our psychophysical experiments. These results suggest that the early visual system (including the retina) is responsible for the generation of the afterimage rotation, and that this illusory rotation may be systematically misinterpreted by our high-level visual system.     

Creative cognition: the diverse operations and the prospect of applying a cognitive neuroscience perspective

Creativity is defined quite simply as "the ability to create" in most lexicons, but, in reality, this is a complex and heterogeneous construct about which there is much to be discovered. The cognitive approach to investigating creativity recognizes and seeks to understand this complexity by investigating the component processes involved in creative thinking. The cognitive neuroscience approach, which has only limitedly been applied in the study of creativity, should ideally build on these ideas in uncovering the neural substrates of these processes. Following an introduction into the early experimental ideas and the cognitive approach to creativity, we discuss the theoretical background and behavioral methods for testing various processes of creative cognition, including conceptual expansion, the constraining influence of examples, creative imagery and insight. The complex relations between the underlying component processes of originality and relevance across these tasks are presented thereafter. We then outline how some of these conceptual distinctions can be evaluated by neuroscientific evidence and elaborate on the neuropsychological approach in the study of creativity. Given the current state of affairs, our recommendation is that despite methodological difficulties that are associated with investigating creativity, adopting the cognitive neuroscience perspective is a highly promising framework for validating and expanding on the critical issues that have been raised in this paper.     

Attention driven phantom vision: measuring the sensory strength of attentional templates and their relation to visual mental imagery and aphantasia

When we search for an object in an array or anticipate attending to a future object, we create an ‘attentional template'' of the object. The definitions of attentional templates and visual imagery share many similarities as well as many of the same neural characteristics. However, the phenomenology of these attentional templates and their neural similarities to visual imagery and perception are rarely, if ever discussed. Here, we investigate the relationship between these two forms of non-retinal phantom vision through the use of the binocular rivalry technique, which allows us to measure the sensory strength of attentional templates in the absence of concurrent perceptual stimuli. We find that attentional templates correlate with both feature-based attention and visual imagery. Attentional templates, like imagery, were significantly disrupted by the presence of irrelevant visual stimuli, while feature-based attention was not. We also found that a special population who lack the ability to visualize (aphantasia), showed evidence of feature-based attention when measured using the binocular rivalry paradigm, but not attentional templates. Taken together, these data suggest functional similarities between attentional templates and visual imagery, advancing the theory of visual imagery as a general simulation tool used across cognition.This article is part of the theme issue ‘Offline perception: voluntary and spontaneous perceptual experiences without matching external stimulation’.     

Rockpool Gobies Change Colour for Camouflage

Camouflage is found in a wide range of species living in numerous habitat types, offering protection from visually guided predators. This includes many species from the intertidal zone, which must cope with background types diverse in appearance and with multiple predator groups foraging at high and low tide. Many animals are capable of either relatively slow (hours, days, weeks) or rapid (seconds and minutes) colour change in order to better resemble the background against which they are found, but most work has been restricted to a few species or taxa. It is often suggested that many small intertidal fish are capable of colour change for camouflage, yet little experimental work has addressed this. Here, we test rock gobies (Gobius paganellus) for colour change abilities, and whether they can tune their appearance to match the background. In two experiments, we place gobies on backgrounds of different brightness (black or white), and of different colours (red and blue) and use digital image analysis and modelling of predator (avian) vision to quantify colour and luminance (perceived lightness) changes and camouflage. We find that gobies are capable of rapid colour change (occurring within one minute), and that they can change their luminance on lighter or darker backgrounds. When presented on backgrounds of different colours, gobies also change their colour (hue and saturation) while keeping luminance the same. These changes lead to predicted improvements in camouflage match to the background. Our study shows that small rockpool fish are capable of rapid visual change for concealment, and that this may be an important mechanism in many species to avoid predation, especially in complex heterogeneous environments.     

Detecting Unidentified Changes

Does becoming aware of a change to a purely visual stimulus necessarily cause the observer to be able to identify or localise the change or can change detection occur in the absence of identification or localisation? Several theories of visual awareness stress that we are aware of more than just the few objects to which we attend. In particular, it is clear that to some extent we are also aware of the global properties of the scene, such as the mean luminance or the distribution of spatial frequencies. It follows that we may be able to detect a change to a visual scene by detecting a change to one or more of these global properties. However, detecting a change to global property may not supply us with enough information to accurately identify or localise which object in the scene has been changed. Thus, it may be possible to reliably detect the occurrence of changes without being able to identify or localise what has changed. Previous attempts to show that this can occur with natural images have produced mixed results. Here we use a novel analysis technique to provide additional evidence that changes can be detected in natural images without also being identified or localised. It is likely that this occurs by the observers monitoring the global properties of the scene.     

Refractive Errors Affect the Vividness of Visual Mental Images

The hypothesis that visual perception and mental imagery are equivalent has never been explored in individuals with vision defects not preventing the visual perception of the world, such as refractive errors. Refractive error (i.e., myopia, hyperopia or astigmatism) is a condition where the refracting system of the eye fails to focus objects sharply on the retina. As a consequence refractive errors cause blurred vision.We subdivided 84 individuals according to their spherical equivalent refraction into Emmetropes (control individuals without refractive errors) and Ametropes (individuals with refractive errors). Participants performed a vividness task and completed a questionnaire that explored their cognitive style of thinking before their vision was checked by an ophthalmologist. Although results showed that Ametropes had less vivid mental images than Emmetropes this did not affect the development of their cognitive style of thinking; in fact, Ametropes were able to use both verbal and visual strategies to acquire and retrieve information. Present data are consistent with the hypothesis of equivalence between imagery and perception.     

Distinct Acute Zones for Visual Stimuli in Different Visual Tasks in Drosophila

The fruit fly Drosophila melanogaster has a sophisticated visual system and exhibits complex visual behaviors. Visual responses, vision processing and higher cognitive processes in Drosophila have been studied extensively. However, little is known about whether the retinal location of visual stimuli can affect fruit fly performance in various visual tasks. We tested the response of wild-type Berlin flies to visual stimuli at several vertical locations. Three paradigms were used in our study: visual operant conditioning, visual object fixation and optomotor response. We observed an acute zone for visual feature memorization in the upper visual field when visual patterns were presented with a black background. However, when a white background was used, the acute zone was in the lower visual field. Similar to visual feature memorization, the best locations for visual object fixation and optomotor response to a single moving stripe were in the lower visual field with a white background and the upper visual field with a black background. The preferred location for the optomotor response to moving gratings was around the equator of the visual field. Our results suggest that different visual processing pathways are involved in different visual tasks and that there is a certain degree of overlap between the pathways for visual feature memorization, visual object fixation and optomotor response.     

Dynamic encoding of natural luminance sequences by LGN bursts

In the lateral geniculate nucleus (LGN) of the thalamus, visual stimulation produces two distinct types of responses known as tonic and burst. Due to the dynamics of the T-type Ca ²⁺ channels involved in burst generation, the type of response evoked by a particular stimulus depends on the resting membrane potential, which is controlled by a network of modulatory connections from other brain areas. In this study, we use simulated responses to natural scene movies to describe how modulatory and stimulus-driven changes in LGN membrane potential interact to determine the luminance sequences that trigger burst responses. We find that at low resting potentials, when the T channels are de-inactivated and bursts are relatively frequent, an excitatory stimulus transient alone is sufficient to evoke a burst. However, to evoke a burst at high resting potentials, when the T channels are inactivated and bursts are relatively rare, prolonged inhibitory stimulation followed by an excitatory transient is required. We also observe evidence of these effects in vivo, where analysis of experimental recordings demonstrates that the luminance sequences that trigger bursts can vary dramatically with the overall burst percentage of the response. To characterize the functional consequences of the effects of resting potential on burst generation, we simulate LGN responses to different luminance sequences at a range of resting potentials with and without a mechanism for generating bursts. Using analysis based on signal detection theory, we show that bursts enhance detection of specific luminance sequences, ranging from the onset of excitatory sequences at low resting potentials to the offset of inhibitory sequences at high resting potentials. These results suggest a dynamic role for burst responses during visual processing that may change according to behavioral state.     

High visual working memory capacity in trait social anxiety

Working memory capacity is one of the most important cognitive functions influencing individual traits, such as attentional control, fluid intelligence, and also psychopathological traits. Previous research suggests that anxiety is associated with impaired cognitive function, and studies have shown low verbal working memory capacity in individuals with high trait anxiety. However, the relationship between trait anxiety and visual working memory capacity is still unclear. Considering that people allocate visual attention more widely to detect danger under threat, visual working memory capacity might be higher in anxious people. In the present study, we show that visual working memory capacity increases as trait social anxiety increases by using a change detection task. When the demand to inhibit distractors increased, however, high visual working memory capacity diminished in individuals with social anxiety, and instead, impaired filtering of distractors was predicted by trait social anxiety. State anxiety was not correlated with visual working memory capacity. These results indicate that socially anxious people could potentially hold a large amount of information in working memory. However, because of an impaired cognitive function, they could not inhibit goal-irrelevant distractors and their performance decreased under highly demanding conditions.