Grapheme-colour synaesthesia improves detection of embedded shapes,but without pre-attentive ‘pop-out’ of synaesthetic colour

For people with synaesthesia letters and numbers may evoke experiences of colour. It has been previously demonstrated that these synaesthetes may be better at detecting a triangle made of 2s among a background of 5s if they perceive 5 and 2 as having different synaesthetic colours. However, other studies using this task (or tasks based on the same principle) have failed to replicate the effect or have suggested alternative explanations of the effect. In this study, we repeat the original study on a larger group of synaesthetes (n = 36) and include, for the first time, an assessment of their self-reported colour experiences. We show that synaesthetes do have a general advantage over controls on this task. However, many synaesthetes report no colour experiences at all during the task. Synaesthetes who do report colour typically experience around one third of the graphemes in the display as coloured. This is more consistent with theories of synaesthesia in which spatial attention needs to be deployed to graphemes for conscious colour experiences to emerge than the interpretation based on ‘pop-out’.     

Psychophysical investigations into the neural basis of synaesthesia

We studied two otherwise normal, synaesthetic subjects who 'saw' a specific colour every time they saw a specific number or letter. We conducted four experiments in order to show that this was a genuine perceptual experience rather than merely a memory association. (i) The synaesthetically induced colours could lead to perceptual grouping, even though the inducing numerals or letters did not. (ii) Synaesthetically induced colours were not experienced if the graphemes were presented peripherally. (iii) Roman numerals were ineffective: the actual number grapheme was required. (iv) If two graphemes were alternated the induced colours were also seen in alternation. However, colours were no longer experienced if the graphemes were alternated at more than 4 Hz. We propose that grapheme colour synaesthesia arises from 'cross-wiring' between the 'colour centre' (area V4 or V8) and the 'number area', both of which lie in the fusiform gyrus. We also suggest a similar explanation for the representation of metaphors in the brain: hence, the higher incidence of synaesthesia among artists and poets.     

Integration of Retinal and Extraretinal Information across Eye Movements

Visual perception is burdened with a highly discontinuous input stream arising from saccadic eye movements. For successful integration into a coherent representation, the visuomotor system needs to deal with these self-induced perceptual changes and distinguish them from external motion. Forward models are one way to solve this problem where the brain uses internal monitoring signals associated with oculomotor commands to predict the visual consequences of corresponding eye movements during active exploration. Visual scenes typically contain a rich structure of spatial relational information, providing additional cues that may help disambiguate self-induced from external changes of perceptual input. We reasoned that a weighted integration of these two inherently noisy sources of information should lead to better perceptual estimates. Volunteer subjects performed a simple perceptual decision on the apparent displacement of a visual target, jumping unpredictably in sync with a saccadic eye movement. In a critical test condition, the target was presented together with a flanker object, where perceptual decisions could take into account the spatial distance between target and flanker object. Here, precision was better compared to control conditions in which target displacements could only be estimated from either extraretinal or visual relational information alone. Our findings suggest that under natural conditions, integration of visual space across eye movements is based upon close to optimal integration of both retinal and extraretinal pieces of information.     

Synaesthetic Colour in the Brain: Beyond Colour Areas. A Functional Magnetic Resonance Imaging Study of Synaesthetes and Matched Controls

Background

In synaesthesia, sensations in a particular modality cause additional experiences in a second, unstimulated modality (e.g., letters elicit colour). Understanding how synaesthesia is mediated in the brain can help to understand normal processes of perceptual awareness and multisensory integration. In several neuroimaging studies, enhanced brain activity for grapheme-colour synaesthesia has been found in ventral-occipital areas that are also involved in real colour processing. Our question was whether the neural correlates of synaesthetically induced colour and real colour experience are truly shared.

Methodology/Principal Findings

First, in a free viewing functional magnetic resonance imaging (fMRI) experiment, we located main effects of synaesthesia in left superior parietal lobule and in colour related areas. In the left superior parietal lobe, individual differences between synaesthetes (projector-associator distinction) also influenced brain activity, confirming the importance of the left superior parietal lobe for synaesthesia. Next, we applied a repetition suppression paradigm in fMRI, in which a decrease in the BOLD (blood-oxygenated-level-dependent) response is generally observed for repeated stimuli. We hypothesized that synaesthetically induced colours would lead to a reduction in BOLD response for subsequently presented real colours, if the neural correlates were overlapping. We did find BOLD suppression effects induced by synaesthesia, but not within the colour areas.

Conclusions/Significance

Because synaesthetically induced colours were not able to suppress BOLD effects for real colour, we conclude that the neural correlates of synaesthetic colour experience and real colour experience are not fully shared. We propose that synaesthetic colour experiences are mediated by higher-order visual pathways that lie beyond the scope of classical, ventral-occipital visual areas. Feedback from these areas, in which the left parietal cortex is likely to play an important role, may induce V4 activation and the percept of synaesthetic colour.     

Disruptive Colouration and Perceptual Grouping

Camouflage is the primary defence of many animals and includes multiple strategies that interfere with figure-ground segmentation and object recognition. While matching background colours and textures is widespread and conceptually straightforward, less well explored are the optical ‘tricks’, collectively called disruptive colouration, that exploit perceptual grouping mechanisms. Adjacent high contrast colours create false edges, but this is not sufficient for an object’s shape to be broken up; some colours must blend with the background. We test the novel hypothesis that this will be particularly effective when the colour patches on the animal appear to belong to, not merely different background colours, but different background objects. We used computer-based experiments where human participants had to find cryptic targets on artificial backgrounds. Creating what appeared to be bi-coloured foreground objects on bi-coloured backgrounds, we generated colour boundaries that had identical local contrast but either lay within or between (illusory) objects. As predicted, error rates for targets matching what appeared to be different background objects were higher than for targets which had otherwise identical local contrast to the background but appeared to belong to single background objects. This provides evidence for disruptive colouration interfering with higher-level feature integration in addition to previously demonstrated low-level effects involving contour detection. In addition, detection was impeded in treatments where targets were on or in close proximity to multiple background colour or tone boundaries. This is consistent with other studies which show a deleterious influence of visual ‘clutter’ or background complexity on search.     

The visual perceptual range of a lizard, <Emphasis Type="Italic">Tiliqua rugosa</Emphasis>

The fragmentation of landscapes produces habitat gaps where the distance between visual landmarks may exceed the perceptual range of a species and impose navigational constraints. We estimated the visual perceptual range of the Australian sleepy lizard, Tiliqua rugosa, by releasing individuals in the centre of a cleared arena in high temperature conditions, with a 0.5-m-high bush placed either 10, 20 or 30 m from the release site. Lizards were more likely to locate those bushes and shelter under them when they were closer, and no lizards found a bush at 30 m. In addition, lizards were less likely to move from the release point when bushes were at 30 m than when they were at the two closer distances. These data suggest that for sleepy lizards the perceptual range for a 0.5-m-high bush is about 20 m. In the uncleared chenopod shrub-land where these lizards live, suitable shelter bushes are an average of 10.5 m from any point in their home range, well within their perceptual range.     

Components of Attention in Grapheme-Color Synesthesia: A Modeling Approach

Grapheme-color synesthesia is a condition where the perception of graphemes consistently and automatically evokes an experience of non-physical color. Many have studied how synesthesia affects the processing of achromatic graphemes, but less is known about the synesthetic processing of physically colored graphemes. Here, we investigated how the visual processing of colored letters is affected by the congruence or incongruence of synesthetic grapheme-color associations. We briefly presented graphemes (10–150 ms) to 9 grapheme-color synesthetes and to 9 control observers. Their task was to report as many letters (targets) as possible, while ignoring digit (distractors). Graphemes were either congruently or incongruently colored with the synesthetes’ reported grapheme-color association. A mathematical model, based on Bundesen’s (1990) Theory of Visual Attention (TVA), was fitted to each observer’s data, allowing us to estimate discrete components of visual attention. The models suggested that the synesthetes processed congruent letters faster than incongruent ones, and that they were able to retain more congruent letters in visual short-term memory, while the control group’s model parameters were not significantly affected by congruence. The increase in processing speed, when synesthetes process congruent letters, suggests that synesthesia affects the processing of letters at a perceptual level. To account for the benefit in processing speed, we propose that synesthetic associations become integrated into the categories of graphemes, and that letter colors are considered as evidence for making certain perceptual categorizations in the visual system. We also propose that enhanced visual short-term memory capacity for congruently colored graphemes can be explained by the synesthetes’ expertise regarding their specific grapheme-color associations.     

Gaze Duration Biases for Colours in Combination with Dissonant and Consonant Sounds: A Comparative Eye-Tracking Study with Orangutans

Research on colour preferences in humans and non-human primates suggests similar patterns of biases for and avoidance of specific colours, indicating that these colours are connected to a psychological reaction. Similarly, in the acoustic domain, approach reactions to consonant sounds (considered as positive) and avoidance reactions to dissonant sounds (considered as negative) have been found in human adults and children, and it has been demonstrated that non-human primates are able to discriminate between consonant and dissonant sounds. Yet it remains unclear whether the visual and acoustic approach–avoidance patterns remain consistent when both types of stimuli are combined, how they relate to and influence each other, and whether these are similar for humans and other primates. Therefore, to investigate whether gaze duration biases for colours are similar across primates and whether reactions to consonant and dissonant sounds cumulate with reactions to specific colours, we conducted an eye-tracking study in which we compared humans with one species of great apes, the orangutans. We presented four different colours either in isolation or in combination with consonant and dissonant sounds. We hypothesised that the viewing time for specific colours should be influenced by dissonant sounds and that previously existing avoidance behaviours with regard to colours should be intensified, reflecting their association with negative acoustic information. The results showed that the humans had constant gaze durations which were independent of the auditory stimulus, with a clear avoidance of yellow. In contrast, the orangutans did not show any clear gaze duration bias or avoidance of colours, and they were also not influenced by the auditory stimuli. In conclusion, our findings only partially support the previously identified pattern of biases for and avoidance of specific colours in humans and do not confirm such a pattern for orangutans.     

Colour spaces in ecology and evolutionary biology

The recognition that animals sense the world in a different way than we do has unlocked important lines of research in ecology and evolutionary biology. In practice, the subjective study of natural stimuli has been permitted by perceptual spaces, which are graphical models of how stimuli are perceived by a given animal. Because colour vision is arguably the best‐known sensory modality in most animals, a diversity of colour spaces are now available to visual ecologists, ranging from generalist and basic models allowing rough but robust predictions on colour perception, to species‐specific, more complex models giving accurate but context‐dependent predictions. Selecting among these models is most often influenced by historical contingencies that have associated models to specific questions and organisms; however, these associations are not always optimal. The aim of this review is to provide visual ecologists with a critical perspective on how models of colour space are built, how well they perform and where their main limitations are with regard to their most frequent uses in ecology and evolutionary biology. We propose a classification of models based on their complexity, defined as whether and how they model the mechanisms of chromatic adaptation and receptor opponency, the nonlinear association between the stimulus and its perception, and whether or not models have been fitted to experimental data. Then, we review the effect of modelling these mechanisms on predictions of colour detection and discrimination, colour conspicuousness, colour diversity and diversification, and for comparing the perception of colour traits between distinct perceivers. While a few rules emerge (e.g. opponent log–linear models should be preferred when analysing very distinct colours), in general model parameters still have poorly known effects. Colour spaces have nonetheless permitted significant advances in ecology and evolutionary biology, and more progress is expected if ecologists compare results between models and perform behavioural experiments more routinely. Such an approach would further contribute to a better understanding of colour vision and its links to the behavioural ecology of animals. While visual ecology is essentially a transfer of knowledge from visual sciences to evolutionary ecology, we hope that the discipline will benefit both fields more evenly in the future.     

Visual attention: insights from brain imaging

We are not passive recipients of the information that impinges on our retinae, but active participants in our own perceptual processes. Visual experience depends critically on attention. We select particular aspects of a visual scene for detailed analysis and control of subsequent behaviour, but ignore other aspects so completely that moments after they disappear from view we cannot report anything about them. Here we show that functional neuroimaging is revealing much more than where attention happens in the brain; it is beginning to answer some of the oldest and deepest questions about what visual attention is and how it works.     

The modelling of avian visual perception predicts behavioural rejection responses to foreign egg colours

How do birds tell the colours of their own and foreign eggs apart? We demonstrate that perceptual modelling of avian visual discrimination can predict behavioural rejection responses to foreign eggs in the nest of wild birds. We use a photoreceptor noise-limited colour opponent model of visual perception to evaluate its accuracy as a predictor of behavioural rates of experimental egg discrimination in the song thrush Turdus philomelos. The visual modelling of experimental and natural eggshell colours suggests that photon capture from the ultraviolet and short wavelength-sensitive cones elicits egg rejection decisions in song thrushes, while inter-clutch variation of egg coloration provides sufficient contrasts for detecting conspecific parasitism in this species. Biologically realistic sensory models provide an important tool for relating variability of behavioural responses to perceived phenotypic variation.     

Evolution and Optimality of Similar Neural Mechanisms for Perception and Action during Search

A prevailing theory proposes that the brain''s two visual pathways, the ventral and dorsal, lead to differing visual processing and world representations for conscious perception than those for action. Others have claimed that perception and action share much of their visual processing. But which of these two neural architectures is favored by evolution? Successful visual search is life-critical and here we investigate the evolution and optimality of neural mechanisms mediating perception and eye movement actions for visual search in natural images. We implement an approximation to the ideal Bayesian searcher with two separate processing streams, one controlling the eye movements and the other stream determining the perceptual search decisions. We virtually evolved the neural mechanisms of the searchers'' two separate pathways built from linear combinations of primary visual cortex receptive fields (V1) by making the simulated individuals'' probability of survival depend on the perceptual accuracy finding targets in cluttered backgrounds. We find that for a variety of targets, backgrounds, and dependence of target detectability on retinal eccentricity, the mechanisms of the searchers'' two processing streams converge to similar representations showing that mismatches in the mechanisms for perception and eye movements lead to suboptimal search. Three exceptions which resulted in partial or no convergence were a case of an organism for which the targets are equally detectable across the retina, an organism with sufficient time to foveate all possible target locations, and a strict two-pathway model with no interconnections and differential pre-filtering based on parvocellular and magnocellular lateral geniculate cell properties. Thus, similar neural mechanisms for perception and eye movement actions during search are optimal and should be expected from the effects of natural selection on an organism with limited time to search for food that is not equi-detectable across its retina and interconnected perception and action neural pathways.     

Spatial heterogeneity in the perception of face and form attributes

The identity of an object is a fixed property, independent of where it appears, and an effective visual system should capture this invariance [1-3]. However, we now report that the perceived gender of a face is strongly biased toward male or female at different locations in the visual field. The spatial pattern of these biases was distinctive and stable for each individual. Identical neutral faces looked different when they were presented simultaneously at locations maximally biased to opposite genders. A similar effect was observed for perceived age of faces. We measured the magnitude of this perceptual heterogeneity for four other visual judgments: perceived aspect ratio, orientation discrimination, spatial-frequency discrimination, and color discrimination. The effect was sizeable for the aspect ratio task but substantially smaller for the other three tasks. We also evaluated perceptual heterogeneity for facial gender and orientation tasks at different spatial scales. Strong heterogeneity was observed even for the orientation task when tested at small scales. We suggest that perceptual heterogeneity is a general property of visual perception and results from undersampling of the visual signal at spatial scales that are small relative to the size of the receptive fields associated with each visual attribute.     

Adaptation to visual feedback delay influences visuomotor learning

Computational theory of motor control suggests that the brain continuously monitors motor commands, to predict their sensory consequences before actual sensory feedback becomes available. Such prediction error is a driving force of motor learning, and therefore appropriate associations between motor commands and delayed sensory feedback signals are crucial. Indeed, artificially introduced delays in visual feedback have been reported to degrade motor learning. However, considering our perceptual ability to causally bind our own actions with sensory feedback, demonstrated by the decrease in the perceived time delay following repeated exposure to an artificial delay, we hypothesized that such perceptual binding might alleviate deficits of motor learning associated with delayed visual feedback. Here, we evaluated this hypothesis by investigating the ability of human participants to adapt their reaching movements in response to a novel visuomotor environment with 3 visual feedback conditions--no-delay, sudden-delay, and adapted-delay. To introduce novelty into the trials, the cursor position, which originally indicated the hand position in baseline trials, was rotated around the starting position. In contrast to the no-delay condition, a 200-ms delay was artificially introduced between the cursor and hand positions during the presence of visual rotation (sudden-delay condition), or before the application of visual rotation (adapted-delay condition). We compared the learning rate (representing how the movement error modifies the movement direction in the subsequent trial) between the 3 conditions. In comparison with the no-delay condition, the learning rate was significantly degraded for the sudden-delay condition. However, this degradation was significantly alleviated by prior exposure to the delay (adapted-delay condition). Our data indicate the importance of appropriate temporal associations between motor commands and sensory feedback in visuomotor learning. Moreover, they suggest that the brain is able to account for such temporal associations in a flexible manner.     

Very Slow Search and Reach: Failure to Maximize Expected Gain in an Eye-Hand Coordination Task

We examined an eye-hand coordination task where optimal visual search and hand movement strategies were inter-related. Observers were asked to find and touch a target among five distractors on a touch screen. Their reward for touching the target was reduced by an amount proportional to how long they took to locate and reach to it. Coordinating the eye and the hand appropriately would markedly reduce the search-reach time. Using statistical decision theory we derived the sequence of interrelated eye and hand movements that would maximize expected gain and we predicted how hand movements should change as the eye gathered further information about target location. We recorded human observers'' eye movements and hand movements and compared them with the optimal strategy that would have maximized expected gain. We found that most observers failed to adopt the optimal search-reach strategy. We analyze and describe the strategies they did adopt.     

Neural mechanisms of human perceptual learning: electrophysiological evidence for a two-stage process

Background

Humans and other animals change the way they perceive the world due to experience. This process has been labeled as perceptual learning, and implies that adult nervous systems can adaptively modify the way in which they process sensory stimulation. However, the mechanisms by which the brain modifies this capacity have not been sufficiently analyzed.

Methodology/Principal Findings

We studied the neural mechanisms of human perceptual learning by combining electroencephalographic (EEG) recordings of brain activity and the assessment of psychophysical performance during training in a visual search task. All participants improved their perceptual performance as reflected by an increase in sensitivity (d'') and a decrease in reaction time. The EEG signal was acquired throughout the entire experiment revealing amplitude increments, specific and unspecific to the trained stimulus, in event-related potential (ERP) components N2pc and P3 respectively. P3 unspecific modification can be related to context or task-based learning, while N2pc may be reflecting a more specific attentional-related boosting of target detection. Moreover, bell and U-shaped profiles of oscillatory brain activity in gamma (30–60 Hz) and alpha (8–14 Hz) frequency bands may suggest the existence of two phases for learning acquisition, which can be understood as distinctive optimization mechanisms in stimulus processing.

Conclusions/Significance

We conclude that there are reorganizations in several neural processes that contribute differently to perceptual learning in a visual search task. We propose an integrative model of neural activity reorganization, whereby perceptual learning takes place as a two-stage phenomenon including perceptual, attentional and contextual processes.     

Reward-Priming of Location in Visual Search

Existing visual search research has demonstrated that the receipt of reward will be beneficial for subsequent perceptual and attentional processing of features that have characterized targets, but detrimental for processing of features that have characterized irrelevant distractors. Here we report a similar effect of reward on location. Observers completed a visual search task in which they selected a target, ignored a salient distractor, and received random-magnitude reward for correct performance. Results show that when target selection garnered rewarding outcome attention is subsequently a.) primed to return to the target location, and b.) biased away from the location that was occupied by the salient, task-irrelevant distractor. These results suggest that in addition to priming features, reward acts to guide visual search by priming contextual locations of visual stimuli.     

Training Top-Down Attention Improves Performance on a Triple-Conjunction Search Task

Training has been shown to improve perceptual performance on limited sets of stimuli. However, whether training can generally improve top-down biasing of visual search in a target-nonspecific manner remains unknown. We trained subjects over ten days on a visual search task, challenging them with a novel target (top-down goal) on every trial, while bottom-up uncertainty (distribution of distractors) remained constant. We analyzed the changes in saccade statistics and visual behavior over the course of training by recording eye movements as subjects performed the task. Subjects became experts at this task, with twofold increased performance, decreased fixation duration, and stronger tendency to guide gaze toward items with color and spatial frequency (but not necessarily orientation) that resembled the target, suggesting improved general top-down biasing of search.     

Corticosterone regulates multiple colour traits in Lacerta [Zootoca] vivipara males

Ornamental colours usually evolve as honest signals of quality, which is supported by the fact that they frequently depend on individual condition. It has generally been suggested that some, but not all types of ornamental colours are condition dependent, indicating that different evolutionary mechanisms underlie the evolution of multiple types of ornamental colours even when these are exhibited by the same species. Stress hormones, which negatively affect condition, have been shown to affect colour traits based on different pigments and structures, suggesting that they mediate condition dependence of multiple ornament types both among and within individuals. However, studies investigating effects of stress hormones on different ornament types within individuals are lacking, and thus, evidence for this hypothesis is scant. Here, we investigated whether corticosterone mediates condition dependence of multiple ornaments by manipulating corticosterone levels and body condition (via food availability) using a two‐factorial design and by assessing their effect on multiple colour traits in male common lizards. Corticosterone negatively affected ventral melanin‐ and carotenoid‐based coloration, whereas food availability did not affect coloration, despite its significant effect on body condition. The corticosterone effect on melanin‐ and carotenoid‐based coloration demonstrates the condition dependence of both ornaments. Moreover, corticosterone affected ventral coloration and had no effect on the nonsexually selected dorsal coloration, showing specific effects of corticosterone on ornamental ventral colours. This suggests that corticosterone simultaneously mediates condition dependence of multiple colour traits and that it therefore accounts for covariation among them, which may influence their evolution via correlational selection.     

Homing in German Cockroaches, Blattella germanica (L.) (Insecta: Dictyoptera): Multi-Channelled Orientation Cues

Cockroaches use navigational cues to elaborate their return path to the shelter. Our experiments investigated how individuals weighted information to choose where to search for the shelter in situations where path integration, visual and olfactory cues were conflicting. We showed that homing relied on a complex set of environmental stimuli, each playing a particular part. Path integration cues give cockroaches an estimation of the position of their goal, visual landmarks guide them to that position from a distance, while olfactory cues indicate the end of the path. Cockroaches gave the greatest importance to the first cues they encountered along their return path. Nevertheless, visual cues placed beyond aggregation pheromone deposits reduced their arrest efficiency and induced search in the area near the visual cues.