Assessment of the Hemispheric Lateralization of Grapheme-Color Synesthesia with Stroop-Type Tests

Grapheme-color synesthesia, the idiosyncratic, arbitrary association of colors to letters or numbers, develops in childhood once reading is mastered. Because language processing is strongly left-lateralized in most individuals, we hypothesized that grapheme-color synesthesia could be left-lateralized as well. We used synesthetic versions of the Stroop test with colored letters and numbers presented either in the right or the left visual field of thirty-four synesthetes. Interference by synesthetic colors was stronger for stimuli in the right hemifield (first experiment, color naming task). Synesthetes were also faster in the right hemifield when naming the synesthetic color of graphemes (second experiment). Overall, the lateralization effect was 7 ms (the 95% confidence interval was [1.5 12] ms), a delay compatible with an additional callosal transfer for stimuli presented in the left hemifield. Though weak, this effect suggests that the association of synesthetic colors to graphemes may be preferentially processed in the left hemisphere. We speculate that this left-lateralization could be a landmark of synesthetic grapheme-color associations, if not found for color associations learnt by non-synesthete adults.     

Components of Attention in Grapheme-Color Synesthesia: A Modeling Approach

Grapheme-color synesthesia is a condition where the perception of graphemes consistently and automatically evokes an experience of non-physical color. Many have studied how synesthesia affects the processing of achromatic graphemes, but less is known about the synesthetic processing of physically colored graphemes. Here, we investigated how the visual processing of colored letters is affected by the congruence or incongruence of synesthetic grapheme-color associations. We briefly presented graphemes (10–150 ms) to 9 grapheme-color synesthetes and to 9 control observers. Their task was to report as many letters (targets) as possible, while ignoring digit (distractors). Graphemes were either congruently or incongruently colored with the synesthetes’ reported grapheme-color association. A mathematical model, based on Bundesen’s (1990) Theory of Visual Attention (TVA), was fitted to each observer’s data, allowing us to estimate discrete components of visual attention. The models suggested that the synesthetes processed congruent letters faster than incongruent ones, and that they were able to retain more congruent letters in visual short-term memory, while the control group’s model parameters were not significantly affected by congruence. The increase in processing speed, when synesthetes process congruent letters, suggests that synesthesia affects the processing of letters at a perceptual level. To account for the benefit in processing speed, we propose that synesthetic associations become integrated into the categories of graphemes, and that letter colors are considered as evidence for making certain perceptual categorizations in the visual system. We also propose that enhanced visual short-term memory capacity for congruently colored graphemes can be explained by the synesthetes’ expertise regarding their specific grapheme-color associations.     

Prevalence of Learned Grapheme-Color Pairings in a Large Online Sample of Synesthetes

In this paper we estimate the minimum prevalence of grapheme-color synesthetes with letter-color matches learned from an external stimulus, by analyzing a large sample of English-speaking grapheme-color synesthetes. We find that at least 6% (400/6588 participants) of the total sample learned many of their matches from a widely available colored letter toy. Among those born in the decade after the toy began to be manufactured, the proportion of synesthetes with learned letter-color pairings approaches 15% for some 5-year periods. Among those born 5 years or more before it was manufactured, none have colors learned from the toy. Analysis of the letter-color matching data suggests the only difference between synesthetes with matches to the toy and those without is exposure to the stimulus. These data indicate learning of letter-color pairings from external contingencies can occur in a substantial fraction of synesthetes, and are consistent with the hypothesis that grapheme-color synesthesia is a kind of conditioned mental imagery.     

Neurocognitive mechanisms of synesthesia

Synesthesia is a condition in which stimulation of one sensory modality causes unusual experiences in a second, unstimulated modality. Although long treated as a curiosity, recent research with a combination of phenomenological, behavioral, and neuroimaging methods has begun to identify the cognitive and neural basis of synesthesia. Here, we review this literature with an emphasis on grapheme-color synesthesia, in which viewing letters and numbers induces the perception of colors. We discuss both the substantial progress that has been made in the past fifteen years and some open questions. In particular, we focus on debates in the field relating to the neural basis of synesthesia, including the relationship between synesthesia and attention and the role of meaning in synesthetic colors. We propose that some, but probably not all, of these differences can be accounted for by differences in the synesthetes studied and discuss some methodological implications of these individual differences.     

Individual differences among grapheme-color synesthetes: brain-behavior correlations

Grapheme-color synesthetes experience specific colors associated with specific number or letter characters. To determine the neural locus of this condition, we compared behavioral and fMRI responses in six grapheme-color synesthetes to control subjects. In our behavioral experiments, we found that a subject's synesthetic experience can aid in texture segregation (experiment 1) and reduce the effects of crowding (experiment 2). For synesthetes, graphemes produced larger fMRI responses in color-selective area human V4 than for control subjects (experiment 3). Importantly, we found a correlation within subjects between the behavioral and fMRI results; subjects with better performance on the behavioral experiments showed larger fMRI responses in early retinotopic visual areas (V1, V2, V3, and hV4). These results suggest that grapheme-color synesthesia is the result of cross-activation between grapheme-selective and color-selective brain areas. The correlation between the behavioral and fMRI results suggests that grapheme-color synesthetes may constitute a heterogeneous group.     

Enhanced cortical excitability in grapheme-color synesthesia and its modulation

Synesthesia is an unusual condition characterized by the over-binding of two or more features and the concomitant automatic and conscious experience of atypical, ancillary images or perceptions. Previous research suggests that synesthetes display enhanced modality-specific perceptual processing, but it remains unclear whether enhanced processing contributes to conscious awareness of color photisms. In three experiments, we investigated whether grapheme-color synesthesia is characterized by enhanced cortical excitability in primary visual cortex and the role played by this hyperexcitability in the expression of synesthesia. Using transcranial magnetic stimulation, we show that synesthetes display 3-fold lower phosphene thresholds than controls during stimulation of the primary visual cortex. We next used transcranial direct current stimulation to discriminate between two competing hypotheses of the role of hyperexcitability in the expression of synesthesia. We demonstrate that synesthesia can be selectively augmented with cathodal stimulation and attenuated with anodal stimulation of primary visual cortex. A control task revealed that the effect of the brain stimulation was specific to the experience of synesthesia. These results indicate that hyperexcitability acts as a source of noise in visual cortex that influences the availability of the neuronal signals underlying conscious awareness of synesthetic photisms.     

Unconscious letter discrimination is enhanced by association with conscious color perception in visual form agnosia

Adaptive behavior guided by unconscious visual cues occurs in patients with various kinds of brain damage as well as in normal observers, all of whom can process visual information of which they are fully unaware [1] [2] [3] [4] [5] [6] [7] [8]. Little is known on the possibility that unconscious vision is influenced by visual cues that have access to consciousness [9]. Here we report a 'blind' letter discrimination induced through a semantic interaction with conscious color processing in a patient who is agnosic for visual shapes, but has normal color vision and visual imagery. In seeing the initial letters of color names printed in different colors, it is normally easier to name the print color when it is congruent with the initial letter of the color name than when it is not [10]. The patient could discriminate the initial letters of the words 'red' and 'green' printed in the corresponding colors significantly above chance but without any conscious accompaniment, whereas he performed at chance with the reverse color-letter mapping as well as in standard tests of letter reading. We suggest that the consciously perceived colors activated a representation of the corresponding word names and their component letters, which in turn brought out a partially successful, unconscious processing of visual inputs corresponding to the activated letter representations.     

The importance of individual differences in grapheme-color synesthesia

In this issue of Neuron, Hubbard et al. show individual differences in how grapheme-color synesthetes perform on cognitive tasks. Importantly, these behavioral differences were correlated with fMRI measures. Such individual differences have important ramifications for synesthesia research. If individual differences are ignored, then synesthesia research will be characterized by erroneous conclusions and failures to replicate.     

Synesthesia as a neuronal palimpsest

Synesthetes, a small fraction of the population, experience systematic, additional associations. For example, they may arbitrarily associate a specific color to each letter or number. Synesthesia has offered for the last ten years to cognitive science a unique opportunity to study the neural bases of subjective experience, drawing on individual differences just like in neuropsychology, but with healthy people. Here we review the current knowledge and propose a new theory, the "palimpsest hypothesis", a variant of the recycling hypothesis for reading. The neural development of written language expertise (a recent cultural invention acquired without any genetic modification) requires indeed the recycling of brain regions predisposed to expertise acquisition into reading regions. The palimpsest hypothesis supposes that for synesthetes recycling involves neuronal networks that were already specialized for color perception. Synesthetic colors would be the remains of this former expertise.     

Behavioral and anatomical consequences of early versus late symbol training in macaques

Distinct brain regions, reproducible from one person to the next, are specialized for processing different kinds of human expertise, such as face recognition?and reading. Here, we explore the relationship between age of learning, learning ability, and specialized brain structures. Specifically, we ask whether the existence of reproducible cortical domains necessarily means that certain abilities are innate, or innately easily learned, or whether reproducible domains can be formed, or refined, by interactions between genetic programs and common early experience. Functional MRI showed that intensive early, but not late, experience caused the formation of category-selective regions in macaque temporal lobe for stimuli never naturally encountered by monkeys. And behaviorally, early training produced more fluent processing of these stimuli than the same training in adults. One explanation for these results is that in higher cortical areas, as in early sensory areas, experience drives functional clustering and functional clustering determines how that information is processed.     

Survival of the Synesthesia Gene: Why Do People Hear Colors and Taste Words?

Synesthesia is a perceptual experience in which stimuli presented through one modality will spontaneously evoke sensations in an unrelated modality. The condition occurs from increased communication between sensory regions and is involuntary, automatic, and stable over time. While synesthesia can occur in response to drugs, sensory deprivation, or brain damage, research has largely focused on heritable variants comprising roughly 4% of the general population. Genetic research on synesthesia suggests the phenomenon is heterogeneous and polygenetic, yet it remains unclear whether synesthesia ever provided a selective advantage or is merely a byproduct of some other useful selected trait. Progress in uncovering the genetic basis of synesthesia will help us understand why synesthesia has been conserved in the population.     

‘When Birds of a Feather Flock Together’: Synesthetic Correspondences Modulate Audiovisual Integration in Non-Synesthetes

Background

Synesthesia is a condition in which the stimulation of one sense elicits an additional experience, often in a different (i.e., unstimulated) sense. Although only a small proportion of the population is synesthetic, there is growing evidence to suggest that neurocognitively-normal individuals also experience some form of synesthetic association between the stimuli presented to different sensory modalities (i.e., between auditory pitch and visual size, where lower frequency tones are associated with large objects and higher frequency tones with small objects). While previous research has highlighted crossmodal interactions between synesthetically corresponding dimensions, the possible role of synesthetic associations in multisensory integration has not been considered previously.

Methodology

Here we investigate the effects of synesthetic associations by presenting pairs of asynchronous or spatially discrepant visual and auditory stimuli that were either synesthetically matched or mismatched. In a series of three psychophysical experiments, participants reported the relative temporal order of presentation or the relative spatial locations of the two stimuli.

Principal Findings

The reliability of non-synesthetic participants'' estimates of both audiovisual temporal asynchrony and spatial discrepancy were lower for pairs of synesthetically matched as compared to synesthetically mismatched audiovisual stimuli.

Conclusions

Recent studies of multisensory integration have shown that the reduced reliability of perceptual estimates regarding intersensory conflicts constitutes the marker of a stronger coupling between the unisensory signals. Our results therefore indicate a stronger coupling of synesthetically matched vs. mismatched stimuli and provide the first psychophysical evidence that synesthetic congruency can promote multisensory integration. Synesthetic crossmodal correspondences therefore appear to play a crucial (if unacknowledged) role in the multisensory integration of auditory and visual information.     

Generalization and categorization of spectral colors in goldfish I. Experiments with one training wavelength

Goldfish have a tetrachromatic color vision with a high discrimination ability for spectral colors as well as for object colors. We investigate the question whether goldfish organize the high number of discriminable colors in terms of color categories, i.e. in a few larger groups of colors independent of wavelength discrimination. Twenty-four goldfish were trained with food reward, each fish on one out of 13 wavelengths between 371 nm and 630 nm. In transfer tests two different wavelengths were presented, one shorter and one longer than the training wavelength, and the choice behavior was determined. Choice frequencies of ≥50% were assumed to indicate similarity to the training color. The wavelength ranges ≥50% were about 100 nm and twice as large as the just noticeable differences measured in wavelength discrimination tests (Fig. 7). The ranges were surprisingly about the same for all training wavelengths, provided the data were plotted on a wavelength scale weighted according to discrimination ability (Fig. 4). Thus, with the training method chosen goldfish showed a kind of categorization which, however, depends on training wavelength and discrimination ability. Generalization tests in which training wavelength and test wavelengths were shown separately for 2 min each gave the same results as wavelength discrimination tests (Figs. 5 and 6) and are, therefore, not indicative for color categories.     

Sensory Perception: Lessons from Synesthesia: Using Synesthesia to Inform the Understanding of Sensory Perception

Synesthesia, the conscious, idiosyncratic, repeatable, and involuntary sensation of one sensory modality in response to another, is a condition that has puzzled both researchers and philosophers for centuries. Much time has been spent proving the condition’s existence as well as investigating its etiology, but what can be learned from synesthesia remains a poorly discussed topic. Here, synaesthesia is presented as a possible answer rather than a question to the current gaps in our understanding of sensory perception. By first appreciating the similarities between normal sensory perception and synesthesia, one can use what is known about synaesthesia, from behavioral and imaging studies, to inform our understanding of “normal” sensory perception. In particular, in considering synesthesia, one can better understand how and where the different sensory modalities interact in the brain, how different sensory modalities can interact without confusion ― the binding problem ― as well as how sensory perception develops.     

A Whole-Genome Scan and Fine-Mapping Linkage Study of Auditory-Visual Synesthesia Reveals Evidence of Linkage to Chromosomes 2q24, 5q33, 6p12, and 12p12

Synesthesia, a neurological condition affecting between 0.05%–1% of the population, is characterized by anomalous sensory perception and associated alterations in cognitive function due to interference from synesthetic percepts. A stimulus in one sensory modality triggers an automatic, consistent response in either another modality or a different aspect of the same modality. Familiality studies show evidence of a strong genetic predisposition; whereas initial pedigree analyses supported a single-gene X-linked dominant mode of inheritance with a skewed F:M ratio and a notable absence of male-to-male transmission, subsequent analyses in larger samples indicated that the mode of inheritance was likely to be more complex. Here, we report the results of a whole-genome linkage scan for auditory-visual synesthesia with 410 microsatellite markers at 9.05 cM density in 43 multiplex families (n = 196) with potential candidate regions fine-mapped at 5 cM density. Using NPL and HLOD analysis, we identified four candidate regions. Significant linkage at the genome-wide level was detected to chromosome 2q24 (HLOD = 3.025, empirical genome-wide p = 0.047). Suggestive linkage was found to chromosomes 5q33, 6p12, and 12p12. No support was found for linkage to the X chromosome; furthermore, we have identified two confirmed cases of male-to-male transmission of synesthesia. Our results demonstrate that auditory-visual synesthesia is likely to be an oligogenic disorder subject to multiple modes of inheritance and locus heterogeneity. This study comprises a significant step toward identifying the genetic substrates underlying synesthesia, with important implications for our understanding of the role of genes in human cognition and perception.     

Toward a framework for the evaluation of feature binding in pigeons

Pigeons were trained in a new procedure to test for visual binding errors between the dimensions of color and shape. In Experiment 1, pigeons learned to discriminate a target compound from 15 non-target compounds (constructed from four colors and shapes) by choosing one of two hoppers in a two-hopper choice task. The similarity of the target to non-target stimuli influenced choice responding. In Experiment 2, pigeons learned to detect a target compound when presented with a non-target compound within the same trial under conditions of simultaneity and sequentiality. Non-target trials were arranged to allow for the testing of binding errors (i.e., false identifications of the target on certain non-target trials). Transient evidence for binding errors in two of the birds occurred at the start of two-item training, but decreased with training. The experiments represent an important step toward developing a framework for the evaluation of visual feature binding in nonhumans.     

Accurate visual memory of colors in controlling the pecking behavior of quail chicks

Animals are predisposed to memorize specific features of objects they encounter, and to link them with behavioral outputs in a selective manner. In this study, we examined whether chicks memorize objects by colors, and how they exploit the memorized color cues for selective pecking in 1- to 2-days-old quail chicks (Coturnix japonica). Ball-shaped beads painted in green (G), yellowish green (YG) and the intermediate color (YGG) were used. Repetitive presentation of a bead (interval: 4.5 min) resulted in gradually fewer pecks (habituation). Subsequent presentation of a different color caused proportionately more pecks (dishabituation); e.g., after habituation to the G bead, the YG bead caused a stronger dishabituation than the YGG bead did. The dishabituation appeared symmetric; e.g., the YG bead caused as strong dishabituation after the G-habituation, as was caused by the G bead after the YG-habituation. Number of pecks could thus reveal the memory-based color perception in chicks. Similar discrimination of beads by memorized color cues was found after one-trial passive avoidance training, where chicks learned to avoid a bitter-tasting object without any differential pre-training experiences. However, proportion of the chicks that discriminated between different colors became progressively smaller at test 15 min, 1 hr, and 24 hr post-training. On the other hand, proportion of chicks that distinguished beads by non-color cues remained unchanged. Chicks may primarily form an accurate memory of colors, but gradually change the link between the color memory and the pecking behavior.     

Generalization gradients and representation modes after absolute and relative discrimination learning in young chickens

Perceptual relativity has become a central issue in animal psychophysics. In order to assess how the interplay of training experience and stimulus dimension might affect perceptual relativity, we investigated the role of 'absolute' and 'relative' training on the learning and representation of stimuli from two dimensions that might favor absolute or relative encoding to a different degree. Young chicks learned to discriminate 3D-objects by either color or size. During 'absolute' training always the choice of the same stimulus of a simultaneously presented pair was reinforced, while choice of the larger (smaller) or greener (bluer) stimulus was reinforced during 'relative' training. Overall, discrimination learning was faster with relative training, but size learning profited more from 'relative' training than color learning. Post-training generalization tests revealed a combined effect of training experience and stimulus dimension: a higher amount of absolute encoding occurred with absolute training and color learning, while relative choices prevailed with relative training and size learning.     

Seeing Life through Positive-Tinted Glasses: Color–Meaning Associations

There is a growing body of literature to show that color can convey information, owing to its emotionally meaningful associations. Most research so far has focused on negative hue–meaning associations (e.g., red) with the exception of the positive aspects associated with green. We therefore set out to investigate the positive associations of two colors (i.e., green and pink), using an emotional facial expression recognition task in which colors provided the emotional contextual information for the face processing. In two experiments, green and pink backgrounds enhanced happy face recognition and impaired sad face recognition, compared with a control color (gray). Our findings therefore suggest that because green and pink both convey positive information, they facilitate the processing of emotionally congruent facial expressions (i.e., faces expressing happiness) and interfere with that of incongruent facial expressions (i.e., faces expressing sadness). Data also revealed a positive association for white. Results are discussed within the theoretical framework of emotional cue processing and color meaning.