The uses of colour vision: behavioural and physiological distinctiveness of colour stimuli

Colour and greyscale (black and white) pictures look different to us, but it is not clear whether the difference in appearance is a consequence of the way our visual system uses colour signals or a by-product of our experience. In principle, colour images are qualitatively different from greyscale images because they make it possible to use different processing strategies. Colour signals provide important cues for segmenting the image into areas that represent different objects and for linking together areas that represent the same object. If this property of colour signals is exploited in visual processing we would expect colour stimuli to look different, as a class, from greyscale stimuli. We would also expect that adding colour signals to greyscale signals should change the way that those signals are processed. We have investigated these questions in behavioural and in physiological experiments. We find that male marmosets (all of which are dichromats) rapidly learn to distinguish between colour and greyscale copies of the same images. The discrimination transfers to new image pairs, to new colours and to image pairs in which the colour and greyscale images are spatially different. We find that, in a proportion of neurons recorded in the marmoset visual cortex, colour-shifts in opposite directions produce similar enhancements of the response to a luminance stimulus. We conclude that colour is, both behaviourally and physiologically, a distinctive property of images.     

MM-MDS: A Multidimensional Scaling Database with Similarity Ratings for 240 Object Categories from the Massive Memory Picture Database

Cognitive theories in visual attention and perception, categorization, and memory often critically rely on concepts of similarity among objects, and empirically require measures of “sameness” among their stimuli. For instance, a researcher may require similarity estimates among multiple exemplars of a target category in visual search, or targets and lures in recognition memory. Quantifying similarity, however, is challenging when everyday items are the desired stimulus set, particularly when researchers require several different pictures from the same category. In this article, we document a new multidimensional scaling database with similarity ratings for 240 categories, each containing color photographs of 16–17 exemplar objects. We collected similarity ratings using the spatial arrangement method. Reports include: the multidimensional scaling solutions for each category, up to five dimensions, stress and fit measures, coordinate locations for each stimulus, and two new classifications. For each picture, we categorized the item''s prototypicality, indexed by its proximity to other items in the space. We also classified pairs of images along a continuum of similarity, by assessing the overall arrangement of each MDS space. These similarity ratings will be useful to any researcher that wishes to control the similarity of experimental stimuli according to an objective quantification of “sameness.”     

Distributed Patterns of Event-Related Potentials Predict Subsequent Ratings of Abstract Stimulus Attributes

Exposure to pleasant and rewarding visual stimuli can bias people''s choices towards either immediate or delayed gratification. We hypothesised that this phenomenon might be based on carry-over effects from a fast, unconscious assessment of the abstract ‘time reference’ of a stimuli, i.e. how the stimulus relates to one''s personal understanding and connotation of time. Here we investigated whether participants'' post-experiment ratings of task-irrelevant, positive background visual stimuli for the dimensions ‘arousal’ (used as a control condition) and ‘time reference’ were related to differences in single-channel event-related potentials (ERPs) and whether they could be predicted from spatio-temporal patterns of ERPs. Participants performed a demanding foreground choice-reaction task while on each trial one task-irrelevant image (depicting objects, people and scenes) was presented in the background. Conventional ERP analyses as well as multivariate support vector regression (SVR) analyses were conducted to predict participants'' subsequent ratings. We found that only SVR allowed both ‘arousal’ and ‘time reference’ ratings to be predicted during the first 200 ms post-stimulus. This demonstrates an early, automatic semantic stimulus analysis, which might be related to the high relevance of ‘time reference’ to everyday decision-making and preference formation.     

Colour Terms Affect Detection of Colour and Colour-Associated Objects Suppressed from Visual Awareness

The idea that language can affect how we see the world continues to create controversy. A potentially important study in this field has shown that when an object is suppressed from visual awareness using continuous flash suppression (a form of binocular rivalry), detection of the object is differently affected by a preceding word prime depending on whether the prime matches or does not match the object. This may suggest that language can affect early stages of vision. We replicated this paradigm and further investigated whether colour terms likewise influence the detection of colours or colour-associated object images suppressed from visual awareness by continuous flash suppression. This method presents rapidly changing visual noise to one eye while the target stimulus is presented to the other. It has been shown to delay conscious perception of a target for up to several minutes. In Experiment 1 we presented greyscale photos of objects. They were either preceded by a congruent object label, an incongruent label, or white noise. Detection sensitivity (d’) and hit rates were significantly poorer for suppressed objects preceded by an incongruent label compared to a congruent label or noise. In Experiment 2, targets were coloured discs preceded by a colour term. Detection sensitivity was significantly worse for suppressed colour patches preceded by an incongruent colour term as compared to a congruent term or white noise. In Experiment 3 targets were suppressed greyscale object images preceded by an auditory presentation of a colour term. On congruent trials the colour term matched the object’s stereotypical colour and on incongruent trials the colour term mismatched. Detection sensitivity was significantly poorer on incongruent trials than congruent trials. Overall, these findings suggest that colour terms affect awareness of coloured stimuli and colour- associated objects, and provide new evidence for language-perception interaction in the brain.     

Affective responses by adults with autism are reduced to social images but elevated to images related to circumscribed interests

Individuals with autism spectrum disorders (ASD) demonstrate increased visual attention and elevated brain reward circuitry responses to images related to circumscribed interests (CI), suggesting that a heightened affective response to CI may underlie their disproportionate salience and reward value in ASD. To determine if individuals with ASD differ from typically developing (TD) adults in their subjective emotional experience of CI object images, non-CI object images and social images, 213 TD adults and 56 adults with ASD provided arousal ratings (sensation of being energized varying along a dimension from calm to excited) and valence ratings (emotionality varying along dimension of approach to withdrawal) for a series of 114 images derived from previous research on CI. The groups did not differ on arousal ratings for any image type, but ASD adults provided higher valence ratings than TD adults for CI-related images, and lower valence ratings for social images. Even after co-varying the effects of sex, the ASD group, but not the TD group, gave higher valence ratings to CI images than social images. These findings provide additional evidence that ASD is characterized by a preference for certain categories of non-social objects and a reduced preference for social stimuli, and support the dissemination of this image set for examining aspects of the circumscribed interest phenotype in ASD.     

CONTEXTUAL SHIFTS IN HEDONIC JUDGMENTS

Taste intensity ratings are higher when many low-intensity stimuli are presented than when many high-intensity stimuli are presented. It is investigated whether similar contextual shifts occur for hedonic judgments. Experiment 1 assessed the perceived pleasantness of a heterogeneous stimulus set containing either a large number of quinine (unpleasant context) or sucrose (pleasant context) solutions. In Experiment 2 subjects made hedonic judgments for a set of sucrose solutions. They were classified pre-experimentally as ‘sweet-likers,’‘neutrals,’ or ‘sweet-dislikers.’ Stimulus frequency distributions were positively or negatively skewed. In Experiment 1, hedonic ratings in the pleasant context were lower than those obtained in the unpleasant context. In Experiment 2, no effect of stimulus context was found. It is argued that contextual shifts in hedonic judgments are found only if the stimuli differ substantially along the hedonic dimension. In addition, the subject groups should be homogeneous with regard to preference rank orders, and should be sufficiently large to allow testing with adequate statistical power.     

Categorization of Natural Dynamic Audiovisual Scenes

This work analyzed the perceptual attributes of natural dynamic audiovisual scenes. We presented thirty participants with 19 natural scenes in a similarity categorization task, followed by a semi-structured interview. The scenes were reproduced with an immersive audiovisual display. Natural scene perception has been studied mainly with unimodal settings, which have identified motion as one of the most salient attributes related to visual scenes, and sound intensity along with pitch trajectories related to auditory scenes. However, controlled laboratory experiments with natural multimodal stimuli are still scarce. Our results show that humans pay attention to similar perceptual attributes in natural scenes, and a two-dimensional perceptual map of the stimulus scenes and perceptual attributes was obtained in this work. The exploratory results show the amount of movement, perceived noisiness, and eventfulness of the scene to be the most important perceptual attributes in naturalistically reproduced real-world urban environments. We found the scene gist properties openness and expansion to remain as important factors in scenes with no salient auditory or visual events. We propose that the study of scene perception should move forward to understand better the processes behind multimodal scene processing in real-world environments. We publish our stimulus scenes as spherical video recordings and sound field recordings in a publicly available database.     

A preference for contralateral stimuli in human object- and face-selective cortex

Visual input from the left and right visual fields is processed predominantly in the contralateral hemisphere. Here we investigated whether this preference for contralateral over ipsilateral stimuli is also found in high-level visual areas that are important for the recognition of objects and faces. Human subjects were scanned with functional magnetic resonance imaging (fMRI) while they viewed and attended faces, objects, scenes, and scrambled images in the left or right visual field. With our stimulation protocol, primary visual cortex responded only to contralateral stimuli. The contralateral preference was smaller in object- and face-selective regions, and it was smallest in the fusiform gyrus. Nevertheless, each region showed a significant preference for contralateral stimuli. These results indicate that sensitivity to stimulus position is present even in high-level ventral visual cortex.     

On the Contribution of Binocular Disparity to the Long-Term Memory for Natural Scenes

Binocular disparity is a fundamental dimension defining the input we receive from the visual world, along with luminance and chromaticity. In a memory task involving images of natural scenes we investigate whether binocular disparity enhances long-term visual memory. We found that forest images studied in the presence of disparity for relatively long times (7s) were remembered better as compared to 2D presentation. This enhancement was not evident for other categories of pictures, such as images containing cars and houses, which are mostly identified by the presence of distinctive artifacts rather than by their spatial layout. Evidence from a further experiment indicates that observers do not retain a trace of stereo presentation in long-term memory.     

Functional mechanisms shaping lateral geniculate responses to artificial and natural stimuli

Functional models of the early visual system should predict responses not only to simple artificial stimuli but also to sequences of complex natural scenes. An ideal testbed for such models is the lateral geniculate nucleus (LGN). Mechanisms shaping LGN responses include the linear receptive field and two fast adaptation processes, sensitive to luminance and contrast. We propose a compact functional model for these mechanisms that operates on sequences of arbitrary images. With the same parameters that fit the firing rate responses to simple stimuli, it predicts the bulk of the firing rate responses to complex stimuli, including natural scenes. Further improvements could result by adding a spiking mechanism, possibly one capable of bursts, but not by adding mechanisms of slow adaptation. We conclude that up to the LGN the responses to natural scenes can be largely explained through insights gained with simple artificial stimuli.     

Does Presentation Format Influence Visual Size Discrimination in Tufted Capuchin Monkeys (Sapajus spp.)?

Most experimental paradigms to study visual cognition in humans and non-human species are based on discrimination tasks involving the choice between two or more visual stimuli. To this end, different types of stimuli and procedures for stimuli presentation are used, which highlights the necessity to compare data obtained with different methods. The present study assessed whether, and to what extent, capuchin monkeys’ ability to solve a size discrimination problem is influenced by the type of procedure used to present the problem. Capuchins’ ability to generalise knowledge across different tasks was also evaluated. We trained eight adult tufted capuchin monkeys to select the larger of two stimuli of the same shape and different sizes by using pairs of food items (Experiment 1), computer images (Experiment 1) and objects (Experiment 2). Our results indicated that monkeys achieved the learning criterion faster with food stimuli compared to both images and objects. They also required consistently fewer trials with objects than with images. Moreover, female capuchins had higher levels of acquisition accuracy with food stimuli than with images. Finally, capuchins did not immediately transfer the solution of the problem acquired in one task condition to the other conditions. Overall, these findings suggest that – even in relatively simple visual discrimination problems where a single perceptual dimension (i.e., size) has to be judged – learning speed strongly depends on the mode of presentation.     

Area summation in human vision at and above detection threshold

The initial image-processing stages of visual cortex are well suited to a local (patchwise) analysis of the viewed scene. But the world's structures extend over space as textures and surfaces, suggesting the need for spatial integration. Most models of contrast vision fall shy of this process because (i) the weak area summation at detection threshold is attributed to probability summation (PS) and (ii) there is little or no advantage of area well above threshold. Both of these views are challenged here. First, it is shown that results at threshold are consistent with linear summation of contrast following retinal inhomogeneity, spatial filtering, nonlinear contrast transduction and multiple sources of additive Gaussian noise. We suggest that the suprathreshold loss of the area advantage in previous studies is due to a concomitant increase in suppression from the pedestal. To overcome this confound, a novel stimulus class is designed where: (i) the observer operates on a constant retinal area, (ii) the target area is controlled within this summation field, and (iii) the pedestal is fixed in size. Using this arrangement, substantial summation is found along the entire masking function, including the region of facilitation. Our analysis shows that PS and uncertainty cannot account for the results, and that suprathreshold summation of contrast extends over at least seven target cycles of grating.     

Assessment of novel binocular colour, motion and contrast tests in glaucoma

The effects of glaucoma on binocular visual sensitivity for the detection of various stimulus attributes are investigated at the fovea and in four paracentral retinal regions. The study employed a number of visual stimuli designed to isolate the processing of various stimulus attributes. We measured absolute contrast detection thresholds and functional contrast sensitivity by using Landolt ring stimuli. This psychophysical Landolt C-based contrast test of detection and gap discrimination allowed us to test parafoveally at 6 ° from fixation and foveally by employing interleaved testing locations. First-order motion perception was examined by using moving stimuli embedded in static luminance contrast noise. Red/green (RG) and yellow/blue (YB) colour thresholds were measured with the Colour Assessment and Diagnosis (CAD) test, which utilises random dynamic luminance contrast noise (± 45 %) to ensure that only colour and not luminance signals are available for target detection. Subjects were normal controls (n?=?65) and glaucoma patients with binocular visual field defects (n?=?15) classified based on their Humphrey Field Analyzer mean deviation (MD) scores. The impairment of visual function varied depending on the stimulus attribute and location tested. Progression of loss was noted for all tests as the degree of glaucoma increased. For subjects with mild glaucoma (MD ?0.01 dB to ?6.00 dB) significantly more data points fell outside the normal age-representative range for RG colour thresholds than for any other visual test, followed by motion thresholds. This was particularly the case for the parafoveal data compared with the foveal data. Thus, a multifaceted measure of binocular visual performance, incorporating RG colour and motion test at multiple locations, might provide a better index for comparison with quality of life measures in glaucoma.     

Cortical sensitivity to visual features in natural scenes

A central hypothesis concerning sensory processing is that the neuronal circuits are specifically adapted to represent natural stimuli efficiently. Here we show a novel effect in cortical coding of natural images. Using spike-triggered average or spike-triggered covariance analyses, we first identified the visual features selectively represented by each cortical neuron from its responses to natural images. We then measured the neuronal sensitivity to these features when they were present in either natural images or random stimuli. We found that in the responses of complex cells, but not of simple cells, the sensitivity was markedly higher for natural images than for random stimuli. Such elevated sensitivity leads to increased detectability of the visual features and thus an improved cortical representation of natural scenes. Interestingly, this effect is due not to the spatial power spectra of natural images, but to their phase regularities. These results point to a distinct visual-coding strategy that is mediated by contextual modulation of cortical responses tuned to the spatial-phase structure of natural scenes.     

Modulation of V1 Spike Response by Temporal Interval of Spatiotemporal Stimulus Sequence

The spike activity of single neurons of the primary visual cortex (V1) becomes more selective and reliable in response to wide-field natural scenes compared to smaller stimuli confined to the classical receptive field (RF). However, it is largely unknown what aspects of natural scenes increase the selectivity of V1 neurons. One hypothesis is that modulation by surround interaction is highly sensitive to small changes in spatiotemporal aspects of RF surround. Such a fine-tuned modulation would enable single neurons to hold information about spatiotemporal sequences of oriented stimuli, which extends the role of V1 neurons as a simple spatiotemporal filter confined to the RF. In the current study, we examined the hypothesis in the V1 of awake behaving monkeys, by testing whether the spike response of single V1 neurons is modulated by temporal interval of spatiotemporal stimulus sequence encompassing inside and outside the RF. We used two identical Gabor stimuli that were sequentially presented with a variable stimulus onset asynchrony (SOA): the preceding one (S1) outside the RF and the following one (S2) in the RF. This stimulus configuration enabled us to examine the spatiotemporal selectivity of response modulation from a focal surround region. Although S1 alone did not evoke spike responses, visual response to S2 was modulated for SOA in the range of tens of milliseconds. These results suggest that V1 neurons participate in processing spatiotemporal sequences of oriented stimuli extending outside the RF.     

Saliency predicts change detection in pictures of natural scenes

It has been proposed that the visual system encodes the salience of objects in the visual field in an explicit two-dimensional map that guides visual selective attention. Experiments were conducted to determine whether salience measurements applied to regions of pictures of outdoor scenes could predict the detection of changes in those regions. To obtain a quantitative measure of change detection, observers located changes in pairs of colour pictures presented across an interstimulus interval (ISI). Salience measurements were then obtained from different observers for image change regions using three independent methods, and all were positively correlated with change detection. Factor analysis extracted a single saliency factor that accounted for 62% of the variance contained in the four measures. Finally, estimates of the magnitude of the image change in each picture pair were obtained, using nine separate visual filters representing low-level vision features (luminance, colour, spatial frequency, orientation, edge density). None of the feature outputs was significantly associated with change detection or saliency. On the other hand it was shown that high-level (structural) properties of the changed region were related to saliency and to change detection: objects were more salient than shadows and more detectable when changed.     

Animal Detection in Natural Images: Effects of Color and Image Database

The visual system has a remarkable ability to extract categorical information from complex natural scenes. In order to elucidate the role of low-level image features for the recognition of objects in natural scenes, we recorded saccadic eye movements and event-related potentials (ERPs) in two experiments, in which human subjects had to detect animals in previously unseen natural images. We used a new natural image database (ANID) that is free of some of the potential artifacts that have plagued the widely used COREL images. Color and grayscale images picked from the ANID and COREL databases were used. In all experiments, color images induced a greater N1 EEG component at earlier time points than grayscale images. We suggest that this influence of color in animal detection may be masked by later processes when measuring reation times. The ERP results of go/nogo and forced choice tasks were similar to those reported earlier. The non-animal stimuli induced bigger N1 than animal stimuli both in the COREL and ANID databases. This result indicates ultra-fast processing of animal images is possible irrespective of the particular database. With the ANID images, the difference between color and grayscale images is more pronounced than with the COREL images. The earlier use of the COREL images might have led to an underestimation of the contribution of color. Therefore, we conclude that the ANID image database is better suited for the investigation of the processing of natural scenes than other databases commonly used.     

Fractal-like image statistics in visual art: similarity to natural scenes

Both natural scenes and visual art are often perceived as esthetically pleasing. It is therefore conceivable that the two types of visual stimuli share statistical properties. For example, natural scenes display a Fourier power spectrum that tends to fall with spatial frequency according to a power-law. This result indicates that natural scenes have fractal-like, scale-invariant properties. In the present study, we asked whether visual art displays similar statistical properties by measuring their Fourier power spectra. Our analysis was restricted to graphic art from the Western hemisphere. For comparison, we also analyzed images, which generally display relatively low or no esthetic quality (household and laboratory objects, parts of plants, and scientific illustrations). Graphic art, but not the other image categories, resembles natural scenes in showing fractal-like, scale-invariant statistics. This property is universal in our sample of graphic art; it is independent of cultural variables, such as century and country of origin, techniques used or subject matter. We speculate that both graphic art and natural scenes share statistical properties because visual art is adapted to the structure of the visual system which, in turn, is adapted to process optimally the image statistics of natural scenes.     

Can video images imitate real stimuli in animal behaviour experiments?

The use of video images in place of natural stimuli in animal behaviour experiments is reviewed. Unlike most other artificial means of stimulus presentation, video stimuli can depict complex moving objects such as other animals, preserving the temporal and spatial patterns of movement precisely as well as colour and sounds for repeated playback. Computer editing can give flexibility and control over all elements of the stimulus. A variety of limitations of video image presentation are also considered. Televisions and video monitors are designed with human vision in mind, and some non-human animals that differ in aspects of visual processing such as their colour vision, critical flicker-fusion threshold, perception of depth and visual acuity, may perceive video images differently to ourselves. The failure of video stimuli to interact with subjects can be a drawback for some studies. For video to be useful, it is important to confirm that the subject animal responds to the image in a comparable way to the real stimulus, and the criteria used to assess this are discussed. Finally, the contribution made by video studies to date in the understanding of animal visual responses is considered, and recommendations as to the future uses of video are made.     

Size as a parameter for tuning visual attention]

The ability of visual attention to tune to the stimulus size (when this size could not be described by spatial frequencies) was studies. Sinusoidal gratings with frequencies of 1.5, 3, and 6 cycle/degree were used as test stimuli. All these stimuli consisted of 3 periods, consequently, they had different sizes: 2 x 2, 1 x 1, and 0.5 x 0.5 degrees. Three reference stimuli had the same sizes but were constructed as a superposition of all the test frequencies. The reference stimulus of suprathreshold contrast was displayed for 400 ms to the left or to the right of a fixation point at a distance of 3 degrees. After that, the test stimulus of threshold contrast was for 100 ms displayed symmetrically to the fixation point on the other side. Subjects were instructed that the sizes of the reference and test stimuli were the same. It was found that the probability of test detection decreased with increase in the difference between the sizes of the reference and test stimuli. Since in our experiments the spatial frequency could not be used for tuning visual attention, the obtained results suggest that there are specialized mechanisms in the visual system for estimation of the general image size.