Emotional Picture and Word Processing: An fMRI Study on Effects of Stimulus Complexity

Neuroscientific investigations regarding aspects of emotional experiences usually focus on one stimulus modality (e.g., pictorial or verbal). Similarities and differences in the processing between the different modalities have rarely been studied directly. The comparison of verbal and pictorial emotional stimuli often reveals a processing advantage of emotional pictures in terms of larger or more pronounced emotion effects evoked by pictorial stimuli. In this study, we examined whether this picture advantage refers to general processing differences or whether it might partly be attributed to differences in visual complexity between pictures and words. We first developed a new stimulus database comprising valence and arousal ratings for more than 200 concrete objects representable in different modalities including different levels of complexity: words, phrases, pictograms, and photographs. Using fMRI we then studied the neural correlates of the processing of these emotional stimuli in a valence judgment task, in which the stimulus material was controlled for differences in emotional arousal. No superiority for the pictorial stimuli was found in terms of emotional information processing with differences between modalities being revealed mainly in perceptual processing regions. While visual complexity might partly account for previously found differences in emotional stimulus processing, the main existing processing differences are probably due to enhanced processing in modality specific perceptual regions. We would suggest that both pictures and words elicit emotional responses with no general superiority for either stimulus modality, while emotional responses to pictures are modulated by perceptual stimulus features, such as picture complexity.     

The effect of odour priming on long latency visual evoked potentials of matching and mismatching objects.

This study reports a cross-modal, olfactory/visual event related potential (ERP) using odours as olfactory primes. The results show a difference in the ERP waveform for the N400 waveform when a visual image does not match the priming odour. An N400 peak was produced for both the matched and mismatched conditions but the peaks were significantly more negative for the mismatched condition. By the use of non-food odours this study extends an earlier finding by Grigor, who, using the same ERP paradigm, obtained similar results for food odours and photographs of food.     

Picture – Object Recognition in Kea (Nestor notabilis)

Although pictures are widely used as stimuli in cognitive experiments with both humans and animals, the question of how subjects interpret pictures receives less attention. Gaining a better understanding of this is especially important when working with avian subjects, as their visual anatomy and processing is different from that of humans, and even differs from one avian species to another. Successful testing for picture recognition in birds has been carried out mainly with pigeons, but no such research has been explicitly performed with ‘brainy’ birds like parrots, despite the fact that these have been the subject of exciting cognitive research. This study tested kea (Nestor notabilis) mountain parrots for picture–object recognition using a procedure which required the transfer of a learned discrimination task between pictures and objects. Kea successfully showed both picture‐to‐object and object‐to‐picture transfer and performed at a comparable level when pictures were displayed on a touch screen or as printed photographs.     

Learning of Multi-Modal Stimuli in Hawkmoths

The hawkmoth, Manduca sexta, uses both colour and odour to find flowers when foraging for nectar. In the present study we investigated how vision and olfaction interact during learning. Manduca sexta were equally attracted to a scented blue coloured feeding target (multimodal stimulus) as to one that does not carry any scent (unimodal stimulus; visual) or to an invisible scented target (unimodal stimulus; odour). This naive attraction to multimodal as well as to unimodal stimuli could be manipulated through training. Moths trained to feed from a blue, scented multimodal feeding target will, when tested in a set-up containing all three feeding targets, select the multimodal target as well as the scented, unimodal target, but ignore the visual target. Interestingly, moths trained to feed from a blue, unimodal visual feeding target will select the visual target as well as the scented, multimodal target, but ignore the unimodal odour target. Our results indicate that a multimodal target is perceived as two separate modalities, colour and odour, rather than as a unique fused target. These findings differ from earlier studies of desert ants that perceive combined visual and odour signals as a unique fused stimulus following learning trials.     

Manipulable objects facilitate cross-modal integration in peripersonal space

Previous studies have shown that tool use often modifies one's peripersonal space--i.e. the space directly surrounding our body. Given our profound experience with manipulable objects (e.g. a toothbrush, a comb or a teapot) in the present study we hypothesized that the observation of pictures representing manipulable objects would result in a remapping of peripersonal space as well. Subjects were required to report the location of vibrotactile stimuli delivered to the right hand, while ignoring visual distractors superimposed on pictures representing everyday objects. Pictures could represent objects that were of high manipulability (e.g. a cell phone), medium manipulability (e.g. a soap dispenser) and low manipulability (e.g. a computer screen). In the first experiment, when subjects attended to the action associated with the objects, a strong cross-modal congruency effect (CCE) was observed for pictures representing medium and high manipulability objects, reflected in faster reaction times if the vibrotactile stimulus and the visual distractor were in the same location, whereas no CCE was observed for low manipulability objects. This finding was replicated in a second experiment in which subjects attended to the visual properties of the objects. These findings suggest that the observation of manipulable objects facilitates cross-modal integration in peripersonal space.     

Neural correlates of face and object perception in an awake chimpanzee (Pan troglodytes) examined by scalp-surface event-related potentials

Background

The neural system of our closest living relative, the chimpanzee, is a topic of increasing research interest. However, electrophysiological examinations of neural activity during visual processing in awake chimpanzees are currently lacking.

Methodology/Principal Findings

In the present report, skin-surface event-related brain potentials (ERPs) were measured while a fully awake chimpanzee observed photographs of faces and objects in two experiments. In Experiment 1, human faces and stimuli composed of scrambled face images were displayed. In Experiment 2, three types of pictures (faces, flowers, and cars) were presented. The waveforms evoked by face stimuli were distinguished from other stimulus types, as reflected by an enhanced early positivity appearing before 200 ms post stimulus, and an enhanced late negativity after 200 ms, around posterior and occipito-temporal sites. Face-sensitive activity was clearly observed in both experiments. However, in contrast to the robustly observed face-evoked N170 component in humans, we found that faces did not elicit a peak in the latency range of 150–200 ms in either experiment.

Conclusions/Significance

Although this pilot study examined a single subject and requires further examination, the observed scalp voltage patterns suggest that selective processing of faces in the chimpanzee brain can be detected by recording surface ERPs. In addition, this non-invasive method for examining an awake chimpanzee can be used to extend our knowledge of the characteristics of visual cognition in other primate species.     

Discrimination of cartoons and photographs in pigeons: effects of scrambling of elements

Four groups of pigeons were trained on four visual discrimination tasks using people photographs, pigeon photographs, cartoons of people, and cartoons of pigeons. After completion of the discrimination, the subjects were tested with new stimuli in the same stimulus categories but never used during the training, and scrambled stimuli in which elements of the original stimulus were spatially randomized. The subjects showed generalization to the new stimuli except for the people cartoons. When scrambled stimuli were presented, the subjects showed suppression of responding to the photographs but not to the people cartoon. These results suggest that the pigeons recognized both the elements and spatial arrangement of the photographs but ignored the spatial arrangement of the human cartoons.     

Discrimination training with multimodal stimuli changes activity in the mushroom body of the hawkmoth Manduca sexta

Background

The mushroom bodies of the insect brain play an important role in olfactory processing, associative learning and memory. The mushroom bodies show odor-specific spatial patterns of activity and are also influenced by visual stimuli.

Methodology/Principal Findings

Functional imaging was used to investigate changes in the in vivo responses of the mushroom body of the hawkmoth Manduca sexta during multimodal discrimination training. A visual and an odour stimulus were presented either together or individually. Initially, mushroom body activation patterns were identical to the odour stimulus and the multimodal stimulus. After training, however, the mushroom body response to the rewarded multimodal stimulus was significantly lower than the response to the unrewarded unimodal odour stimulus, indicating that the coding of the stimuli had changed as a result of training. The opposite pattern was seen when only the unimodal odour stimulus was rewarded. In this case, the mushroom body was more strongly activated by the multimodal stimuli after training. When no stimuli were rewarded, the mushroom body activity decreased for both the multimodal and unimodal odour stimuli. There was no measurable response to the unimodal visual stimulus in any of the experiments. These results can be explained using a connectionist model where the mushroom body is assumed to be excited by olfactory stimulus components, and suppressed by multimodal configurations.

Conclusions

Discrimination training with multimodal stimuli consisting of visual and odour cues leads to stimulus specific changes in the in vivo responses of the mushroom body of the hawkmoth.     

Study of effects of low level microwave field by method of face masking

The aim of this study was to examine experimentally effects of low level, modulated microwaves on human central nervous system function utilizing the phenomenon of visual masking. Ten healthy volunteers, four males and six females, were exposed to electromagnetic field (450 MHz, 0.16 mW/cm2) with 7 Hz modulation frequency. Two photo series (visual stimuli) of unfamiliar, young male faces were presented to the subjects, one picture after another. All the photos were frontal views of unfamiliar faces, which could be recognized only by their unique combinations of features. The task was to identify the pictures from a group of six photos and to decide which order they were presented in. The phenomenon of visual masking is revealed as anamorphosis in subject's perception of two instantaneous visual stimuli presented within a short time interval. When both stimuli were to be recognized correctly and put in the right order, there was a statistically significant difference (P < 0.05) between the identification of the stimulus with microwave electromagnetic field and sham exposure. Recognition of both stimuli in a pair was better under the sham exposure conditions but the actual difference was only 5%. It was concluded that early stages of visual information processing are overwhelmingly robust and routine (and adaptively significant) activities, so that the low level 7 Hz modulated electromagnetic field effects exerted upon it are extremely weak.     

Flower and fruit volatiles assist host‐plant location in the Tomato fruit fly Neoceratitis cyanescens

Mature females of the tomato fruit fly Neoceratitis cyanescens can detect host fruit at a short distance using only visual stimuli, but little is known about the role of airborne volatile cues in the host searching strategy. A series of experiments is conducted in a laboratory wind tunnel, in which the behavioural responses of individual flies to volatiles from Solanaceae host plants (including tomato Lycopersicum esculentum Mill., bug weed Solanum mauritianum Scop. and Turkey berry Solanum torvum Sw.) are observed, according to some environmental (air speed) and physiological (age and mating status of females, time of day) factors. Mature females respond primarily to specific olfactory cues from blends of flowers or host fruit, preferentially unripe fruit for bug weed, as opposed to ripe fruit for Turkey berry or tomato. Males are also highly attracted by the odour of unripe fruit of bug weed. Wind plays a key role, as shown by the proportion of flies that reach the upwind section of the tunnel in the presence of both fruit odour and air flow (66.7%) and in the absence of either fruit odour (13.3%) or wind (36.7%). In response to fruit volatiles carried by wind, flies embark in a ‘plume tracking’ or ‘aim and shoot' flight, consistent with odour‐conditioned anemotaxis. Females respond to host fruit odour regardless of their age, egg load or mating status, and also more consistently in the afternoon, which is their preferential time of day for egg‐laying. Searching behaviour and response to host volatiles in N. cyanescens are discussed in the light of host‐finding and an adaptive strategy.     

Aesthetic issues in spatial composition: effects of position and direction on framing single objects

Artists who work in two-dimensional visual media regularly face the problem of how to compose their subjects in aesthetically pleasing ways within a surrounding rectangular frame. We performed psychophysical investigations of viewers' aesthetic preferences for the position and facing direction of single, directed objects (e.g. people, cars, teapots and flowers) within such rectangular frames. Preferences were measured using two-alternative forced-choice preference judgments, the method of adjustment, and free choice in taking photographs. In front-facing conditions, preference was greatest for pictures whose subject was located at or near the center of the frame and decreased monotonically and symmetrically with distance from the center (the center bias). In the left- or right-facing conditions, there was an additional preference for objects to face into rather than out of the frame (the inward bias). Similar biases were evident using a method of adjustment, in which participants positioned objects along a horizontal axis, and in free choice photographs, in which participants were asked to take 'the most aesthetically pleasing picture' they could of everyday objects. The results are discussed as affirming the power of the center and facing direction in the aesthetic biases viewers bring to their appreciation of framed works of visual art (e.g. Alexander, 2002; Arnheim, 1988).     

Odour-mediated foraging by yellowjacket wasps (Hymenoptera: Vespidae): predation on leks of pheromone-calling Mediterranean fruit fly males (Diptera: Tephritidae)

Predation is probably the most important male mortality factor in insect species with courtship displays that render males performing them conspicuous targets of predators. Sexually active Mediterranean fruit fly males, Ceratitis capitata (Wied.), aggregate in leks, where they participate in agonistic encounters and engage in visual, acoustic and pheromone-calling displays to attract receptive females. The objective of this study was to assess: a) whether sexually displaying C. capitata males in leks inside host and non-host foliage are subject to predation by the most prominent predators yellow-jacket wasps, Vespula germanica (F.), and if so, b) whether olfactory, visual or auditive stimuli are used by foraging wasps in locating male C. capitata prey. Studies were carried out in a citrus orchard and surroundings on the island of Chios, Greece. Observations were conducted using perforated containers hung within mulberry, fig or citrus foliage. Living C. capitata flies of different sex and either mature or immature were placed inside. Our results show that the yellowjacket wasps have learned to associate the presence of sexually active medfly males aggregated in leks with their prey's pheromone (kairomone). Foraging wasps, flying through the crowns of host trees, responded to the odour source of C. capitata male pheromone by approaching from downwind. Even inside dense citrus tree foliage, wasps keyed in on aggregations of pheromone-calling males using olfactory stimuli. Stimuli of visual and acoustic male signalling were only used at close range, after having followed the pheromone plume close to its source. Visual cues played a greater role in directing wasp foraging under more open and exposed host foliage conditions. Odour-based foraging of wasps inside host foliage in the mid-morning hours, when medfly male lekking activities peak, shifted gradually to a more visual-based host fruit patrolling in the afternoons to capture ovipositing and feeding medfly females. On ripe fruit, particularly fig, V. germanica visual prey hunting also included the capture of feeding medfly males, other feeding Diptera, as well as medfly larvae extracted from wasp-made perforations in the fruit.     

North-American Norms for Name Disagreement: Pictorial Stimuli Naming Discrepancies

Pictorial stimuli are commonly used by scientists to explore central processes; including memory, attention, and language. Pictures that have been collected and put into sets for these purposes often contain visual ambiguities that lead to name disagreement amongst subjects. In the present work, we propose new norms which reflect these sources of name disagreement, and we apply this method to two sets of pictures: the Snodgrass and Vanderwart (S&V) set and the Bank of Standardized Stimuli (BOSS). Naming responses of the presented pictures were classified within response categories based on whether they were correct, incorrect, or equivocal. To characterize the naming strategy where an alternative name was being used, responses were further divided into different sub-categories that reflected various sources of name disagreement. Naming strategies were also compared across the two sets of stimuli. Results showed that the pictures of the S&V set and the BOSS were more likely to elicit alternative specific and equivocal names, respectively. It was also found that the use of incorrect names was not significantly different across stimulus sets but that errors were more likely caused by visual ambiguity in the S&V set and by a misuse of names in the BOSS. Norms for name disagreement presented in this paper are useful for subsequent research for their categorization and elucidation of name disagreement that occurs when choosing visual stimuli from one or both stimulus sets. The sources of disagreement should be examined carefully as they help to provide an explanation of errors and inconsistencies of many concepts during picture naming tasks.     

Emotional stimulation alters olfactory sensitivity and odor judgment

Emotions have a strong influence on the perception of visual and auditory stimuli. Only little is known about the relation between emotional stimulation and olfactory functions. The present study investigated the relationship between the presentation of affective pictures, olfactory functions, and sex. Olfactory performance was assessed in 32 subjects (16 male). Olfactory sensitivity was significantly reduced following unpleasant picture presentation for all subjects and following pleasant picture presentation for male subjects only. Pleasantness and intensity ratings of a neutral suprathreshold odor were related to the valence of the pictures: After unpleasant picture presentation, the odor was rated as less pleasant and more intense, whereas viewing positive pictures induced a significant increase in reported odor pleasantness. We conclude that inducing a negative emotional state reduces olfactory sensitivity. A relation to functional deviations within the primary olfactory cortices is discussed.     

Analysis of event-related potentials to verbal stimuli in healthy subjects and schizophrenia patients

In 30 healthy subjects and 32 patients after the first episode of schizophrenia 19 channel-EEG was recorded during visual presentation of a random sequence of words and pseudo-words. In the first series of the experiments, subjects had to read the presented verbal stimuli, in the second series they had to press a button when seeing a word, and in the third series they were instructed to press the button when seeing a pseudo-word. We studied components N170, P300 and N400. In the group of healthy subjects, the amplitude of N170 increased to words in the situation of their relevance, which corresponds to the "recognition potential", whereas in the group of patients, the amplitude of N170 increased to pseudo-words when they were relevant. So it was a paradoxical response. The amplitude of the ERP later waves (P300 and N400) in the group of schizophrenic patients was smaller and the relevance effect was impaired when the target stimuli were pseudo-words. However, the incongruity effect consisting in an increase in N400 amplitude to a non-target stimulus remained intact in patients.     

Functionally independent components of early event-related potentials in a visual spatial attention task.

Spatial visual attention modulates the first negative-going deflection in the human averaged event-related potential (ERP) in response to visual target and non-target stimuli (the N1 complex). Here we demonstrate a decomposition of N1 into functionally independent subcomponents with functionally distinct relations to task and stimulus conditions. ERPs were collected from 20 subjects in response to visual target and non-target stimuli presented at five attended and non-attended screen locations. Independent component analysis, a new method for blind source separation, was trained simultaneously on 500 ms grand average responses from all 25 stimulus-attention conditions and decomposed the non-target N1 complexes into five spatially fixed, temporally independent and physiologically plausible components. Activity of an early, laterally symmetrical component pair (N1aR and N1aL) was evoked by the left and right visual field stimuli, respectively. Component N1aR peaked ca. 9 ms earlier than N1aL. Central stimuli evoked both components with the same peak latency difference, producing a bilateral scalp distribution. The amplitudes of these components were no reliably augmented by spatial attention. Stimuli in the right visual field evoked activity in a spatio-temporally overlapping bilateral component (N1b) that peaked at ca. 180 ms and was strongly enhanced by attention. Stimuli presented at unattended locations evoked a fourth component (P2a) peaking near 240 ms. A fifth component (P3f) was evoked only by targets presented in either visual field. The distinct response patterns of these components across the array of stimulus and attention conditions suggest that they reflect activity in functionally independent brain systems involved in processing attended and unattended visuospatial events.     

Summation of perceptual cues in natural visual scenes

Natural visual scenes are rich in information, and any neural system analysing them must piece together the many messages from large arrays of diverse feature detectors. It is known how threshold detection of compound visual stimuli (sinusoidal gratings) is determined by their components' thresholds. We investigate whether similar combination rules apply to the perception of the complex and suprathreshold visual elements in naturalistic visual images. Observers gave magnitude estimations (ratings) of the perceived differences between pairs of images made from photographs of natural scenes. Images in some pairs differed along one stimulus dimension such as object colour, location, size or blur. But, for other image pairs, there were composite differences along two dimensions (e.g. both colour and object-location might change). We examined whether the ratings for such composite pairs could be predicted from the two ratings for the respective pairs in which only one stimulus dimension had changed. We found a pooling relationship similar to that proposed for simple stimuli: Minkowski summation with exponent 2.84 yielded the best predictive power (r=0.96), an exponent similar to that generally reported for compound grating detection. This suggests that theories based on detecting simple stimuli can encompass visual processing of complex, suprathreshold stimuli.     

Olfactory and visual stimuli used in orientation to conifer seedlings by the pine weevil, Hylobius abietis

Abstract. The influence of noncontact plant cues is investigated on the likelihood that individual conifer seedlings will be found by walking adults of the pine weevil, Hylobius abietis, in the field. Traps with solely odour or solely visual stimuli catch significantly more weevils than stimulus-free traps, and traps with the combination of odour and visual stimuli catch more weevils than traps with odour or visual stimuli alone. There is essentially an additive effect between odour and visual stimuli. The reactions to odour and visual stimuli are similar for three phases of the pine weevil's life cycle associated with three ages of clear-cuttings (i.e. sites where all trees have been harvested). Visual stimuli appear to be at least as important as odour for the pine weevil in finding an undamaged conifer seedling.     

Stimulus dependency of object-evoked responses in human visual cortex: an inverse problem for category specificity

Many studies have linked the processing of different object categories to specific event-related potentials (ERPs) such as the face-specific N170. Despite reports showing that object-related ERPs are influenced by visual stimulus features, there is consensus that these components primarily reflect categorical aspects of the stimuli. Here, we re-investigated this idea by systematically measuring the effects of visual feature manipulations on ERP responses elicited by both structure-from-motion (SFM)-defined and luminance-defined object stimuli. SFM objects elicited a novel component at 200-250 ms (N250) over parietal and posterior temporal sites. We found, however, that the N250 amplitude was unaffected by restructuring SFM stimuli into meaningless objects based on identical visual cues. This suggests that this N250 peak was not uniquely linked to categorical aspects of the objects, but is strongly determined by visual stimulus features. We provide strong support for this hypothesis by parametrically manipulating the depth range of both SFM- and luminance-defined object stimuli and showing that the N250 evoked by SFM stimuli as well as the well-known N170 to static faces were sensitive to this manipulation. Importantly, this effect could not be attributed to compromised object categorization in low depth stimuli, confirming a strong impact of visual stimulus features on object-related ERP signals. As ERP components linked with visual categorical object perception are likely determined by multiple stimulus features, this creates an interesting inverse problem when deriving specific perceptual processes from variations in ERP components.     

Role of the Frontal Cortical Areas in the Analysis of Visual Stimuli at Conscious and Unconscious Levels

Event-related potentials (ERP) in response to complex target stimuli, which consisted of a central recognizable picture and a lateral masked image (analyzed at the unconscious level) were recorded in adult subjects and seven-year-old children. ERP components N200, N300, and P400/N400 had different topography and were differently pronounced in adults and children. In adult subjects, the N200 component that reflects the processing of a sensory stimulus was recorded in the temporo-parieto-occipital and occipital areas. In children, N200 was recorded in the caudal regions and the frontal areas of the cortex. Analysis of different waveforms obtained by subtraction of the ERP to the central stimulus from the ERP to the complex stimulus showed that unconscious stimulus processing in adult subjects is not reflected in the ERP structure. In children, an unconsciously processed image incorporated into a complex stimulus evokes processing negativity in the occipital and frontal cortical areas. Comparison of ERP in groups of children divided by their reflectivity/impulsivity showed that, predominantly, the left frontal area is involved in image analysis at the unconscious level in reflective children and, predominantly, the right frontal area participates in unconscious image analysis in impulsive children. It is suggested that the perfection of the visual recognition of a target stimulus, which contains additional unconsciously processed information, consists in growth of the involvement of the left-hemispheric mechanisms (with respective growth of significance of the left-hemispheric mechanisms) and in a decrease in the role of the frontal areas in analysis of sensory information.