Falling Out of Time: Enhanced Memory for Scenes Presented at Behaviorally Irrelevant Points in Time in Posttraumatic Stress Disorder (PTSD)

Spontaneous encoding of the visual environment depends on the behavioral relevance of the task performed simultaneously. If participants identify target letters or auditory tones while viewing a series of briefly presented natural and urban scenes, they demonstrate effective scene recognition only when a target, but not a behaviorally irrelevant distractor, appears together with the scene. Here, we show that individuals with posttraumatic stress disorder (PTSD), who witnessed the red sludge disaster in Hungary, show the opposite pattern of performance: enhanced recognition of scenes presented together with distractors and deficient recognition of scenes presented with targets. The recognition of trauma-related and neutral scenes was not different in individuals with PTSD. We found a positive correlation between memory for scenes presented with auditory distractors and re-experiencing symptoms (memory intrusions and flashbacks). These results suggest that abnormal encoding of visual scenes at behaviorally irrelevant events might be associated with intrusive experiences by disrupting the flow of time.     

Deployment of Spatial Attention towards Locations in Memory Representations. An EEG Study

Recalling information from visual short-term memory (VSTM) involves the same neural mechanisms as attending to an actually perceived scene. In particular, retrieval from VSTM has been associated with orienting of visual attention towards a location within a spatially-organized memory representation. However, an open question concerns whether spatial attention is also recruited during VSTM retrieval even when performing the task does not require access to spatial coordinates of items in the memorized scene. The present study combined a visual search task with a modified, delayed central probe protocol, together with EEG analysis, to answer this question. We found a temporal contralateral negativity (TCN) elicited by a centrally presented go-signal which was spatially uninformative and featurally unrelated to the search target and informed participants only about a response key that they had to press to indicate a prepared target-present vs. -absent decision. This lateralization during VSTM retrieval (TCN) provides strong evidence of a shift of attention towards the target location in the memory representation, which occurred despite the fact that the present task required no spatial (or featural) information from the search to be encoded, maintained, and retrieved to produce the correct response and that the go-signal did not itself specify any information relating to the location and defining feature of the target.     

Do Simultaneously Viewed Objects Influence Scene Recognition Individually or as Groups? Two Perceptual Studies

The ability to quickly categorize visual scenes is critical to daily life, allowing us to identify our whereabouts and to navigate from one place to another. Rapid scene categorization relies heavily on the kinds of objects scenes contain; for instance, studies have shown that recognition is less accurate for scenes to which incongruent objects have been added, an effect usually interpreted as evidence of objects'' general capacity to activate semantic networks for scene categories they are statistically associated with. Essentially all real-world scenes contain multiple objects, however, and it is unclear whether scene recognition draws on the scene associations of individual objects or of object groups. To test the hypothesis that scene recognition is steered, at least in part, by associations between object groups and scene categories, we asked observers to categorize briefly-viewed scenes appearing with object pairs that were semantically consistent or inconsistent with the scenes. In line with previous results, scenes were less accurately recognized when viewed with inconsistent versus consistent pairs. To understand whether this reflected individual or group-level object associations, we compared the impact of pairs composed of mutually related versus unrelated objects; i.e., pairs, which, as groups, had clear associations to particular scene categories versus those that did not. Although related and unrelated object pairs equally reduced scene recognition accuracy, unrelated pairs were consistently less capable of drawing erroneous scene judgments towards scene categories associated with their individual objects. This suggests that scene judgments were influenced by the scene associations of object groups, beyond the influence of individual objects. More generally, the fact that unrelated objects were as capable of degrading categorization accuracy as related objects, while less capable of generating specific alternative judgments, indicates that the process by which objects interfere with scene recognition is separate from the one through which they inform it.     

Does Consolidation of Visuospatial Sequence Knowledge Depend on Eye Movements?

In the current study, we assessed whether visuospatial sequence knowledge is retained over 24 hours and whether this retention is dependent on the occurrence of eye movements. Participants performed two sessions of a serial reaction time (SRT) task in which they had to manually react to the identity of a target letter pair presented in one of four locations around a fixation cross. When the letter pair ‘XO’ was presented, a left response had to be given, when the letter pair ‘OX’ was presented, a right response was required. In the Eye Movements (EM) condition, eye movements were necessary to perform the task since the fixation cross and the target were separated by at least 9° visual angle. In the No Eye Movements (NEM) condition, on the other hand, eye movements were minimized by keeping the distance from the fixation cross to the target below 1° visual angle and by limiting the stimulus presentation to 100 ms. Since the target identity changed randomly in both conditions, no manual response sequence was present in the task. However, target location was structured according to a deterministic sequence in both the EM and NEM condition. Learning of the target location sequence was determined at the end of the first session and 24 hours after initial learning. Results indicated that the sequence learning effect in the SRT task diminished, yet remained significant, over the 24 hour interval in both conditions. Importantly, the difference in eye movements had no impact on the transfer of sequence knowledge. These results suggest that the retention of visuospatial sequence knowledge occurs alike, irrespective of whether this knowledge is supported by eye movements or not.     

Dissociation between Dorsal and Ventral Posterior Parietal Cortical Responses to Incidental Changes in Natural Scenes

Background

The posterior parietal cortex (PPC) is thought to interact with the medial temporal lobe (MTL) to support spatial cognition and topographical memory. While the response of medial temporal lobe regions to topographical stimuli has been intensively studied, much less research has focused on the role of PPC and its functional connectivity with the medial temporal lobe.

Methodology/Principle Findings

Here we report a dissociation between dorsal and ventral regions of PPC in response to different types of change in natural scenes using an fMRI adaptation paradigm. During scanning subjects performed an incidental target detection task whilst viewing trial unique sequentially presented pairs of natural scenes, each containing a single prominent object. We observed a dissociation between the superior parietal gyrus and the angular gyrus, with the former showing greater sensitivity to spatial change, and the latter showing greater sensitivity to scene novelty. In addition, we observed that the parahippocampal cortex has increased functional connectivity with the angular gyrus, but not superior parietal gyrus, when subjects view change to the scene content.

Conclusions/Significance

Our findings provide support for proposed dissociations between dorsal and ventral regions of PPC and suggest that the dorsal PPC may support the spatial coding of the visual environment even when this information is incidental to the task at hand. Further, through revealing the differential functional interactions of the SPG and AG with the MTL our results help advance our understanding of how the MTL and PPC cooperate to update representations of the world around us.     

Relational Memory Is Evident in Eye Movement Behavior despite the Use of Subliminal Testing Methods

While it is generally agreed that perception can occur without awareness, there continues to be debate about the type of representational content that is accessible when awareness is minimized or eliminated. Most investigations that have addressed this issue evaluate access to well-learned representations. Far fewer studies have evaluated whether or not associations encountered just once prior to testing might also be accessed and influence behavior. Here, eye movements were used to examine whether or not memory for studied relationships is evident following the presentation of subliminal cues. Participants assigned to experimental or control groups studied scene-face pairs and test trials evaluated implicit and explicit memory for these pairs. Each test trial began with a subliminal scene cue, followed by three visible studied faces. For experimental group participants, one face was the studied associate of the scene (implicit test); for controls none were a match. Subsequently, the display containing a match was presented to both groups, but now it was preceded by a visible scene cue (explicit test). Eye movements were recorded and recognition memory responses were made. Participants in the experimental group looked disproportionately at matching faces on implicit test trials and participants from both groups looked disproportionately at matching faces on explicit test trials, even when that face had not been successfully identified as the associate. Critically, implicit memory-based viewing effects seemed not to depend on residual awareness of subliminal scene cues, as subjective and objective measures indicated that scenes were successfully masked from view. The reported outcomes indicate that memory for studied relationships can be expressed in eye movement behavior without awareness.     

The neural fate of consciously perceived and missed events in the attentional blink

Cognitive models of attention propose that visual perception is a product of two stages of visual processing: early operations permit rapid initial categorization of the visual world, while later attention-demanding capacity-limited stages are necessary for the conscious report of the stimuli. Here we used the attentional blink paradigm and fMRI to neurally distinguish these two stages of vision. Subjects detected a face target and a scene target presented rapidly among distractors at fixation. Although the second, scene target frequently went undetected by the subjects, it nonetheless activated regions of the medial temporal cortex involved in high-level scene representations, the parahippocampal place area (PPA). This PPA activation was amplified when the stimulus was consciously perceived. By contrast, the frontal cortex was activated only when scenes were successfully reported. These results suggest that medial temporal cortex permits rapid categorization of the visual input, while the frontal cortex is part of a capacity-limited attentional bottleneck to conscious report.     

Recognition Memory for Colored and Black-and-White Scenes in Normal and Color Deficient Observers (Dichromats)

Color deficient (dichromat) and normal observers’ recognition memory for colored and black-and-white natural scenes was evaluated through several parameters: the rate of recognition, discrimination (A’), response bias (B”D), response confidence, and the proportion of conscious recollections (Remember responses) among hits. At the encoding phase, 36 images of natural scenes were each presented for 1 sec. Half of the images were shown in color and half in black-and-white. At the recognition phase, these 36 pictures were intermixed with 36 new images. The participants’ task was to indicate whether an image had been presented or not at the encoding phase, to rate their level of confidence in his her/his response, and in the case of a positive response, to classify the response as a Remember, a Know or a Guess response. Results indicated that accuracy, response discrimination, response bias and confidence ratings were higher for colored than for black-and-white images; this advantage for colored images was similar in both groups of participants. Rates of Remember responses were not higher for colored images than for black-and-white ones, whatever the group. However, interestingly, Remember responses were significantly more often based on color information for colored than for black-and-white images in normal observers only, not in dichromats.     

Animal Detection Precedes Access to Scene Category

The processes underlying object recognition are fundamental for the understanding of visual perception. Humans can recognize many objects rapidly even in complex scenes, a task that still presents major challenges for computer vision systems. A common experimental demonstration of this ability is the rapid animal detection protocol, where human participants earliest responses to report the presence/absence of animals in natural scenes are observed at 250–270 ms latencies. One of the hypotheses to account for such speed is that people would not actually recognize an animal per se, but rather base their decision on global scene statistics. These global statistics (also referred to as spatial envelope or gist) have been shown to be computationally easy to process and could thus be used as a proxy for coarse object recognition. Here, using a saccadic choice task, which allows us to investigate a previously inaccessible temporal window of visual processing, we showed that animal – but not vehicle – detection clearly precedes scene categorization. This asynchrony is in addition validated by a late contextual modulation of animal detection, starting simultaneously with the availability of scene category. Interestingly, the advantage for animal over scene categorization is in opposition to the results of simulations using standard computational models. Taken together, these results challenge the idea that rapid animal detection might be based on early access of global scene statistics, and rather suggests a process based on the extraction of specific local complex features that might be hardwired in the visual system.     

Reward-motivated learning: mesolimbic activation precedes memory formation

We examined anticipatory mechanisms of reward-motivated memory formation using event-related FMRI. In a monetary incentive encoding task, cues signaled high- or low-value reward for memorizing an upcoming scene. When tested 24 hr postscan, subjects were significantly more likely to remember scenes that followed cues for high-value rather than low-value reward. A monetary incentive delay task independently localized regions responsive to reward anticipation. In the encoding task, high-reward cues preceding remembered but not forgotten scenes activated the ventral tegmental area, nucleus accumbens, and hippocampus. Across subjects, greater activation in these regions predicted superior memory performance. Within subject, increased correlation between the hippocampus and ventral tegmental area was associated with enhanced long-term memory for the subsequent scene. These findings demonstrate that brain activation preceding stimulus encoding can predict declarative memory formation. The findings are consistent with the hypothesis that reward motivation promotes memory formation via dopamine release in the hippocampus prior to learning.     

The Processing of Object Identity Information by Women and Men

The study examined whether women excel at tasks which require processing the identity of objects information as has been suggested in the context of the well-known object location memory task. In a computer-simulated task, university students were shown simulated indoor and outdoor house scenes. After studying a scene the students were presented with two images. One was the original image and the other a modified version in which one object was either rotated by ninety degrees or substituted with a similar looking object. The participants were asked to indicate the original image. The main finding was that no sex effect was obtained in this task. The female and male students did not differ on a verbal ability test, and their 2D:4D ratios were found to be comparable.     

Overt attention and context factors: the impact of repeated presentations, image type, and individual motivation

The present study investigated the dynamic of the attention focus during observation of different categories of complex scenes and simultaneous consideration of individuals' memory and motivational state. We repeatedly presented four types of complex visual scenes in a pseudo-randomized order and recorded eye movements. Subjects were divided into groups according to their motivational disposition in terms of action orientation and individual rating of scene interest.Statistical analysis of eye-tracking data revealed that the attention focus successively became locally expressed by increasing fixation duration; decreasing saccade length, saccade frequency, and single subject's fixation distribution over images; and increasing inter-subject variance of fixation distributions. The validity of these results was supported by verbal reports. This general tendency was weaker for the group of subjects who rated the image set as interesting as compared to the other group. Additionally, effects were partly mediated by subjects' motivational disposition. Finally, we found a generally strong impact of image type on eye movement parameters. We conclude that motivational tendencies linked to personality as well as individual preferences significantly affected viewing behaviour. Hence, it is important and fruitful to consider inter-individual differences on the level of motivation and personality traits within investigations of attention processes. We demonstrate that future studies on memory's impact on overt attention have to deal appropriately with several aspects that had been out of the research focus until now.     

Attention-memory interactions in scene perception

The perception of natural scenes relies on the integration of pre-existing knowledge with the immediate results of attentional processing, and what can be remembered from a scene depends in turn on how that scene is perceived and understood. However, there are conflicting results in the literature as to whether people are more likely to remember those objects that are consistent with the scene or those that are not. Moreover, whether any discrepancy between the likelihood of remembering schema-consistent or schema-inconsistent objects should be attributed to the schematic effects on attention or on memory remains unclear. To address this issue, the current study attempted to directly manipulate attention allocation by requiring participants to look at (i) schema-consistent objects, (ii) schema-inconsistent objects, or (iii) to share attention equally across both. Regardless of the differential allocation of attention or object fixation, schema-consistent objects were better recalled whereas recognition was independent of schema-consistency, but depended on task instruction. These results suggest that attention is important both for remembering low-level object properties, and information whose retrieval is not supported by the currently active schema. Specific knowledge of the scenes being viewed can result in the recall of non-fixated objects, but without such knowledge attention is required to encode sufficient detail for subsequent recognition. Our results demonstrate therefore that attention is not critical for the retrieval of objects that are consistent with a scene's schematic content.     

Associative Processing Is Inherent in Scene Perception

How are complex visual entities such as scenes represented in the human brain? More concretely, along what visual and semantic dimensions are scenes encoded in memory? One hypothesis is that global spatial properties provide a basis for categorizing the neural response patterns arising from scenes. In contrast, non-spatial properties, such as single objects, also account for variance in neural responses. The list of critical scene dimensions has continued to grow—sometimes in a contradictory manner—coming to encompass properties such as geometric layout, big/small, crowded/sparse, and three-dimensionality. We demonstrate that these dimensions may be better understood within the more general framework of associative properties. That is, across both the perceptual and semantic domains, features of scene representations are related to one another through learned associations. Critically, the components of such associations are consistent with the dimensions that are typically invoked to account for scene understanding and its neural bases. Using fMRI, we show that non-scene stimuli displaying novel associations across identities or locations recruit putatively scene-selective regions of the human brain (the parahippocampal/lingual region, the retrosplenial complex, and the transverse occipital sulcus/occipital place area). Moreover, we find that the voxel-wise neural patterns arising from these associations are significantly correlated with the neural patterns arising from everyday scenes providing critical evidence whether the same encoding principals underlie both types of processing. These neuroimaging results provide evidence for the hypothesis that the neural representation of scenes is better understood within the broader theoretical framework of associative processing. In addition, the results demonstrate a division of labor that arises across scene-selective regions when processing associations and scenes providing better understanding of the functional roles of each region within the cortical network that mediates scene processing.     

The Power of the Picture: How Narrative Film Captures Attention and Disrupts Goal Pursuit

Narrative transportation is described as a state of detachment that arises when one becomes immersed in the narrative of a story. Participants viewed either an intact version of an engaging 20 min film, “Bang You’re Dead!,” (1961) by Alfred Hitchcock (contiguous condition), or a version of the same film with scenes presented out of order (noncontiguous condition). In this latter condition, the individual scenes were intact but were presented out of chronological order. Participants were told a cover story that we were interested in the amount of gun violence depicted in films. Both groups were given the goal to remember to lift their hand every time they heard the word “gun” spoken during the film. Results revealed that participants were significantly less likely to remember to execute their goal in the contiguous condition, presumably because this narrative transported viewers’ attention and thereby “hijacked” processing resources away from internal goals.     

Modelling human visual navigation using multi-view scene reconstruction

It is often assumed that humans generate a 3D reconstruction of the environment, either in egocentric or world-based coordinates, but the steps involved are unknown. Here, we propose two reconstruction-based models, evaluated using data from two tasks in immersive virtual reality. We model the observer’s prediction of landmark location based on standard photogrammetric methods and then combine location predictions to compute likelihood maps of navigation behaviour. In one model, each scene point is treated independently in the reconstruction; in the other, the pertinent variable is the spatial relationship between pairs of points. Participants viewed a simple environment from one location, were transported (virtually) to another part of the scene and were asked to navigate back. Error distributions varied substantially with changes in scene layout; we compared these directly with the likelihood maps to quantify the success of the models. We also measured error distributions when participants manipulated the location of a landmark to match the preceding interval, providing a direct test of the landmark-location stage of the navigation models. Models such as this, which start with scenes and end with a probabilistic prediction of behaviour, are likely to be increasingly useful for understanding 3D vision.     

Task attention facilitates learning of task-irrelevant stimuli

Attention plays a fundamental role in visual learning and memory. One highly established principle of visual attention is that the harder a central task is, the more attentional resources are used to perform the task and the smaller amount of attention is allocated to peripheral processing because of limited attention capacity. Here we show that this principle holds true in a dual-task setting but not in a paradigm of task-irrelevant perceptual learning. In Experiment 1, eight participants were asked to identify either bright or dim number targets at the screen center and to remember concurrently presented scene backgrounds. Their recognition performances for scenes paired with dim/hard targets were worse than those for scenes paired with bright/easy targets. In Experiment 2, eight participants were asked to identify either bright or dim letter targets at the screen center while a task-irrelevant coherent motion was concurrently presented in the background. After five days of training on letter identification, participants improved their motion sensitivity to the direction paired with hard/dim targets improved but not to the direction paired with easy/bright targets. Taken together, these results suggest that task-irrelevant stimuli are not subject to the attentional control mechanisms that task-relevant stimuli abide.     

A Dual Task Procedure Combined with Rapid Serial Visual Presentation to Test Attentional Blink for Nontargets

When viewers search for targets in a rapid serial visual presentation (RSVP) stream, if two targets are presented within about 500 msec of each other, the first target may be easy to spot but the second is likely to be missed. This phenomenon of attentional blink (AB) has been widely studied to probe the temporal capacity of attention for detecting visual targets. However, with the typical procedure of AB experiments, it is not possible to examine how the processing of non-target items in RSVP may be affected by attention. This paper describes a novel dual task procedure combined with RSVP to test effects of AB for nontargets at varied stimulus onset asynchronies (SOAs). In an exemplar experiment, a target category was first displayed, followed by a sequence of 8 nouns. If one of the nouns belonged to the target category, participants would respond ‘yes’ at the end of the sequence, otherwise participants would respond ‘no’. Two 2-alternative forced choice memory tasks followed the response to determine if participants remembered the words immediately before or after the target, as well as a random word from another part of the sequence. In a second exemplar experiment, the same design was used, except that 1) the memory task was counterbalanced into two groups with SOAs of either 120 or 240 msec and 2) three memory tasks followed the sequence and tested remembrance for nontarget nouns in the sequence that could be anywhere from 3 items prior the target noun position to 3 items following the target noun position. Representative results from a previously published study demonstrate that our procedure can be used to examine divergent effects of attention that not only enhance targets but also suppress nontargets. Here we show results from a representative participant that replicated the previous finding.      

The Effects of Feature-Based Priming and Visual Working Memory on Oculomotor Capture

Recently, it has been demonstrated that objects held in working memory can influence rapid oculomotor selection. This has been taken as evidence that perceptual salience can be modified by active working memory representations. The goal of the present study was to examine whether these results could also be caused by feature-based priming. In two experiments, participants were asked to saccade to a target line segment of a certain orientation that was presented together with a to-be-ignored distractor. Both objects were given a task-irrelevant color that varied per trial. In a secondary task, a color had to be memorized, and that color could either match the color of the target, match the color of the distractor, or it did not match the color of any of the objects in the search task. The memory task was completed either after the search task (Experiment 1), or before it (Experiment 2). The results showed that in both experiments the memorized color biased oculomotor selection. Eye movements were more frequently drawn towards objects that matched the memorized color, irrespective of whether the memory task was completed after (Experiment 1) or before (Experiment 2) the search task. This bias was particularly prevalent in short-latency saccades. The results show that early oculomotor selection performance is not only affected by properties that are actively maintained in working memory but also by those previously memorized. Both working memory and feature priming can cause early biases in oculomotor selection.     

Experience shapes the utility of natural statistics for perceptual contour integration

Segmenting meaningful targets from cluttered scenes is a fundamental function of the visual system. Evolution and development have been suggested to optimize the brain's solution to this computationally challenging task by tuning the visual system to features that co-occur frequently in natural scenes (e.g., collinear edges) [1, 2, 3]. However, the role of shorter-term experience in shaping the utility of scene statistics remains largely unknown. Here, we ask whether collinearity is a specialized case, or whether the brain can learn to recruit any image regularity for the purpose of target identification. Consistent with long-term optimization for typical scene statistics, observers were better at detecting collinear contours than configurations of elements oriented at orthogonal or acute angles to the contour path. However, training resulted in improved detection of orthogonal contours that lasted for several months, suggesting retuning rather than transient changes of visual sensitivity. Improvement was also observed for acute contours but only after longer training. These results demonstrate that the brain flexibly exploits image regularities and learns to use discontinuities typically associated with surface boundaries (orthogonal, acute alignments) for contour linking and target identification. Thus, short-term experience in adulthood shapes the interpretation of scenes by assigning new statistical utility to image regularities.