The purity of auditory memory

Recent evidence from experiments on immediate memory indicates unambiguously that silent speech perception can produce typically 'auditory' effects while there is either active or passive mouthing of the relevant articulatory gestures. This result falsifies previous theories of auditory sensory memory (pre-categorical acoustic store) that insisted on external auditory stimulation as indispensable for access to the system. A resolution is proposed that leaves the properties of pre-categorical acoustic store much as they were assumed to be before but adds the possibility that visual information can affect the selection of auditory features in a pre-categorical stage of speech perception. In common terms, a speaker's facial gestures (or one's own) can influence auditory experience independently of determining what it was that was said. Some results in word perception that encourage this view are discussed.     

Are There Multiple Visual Short-Term Memory Stores?

Background

Classic work on visual short-term memory (VSTM) suggests that people store a limited amount of items for subsequent report. However, when human observers are cued to shift attention to one item in VSTM during retention, it seems as if there is a much larger representation, which keeps additional items in a more fragile VSTM store. Thus far, it is not clear whether the capacity of this fragile VSTM store indeed exceeds the traditional capacity limits of VSTM. The current experiments address this issue and explore the capacity, stability, and duration of fragile VSTM representations.

Methodology/Principal Findings

We presented cues in a change-detection task either just after off-set of the memory array (iconic-cue), 1,000 ms after off-set of the memory array (retro-cue) or after on-set of the probe array (post-cue). We observed three stages in visual information processing 1) iconic memory with unlimited capacity, 2) a four seconds lasting fragile VSTM store with a capacity that is at least a factor of two higher than 3) the robust and capacity-limited form of VSTM. Iconic memory seemed to depend on the strength of the positive after-image resulting from the memory display and was virtually absent under conditions of isoluminance or when intervening light masks were presented. This suggests that iconic memory is driven by prolonged retinal activation beyond stimulus duration. Fragile VSTM representations were not affected by light masks, but were completely overwritten by irrelevant pattern masks that spatially overlapped the memory array.

Conclusions/Significance

We find that immediately after a stimulus has disappeared from view, subjects can still access information from iconic memory because they can see an after-image of the display. After that period, human observers can still access a substantial, but somewhat more limited amount of information from a high-capacity, but fragile VSTM that is overwritten when new items are presented to the eyes. What is left after that is the traditional VSTM store, with a limit of about four objects. We conclude that human observers store more sustained representations than is evident from standard change detection tasks and that these representations can be accessed at will.     

A model of localized iconic memory, replicable and associative, using time sharing and 3 neuronal communication modes

A model is proposed for the functioning of an iconic memory involving several layers of neurones. A small group of neurones in one layer project their terminations over the terminations of a single neurone of the superior layer. According to the communication mode (emission or reception), a neurone in one layer can memorize the state of the terminations of a neurone of the superior layer, or impose on the latter the state of its own terminations. In the comparison mode, an emitting neurone compares its state to another emitting neurone and, in case of sufficient similarity, switches to the reception mode (associative recall). The first layer, corresponding to short-term memory, communicates with the cells involved in the representation of the perceived image. This model makes possible the establishment of a correspondence between a percept and a neurone, the replication of memorized configurations, the restructuration of memory and, starting with a percept or a memorized item, the integral associative recall of all similar memorized items.     

Landmarks help guide attention during visual search

Using a novel visual search paradigm McCarley et al. (2003) concluded that the oculomotor system keeps a history of 3-4 previously attended objects. However, their displays were visually sparse, denying participants structural information which might be used during normal search. This might have underestimated memory capacity. To examine this possibility, we included landmarks in the same search paradigm. Previously examined items were re-examined less frequently when landmarks were present compared to when they were absent. Results indicate that objects in the environment that share no features with search items are used as external support to aid memory in guiding visual search.     

Bottlenecks of Motion Processing during a Visual Glance: The Leaky Flask Model

Where do the bottlenecks for information and attention lie when our visual system processes incoming stimuli? The human visual system encodes the incoming stimulus and transfers its contents into three major memory systems with increasing time scales, viz., sensory (or iconic) memory, visual short-term memory (VSTM), and long-term memory (LTM). It is commonly believed that the major bottleneck of information processing resides in VSTM. In contrast to this view, we show major bottlenecks for motion processing prior to VSTM. In the first experiment, we examined bottlenecks at the stimulus encoding stage through a partial-report technique by delivering the cue immediately at the end of the stimulus presentation. In the second experiment, we varied the cue delay to investigate sensory memory and VSTM. Performance decayed exponentially as a function of cue delay and we used the time-constant of the exponential-decay to demarcate sensory memory from VSTM. We then decomposed performance in terms of quality and quantity measures to analyze bottlenecks along these dimensions. In terms of the quality of information, two thirds to three quarters of the motion-processing bottleneck occurs in stimulus encoding rather than memory stages. In terms of the quantity of information, the motion-processing bottleneck is distributed, with the stimulus-encoding stage accounting for one third of the bottleneck. The bottleneck for the stimulus-encoding stage is dominated by the selection compared to the filtering function of attention. We also found that the filtering function of attention is operating mainly at the sensory memory stage in a specific manner, i.e., influencing only quantity and sparing quality. These results provide a novel and more complete understanding of information processing and storage bottlenecks for motion processing.     

A model of the ventral visual system based on temporal stability and local memory

The cerebral cortex is a remarkably homogeneous structure suggesting a rather generic computational machinery. Indeed, under a variety of conditions, functions attributed to specialized areas can be supported by other regions. However, a host of studies have laid out an ever more detailed map of functional cortical areas. This leaves us with the puzzle of whether different cortical areas are intrinsically specialized, or whether they differ mostly by their position in the processing hierarchy and their inputs but apply the same computational principles. Here we show that the computational principle of optimal stability of sensory representations combined with local memory gives rise to a hierarchy of processing stages resembling the ventral visual pathway when it is exposed to continuous natural stimuli. Early processing stages show receptive fields similar to those observed in the primary visual cortex. Subsequent stages are selective for increasingly complex configurations of local features, as observed in higher visual areas. The last stage of the model displays place fields as observed in entorhinal cortex and hippocampus. The results suggest that functionally heterogeneous cortical areas can be generated by only a few computational principles and highlight the importance of the variability of the input signals in forming functional specialization.     

Dealing with Illumination in Visual Scenes: Effects of Ageing and Alzheimer's Disease

Various visual functions decline in ageing and even more so in patients with Alzheimer''s disease (AD). Here we investigated whether the complex visual processes involved in ignoring illumination-related variability (specifically, cast shadows) in visual scenes may also be compromised. Participants searched for a discrepant target among items which appeared as posts with shadows cast by light-from-above when upright, but as angled objects when inverted. As in earlier reports, young participants gave slower responses with upright than inverted displays when the shadow-like part was dark but not white (control condition). This is consistent with visual processing mechanisms making shadows difficult to perceive, presumably to assist object recognition under varied illumination. Contrary to predictions, this interaction of “shadow” colour with item orientation was maintained in healthy older and AD groups. Thus, the processing mechanisms which assist complex light-independent object identification appear to be robust to the effects of both ageing and AD. Importantly, this means that the complexity of a function does not necessarily determine its vulnerability to age- or AD-related decline.We also report slower responses to dark than light “shadows” of either orientation in both ageing and AD, in keeping with increasing light scatter in the ageing eye. Rather curiously, AD patients showed further slowed responses to “shadows” of either colour at the bottom than the top of items as if they applied shadow-specific rules to non-shadow conditions. This suggests that in AD, shadow-processing mechanisms, while preserved, might be applied in a less selective way.     

Effective visual working memory capacity: an emergent effect from the neural dynamics in an attractor network

The study of working memory capacity is of outmost importance in cognitive psychology as working memory is at the basis of general cognitive function. Although the working memory capacity limit has been thoroughly studied, its origin still remains a matter of strong debate. Only recently has the role of visual saliency in modulating working memory storage capacity been assessed experimentally and proved to provide valuable insights into working memory function. In the computational arena, attractor networks have successfully accounted for psychophysical and neurophysiological data in numerous working memory tasks given their ability to produce a sustained elevated firing rate during a delay period. Here we investigate the mechanisms underlying working memory capacity by means of a biophysically-realistic attractor network with spiking neurons while accounting for two recent experimental observations: 1) the presence of a visually salient item reduces the number of items that can be held in working memory, and 2) visually salient items are commonly kept in memory at the cost of not keeping as many non-salient items.OUR MODEL SUGGESTS THAT WORKING MEMORY CAPACITY IS DETERMINED BY TWO FUNDAMENTAL PROCESSES: encoding of visual items into working memory and maintenance of the encoded items upon their removal from the visual display. While maintenance critically depends on the constraints that lateral inhibition imposes to the mnemonic activity, encoding is limited by the ability of the stimulated neural assemblies to reach a sufficiently high level of excitation, a process governed by the dynamics of competition and cooperation among neuronal pools. Encoding is therefore contingent upon the visual working memory task and has led us to introduce the concept of effective working memory capacity (eWMC) in contrast to the maximal upper capacity limit only reached under ideal conditions.     

Auditory short-term memory behaves like visual short-term memory

Are the information processing steps that support short-term sensory memory common to all the senses? Systematic, psychophysical comparison requires identical experimental paradigms and comparable stimuli, which can be challenging to obtain across modalities. Participants performed a recognition memory task with auditory and visual stimuli that were comparable in complexity and in their neural representations at early stages of cortical processing. The visual stimuli were static and moving Gaussian-windowed, oriented, sinusoidal gratings (Gabor patches); the auditory stimuli were broadband sounds whose frequency content varied sinusoidally over time (moving ripples). Parallel effects on recognition memory were seen for number of items to be remembered, retention interval, and serial position. Further, regardless of modality, predicting an item's recognizability requires taking account of (1) the probe's similarity to the remembered list items (summed similarity), and (2) the similarity between the items in memory (inter-item homogeneity). A model incorporating both these factors gives a good fit to recognition memory data for auditory as well as visual stimuli. In addition, we present the first demonstration of the orthogonality of summed similarity and inter-item homogeneity effects. These data imply that auditory and visual representations undergo very similar transformations while they are encoded and retrieved from memory.     

Bilateral field advantage in visual enumeration

A number of recent studies have demonstrated superior visual processing when the information is distributed across the left and right visual fields than if the information is presented in a single hemifield (the bilateral field advantage). This effect is thought to reflect independent attentional resources in the two hemifields and the capacity of the neural responses to the left and right hemifields to process visual information in parallel. Here, we examined whether a bilateral field advantage can also be observed in a high-level visual task that requires the information from both hemifields to be combined. To this end, we used a visual enumeration task--a task that requires the assimilation of separate visual items into a single quantity--where the to-be-enumerated items were either presented in one hemifield or distributed between the two visual fields. We found that enumerating large number (>4 items), but not small number (<4 items), exhibited the bilateral field advantage: enumeration was more accurate when the visual items were split between the left and right hemifields than when they were all presented within the same hemifield. Control experiments further showed that this effect could not be attributed to a horizontal alignment advantage of the items in the visual field, or to a retinal stimulation difference between the unilateral and bilateral displays. These results suggest that a bilateral field advantage can arise when the visual task involves inter-hemispheric integration. This is in line with previous research and theory indicating that, when the visual task is attentionally demanding, parallel processing by the neural responses to the left and right hemifields can expand the capacity of visual information processing.     

The persistences of vision

Human observers continue to experience a visual stimulus for some time after the offset that stimulus. The neural activity evoked by a visual stimulus continues for some time after its offset. The information extracted from a visual stimulus continues to be registered in a visual form of memory ('iconic memory') for some time after its offset. We may thus distinguish three distinct senses in which a visual stimulus may be said to persist after its physical offset: there is phenomenological persistence, neural persistence and informational persistence. Various assumptions have been made about the relation between these forms of visual persistence. The most frequent assumption is that they correspond simply to three different methods for studying a single entity. Detailed consideration of what is known about the properties of these three forms of persistence suggests, however, that this assumption is not correct. It can reasonably be proposed that visible persistence is the phenomenological correlate of neural persistence occurring at various stages of the visual system: photoreceptors, ganglion cells and the stereopsis system. Iconic memory on the other hand, does not correspond to visible persistence, nor to neural persistence in any stage of the visual system. Recent work, in fact, suggests that iconic memory is a property of some relatively late stage in the visual information-processing system, rather than being a peripheral sensory buffer store. This suggestion raises some fundamental theoretical issues concerning the psychology of visual perception, issues with which cognitive psychology has yet to come to grips.     

On recency and echoic memory

In short-term memory, the tendency for the last few (recency) items from a verbal sequence to be increasingly well recalled is more pronounced if the items are spoken rather than written. This auditory recency advantage has been quite generally attributed to echoic memory, on the grounds that in the auditory, but not the visual, mode, sensory memory persists just long enough to supplement recall of the most recent items. This view no longer seems tenable. There are now several studies showing that an auditory recency advantage occurs not only in long-term memory, but under conditions in which it cannot possibly be attributed to echoic memory. Also, similar recency phenomena have been discovered in short-term memory when the items are lip-read, or presented in sign-language, rather than heard. This article provides a partial review of these studies, taking a broad theoretical position from which these particular recency phenomena are approached as possible exceptions, to a general theory according to which recency is due to temporal distinctiveness. Much of the fresh evidence reviewed is of a somewhat preliminary nature and it is as yet unexplained by any theory of memory. The need for additional, converging experimental tests is obvious; so too is the need for further theoretical development. Several alternative theoretical resolutions are mentioned, including the possibility that enhanced recency may reflect movement, from sequentially occurring stimulus features, and the suggestion that it may be associated with the primary linguistic mode of the individuals concerned. But special weight is attached to the conjecture that all these recency phenomena might be accounted for in terms of distinctiveness or discriminability. On this view, the enhanced recency effects observed with certain modes, including the auditory mode, are attributed to items possessing greater temporal discriminability in those modes.     

Fine-Grained,Local Maps and Coarse,Global Representations Support Human Spatial Working Memory

While sensory processes are tuned to particular features, such as an object''s specific location, color or orientation, visual working memory (vWM) is assumed to store information using representations, which generalize over a feature dimension. Additionally, current vWM models presume that different features or objects are stored independently. On the other hand, configurational effects, when observed, are supposed to mainly reflect encoding strategies. We show that the location of the target, relative to the display center and boundaries, and overall memory load influenced recall precision, indicating that, like sensory processes, capacity limited vWM resources are spatially tuned. When recalling one of three memory items the target distance from the display center was overestimated, similar to the error when only one item was memorized, but its distance from the memory items'' average position was underestimated, showing that not only individual memory items'' position, but also the global configuration of the memory array may be stored. Finally, presenting the non-target items at recall, consequently providing landmarks and configurational information, improved precision and accuracy of target recall. Similarly, when the non-target items were translated at recall, relative to their position in the initial display, a parallel displacement of the recalled target was observed. These findings suggest that fine-grained spatial information in vWM is represented in local maps whose resolution varies with distance from landmarks, such as the display center, while coarse representations are used to store the memory array configuration. Both these representations are updated at the time of recall.     

The role of attentional priority and saliency in determining capacity limits in enumeration and visual working memory

Many common tasks require us to individuate in parallel two or more objects out of a complex scene. Although the mechanisms underlying our abilities to count the number of items, remember the visual properties of objects and to make saccadic eye movements towards targets have been studied separately, each of these tasks require selection of individual objects and shows a capacity limit. Here we show that a common factor--salience--determines the capacity limit in the various tasks. We manipulated bottom-up salience (visual contrast) and top-down salience (task relevance) in enumeration and visual memory tasks. As one item became increasingly salient, the subitizing range was reduced and memory performance for all other less-salient items was decreased. Overall, the pattern of results suggests that our abilities to enumerate and remember small groups of stimuli are grounded in an attentional priority or salience map which represents the location of important items.     

Does consciousness overflow cognitive access? Novel insights from the new phenomenon of attribute amnesia

The moment we open our eyes, we experience a rich and detailed visual world, but the amount of information available to report is rather limited. This dissociation relates to a major debate regarding the nature of visual consciousness. The overflow argument suggests that our conscious experience is quite rich and far beyond what can be reported, standing in sharp contrast to the nooverflow argument that visual consciousness is severely impoverished and limited to what can be reported. In this paper, we systematically reviewed existing evidence in favor of the overflow argument, including studies of several variations of the iconic memory paradigm and the divided attention paradigm, as well as studies of neural correlates of consciousness. Simultaneously,we expounded some critical objections and alternative interpretations to such evidence, as well as some opposing evidence.Finally, we introduced a series of our recent studies based on a striking phenomenon of attribute amnesia, which we believe could provide new insight into the overflow view of visual consciousness.     

Spatial and temporal dynamics of attentional guidance during inefficient visual search

Spotting a prey or a predator is crucial in the natural environment and relies on the ability to extract quickly pertinent visual information. The experimental counterpart of this behavior is visual search (VS) where subjects have to identify a target amongst several distractors. In difficult VS tasks, it has been found that the reaction time (RT) is influenced by salience factors, such as the target-distractor similarity, and this finding is usually regarded as evidence for a guidance of attention by preattentive mechanisms. However, the use of RT measurements, a parameter which depends on multiple factors, allows only very indirect inferences about the underlying attentional mechanisms. The purpose of the present study was to determine the influence of salience factors on attentional guidance during VS, by measuring directly attentional allocation. We studied attention allocation by using a dual covert VS task in subjects who had 1) to detect a target amongst different items and 2) to report letters briefly flashed inside those items at different delays. As predicted, we showed that parallel processes guide attention towards the most relevant item by virtue of both goal-directed and stimulus-driven factors, and we demonstrated that this attentional selection is a prerequisite for target detection. In addition, we show that when the target is characterized by two features (conjunction VS), the goal-directed effects of both features are initially combined into a unique salience value, but at a later stage, grouping phenomena interact with the salience computation, and lead to the selection of a whole group of items. These results, in line with Guided Search Theory, show that efficient and rapid preattentive processes guide attention towards the most salient item, allowing to reduce the number of attentional shifts needed to find the target.     

Cortical mechanisms for trans-saccadic memory and integration of multiple object features

Constructing an internal representation of the world from successive visual fixations, i.e. separated by saccadic eye movements, is known as trans-saccadic perception. Research on trans-saccadic perception (TSP) has been traditionally aimed at resolving the problems of memory capacity and visual integration across saccades. In this paper, we review this literature on TSP with a focus on research showing that egocentric measures of the saccadic eye movement can be used to integrate simple object features across saccades, and that the memory capacity for items retained across saccades, like visual working memory, is restricted to about three to four items. We also review recent transcranial magnetic stimulation experiments which suggest that the right parietal eye field and frontal eye fields play a key functional role in spatial updating of objects in TSP. We conclude by speculating on possible cortical mechanisms for governing egocentric spatial updating of multiple objects in TSP.     

The Focus of Attention in Visual Working Memory: Protection of Focused Representations and Its Individual Variation

Visual working memory can be modulated according to changes in the cued task relevance of maintained items. Here, we investigated the mechanisms underlying this modulation. In particular, we studied the consequences of attentional selection for selected and unselected items, and the role of individual differences in the efficiency with which attention is deployed. To this end, performance in a visual working memory task as well as the CDA/SPCN and the N2pc, ERP components associated with visual working memory and attentional processes, were analysed. Selection during the maintenance stage was manipulated by means of two successively presented retrocues providing spatial information as to which items were most likely to be tested. Results show that attentional selection serves to robustly protect relevant representations in the focus of attention while unselected representations which may become relevant again still remain available. Individuals with larger retrocueing benefits showed higher efficiency of attentional selection, as indicated by the N2pc, and showed stronger maintenance-associated activity (CDA/SPCN). The findings add to converging evidence that focused representations are protected, and highlight the flexibility of visual working memory, in which information can be weighted according its relevance.     

The Role of Visual Processing Speed in Reading Speed Development

A steady increase in reading speed is the hallmark of normal reading acquisition. However, little is known of the influence of visual attention capacity on children''s reading speed. The number of distinct visual elements that can be simultaneously processed at a glance (dubbed the visual attention span), predicts single-word reading speed in both normal reading and dyslexic children. However, the exact processes that account for the relationship between the visual attention span and reading speed remain to be specified. We used the Theory of Visual Attention to estimate visual processing speed and visual short-term memory capacity from a multiple letter report task in eight and nine year old children. The visual attention span and text reading speed were also assessed. Results showed that visual processing speed and visual short term memory capacity predicted the visual attention span. Furthermore, visual processing speed predicted reading speed, but visual short term memory capacity did not. Finally, the visual attention span mediated the effect of visual processing speed on reading speed. These results suggest that visual attention capacity could constrain reading speed in elementary school children.     

Attention and available long-term memory in an activation-based model

The influence of attention on memorizing related items and on available long-term memory (ALTM) was explored, showing that N400 of no-memory items was more negative than that of the memory item. The results of the category comparison task indicated that information processing under attention-driven in WM determined the availability of related long-term memory, i.e., specific content, which was formerly concerned or ignored, yielding different indirect semantic priming effects. These indicate that the orientation of conceptual attention leads the related representations of LTM to diverse activation patterns, supporting the activation-based model.