Is object search mediated by object-based or image-based representations?

Recent research suggests that visually specific memory representations for previously fixated objects are maintained during scene perception. Here we investigate the degree of visual specificity by asking whether the memory representations are image-based or object-based. To that end we measured the effects of object orientation on the time to search for a familiar object from amongst a set of 7 familiar distractors arranged in a circular array. Search times were found to depend on the relative orientations of the target object and the probe object for both familiar and novel objects. This effect was found to be partly an image matching effect but there was also an advantage shown for the object's canonical view for familiar objects. Orientation effects were maintained even when the target object was specified as having unique or similar shape properties relative to the distractors. Participants' eye movements were monitored during two of the experiments. Eye movement patterns revealed selection for object shape and object orientation during the search process. Our findings provide evidence for object representations during search that are detailed and share image-based characteristics with more high-level characteristics from object memory.     

How Lateral Connections and Spiking Dynamics May Separate Multiple Objects Moving Together

Over successive stages, the ventral visual system of the primate brain develops neurons that respond selectively to particular objects or faces with translation, size and view invariance. The powerful neural representations found in Inferotemporal cortex form a remarkably rapid and robust basis for object recognition which belies the difficulties faced by the system when learning in natural visual environments. A central issue in understanding the process of biological object recognition is how these neurons learn to form separate representations of objects from complex visual scenes composed of multiple objects. We show how a one-layer competitive network comprised of ‘spiking’ neurons is able to learn separate transformation-invariant representations (exemplified by one-dimensional translations) of visual objects that are always seen together moving in lock-step, but separated in space. This is achieved by combining ‘Mexican hat’ functional lateral connectivity with cell firing-rate adaptation to temporally segment input representations of competing stimuli through anti-phase oscillations (perceptual cycles). These spiking dynamics are quickly and reliably generated, enabling selective modification of the feed-forward connections to neurons in the next layer through Spike-Time-Dependent Plasticity (STDP), resulting in separate translation-invariant representations of each stimulus. Variations in key properties of the model are investigated with respect to the network’s ability to develop appropriate input representations and subsequently output representations through STDP. Contrary to earlier rate-coded models of this learning process, this work shows how spiking neural networks may learn about more than one stimulus together without suffering from the ‘superposition catastrophe’. We take these results to suggest that spiking dynamics are key to understanding biological visual object recognition.     

Size-Sensitive Perceptual Representations Underlie Visual and Haptic Object Recognition

A variety of similarities between visual and haptic object recognition suggests that the two modalities may share common representations. However, it is unclear whether such common representations preserve low-level perceptual features or whether transfer between vision and haptics is mediated by high-level, abstract representations. Two experiments used a sequential shape-matching task to examine the effects of size changes on unimodal and crossmodal visual and haptic object recognition. Participants felt or saw 3D plastic models of familiar objects. The two objects presented on a trial were either the same size or different sizes and were the same shape or different but similar shapes. Participants were told to ignore size changes and to match on shape alone. In Experiment 1, size changes on same-shape trials impaired performance similarly for both visual-to-visual and haptic-to-haptic shape matching. In Experiment 2, size changes impaired performance on both visual-to-haptic and haptic-to-visual shape matching and there was no interaction between the cost of size changes and direction of transfer. Together the unimodal and crossmodal matching results suggest that the same, size-specific perceptual representations underlie both visual and haptic object recognition, and indicate that crossmodal memory for objects must be at least partly based on common perceptual representations.     

The role of prefrontal cortex in working memory: examining the contents of consciousness.

Working memory enables us to hold in our ''mind''s eye'' the contents of our conscious awareness, even in the absence of sensory input, by maintaining an active representation of information for a brief period of time. In this review we consider the functional organization of the prefrontal cortex and its role in this cognitive process. First, we present evidence from brain-imaging studies that prefrontal cortex shows sustained activity during the delay period of visual working memory tasks, indicating that this cortex maintains on-line representations of stimuli after they are removed from view. We then present evidence for domain specificity within frontal cortex based on the type of information, with object working memory mediated by more ventral frontal regions and spatial working memory mediated by more dorsal frontal regions. We also propose that a second dimension for domain specificity within prefrontal cortex might exist for object working memory on the basis of the type of representation, with analytic representations maintained preferentially in the left hemisphere and image-based representations maintained preferentially in the right hemisphere. Furthermore, we discuss the possibility that there are prefrontal areas brought into play during the monitoring and manipulation of information in working memory in addition to those engaged during the maintenance of this information. Finally, we consider the relationship of prefrontal areas important for working memory, both to posterior visual processing areas and to prefrontal areas associated with long-term memory.     

Intensity generalization: physiology and modelling of a neglected topic

I briefly review empirical data about the generalization of acquired behaviour to novel stimuli, showing that variations in stimulus intensity affect behaviour differently from variations in characteristics such as, for instance, visual shape or sound frequency. I argue that such differences can be seen already in how the sense organs react to changes in intensity compared to changes in other stimulus characteristics. I then evaluate a number of models of generalization with respect to their ability to reproduce intensity generalization. I reach three main conclusions. First, realistic stimulus representations, based on knowledge of the sense organs, are necessary to account for intensity effects. Models employing stimulus representations too remote from the sense organs are unable to reproduce the data. Second, the intuitive notion that generalization is based on similarities between stimuli, possibly modelled as distances in an appropriate representation space, is difficult to reconcile with data about intensity generalization. Third, several simple models, in conjunction with realistic stimulus representations, can account for a wide array of generalization phenomena along both intensity and non-intensity stimulus dimensions. The paper also introduces concepts which may be generally useful to evaluate and compare different models of behaviour.     

Perception of novel changes in a familiar environment by socially‐housed rhesus monkeys

Many animals appear to have a sophisticated spatial representation of their environment. The development of these representations depends on the joint abilities of discriminating novel objects and remembering their locations. Variations of a detection of novelty paradigm were used to determine the nature and limitations of these abilities in rhesus monkeys. Socially‐housed monkeys at two facilities (UMASS Primate Laboratory and the New England Regional Primate Research Center) were exposed to novelty detection tasks using a vertical object grid arranged on a mesh wall of the animals' pens. Monkeys rapidly responded with increased exploration to the replacement of one familiar object with a novel object, to the movement of a familiar object to a novel location, and to the swapping of two familiar objects. However, novelty of object was more salient than novelty of place. In these initial studies, monkeys were given continuous access to the grid, and only one or two changes occurred on a given day. In subsequent studies, the task difficulty was varied either by reducing the length of grid exposure or increasing the number of changed objects/session. Surprisingly, only a reduction in length of exposure markedly affected novelty detecting abilities. Rhesus monkeys clearly possessed the dual novelty detecting abilities. These skills were negatively affected only when monkeys' access to the grid was limited. The procedure employed here provided a convenient way to assess complex cognitive abilities in a group setting. It also relied on rhesus monkeys' inherent attraction to novelty and required only their species‐typical behavior for assessment. Am. J. Primatol. 47:117–131, 1999. © 1999 Wiley‐Liss, Inc.     

Object/Context Specific Memory Deficits following Medial Frontal Cortex Damage in Mice

Recent evidence suggests that the medial prefrontal cortex (MFC) is important for processing contextual information. Here we evaluate the performance of mice with MFC damage in a discrimination task that requires an association between an object and the context in which it was experienced (the object/context mismatch task), as well as a version of the novel object preference task that does not require knowledge of contextual information to resolve. Adult C57/BL6 mice received aspiration lesions of the MFC or control surgery. Upon recovery, mice were tested in the object/context mismatch and novel object preference tasks. The object/context mismatch task involved exposing mice to two different contexts, each of which housed a unique pair of identical objects. After a brief delay, mice were re-exposed to one of the contexts, this time with one object that was congruent with that context and one that was not. Novel object preference was performed within a single context, housing an identical pair of objects. After the initial exposure and following a brief delay, mice were re-exposed to the context, this time housing a familiar and a novel object. Control mice were able to successfully resolve the object/context mismatch and novel object preference discriminations, investigating the incongruent/novel object within each task significantly greater than chance. Mice with MFC damage experienced deficits in the object/context mismatch task but not the novel object preference task. These findings add to a growing body of evidence that demonstrate a critical role for the MFC in contextual information processing.     

Ganzfeld Stimulation or Sleep Enhance Long Term Motor Memory Consolidation Compared to Normal Viewing in Saccadic Adaptation Paradigm

Adaptation of saccade amplitude in response to intra-saccadic target displacement is a type of implicit motor learning which is required to compensate for physiological changes in saccade performance. Once established trials without intra-saccadic target displacement lead to de-adaptation or extinction, which has been attributed either to extra-retinal mechanisms of spatial constancy or to the influence of the stable visual surroundings. Therefore we investigated whether visual deprivation (“Ganzfeld”-stimulation or sleep) can partially maintain this motor learning compared to free viewing of the natural surroundings. Thirty-five healthy volunteers performed two adaptation blocks of 100 inward adaptation trials – interspersed by an extinction block – which were followed by a two-hour break with or without visual deprivation (VD). Using additional adaptation and extinction blocks short and long (4 weeks) term memory of this implicit motor learning were tested. In the short term, motor memory tested immediately after free viewing was superior to adaptation performance after VD. In the long run, however, effects were opposite: motor memory and relearning of adaptation was superior in the VD conditions. This could imply independent mechanisms that underlie the short-term ability of retrieving learned saccadic gain and its long term consolidation. We suggest that subjects mainly rely on visual cues (i.e., retinal error) in the free viewing condition which makes them prone to changes of the visual stimulus in the extinction block. This indicates the role of a stable visual array for resetting adapted saccade amplitudes. In contrast, visual deprivation (GS and sleep), might train subjects to rely on extra-retinal cues, e.g., efference copy or prediction to remap their internal representations of saccade targets, thus leading to better consolidation of saccadic adaptation.     

An instance of viewpoint consistency in pigeon object recognition

Pigeons were trained on a visual discrimination, a task using a TV monitor. Two different types of stimuli appeared as pictures on the TV screen, one was a feeder used in their home cages and the other a coffee mug. One group of the pigeons was trained to peck the screen when the feeder appeared on it, while the other group was trained to peck the screen when the mug appeared. The feeder was considered to be a `familiar object' but the mug an `unfamiliar object' for the subjects. After training, to peck the familiar object, the subjects showed generalization to unusual view pictures of the object, but they did not show such generalization after training to peck the unfamiliar object. These results suggest that view point consistency is limited to familiar objects in pigeons.     

Position specificity of adaptation-related face aftereffects

It has been shown that prolonged exposure to a human face leads to shape-selective visual aftereffects. It seems that these face-specific aftereffects (FAEs) have multiple components, related to the adaptation of earlier and higher level processing of visual stimuli. The largest magnitude of FAE, using long-term adaptation periods, is usually observed at the retinotopic position of the preceding adaptor stimulus. However, FAE is also detected, to a smaller degree, at other retinal positions in a spatially invariant way and this component depends less on the adaptation duration. Several lines of evidences suggest that while the position-specific FAE involves lower level areas of the ventral processing stream, the position-invariant FAE depends on the activation of higher level face-processing areas and the fusiform gyrus in particular. In the present paper, we summarize the available behavioural, electrophysiological and neuroimaging results regarding the spatial selectivity of FAE and discuss their implications for the visual stability of object representations across saccadic eye movements.     

Neuro-robotics study on integrative learning of proactive visual attention and motor behaviors

The current paper proposes a novel model for integrative learning of proactive visual attention and sensory-motor control as inspired by the premotor theory of visual attention. The model is characterized by coupling a slow dynamics network with a fast dynamics network and by inheriting our prior proposed multiple timescales recurrent neural networks model (MTRNN) that may correspond to the fronto-parietal networks in the cortical brains. The neuro-robotics experiments in a task of manipulating multiple objects utilizing the proposed model demonstrated that some degrees of generalization in terms of position and object size variation can be achieved by organizing seamless integration of the proactive object-related visual attention and the related sensory-motor control into a set of action primitives in the distributed neural activities appearing in the fast dynamics network. It was also shown that such action primitives can be combined in compositional ways in acquiring novel actions in the slow dynamics network. The experimental results presented substantiate the premotor theory of visual attention.     

Climate interacts with anthropogenic drivers to determine extirpation dynamics

Theoretical studies suggest that the dynamics of a species’ range during a period of climate change depends upon the existence and interplay of various ecological and evolutionary processes. Here we tested how anthropogenic pressures contribute to climate‐mediated extirpation patterns of 32 freshwater fish species over the last 20 yr. We contrasted two extreme cases to determine whether extirpations were governed by patterns of climate exposure, assuming full adaptation of species to local climate, or instead by the interplay between climate exposure and the distance from the centroid of species’ climatic niches, assuming a fixed niche, and asked whether anthropogenic disturbances interact with these climatic drivers. We found strong support for the fixed niche hypothesis, but showed that species‐specific local adaptation to climate may also be important in determining extirpation dynamics. We also demonstrated that anthropogenic disturbance acted in concert with climate, ultimately determining population changes. Our results add novel evidence that unravelling the direct links between range dynamics and climate requires a multifaceted treatment, and that accounting for the cumulative effects of anthropogenic pressures deserves special attention in the context of climate change.     

Dissociation between ventral and dorsal fMRI activation during object and action recognition

Neuropsychological case studies suggest the existence of two functionally separate visual streams: the ventral pathway, central for object recognition; and the dorsal pathway, engaged in visually guided actions. However, a clear dissociation between the functions of the two streams has not been decisively shown in intact humans. In this study, we demonstrate dissociation between dorsal and ventral fMRI activation patterns during observation of object manipulation video clips. Parietal areas, such as anterior intraparietal sulcus (aIPS) display grasp viewing-dependent adaptation (i.e., fMR adaptation during repeated viewing of the same object-grasping movement) as well as a contralateral preference for the viewed manipulating hand. Ventral regions, such as the fusiform gyrus, show similar characteristics (i.e., adaptation, contralateral preference), but these depend on object identity. Our results support the hypothesized functional specialization in the visual system and suggest that parietal areas (such as aIPS) are engaged in action recognition, as well as in action planning.     

The Time Course of Activation of Object Shape and Shape+Colour Representations during Memory Retrieval

Little is known about the timing of activating memory for objects and their associated perceptual properties, such as colour, and yet this is important for theories of human cognition. We investigated the time course associated with early cognitive processes related to the activation of object shape and object shape+colour representations respectively, during memory retrieval as assessed by repetition priming in an event-related potential (ERP) study. The main findings were as follows: (1) we identified a unique early modulation of mean ERP amplitude during the N1 that was associated with the activation of object shape independently of colour; (2) we also found a subsequent early P2 modulation of mean amplitude over the same electrode clusters associated with the activation of object shape+colour representations; (3) these findings were apparent across both familiar (i.e., correctly coloured – yellow banana) and novel (i.e., incorrectly coloured - blue strawberry) objects; and (4) neither of the modulations of mean ERP amplitude were evident during the P3. Together the findings delineate the timing of object shape and colour memory systems and support the notion that perceptual representations of object shape mediate the retrieval of temporary shape+colour representations for familiar and novel objects.     

Visual, haptic and cross-modal recognition of objects and scenes.

In this article we review current literature on cross-modal recognition and present new findings from our studies on object and scene recognition. Specifically, we address the questions of what is the nature of the representation underlying each sensory system that facilitates convergence across the senses and how perception is modified by the interaction of the senses. In the first set of our experiments, the recognition of unfamiliar objects within and across the visual and haptic modalities was investigated under conditions of changes in orientation (0 degrees or 180 degrees ). An orientation change increased recognition errors within each modality but this effect was reduced across modalities. Our results suggest that cross-modal object representations of objects are mediated by surface-dependent representations. In a second series of experiments, we investigated how spatial information is integrated across modalities and viewpoint using scenes of familiar, 3D objects as stimuli. We found that scene recognition performance was less efficient when there was either a change in modality, or in orientation, between learning and test. Furthermore, haptic learning was selectively disrupted by a verbal interpolation task. Our findings are discussed with reference to separate spatial encoding of visual and haptic scenes. We conclude by discussing a number of constraints under which cross-modal integration is optimal for object recognition. These constraints include the nature of the task, and the amount of spatial and temporal congruency of information across the modalities.     

Adaptation-induced plasticity of orientation tuning in adult visual cortex

A key emergent property of the primary visual cortex (V1) is the orientation selectivity of its neurons. The extent to which adult visual cortical neurons can exhibit changes in orientation selectivity is unknown. Here we use single-unit recording and intrinsic signal imaging in V1 of adult cats to demonstrate systematic repulsive shifts in orientation preference following short-term exposure (adaptation) to one stimulus orientation. In contrast to the common view of adaptation as a passive process by which responses around the adapting orientation are reduced, we show that changes in orientation tuning also occur due to response increases at orientations away from the adapting stimulus. Adaptation-induced orientation plasticity is thus an active time-dependent process that involves network interactions and includes both response depression and enhancement.     

The role of environmental familiarization in novel-object preference

The experiments in this report examined the behavioral variables modulating novel-object preference in a widely used 'object recognition' preparation. This preparation takes advantage of the tendency of rats to interact more with a novel than a familiar 'sample' object in a free-choice situation. Experiment 1 examined the interaction between environmental familiarization and the duration of sample-object exposure in the development of a novel-object preference. Interaction with the sample object during sample-object exposure was increased when rats were given time in the environment prior to presentation of the sample object. Also, provided that the sample-object exposure was greater than 2 min, rats given environmental familiarization time prior to sample-object exposure displayed a novel-object preference. Rats that received the same amount of sample-object exposure without prior exposure to the environment alone did not discriminate between the sample and novel object. In Experiments 2 and 3, sample-object exposure occurred in a different environment than the novel-object test. Novel-object preference was not affected regardless of whether that testing environment was familiar or novel. This result differs from previous work that finds that an object recovers some novelty when moved to a new spatial location.     

Long-range, pattern-dependent contextual effects in early human visual cortex

The standard view of neurons in early visual cortex is that they behave like localized feature detectors. Here we demonstrate that processing in early visual areas goes beyond feature detection by showing that neural responses are greater when a feature deviates from its context compared to when it does not deviate from its context. Using psychophysics, fMRI, and electroencephalography methodologies, we measured neural responses to an oriented Gabor ("target") embedded in various visual patterns as defined by the relative orientation of flanking stimuli. We first show using psychophysical contrast adaptation and fMRI that a target that differs from its context results in more neural activity compared to a target that is contained within an alternating sequence, suggesting that neurons in early visual cortex are sensitive to large-scale orientation patterns. Next, we use event-related potentials to show that orientation deviations affect the earliest sensory components of the target response. Finally, we use forced-choice classification of "noise" stimuli to show that we are more likely to "see" orientations that deviate from the context. Our results suggest that early visual cortex is sensitive to global patterns in images in a way that is markedly different from the predictions of standard models of cortical visual processing.     

From Sensory Signals to Modality-Independent Conceptual Representations: A Probabilistic Language of Thought Approach

People learn modality-independent, conceptual representations from modality-specific sensory signals. Here, we hypothesize that any system that accomplishes this feat will include three components: a representational language for characterizing modality-independent representations, a set of sensory-specific forward models for mapping from modality-independent representations to sensory signals, and an inference algorithm for inverting forward models—that is, an algorithm for using sensory signals to infer modality-independent representations. To evaluate this hypothesis, we instantiate it in the form of a computational model that learns object shape representations from visual and/or haptic signals. The model uses a probabilistic grammar to characterize modality-independent representations of object shape, uses a computer graphics toolkit and a human hand simulator to map from object representations to visual and haptic features, respectively, and uses a Bayesian inference algorithm to infer modality-independent object representations from visual and/or haptic signals. Simulation results show that the model infers identical object representations when an object is viewed, grasped, or both. That is, the model’s percepts are modality invariant. We also report the results of an experiment in which different subjects rated the similarity of pairs of objects in different sensory conditions, and show that the model provides a very accurate account of subjects’ ratings. Conceptually, this research significantly contributes to our understanding of modality invariance, an important type of perceptual constancy, by demonstrating how modality-independent representations can be acquired and used. Methodologically, it provides an important contribution to cognitive modeling, particularly an emerging probabilistic language-of-thought approach, by showing how symbolic and statistical approaches can be combined in order to understand aspects of human perception.     

Individuals with autism spectrum disorder have altered visual encoding capacity

Perceptual anomalies in individuals with autism spectrum disorder (ASD) have been attributed to an imbalance in weighting incoming sensory evidence with prior knowledge when interpreting sensory information. Here, we show that sensory encoding and how it adapts to changing stimulus statistics during feedback also characteristically differs between neurotypical and ASD groups. In a visual orientation estimation task, we extracted the accuracy of sensory encoding from psychophysical data by using an information theoretic measure. Initially, sensory representations in both groups reflected the statistics of visual orientations in natural scenes, but encoding capacity was overall lower in the ASD group. Exposure to an artificial (i.e., uniform) distribution of visual orientations coupled with performance feedback altered the sensory representations of the neurotypical group toward the novel experimental statistics, while also increasing their total encoding capacity. In contrast, neither total encoding capacity nor its allocation significantly changed in the ASD group. Across both groups, the degree of adaptation was correlated with participants’ initial encoding capacity. These findings highlight substantial deficits in sensory encoding—independent from and potentially in addition to deficits in decoding—in individuals with ASD.

It is increasingly recognized that individuals with Autism Spectrum Disorder (ASD) show anomalies in perception, and these have been recently attributed to altered decoding (i.e. interpretation of sensory signals). This study reveals that independent of these changes, individuals with ASD show upstream deficits in sensory encoding (i.e., how samples are drawn from the environment).