Visual exploration and object recognition by lattice deformation

Mechanisms of explicit object recognition are often difficult to investigate and require stimuli with controlled features whose expression can be manipulated in a precise quantitative fashion. Here, we developed a novel method (called "Dots"), for generating visual stimuli, which is based on the progressive deformation of a regular lattice of dots, driven by local contour information from images of objects. By applying progressively larger deformation to the lattice, the latter conveys progressively more information about the target object. Stimuli generated with the presented method enable a precise control of object-related information content while preserving low-level image statistics, globally, and affecting them only little, locally. We show that such stimuli are useful for investigating object recognition under a naturalistic setting--free visual exploration--enabling a clear dissociation between object detection and explicit recognition. Using the introduced stimuli, we show that top-down modulation induced by previous exposure to target objects can greatly influence perceptual decisions, lowering perceptual thresholds not only for object recognition but also for object detection (visual hysteresis). Visual hysteresis is target-specific, its expression and magnitude depending on the identity of individual objects. Relying on the particular features of dot stimuli and on eye-tracking measurements, we further demonstrate that top-down processes guide visual exploration, controlling how visual information is integrated by successive fixations. Prior knowledge about objects can guide saccades/fixations to sample locations that are supposed to be highly informative, even when the actual information is missing from those locations in the stimulus. The duration of individual fixations is modulated by the novelty and difficulty of the stimulus, likely reflecting cognitive demand.     

Creating Objects and Object Categories for Studying Perception and Perceptual Learning

In order to quantitatively study object perception, be it perception by biological systems or by machines, one needs to create objects and object categories with precisely definable, preferably naturalistic, properties¹. Furthermore, for studies on perceptual learning, it is useful to create novel objects and object categories (or object classes) with such properties².Many innovative and useful methods currently exist for creating novel objects and object categories^3-6 (also see refs. 7,8). However, generally speaking, the existing methods have three broad types of shortcomings.First, shape variations are generally imposed by the experimenter^5,9,10, and may therefore be different from the variability in natural categories, and optimized for a particular recognition algorithm. It would be desirable to have the variations arise independently of the externally imposed constraints.Second, the existing methods have difficulty capturing the shape complexity of natural objects^11-13. If the goal is to study natural object perception, it is desirable for objects and object categories to be naturalistic, so as to avoid possible confounds and special cases.Third, it is generally hard to quantitatively measure the available information in the stimuli created by conventional methods. It would be desirable to create objects and object categories where the available information can be precisely measured and, where necessary, systematically manipulated (or ''tuned''). This allows one to formulate the underlying object recognition tasks in quantitative terms.Here we describe a set of algorithms, or methods, that meet all three of the above criteria. Virtual morphogenesis (VM) creates novel, naturalistic virtual 3-D objects called ''digital embryos'' by simulating the biological process of embryogenesis¹⁴. Virtual phylogenesis (VP) creates novel, naturalistic object categories by simulating the evolutionary process of natural selection^9,12,13. Objects and object categories created by these simulations can be further manipulated by various morphing methods to generate systematic variations of shape characteristics^15,16. The VP and morphing methods can also be applied, in principle, to novel virtual objects other than digital embryos, or to virtual versions of real-world objects^9,13. Virtual objects created in this fashion can be rendered as visual images using a conventional graphical toolkit, with desired manipulations of surface texture, illumination, size, viewpoint and background. The virtual objects can also be ''printed'' as haptic objects using a conventional 3-D prototyper.We also describe some implementations of these computational algorithms to help illustrate the potential utility of the algorithms. It is important to distinguish the algorithms from their implementations. The implementations are demonstrations offered solely as a ''proof of principle'' of the underlying algorithms. It is important to note that, in general, an implementation of a computational algorithm often has limitations that the algorithm itself does not have.Together, these methods represent a set of powerful and flexible tools for studying object recognition and perceptual learning by biological and computational systems alike. With appropriate extensions, these methods may also prove useful in the study of morphogenesis and phylogenesis.     

Activation of kinetically distinct synaptic conductances on inhibitory interneurons by electrotonically overlapping afferents

We measured brain activity during mental rotation and object recognition with objects rotated around three different axes. Activity in the superior parietal lobe (SPL) increased proportionally to viewpoint disparity during mental rotation, but not during object recognition. In contrast, the fusiform gyrus was preferentially recruited in a viewpoint-dependent manner in recognition as compared to mental rotation. In addition, independent of the effect of viewpoint, object recognition was associated with ventral areas and mental rotation with dorsal areas. These results indicate that the similar behavioral effects of viewpoint obtained in these two tasks are based on different neural substrates. Such findings call into question the hypothesis that mental rotation is used to compensate for changes in viewpoint during object recognition.     

Pigeons and humans are more sensitive to nonaccidental than to metric changes in visual objects

Humans and macaques are more sensitive to differences in nonaccidental image properties, such as straight vs. curved contours, than to differences in metric properties, such as degree of curvature [Biederman, I., Bar, M., 1999. One-shot viewpoint invariance in matching novel objects. Vis. Res. 39, 2885-2899; Kayaert, G., Biederman, I., Vogels, R., 2003. Shape tuning in macaque inferior temporal cortex. J. Neurosci. 23, 3016-3027; Kayaert, G., Biederman, I., Vogels, R., 2005. Representation of regular and irregular shapes in macaque inferotemporal cortex. Cereb. Cortex 15, 1308-1321]. This differential sensitivity allows facile recognition when the object is viewed at an orientation in depth not previously experienced. In Experiment 1, we trained pigeons to discriminate grayscale, shaded images of four shapes. Pigeons made more confusion errors to shapes that shared more nonaccidental properties. Although the images in that experiment were not well controlled for incidental changes in metric properties, the same results were apparent with better controlled stimuli in Experiment 2: pigeons trained to discriminate a target shape from a metrically changed shape and a nonaccidentally changed shape committed more confusion errors to the metrically changed shape, suggesting that they perceived it to be more similar to the target shape. Humans trained with similar stimuli and procedure exhibited the same tendency to make more errors to the metrically changed shape. These results document the greater saliency of nonaccidental differences for shape recognition and discrimination in a non-primate species and suggest that nonaccidental sensitivity may be characteristic of all shape-discriminating species.     

Active manual control of object views facilitates visual recognition

Active exploration of large-scale environments leads to better learning of spatial layout than does passive observation [1] [2] [3]. But active exploration might also help us to remember the appearance of individual objects in a scene. In fact, when we encounter new objects, we often manipulate them so that they can be seen from a variety of perspectives. We present here the first evidence that active control of the visual input in this way facilitates later recognition of objects. Observers who actively rotated novel, three-dimensional objects on a computer screen later showed more efficient visual recognition than observers who passively viewed the exact same sequence of images of these virtual objects. During active exploration, the observers focused mainly on the 'side' or 'front' views of the objects (see also [4] [5] [6]). The results demonstrate that how an object is represented for later recognition is influenced by whether or not one controls the presentation of visual input during learning.     

A neuromorphic architecture for object recognition and motion anticipation using burst-STDP

In this work we investigate the possibilities offered by a minimal framework of artificial spiking neurons to be deployed in silico. Here we introduce a hierarchical network architecture of spiking neurons which learns to recognize moving objects in a visual environment and determine the correct motor output for each object. These tasks are learned through both supervised and unsupervised spike timing dependent plasticity (STDP). STDP is responsible for the strengthening (or weakening) of synapses in relation to pre- and post-synaptic spike times and has been described as a Hebbian paradigm taking place both in vitro and in vivo. We utilize a variation of STDP learning, called burst-STDP, which is based on the notion that, since spikes are expensive in terms of energy consumption, then strong bursting activity carries more information than single (sparse) spikes. Furthermore, this learning algorithm takes advantage of homeostatic renormalization, which has been hypothesized to promote memory consolidation during NREM sleep. Using this learning rule, we design a spiking neural network architecture capable of object recognition, motion detection, attention towards important objects, and motor control outputs. We demonstrate the abilities of our design in a simple environment with distractor objects, multiple objects moving concurrently, and in the presence of noise. Most importantly, we show how this neural network is capable of performing these tasks using a simple leaky-integrate-and-fire (LIF) neuron model with binary synapses, making it fully compatible with state-of-the-art digital neuromorphic hardware designs. As such, the building blocks and learning rules presented in this paper appear promising for scalable fully neuromorphic systems to be implemented in hardware chips.     

Learning viewpoint invariant object representations using a temporal coherence principle

Invariant object recognition is arguably one of the major challenges for contemporary machine vision systems. In contrast, the mammalian visual system performs this task virtually effortlessly. How can we exploit our knowledge on the biological system to improve artificial systems? Our understanding of the mammalian early visual system has been augmented by the discovery that general coding principles could explain many aspects of neuronal response properties. How can such schemes be transferred to system level performance? In the present study we train cells on a particular variant of the general principle of temporal coherence, the “stability” objective. These cells are trained on unlabeled real-world images without a teaching signal. We show that after training, the cells form a representation that is largely independent of the viewpoint from which the stimulus is looked at. This finding includes generalization to previously unseen viewpoints. The achieved representation is better suited for view-point invariant object classification than the cells’ input patterns. This property to facilitate view-point invariant classification is maintained even if training and classification take place in the presence of an – also unlabeled – distractor object. In summary, here we show that unsupervised learning using a general coding principle facilitates the classification of real-world objects, that are not segmented from the background and undergo complex, non-isomorphic, transformations.     

Flexible representations of dynamics are used in object manipulation

To manipulate an object skillfully, the brain must learn its dynamics, specifying the mapping between applied force and motion. A fundamental issue in sensorimotor control is whether such dynamics are represented in an extrinsic frame of reference tied to the object or an intrinsic frame of reference linked to the arm. Although previous studies have suggested that objects are represented in arm-centered coordinates [1-6], all of these studies have used objects with unusual and complex dynamics. Thus, it is not known how objects with natural dynamics are represented. Here we show that objects with simple (or familiar) dynamics and those with complex (or unfamiliar) dynamics are represented in object- and arm-centered coordinates, respectively. We also show that objects with simple dynamics are represented with an intermediate coordinate frame when vision of the object is removed. These results indicate that object dynamics can be flexibly represented in different coordinate frames by the brain. We suggest that with experience, the representation of the dynamics of a manipulated object may shift from a coordinate frame tied to the arm toward one that is linked to the object. The additional complexity required to represent dynamics in object-centered coordinates would be economical for familiar objects because such a representation allows object use regardless of the orientation of the object in hand.     

Neuropsychological studies of object recognition

It is well established that disorders of visual perception are associated with lesions in the right hemisphere. Performances on tasks as disparate as the identification of objects from unusual views of objects drawn so as to overlap, of fragmented letters, of familiar faces, and of anomalous features in drawings, have been shown to be impaired in patients with focal right posterior lesions. A series of investigations are reviewed, directed towards analysing the basis of these deficits. Explanations in terms of primary visual impairment can be rejected, as can an account in terms of faulty figure-ground organization. It is argued that a wide variety of such perceptual deficits--all of which are concerned with meaningful visual stimuli--can be encompassed by the notion of faulty perceptual categorization at an early post-sensory stage of object recognition. Moreover, there is evidence suggesting that some of these various perceptual deficits can be mutually dissociated. The concept of perceptual categorization is discussed in the wider context of tentative model of object recognition.     

The effects of visual object priming on brain activation before and after recognition

BACKGROUND: Recognizing an object is improved by recent experience with that object even if one cannot recall seeing the object. This perceptual facilitation as a result of previous experience is called priming. In neuroimaging studies, priming is often associated with a decrease in activation in brain regions involved in object recognition. It is thought that this occurs because priming causes a sharpening of object representations which leads to more efficient processing and, consequently, a reduction in neural activity. Recent evidence has suggested, however, that the apparent effect of priming on brain activation may vary as a function of whether the neural activity is measured before or after recognition has taken place. RESULTS: Using a gradual 'unmasking' technique, we presented primed and non-primed objects to subjects, and measured activation time courses using high-field functional magnetic resonance imaging (fMRI). As the objects were slowly revealed, but before recognition had occurred, activation increased from baseline level to a peak that corresponded in time to the subjects' behavioural recognition responses. The activation peak for primed objects occurred sooner than the peak for non-primed objects, and subjects responded sooner when presented with a primed object than with a non-primed object. During this pre-recognition phase, primed objects produced more activation than non-primed objects. After recognition, activation declined rapidly for both primed and non-primed objects, but now activation was lower for the primed objects. CONCLUSIONS: Priming did not produce a general decrease in activation in the brain regions involved in object recognition but, instead, produced a shift in the time of peak activation that corresponded to the shift in time seen in the subjects' behavioural recognition performance.     

Temporal brain dynamics of multiple object processing: the flexibility of individuation

The ability to process concurrently multiple visual objects is fundamental for a coherent perception of the world. A core component of this ability is the simultaneous individuation of multiple objects. Many studies have addressed the mechanism of object individuation but it remains unknown whether the visual system mandatorily individuates all relevant elements in the visual field, or whether object indexing depends on task demands. We used a neural measure of visual selection, the N2pc component, to evaluate the flexibility of multiple object individuation. In three ERP experiments, participants saw a variable number of target elements among homogenous distracters and performed either an enumeration task (Experiment 1) or a detection task, reporting whether at least one (Experiment 2) or a specified number of target elements (Experiment 3) was present. While in the enumeration task the N2pc response increased as a function of the number of targets, no such modulation was found in Experiment 2, indicating that individuation of multiple targets is not mandatory. However, a modulation of the N2pc similar to the enumeration task was visible in Experiment 3, further highlighting that object individuation is a flexible mechanism that binds indexes to object properties and locations as needed for further object processing.     

Effects of gonadectomy and hormone replacement on a spontaneous novel object recognition task in adult male rats

Recent studies in adult male rats have shown that gonadal hormones influence performance on certain working memory and other types of cognitive tasks that are sensitive to lesions of the medial and/or orbital prefrontal cortices. This study asked whether gonadal hormone modulation of prefrontal cortical function in males also extends to the perirhinal division of the rat prefrontal cortex. Specifically, sham-operated control, gonadectomized, and gonadectomized rats supplemented with testosterone propionate or estradiol were tested on a spontaneous novel object recognition task, a paradigm where performance has been shown to be impaired by perirhinal cortical lesions. Using analyses of variance, regression analyses and post-hoc testing to evaluate group differences, it was found that during both the sample and test trials of the task all four groups spent similar absolute and proportional amounts of time ambulating, rearing, stationary, and exploring the two objects present. All groups also explored each of the two identical objects present during sample trials equally. However, during the test trials, only the control and gonadectomized rats given testosterone showed the expected increase in exploration of the novel objects presented, whereas the gonadectomized and gonadectomized, estradiol-supplemental groups continued to explore the novel and familiar objects equally. That regression analyses also identified significant correlations between low bulbospongiosus muscle weight and impaired novel vs. familiar object discrimination further indicates that gonadectomy in adult male rats adversely affects spontaneous novel object recognition in an androgen-sensitive, estrogen-insensitive manner.     

Border collie comprehends object names as verbal referents

Four experiments investigated the ability of a border collie (Chaser) to acquire receptive language skills. Experiment 1 demonstrated that Chaser learned and retained, over a 3-year period of intensive training, the proper-noun names of 1022 objects. Experiment 2 presented random pair-wise combinations of three commands and three names, and demonstrated that she understood the separate meanings of proper-noun names and commands. Chaser understood that names refer to objects, independent of the behavior directed toward those objects. Experiment 3 demonstrated Chaser's ability to learn three common nouns - words that represent categories. Chaser demonstrated one-to-many (common noun) and many-to-one (multiple-name) name-object mappings. Experiment 4 demonstrated Chaser's ability to learn words by inferential reasoning by exclusion - inferring the name of an object based on its novelty among familiar objects that already had names. Together, these studies indicate that Chaser acquired referential understanding of nouns, an ability normally attributed to children, which included: (a) awareness that words may refer to objects, (b) awareness of verbal cues that map words upon the object referent, and (c) awareness that names may refer to unique objects or categories of objects, independent of the behaviors directed toward those objects.     

How Can Selection of Biologically Inspired Features Improve the Performance of a Robust Object Recognition Model?

Humans can effectively and swiftly recognize objects in complex natural scenes. This outstanding ability has motivated many computational object recognition models. Most of these models try to emulate the behavior of this remarkable system. The human visual system hierarchically recognizes objects in several processing stages. Along these stages a set of features with increasing complexity is extracted by different parts of visual system. Elementary features like bars and edges are processed in earlier levels of visual pathway and as far as one goes upper in this pathway more complex features will be spotted. It is an important interrogation in the field of visual processing to see which features of an object are selected and represented by the visual cortex. To address this issue, we extended a hierarchical model, which is motivated by biology, for different object recognition tasks. In this model, a set of object parts, named patches, extracted in the intermediate stages. These object parts are used for training procedure in the model and have an important role in object recognition. These patches are selected indiscriminately from different positions of an image and this can lead to the extraction of non-discriminating patches which eventually may reduce the performance. In the proposed model we used an evolutionary algorithm approach to select a set of informative patches. Our reported results indicate that these patches are more informative than usual random patches. We demonstrate the strength of the proposed model on a range of object recognition tasks. The proposed model outperforms the original model in diverse object recognition tasks. It can be seen from the experiments that selected features are generally particular parts of target images. Our results suggest that selected features which are parts of target objects provide an efficient set for robust object recognition.     

Symmetry perception by poultry chicks and its implications for three-dimensional object recognition

Bilateral symmetry is visually salient to diverse animals including birds, but whereas experimental studies typically use bilaterally symmetrical two-dimensional patterns that are viewed approximately fronto-parallel; in nature, animals observe three-dimensional objects from all angles. Many animals and plant structures have a plane of bilateral symmetry. Here, we first (experiment I) give evidence that young poultry chicks readily generalize bilateral symmetry as a feature of two-dimensional patterns in fronto-parallel view. We then test the ability of chicks to recognize symmetry in images that would be produced by the transformed view produced by a 40° horizontal combined with a 20° vertical rotation of a pattern on a spherical surface. Experiment II gives evidence that chicks trained to distinguish symmetrical from asymmetrical patterns treat rotated views of symmetrical 'objects' as symmetrical. Experiment III gives evidence that chicks trained to discriminate rotated views of symmetrical 'objects' from asymmetrical patterns generalize to novel symmetrical objects either in fronto-parallel or rotated view. These findings emphasize the importance of bilateral symmetry for three-dimensional object recognition and raise questions about the underlying mechanisms of symmetry perception.     

A hierarchical probabilistic model for rapid object categorization in natural scenes

Humans can categorize objects in complex natural scenes within 100-150 ms. This amazing ability of rapid categorization has motivated many computational models. Most of these models require extensive training to obtain a decision boundary in a very high dimensional (e.g., ～6,000 in a leading model) feature space and often categorize objects in natural scenes by categorizing the context that co-occurs with objects when objects do not occupy large portions of the scenes. It is thus unclear how humans achieve rapid scene categorization.To address this issue, we developed a hierarchical probabilistic model for rapid object categorization in natural scenes. In this model, a natural object category is represented by a coarse hierarchical probability distribution (PD), which includes PDs of object geometry and spatial configuration of object parts. Object parts are encoded by PDs of a set of natural object structures, each of which is a concatenation of local object features. Rapid categorization is performed as statistical inference. Since the model uses a very small number (～100) of structures for even complex object categories such as animals and cars, it requires little training and is robust in the presence of large variations within object categories and in their occurrences in natural scenes. Remarkably, we found that the model categorized animals in natural scenes and cars in street scenes with a near human-level performance. We also found that the model located animals and cars in natural scenes, thus overcoming a flaw in many other models which is to categorize objects in natural context by categorizing contextual features. These results suggest that coarse PDs of object categories based on natural object structures and statistical operations on these PDs may underlie the human ability to rapidly categorize scenes.     

Size-Sensitive Perceptual Representations Underlie Visual and Haptic Object Recognition

A variety of similarities between visual and haptic object recognition suggests that the two modalities may share common representations. However, it is unclear whether such common representations preserve low-level perceptual features or whether transfer between vision and haptics is mediated by high-level, abstract representations. Two experiments used a sequential shape-matching task to examine the effects of size changes on unimodal and crossmodal visual and haptic object recognition. Participants felt or saw 3D plastic models of familiar objects. The two objects presented on a trial were either the same size or different sizes and were the same shape or different but similar shapes. Participants were told to ignore size changes and to match on shape alone. In Experiment 1, size changes on same-shape trials impaired performance similarly for both visual-to-visual and haptic-to-haptic shape matching. In Experiment 2, size changes impaired performance on both visual-to-haptic and haptic-to-visual shape matching and there was no interaction between the cost of size changes and direction of transfer. Together the unimodal and crossmodal matching results suggest that the same, size-specific perceptual representations underlie both visual and haptic object recognition, and indicate that crossmodal memory for objects must be at least partly based on common perceptual representations.     

Haptic object perception: spatial dimensionality and relation to vision

Enabled by the remarkable dexterity of the human hand, specialized haptic exploration is a hallmark of object perception by touch. Haptic exploration normally takes place in a spatial world that is three-dimensional; nevertheless, stimuli of reduced spatial dimensionality are also used to display spatial information. This paper examines the consequences of full (three-dimensional) versus reduced (two-dimensional) spatial dimensionality for object processing by touch, particularly in comparison with vision. We begin with perceptual recognition of common human-made artefacts, then extend our discussion of spatial dimensionality in touch and vision to include faces, drawing from research on haptic recognition of facial identity and emotional expressions. Faces have often been characterized as constituting a specialized input for human perception. We find that contrary to vision, haptic processing of common objects is impaired by reduced spatial dimensionality, whereas haptic face processing is not. We interpret these results in terms of fundamental differences in object perception across the modalities, particularly the special role of manual exploration in extracting a three-dimensional structure.     

Disruptive camouflage impairs object recognition

Whether hiding from predators, or avoiding battlefield casualties, camouflage is widely employed to prevent detection. Disruptive coloration is a seemingly well-known camouflage mechanism proposed to function by breaking up an object''s salient features (for example their characteristic outline), rendering objects more difficult to recognize. However, while a wide range of animals are thought to evade detection using disruptive patterns, there is no direct experimental evidence that disruptive coloration impairs recognition. Using humans searching for computer-generated moth targets, we demonstrate that the number of edge-intersecting patches on a target reduces the likelihood of it being detected, even at the expense of reduced background matching. Crucially, eye-tracking data show that targets with more edge-intersecting patches were looked at for longer periods prior to attack, and passed-over more frequently during search tasks. We therefore show directly that edge patches enhance survivorship by impairing recognition, confirming that disruptive coloration is a distinct camouflage strategy, not simply an artefact of background matching.     

Computational object recognition: a biologically motivated approach

We propose a conceptual framework for artificial object recognition systems based on findings from neurophysiological and neuropsychological research on the visual system in primate cortex. We identify some essential questions, which have to be addressed in the course of designing object recognition systems. As answers, we review some major aspects of biological object recognition, which are then translated into the technical field of computer vision. The key suggestions are the use of incremental and view-based approaches together with the ability of online feature selection and the interconnection of object-views to form an overall object representation. The effectiveness of the computational approach is estimated by testing a possible realization in various tasks and conditions explicitly designed to allow for a direct comparison with the biological counterpart. The results exhibit excellent performance with regard to recognition accuracy, the creation of sparse models and the selection of appropriate features.