Medial axis shape coding in macaque inferotemporal cortex

The basic, still unanswered question about visual object representation is this: what specific information is encoded by neural signals? Theorists have long predicted that neurons would encode medial axis or skeletal object shape, yet recent studies reveal instead neural coding of boundary or surface shape. Here, we addressed this theoretical/experimental disconnect, using adaptive shape sampling to demonstrate explicit coding of medial axis shape in high-level object cortex (macaque monkey inferotemporal cortex or IT). Our metric shape analyses revealed a coding continuum, along which most neurons represent a configuration of both medial axis and surface components. Thus, IT response functions embody a rich basis set for simultaneously representing skeletal and external shape of complex objects. This would be especially useful for representing biological shapes, which are often characterized by both complex, articulated skeletal structure and specific surface features.     

Online learning of objects in a biologically motivated visual architecture

We present a biologically motivated architecture for object recognition that is capable of online learning of several objects based on interaction with a human teacher. The system combines biological principles such as appearance-based representation in topographical feature detection hierarchies and context-driven transfer between different levels of object memory. Training can be performed in an unconstrained environment by presenting objects in front of a stereo camera system and labeling them by speech input. The learning is fully online and thus avoids an artificial separation of the interaction into training and test phases. We demonstrate the performance on a challenging ensemble of 50 objects.     

A measure for the angle between projections based on the extent of correlation between corresponding central sections

A pre-condition for the ab initio assignment of Euler angles to a set of projections from an asymmetric object is that at least three of the available projections correspond to rotations about different axes. For symmetric objects this condition may be relaxed. There are some applications of single-particle electron microscopy, such as the reconstruction of filamentous macromolecular assemblies, where all available projections more-or-less correspond to rotations about a common rotation axis making it difficult to satisfy this condition. Here, a method has been developed to overcome this problem, based on the fact that the correlation between two central sections of the Fourier transform of a compact object will not be limited to an infinitesimal central line but will have a finite extent, which is related to the angle between the corresponding projections. Projections from model filaments, with different degrees of rotational symmetry about the long axis, have been used to test the methodology. The results show that angle determination is robust down to signal-to-noise ratios as low as 2 and that, in general, the error decreases as the degree of symmetry increases. The method has been used to assign angles to a set of negatively stained muscle thick filament projections to obtain an initial 3D reconstruction. The main features of the projections are seen to be faithfully reproduced in the reprojections from the reconstruction. A real-space adaptation of this method is also discussed.     

An Elliptometer: A Simple Method of Measuring the Areas of Structures in Microscopic Sections

A simple method of measuring the cross sectional areas of objects (such as cells) in microscopic sections is described. A beam of light is passed through an adjustable diaphragm and focussed by a lens on a screen of mm. ruled graph paper. The screen may be rotated about a horizontal axis. Adjustment of the aperture of the diaphragm and the plane of the screen yields illuminated areas of variable size and degree of ellipticity. As close a “fit” as possible is made between the illuminated area and the camera lucida tracing of the object to be measured. The lengths of the major and minor axes of the appropriate ellipse are read from the screen. The application of the formula for the area of an ellipse to mean major and minor axes for a group of cells gives the mean cross sectional area for the cell population under investigation.     

Contextual Effects of Scene on the Visual Perception of Object Orientation in Depth

We investigated the effect of background scene on the human visual perception of depth orientation (i.e., azimuth angle) of three-dimensional common objects. Participants evaluated the depth orientation of objects. The objects were surrounded by scenes with an apparent axis of the global reference frame, such as a sidewalk scene. When a scene axis was slightly misaligned with the gaze line, object orientation perception was biased, as if the gaze line had been assimilated into the scene axis (Experiment 1). When the scene axis was slightly misaligned with the object, evaluated object orientation was biased, as if it had been assimilated into the scene axis (Experiment 2). This assimilation may be due to confusion between the orientation of the scene and object axes (Experiment 3). Thus, the global reference frame may influence object orientation perception when its orientation is similar to that of the gaze-line or object.     

Object localization with whiskers

Rats use their large facial hairs (whiskers) to detect, localize and identify objects in their proximal three-dimensional (3D) space. Here, we focus on recent evidence of how object location is encoded in the neural sensory pathways of the rat whisker system. Behavioral and neuronal observations have recently converged to the point where object location in 3D appears to be encoded by an efficient orthogonal scheme supported by primary sensory-afferents: each primary-afferent can signal object location by a spatial (labeled-line) code for the vertical axis (along whisker arcs), a temporal code for the horizontal axis (along whisker rows), and an intensity code for the radial axis (from the face out). Neuronal evidence shows that (i) the identities of activated sensory neurons convey information about the vertical coordinate of an object, (ii) the timing of their firing, in relation to other reference signals, conveys information about the horizontal object coordinate, and (iii) the intensity of firing conveys information about the radial object coordinate. Such a triple-coding scheme allows for efficient multiplexing of 3D object location information in the activity of single neurons. Also, this scheme provides redundancy since the same information may be represented in the activity of many neurons. These features of orthogonal coding increase accuracy and reliability. We propose that the multiplexed information is conveyed in parallel to different readout circuits, each decoding a specific spatial variable. Such decoding reduces ambiguity, and simplifies the required decoding algorithms, since different readout circuits can be optimized for a particular variable.     

Activation of kinetically distinct synaptic conductances on inhibitory interneurons by electrotonically overlapping afferents

We measured brain activity during mental rotation and object recognition with objects rotated around three different axes. Activity in the superior parietal lobe (SPL) increased proportionally to viewpoint disparity during mental rotation, but not during object recognition. In contrast, the fusiform gyrus was preferentially recruited in a viewpoint-dependent manner in recognition as compared to mental rotation. In addition, independent of the effect of viewpoint, object recognition was associated with ventral areas and mental rotation with dorsal areas. These results indicate that the similar behavioral effects of viewpoint obtained in these two tasks are based on different neural substrates. Such findings call into question the hypothesis that mental rotation is used to compensate for changes in viewpoint during object recognition.     

Symmetry cues for matching mirrored objects

The study explores cues for matching pairs of objects. These objects were arranged into mirror-symmetrical displays and the task was to judge whether a pre-specified 180 degree rotation around the X, Y or Z axis carries one object into the mirror object. For some rotations, the object's mirror symmetry (M) and, for other rotations, the object's point-symmetry (P) could serve as a cue. The matching results suggest that M is a better cue than P, say M > P. Various attempts are made to explain this effect. The most promising one focuses on simplest structural object representations as these capture M and not P. It is furthermore plausible that M captured by reference frames at high hierarchical representation levels, say M1, serves as a better cue than M captured at low levels, say M2. The prediction M1 > M2 > P merely applies to the open surface objects in the experiment. For the closed solid objects in the experiment the expectation is M1 = M2 = P. Both predictions roughly agree with the accuracy and reaction-time data. The results suggest the perceptual relevance of representation cues and, aditionally, that cues stemming from reference frames at higher hierarchical representation levels are more effective than those from lower levels.     

An active method of extracting egomotion parameters from optical flow

Instead of the former passive method characterized by a camera fixed on a translating and rotating vehicle, an active method characterized by tracking an object at the center of visual field is proposed. The method extracts egomotion parameters such as the instantaneous direction of translation, the axis of rotation and its angular velocity. A unit spherical surface is used as the projection surface.The theory allows clear vision where needed at the center of the visual field, and simultaneously permits the extraction of egomotion parameters from the periphery of the visual field.     

Animal Scanner: Software for classifying humans,animals, and empty frames in camera trap images

Camera traps are a popular tool to sample animal populations because they are noninvasive, detect a variety of species, and can record many thousands of animal detections per deployment. Cameras are typically set to take bursts of multiple photographs for each detection and are deployed in arrays of dozens or hundreds of sites, often resulting in millions of photographs per study. The task of converting photographs to animal detection records from such large image collections is daunting, and made worse by situations that generate copious empty pictures from false triggers (e.g., camera malfunction or moving vegetation) or pictures of humans. We developed computer vision algorithms to detect and classify moving objects to aid the first step of camera trap image filtering—separating the animal detections from the empty frames and pictures of humans. Our new work couples foreground object segmentation through background subtraction with deep learning classification to provide a fast and accurate scheme for human–animal detection. We provide these programs as both Matlab GUI and command prompt developed with C++. The software reads folders of camera trap images and outputs images annotated with bounding boxes around moving objects and a text file summary of results. This software maintains high accuracy while reducing the execution time by 14 times. It takes about 6 seconds to process a sequence of ten frames (on a 2.6 GHZ CPU computer). For those cameras with excessive empty frames due to camera malfunction or blowing vegetation automatically removes 54% of the false‐triggers sequences without influencing the human/animal sequences. We achieve 99.58% on image‐level empty versus object classification of Serengeti dataset. We offer the first computer vision tool for processing camera trap images providing substantial time savings for processing large image datasets, thus improving our ability to monitor wildlife across large scales with camera traps.     

Neural response to movement of the hand and mouth in the secondary somatosensory cortex of Japanese monkeys during a simple feeding task

Abstract

Neural activity was recorded in the secondary somatosensory cortex (SII) of macaque monkeys during a simple feeding task. Around the border between the representations of the hand and face in SII, we found neurons that became active during both retrieving with the hand and eating; 59% had receptive fields (RFs) in the hand/face and the remaining 41% had no RFs. Neurons that responded to touching objects were rarely found. This suggests their sensorimotor function rather than tactile object recognition.     

Controlled rotation of biological microscopic objects using optical line tweezers

Controlled, continuous rotation of cells or intracellular objects was achieved using optical tweezers with an elliptic beam profile (line tweezers), which was generated by placing a cylindrical lens in the path of the trapping beam. By rotating the cylindrical lens, rotation of the elliptic trapping beam and hence of the object trapped therein was achieved. Compared to previously reported techniques for rotation of microscopic objects, this approach is much simpler, gives better utilization of available laser power and also allows much easier control of the trap beam profile. We have used this approach for rotation of biological objects varying in size from 2 to 40 m. At 25 mW trapping beam power at the object plane E. coli bacteria could be rotated at speeds approaching 10 Hz and an intracellular object (presumably a calcium oxalate crystal) trapped inside Elodea densa plant cell could be rotated with speeds of up to 4 Hz. To our knowledge, this is the first report for rotation of an intracellular object.     

A neural network model of semantic memory linking feature-based object representation and words

Recent theories in cognitive neuroscience suggest that semantic memory is a distributed process, which involves many cortical areas and is based on a multimodal representation of objects. The aim of this work is to extend a previous model of object representation to realize a semantic memory, in which sensory-motor representations of objects are linked with words. The model assumes that each object is described as a collection of features, coded in different cortical areas via a topological organization. Features in different objects are segmented via γ-band synchronization of neural oscillators. The feature areas are further connected with a lexical area, devoted to the representation of words. Synapses among the feature areas, and among the lexical area and the feature areas are trained via a time-dependent Hebbian rule, during a period in which individual objects are presented together with the corresponding words. Simulation results demonstrate that, during the retrieval phase, the network can deal with the simultaneous presence of objects (from sensory-motor inputs) and words (from acoustic inputs), can correctly associate objects with words and segment objects even in the presence of incomplete information. Moreover, the network can realize some semantic links among words representing objects with shared features. These results support the idea that semantic memory can be described as an integrated process, whose content is retrieved by the co-activation of different multimodal regions. In perspective, extended versions of this model may be used to test conceptual theories, and to provide a quantitative assessment of existing data (for instance concerning patients with neural deficits).     

Coherent interaction of dynamical attractors for object-based selective attention

I investigate essential neuronal mechanisms of visual attention based on object-based theory and a biased-competition scheme. A neural network model is proposed that consists of two feature networks, FI and FII, and one object network, OJ. The FI and FII networks send feedforward projections to the OJ network and receive feedback projections from the OJ network in a convergent/divergent manner. The OJ network integrates information about sensory features originated from the FI and FII networks into information about objects. I let the feature networks and the object network memorize individual features and objects according to the Hebbian learning rule and create the point attractors corresponding to these features and objects as long-term memories in the network dynamics. When the model tries to attend to objects that are superimposed, the point attractors relevant to the two objects emerge in each network. After a short interval (hundreds of milliseconds), the point attractors relevant to one of the two objects are selected and the other point attractors are completely suppressed. I suggest that coherent interactions of dynamical attractors relevant to the selected object may be the neuronal substrate for object-based selective attention. Bottom-up (FI-to-OJ and FI-to-OJ) neuronal mechanisms separate candidate objects from the background, and top-down (OJ-to-FI and OJ-to-FII) mechanisms resolve object-competition by which one relevant object is selected from candidate objects.     

Invariant visual object recognition: a model, with lighting invariance.

How are invariant representations of objects formed in the visual cortex? We describe a neurophysiological and computational approach which focusses on a feature hierarchy model in which invariant representations can be built by self-organizing learning based on the statistics of the visual input. The model can use temporal continuity in an associative synaptic learning rule with a short term memory trace, and/or it can use spatial continuity in Continuous Transformation learning. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and in this paper we show also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in for example spatial and object search tasks. The model has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene.     

VISUAL ACUITY AND ILLUMINATION IN DIFFERENT SPECTRAL REGIONS

The relation between visual acuity and illumination was measured in red and blue light, using a broken circle or C and a grating as test objects. The red light data fall on single continuous curves representing pure cone vision. The blue light data fall on two distinct curves with a transition at about 0.03 photons. Values below this intensity represent pure rod vision. Those immediately above represent the cooperative activity of rods and cones, and yield higher visual acuities than either. Pure cone vision in this intensity region is given by central fixation (C test object). All the rest of the values above this transition region represent pure cone vision. In blue light the rod data with the C lie about 1.5 log units lower on the intensity axis (cone scale) than they do in white light, while with the grating they lie about 1.0 log unit lower than in white light. Both the pure rod and cone data with the C test object are precisely described by one form of the stationary state equation. With the grating test object and a non-limiting pupil, the pure rod and cone data are described by another form of the same equation in which the curve is half as steep. The introduction of a small pupil, which limits maximum visual acuity, makes the relation between visual acuity and illumination appear steeper. Determinations of maximum visual acuities under a variety of conditions show that for the grating the pupil has to be larger, the longer the wavelength of the light, in order for the pupil not to be the limiting factor. Similar measurements with the C show that when intensity discrimination at the retina is experimentally made the limiting factor in resolution, visual acuity is improved by conditions designed to increase image contrast. However, intensity discrimination cannot be the limiting factor for the ordinary test object resolution because the conditions designed to improve image contrast do not improve maximum visual acuity, while those which reduce image contrast do not produce proportional reductions of visual acuity.     

caBIONet--A .NET wrapper to access and process genomic data stored at the National Cancer Institute's Center for Bioinformatics databases

MOTIVATION: The National Cancer Institute's Center for Bioinformatics (NCICB) has developed a Java based data management and information system called caCORE. One component of this software suite is the object oriented API (caBIO) used to access the rich biological datasets collected at the NCI. This API can access the data using native Java classes, SOAP requests or HTTP calls. Non-Java based clients wanting to use this API have to use the SOAP or HTTP interfaces with the data being returned from the NCI servers as an XML data stream. Although the XML can be read and manipulated using DOM or SAX parsers, one loses the convenience and usability of an object oriented programming paradigm. caBIONet is a set of .NET wrapper classes (managers, genes, chromosomes, sequences, etc.) capable of serializing the XML data stream into local .NET objects. The software is able to search NCICB databases and provide local objects representing the data that can be manipulated and used by other .NET programs. The software was written in C# and compiled as a .NET DLL.     

Perception of successive brief objects as a function of stimulus onset asynchrony: model experiments based on two-stage synchronization of neuronal oscillators

Recently we introduced a new version of the perceptual retouch model incorporating two interactive binding operations—binding features for objects and binding the bound feature-objects with a large scale oscillatory system that acts as a mediary for the perceptual information to reach consciousness-level representation. The relative level of synchronized firing of the neurons representing the features of an object obtained after the second-stage synchronizing modulation is used as the equivalent of conscious perception of the corresponding object. Here, this model is used for simulating interaction of two successive featured objects as a function of stimulus onset asynchrony (SOA). Model output reproduces typical results of mutual masking—with shortest and longest SOAs first and second object correct perception rate is comparable while with intermediate SOAs second object dominates over the first one. Additionally, with shortest SOAs misbinding of features to form illusory objects is simulated by the model.     

Object detection in the fly during simulated translatory flight

Translatory movement of an animal in its environment induces optic flow that contains information about the three-dimensional layout of the surroundings: as a rule, images of objects that are closer to the animal move faster across the retina than those of more distant objects. Such relative motion cues are used by flies to detect objects in front of a structured background. We confronted flying flies, tethered to a torque meter, with front-to-back motion of patterns displayed on two CRT screens, thereby simulating translatory motion of the background as experienced by an animal during straight flight. The torque meter measured the instantaneous turning responses of the fly around its vertical body axis. During short time intervals, object motion was superimposed on background pattern motion. The average turning response towards such an object depends on both object and background velocity in a characteristic way: (1) in order to elicit significant responses object motion has to be faster than background motion; (2) background motion within a certain range of velocities improves object detection. These properties can be interpreted as adaptations to situations as they occur in natural free flight. We confirmed that the measured responses were mediated mainly by a control system specialized for the detection of objects rather than by the compensatory optomotor system responsible for course stabilization. Accepted: 20 March 1997     

Use of temporary and semipermanent enrichment objects by five chimpanzees

At the Chimpanzee and Human Communication Institute, caregivers provide the 5 chimpanzees who reside there with many different forms of social, food, habitat, and object enrichment. In this study, we examined the chimpanzees' use of both semipermanent and temporary objects. Semipermanent objects included cargo nets, climbing structures, a treat mound, and other objects that were present at the chimpanzees' enclosure throughout the duration of this study. Each day, 50 temporary objects were placed in the chimpanzees' outdoor enclosure or indoor exercise rooms. Frequency of use was examined in 2 conditions: rotated and same. In the rotated condition, temporary objects were replaced with different temporary objects after 3 hr. In the same condition, temporary objects were presented for the entire day. Focal and scan sampling were used to record the chimpanzees' use of enrichment objects. Observers collected focal sample data to record the chimpanzees' initial reaction to objects when entering the indoor exercise rooms, outdoor enclosures, or both at 9:00 a.m. and 1.00 p.m. A total of 35 hr of focal data and 156 hr of scan data were collected over an 8-week period. Temporary object rotation in- creased the overall frequency of temporary object use both in the initial 15 min of focal sample data and during the following 6 hr of scan sample data for 4 of the chimpanzees. All of these chimpanzees used both semipermanent and temporary objects throughout the day. Each chimpanzee's pattern of use was unique. The results of this study emphasize the importance of temporary object rotation and presentation of both temporary and semipermanent objects to captive chimpanzee environments.