Effects of Specular Highlights on Perceived Surface Convexity

Shading is known to produce vivid perceptions of depth. However, the influence of specular highlights on perceived shape is unclear: some studies have shown that highlights improve quantitative shape perception while others have shown no effect. Here we ask how specular highlights combine with Lambertian shading cues to determine perceived surface curvature, and to what degree this is based upon a coherent model of the scene geometry. Observers viewed ambiguous convex/concave shaded surfaces, with or without highlights. We show that the presence/absence of specular highlights has an effect on qualitative shape, their presence biasing perception toward convex interpretations of ambiguous shaded objects. We also find that the alignment of a highlight with the Lambertian shading modulates its effect on perceived shape; misaligned highlights are less likely to be perceived as specularities, and thus have less effect on shape perception. Increasing the depth of the surface or the slant of the illuminant also modulated the effect of the highlight, increasing the bias toward convexity. The effect of highlights on perceived shape can be understood probabilistically in terms of scene geometry: for deeper objects and/or highly slanted illuminants, highlights will occur on convex but not concave surfaces, due to occlusion of the illuminant. Given uncertainty about the exact object depth and illuminant direction, the presence of a highlight increases the probability that the surface is convex.     

Visual, haptic and cross-modal recognition of objects and scenes.

In this article we review current literature on cross-modal recognition and present new findings from our studies on object and scene recognition. Specifically, we address the questions of what is the nature of the representation underlying each sensory system that facilitates convergence across the senses and how perception is modified by the interaction of the senses. In the first set of our experiments, the recognition of unfamiliar objects within and across the visual and haptic modalities was investigated under conditions of changes in orientation (0 degrees or 180 degrees ). An orientation change increased recognition errors within each modality but this effect was reduced across modalities. Our results suggest that cross-modal object representations of objects are mediated by surface-dependent representations. In a second series of experiments, we investigated how spatial information is integrated across modalities and viewpoint using scenes of familiar, 3D objects as stimuli. We found that scene recognition performance was less efficient when there was either a change in modality, or in orientation, between learning and test. Furthermore, haptic learning was selectively disrupted by a verbal interpolation task. Our findings are discussed with reference to separate spatial encoding of visual and haptic scenes. We conclude by discussing a number of constraints under which cross-modal integration is optimal for object recognition. These constraints include the nature of the task, and the amount of spatial and temporal congruency of information across the modalities.     

Orientation discrimination and categorization of photographs of natural objects by pigeons

In Experiment 1, pigeons trained to discriminate rightside-up and upside-down orientations of slides of natural scenes with humans successfully transferred to new slides of the same kind. Experiment 2 revealed that both the orientations of the human figures and of the background scenes controlled the discrimination. When they were oppositely oriented, the background orientation cue was dominant. In Experiment 3 slides showing objects on a white background were presented either rightside up or upside down, with each slide presented in one orientation only. One group of pigeons learned to classify the slides according to their orientations. The other group learned to classify the slides according to arbitrary groupings. When the slides were shown rotated by 180 degrees, the latter group continued to discriminate the individual slides (i.e., the pigeons showed orientation invariance). The former group classified the rotated slides according to their orientations (i.e., orientation discrimination). In Experiment 4, pigeons learned the orientation discrimination with separate sets of human and bird figures. Partial reversal training in one object class transferred to the rest of stimuli in this object class but did not to the other object class. These results suggest that pigeons can learn to discriminate photographs on the basis of orientation but that orientation-based equivalence relationship is not formed between object classes.     

Is object search mediated by object-based or image-based representations?

Recent research suggests that visually specific memory representations for previously fixated objects are maintained during scene perception. Here we investigate the degree of visual specificity by asking whether the memory representations are image-based or object-based. To that end we measured the effects of object orientation on the time to search for a familiar object from amongst a set of 7 familiar distractors arranged in a circular array. Search times were found to depend on the relative orientations of the target object and the probe object for both familiar and novel objects. This effect was found to be partly an image matching effect but there was also an advantage shown for the object's canonical view for familiar objects. Orientation effects were maintained even when the target object was specified as having unique or similar shape properties relative to the distractors. Participants' eye movements were monitored during two of the experiments. Eye movement patterns revealed selection for object shape and object orientation during the search process. Our findings provide evidence for object representations during search that are detailed and share image-based characteristics with more high-level characteristics from object memory.     

Coordinates of Human Visual and Inertial Heading Perception

Heading estimation involves both inertial and visual cues. Inertial motion is sensed by the labyrinth, somatic sensation by the body, and optic flow by the retina. Because the eye and head are mobile these stimuli are sensed relative to different reference frames and it remains unclear if a perception occurs in a common reference frame. Recent neurophysiologic evidence has suggested the reference frames remain separate even at higher levels of processing but has not addressed the resulting perception. Seven human subjects experienced a 2s, 16 cm/s translation and/or a visual stimulus corresponding with this translation. For each condition 72 stimuli (360° in 5° increments) were delivered in random order. After each stimulus the subject identified the perceived heading using a mechanical dial. Some trial blocks included interleaved conditions in which the influence of ±28° of gaze and/or head position were examined. The observations were fit using a two degree-of-freedom population vector decoder (PVD) model which considered the relative sensitivity to lateral motion and coordinate system offset. For visual stimuli gaze shifts caused shifts in perceived head estimates in the direction opposite the gaze shift in all subjects. These perceptual shifts averaged 13 ± 2° for eye only gaze shifts and 17 ± 2° for eye-head gaze shifts. This finding indicates visual headings are biased towards retina coordinates. Similar gaze and head direction shifts prior to inertial headings had no significant influence on heading direction. Thus inertial headings are perceived in body-centered coordinates. Combined visual and inertial stimuli yielded intermediate results.     

Multiple Strategies for Spatial Integration of 2D Layouts within Working Memory

Prior results on the spatial integration of layouts within a room differed regarding the reference frame that participants used for integration. We asked whether these differences also occur when integrating 2D screen views and, if so, what the reasons for this might be. In four experiments we showed that integrating reference frames varied as a function of task familiarity combined with processing time, cues for spatial transformation, and information about action requirements paralleling results in the 3D case. Participants saw part of an object layout in screen 1, another part in screen 2, and reacted on the integrated layout in screen 3. Layout presentations between two screens coincided or differed in orientation. Aligning misaligned screens for integration is known to increase errors/latencies. The error/latency pattern was thus indicative of the reference frame used for integration. We showed that task familiarity combined with self-paced learning, visual updating, and knowing from where to act prioritized the integration within the reference frame of the initial presentation, which was updated later, and from where participants acted respectively. Participants also heavily relied on layout intrinsic frames. The results show how humans flexibly adjust their integration strategy to a wide variety of conditions.     

The object behind the echo: dolphins (Tursiops truncatus) perceive object shape globally through echolocation

Two experiments tested a bottlenosed dolphin's ability to match objects across echolocation and vision. Matching was tested from echolocation sample to visual alternatives (E-V) and from visual sample to echolocation alternatives (V-E). In Experiment 1, the dolphin chose a match from among three-alternative objects that differed in overall (global) shape, but shared several 'local' features with the sample. The dolphin conducted a right-to-left serial nonexhaustive search among the alternatives, stopping when a match was encountered. It matched correctly on 93% of V-E trials and on 99% of E-V trials with completely novel combinations of objects despite the presence of many overlapping features. In Experiment 2, a fourth alternative was added in the form of a paddle that the dolphin could press if it decided that none of the three-alternatives matched the sample. When a match was present, the dolphin selected it on 94% of V-E trials and 95% of E-V trials. When a match was absent, the dolphin pressed the paddle on 74% and 76%, respectively, of V-E and E-V trials. The approximate 25% error rate, which consisted of a choice of one of the three non-matching alternatives in lieu of the paddle press, increased from right to center to left alternative object, reflecting successively later times in the dolphin's search path. A weakening in memory for the sample seemed the most likely cause of this error pattern. Overall, the results gave strong support to the hypothesis that the echolocating dolphin represents an object by its global appearance rather than by local features.     

Spatial modulation of primate inferotemporal responses by eye position

Background

A key aspect of representations for object recognition and scene analysis in the ventral visual stream is the spatial frame of reference, be it a viewer-centered, object-centered, or scene-based coordinate system. Coordinate transforms from retinocentric space to other reference frames involve combining neural visual responses with extraretinal postural information.

Methodology/Principal Findings

We examined whether such spatial information is available to anterior inferotemporal (AIT) neurons in the macaque monkey by measuring the effect of eye position on responses to a set of simple 2D shapes. We report, for the first time, a significant eye position effect in over 40% of recorded neurons with small gaze angle shifts from central fixation. Although eye position modulates responses, it does not change shape selectivity.

Conclusions/Significance

These data demonstrate that spatial information is available in AIT for the representation of objects and scenes within a non-retinocentric frame of reference. More generally, the availability of spatial information in AIT calls into questions the classic dichotomy in visual processing that associates object shape processing with ventral structures such as AIT but places spatial processing in a separate anatomical stream projecting to dorsal structures.     

Depth perception in disparity-defined objects: finding the balance between averaging and segregation

Deciding what constitutes an object, and what background, is an essential task for the visual system. This presents a conundrum: averaging over the visual scene is required to obtain a precise signal for object segregation, but segregation is required to define the region over which averaging should take place. Depth, obtained via binocular disparity (the differences between two eyes’ views), could help with segregation by enabling identification of object and background via differences in depth. Here, we explore depth perception in disparity-defined objects. We show that a simple object segregation rule, followed by averaging over that segregated area, can account for depth estimation errors. To do this, we compared objects with smoothly varying depth edges to those with sharp depth edges, and found that perceived peak depth was reduced for the former. A computational model used a rule based on object shape to segregate and average over a central portion of the object, and was able to emulate the reduction in perceived depth. We also demonstrated that the segregated area is not predefined but is dependent on the object shape. We discuss how this segregation strategy could be employed by animals seeking to deter binocular predators.This article is part of the themed issue ‘Vision in our three-dimensional world’.     

Flexible representations of dynamics are used in object manipulation

To manipulate an object skillfully, the brain must learn its dynamics, specifying the mapping between applied force and motion. A fundamental issue in sensorimotor control is whether such dynamics are represented in an extrinsic frame of reference tied to the object or an intrinsic frame of reference linked to the arm. Although previous studies have suggested that objects are represented in arm-centered coordinates [1-6], all of these studies have used objects with unusual and complex dynamics. Thus, it is not known how objects with natural dynamics are represented. Here we show that objects with simple (or familiar) dynamics and those with complex (or unfamiliar) dynamics are represented in object- and arm-centered coordinates, respectively. We also show that objects with simple dynamics are represented with an intermediate coordinate frame when vision of the object is removed. These results indicate that object dynamics can be flexibly represented in different coordinate frames by the brain. We suggest that with experience, the representation of the dynamics of a manipulated object may shift from a coordinate frame tied to the arm toward one that is linked to the object. The additional complexity required to represent dynamics in object-centered coordinates would be economical for familiar objects because such a representation allows object use regardless of the orientation of the object in hand.     

The role of observers' gaze behaviour when watching object manipulation tasks: predicting and evaluating the consequences of action

When watching an actor manipulate objects, observers, like the actor, naturally direct their gaze to each object as the hand approaches and typically maintain gaze on the object until the hand departs. Here, we probed the function of observers'' eye movements, focusing on two possibilities: (i) that observers'' gaze behaviour arises from processes involved in the prediction of the target object of the actor''s reaching movement and (ii) that this gaze behaviour supports the evaluation of mechanical events that arise from interactions between the actor''s hand and objects. Observers watched an actor reach for and lift one of two presented objects. The observers'' task was either to predict the target object or judge its weight. Proactive gaze behaviour, similar to that seen in self-guided action–observation, was seen in the weight judgement task, which requires evaluating mechanical events associated with lifting, but not in the target prediction task. We submit that an important function of gaze behaviour in self-guided action observation is the evaluation of mechanical events associated with interactions between the hand and object. By comparing predicted and actual mechanical events, observers, like actors, can gain knowledge about the world, including information about objects they may subsequently act upon.     

Reference frame for rapid visual processing of line orientation.

Detection of a uniquely oriented line element in a background field of uniformly oriented line elements depends on the orientation of the background field. Is the orientational reference frame for this anisotropy entirely dependent on the orientations of structures outside the line-element display, the spatial regularity of the stimulus elements, and the direction of gravity? The effects of these potential cues were investigated in target-detection experiments with brief displays. The anisotropy was found whether or not gravitational or visual cues defined an orientational reference frame. Stimulus orientation may be coded with respect to the retina or body axis in rapid visual processing.     

Directed attention eliminates 'change deafness' in complex auditory scenes

In natural environments that contain multiple sound sources, acoustic energy arising from the different sources sums to produce a single complex waveform at each of the listener's ears. The auditory system must segregate this waveform into distinct streams to permit identification of the objects from which the signals emanate [1]. Although the processes involved in stream segregation are now reasonably well understood [1, 2 and 3], little is known about the nature of our perception of complex auditory scenes. Here, we examined complex scene perception by having listeners detect a discrete change to an auditory scene comprising multiple concurrent naturalistic sounds. We found that listeners were remarkably poor at detecting the disappearance of an individual auditory object when listening to scenes containing more than four objects, but they performed near perfectly when their attention was directed to the identity of a potential change. In the absence of directed attention, this "change deafness" [4] was greater for objects arising from a common location in space than for objects separated in azimuth. Change deafness was also observed for changes in object location, suggesting that it may reflect a general effect of the dependence of human auditory perception on attention.     

Oculomotor Examination of the Weapon Focus Effect: Does a Gun Automatically Engage Visual Attention?

Background

A person is less likely to be accurately remembered if they appear in a visual scene with a gun, a result that has been termed the weapon focus effect (WFE). Explanations of the WFE argue that weapons engage attention because they are unusual and/or threatening, which causes encoding deficits for the other items in the visual scene. Previous WFE research has always embedded the weapon and nonweapon objects within a larger context that provides information about an actor''s intention to use the object. As such, it is currently unknown whether a gun automatically engages attention to a greater extent than other objects independent of the context in which it is presented.

Method

Reflexive responding to a gun compared to other objects was examined in two experiments. Experiment 1 employed a prosaccade gap-overlap paradigm, whereby participants looked toward a peripheral target, and Experiment 2 employed an antisaccade gap-overlap paradigm, whereby participants looked away from a peripheral target. In both experiments, the peripheral target was a gun or a nonthreatening object (i.e., a tomato or pocket watch). We also controlled how unexpected the targets were and compared saccadic reaction times across types of objects.

Results

A gun was not found to differentially engage attention compared to the unexpected object (i.e., a pocket watch). Some evidence was found (Experiment 2) that both the gun and the unexpected object engaged attention to a greater extent compared the expected object (i.e., a tomato).

Conclusion

An image of a gun did not engage attention to a larger extent than images of other types of objects (i.e., a pocket watch or tomato). The results suggest that context may be an important determinant of WFE. The extent to which an object is threatening may depend on the larger context in which it is presented.     

The functional neuroanatomy of implicit-motion perception or representational momentum

BACKGROUND: When we view static scenes that imply motion - such as an object dropping off a shelf - recognition memory for the position of the object is extrapolated forward. It is as if the object in our mind's eye comes alive and continues on its course. This phenomenon is known as representational momentum and results in a distortion of recognition memory in the implied direction of motion. Representational momentum is modifiable; simply labelling a drawing of a pointed object as 'rocket' will facilitate the effect, whereas the label 'steeple' will impede it. We used functional magnetic resonance imaging (fMRI) to explore the neural substrate for representational momentum. RESULTS: Subjects participated in two experiments. In the first, they were presented with video excerpts of objects in motion (versus the same objects in a resting position). This identified brain areas responsible for motion perception. In the second experiment, they were presented with still photographs of the same target items, only some of which implied motion (representational momentum stimuli). When viewing still photographs of scenes implying motion, activity was revealed in secondary visual cortical regions that overlap with areas responsible for the perception of actual motion. Additional bilateral activity was revealed within a posterior satellite of V5 for the representational momentum stimuli. Activation was also engendered in the anterior cingulate cortex. CONCLUSIONS: Considering the implicit nature of representational momentum and its modifiability, the findings suggest that higher-order semantic information can act on secondary visual cortex to alter perception without explicit awareness.     

Generalization of Dexterous Manipulation Is Sensitive to the Frame of Reference in Which It Is Learned

Studies have shown that internal representations of manipulations of objects with asymmetric mass distributions that are generated within a specific orientation are not generalizable to novel orientations, i.e., subjects fail to prevent object roll on their first grasp-lift attempt of the object following 180° object rotation. This suggests that representations of these manipulations are specific to the reference frame in which they are formed. However, it is unknown whether that reference frame is specific to the hand, the body, or both, because rotating the object 180° modifies the relation between object and body as well as object and hand. An alternative, untested explanation for the above failure to generalize learned manipulations is that any rotation will disrupt grasp performance, regardless if the reference frame in which the manipulation was learned is maintained or modified. We examined the effect of rotations that (1) maintain and (2) modify relations between object and body, and object and hand, on the generalizability of learned two-digit manipulation of an object with an asymmetric mass distribution. Following rotations that maintained the relation between object and body and object and hand (e.g., rotating the object and subject 180°), subjects continued to use appropriate digit placement and load force distributions, thus generating sufficient compensatory moments to minimize object roll. In contrast, following rotations that modified the relation between (1) object and hand (e.g. rotating the hand around to the opposite object side), (2) object and body (e.g. rotating subject and hand 180°), or (3) both (e.g. rotating the subject 180°), subjects used the same, yet inappropriate digit placement and load force distribution, as those used prior to the rotation. Consequently, the compensatory moments were insufficient to prevent large object rolls. These findings suggest that representations of learned manipulation of objects with asymmetric mass distributions are specific to the body- and hand-reference frames in which they were learned.     

Predictive adjustment of the perceived direction of gaze during saccadic eye movements

When we look at a stationary object, the perceived direction of gaze (where we are looking) is aligned with the physical direction of eyes (where our eyes are oriented) by which the object is foveated. However, this alignment may not hold in a dynamic situation. Our experiments assessed the perceived locations of two brief stimuli (1 ms) simultaneously displayed at two different physical locations during a saccade. The first stimulus was in the instantaneous location to which the eyes were oriented and the second one was always in the same location as the initial fixation point. When the timing of these stimuli was changed intra-saccadically, their perceived locations were dissociated. The first stimuli were consistently perceived near the target that will be foveated at saccade termination. The second stimuli once perceived near the target location, shifted in the direction opposite to that of saccades, as its latency from saccades increased. These results suggested an independent adjustment of gaze orientation from the physical orientation of eyes during saccades. The spatial dissociation of two stimuli may reflect sensorimotor control of gaze during saccades.     

Effects of Haptic Information on the Perception of Dynamic 3-D Movement

This study examined effects of hand movement on visual perception of 3-D movement. I used an apparatus in which a cursor position in a simulated 3-D space and the position of a stylus on a haptic device could coincide using a mirror. In three experiments, participants touched the center of a rectangle in the visual display with the stylus of the force-feedback device. Then the rectangle''s surface stereoscopically either protruded toward a participant or indented away from the participant. Simultaneously, the stylus either pushed back participant''s hand, pulled away, or remained static. Visual and haptic information were independently manipulated. Participants judged whether the rectangle visually protruded or dented. Results showed that when the hand was pulled away, subjects were biased to perceive rectangles indented; however, when the hand was pushed back, no effect of haptic information was observed (Experiment 1). This effect persisted even when the cursor position was spatially separated from the hand position (Experiment 2). But, when participants touched an object different from the visual stimulus, this effect disappeared (Experiment 3). These results suggest that the visual system tried to integrate the dynamic visual and haptic information when they coincided cognitively, and the effect of haptic information on visually perceived depth was direction-dependent.     

Temporal brain dynamics of multiple object processing: the flexibility of individuation

The ability to process concurrently multiple visual objects is fundamental for a coherent perception of the world. A core component of this ability is the simultaneous individuation of multiple objects. Many studies have addressed the mechanism of object individuation but it remains unknown whether the visual system mandatorily individuates all relevant elements in the visual field, or whether object indexing depends on task demands. We used a neural measure of visual selection, the N2pc component, to evaluate the flexibility of multiple object individuation. In three ERP experiments, participants saw a variable number of target elements among homogenous distracters and performed either an enumeration task (Experiment 1) or a detection task, reporting whether at least one (Experiment 2) or a specified number of target elements (Experiment 3) was present. While in the enumeration task the N2pc response increased as a function of the number of targets, no such modulation was found in Experiment 2, indicating that individuation of multiple targets is not mandatory. However, a modulation of the N2pc similar to the enumeration task was visible in Experiment 3, further highlighting that object individuation is a flexible mechanism that binds indexes to object properties and locations as needed for further object processing.