Perception of solid shape from shading

Observers judged the slants and tilts of numerous regions within shaded images of ellipsoid surfaces that varied in shape, orientation, surface reflectance, and direction of illumination. The perceived three-dimensional structure of each surface was calculated from these judgments. Much of the error in observers' responses resulted from a tendency to perceive surfaces whose axes were aligned with the display screen. The presence of specular highlights or cast shadows, in contrast, had no effect on performance. The results of the experiment indicate that several assumption of certain formal models for perception of shape from shading are not psychologically valid. The most notable of these assumptions are that the visual system initially assumes that all surfaces have Lambertian reflectance and that illuminant direction must be known before shape detection can proceed. These assumptions are often accompanied by a third assumption that surface orientation is detected locally, and global shape determined by smoothing over local surface orientation estimates. The present experiment indicates that an alternative approach offered by Koenderink and van Doorn may be more psychologically accurate, as it avoids all three assumptions.Supported in part by the Air Force Office of Scientific Research (AFOSR 82-0148). The research reported in this paper was performed while the first author was a graduate student at the University of Connecticut at Storrs     

Learning to use illumination gradients as an unambiguous cue to three dimensional shape

The luminance and colour gradients across an image are the result of complex interactions between object shape, material and illumination. Using such variations to infer object shape or surface colour is therefore a difficult problem for the visual system. We know that changes to the shape of an object can affect its perceived colour, and that shading gradients confer a sense of shape. Here we investigate if the visual system is able to effectively utilise these gradients as a cue to shape perception, even when additional cues are not available. We tested shape perception of a folded card object that contained illumination gradients in the form of shading and more subtle effects such as inter-reflections. Our results suggest that observers are able to use the gradients to make consistent shape judgements. In order to do this, observers must be given the opportunity to learn suitable assumptions about the lighting and scene. Using a variety of different training conditions, we demonstrate that learning can occur quickly and requires only coarse information. We also establish that learning does not deliver a trivial mapping between gradient and shape; rather learning leads to the acquisition of assumptions about lighting and scene parameters that subsequently allow for gradients to be used as a shape cue. The perceived shape is shown to be consistent for convex and concave versions of the object that exhibit very different shading, and also similar to that delivered by outline, a largely unrelated cue to shape. Overall our results indicate that, although gradients are less reliable than some other cues, the relationship between gradients and shape can be quickly assessed and the gradients therefore used effectively as a visual shape cue.     

Perceiving Object Shape from Specular Highlight Deformation,Boundary Contour Deformation,and Active Haptic Manipulation

It is well known that motion facilitates the visual perception of solid object shape, particularly when surface texture or other identifiable features (e.g., corners) are present. Conventional models of structure-from-motion require the presence of texture or identifiable object features in order to recover 3-D structure. Is the facilitation in 3-D shape perception similar in magnitude when surface texture is absent? On any given trial in the current experiments, participants were presented with a single randomly-selected solid object (bell pepper or randomly-shaped “glaven”) for 12 seconds and were required to indicate which of 12 (for bell peppers) or 8 (for glavens) simultaneously visible objects possessed the same shape. The initial single object’s shape was defined either by boundary contours alone (i.e., presented as a silhouette), specular highlights alone, specular highlights combined with boundary contours, or texture. In addition, there was a haptic condition: in this condition, the participants haptically explored with both hands (but could not see) the initial single object for 12 seconds; they then performed the same shape-matching task used in the visual conditions. For both the visual and haptic conditions, motion (rotation in depth or active object manipulation) was present in half of the trials and was not present for the remaining trials. The effect of motion was quantitatively similar for all of the visual and haptic conditions–e.g., the participants’ performance in Experiment 1 was 93.5 percent higher in the motion or active haptic manipulation conditions (when compared to the static conditions). The current results demonstrate that deforming specular highlights or boundary contours facilitate 3-D shape perception as much as the motion of objects that possess texture. The current results also indicate that the improvement with motion that occurs for haptics is similar in magnitude to that which occurs for vision.     

Cuttlefish see shape from shading,fine-tuning coloration in response to pictorial depth cues and directional illumination

Humans use shading as a cue to three-dimensional form by combining low-level information about light intensity with high-level knowledge about objects and the environment. Here, we examine how cuttlefish Sepia officinalis respond to light and shadow to shade the white square (WS) feature in their body pattern. Cuttlefish display the WS in the presence of pebble-like objects, and they can shade it to render the appearance of surface curvature to a human observer, which might benefit camouflage. Here we test how they colour the WS on visual backgrounds containing two-dimensional circular stimuli, some of which were shaded to suggest surface curvature, whereas others were uniformly coloured or divided into dark and light semicircles. WS shading, measured by lateral asymmetry, was greatest when the animal rested on a background of shaded circles and three-dimensional hemispheres, and less on plain white circles or black/white semicircles. In addition, shading was enhanced when light fell from the lighter side of the shaded stimulus, as expected for real convex surfaces. Thus, the cuttlefish acts as if it perceives surface curvature from shading, and takes account of the direction of illumination. However, the direction of WS shading is insensitive to the directions of background shading and illumination; instead the cuttlefish tend to turn to face the light source.     

Selection of visual information for lightness judgements by eye movements

When judging the lightness of objects, the visual system has to take into account many factors such as shading, scene geometry, occlusions or transparency. The problem then is to estimate global lightness based on a number of local samples that differ in luminance. Here, we show that eye fixations play a prominent role in this selection process. We explored a special case of transparency for which the visual system separates surface reflectance from interfering conditions to generate a layered image representation. Eye movements were recorded while the observers matched the lightness of the layered stimulus. We found that observers did focus their fixations on the target layer, and this sampling strategy affected their lightness perception. The effect of image segmentation on perceived lightness was highly correlated with the fixation strategy and was strongly affected when we manipulated it using a gaze-contingent display. Finally, we disrupted the segmentation process showing that it causally drives the selection strategy. Selection through eye fixations can so serve as a simple heuristic to estimate the target reflectance.     

Depth perception in disparity-defined objects: finding the balance between averaging and segregation

Deciding what constitutes an object, and what background, is an essential task for the visual system. This presents a conundrum: averaging over the visual scene is required to obtain a precise signal for object segregation, but segregation is required to define the region over which averaging should take place. Depth, obtained via binocular disparity (the differences between two eyes’ views), could help with segregation by enabling identification of object and background via differences in depth. Here, we explore depth perception in disparity-defined objects. We show that a simple object segregation rule, followed by averaging over that segregated area, can account for depth estimation errors. To do this, we compared objects with smoothly varying depth edges to those with sharp depth edges, and found that perceived peak depth was reduced for the former. A computational model used a rule based on object shape to segregate and average over a central portion of the object, and was able to emulate the reduction in perceived depth. We also demonstrated that the segregated area is not predefined but is dependent on the object shape. We discuss how this segregation strategy could be employed by animals seeking to deter binocular predators.This article is part of the themed issue ‘Vision in our three-dimensional world’.     

Contextual Effects of Scene on the Visual Perception of Object Orientation in Depth

We investigated the effect of background scene on the human visual perception of depth orientation (i.e., azimuth angle) of three-dimensional common objects. Participants evaluated the depth orientation of objects. The objects were surrounded by scenes with an apparent axis of the global reference frame, such as a sidewalk scene. When a scene axis was slightly misaligned with the gaze line, object orientation perception was biased, as if the gaze line had been assimilated into the scene axis (Experiment 1). When the scene axis was slightly misaligned with the object, evaluated object orientation was biased, as if it had been assimilated into the scene axis (Experiment 2). This assimilation may be due to confusion between the orientation of the scene and object axes (Experiment 3). Thus, the global reference frame may influence object orientation perception when its orientation is similar to that of the gaze-line or object.     

Where Do the Eyes Really Go in the Hollow-Face Illusion?

The hollow-face illusion refers to the finding that people typically perceive a concave (hollow) mask as being convex, despite the presence of binocular disparity cues that indicate the contrary. Unlike other illusions of depth, recent research has suggested that the eyes tend to converge at perceived, rather than actual, depths. However, technical and methodological limitations prevented one from knowing whether disparity cues may still have influenced vergence. In the current study, we presented participants with virtual normal or hollow masks and asked them to fixate the tip of the face's nose until they had indicated whether they perceived it as pointing towards or away from them. The results showed that the direction of vergence was indeed determined by perceived depth, although vergence responses were both somewhat delayed and of smaller amplitude (by a factor of about 0.5) for concave than convex masks. These findings demonstrate how perceived depth can override disparity cues when it comes to vergence, albeit not entirely.     

Object Recognition in Flight: How Do Bees Distinguish between 3D Shapes?

Honeybees (Apis mellifera) discriminate multiple object features such as colour, pattern and 2D shape, but it remains unknown whether and how bees recover three-dimensional shape. Here we show that bees can recognize objects by their three-dimensional form, whereby they employ an active strategy to uncover the depth profiles. We trained individual, free flying honeybees to collect sugar water from small three-dimensional objects made of styrofoam (sphere, cylinder, cuboids) or folded paper (convex, concave, planar) and found that bees can easily discriminate between these stimuli. We also tested possible strategies employed by the bees to uncover the depth profiles. For the card stimuli, we excluded overall shape and pictorial features (shading, texture gradients) as cues for discrimination. Lacking sufficient stereo vision, bees are known to use speed gradients in optic flow to detect edges; could the bees apply this strategy also to recover the fine details of a surface depth profile? Analysing the bees’ flight tracks in front of the stimuli revealed specific combinations of flight maneuvers (lateral translations in combination with yaw rotations), which are particularly suitable to extract depth cues from motion parallax. We modelled the generated optic flow and found characteristic patterns of angular displacement corresponding to the depth profiles of our stimuli: optic flow patterns from pure translations successfully recovered depth relations from the magnitude of angular displacements, additional rotation provided robust depth information based on the direction of the displacements; thus, the bees flight maneuvers may reflect an optimized visuo-motor strategy to extract depth structure from motion signals. The robustness and simplicity of this strategy offers an efficient solution for 3D-object-recognition without stereo vision, and could be employed by other flying insects, or mobile robots.     

Colour contrast influences perceived shape in combined shading and texture patterns

The 'colour-shading effect' describes the phenomenon whereby a chromatic pattern influences perceived shape-from-shading in a luminance pattern. Specifically, the depth corrugations perceived in sinusoidal luminance gratings can be enhanced by spatially non-aligned, and suppressed by spatially aligned sinusoidal chromatic gratings. Here we examine whether colour contrast can influence perceived shape in patterns that combine shape-from-shading with shape-from-texture. Stimuli consisted of sinusoidal modulations of texture (defined by orientation), luminance and colour. When the texture and luminance modulations were suitably combined, one obtained a vivid impression of a corrugated depth surface. The addition of a colour grating to the texture-luminance combination was found to enhance the impression of depth when out-of-phase with the luminance modulation, and suppress the impression of depth when in-phase with the luminance modulation. The degree of depth enhancement and depth suppression was approximately constant across texture amplitude when measured linearly. In the absence of the luminance grating however, the colour grating had no phase-dependent affect on perceived depth. These results show that colour contrast modulates the contribution of shading to perceived shape in combined shading and texture patterns.     

Unifying shading and texture through an active observer

Shading (variations of image intensity) provides an important cue for understanding the shape of three-dimensional surfaces from monocular views. On the other hand, texture (distribution of discontinuities on the surface) is a strong cue for recovering surface orientation by using monocular images. But given the image of an object or scene, what technique should we use to recover the shape of what is image? Resolution of shape from shading requires knowledge of the reflectance of the imaged surface and, usually, the fact that it is smooth (i.e. it shows no discontinuities). Determination of shape from texture requires knowledge of the distribution of surface markings (i.e. discontinuities). One might expect that one method would work when the other does not. I present a theory on how an active observer can determine shape from the image of an object or scene regardless of whether the image is shaded, textured, or both, and without any knowledge of reflectance maps or the distribution of surface markings. The approach is successful because the active observer is able to manipulate the constraints behind the perceptual phenomenon at hand and thus derive a simple solution. Several experimental results are presented with real and synthetic images.     

An advantage for concavities in shape perception by chimpanzees (Pan troglodytes)

The significance of concavity in object shape perception by chimpanzees (Pan troglodytes) was investigated in a matching-to-sample procedure. For the task, chimpanzees were required to choose a polygon stimulus that was identical in shape to a sample. The incorrect alternative was defined by the addition or subtraction of a concave or convex apex. Chimpanzees were more sensitive to the concave deformation than to the convex deformation. This tendency conforms to the theories of human visual perception that have treated concave features as important factors in reconstructing three-dimensional structures from two-dimensional images. Our results suggest that shape representation in chimpanzees is similar to that in humans and that chimpanzees visually process two-dimensional images in the same manner as humans.     

Depth Structure from Asymmetric Shading Supports Face Discrimination

To examine the effect of illumination direction on the ability of observers to discriminate between faces, we manipulated the direction of illumination on scanned 3D face models. In order to dissociate the surface reflectance and illumination components of front-view face images, we introduce a symmetry algorithm that can separate the symmetric and asymmetric components of the face in both low and high spatial frequency bands. Based on this approach, hybrid faces stimuli were constructed with different combinations of symmetric and asymmetric spatial content. Discrimination results with these images showed that asymmetric illumination information biased face perception toward the structure of the shading component, while the symmetric illumination information had little, if any, effect. Measures of perceived depth showed that this property increased systematically with the asymmetric but not the symmetric low spatial frequency component. Together, these results suggest that (1) the asymmetric 3D shading information dramatically affects both the perceived facial information and the perceived depth of the facial structure; and (2) these effects both increase as the illumination direction is shifted to the side. Thus, our results support the hypothesis that face processing has a strong 3D component.     

Art pieces that 'move' in our minds--an explanation of illusory motion based on depth reversal

Certain art forms, such as Patrick Hughes's 'reverspectives', Dick Termes's 'Termespheres', intaglios, and hollow masks, appear to move vividly as viewers move in front of them, even though they are stationary. This illusory motion is accompanied by a perceived reversal of depth, where physical convex and concave surfaces are falsely seen as concave and convex, respectively. A geometric explanation is presented that considers this illusory motion as a result of the perceived depth reversal. The main argument is that the visual system constructs a three-dimensional representation of the surfaces, and that this representation is one of the sources that contribute to the illusory motion, together with vestibular signals of self-motion and signals of eye movements. This explanation is extended to stereograms that are also known to appear to move as viewers move in front of them. A quantitative model can be developed around this geometric explanation to examine the extent to which the visual system tolerates large distortions in size and shape and still maintains the illusion.     

Effect of pictorial depth cues, binocular disparity cues and motion parallax depth cues on lightness perception in three-dimensional virtual scenes

Background

Surface lightness perception is affected by scene interpretation. There is some experimental evidence that perceived lightness under bi-ocular viewing conditions is different from perceived lightness in actual scenes but there are also reports that viewing conditions have little or no effect on perceived color. We investigated how mixes of depth cues affect perception of lightness in three-dimensional rendered scenes containing strong gradients of illumination in depth.

Methodology/Principal Findings

Observers viewed a virtual room (4 m width×5 m height×17.5 m depth) with checkerboard walls and floor. In four conditions, the room was presented with or without binocular disparity (BD) depth cues and with or without motion parallax (MP) depth cues. In all conditions, observers were asked to adjust the luminance of a comparison surface to match the lightness of test surfaces placed at seven different depths (8.5–17.5 m) in the scene. We estimated lightness versus depth profiles in all four depth cue conditions. Even when observers had only pictorial depth cues (no MP, no BD), they partially but significantly discounted the illumination gradient in judging lightness. Adding either MP or BD led to significantly greater discounting and both cues together produced the greatest discounting. The effects of MP and BD were approximately additive. BD had greater influence at near distances than far.

Conclusions/Significance

These results suggest the surface lightness perception is modulated by three-dimensional perception/interpretation using pictorial, binocular-disparity, and motion-parallax cues additively. We propose a two-stage (2D and 3D) processing model for lightness perception.     

Shape and orientation of articular facets of cervical vertebrae (C3-C7) in dogs denoting axial rotational ability: an osteological study

Macerated cervical vertebrae of 139 adult and 39 juvenile dogs were assessed for morphological evidence of axial rotation, which is denoted by the shape and orientation of the articular facets. The potential effect of the presence of caudal costal foveae at C7 on facet geometry was also investigated. Four variations of caudal facet shape were evident, namely, plane, concave, convex and sigmoid. The earliest onset of change in shape from plane to curved was noticed in juvenile dogs who were older than 8 weeks. The incidence of curved facets denoting axial rotation ability was observed to increase on descending down the cervical spine. Concave caudal facets were more frequently present in males and in large dogs (P < 0.001), compared to Dachshunds and small breeds. The degree of concavity was not related to age but was associated with the transverse distance between the most medial aspects of the caudal and corresponding cranial facets (P < 0.05). In large breeds, the presence of concave caudal facets at C7 was inversely related to the presence of caudal costal foveae (P < 0.01). The effect of axial rotation is discussed in context with clinical literature suggesting that axial rotation might attribute to the development of the wobbler syndrome.     

Shading Beats Binocular Disparity in Depth from Luminance Gradients: Evidence against a Maximum Likelihood Principle for Cue Combination

Perceived depth is conveyed by multiple cues, including binocular disparity and luminance shading. Depth perception from luminance shading information depends on the perceptual assumption for the incident light, which has been shown to default to a diffuse illumination assumption. We focus on the case of sinusoidally corrugated surfaces to ask how shading and disparity cues combine defined by the joint luminance gradients and intrinsic disparity modulation that would occur in viewing the physical corrugation of a uniform surface under diffuse illumination. Such surfaces were simulated with a sinusoidal luminance modulation (0.26 or 1.8 cy/deg, contrast 20%-80%) modulated either in-phase or in opposite phase with a sinusoidal disparity of the same corrugation frequency, with disparity amplitudes ranging from 0’-20’. The observers’ task was to adjust the binocular disparity of a comparison random-dot stereogram surface to match the perceived depth of the joint luminance/disparity-modulated corrugation target. Regardless of target spatial frequency, the perceived target depth increased with the luminance contrast and depended on luminance phase but was largely unaffected by the luminance disparity modulation. These results validate the idea that human observers can use the diffuse illumination assumption to perceive depth from luminance gradients alone without making an assumption of light direction. For depth judgments with combined cues, the observers gave much greater weighting to the luminance shading than to the disparity modulation of the targets. The results were not well-fit by a Bayesian cue-combination model weighted in proportion to the variance of the measurements for each cue in isolation. Instead, they suggest that the visual system uses disjunctive mechanisms to process these two types of information rather than combining them according to their likelihood ratios.     

Visual effects of haptic feedback are large but local

Vision generally provides reliable predictions for touch and motor-control, but some classes of stimuli evoke visual illusions. Using haptic feedback on virtual 3-D surfaces, we tested the function of touch in such cases. Our experiments show that in the perception of 3-D shapes from texture cues, haptic information can dominate vision in some cases, changing percepts qualitatively from convex to concave and concave to slant. The effects take time to develop, do not outlive the cessation of the feedback, are attenuated by distance, and drastically reduced by gaps in the surface. These dynamic shifts in qualitative perceived shapes could be invaluable in neural investigations that test whether haptic feedback modifies selective activation of neurons or changes the shape-tuning of neurons responsible for percepts of 3-D shapes.     

Neural response to perception of volume in the lateral occipital complex

Projection of a 3D scene onto the 2D retina necessarily entails a loss of information, yet perceivers experience a world populated with volumetric objects. Using simultaneous behavioral and neural (fMRI) measures, we identify neural bases of volume perception. Neural activity in the lateral occipital cortex increased with presentation of 3D volumes relative to presentation of 2D shapes. Neural activity also modulated with perceived volume, independent of image information. When behavioral responses indicated that observers saw ambiguous images as 3D volumes, neural response increased; when behavioral data revealed a 2D interpretation, neural response waned. Crucially, the physical stimulus was identical under both interpretations; only the percept of volume can account for the increased neural activity.