Visual perception: bizarre contours go against the odds

A new study shows that the brain sometimes invents visual contours even when they would be highly unlikely to occur in the real world. This presents a challenge to theories assuming that the brain prefers the most probable interpretation of the retinal image.     

Vision without Frames: A Semiotic Paradigm of Event Based Computer Vision

Conventional imagers and almost all vision processes use and rely on theories that are based on the principle of static image-frames. A frame is a 2D matrix that represents the spatial locations of intensities of a scene projected on the imager. The notion of a frame itself is so embedded in machine vision, that it is usually taken for granted that this is how biological systems store light information. This paper presents a biosinpired event-based image formation principle, which output data rely on an asynchronous acquisition process. The generated information is stored in temporal volumes, which size and information depend only on the dynamic content of observed scenes. Practical analysis of such information will shows that the processing of visual information can only be based on a semiotic process. The paper also provides a general definition of the notion of visual features as the interpretation of signs according to different possible readings of the codified visual signal.     

Disambiguating ambiguous figures by a model of selective attention

Ambiguous figures are visual stimuli which are interpreted multiply by the human visual system. A model is proposed which disambiguates the ambiguous figures. The model was formulated based on the characteristics of visual information processing, accompanied with selective attention. In the ambiguous figure my husband and my father-in-law, it was necessary to simulate visual information processing so that attention was directed to the multiple features in the figure to disambiguate the ambiguous figure. Pictures, obtained from the model, were examined as to whether they were interpreted unambiguously or not. Results show that the model, simulated selective attention, can disambiguate the ambiguous figures. This suggests that the image per se, viewed through selective attention, becomes unambiguous before the figure is interpreted in the higher level. Results also show that the computer simulation of selective attention would make it possible to examine factors affecting the initial interpretation of the figure.     

Medical images and automated interpretation

The application of automated interpretation to medical images is discussed and the main methods of medical imaging are briefly described. The factors behind the process of human clinical interpretation are also considered. Because human interpretation can be aided by processing of the raw image, the standard methods of image processing are mentioned. Automated interpretation of images in other fields is relevant, and segmentation of images is an important initial part of the process. There are already some applications of automated interpretation of medical images. Some are more complete than others. This will undoubtedly be an important developing field of work which will draw on experience from other areas, and be spurred on by the increasing complexity of medical imaging methods and shortage of expertise for human interpretation.     

Estimation of the Horizon in Photographed Outdoor Scenes by Human and Machine

We present three experiments on horizon estimation. In Experiment 1 we verify the human ability to estimate the horizon in static images from only visual input. Estimates are given without time constraints with emphasis on precision. The resulting estimates are used as baseline to evaluate horizon estimates from early visual processes. Stimuli are presented for only ms and then masked to purge visual short-term memory and enforcing estimates to rely on early processes, only. The high agreement between estimates and the lack of a training effect shows that enough information about viewpoint is extracted in the first few hundred milliseconds to make accurate horizon estimation possible. In Experiment 3 we investigate several strategies to estimate the horizon in the computer and compare human with machine “behavior” for different image manipulations and image scene types.     

Dual-Force ISOMAP: A New Relevance Feedback Method for Medical Image Retrieval

With great potential for assisting radiological image interpretation and decision making, content-based image retrieval in the medical domain has become a hot topic in recent years. Many methods to enhance the performance of content-based medical image retrieval have been proposed, among which the relevance feedback (RF) scheme is one of the most promising. Given user feedback information, RF algorithms interactively learn a user’s preferences to bridge the “semantic gap” between low-level computerized visual features and high-level human semantic perception and thus improve retrieval performance. However, most existing RF algorithms perform in the original high-dimensional feature space and ignore the manifold structure of the low-level visual features of images. In this paper, we propose a new method, termed dual-force ISOMAP (DFISOMAP), for content-based medical image retrieval. Under the assumption that medical images lie on a low-dimensional manifold embedded in a high-dimensional ambient space, DFISOMAP operates in the following three stages. First, the geometric structure of positive examples in the learned low-dimensional embedding is preserved according to the isometric feature mapping (ISOMAP) criterion. To precisely model the geometric structure, a reconstruction error constraint is also added. Second, the average distance between positive and negative examples is maximized to separate them; this margin maximization acts as a force that pushes negative examples far away from positive examples. Finally, the similarity propagation technique is utilized to provide negative examples with another force that will pull them back into the negative sample set. We evaluate the proposed method on a subset of the IRMA medical image dataset with a RF-based medical image retrieval framework. Experimental results show that DFISOMAP outperforms popular approaches for content-based medical image retrieval in terms of accuracy and stability.     

Human psychophysics: Functional interpretation for contrast sensitivity versus spatial frequency curve

The hypothesis that neural processing in the human visual pathways compensates for both optical degradation as well as noise contamination at the photoreceptor level is introduced and shown to be consistent with the high frequency portion of the contrast sensitivity function for threshold detection of sinusoidal gratings in addition to the suprathreshold phenomenon of matching sinusoidal gratings of different spatial frequencies. This offers a unifying interpretation for why, at threshold conditions, the high spatial frequency portion of the image is blurred as severely by the nervous system as it is by the optics (e.g. Campbell and Green, 1965) while in extreme suprathreshold conditions the nervous system effectively deblurs the image (e.g. Georgeson and sullivan, 1975; Kulikowski, 1976). These conclusions do not necessitate a highly specific form of visual processing such as Fourier channeling.This research was conducted at Yale University, Department of Ophthalmology and Visual Science, New Haven, Connecticut, USA, throughout which period A.W.S. was a John Simon Guggenheim fellow     

A model for the estimate of local image velocity by cells in the visual cortex

Some computational theories of motion perception assume that the first stage en route to this perception is the local estimate of image velocity. However, this assumption is not supported by data from the primary visual cortex. Its motion sensitive cells are not selective to velocity, but rather are directionally selective and tuned to spatio-temporal frequencies. Accordingly, physiologically based theories start with filters selective to oriented spatio-temporal frequencies. This paper shows that computational and physiological theories do not necessarily conflict, because such filters may, as a population, compute velocity locally. To prove this point, we show how to combine the outputs of a class of frequency tuned filters to detect local image velocity. Furthermore, we show that the combination of filters may simulate 'Pattern' cells in the middle temporal area (MT), whereas each filter simulates primary visual cortex cells. These simulations include three properties of the primary cortex. First, the spatio-temporal frequency tuning curves of the individual filters display approximate space-time separability. Secondly, their direction-of-motion tuning curves depend on the distribution of orientations of the components of the Fourier decomposition and speed of the stimulus. Thirdly, the filters show facilitation and suppression for responses to apparent motions in the preferred and null directions, respectively. It is suggested that the MT's role is not to solve the aperture problem, but to estimate velocities from primary cortex information. The spatial integration that accounts for motion coherence may be postponed to a later cortical stage.     

Productive Perception

Many generations of psychologists have been concerned with the problem of how we see things as they are in reality. The science of vision has accumulated extensive material on this subject; and many more or less correct hypotheses have been proposed, formulated in the various conceptual terms of philosophy, art, psychology, physiology, biophysics, mathematics, and technology. Investigation into the working of the visual system has not developed equally in these fields. One instructive fact will not go unnoticed in the future history of the science of vision: in psychology, interest in image phenomena has fallen sharply, first because of behaviorism, then because of too broad an interpretation of the classic teachings on conditioned reflexes, and later because of cybernetic ideas. It has been maintained probably in a single area, psychiatry, in which investigative and medical material has not permitted image and hallucinatory phenomena to be ignored. In the early 1960s representatives of various psychological and physiological schools and disciplines returned anew to the long-for gotten topic of image phenomena. As Holt has aptly remarked (15), images are returning from exile. As generally happens in science, the rekindling of interest in a traditional topic occurred through the internal logic of studying the problems of perception in the light of the relatively new fields in science and technology. If we trace the development of only some of them — design theory, engineering psychology, and the theory of creative activity (as a separate subdivision of art, science, and the psychology of creativity) — it becomes clear that the return to image phenomena is due to the inability of behaviorism and associated physiological conditioning theories to explain complex forms of behavior and mental activity.     

Discrimination strategies of humans and rhesus monkeys for complex visual displays

By learning to discriminate among visual stimuli, human observers can become experts at specific visual tasks. The same is true for Rhesus monkeys, the major animal model of human visual perception. Here, we systematically compare how humans and monkeys solve a simple visual task. We trained humans and monkeys to discriminate between the members of small natural-image sets. We employed the "Bubbles" procedure to determine the stimulus features used by the observers. On average, monkeys used image features drawn from a diagnostic region covering about 7% +/- 2% of the images. Humans were able to use image features drawn from a much larger diagnostic region covering on average 51% +/- 4% of the images. Similarly for the two species, however, about 2% of the image needed to be visible within the diagnostic region on any individual trial for correct performance. We characterize the low-level image properties of the diagnostic regions and discuss individual differences among the monkeys. Our results reveal that monkeys base their behavior on confined image patches and essentially ignore a large fraction of the visual input, whereas humans are able to gather visual information with greater flexibility from large image regions.     

Comparative measurement of visual stability in Earth and cosmic space.

Three theories have been suggested as to the cause of space motion sickness: 1) eye and vestibular sensory mismatch, 2) abnormal shift of body fluids producing increased intracranial pressure and 3) pre-warning signals for unpleasant physical situations by self-produced neurotoxic substances released in the body. We are interested in the possible functional disabilities/incongruities of eye, head and body movements in 0-G. Space motion sickness might be explained from the viewpoint of lack of coordination of the movements of the eye and head. It is important to ascertain the significance of gravity in the maintenance of human visual stability. We will examine the coordination of Japanese Payload Specialist (JPS) eye and head movement by electrooculogram and neck muscle electromyogram recordings, as well as obtaining a subjective evaluation of visual stability from the PS during space flight. We hypothesize that 1) poor performance of the eye movement will be observed, 2) unusual neck muscle activity will be observed and 3) there will be decreased visual stability in micro gravity. We obtained all digital data and VCR taped image data in [TEXT MISSING]     

Stereoscopic surface perception

Physiological, computational, and psychophysical studies of stereopsis have assumed that the perceived surface structure of binocularly viewed images is primarily specified by the pattern of binocular disparities in the two eyes' views. A novel set of stereoscopic phenomena are reported that demonstrate the insufficiency of this view. It is shown that the visual system computes the contrast relationships along depth discontinuities to infer the depth, lightness, and opacity of stereoscopically viewed surfaces. A novel theoretical framework is introduced to explain these results. It is argued that the visual system contains mechanisms that enforce two principles of scene interpretation: a generic view principle that determines qualitative scene geometry, and anchoring principles that determine how image data are quantitatively partitioned between different surface attributes.     

The evolution of the human mind and logic--mathematics structures

The evolution of the human mind is discussed based on: (i) the fact that living beings interchange matter, energy and information with their environment, (ii) an ontological interpretation of the "reality" of the quantum world, of which logic-mathematics structures are considered constitutive parts, (iii) recent theories according to which living beings are considered as dynamic complex systems organized by information, and (iv) the fact that the evolution of living beings is guided by information about the environment and by intrinsic information on living systems (auto-organization). Assuming the evolution of vision as a model we observe that the driving forces that directed the evolution of the eyes, as dynamic complex systems, are the information about the environment supplied by sunlight and the intrinsic information-gaining mechanism of living organisms. Thus, there exists a convergence toward a visual system with the greatest ability to obtain light information, like the human eye, and also a divergence that leads to the development of specific qualities in some species. As in the case of vision the evolution of the human mind-brain cannot be a consequence of factors unrelated to the object of its own functioning. The human mind was structured for the acquisition from reality of the logic-mathematics structures that underlie the whole universe and consequently of an internal representation of the external world and of its own self. Thus, these structures are, together with the intrinsic capacity for auto-organization of the human brain, the predominant driving force of the human mind evolution. Both factors are complementary.     

A computational model to link psychophysics and cortical cell activation patterns in human texture processing

The human visual system uses texture information to automatically, or pre-attentively, segregate parts of the visual scene. We investigate the neural substrate underlying human texture processing using a computational model that consists of a hierarchy of bi-directionally linked model areas. The model builds upon two key hypotheses, namely that (i) texture segregation is based on boundary detection--rather than clustering of homogeneous items--and (ii) texture boundaries are detected mainly on the basis of a large scenic context that is analyzed by higher cortical areas within the ventral visual pathway, such as area V4. Here, we focus on the interpretation of key results from psychophysical studies on human texture segmentation. In psychophysical studies, texture patterns were varied along several feature dimensions to systematically characterize human performance. We use simulations to demonstrate that the activation patterns of our model directly correlate with the psychophysical results. This allows us to identify the putative neural mechanisms and cortical key areas which underlie human behavior. In particular, we investigate (i) the effects of varying texture density on target saliency, and the impact of (ii) element alignment and (iii) orientation noise on the detectability of a pop-out bar. As a result, we demonstrate that the dependency of target saliency on texture density is linked to a putative receptive field organization of orientation-selective neurons in V4. The effect of texture element alignment is related to grouping mechanisms in early visual areas. Finally, the modulation of cell activity by feedback activation from higher model areas, interacting with mechanisms of intra-areal center-surround competition, is shown to result in the specific suppression of noise-related cell activities and to improve the overall model capabilities in texture segmentation. In particular, feedback interaction is crucial to raise the model performance to the level of human observers.     

Human stereovision without localized image features

Many theories of human stereovision are based on feature matching and the related correspondence problem. In this paper, we present psychophysical experiments indicating that localized image features such as Laplacian zerocrossings, intensity extrema, or centroids are not necessary for binocular depth perception. Smooth one-dimensional intensity profiles were combined into stereograms with mirror-symmetric half-images such that these localized image features were either absent or did not carry stereo information. In a discrimination task, subjects were asked to distinguish between stereograms differing only by an exchange of these half-images (ortho- vs. pseudoscopic stereograms). In a depth ordering task, subjects had to judge which of the two versions appeared in front. Subjects are able to solve both tasks even in the absence of the mentioned image features. The performance is compared to various possible stereo mechanisms. We conclude that localized image features and the correspondences between them are not necessary to perceive stereoscopic depth. One mechanism accounting for our data is correlation or mean square difference. Received: 8 February 1994 / Accepted in revised form: 15 September 1994     

Feature-based attention: it is all bottom-up priming

Feature-based attention (FBA) enhances the representation of image characteristics throughout the visual field, a mechanism that is particularly useful when searching for a specific stimulus feature. Even though most theories of visual search implicitly or explicitly assume that FBA is under top-down control, we argue that the role of top-down processing in FBA may be limited. Our review of the literature indicates that all behavioural and neuro-imaging studies investigating FBA suffer from the shortcoming that they cannot rule out an effect of priming. The mere attending to a feature enhances the mandatory processing of that feature across the visual field, an effect that is likely to occur in an automatic, bottom-up way. Studies that have investigated the feasibility of FBA by means of cueing paradigms suggest that the role of top-down processing in FBA is limited (e.g. prepare for red). Instead, the actual processing of the stimulus is needed to cause the mandatory tuning of responses throughout the visual field. We conclude that it is likely that all FBA effects reported previously are the result of bottom-up priming.     

Innate and learned components of human visual preference.

BACKGROUND: Recent claims in neuroscience and evolutionary biology suggest that the aesthetic sense reflects preferences for image signals whose characteristics best fit innate brain mechanisms of visual recognition. RESULTS: This hypothesis was tested by behaviourally measuring, for a set of initially unfamiliar images, the effects of category learning on preference judgements by humans, and by relating the observed data to computationally reconstructed internal representations of categorical concepts. Category learning induced complex shifts in preference behaviour. Two distinct factors - complexity and bilateral symmetry - could be identified from the data as determinants of preference judgements. The effect of the complexity factor varied with object knowledge acquired through category learning. In contrast, the impact of the symmetry factor proved to be unaffected by learning experience. Computer simulations suggested that the preference for pattern complexity relies on active (top-down) mechanisms of visual recognition, whereas the preference for pattern symmetry depends on automatic (bottom-up) mechanisms. CONCLUSIONS: Human visual preferences are not fully determined by (objective) structural regularities of image stimuli but also depend on their learned (subjective) interpretation. These two aspects are reflected in distinct complementary factors underlying preference judgements, and may be related to complementary modes of visual processing in the brain.     

Model-based interpretation of complex and variable images.

The ultimate goal of machine vision is image understanding-the ability not only to recover image structure but also to know what it represents. By definition, this involves the use of models which describe and label the expected structure of the world. Over the past decade, model-based vision has been applied successfully to images of man-made objects. It has proved much more difficult to develop model-based approaches to the interpretation of images of complex and variable structures such as faces or the internal organs of the human body (as visualized in medical images). In such cases it has been problematic even to recover image structure reliably, without a model to organize the often noisy and incomplete image evidence. The key problem is that of variability. To be useful, a model needs to be specific-that is, to be capable of representing only ''legal'' examples of the modelled object(s). It has proved difficult to achieve this whilst allowing for natural variability. Recent developments have overcome this problem; it has been shown that specific patterns of variability in shape and grey-level appearance can be captured by statistical models that can be used directly in image interpretation. The details of the approach are outlined and practical examples from medical image interpretation and face recognition are used to illustrate how previously intractable problems can now be tackled successfully. It is also interesting to ask whether these results provide any possible insights into natural vision; for example, we show that the apparent changes in shape which result from viewing three-dimensional objects from different viewpoints can be modelled quite well in two dimensions; this may lend some support to the ''characteristic views'' model of natural vision.