Predicting the eye fixation locations in the gray scale images in the visual scenes with different semantic contents

In recent years, there has been considerable interest in visual attention models (saliency map of visual attention). These models can be used to predict eye fixation locations, and thus will have many applications in various fields which leads to obtain better performance in machine vision systems. Most of these models need to be improved because they are based on bottom-up computation that does not consider top-down image semantic contents and often does not match actual eye fixation locations. In this study, we recorded the eye movements (i.e., fixations) of fourteen individuals who viewed images which consist natural (e.g., landscape, animal) and man-made (e.g., building, vehicles) scenes. We extracted the fixation locations of eye movements in two image categories. After extraction of the fixation areas (a patch around each fixation location), characteristics of these areas were evaluated as compared to non-fixation areas. The extracted features in each patch included the orientation and spatial frequency. After feature extraction phase, different statistical classifiers were trained for prediction of eye fixation locations by these features. This study connects eye-tracking results to automatic prediction of saliency regions of the images. The results showed that it is possible to predict the eye fixation locations by using of the image patches around subjects’ fixation points.     

视觉图像辨认眼动中的Top－down信息处理

在视觉图像辨认过程中,眼球不是均匀地扫描全幅图像,而是通过一系列快速的眼球跳动来改变注视点位置,有选择地通过注视停顿来采集图象中的关键信息。通过实验对不同图像刺激时的眼动轨迹进行记录与分析,发现：（１）对于简单的几何图形,眼动注视停顿主要集中在图像中几何特征之处,亦即与周围不同的奇异点上;（２）对复杂图象刺激,眼动注视点位置决定于受试者的已有概念模型及其兴趣所在;（３）对中文单字进行辩认时,其眼动模式也是取决于受试者对该单字的知识（也即概念模型）。以上结果提示,视觉图象辨认主要是通过自上而下（ｔｏｐ－ｄｏｗｎ）的信息处理方式才完成．由中枢控制眼球运动,将注视点落到中枢决定的图形奇点上来,通过注视停顿对中枢认为的关键信息之处进行抽提,以实现辨认。这种处理方式不是只取决于输入的图像信息,也不必对目标图像的每个象素进行处理,而只需对图象中少量的关键信息部位进行重点的检测和处理,从而提高了图象信息处理的能力及效率。     

视觉图像辨认眼动中的Top－down信息处理

在视觉图像辨认过程中,眼球不是均匀地扫描全幅图像,而是通过一系列快速的眼球跳动来改变注视点位置,有选择地通过注视停顿来采集图象中的关键信息。通过实验对不同图像刺激时的眼动轨迹进行记录与分析,发现：（１）对于简单的几何图形,眼动注视停顿主要集中在图像中几何特征之处,亦即与周围不同的奇异点上;（２）对复杂图象刺激,眼动注视点位置决定于受试者的已有概念模型及其兴趣所在;（３）对中文单字进行辩认时,其眼动模式也是取决于受试者对该单字的知识（也即概念模型）。以上结果提示,视觉图象辨认主要是通过自上而下（ｔｏｐ－ｄｏｗｎ）的信息处理方式才完成．由中枢控制眼球运动,将注视点落到中枢决定的图形奇点上来,通过注视停顿对中枢认为的关键信息之处进行抽提,以实现辨认。这种处理方式不是只取决于输入的图像信息,也不必对目标图像的每个象素进行处理,而只需对图象中少量的关键信息部位进行重点的检测和处理,从而提高了图象信息处理的能力及效率。     

Predicting the Valence of a Scene from Observers’ Eye Movements

Multimedia analysis benefits from understanding the emotional content of a scene in a variety of tasks such as video genre classification and content-based image retrieval. Recently, there has been an increasing interest in applying human bio-signals, particularly eye movements, to recognize the emotional gist of a scene such as its valence. In order to determine the emotional category of images using eye movements, the existing methods often learn a classifier using several features that are extracted from eye movements. Although it has been shown that eye movement is potentially useful for recognition of scene valence, the contribution of each feature is not well-studied. To address the issue, we study the contribution of features extracted from eye movements in the classification of images into pleasant, neutral, and unpleasant categories. We assess ten features and their fusion. The features are histogram of saccade orientation, histogram of saccade slope, histogram of saccade length, histogram of saccade duration, histogram of saccade velocity, histogram of fixation duration, fixation histogram, top-ten salient coordinates, and saliency map. We utilize machine learning approach to analyze the performance of features by learning a support vector machine and exploiting various feature fusion schemes. The experiments reveal that ‘saliency map’, ‘fixation histogram’, ‘histogram of fixation duration’, and ‘histogram of saccade slope’ are the most contributing features. The selected features signify the influence of fixation information and angular behavior of eye movements in the recognition of the valence of images.     

Overt attention and context factors: the impact of repeated presentations, image type, and individual motivation

The present study investigated the dynamic of the attention focus during observation of different categories of complex scenes and simultaneous consideration of individuals' memory and motivational state. We repeatedly presented four types of complex visual scenes in a pseudo-randomized order and recorded eye movements. Subjects were divided into groups according to their motivational disposition in terms of action orientation and individual rating of scene interest.Statistical analysis of eye-tracking data revealed that the attention focus successively became locally expressed by increasing fixation duration; decreasing saccade length, saccade frequency, and single subject's fixation distribution over images; and increasing inter-subject variance of fixation distributions. The validity of these results was supported by verbal reports. This general tendency was weaker for the group of subjects who rated the image set as interesting as compared to the other group. Additionally, effects were partly mediated by subjects' motivational disposition. Finally, we found a generally strong impact of image type on eye movement parameters. We conclude that motivational tendencies linked to personality as well as individual preferences significantly affected viewing behaviour. Hence, it is important and fruitful to consider inter-individual differences on the level of motivation and personality traits within investigations of attention processes. We demonstrate that future studies on memory's impact on overt attention have to deal appropriately with several aspects that had been out of the research focus until now.     

Scene content selected by active vision

The primate visual system actively selects visual information from the environment for detailed processing through mechanisms of visual attention and saccadic eye movements. This study examines the statistical properties of the scene content selected by active vision. Eye movements were recorded while participants free-viewed digitized images of natural and artificial scenes. Fixation locations were determined for each image and image patches were extracted around the observed fixation locations. Measures of local contrast, local spatial correlation and spatial frequency content were calculated on the extracted image patches. Replicating previous results, local contrast was found to be greater at the points of fixation when compared to either the contrast for image patches extracted at random locations or at the observed fixation locations using an image-shuffled database. Contrary to some results and in agreement with other results in the literature, a significant decorrelation of image intensity is observed between the locations of fixation and other neighboring locations. A discussion and analysis of methodological techniques is given that provides an explanation for the discrepancy in results. The results of our analyses indicate that both the local contrast and correlation at the points of fixation are a function of image type and, furthermore, that the magnitude of these effects depend on the levels of contrast and correlation present overall in the images. Finally, the largest effect sizes in local contrast and correlation are found at distances of approximately 1 deg of visual angle, which agrees well with measures of optimal spatial scale selectivity in the visual periphery where visual information for potential saccade targets is processed.     

Object Learning Improves Feature Extraction but Does Not Improve Feature Selection

A single glance at your crowded desk is enough to locate your favorite cup. But finding an unfamiliar object requires more effort. This superiority in recognition performance for learned objects has at least two possible sources. For familiar objects observers might: 1) select more informative image locations upon which to fixate their eyes, or 2) extract more information from a given eye fixation. To test these possibilities, we had observers localize fragmented objects embedded in dense displays of random contour fragments. Eight participants searched for objects in 600 images while their eye movements were recorded in three daily sessions. Performance improved as subjects trained with the objects: The number of fixations required to find an object decreased by 64% across the 3 sessions. An ideal observer model that included measures of fragment confusability was used to calculate the information available from a single fixation. Comparing human performance to the model suggested that across sessions information extraction at each eye fixation increased markedly, by an amount roughly equal to the extra information that would be extracted following a 100% increase in functional field of view. Selection of fixation locations, on the other hand, did not improve with practice.     

Through a barn owl’s eyes: interactions between scene content and visual attention

In this study we investigated visual attention properties of freely behaving barn owls, using a miniature wireless camera attached to their heads. The tubular eye structure of barn owls makes them ideal subjects for this research since it limits their eye movements. Video sequences recorded from the owl’s point of view capture part of the visual scene as seen by the owl. Automated analysis of video sequences revealed that during an active search task, owls repeatedly and consistently direct their gaze in a way that brings objects of interest to a specific retinal location (retinal fixation area). Using a projective model that captures the geometry between the eye and the camera, we recovered the corresponding location in the recorded images (image fixation area). Recording in various types of environments (aviary, office, outdoors) revealed significant statistical differences of low level image properties at the image fixation area compared to values extracted at random image patches. These differences are in agreement with results obtained in primates in similar studies. To investigate the role of saliency and its contribution to drawing the owl’s attention, we used a popular bottom-up computational model. Saliency values at the image fixation area were typically greater than at random patches, yet were only 20% out of the maximal saliency value, suggesting a top-down modulation of gaze control.     

Epigenetic randomness, complexity and singularity of human iris patterns

We investigated the randomness and uniqueness of human iris patterns by mathematically comparing 2.3 million different pairs of eye images. The phase structure of each iris pattern was extracted by demodulation with quadrature wavelets spanning several scales of analysis. The resulting distribution of phase sequence variation among different eyes was precisely binomial, revealing 244 independent degrees of freedom. This amount of statistical variability corresponds to an entropy (information density) of about 3.2 bits mm(-2) over the iris. It implies that the probability of two different irides agreeing by chance in more than 70% of their phase sequence is about one in 7 billion. We also compared images of genetically identical irides, from the left and right eyes of 324 persons, and from monozygotic twins. Their relative phase sequence variation generated the same statistical distribution as did unrelated eyes. This indicates that apart from overall form and colour, iris patterns are determined epigenetically by random events in the morphogenesis of this tissue. The resulting diversity, and the combinatorial complexity created by so many dimensions of random variation, mean that the failure of a simple test of statistical independence performed on iris patterns can serve as a reliable rapid basis for automatic personal identification.     

Preliminary Study towards the Use of In‐situ Microscopy for the Online Analysis of Microcarrier Cultivations

In order to apply newly developed non‐invasive in‐situ microscope systems for the monitoring of microcarrier‐based cultivations, appropriate image analysis systems must be available. Thus a simple, but effective greyscale distribution scan algorithm was tested for the evaluation of images generated by either a standard phase‐contrast microscope or an in‐situ microscope. The images were analyzed according to their greyscale pattern in order to examine whether the greyscale distribution is a possibility to gain information about the covering ratio. The mouse fibroblast cell line (NIH–3T3) was grown on different microcarrier spheres. At first, different microcarriers were tested with respect to their suitability for microscopic observation. In a second part, the phase‐contrast pictures and in‐situ microscope pictures of the microcarrier were separately analyzed using the histogram function of CorelPhotopaint, which analyzes the greyscale distribution within the chosen area. Due to the low optical density of the polydextrin matrix, the images of Cytodex 1 microcarriers proved to be an ideal model for the image analysis. Significant differences in the greyscale distribution of this microcarrier without cells and with increased cell density were observed. Therefore a relationship between the cell density on the microcarriers and the greyscale pattern can be assumed. After automating this image analysis and calibrating the cell number/greyscale pattern relationship, it should be possible to analyze the plating efficiency/covering ratio on the microcarrier online by in‐situ microscopy.     

Analysis of the hierarchical structure of biological tissues by scanning X-ray scattering using a micro-beam.

The outstanding mechanical properties of biological tissues such as wood or bone are mainly due to their hierarchical structure and to their optimization at all levels of hierarchy. It is therefore essential to characterize the structure at all levels to understand the complex behavior of such tissues. Structures down to the micrometer level are accessible to light or scanning electron microscopic observation. In the case of bone this includes, for example, morphometry of the trabecular architecture or the bone mineral density distribution in cortical and trabecular bone. To characterize the sub-micrometer structure of, e.g., the collagen-mineral composite in the case of bone or the cellulose microfibrils in the case of wood, other methods, such as transmission electron microscopy or X-ray scattering are necessary. The recent availability of extremely brilliant synchrotron X-ray sources has led to the development of the new techniques of scanning small-angle X-ray scattering and scanning X-ray microdiffraction, which are capable of providing structural information on the micrometer and the nanometer level, simultaneously. As a basic principle of the method the specimen is scanned across an X-ray beam which has a diameter of few micrometers. Measuring the X-ray absorption at each position provides an image of the specimen (microradiography) with resolution similar to light microscopy, in the micrometer range. Moreover, the X-ray scattering pattern is analyzed at each specimen position to provide parameters characterizing the structure in the nanometer range. The present paper reviews the principles of the techniques and demonstrates their application to biological materials, such as wood or bone.     

人工红松树干内部节子体积预测模型

基于黑龙江省林口林业局林场和东京城林业局林场29块标准地中49株人工红松1207个节子数据,使用图片处理软件Digimizer对节子纵剖面图片数据进行提取,将节子形状用一个二维散点图表示。根据节子二维形状散点图,把人工红松节子分为3种类型: 活节(整个节子为健全节)、未包藏死节(节子由健全节和疏松节组成)和包藏死节(节子的健全节和疏松节部分被树干包藏)。3个类型节子的健全节体积通过对健全节形状参数方程求积得到;疏松节体积利用圆柱体的体积计算得到;节子总体积等于健全节体积与疏松节体积之和。最后,基于节子变量(节子直径、节子相对高、节子总长度)和树木变量(胸径),采用样地和树木水平的线性混合模型建立了红松人工林健全节体积、疏松节体积和节子总体积的预测模型。与基础模型相比,考虑样地和树木水平的混合效应所建立的健全节体积、疏松节体积和节子总体积预测模型,其参数估计更精准,残差分布更均匀,拟合精度明显提高。检验结果表明,基础模型预估精度均在90%以上,引入样地和树木效应的混合模型的预估精度均在93%以上,说明所建模型可以很好地预测红松人工林节子体积大小。     

Cell-by-Cell Alignment of Repeated Specular Microscopy Images from the Same Eye

Purpose

Modern specular microscopes (SM) robustly depict the same central area of the corneal endothelium at different time points through a built-in fixation light. However, repeated image acquisitions slightly shift and rotate because of minute changes in head position in the chin and forehead rest. This prevents the manual retrieval of individual corneal endothelial cells (CECs) in repeated measurements because SM images usually lack obvious landmarks. We devised and validated an image registration algorithm that aligns SM images from the same eye to make corresponding CECs coincide.

Methods

We retrospectively selected 27 image pairs for the presence of significant image overlap. Each image pair had been recorded on the same day and of the same eye. We applied our registration method in each image pair. Two observers independently validated, by means of alternation flicker, that the image pairs had been correctly aligned. We also repeatedly applied our registration method on unrelated image pairs by randomly drawing images and making certain that the images did not originate from the same eye. This was done to assess the specifity of our method.

Results

All automated registrations of the same-day and same-eye image pairs were accurate. However, one single image incorrectly failed to trigger the non-match diagnosis twice in 81 registration attempts between unrelated images. As it turned out, this particular image depicted only 73 CECs. The average number of CECs was 253 (range 73–393).

Conclusion

Repeated non-contact SM images can be automatedly aligned so that the corresponding CECs coincide. Any successful alignment can be considered as proof of the retrieval of identical CECs as soon as at least 100 CEC centroids have been identified. We believe our method is the first to robustly confirm endothelial stability in individual eyes.     

How to Create and Use Binocular Rivalry

Each of our eyes normally sees a slightly different image of the world around us. The brain can combine these two images into a single coherent representation. However, when the eyes are presented with images that are sufficiently different from each other, an interesting thing happens: Rather than fusing the two images into a combined conscious percept, what transpires is a pattern of perceptual alternations where one image dominates awareness while the other is suppressed; dominance alternates between the two images, typically every few seconds. This perceptual phenomenon is known as binocular rivalry. Binocular rivalry is considered useful for studying perceptual selection and awareness in both human and animal models, because unchanging visual input to each eye leads to alternations in visual awareness and perception. To create a binocular rivalry stimulus, all that is necessary is to present each eye with a different image at the same perceived location. There are several ways of doing this, but newcomers to the field are often unsure which method would best suit their specific needs. The purpose of this article is to describe a number of inexpensive and straightforward ways to create and use binocular rivalry. We detail methods that do not require expensive specialized equipment and describe each method''s advantages and disadvantages. The methods described include the use of red-blue goggles, mirror stereoscopes and prism goggles.     

What Image Properties Regulate Eye Growth?

The growth of the eye, unlike other parts of the body, is not ballistic. It is guided by visual feedback with the eventual aim being optimal focus of the retinal image or emmetropization . It has been shown in animal models that interference with the quality of the retinal image leads to a disruption to the normal growth pattern, resulting in the development of refractive errors and defocused retinal images . While it is clear that retinal images rich in pattern information are needed to control eye growth, it is unclear what particular aspect of image structure is relevant. Retinal images comprise a range of spatial frequencies at different absolute and relative contrasts and in different degrees of spatial alignment. Here we show, by using synthetic images, that it is not the local edge structure produced by relative spatial frequency alignments within an image but rather the spatial frequency composition per se that is used to regulate the growth of the eye. Furthermore, it is the absolute energy at high spatial frequencies regardless of the spectral slope that is most effective. Neither result would be expected from currently accepted ideas of how human observers judge the degree of image "blur" in a scene where both phase alignments and the relative energy distribution across spatial frequency (i.e., spectral slope) are important.     

Accurate Image Analysis of the Retina Using Hessian Matrix and Binarisation of Thresholded Entropy with Application of Texture Mapping

In this paper, we demonstrate a comprehensive method for segmenting the retinal vasculature in camera images of the fundus. This is of interest in the area of diagnostics for eye diseases that affect the blood vessels in the eye. In a departure from other state-of-the-art methods, vessels are first pre-grouped together with graph partitioning, using a spectral clustering technique based on morphological features. Local curvature is estimated over the whole image using eigenvalues of Hessian matrix in order to enhance the vessels, which appear as ridges in images of the retina. The result is combined with a binarized image, obtained using a threshold that maximizes entropy, to extract the retinal vessels from the background. Speckle type noise is reduced by applying a connectivity constraint on the extracted curvature based enhanced image. This constraint is varied over the image according to each region''s predominant blood vessel size. The resultant image exhibits the central light reflex of retinal arteries and veins, which prevents the segmentation of whole vessels. To address this, the earlier entropy-based binarization technique is repeated on the original image, but crucially, with a different threshold to incorporate the central reflex vessels. The final segmentation is achieved by combining the segmented vessels with and without central light reflex. We carry out our approach on DRIVE and REVIEW, two publicly available collections of retinal images for research purposes. The obtained results are compared with state-of-the-art methods in the literature using metrics such as sensitivity (true positive rate), selectivity (false positive rate) and accuracy rates for the DRIVE images and measured vessel widths for the REVIEW images. Our approach out-performs the methods in the literature.     

Ontogenese de la fixation visuelle au cours de la marche orientee chez le grillon des bois Nemobius sylvestris

Wood-crickets $\text{Nemobius sylvestris}$ of various ages, usually orientate towards black targets on a white background. During this walking movement, the strategy they use to maintain the image of the target edges on the retina is studied through a frame by frame analysis. For larvae and adults, the visually orientated locomotion is jerky with successive runs and pauses but the characteristics of the rotations and translations of the body long axis are different according to the larval instar considered.Many runs and pauses of long duration occur during the first larval instar paths. The young insects make greater body long axis rotations and their changes of direction also have a larger amplitude. The images of the edges of the target are not recorded by exactly the same part of the retina during ontogeny.For the early larval instars, a wide lateral ocular area has this function, but for the adults a well differentiated visual fixation area occurs. The results for the older larval instars are intermediate.     

Why um helps auditory word recognition: the temporal delay hypothesis

Several studies suggest that speech understanding can sometimes benefit from the presence of filled pauses (uh, um, and the like), and that words following such filled pauses are recognised more quickly. Three experiments examined whether this is because filled pauses serve to delay the onset of upcoming words and these delays facilitate auditory word recognition, or whether the fillers themselves serve to signal upcoming delays in a way which informs listeners' reactions. Participants viewed pairs of images on a computer screen, and followed recorded instructions to press buttons corresponding to either an easy (unmanipulated, with a high-frequency name) or a difficult (visually blurred, low-frequency) image. In all three experiments, participants were faster to respond to easy images. In 50% of trials in each experiment, the name of the image was directly preceded by a delay; in the remaining trials an equivalent delay was included earlier in the instruction. Participants were quicker to respond when a name was directly preceded by a delay, regardless of whether this delay was filled with a spoken um, was silent, or contained an artificial tone. This effect did not interact with the effect of image difficulty, nor did it change over the course of each experiment. Taken together, our consistent finding that delays of any kind help word recognition indicates that natural delays such as fillers need not be seen as 'signals' to explain the benefits they have to listeners' ability to recognise and respond to the words which follow them.     

Toward objective selection of representative microscope images

Scientists wishing to communicate the essential characteristics of a pattern (such as an immunofluorescence distribution) currently must make a subjective choice of one or two images to publish. We therefore developed methods for objectively choosing a typical image from a set, with emphasis on images from cell biology. The methods involve calculation of numerical features to describe each image, calculation of similarity between images as a distance in feature space, and ranking of images by distance from the center of the feature distribution. Two types of features were explored, image texture measures and Zernike polynomial moments, and various distance measures were utilized. Criteria for evaluating methods for assigning typicality were proposed and applied to sets of images containing more than one pattern. The results indicate the importance of using distance measures that are insensitive to the presence of outliers. For collections of images of the distributions of a lysosomal protein, a Golgi protein, and nuclear DNA, the images chosen as most typical were in good agreement with the conventional understanding of organelle morphologies. The methods described here have been implemented in a web server (http://murphylab.web.cmu.edu/services/TyplC).     

基于百度街景图像的行人视角城市街道植被绿化格局分析

城市街道绿化植被作为城市景观的重要组成部分, 其分布格局对城市景观美学发展及行人身心健康有显著影响, 立足行人视角准确监测街道绿植分布信息对城市规划与管理有明确的辅助作用。该文针对已有研究多采用沿天底方向垂直向下观测的遥感影像监测地表植被而对行人视角的绿色植被分布格局研究涉及不多的现状, 基于免费获取的百度街景图像, 选取绿植覆被典型的泰安市区为案例区, 结合网络信息抓取与空间地理信息处理技术, 分析百度街景图像提取侧视绿植信息的可行性, 统计并对比其计算结果与遥感影像提取结果的关系, 以期为城市规划与管理提供辅助参考信息。网络抓取案例区273个样点共3 276幅百度街景图像, 利用计算机监督分类提取图像中的绿植区域; 基于空间分析模型分析街道绿色植被的分布格局; 利用SPSS软件趋势拟合模块分析百度街景图像与遥感影像提取的植被信息的相关性。主要结果为: 百度街景图像可作为主数据源提取城市街道的侧视绿植分布情况; 案例区不同区域植被分布指数区别较大, 空间格局差异明显; 百度街道植被分布指数与基于遥感图像提取的10、20、50 m缓冲距离范围内植被覆盖面积呈显著正相关关系, 但两者的变化趋势并非完全一致。百度街道植被分布结果可作为遥感监测结果的辅助信息更好地指导城市绿色景观规划与精准管理。