Measurement of the 2D affine Lie group parameters for visual motion analysis.

A class of linear operators is presented for estimating the local components of 2D translation, dilatation, rotation, and the shear/deformations which span the six degrees of freedom of motion of arbitrarily textured surfaces. This results in a model of visual motion analysis which proposes that the local transformations in the image are analysed by decomposing them into the six one-parameter subgroups of the 2D affine group. Each of the required invariant integral operators are easily specified by the characters of these six subgroups. The 2D affine group, however, does not have a simple structure. It is a Lie group which possesses a semi-direct product manifold, and classical harmonic analysis cannot proceed unless some mechanism is prescribed to isolate the 2D 'general linear' transformations from the 2D translations. It must also do so using measures which receive only local support from the image, since the global affine group model is only valid tangentially. A form of 'active perception' is thereby implicated; it is proposed that spatial indexing and 2D tracking is needed in order to form reliable estimates of 3D motion parameters using local operators in a data-driven fashion.     

An automatic 2D–3D image matching method for reproducing spatial knee joint positions using single or dual fluoroscopic images

Fluoroscopic image technique, using either a single image or dual images, has been widely applied to measure in vivo human knee joint kinematics. However, few studies have compared the advantages of using single and dual fluoroscopic images. Furthermore, due to the size limitation of the image intensifiers, it is possible that only a portion of the knee joint could be captured by the fluoroscopy during dynamic knee joint motion. In this paper, we presented a systematic evaluation of an automatic 2D–3D image matching method in reproducing spatial knee joint positions using either single or dual fluoroscopic image techniques. The data indicated that for the femur and tibia, their spatial positions could be determined with an accuracy and precision less than 0.2 mm in translation and less than 0.4° in orientation when dual fluoroscopic images were used. Using single fluoroscopic images, the method could produce satisfactory accuracy in joint positions in the imaging plane (in average up to 0.5 mm in translation and 1.3° in rotation), but large variations along the out-plane direction (in average up to 4.0 mm in translation and 2.2° in rotation). The precision of using single fluoroscopic images to determine the actual knee positions was worse than its accuracy obtained. The data also indicated that when using dual fluoroscopic image technique, if the knee joint outlines in one image were incomplete by 80%, the algorithm could still reproduce the joint positions with high precisions.     

An automatic 2D-3D image matching method for reproducing spatial knee joint positions using single or dual fluoroscopic images

Fluoroscopic image technique, using either a single image or dual images, has been widely applied to measure in vivo human knee joint kinematics. However, few studies have compared the advantages of using single and dual fluoroscopic images. Furthermore, due to the size limitation of the image intensifiers, it is possible that only a portion of the knee joint could be captured by the fluoroscopy during dynamic knee joint motion. In this paper, we presented a systematic evaluation of an automatic 2D-3D image matching method in reproducing spatial knee joint positions using either single or dual fluoroscopic image techniques. The data indicated that for the femur and tibia, their spatial positions could be determined with an accuracy and precision less than 0.2?mm in translation and less than 0.4° in orientation when dual fluoroscopic images were used. Using single fluoroscopic images, the method could produce satisfactory accuracy in joint positions in the imaging plane (in average up to 0.5?mm in translation and 1.3° in rotation), but large variations along the out-plane direction (in average up to 4.0?mm in translation and 2.2° in rotation). The precision of using single fluoroscopic images to determine the actual knee positions was worse than its accuracy obtained. The data also indicated that when using dual fluoroscopic image technique, if the knee joint outlines in one image were incomplete by 80%, the algorithm could still reproduce the joint positions with high precisions.     

Automated Protein Subcellular Localization Based on Local Invariant Features

To understand the function of the encoded proteins, we need to be able to know the subcellular location of a protein. The most common method used for determining subcellular location is fluorescence microscopy which allows subcellular localizations to be imaged in high throughput. Image feature calculation has proven invaluable in the automated analysis of cellular images. This article proposes a novel method named LDPs for feature extraction based on invariant of translation and rotation from given images, the nature which is to count the local difference features of images, and the difference features are given by calculating the D-value between the gray value of the central pixel c and the gray values of eight pixels in the neighborhood. The novel method is tested on two image sets, the first set is which fluorescently tagged protein was endogenously expressed in 10 sebcellular locations, and the second set is which protein was transfected in 11 locations. A SVM was trained and tested for each image set and classification accuracies of 96.7 and 92.3 % were obtained on the endogenous and transfected sets respectively.     

Single particle 3D reconstruction for 2D crystal images of membrane proteins

In cases where ultra-flat cryo-preparations of well-ordered two-dimensional (2D) crystals are available, electron crystallography is a powerful method for the determination of the high-resolution structures of membrane and soluble proteins. However, crystal unbending and Fourier-filtering methods in electron crystallography three-dimensional (3D) image processing are generally limited in their performance for 2D crystals that are badly ordered or non-flat. Here we present a single particle image processing approach, which is implemented as an extension of the 2D crystallographic pipeline realized in the 2dx software package, for the determination of high-resolution 3D structures of membrane proteins. The algorithm presented, addresses the low single-to-noise ratio (SNR) of 2D crystal images by exploiting neighborhood correlation between adjacent proteins in the 2D crystal. Compared with conventional single particle processing for randomly oriented particles, the computational costs are greatly reduced due to the crystal-induced limited search space, which allows a much finer search space compared to classical single particle processing. To reduce the considerable computational costs, our software features a hybrid parallelization scheme for multi-CPU clusters and computer with high-end graphic processing units (GPUs). We successfully apply the new refinement method to the structure of the potassium channel MloK1. The calculated 3D reconstruction shows more structural details and contains less noise than the map obtained by conventional Fourier-filtering based processing of the same 2D crystal images.     

A maximum likelihood approach to two-dimensional crystals

Maximum likelihood (ML) processing of transmission electron microscopy images of protein particles can produce reconstructions of superior resolution due to a reduced reference bias. We have investigated a ML processing approach to images centered on the unit cells of two-dimensional (2D) crystal images. The implemented software makes use of the predictive lattice node tracking in the MRC software, which is used to window particle stacks. These are then noise-whitened and subjected to ML processing. Resulting ML maps are translated into amplitudes and phases for further processing within the 2dx software package. Compared with ML processing for randomly oriented single particles, the required computational costs are greatly reduced as the 2D crystals restrict the parameter search space. The software was applied to images of negatively stained or frozen hydrated 2D crystals of different crystal order. We find that the ML algorithm is not free from reference bias, even though its sensitivity to noise correlation is lower than for pure cross-correlation alignment. Compared with crystallographic processing, the newly developed software yields better resolution for 2D crystal images of lower crystal quality, and it performs equally well for well-ordered crystal images.     

Pattern recognition and associative memory as dynamical processes in a synergetic system

We first present numerical results for the decomposition procedure for complex scenes described in Part I of these papers. Part II then mainly deals with a formalism that allows a formulation of our approach to pattern recognition and associative memory that is simultaneously invariant against translation, rotation, and scaling, Part II thus contains an explicit elaboration of ideas of Part I.     

JULIDE: a software tool for 3D reconstruction and statistical analysis of autoradiographic mouse brain sections

In this article we introduce JULIDE, a software toolkit developed to perform the 3D reconstruction, intensity normalization, volume standardization by 3D image registration and voxel-wise statistical analysis of autoradiographs of mouse brain sections. This software tool has been developed in the open-source ITK software framework and is freely available under a GPL license. The article presents the complete image processing chain from raw data acquisition to 3D statistical group analysis. Results of the group comparison in the context of a study on spatial learning are shown as an illustration of the data that can be obtained with this tool.     

Pattern association and retrieval in a continuous neural system

This paper studies the behavior of a large body of neurons in the continuum limit. A mathematical characterization of such systems is obtained by approximating the inverse input-output nonlinearity of a cell (or an assembly of cells) by three adjustable linearized sections. The associative spatio-temporal patterns for storage in the neural system are obtained by using approaches analogous to solving space-time field equations in physics. A noise-reducing equation is also derived from this neural model. In addition, conditions that make a noisy pattern retrievable are identified. Based on these analyses, a visual cortex model is proposed and an exact characterization of the patterns that are storable in this cortex is obtained. Furthermore, we show that this model achieves pattern association that is invariant to scaling, translation, rotation and mirror-reflection.     

A tiger cannot change its stripes: using a three-dimensional model to match images of living tigers and tiger skins

The tiger is one of many species in which individuals can be identified by surface patterns. Camera traps can be used to record individual tigers moving over an array of locations and provide data for monitoring and studying populations and devising conservation strategies. We suggest using a combination of algorithms to calculate similarity scores between pattern samples scanned from the images to automate the search for a match to a new image. We show how using a three-dimensional surface model of a tiger to scan the pattern samples allows comparison of images that differ widely in camera angles and body posture. The software, which is free to download, considerably reduces the effort required to maintain an image catalogue and we suggest it could be used to trace the origin of a tiger skin by searching a central database of living tigers'' images for matches to an image of the skin.     

In vivo determination of normal and anterior cruciate ligament-deficient knee kinematics

The objective of the current study was to use fluoroscopy to accurately determine the three-dimensional (3D), in vivo, weight-bearing kinematics of 10 normal and five anterior cruciate ligament deficient (ACLD) knees. Patient-specific bone models were derived from computed tomography (CT) data. 3D computer bone models of each subject's femur, tibia, and fibula were recreated from the CT 3D bone density data. Using a model-based 3D-to-2D imaging technique registered CT images were precisely fit onto fluoroscopic images, the full six degrees of freedom motion of the bones was measured from the images. The computer-generated 3D models of each subject's femur and tibia were precisely registered to the 2D digital fluoroscopic images using an optimization algorithm that automatically adjusts the pose of the model at various flexion/extension angles. Each subject performed a weight-bearing deep knee bend while under dynamic fluoroscopic surveillance. All 10 normal knees experienced posterior femoral translation of the lateral condyle and minimal change in position of the medial condyle with progressive knee flexion. The average amount of posterior femoral translation of the lateral condyle was 21.07 mm, whereas the average medial condyle translation was 1.94 mm, in the posterior direction. In contrast, all five ACLD knees experienced considerable change in the position of the medial condyle. The average amount of posterior femoral translation of the lateral condyle was 17.00 mm, while the medial condyle translation was 4.65 mm, in the posterior direction. In addition, the helical axis of motion was determined between maximum flexion and extension. A considerable difference was found between the center of rotation locations of the normal and ACLD subjects, with ACLD subjects exhibiting substantially higher variance in kinematic patterns.     

A computational approach to motion perception

In this paper it is shown that the computation of the optical flow from a sequence of timevarying images is not, in general, an underconstrained problem. A local algorithm for the computation of the optical flow which uses second order derivatives of the image brightness pattern, and that avoids the aperture problem, is presented. The obtained optical flow is very similar to the true motion field — which is the vector field associated with moving features on the image plane — and can be used to recover 3D motion information. Experimental results on sequences of real images, together with estimates of relevant motion parameters, like time-to-crash for translation and angular velocity for rotation, are presented and discussed. Due to the remarkable accuracy which can be achieved in estimating motion parameters, the proposed method is likely to be very useful in a number of computer vision applications.     

Radiomics software for breast imaging optimization and simulation studies

Background and ObjectiveThe development, control and optimisation of new x-ray breast imaging modalities could benefit from a quantitative assessment of the resulting image textures. The aim of this work was to develop a software tool for routine radiomics applications in breast imaging, which will also be available upon request.MethodsThe tool (developed in MATLAB) allows image reading, selection of Regions of Interest (ROI), analysis and comparison. Requirements towards the tool also included convenient handling of common medical and simulated images, building and providing a library of commonly applied algorithms and a friendly graphical user interface. Initial set of features and analyses have been selected after a literature search. Being open, the tool can be extended, if necessary.ResultsThe tool allows semi-automatic extracting of ROIs, calculating and processing a total of 23 different metrics or features in 2D images and/or in 3D image volumes. Computations of the features were verified against computations with other software packages performed with test images. Two case studies illustrate the applicability of the tool – (i) features on a series of 2D ‘left’ and ‘right’ CC mammograms acquired on a Siemens Inspiration system were computed and compared, and (ii) evaluation of the suitability of newly proposed and developed breast phantoms for x-ray-based imaging based on reference values from clinical mammography images. Obtained results could steer the further development of the physical breast phantoms.ConclusionsA new image analysis toolbox was realized and can now be used in a multitude of radiomics applications, on both clinical and test images.     

A Fast Image Alignment Approach for 2D Classification of Cryo-EM Images Using Spectral Clustering

Three-dimensional (3D) reconstruction in single-particle cryo-electron microscopy (cryo-EM) is a significant technique for recovering the 3D structure of proteins or other biological macromolecules from their two-dimensional (2D) noisy projection images taken from unknown random directions. Class averaging in single-particle cryo-EM is an important procedure for producing high-quality initial 3D structures, where image alignment is a fundamental step. In this paper, an efficient image alignment algorithm using 2D interpolation in the frequency domain of images is proposed to improve the estimation accuracy of alignment parameters of rotation angles and translational shifts between the two projection images, which can obtain subpixel and subangle accuracy. The proposed algorithm firstly uses the Fourier transform of two projection images to calculate a discrete cross-correlation matrix and then performs the 2D interpolation around the maximum value in the cross-correlation matrix. The alignment parameters are directly determined according to the position of the maximum value in the cross-correlation matrix after interpolation. Furthermore, the proposed image alignment algorithm and a spectral clustering algorithm are used to compute class averages for single-particle 3D reconstruction. The proposed image alignment algorithm is firstly tested on a Lena image and two cryo-EM datasets. Results show that the proposed image alignment algorithm can estimate the alignment parameters accurately and efficiently. The proposed method is also used to reconstruct preliminary 3D structures from a simulated cryo-EM dataset and a real cryo-EM dataset and to compare them with RELION. Experimental results show that the proposed method can obtain more high-quality class averages than RELION and can obtain higher reconstruction resolution than RELION even without iteration.     

Errors in Quantitative Image Analysis due to Platform-Dependent Image Scaling

PURPOSE: To evaluate the ability of various software (SW) tools used for quantitative image analysis to properly account for source-specific image scaling employed by magnetic resonance imaging manufacturers. METHODS: A series of gadoteridol-doped distilled water solutions (0%, 0.5%, 1%, and 2% volume concentrations) was prepared for manual substitution into one (of three) phantom compartments to create “variable signal,” whereas the other two compartments (containing mineral oil and 0.25% gadoteriol) were held unchanged. Pseudodynamic images were acquired over multiple series using four scanners such that the histogram of pixel intensities varied enough to provoke variable image scaling from series to series. Additional diffusion-weighted images were acquired of an ice-water phantom to generate scanner-specific apparent diffusion coefficient (ADC) maps. The resulting pseudodynamic images and ADC maps were analyzed by eight centers of the Quantitative Imaging Network using 16 different SW tools to measure compartment-specific region-of-interest intensity. RESULTS: Images generated by one of the scanners appeared to have additional intensity scaling that was not accounted for by the majority of tested quantitative image analysis SW tools. Incorrect image scaling leads to intensity measurement bias near 100%, compared to nonscaled images. CONCLUSION: Corrective actions for image scaling are suggested for manufacturers and quantitative imaging community.     

Invariant visual object recognition: a model, with lighting invariance.

How are invariant representations of objects formed in the visual cortex? We describe a neurophysiological and computational approach which focusses on a feature hierarchy model in which invariant representations can be built by self-organizing learning based on the statistics of the visual input. The model can use temporal continuity in an associative synaptic learning rule with a short term memory trace, and/or it can use spatial continuity in Continuous Transformation learning. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and in this paper we show also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in for example spatial and object search tasks. The model has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene.     

Computer-aided studies of vision in crabs

A system for the back projection of computer-generated visual images onto a screen or screens that cover 240° of the horizontal visual field is described. Its applicability for the study of crab vision is tested by comparing the frequency response of the optokinetic response of the land crab, Cardisoma guanhumi, to sinusoidal oscillation of computer-generated striped patterns and a real striped drum. Significant differences were observed only at the low end of the frequency spectrum. The flexibility of computer-generated visual stimulation and its advantages for the study of optic flow are illustrated by experiments that: (a) demonstrate how well crabs separate the translational and rotational components of optic flow by showing compensatory eye movements to only the latter; (b) show that the ability to compensate for rotation is not impaired by combinations of rotation and translation; (c) show that motion parallax cues are used in addition to previously-described global cues for making the distinction between rotation and translation. Finally, the use of these methods in a successful search for visual interneurones sensitive to optic flow stimuli is demonstrated for the shore crab, Carcinus maenas.