How to measure diagnosis-associated information in virtual slides

The distribution of diagnosis-associated information in histological slides is often spatial dependent. A reliable selection of the slide areas containing the most significant information to deriving the associated diagnosis is a major task in virtual microscopy. Three different algorithms can be used to select the appropriate fields of view: 1) Object dependent segmentation combined with graph theory; 2) time series associated texture analysis; and 3) geometrical statistics based upon geometrical primitives. These methods can be applied by sliding technique (i.e., field of view selection with fixed frames), and by cluster analysis. The implementation of these methods requires a standardization of images in terms of vignette correction and gray value distribution as well as determination of appropriate magnification (method 1 only). A principle component analysis of the color space can significantly reduce the necessary computation time. Method 3 is based upon gray value dependent segmentation followed by graph theory application using the construction of (associated) minimum spanning tree and Voronoi's neighbourhood condition. The three methods have been applied on large sets of histological images comprising different organs (colon, lung, pleura, stomach, thyroid) and different magnifications, The trials resulted in a reproducible and correct selection of fields of view in all three methods. The different algorithms can be combined to a basic technique of field of view selection, and a general theory of "image information" can be derived. The advantages and constraints of the applied methods will be discussed.     

A methodology for registration of a histological slide and in vivo MRI volume based on optimizing mutual information

We present a method for registering histology and in vivo imaging that requires minimal microtoming and is automatic following the user's initialization. In this demonstration, we register a single hematoxylin-and-eosin-stained histological slide of a coronal section of a rat brain harboring a 9L gliosarcoma with an in vivo 7T MR image volume of the same brain. Because the spatial resolution of the in vivo MRI is limited, we add the step of obtaining a high spatial resolution, ex vivo MRI in situ for intermediate registration. The approach taken was to maximize mutual information in order to optimize the registration between all pairings of image data whether the sources are MRI, tissue block photograph, or stained sample photograph. The warping interpolant used was thin plate splines with the appropriate basis function for either 2-D or 3-D applications. All registrations were implemented by user initialization of the approximate pose between the two data sets, followed by automatic optimization based on maximizing mutual information. Only the higher quality anatomical images were used in the registration process; however, the spatial transformation was directly applied to a quantitative diffusion image. Quantitative diffusion maps from the registered location appeared highly correlated with the H&E slide. Overall, this approach provides a robust method for coregistration of in vivo images with histological sections and will have broad applications in the field of functional and molecular imaging.     

Methods for acquisition of quantitative from confocal images of gene expression in situ

In this review we summarize original methods for the extraction quantitative information from the confocal images of gene expression patterns. These methods include image segmentation, extraction of quantitative numerical data on gene expression, removal of background signal and spatial registration. Finally it is possible to construct a spatiotemporal atlas of gene expression form individual images obtained at each developmental stage. Initially all methods were developed to extract quantitative numerical information form confocal images of segmentation gene expression in Drosophila melanogaster. Application of these methods to Drosophila images makes it possible to reveal new mechanisms of formation of segmentation gene expression domains, as well as to construct the quantitative atlas of segmentation gene expression. Most image processing procedures can be easily adapted to process a wide range of biological images.     

Methods for acquisition of quantitative data from confocal images of gene expression in situ

In this review, we summarize original methods for the extraction of quantitative information from confocal images of gene-expression patterns. These methods include image segmentation, the extraction of quantitative numerical data on gene expression, and the removal of background signal and spatial registration. Finally, it is possible to construct a spatiotemporal atlas of gene expression from individual images recorded at each developmental stage. Initially all methods were developed to extract quantitative numerical information from confocal images of segmentation gene expression in Drosophila melanogaster. The application of these methods to Drosophila images makes it possible to reveal new mechanisms in the formation of segmentation gene expression domains, as well as to construct a quantitative atlas of segmentation gene expression. Most image processing procedures can be easily adapted to process a wide range of biological images.     

Magnetic Resonance Imaging of the Newborn Brain: Automatic Segmentation of Brain Images into 50 Anatomical Regions

We studied methods for the automatic segmentation of neonatal and developing brain images into 50 anatomical regions, utilizing a new set of manually segmented magnetic resonance (MR) images from 5 term-born and 15 preterm infants imaged at term corrected age called ALBERTs. Two methods were compared: individual registrations with label propagation and fusion; and template based registration with propagation of a maximum probability neonatal ALBERT (MPNA). In both cases we evaluated the performance of different neonatal atlases and MPNA, and the approaches were compared with the manual segmentations by means of the Dice overlap coefficient. Dice values, averaged across regions, were 0.81±0.02 using label propagation and fusion for the preterm population, and 0.81±0.02 using the single registration of a MPNA for the term population. Segmentations of 36 further unsegmented target images of developing brains yielded visibly high-quality results. This registration approach allows the rapid construction of automatically labeled age-specific brain atlases for neonates and the developing brain.     

Registration methods in radiotherapy

PurposeThe aim of this study is to present a short and comprehensive review of the methods of medical image registration, their conditions and applications in radiotherapy. A particular focus was placed on the methods of deformable image registration.MethodsTo structure and deepen the knowledge on medical image registration in radiotherapy, a medical literature analysis was made using the Google Scholar browser and the medical database of the PubMed library.ResultsChronological review of image registration methods in radiotherapy based on 34 selected articles. A particular attention was given to show: (i) potential regions of the application of different methods of registration, (ii) mathematical basis of the deformable methods and (iii) the methods of quality control for the registration process.ConclusionsThe primary aim of the medical image registration process is to connect the contents of images. What we want to achieve is a complementary or extended knowledge that can be used for more precise localisation of pathogenic lesions and continuous improvement of patient treatment. Therefore, the choice of imaging mode is dependent on the type of clinical study. It is impossible to visualise all anatomical details or functional changes using a single modality machine. Therefore, fusion of various modality images is of great clinical relevance. A natural problem in analysing the fusion of medical images is geographical errors related to displacement. The registered images are performed not at the same time and, very often, at different respiratory phases.     

Auto-align - multi-modality fluorescence microscopy image co-registration

Multi-modality microscopes incorporate multiple microscopy techniques into one module, imaging through a common objective lens. Simultaneous or consecutive image acquisition of a single specimen, using multiple techniques, increases the amount of measurable information available. In order to benefit from each modality, it is necessary to accurately co-register data sets. Intrinsic differences in the image formation process employed by each modality result in images which possess different characteristics. In addition, as a result of using different measurement devices, images often differ in size and can suffer relative geometrical deformations including rotation, scale and translation, making registration a complex problem. Current methods generally rely on manual input and are therefore subject to human error. Here, we present an automated image registration tool for fluorescence microscopy. We show that it successfully registers images obtained via total internal reflection fluorescence (TIRF), or epi-fluorescence, and confocal microscopy. Furthermore, we provide several other applications including channel merging following image acquisition through an emission beam splitter, and lateral stage drift correction. We also discuss areas of membrane trafficking which could benefit from application of Auto-Align. Auto-Align is an essential item in the advanced microscopist's toolbox which can create a synergy of single or multi-modality image data.     

Registration accuracy and quality of real-life images

Background

A common registration problem for the application of consumer device is to align all the acquired image sequences into a complete scene. Image alignment requires a registration algorithm that will compensate as much as possible for geometric variability among images. However, images captured views from a real scene usually produce different distortions. Some are derived from the optic characteristics of image sensors, and others are caused by the specific scenes and objects.

Methodology/Principal Findings

An image registration algorithm considering the perspective projection is proposed for the application of consumer devices in this study. It exploits a multiresolution wavelet-based method to extract significant features. An analytic differential approach is then proposed to achieve fast convergence of point matching. Finally, the registration accuracy is further refined to obtain subpixel precision by a feature-based modified Levenberg-Marquardt method. Due to its feature-based and nonlinear characteristic, it converges considerably faster than most other methods. In addition, vignette compensation and color difference adjustment are also performed to further improve the quality of registration results.

Conclusions/Significance

The performance of the proposed method is evaluated by testing the synthetic and real images acquired by a hand-held digital still camera and in comparison with two registration techniques in terms of the squared sum of intensity differences (SSD) and correlation coefficient (CC). The results indicate that the proposed method is promising in registration accuracy and quality, which are statistically significantly better than other two approaches.     

A comparison of seven methods of within-subjects rigid-body pedobarographic image registration

Image registration, the process of transforming images such that homologous structures optimally overlap, provides the pre-processing foundation for pixel-level functional image analysis. The purpose of this study was to compare the performances of seven methods of within-subjects pedobarographic image registration: (1) manual, (2) principal axes, (3) centre of pressure trajectory, (4) mean squared error, (5) probability-weighted variance, (6) mutual information, and (7) exclusive OR. We assumed that foot-contact geometry changes were negligibly small trial-to-trial and thus that a rigid-body transformation could yield optimum registration performance. Thirty image pairs were randomly selected from our laboratory database and were registered using each method. To compensate for inter-rater variability, the mean registration parameters across 10 raters were taken as representative of manual registration. Registration performance was assessed using four dissimilarity metrics (#4-7 above). One-way MANOVA found significant differences between the methods (p<0.001). Bonferroni post-hoc tests revealed that the centre of pressure method performed the poorest (p<0.001) and that the principal axes method tended to perform more poorly than remaining methods (p<0.070). Average manual registration was not different from the remaining methods (p=1.000). The results suggest that a variety of linear registration methods are appropriate for within-subjects pedobarographic images, and that manual image registration is a viable alternative to algorithmic registration when parameters are averaged across raters. The latter finding, in particular, may be useful for cases of image peculiarities resulting from outlier trials or from experimental manipulations that induce substantial changes in contact area or pressure profile geometry.     

An automatic matching technique for patient alignment

An automatic system of patient alignment is required in order to monitor changes that occur in the period between magnetic resonance scans. For each scan of the patient a prime requisite is to register the images with respect to each other. The orthogonal relationship between the sagittal and transverse images should, in principle, identify a single common line at the intersection of the two image planes. The basis of the comparison requires spatial registration of the two images to correct for the probable translational and rotational tilts as well as for the geometrical and intensity distortions. This paper describes a number of automatic techniques which compare, pixel-by-pixel, first two synthetic images, and then their application to real images obtained separately from the same head and neck object field. The robustness, computational cost and effectiveness of the techniques presented are discussed, and computed results on real data for the most promising technique based on the Ratio Absolute Difference algorithm are presented.     

Dual-Force ISOMAP: A New Relevance Feedback Method for Medical Image Retrieval

With great potential for assisting radiological image interpretation and decision making, content-based image retrieval in the medical domain has become a hot topic in recent years. Many methods to enhance the performance of content-based medical image retrieval have been proposed, among which the relevance feedback (RF) scheme is one of the most promising. Given user feedback information, RF algorithms interactively learn a user’s preferences to bridge the “semantic gap” between low-level computerized visual features and high-level human semantic perception and thus improve retrieval performance. However, most existing RF algorithms perform in the original high-dimensional feature space and ignore the manifold structure of the low-level visual features of images. In this paper, we propose a new method, termed dual-force ISOMAP (DFISOMAP), for content-based medical image retrieval. Under the assumption that medical images lie on a low-dimensional manifold embedded in a high-dimensional ambient space, DFISOMAP operates in the following three stages. First, the geometric structure of positive examples in the learned low-dimensional embedding is preserved according to the isometric feature mapping (ISOMAP) criterion. To precisely model the geometric structure, a reconstruction error constraint is also added. Second, the average distance between positive and negative examples is maximized to separate them; this margin maximization acts as a force that pushes negative examples far away from positive examples. Finally, the similarity propagation technique is utilized to provide negative examples with another force that will pull them back into the negative sample set. We evaluate the proposed method on a subset of the IRMA medical image dataset with a RF-based medical image retrieval framework. Experimental results show that DFISOMAP outperforms popular approaches for content-based medical image retrieval in terms of accuracy and stability.     

aBEAT: A Toolbox for Consistent Analysis of Longitudinal Adult Brain MRI

Longitudinal brain image analysis is critical for revealing subtle but complex structural and functional changes of brain during aging or in neurodevelopmental disease. However, even with the rapid increase of clinical research and trials, a software toolbox dedicated for longitudinal image analysis is still lacking publicly. To cater for this increasing need, we have developed a dedicated 4D Adult Brain Extraction and Analysis Toolbox (aBEAT) to provide robust and accurate analysis of the longitudinal adult brain MR images. Specially, a group of image processing tools were integrated into aBEAT, including 4D brain extraction, 4D tissue segmentation, and 4D brain labeling. First, a 4D deformable-surface-based brain extraction algorithm, which can deform serial brain surfaces simultaneously under temporal smoothness constraint, was developed for consistent brain extraction. Second, a level-sets-based 4D tissue segmentation algorithm that incorporates local intensity distribution, spatial cortical-thickness constraint, and temporal cortical-thickness consistency was also included in aBEAT for consistent brain tissue segmentation. Third, a longitudinal groupwise image registration framework was further integrated into aBEAT for consistent ROI labeling by simultaneously warping a pre-labeled brain atlas to the longitudinal brain images. The performance of aBEAT has been extensively evaluated on a large number of longitudinal MR T1 images which include normal and dementia subjects, achieving very promising results. A Linux-based standalone package of aBEAT is now freely available at http://www.nitrc.org/projects/abeat.     

Robust Spectral Clustering Using Statistical Sub-Graph Affinity Model

Spectral clustering methods have been shown to be effective for image segmentation. Unfortunately, the presence of image noise as well as textural characteristics can have a significant negative effect on the segmentation performance. To accommodate for image noise and textural characteristics, this study introduces the concept of sub-graph affinity, where each node in the primary graph is modeled as a sub-graph characterizing the neighborhood surrounding the node. The statistical sub-graph affinity matrix is then constructed based on the statistical relationships between sub-graphs of connected nodes in the primary graph, thus counteracting the uncertainty associated with the image noise and textural characteristics by utilizing more information than traditional spectral clustering methods. Experiments using both synthetic and natural images under various levels of noise contamination demonstrate that the proposed approach can achieve improved segmentation performance when compared to existing spectral clustering methods.     

Application of Dual Modality Contrast Agent Combined with Multi-Scale Representation in Ultrasound-Magnetic Resonance Imaging Registration Scheme

To achieve the image registration/fusion and perfect the quality of the integration, with dual modality contrast agent (DMCA), a novel multi-scale representation registration method between ultrasound imaging (US) and magnetic resonance imaging (MRI) is presented in the paper, and how DMCA influence on registration accuracy is chiefly discussed. Owing to US’s intense speckle noise, it is a tremendous challenge to register US with any other modality images. How to improve the algorithms for US processing has become the bottleneck, and in the short term it is difficult to have a breakthrough. In that case, DMCA is employed in both US and MRI to enhance the region of interest. Then, because multi-scale representation is a strategy that attempts to diminish or eliminate several possible local minima and lead to convex optimization problems to be solved quickly and more efficiently, a multi-scale representation Gaussian pyramid based affine registration (MRGP-AR) scheme is constructed to complete the US-MRI registration process. In view of the above-mentioned method, the comparison tests indicate that US-MRI registration/fusion may be a remarkable method for gaining high-quality registration image. The experiments also show that it is feasible that novel nano-materials combined with excellent algorithm are used to solve some hard tasks in medical image processing field.     

A Practical Approach Based on Analytic Deformable Algorithm for Scenic Image Registration

Background

Image registration is to produce an entire scene by aligning all the acquired image sequences. A registration algorithm is necessary to tolerance as much as possible for intensity and geometric variation among images. However, captured image views of real scene usually produce unexpected distortions. They are generally derived from the optic characteristics of image sensors or caused by the specific scenes and objects.

Methods and Findings

An analytic registration algorithm considering the deformation is proposed for scenic image applications in this study. After extracting important features by the wavelet-based edge correlation method, an analytic registration approach is then proposed to achieve deformable and accurate matching of point sets. Finally, the registration accuracy is further refined to obtain subpixel precision by a feature-based Levenberg-Marquardt (FLM) method. It converges evidently faster than most other methods because of its feature-based characteristic.

Conclusions

We validate the performance of proposed method by testing with synthetic and real image sequences acquired by a hand-held digital still camera (DSC) and in comparison with an optical flow-based motion technique in terms of the squared sum of intensity differences (SSD) and correlation coefficient (CC). The results indicate that the proposed method is satisfactory in the registration accuracy and quality of DSC images.     

Rapid pedobarographic image registration based on contour curvature and optimization

Image registration, the process of optimally aligning homologous structures in multiple images, has recently been demonstrated to support automated pixel-level analysis of pedobarographic images and, subsequently, to extract unique and biomechanically relevant information from plantar pressure data. Recent registration methods have focused on robustness, with slow but globally powerful algorithms. In this paper, we present an alternative registration approach that affords both speed and accuracy, with the goal of making pedobarographic image registration more practical for near-real-time laboratory and clinical applications. The current algorithm first extracts centroid-based curvature trajectories from pressure image contours, and then optimally matches these curvature profiles using optimization based on dynamic programming. Special cases of disconnected images (that occur in high-arched subjects, for example) are dealt with by introducing an artificial spatially linear bridge between adjacent image clusters. Two registration algorithms were developed: a ‘geometric’ algorithm, which exclusively matched geometry, and a ‘hybrid’ algorithm, which performed subsequent pseudo-optimization. After testing the two algorithms on 30 control image pairs considered in a previous study, we found that, when compared with previously published results, the hybrid algorithm improved overlap ratio (p=0.010), but both current algorithms had slightly higher mean-squared error, assumedly because they did not consider pixel intensity. Nonetheless, both algorithms greatly improved the computational efficiency (25±8 and 53±9 ms per image pair for geometric and hybrid registrations, respectively). These results imply that registration-based pixel-level pressure image analyses can, eventually, be implemented for practical clinical purposes.     

Automatic model-based roentgen stereophotogrammetric analysis (RSA) of total knee prostheses

Conventional radiography is insensitive for early and accurate estimation of the mal-alignment and wear of knee prostheses. The two-staged (rough and fine) registration of the model-based RSA technique has recently been developed to in vivo estimate the prosthetic pose (i.e, location and orientation). In the literature, rough registration often uses template match or manual adjustment of the roentgen images. Additionally, possible error induced by the nonorthogonality of taking two roentgen images neither examined nor calibrated prior to fine registration. This study developed two RSA methods for automate the estimation of the prosthetic pose and decrease the nonorthogonality-induced error. The predicted results were validated by both simulative and experimental tests and compared with reported findings in the literature. The outcome revealed that the feature-recognized method automates pose estimation and significantly increases the execution efficiency up to about 50 times in comparison with the literature counterparts. Although the nonorthogonal images resulted in undesirable errors, the outline-optimized method can effectively compensate for the induced errors prior to fine registration. The superiority in automation, efficiency, and accuracy demonstrated the clinical practicability of the two proposed methods especially for the numerous fluoroscopic images of dynamic motion.     

Speckle characterization methods in ultrasound images – A review

Speckle has been widely considered a noisy feature in ultrasound images, thus it is intended to be suppressed and eliminated. On the other hand, speckle can be studied as a signal modeled by various statistical distributions or by analyzing its intensity with spatial relations in image space that characterize its nature, and hence, the nature of the underlying tissue. This knowledge can then be used in order to classify the different speckle regions into anatomical structures. In fact, speckle characterization in echocardiography and other ultrasonic images is important for motion tracking, tissue characterization, image segmentation, registration, and other medical applications for diagnosis, therapy planning and decision making. In this paper, we review and discuss various speckle characterization methods, which are often applied to confirm the speckle nature of the elements.     

Nonrigid Registration Regularized by Shape Information: Application to Atlas Construction of Cardiac CT Images

Cardiac atlases play an important role in the computer-aided diagnosis of cardiovascular diseases, in particular they need to deal with large and highly variable image datasets. In this paper, we propose a new nonrigid registration algorithm incorporating shape information, to produce comprehensive atlases. For one thing, the multiscale gradient orientation features of images are combined to form the construction of multifeature mutual information. Additionally, the shape information of multiple-objects in images is incorporated into the cost function for registration. We demonstrate the merits of the new registration algorithm on the 3D data sets of 15 patients. The experimental results show that the new registration algorithm can outperform the conventional intensity-based registration method. The obtained atlas can represent the cardiac structures more accurately.     

Optical-CT Dual-Modality Mapping Base on DRR Registration

Optical-CT dual-modality imaging requires the mapping between 2D fluorescence images and 3D body surface light flux. In this paper, we proposed an optical-CT dual-modality image mapping algorithm based on the Digitally Reconstructed Radiography (DRR) registration. In the process of registration, a series of DRR images were computed from CT data using the ray casting algorithm. Then, the improved HMNI similarity strategy based on Hausdorff distance was used to complete the registration of the white-light optical images and DRR virtual images. According to the corresponding relationship obtained by the image registration and the Lambert’s cosine law based on the pin-hole imaging model, the 3D light intensity distribution on the surface of the object could be solved. The feasibility and effectiveness of the mapping algorithm are verified by the irregular phantom and mouse experiments.