Feature extraction from three-dimensional images in quantitative microscopy

Different methods are investigated in selecting and generating the appropriate microscope images for analysis of three-dimensional objects in quantitative microscopy. Traditionally, the ‘best’ focused image from a set is used for quantitative analysis. Such an objectively determined image is optimal for the extraction of some features, but may not be the best image for the extraction of all features. Various methods using multiple images are here developed to obtain a tighter distribution for all features.Three different approaches for analysis of images of stained cervical cells were analyzed. In the first approach, features are extracted from each image in the set. The feature values are then averaged to give the final result. In the second approach, a set of varying focused images are reconstructed to obtain a set of in-focus images. Features are then extracted from this set and averaged. In the third approach, a set of images in the three-dimensional scene is compressed into a single two-dimensional image. Four different compression methods are used. Features are then extracted from the resulting two-dimensional image. The third approach is employed on both the raw and transformed images.Each approach has its advantages and disadvantages. The first approach is fast and produces reasonable results. The second approach is more computationally expensive but produces the best results. The last approach overcomes the memory storage problem of the first two approaches since the set of images is compressed into one. The method of compression using the highest gradient pixel produces better results overall than other data reduction techniques and produces results comparable to the first approach.     

A novel approach to spot detection for two-dimensional gel electrophoresis images using pixel value collection

Separation of complex mixtures of proteins by two-dimensional gel electrophoresis (2-DE) is a fundamental component of current proteomic technology. Quantitative analysis of the images generated by digitization of such gels is critical for the identification of alterations in protein expression within a given biological system. Despite the availability of several commercially available software packages designed for this purpose, image analysis is extremely resource intensive, subjective and remains a major bottleneck. In addition to reducing throughput, the requirement for manual intervention results in the introduction of operator subjectivity, which can limit the statistical significance of the numerical data generated. A key requirement of image analysis is the accurate definition of protein spot boundaries using a suitable method of image segmentation. We describe a method of spot detection applicable to 2-DE image files using a segmentation method involving pixel value collection via serial analysis of the image through its range of density levels. This algorithm is reproducible, sensitive, accurate and primarily designed to be automatic, removing operator subjectivity. Furthermore, it is believed that this method may offer the potential for improved spot detection over currently available software.     

Iterative stable alignment and clustering of 2D transmission electron microscope images

Identification of homogeneous subsets of images in a macromolecular electron microscopy (EM) image data set is a critical step in single-particle analysis. The task is handled by iterative algorithms, whose performance is compromised by the compounded limitations of image alignment and K-means clustering. Here we describe an approach, iterative stable alignment and clustering (ISAC) that, relying on a new clustering method and on the concepts of stability and reproducibility, can extract validated, homogeneous subsets of images. ISAC requires only a small number of simple parameters and, with minimal human intervention, can eliminate bias from two-dimensional image clustering and maximize the quality of group averages that can be used for ab initio three-dimensional structural determination and analysis of macromolecular conformational variability. Repeated testing of the stability and reproducibility of a solution within ISAC eliminates heterogeneous or incorrect classes and introduces critical validation to the process of EM image clustering.     

Correlation analysis of two-dimensional gel electrophoretic protein patterns and biological variables

Background  

Two-dimensional gel electrophoresis (2DE) is a powerful technique to examine post-translational modifications of complexly modulated proteins. Currently, spot detection is a necessary step to assess relations between spots and biological variables. This often proves time consuming and difficult when working with non-perfect gels. We developed an analysis technique to measure correlation between 2DE images and biological variables on a pixel by pixel basis. After image alignment and normalization, the biological parameters and pixel values are replaced by their specific rank. These rank adjusted images and parameters are then put into a standard linear Pearson correlation and further tested for significance and variance.     

Alignment of two-dimensional electrophoresis gels

Two-dimensional electrophoresis is a major separating technique for proteins in proteomics. Alignment of gel images is critical for intra-laboratory or even more difficult inter-laboratory gel comparisons. In the paper, we propose a novel iterative closest point (ICP) method for 2D-gel electrophoresis image alignment. The paper seeks to introduce an information theoretic measure as one part of distance metric to gel image alignment. We combine intensity information of spots with geometric information of landmarks by applying information potential idea. The proposed method has been applied to both synthetic and real gel images accessible in public 2D-electrophoresis gel protein databases. The high accuracy and robustness of the algorithm indicate that it is promising for gel image alignment.     

Segmentation of center brains and optic lobes in 3D confocal images of adult fruit fly brains

Automatic alignment (registration) of 3D images of adult fruit fly brains is often influenced by the significant displacement of the relative locations of the two optic lobes (OLs) and the center brain (CB). In one of our ongoing efforts to produce a better image alignment pipeline of adult fruit fly brains, we consider separating CB and OLs and align them independently. This paper reports our automatic method to segregate CB and OLs, in particular under conditions where the signal to noise ratio (SNR) is low, the variation of the image intensity is big, and the relative displacement of OLs and CB is substantial.We design an algorithm to find a minimum-cost 3D surface in a 3D image stack to best separate an OL (of one side, either left or right) from CB. This surface is defined as an aggregation of the respective minimum-cost curves detected in each individual 2D image slice. Each curve is defined by a list of control points that best segregate OL and CB. To obtain the locations of these control points, we derive an energy function that includes an image energy term defined by local pixel intensities and two internal energy terms that constrain the curve’s smoothness and length. Gradient descent method is used to optimize this energy function. To improve both the speed and robustness of the method, for each stack, the locations of optimized control points in a slice are taken as the initialization prior for the next slice. We have tested this approach on simulated and real 3D fly brain image stacks and demonstrated that this method can reasonably segregate OLs from CBs despite the aforementioned difficulties.     

Mapping Diffusion in a Living Cell via the Phasor Approach

Diffusion of a fluorescent protein within a cell has been measured using either fluctuation-based techniques (fluorescence correlation spectroscopy (FCS) or raster-scan image correlation spectroscopy) or particle tracking. However, none of these methods enables us to measure the diffusion of the fluorescent particle at each pixel of the image. Measurement using conventional single-point FCS at every individual pixel results in continuous long exposure of the cell to the laser and eventual bleaching of the sample. To overcome this limitation, we have developed what we believe to be a new method of scanning with simultaneous construction of a fluorescent image of the cell. In this believed new method of modified raster scanning, as it acquires the image, the laser scans each individual line multiple times before moving to the next line. This continues until the entire area is scanned. This is different from the original raster-scan image correlation spectroscopy approach, where data are acquired by scanning each frame once and then scanning the image multiple times. The total time of data acquisition needed for this method is much shorter than the time required for traditional FCS analysis at each pixel. However, at a single pixel, the acquired intensity time sequence is short; requiring nonconventional analysis of the correlation function to extract information about the diffusion. These correlation data have been analyzed using the phasor approach, a fit-free method that was originally developed for analysis of FLIM images. Analysis using this method results in an estimation of the average diffusion coefficient of the fluorescent species at each pixel of an image, and thus, a detailed diffusion map of the cell can be created.     

A secure lossless recovery for medical images based on image encoding and data self-embedding

To protect the security of data outsourced to the cloud, the tampers detection and recovery for outsourced image have aroused the concern of people. A secure tampering detection and lossless recovery for medical images (MI) using permutation ordered binary (POB) number system is proposed. In the proposed scheme, the region of interest (ROI) of MI is first extracted, and then, ROI is divided into some no-overlapping blocks, and image encoding is conducted on these blocks based on the better compression performance of JPEG-LS for medical image. After that, the generated compression data by all the blocks are divided into high 4-bit and low 4-bit planes, and shuffling and combination are used to generate two plane images. Owing to the substantial redundancies space in the compressed data, the data of each plane are spread to the size of the original image. Lastly, authentication data of two bits is obtained for every pixel and inserted into the pixel itself within the each plane, and the corresponding 10-bit data is transformed into the POB value of 8-bit. Furthermore, encryption is implemented on the above image to produce two shares which can be outsourced to the cloud server. The users can detect tampered part and recover original image when they down load the shares from the cloud. Extensive experiments on some ordinary medical image and COVID-19 image datasets show that the proposed approach can locate the tampered parts within the MI, and the original MI can be recovered without any loss even if one of the shares are totally destroyed, or two shares are tampered at the ration not more than 50%. Some comparisons and analysis are given to show the better performance of the scheme.

     

Application of novel lossless compression of medical images using prediction and contextual error modeling

Conduction of tele-3D-computer assisted operations as well as other telemedicine procedures often requires highest possible quality of transmitted medical images and video. Unfortunately, those data types are always associated with high telecommunication and storage costs that sometimes prevent more frequent usage of such procedures. We present a novel algorithm for lossless compression of medical images that is extremely helpful in reducing the telecommunication and storage costs. The algorithm models the image properties around the current, unknown pixel and adjusts itself to the local image region. The main contribution of this work is the enhancement of the well known approach of predictor blends through highly adaptive determination of blending context on a pixel-by-pixel basis using classification technique. We show that this approach is well suited for medical image data compression. Results obtained with the proposed compression method on medical images are very encouraging, beating several well known lossless compression methods. The predictor proposed can also be used in other image processing applications such as segmentation and extraction of image regions.     

Pattern recognition and classification of images of biological macromolecules using artificial neural networks.

The goal of this work was to analyze an image data set and to detect the structural variability within this set. Two algorithms for pattern recognition based on neural networks are presented, one that performs an unsupervised classification (the self-organizing map) and the other a supervised classification (the learning vector quantization). The approach has a direct impact in current strategies for structural determination from electron microscopic images of biological macromolecules. In this work we performed a classification of both aligned but heterogeneous image data sets as well as basically homogeneous but otherwise rotationally misaligned image populations, in the latter case completely avoiding the typical reference dependency of correlation-based alignment methods. A number of examples on chaperonins are presented. The approach is computationally fast and robust with respect to noise. Programs are available through ftp.     

A quantitative determination of multi-protein interactions by the analysis of confocal images using a pixel-by-pixel assessment algorithm

MOTIVATION: Recent advances in confocal microscopy have allowed scientists to assess the expression, and to some extent, the interaction/colocalization of multiple molecules within cells and tissues. In some instances, accurately quantifying the colocalization of two or more proteins may be critical. This can require the acquisition of multiple Z plane images (Z stacks) throughout a specimen and, as such, we report here the successful development of a freeware, open-source image analysis tool, IMAJIN_COLOC, developed in PERL (v. 5.8, build 806), using the PERLMagick libraries (ImageMagick). Using a pixel-by-pixel analysis algorithm, IMAJIN_COLOC can analyze images for antigen expression (any number of colors) and can measure all possible combinations of colocalization for up to three colors by analyzing a Z stack gallery acquired for each sample. The simultaneous (i.e. in a single pass) analysis of three-color colocalization, and batch analysis capabilities are distinctive features of this program. RESULTS: A control image, containing known individual and colocalized pixel counts, was used to validate the accuracy of IMAJIN_COLOC. As further validation, pixel counts and colocalization values from the control image were compared to those obtained with the software packaged with the Zeiss laser-scanning microscope (LSM AIM, version 3.2). The values from both programs were found to be identical. To demonstrate the applicability of this program in addressing novel biological questions, we examined the role of neurons in eliciting an immune reaction in response to viral infection. Specifically, we successfully examined expression of the chemokine RANTES in measles virus (MV) infected hippocampal neurons and quantified changes in RANTES production throughout the disease period. The resultant quantitative data were also evaluated visually, using a gif image created during the analysis. AVAILABILITY: PERL (ActivePerl, version 5.8) is available at activestate.com; the PERLMagick libraries are available at imagemagick.org, and IMAJIN_COLOC, the source code and user documentation can be downloaded from http://www.fda.gov/cber/research/imaging/imageanalysis.htm.     

Statistical evaluation of confocal microscopy images

BACKGROUND: The coefficient of variation (CV) is defined as the standard deviation (sigma) of the fluorescent intensity of a population of beads or pixels expressed as a proportion or percentage of the mean (mu) intensity (CV = sigma/mu). The field of flow cytometry has used the CV of a population of bead intensities to determine if the flow cytometer is aligned correctly and performing properly. In a similar manner, the analysis of CV has been applied to the confocal laser scanning microscope (CLSM) to determine machine performance and sensitivity. METHODS: Instead of measuring 10,000 beads using a flow cytometer and determining the CV of this distribution of intensities, thousands of pixels are measured from within one homogeneous Spherotech 10-microm bead. Similar to a typical flow cytometry population that consists of 10,000 beads, a CLSM scanned image consists of a distribution of pixel intensities representing a population of approximately 100,000 pixels. In order to perform this test properly, it is important to have a population of homogeneous particles. A biological particle usually has heterogeneous pixel intensities that correspond to the details in the biological image and thus shows more variability as a test particle. RESULTS: The bead CV consisting of a population of pixel intensities is dependent on a number of machine variables that include frame averaging, photomultiplier tube (PMT) voltage, PMT noise, and laser power. The relationship among these variables suggests that the machine should be operated with lower PMT values in order to generate superior image quality. If this cannot be achieved, frame averaging will be necessary to reduce the CV and improve image quality. There is more image noise at higher PMT settings, making it is necessary to average more frames to reduce the CV values and improve image quality. The sensitivity of a system is related to system noise, laser light efficiency, and proper system alignment. It is possible to compare different systems for system performance and sensitivity if the laser power is maintained at a constant value. Using this bead CV test, 1 mW of 488 nm laser light measured on the scan head yielded a CV value of 4% with a Leica TCS-SP1 (75-mW argon-krypton laser) and a CV value of 1.3% with a Zeiss 510 (25-mW argon laser). A biological particle shows the same relationship between laser power, averaging, PMT voltage, and CV as do the beads. However, because the biological particle has heterogeneous pixel intensities, there is more particle variability, which does not make as useful as a test particle. CONCLUSIONS: This CV analysis of a 10-microm Spherotech fluorescent bead can help determine the sensitivity in a confocal microscope and the system performance. The relationship among the factors that influence image quality is explained from a statistical endpoint. The data obtained from this test provides a systematic method of reducing noise and increasing image clarity. Many components of a CLSM, including laser power, laser stability, PMT functionality, and alignment, influence the CV and determine if the equipment is performing properly. Preliminary results have shown that the bead CV can be used to compare different confocal microscopy systems with regard to performance and sensitivity. The test appears to be analogous to CV tests made on the flow cytometer to assess instrument performance and sensitivity. Published 2001 Wiley-Liss, Inc.     

Fast rotational matching of single-particle images

The presence of noise and absence of contrast in electron micrographs lead to a reduced resolution of the final 3D reconstruction, due to the inherent limitations of single-particle image alignment. The fast rotational matching (FRM) algorithm was introduced recently for an accurate alignment of 2D images under such challenging conditions. Here, we implemented this algorithm for the first time in a standard 3D reconstruction package used in electron microscopy. This allowed us to carry out exhaustive tests of the robustness and reliability in iterative orientation determination, classification, and 3D reconstruction on simulated and experimental image data. A classification test on GroEL chaperonin images demonstrates that FRM assigns up to 13% more images to their correct reference orientation, compared to the classical self-correlation function method. Moreover, at sub-nanometer resolution, GroEL and rice dwarf virus reconstructions exhibit a remarkable resolution gain of 10-20% that is attributed to the novel image alignment kernel.     

Application of multiplexed cysteine‐labeled complex protein sample for 2D electrophoretic gel alignment

The analysis of cellular subproteomes by 2DE is hampered by the difficulty of aligning gel images from samples that have very different protein composition. Here, we present a sensitive and cost‐effective fluorescent labeling method for analyzing protein samples that is not dependent on their composition. The alignment is guided by inclusion of a complex mixture of proteins that is co‐run with the sample. Maleimide‐conjugated fluorescent dyes Dy‐560 and Dy‐635 are used to label the cysteine residues of the sample of interest and the alignment standard, respectively. The two differently labeled mixtures are then combined and separated on a 2D gel and, after selective fluorescence detection, an unsupervised image registration process is used to align the protein patters. In a pilot study, this protocol significantly improved the accuracy of alignment of nuclear proteins with total cellular proteins.     

Preprocessing of two-dimensional gel electrophoresis images

Proteomics produces a huge amount of two-dimensional gel electrophoresis images. Their analysis can yield a lot of information concerning proteins responsible for different diseases or new unidentified proteins. However, an automatic analysis of such images requires an efficient tool for reducing noise in images. This allows proper detection of the spots' borders, which is important in protein quantification (as the spots' areas are used to determine the amounts of protein present in an analyzed mixture). Also in the feature-based matching methods the detected features (spots) can be described by additional attributes, such as area or shape. In our study, a comparison of different methods of noise reduction is performed in order to find out a method best suited for reducing noise in gel images. Among the compared methods there are the classical methods of linear filtering, e.g., the mean and Gaussian filtering, the nonlinear method, i.e., median filtering, and also the methods better suited for processing of nonstationary signals, such as spatially adaptive linear filtering and filtering in the wavelet domain. The best results are obtained by filtering of gel images in the wavelet domain, using the BayesThresh method of threshold value determination.     

Computer‐automated bird detection and counts in high‐resolution aerial images: a review

Bird surveys conducted using aerial images can be more accurate than those using airborne observers, but can also be more time‐consuming if images must be analyzed manually. Recent advances in digital cameras and image‐analysis software offer unprecedented potential for computer‐automated bird detection and counts in high‐resolution aerial images. We review the literature on this subject and provide an overview of the main image‐analysis techniques. Birds that contrast sharply with image backgrounds (e.g., bright birds on dark ground) are generally the most amenable to automated detection, in some cases requiring only basic image‐analysis software. However, the sophisticated analysis capabilities of modern object‐based image analysis software provide ways to detect birds in more challenging situations based on a variety of attributes including color, size, shape, texture, and spatial context. Some techniques developed to detect mammals may also be applicable to birds, although the prevalent use of aerial thermal‐infrared images for detecting large mammals is of limited applicability to birds because of the low pixel resolution of thermal cameras and the smaller size of birds. However, the increasingly high resolution of true‐color cameras and availability of small unmanned aircraft systems (drones) that can fly at very low altitude now make it feasible to detect even small shorebirds in aerial images. Continued advances in camera and drone technology, in combination with increasingly sophisticated image analysis software, now make it possible for investigators involved in monitoring bird populations to save time and resources by increasing their use of automated bird detection and counts in aerial images. We recommend close collaboration between wildlife‐monitoring practitioners and experts in the fields of remote sensing and computer science to help generate relevant, accessible, and readily applicable computer‐automated aerial photographic census techniques.     

A scene-segmentation method for multispectral cell images

A scene-segmentation method for two-color digitized images acquired from a Papanicolaou-stained cervical smear is proposed. The method first segments a scene into background, red cytoplasm, blue cytoplasm and nuclear regions by a pixel-wise classification and then merges the segmented regions for both types of cytoplasm into a single region. To create the minimum-distance classifier used for the pixel classification, class median vectors are selected from a two-dimensional histogram formed from the optical densities in the red and green images (scanned at 610 nm and 535 nm, respectively). Reference points defined from knowledge about the two-color images played an important role in selecting the vectors for the red and blue cytoplasm. This method was applied to 33 sets of the two-color images. The resulting segmented regions corresponded well with regions apparent to the the human observer. Three different investigations related to the method were carried out; these studies confirmed the suitability of the proposed method.     

The digital processing of the images in x-ray microanalysis

Image analysis of X-ray microanalytical maps of sodium, potassium, phosphorus, chloride and calcium was carried out on the IBAS image analysis system (OPTON, FRG). In the present work, algorithms were developed for getting a new semiquantitative information from the binary images of maps: the element profile on the chosen line, the isoconcentration mapping and the multiple elemental mapping. It is possible to get as many as eight different elemental maps on the same image (with 8-bit resolution of pixel) and to extract the pixels with element overlappings.     

A recognition method of multispectral images of soybean canopies based on neural network

Multispectral images of soybean canopies can reflect plant physiological information and growth status effectively, which is of great significance for soybean high-quality breeding, scientific cultivation, and fine management. At present, it is uneven of the gray level difference of the soybean multispectral images occurred in the leaf edge, and is also small of the gray level difference between the target and the background, resulting in inaccurate recognition of the soybean canopies from the multispectral images. Thus, a multispectral images' recognition method of soybean canopies was proposed based on the neural network. First, the method of Gaussian smoothing filter was used to preprocess the raw soybean multispectral images (green light, near-infrared, red light, red edge, and visible light), which maintained the leaf edge details of the soybean multispectral image. Second, the feedforward neural network model was established to extract the canopy region in the soybean multispectral images. In addition, image morphology operation was used to improve the recognition effects of the soybean canopy. Finally, four quantitative indexes were used to evaluate the experimental results. The results showed that the average effective segmentation rate of the proposed method was 91.69%, the under-segmentation rate was reduced by 33.34%, and the over-segmentation rate was reduced by 48.43%. The difference between the pixel average entropy of the proposed method and the standard canopy image was only 0.2295. The research results can provide not only reliable data for further analysis of physiological and ecological indexes of the soybean canopy, but also technical support for multispectral image recognition of other crop canopies.     

Non-rigid registration of serial dedicated breast CT,longitudinal dedicated breast CT and PET/CT images using the diffeomorphic demons method

Rationale and objectivesDedicated breast CT and PET/CT scanners provide detailed 3D anatomical and functional imaging data sets and are currently being investigated for applications in breast cancer management such as diagnosis, monitoring response to therapy and radiation therapy planning. Our objective was to evaluate the performance of the diffeomorphic demons (DD) non-rigid image registration method to spatially align 3D serial (pre- and post-contrast) dedicated breast computed tomography (CT), and longitudinally-acquired dedicated 3D breast CT and positron emission tomography (PET)/CT images.MethodsThe algorithmic parameters of the DD method were optimized for the alignment of dedicated breast CT images using training data and fixed. The performance of the method for image alignment was quantitatively evaluated using three separate data sets; (1) serial breast CT pre- and post-contrast images of 20 women, (2) breast CT images of 20 women acquired before and after repositioning the subject on the scanner, and (3) dedicated breast PET/CT images of 7 women undergoing neo-adjuvant chemotherapy acquired pre-treatment and after 1 cycle of therapy.ResultsThe DD registration method outperformed no registration (p < 0.001) and conventional affine registration (p ≤ 0.002) for serial and longitudinal breast CT and PET/CT image alignment. In spite of the large size of the imaging data, the computational cost of the DD method was found to be reasonable (3–5 min).ConclusionsCo-registration of dedicated breast CT and PET/CT images can be performed rapidly and reliably using the DD method. This is the first study evaluating the DD registration method for the alignment of dedicated breast CT and PET/CT images.