Markov random field based automatic image alignment for electron tomography

We present a method for automatic full-precision alignment of the images in a tomographic tilt series. Full-precision automatic alignment of cryo electron microscopy images has remained a difficult challenge to date, due to the limited electron dose and low image contrast. These facts lead to poor signal to noise ratio (SNR) in the images, which causes automatic feature trackers to generate errors, even with high contrast gold particles as fiducial features. To enable fully automatic alignment for full-precision reconstructions, we frame the problem probabilistically as finding the most likely particle tracks given a set of noisy images, using contextual information to make the solution more robust to the noise in each image. To solve this maximum likelihood problem, we use Markov Random Fields (MRF) to establish the correspondence of features in alignment and robust optimization for projection model estimation. The resulting algorithm, called Robust Alignment and Projection Estimation for Tomographic Reconstruction, or RAPTOR, has not needed any manual intervention for the difficult datasets we have tried, and has provided sub-pixel alignment that is as good as the manual approach by an expert user. We are able to automatically map complete and partial marker trajectories and thus obtain highly accurate image alignment. Our method has been applied to challenging cryo electron tomographic datasets with low SNR from intact bacterial cells, as well as several plastic section and X-ray datasets.     

Automatic alignment of transmission electron microscope tilt series without fiducial markers

Accurate image alignment is needed for computing three-dimensional reconstructions from transmission electron microscope tilt series. So far, the best results have been obtained by using colloidal gold beads as fiducial markers. If their use has not been possible for some reason, the only option has been the automatic cross-correlation-based registration methods. However, the latter methods are inaccurate and, as we will show, inappropriate for the whole problem. Conversely, we propose a novel method that uses the actual 3D motion model but works without any fiducial markers in the images. The method is based on matching and tracking some interest points of the intensity surface by first solving the underlying geometrical constraint of consecutive images in the tilt series. The results show that our method is near the gold marker alignment in the level of accuracy and hence opens the way for new opportunities in the analysis of electron tomography reconstructions, especially when markers cannot be used.     

Marker-free image registration of electron tomography tilt-series

Background  

Tilt series are commonly used in electron tomography as a means of collecting three-dimensional information from two-dimensional projections. A common problem encountered is the projection alignment prior to 3D reconstruction. Current alignment techniques usually employ gold particles or image derived markers to correctly align the images. When these markers are not present, correlation between adjacent views is used to align them. However, sequential pairwise correlation is prone to bias and the resulting alignment is not always optimal.     

Three-dimensional localization of immunogold markers using two tilted electron microscope recordings.

A method is presented to determine the three-dimensional positions of immuno-labeled gold markers from tilted electron micrograph recordings by using image processing techniques. The method consists of three basic modules: localization of the markers in the recordings, estimation of the motion parameters, and matching corresponding markers between the views. Localization consists of a segmentation step based on edge detection and region growing. It also allows for the separation of (visually) aggregated markers. Initial estimates for the motion parameters are obtained from a small number of user-indicated correspondences. A matching algorithm based on simulated annealing is used to find corresponding markers. With the resulting mapping, the motion parameters are updated and used in a new matching step, etc. Once the parameters are stable, the marker depths are retrieved. The developed method has been applied to semithin resin sections of A431 cells labeled for DNA and detected by silver-enhanced ultrasmall gold particles. It represents a reliable method to analyze the three-dimensional distribution of gold markers in electron microscope samples.     

2D fast rotational matching for image processing of biophysical data

In 3D single particle reconstruction, which involves the translational and rotational matching of a large number of electron microscopy (EM) images, the algorithmic performance is largely dependent on the efficiency and accuracy of the underlying 2D image alignment kernel. We present a novel fast rotational matching kernel for 2D images (FRM2D) that significantly reduces the cost of this alignment. The alignment problem is formulated using one translational and two rotational degrees of freedom. This allows us to take advantage of fast Fourier transforms (FFTs) in rotational space to accelerate the search of the two angular parameters, while the remaining translational parameter is explored, within a limited range, by exhaustive search. Since there are no boundary effects in FFTs of cyclic angular variables, we avoid the expensive zero padding associated with Fourier transforms in linear space. To verify the robustness of our method, efficiency and accuracy tests were carried out over a range of noise levels in realistic simulations of EM images. Performance tests against two standard alignment methods, resampling to polar coordinates and self-correlation, demonstrate that FRM2D compares very favorably to the traditional methods. FRM2D exhibits a comparable or higher robustness against noise and a significant gain in efficiency that depends on the fineness of the angular sampling and linear search range.     

CELL SURFACE LOCALIZATION OF THE 60 kDa HEAT SHOCK CHAPERONIN PROTEIN (hsp60) IN MAMMALIAN CELLS

To investigate whether the 60-kDa heat shock chaperonin protein (hsp60) is present on the surface of mammalian cells, we used immunogold labeling of intact cells and backscattered electron imaging to image gold particles. Chinese hamster ovary cells and the human leukemic CD4-positive T-cell line CEM-SS on glass coverslips were labeled using affinity-purified monoclonal and polyclonal antibodies specific for hsp60 and 30 nm gold markers. Cells were imaged using the scanning mode of the conventional transmission electron microscope. Backscattered electron imaging provided definitive identification of the gold markers while secondary electron imaging gave information on surface architecture. Labeling intensity was 250–800 gold particles per cell in Chinese hamster ovary cells and 600–2000 in CEM-SS human lymphoblasts. The finding of hsp60 on the cell surface of mammalian cells may signify chaperone involvement in surface functions.     

TMaCS: A hybrid template matching and classification system for partially-automated particle selection

Selection of particle images from electron micrographs presents a bottleneck in determining the structures of macromolecular assemblies by single particle electron cryomicroscopy (cryo-EM). The problem is particularly important when an experimentalist wants to improve the resolution of a 3D map by increasing by tens or hundreds of thousands of images the size of the dataset used for calculating the map. Although several existing methods for automatic particle image selection work well for large protein complexes that produce high-contrast images, it is well known in the cryo-EM community that small complexes that give low-contrast images are often refractory to existing automated particle image selection schemes. Here we develop a method for partially-automated particle image selection when an initial 3D map of the protein under investigation is already available. Candidate particle images are selected from micrographs by template matching with template images derived from projections of the existing 3D map. The candidate particle images are then used to train a support vector machine, which classifies the candidates as particle images or non-particle images. In a final step in the analysis, the selected particle images are subjected to projection matching against the initial 3D map, with the correlation coefficient between the particle image and the best matching map projection used to assess the reliability of the particle image. We show that this approach is able to rapidly select particle images from micrographs of a rotary ATPase, a type of membrane protein complex involved in many aspects of biology.     

Gold nanoparticle-protein arrays improve resolution for cryo-electron microscopy

Cryo-electron microscopy single particle analysis shows limited resolution due to poor alignment precision of noisy images taken under low electron exposure. Certain advantages can be obtained by assembling proteins into two-dimensional (2D) arrays since protein particles are locked into repetitive orientation, thus improving alignment precision. We present a labeling method to prepare protein 2D arrays using gold nanoparticles (NPs) interconnecting genetic tag sites on proteins. As an example, mycobacterium tuberculosis 20S proteasomes tagged with 6x-histidine were assembled into 2D arrays using 3.9-nm Au NPs functionalized with nickel-nitrilotriacetic acid. The averaged top-view images from the array particles showed higher resolution (by 6-8A) compared to analysis of single particles. The correct 7-fold symmetry was also evident by using array particles whereas it was not clear by analysis of a comparable number of single particles. The applicability of this labeling method for three-dimensional reconstruction of biological macromolecules is discussed.     

Image-based RSA: Roentgen stereophotogrammetric analysis based on 2D-3D image registration

Image-based Roentgen stereophotogrammetric analysis (IBRSA) integrates 2D-3D image registration and conventional RSA. Instead of radiopaque RSA bone markers, IBRSA uses 3D CT data, from which digitally reconstructed radiographs (DRRs) are generated. Using 2D-3D image registration, the 3D pose of the CT is iteratively adjusted such that the generated DRRs resemble the 2D RSA images as closely as possible, according to an image matching metric. Effectively, by registering all 2D follow-up moments to the same 3D CT, the CT volume functions as common ground. In two experiments, using RSA and using a micromanipulator as gold standard, IBRSA has been validated on cadaveric and sawbone scapula radiographs, and good matching results have been achieved. The accuracy was: |mu |< 0.083 mm for translations and |mu| < 0.023 degrees for rotations. The precision sigma in x-, y-, and z-direction was 0.090, 0.077, and 0.220 mm for translations and 0.155 degrees , 0.243 degrees , and 0.074 degrees for rotations. Our results show that the accuracy and precision of in vitro IBRSA, performed under ideal laboratory conditions, are lower than in vitro standard RSA but higher than in vivo standard RSA. Because IBRSA does not require radiopaque markers, it adds functionality to the RSA method by opening new directions and possibilities for research, such as dynamic analyses using fluoroscopy on subjects without markers and computer navigation applications.     

Automatic particle pickup method using a neural network has high accuracy by applying an initial weight derived from eigenimages: a new reference free method for single-particle analysis

The single-particle analysis is a structure-determining method for electron microscope (EM) images which does not require crystal. In this method, the projections are picked up and averaged by the images of similar Euler angles to improve the signal to noise ratio, and then create a 3-D reconstruction. The selection of a large number of particles from the cryo-EM micrographs is a pre-requisite for obtaining a high resolution. To pickup a low-contrast cryo-EM protein image, we have recently found that a three-layer pyramidal-type neural network is successful in detecting such a faint image, which had been difficult to detect by other methods. The connection weights between the input and hidden layers, which work as a matching filter, have revealed that they reflect characters of the particle projections in the training data. The images stored in terms of the connection weights were complex, more similar to the eigenimages which are created by the principal component analysis of the learning images rather than to the averages of the particle projections. When we set the initial learning weights according to the eigenimages in advance, the learning period was able to be shortened to less than half the time of the NN whose initial weights had been set randomly. Further, the pickup accuracy increased from 90 to 98%, and a combination of the matching filters were found to work as an integrated matching filter there. The integrated filters were amazingly similar to averaged projections and can be used directly as references for further two-dimensional averaging. Therefore, this research also presents a brand-new reference-free method for single-particle analysis.     

Kinematic study of root elongation in Arabidopsis thaliana with a novel image-analysis program

The measurement of the spatial profile of root elongation needs to determine matching points between time-lapse images and calculate their displacement. These data have been obtained by laborious manual methods in the past. Some computer-based programs have been developed to improve the measurement, but they require many time-series digital images or sprinkling graphite particles on the root prior to image capture. Here, we have developed GrowthTracer, a new image-analysis program for the kinematic study of root elongation. GrowthTracer employs a multiresolution image matching method with a nonlinear filter called the critical point filter (CPF), which extracts critical points from images at each resolution and can determine precise matching points by analysis of only two intact images, without pre-marking by graphite particles. This program calculates the displacement of each matching point and determines the displacement velocity profile along the medial axis of the root. In addition, a manual input of distinct matching points increases the matching accuracy. We show a successful application of this novel program for the kinematic analysis of root growth in Arabidopsis thaliana.     

Auto-accumulation method using simulated annealing enables fully automatic particle pickup completely free from a matching template or learning data

Single-particle analysis is a 3-D structure determining method using electron microscopy (EM). In this method, a large number of projections is required to create 3-D reconstruction. In order to enable completely automatic pickup without a matching template or a training data set, we established a brand-new method in which the frames to pickup particles are randomly shifted and rotated over the electron micrograph and, using the total average image of the framed images as an index, each frame reaches a particle. In this process, shifts are selected to increase the contrast of the average. By iterated shifts and further selection of the shifts, the frames are induced to shift so as to surround particles. In this algorithm, hundreds of frames are initially distributed randomly over the electron micrograph in which multi-particle images are dispersed. Starting with these frames, one of them is selected and shifted randomly, and acceptance or non-acceptance of its new position is judged using the simulated annealing (SA) method in which the contrast score of the total average image is adopted as an index. After iteration of this process, the position of each frame converges so as to surround a particle and the framed images are picked up. This method is the first unsupervised fully automatic particle picking method which is applicable to EM of various kinds of proteins, especially to low-contrasted cryo-EM protein images.     

Silver-enhanced colloidal gold as a cell surface marker for photoelectron microscopy

Colloidal gold labeling in conjunction with silver enhancement was investigated as a labeling technique for photoelectron microscopy (PEM). PEM uses UV-stimulated electron emission to image uncoated cell surfaces, and markers for cell surfaces need to be sufficiently photoemissive to be clearly visible against this background. Label contrast provided by 6 nm or 20 nm colloidal gold markers alone was compared to that provided by 6 nm markers after silver enhancement, using both direct and indirect labeling methods for fibronectin on human fibroblast cell surfaces. In all cases, details of the fibrillar fibronectin labeling distribution which were barely discernible before silver enhancement became highly visible against the cellular surface features. Two factors evidently contribute to the pronounced increase in label contrast with silver enhancement: (1) Increased particle size, which was documented by transmission electron microscopy, and (2) increased photoemission resulting from a silver coating on the enhanced gold markers, compared with the protein coating on the unenhanced gold markers. These data demonstrate that silver enhancement of colloidal gold labeling patterns in PEM images is a highly effective method for localization of specific sites on cell surfaces.     

Automated acquisition of electron microscopic random conical tilt sets

Single particle reconstruction using the random conical tilt data collection geometry is a robust method for the initial determination of macromolecular structures by electron microscopy. Unfortunately, the broad adoption of this powerful approach has been limited by the practical challenges inherent in manual data collection of the required pairs of matching high and low tilt images (typically 60 degrees and 0 degrees). The microscopist is obliged to keep the imaging area centered during tilting as well as to maintain accurate focus in the tilted image while minimizing the overall electron dose, a challenging and time consuming process. To help solve these problems, we have developed an automated system for the rapid acquisition of accurately aligned and focused tilt pairs. The system has been designed to minimize the dose incurred during alignment and focusing, making it useful in both negative stain and cryo-electron microscopy. The system includes a feature for montaging untilted images to ensure that all of the particles in the tilted image may be used in the reconstruction.     

Nano-fEM: Protein Localization Using Photo-activated Localization Microscopy and Electron Microscopy

Mapping the distribution of proteins is essential for understanding the function of proteins in a cell. Fluorescence microscopy is extensively used for protein localization, but subcellular context is often absent in fluorescence images. Immuno-electron microscopy, on the other hand, can localize proteins, but the technique is limited by a lack of compatible antibodies, poor preservation of morphology and because most antigens are not exposed to the specimen surface. Correlative approaches can acquire the fluorescence image from a whole cell first, either from immuno-fluorescence or genetically tagged proteins. The sample is then fixed and embedded for electron microscopy, and the images are correlated ^1-3. However, the low-resolution fluorescence image and the lack of fiducial markers preclude the precise localization of proteins. Alternatively, fluorescence imaging can be done after preserving the specimen in plastic. In this approach, the block is sectioned, and fluorescence images and electron micrographs of the same section are correlated ^4-7. However, the diffraction limit of light in the correlated image obscures the locations of individual molecules, and the fluorescence often extends beyond the boundary of the cell. Nano-resolution fluorescence electron microscopy (nano-fEM) is designed to localize proteins at nano-scale by imaging the same sections using photo-activated localization microscopy (PALM) and electron microscopy. PALM overcomes the diffraction limit by imaging individual fluorescent proteins and subsequently mapping the centroid of each fluorescent spot ^8-10. We outline the nano-fEM technique in five steps. First, the sample is fixed and embedded using conditions that preserve the fluorescence of tagged proteins. Second, the resin blocks are sectioned into ultrathin segments (70-80 nm) that are mounted on a cover glass. Third, fluorescence is imaged in these sections using the Zeiss PALM microscope. Fourth, electron dense structures are imaged in these same sections using a scanning electron microscope. Fifth, the fluorescence and electron micrographs are aligned using gold particles as fiducial markers. In summary, the subcellular localization of fluorescently tagged proteins can be determined at nanometer resolution in approximately one week.     

Application of template matching technique to particle detection in electron micrographs

Template matching together with the comprehensive theory of image formation in electron microscope provides an optimal (in Bayesian sense) tool for solving one of the outstanding problems in single particle analysis, i.e., automatic selection of particle views from noisy micrograph fields. The method is based on the assumption that the reference three-dimensional structure is known and that the relevant parameters of the model of the image formation process can be estimated. In the first stage of the procedure, a set of possible particle views is generated using the available reference structure. The template images are constructed as linear combinations of available particle views using a clustering technique. Next, the micrograph noise characteristic is established using an automated contrast transfer function (CTF) estimation procedure. Finally, the CTF parameters calculated are used to construct a matched filter and correlation functions corresponding to the available template images are calculated. In order to alleviate the problem of the biased caused by varying image formation conditions, a decision making strategy based on the predicted distribution of correlation coefficients is proposed. It is demonstrated that due to the inclusion of CTF considerations, the template matching method performed very well in a broad range of microscopy conditions.     

A microcomputer based system for three-dimensional reconstructions from tomographic or histologic sections

The reconstructions of three-dimensional (3-D) objects from serial two-dimensional (2-D) images can contribute to the understanding of many biologic structures, from organelles to organs and tissues. The 3-D reconstruction of sections can be divided into several major tasks: image acquisition, alignment of slices, internal object definition, object reconstruction and rotation of the completed image. A fast, versatile, interactive system was devised for the reconstruction of 3-D objects from serial 2-D images using a low-cost microcomputer, original programs and commercial software. The system allows reconstruction from any serial images, e.g., electron micrographs, histologic sections or computed tomograms. A photographic image or a microscopic field is acquired into the computer memory using a video digitizer. Slices are superimposed and aligned to each other using an operator-interactive program. A contour-(edge-) finding algorithm isolates an object of interest from the background image by "subtraction" of the image from an overlaid, slightly shifted identical image. Contours for each slice are input to a reconstruction procedure, which calculates the x, y and z coordinates of every point in a slice and the thickness and number of slices. It then calculates the illumination for every point using a given point source of light and an intensity-fading coefficient. Finally, the points are represented by cubes to provide dimension and reflective surfaces. A cube of appropriate shade and color represents in 2-D the equivalent of a 3-D object; this results in a very effective 3-D image. The reconstruction is rotated by recalculating the positions of every point defining the object and rebuilding the image.(ABSTRACT TRUNCATED AT 250 WORDS)     

Fast rotational matching of single-particle images

The presence of noise and absence of contrast in electron micrographs lead to a reduced resolution of the final 3D reconstruction, due to the inherent limitations of single-particle image alignment. The fast rotational matching (FRM) algorithm was introduced recently for an accurate alignment of 2D images under such challenging conditions. Here, we implemented this algorithm for the first time in a standard 3D reconstruction package used in electron microscopy. This allowed us to carry out exhaustive tests of the robustness and reliability in iterative orientation determination, classification, and 3D reconstruction on simulated and experimental image data. A classification test on GroEL chaperonin images demonstrates that FRM assigns up to 13% more images to their correct reference orientation, compared to the classical self-correlation function method. Moreover, at sub-nanometer resolution, GroEL and rice dwarf virus reconstructions exhibit a remarkable resolution gain of 10-20% that is attributed to the novel image alignment kernel.     

A Fast Image Alignment Approach for 2D Classification of Cryo-EM Images Using Spectral Clustering

Three-dimensional (3D) reconstruction in single-particle cryo-electron microscopy (cryo-EM) is a significant technique for recovering the 3D structure of proteins or other biological macromolecules from their two-dimensional (2D) noisy projection images taken from unknown random directions. Class averaging in single-particle cryo-EM is an important procedure for producing high-quality initial 3D structures, where image alignment is a fundamental step. In this paper, an efficient image alignment algorithm using 2D interpolation in the frequency domain of images is proposed to improve the estimation accuracy of alignment parameters of rotation angles and translational shifts between the two projection images, which can obtain subpixel and subangle accuracy. The proposed algorithm firstly uses the Fourier transform of two projection images to calculate a discrete cross-correlation matrix and then performs the 2D interpolation around the maximum value in the cross-correlation matrix. The alignment parameters are directly determined according to the position of the maximum value in the cross-correlation matrix after interpolation. Furthermore, the proposed image alignment algorithm and a spectral clustering algorithm are used to compute class averages for single-particle 3D reconstruction. The proposed image alignment algorithm is firstly tested on a Lena image and two cryo-EM datasets. Results show that the proposed image alignment algorithm can estimate the alignment parameters accurately and efficiently. The proposed method is also used to reconstruct preliminary 3D structures from a simulated cryo-EM dataset and a real cryo-EM dataset and to compare them with RELION. Experimental results show that the proposed method can obtain more high-quality class averages than RELION and can obtain higher reconstruction resolution than RELION even without iteration.