Markov random field based automatic image alignment for electron tomography

We present a method for automatic full-precision alignment of the images in a tomographic tilt series. Full-precision automatic alignment of cryo electron microscopy images has remained a difficult challenge to date, due to the limited electron dose and low image contrast. These facts lead to poor signal to noise ratio (SNR) in the images, which causes automatic feature trackers to generate errors, even with high contrast gold particles as fiducial features. To enable fully automatic alignment for full-precision reconstructions, we frame the problem probabilistically as finding the most likely particle tracks given a set of noisy images, using contextual information to make the solution more robust to the noise in each image. To solve this maximum likelihood problem, we use Markov Random Fields (MRF) to establish the correspondence of features in alignment and robust optimization for projection model estimation. The resulting algorithm, called Robust Alignment and Projection Estimation for Tomographic Reconstruction, or RAPTOR, has not needed any manual intervention for the difficult datasets we have tried, and has provided sub-pixel alignment that is as good as the manual approach by an expert user. We are able to automatically map complete and partial marker trajectories and thus obtain highly accurate image alignment. Our method has been applied to challenging cryo electron tomographic datasets with low SNR from intact bacterial cells, as well as several plastic section and X-ray datasets.     

Multiphase method for automatic alignment of transmission electron microscope images using markers

In order to successfully perform the 3D reconstruction in electron tomography, transmission electron microscope images must be accurately aligned or registered. So far, the problem is solved by either manually showing the corresponding fiducial markers from the set of images or automatically using simple correlation between the images on several rotations and scales. The present solutions, however, share the problem of being inefficient and/or inaccurate. We therefore propose a method in which the registration is automated using conventional colloidal gold particles as reference markers between images. We approach the problem from the computer vision viewpoint; hence, the alignment problem is divided into several subproblems: (1) finding initial matches from successive images, (2) estimating the epipolar geometry between consecutive images, (3) finding and localizing the gold particles with subpixel accuracy in each image, (4) predicting the probable matching gold particles using the epipolar constraint and its uncertainty, (5) matching and tracking the gold beads through the tilt series, and (6) optimizing the transformation parameters for the whole image set. The results show not only the reliability of the suggested method but also a high level of accuracy in alignment, since practically all the visible gold markers can be used.     

A procedure to deposit fiducial markers on vitreous cryo-sections for cellular tomography

We describe a novel approach for the accurate alignment of images in electron tomography of vitreous cryo-sections. Quantum dots, suspended in organic solvents at cryo-temperatures, are applied directly onto the sections and are subsequently used as fiducial markers to align the tilt series. Data collection can be performed from different regions of the vitreous sections, even when the sections touch the grid only at a few places. We present high-resolution tomograms of some organelles in cryo-sections of human skin cells using this method. The average error in image alignment was about 1nm and the resolution was estimated to be 5-7nm. Thus, the use of section-attached quantum dots as fiducial markers in electron tomography of vitreous cryo-sections facilitates high-resolution in situ 3D imaging of organelles and macromolecular complexes in their native hydrated state.     

Transform-based backprojection for volume reconstruction of large format electron microscope tilt series

Alignment of the individual images of a tilt series is a critical step in obtaining high-quality electron microscope reconstructions. We report on general methods for producing good alignments, and utilizing the alignment data in subsequent reconstruction steps. Our alignment techniques utilize bundle adjustment. Bundle adjustment is the simultaneous calculation of the position of distinguished markers in the object space and the transforms of these markers to their positions in the observed images, along the bundle of particle trajectories along which the object is projected to each EM image. Bundle adjustment techniques are general enough to encompass the computation of linear, projective or nonlinear transforms for backprojection, and can compensate for curvilinear trajectories through the object, sample warping, and optical aberration. We will also report on new reconstruction codes and describe our results using these codes.     

Fiducial-less alignment of cryo-sections

Cryo-electron tomography of vitreous sections is currently the most promising technique for visualizing arbitrary regions of eukaryotic cells or tissue at molecular resolution. Despite significant progress in the sample preparation techniques over the past few years, the three dimensional reconstruction using electron tomography is not as simple as in plunge frozen samples for various reasons, but mainly due to the effects of irradiation on the sections and the resulting poor alignment. Here, we present a new algorithm, which can provide a useful three-dimensional marker model after investigation of hundreds to thousands of observations calculated using local cross-correlation throughout the tilt series. The observations are chosen according to their coherence to a particular model and assigned to virtual markers. Through this type of measurement a merit figure can be calculated, precisely estimating the quality of the reconstruction. The merit figures of this alignment method are comparable to those obtained with plunge frozen samples using fiducial gold markers. An additional advantage of the algorithm is the implicit detection of areas in the sections that behave as rigid bodies and can thus be properly reconstructed.     

Reprint of "Fiducial-less alignment of cryo-sections" [J. Struct. Biol. 159 (2007) 413-423

Cryo-electron tomography of vitreous sections is currently the most promising technique for visualizing arbitrary regions of eukaryotic cells or tissue at molecular resolution. Despite significant progress in the sample preparation techniques over the past few years, the three dimensional reconstruction using electron tomography is not as simple as in plunge frozen samples for various reasons, but mainly due to the effects of irradiation on the sections and the resulting poor alignment. Here, we present a new algorithm, which can provide a useful three-dimensional marker model after investigation of hundreds to thousands of observations calculated using local cross-correlation throughout the tilt series. The observations are chosen according to their coherence to a particular model and assigned to virtual markers. Through this type of measurement a merit figure can be calculated, precisely estimating the quality of the reconstruction. The merit figures of this alignment method are comparable to those obtained with plunge frozen samples using fiducial gold markers. An additional advantage of the algorithm is the implicit detection of areas in the sections that behave as rigid bodies and can thus be properly reconstructed.     

Zernike phase contrast cryo-electron tomography of whole bacterial cells

Cryo-electron tomography (cryo-ET) provides three-dimensional (3D) structural information of bacteria preserved in a native, frozen-hydrated state. The typical low contrast of tilt-series images, a result of both the need for a low electron dose and the use of conventional defocus phase-contrast imaging, is a challenge for high-quality tomograms. We show that Zernike phase-contrast imaging allows the electron dose to be reduced. This limits movement of gold fiducials during the tilt series, which leads to better alignment and a higher-resolution reconstruction. Contrast is also enhanced, improving visibility of weak features. The reduced electron dose also means that more images at more tilt angles could be recorded, further increasing resolution.     

Electron tomography of vitreous sections from cultured mammalian cells

Cryo-electron tomography of appropriately thin, frozen-hydrated biological specimens has excellent potential for investigating the 3D macromolecular architecture of eukaryotic cells and tissues. Since cardiomyocytes are too thick to be visualised in an intact state, we grew immortalised cell line HL-1 to sub-confluency and harvested the cells by enzymatic detachment prior to hyperbaric freezing, ultramicrotomy, and tomography. We improved the efficiency of tomographic acquisition from vitreous cryosections by implementing two new features: (1) fluorescence microscopy at cryogenic temperatures to search for features of interest without expending any of the tolerable electron dose on secondary (non-imaging) tasks, and (2) the use of colloidal gold as fiducial markers. Vital fluorescent staining and subsequent cryo-fluorescence microscopy of vitreous sections were used to localise mitochondria lying in positions suitable for acquiring tilt series, taking into account section flatness, presence of contamination and proximity to grid bars. To provide a simple and robust means of aligning tomograms, we developed a universally applicable protocol for depositing colloidal gold onto vitreous sections, analogous to the method for applying quantum dots described by Masich et al. [Masich, S., Östberg, T., Norlén, L., Shupliakov, O., Daneholt, B., 2006. A procedure to deposit fiducial markers on vitreous cryo-sections for cellular tomography. J. Struct. Biol. 156, 461–468]. Tomograms of thin sections (nominal thickness 65–85 nm) of cardiac mitochondria revealed the interconnectivity of cristae and junctions with the inner mitochondrial membrane. In some cases, ATP synthases could be identified without ambiguity. These findings confirm the feasibility of investigating the structural biology of mammalian cells in three dimensions and at a resolution of 6–8 nm.     

Automated acquisition of electron microscopic random conical tilt sets

Single particle reconstruction using the random conical tilt data collection geometry is a robust method for the initial determination of macromolecular structures by electron microscopy. Unfortunately, the broad adoption of this powerful approach has been limited by the practical challenges inherent in manual data collection of the required pairs of matching high and low tilt images (typically 60 degrees and 0 degrees). The microscopist is obliged to keep the imaging area centered during tilting as well as to maintain accurate focus in the tilted image while minimizing the overall electron dose, a challenging and time consuming process. To help solve these problems, we have developed an automated system for the rapid acquisition of accurately aligned and focused tilt pairs. The system has been designed to minimize the dose incurred during alignment and focusing, making it useful in both negative stain and cryo-electron microscopy. The system includes a feature for montaging untilted images to ensure that all of the particles in the tilted image may be used in the reconstruction.     

Marker-free image registration of electron tomography tilt-series

Background  

Tilt series are commonly used in electron tomography as a means of collecting three-dimensional information from two-dimensional projections. A common problem encountered is the projection alignment prior to 3D reconstruction. Current alignment techniques usually employ gold particles or image derived markers to correctly align the images. When these markers are not present, correlation between adjacent views is used to align them. However, sequential pairwise correlation is prone to bias and the resulting alignment is not always optimal.     

Local refinement: an attempt to correct for shrinkage and distortion in electron tomography

A critical problem in electron tomography is the deformation of the specimen due to radiation, or "shrinkage," which interferes with image alignment and thereby limits resolution. Here, we describe a general strategy for refining preliminary reconstructions which allows the damage due to the shrinkage of plastic-embedded thin sectioned specimens (50-80 nm) to be corrected. The basic steps of the strategy involve: (a) the partition of the preliminary reconstruction into sub-volumes; (b) the extraction of corresponding sub-areas for each sub-volume from the micrographs of the tilt series; (c) the re-projection of each sub-volume according to the orientation parameters; and (d) the refinement of these parameters by correlating each sub-area to the corresponding computed projection. We tested the strategy by refining chemical synapses reconstructed from series imaged with conical, double and single tilt geometries. The results gathered with local refinement were evaluated by visually inspecting the structure of biological membranes in the maps. In an effort to quantify these improvements, we studied the refined maps using correlation criteria and mapped the corrections applied to the orientation parameters in each sub-volume of the reconstruction. Simulation experiments complemented the data gathered by correlation analysis. Based on these criteria, we concluded that local refinement significantly improves the overall quality of the reconstructions of chemical synapses calculated from series imaged with conical and double tilt geometries.     

Angle determination for side views in single particle electron microscopy

In order to build a first model in single particle electron microscopy the relative angular orientation of each image of a protein complex must be determined. These orientations can be described by three Eulerian angles. Images of complexes that present the same view can be aligned in two-dimensions and averaged in order to increase their signal-to-noise ratio. Based on these averaged images, several standard approaches exist for determining Euler angles for randomly oriented projection images. The common lines and angular reconstitution methods work well for particles with symmetry while the random conical tilting and related orthogonal tilt reconstruction methods work in most cases but require the acquisition of tilt pairs of images. For the situation where views of particles can be identified that are rotations about a single axis parallel to the grid, an alternative algorithm to determine the orientations of class averages without the need to acquire tilt pairs can be applied. This type of view of a complex is usually called a side view. This paper describes the detailed workings and characterization of an algorithm, named rotational analysis, which uses real-space fiducial markers derived from the averages themselves to determine the Euler angles for side views. We demonstrate how this algorithm works in practice by applying it to a data set of images of affinity-purified bovine mitochondrial ATP synthase.     

Focus gradient correction applied to tilt series image data used in electron tomography

The resolution in 3D reconstructions from tilt series is limited to the information below the first zero of the contrast transfer function unless the signal is corrected computationally. The restoration is usually based on the assumption of a linear space-invariant system and a linear relationship between object mass density and observed image contrast. The space-invariant model is no longer valid when applied to tilted micrographs because the defocus varies in a direction perpendicular to the tilt axis and with it the shape of the associated point spread function. In this paper, a method is presented for determining the defocus gradient in thin specimens such as sections and 2D crystals, and for restoration of the images subsequently used for 3D reconstruction. The alignment procedure for 3D reconstruction includes area matching and tilt geometry refinement. A map with limited resolution computed from uncorrected micrographs is compared to a volume computed from corrected micrographs with extended resolution.     

High resolution electron tomography and segmentation‐by‐modeling interpretation in Bsoft

Bsoft offers many tools for the processing of tomographic tilt series and the interpretation of tomograms. Since I introduced tomography into Bsoft almost two decades ago, the field has advanced significantly, requiring refinement of old algorithms and development of new ones. The current direct detectors allow us to collect data more efficiently and with better quality, progressing towards automation. The goal is then to also automate alignment of tilt series and reconstruction. I added an estimation of the specimen thickness as well as fiducialless alignment, to augment the existing fiducial‐based alignment. High‐resolution work requires correction for the contrast transfer function, in tomography complicated by the tilted specimen. For this, I developed a method to generate a power spectrum using the whole micrograph, compensating for tilting. This is followed by routine determination of the contrast transfer function, and correction for it during reconstruction. The next steps involve interpretation of the tomogram, either by subtomogram averaging where possible, or by segmentation and modeling otherwise. Such interpretation actually constitutes the main time‐consuming part of tomography and is less amenable to automation compared to the initial reconstruction.     

Reprint of "On the feasibility of visualizing ultrasmall gold labels in biological specimens by STEM tomography" [J. Struct. Biol. 159 (2007) 507-522

Labeling with heavy atom clusters attached to antibody fragments is an attractive technique for determining the 3D distribution of specific proteins in cells using electron tomography. However, the small size of the labels makes them very difficult to detect by conventional bright-field electron tomography. Here, we evaluate quantitative scanning transmission electron microscopy (STEM) at a beam voltage of 300 kV for detecting 11-gold atom clusters (Undecagold) and 1.4 nm-diameter nanoparticles (Nanogold) for a variety of specimens and imaging conditions. STEM images as well as tomographic tilt series are simulated by means of the NIST Elastic-Scattering Cross-Section Database for gold clusters embedded in carbon. The simulations indicate that the visibility in 2D of Undecagold clusters in a homogeneous matrix is maximized for low inner collection semi-angles of the STEM annular dark-field detector (15–20 mrad). Furthermore, our calculations show that the visibility of Undecagold in 3D reconstructions is significantly higher than in 2D images for an inhomogeneous matrix corresponding to fluctuations in local density. The measurements demonstrate that it is possible to detect Nanogold particles in plastic sections of tissue freeze-substituted in the presence of osmium. STEM tomography has the potential to localize specific proteins in permeabilized cells using antibody fragments tagged with small heavy atom clusters. Our quantitative analysis provides a framework for determining the detection limits and optimal experimental conditions for localizing these small clusters.     

On the feasibility of visualizing ultrasmall gold labels in biological specimens by STEM tomography

Labeling with heavy atom clusters attached to antibody fragments is an attractive technique for determining the 3D distribution of specific proteins in cells using electron tomography. However, the small size of the labels makes them very difficult to detect by conventional bright-field electron tomography. Here, we evaluate quantitative scanning transmission electron microscopy (STEM) at a beam voltage of 300 kV for detecting 11-gold atom clusters (Undecagold) and 1.4 nm-diameter nanoparticles (Nanogold) for a variety of specimens and imaging conditions. STEM images as well as tomographic tilt series are simulated by means of the NIST Elastic-Scattering Cross-Section Database for gold clusters embedded in carbon. The simulations indicate that the visibility in 2D of Undecagold clusters in a homogeneous matrix is maximized for low inner collection semi-angles of the STEM annular dark-field detector (15–20 mrad). Furthermore, our calculations show that the visibility of Undecagold in 3D reconstructions is significantly higher than in 2D images for an inhomogeneous matrix corresponding to fluctuations in local density. The measurements demonstrate that it is possible to detect Nanogold particles in plastic sections of tissue freeze-substituted in the presence of osmium. STEM tomography has the potential to localize specific proteins in permeabilized cells using antibody fragments tagged with small heavy atom clusters. Our quantitative analysis provides a framework for determining the detection limits and optimal experimental conditions for localizing these small clusters.     

An approach to examining model dependence in EM reconstructions using cross-validation

Reference bias refers to a common problem in fitting experimental data to an initial model. Given enough free parameters, a good fit of any experimental data to the model can be obtained, even if the experimental data contain only noise. Reference-based alignment methods used in electron microscopy (EM) are subject to this type of bias, in that images containing pure noise can regenerate the reference. Cross-validation is based on the idea that the experimental data used to assess the validity of the fitting should not be the same data as were used to do the fitting. Here we present the application of cross-validation to one form of reference-based alignment: 3D-projection matching in single-particle reconstructions. Our results show that reference bias is indeed present in reconstructions, but that the effect is small for real data compared to that for random noise, and that this difference in behavior is magnified, rather than diminished, during iterative refinement.     

Electron cryotomography sample preparation using the Vitrobot

Electron cryotomography is the highest-resolution structural technique currently available that can be applied to unique objects such as flexible large protein complexes, irregular viruses, organelles and small cells. Specimens are preserved in a near-native, 'frozen-hydrated' state by vitrification. The thickness of the vitreous ice must be optimized for each specimen, and gold fiducials are typically added to facilitate image alignment. Here, we describe in detail our protocols for electron cryotomography sample preparation including (i) introduction of fiducial markers into the sample and (ii) sample vitrification. Because we almost exclusively use an automated, climate-controlled plunge-freezing device (the FEI Vitrobot) to vitrify our samples, we discuss its operation and parameters in detail. A session in which eight grids are prepared takes 1.5-2 h.     

Perspective: Emerging strategies for determining atomic-resolution structures of macromolecular complexes within cells

In principle, electron cryo-tomography (cryo-ET) of thin portions of cells provides high-resolution images of the three-dimensional spatial arrangement of all members of the proteome. In practice, however, radiation damage creates a tension between recording images at many different tilt angles, but at correspondingly reduced exposure levels, versus limiting the number of tilt angles in order to improve the signal-to-noise ratio (SNR). Either way, it is challenging to read the available information out at the level of atomic structure. Here, we first review work that explores the optimal strategy for data collection, which currently seems to favor the use of a limited angular range for tilting the sample or even the use of a single image to record the high-resolution information. Looking then to the future, we point to the alternative of so-called “deconvolution microscopy”, which may be applied to tilt-series or optically-sectioned, focal series data. Recording data as a focal series has the advantage that little or no translational alignment of frames might be needed, and a three-dimensional reconstruction might require only 2/3 the number of images as does standard tomography. We also point to the unexploited potential of phase plates to increase the contrast, and thus to reduce the electron exposure levels while retaining the ability align and merge the data. In turn, using much lower exposures per image could have the advantage that high-resolution information is retained throughout the full data-set, whether recorded as a tilt series or a focal series of images.