Retrospective: radiation damage and its associated "information limitations"

The fact that radiation damage would limit the usefulness of electron microscopy with biological specimens was a concern in the earliest days of the field. Good estimates of what that limitation must be can be made by using Rose's empirical relationship between the inherent image contrast, the exposure used to record an image, and the smallest feature size that is detectable. Such estimates show that it is necessary to average many images in order to obtain statistically well-defined data at high resolution. Structures are now routinely obtained by averaging large numbers of shot-noise limited images, and some of these extend to atomic resolution. The signal level in current images is nevertheless far below what physics would allow it to be. A possible explanation is that beam-induced movement limits the quality of images recorded by electron microscopy. For specimens embedded in vitreous ice, beam-induced movement can even be severe enough to limit the resolution achieved during tomographic reconstruction. The fact that very high-quality images can nevertheless be obtained, although only unpredictably, suggests that it may be possible to devise new techniques of specimen preparation and/or data collection that at least partially overcome beam-induced movement. If so, the need for image averaging would be correspondingly reduced.     

Fast rotational matching of single-particle images

The presence of noise and absence of contrast in electron micrographs lead to a reduced resolution of the final 3D reconstruction, due to the inherent limitations of single-particle image alignment. The fast rotational matching (FRM) algorithm was introduced recently for an accurate alignment of 2D images under such challenging conditions. Here, we implemented this algorithm for the first time in a standard 3D reconstruction package used in electron microscopy. This allowed us to carry out exhaustive tests of the robustness and reliability in iterative orientation determination, classification, and 3D reconstruction on simulated and experimental image data. A classification test on GroEL chaperonin images demonstrates that FRM assigns up to 13% more images to their correct reference orientation, compared to the classical self-correlation function method. Moreover, at sub-nanometer resolution, GroEL and rice dwarf virus reconstructions exhibit a remarkable resolution gain of 10-20% that is attributed to the novel image alignment kernel.     

Automatic particle pickup method using a neural network has high accuracy by applying an initial weight derived from eigenimages: a new reference free method for single-particle analysis

The single-particle analysis is a structure-determining method for electron microscope (EM) images which does not require crystal. In this method, the projections are picked up and averaged by the images of similar Euler angles to improve the signal to noise ratio, and then create a 3-D reconstruction. The selection of a large number of particles from the cryo-EM micrographs is a pre-requisite for obtaining a high resolution. To pickup a low-contrast cryo-EM protein image, we have recently found that a three-layer pyramidal-type neural network is successful in detecting such a faint image, which had been difficult to detect by other methods. The connection weights between the input and hidden layers, which work as a matching filter, have revealed that they reflect characters of the particle projections in the training data. The images stored in terms of the connection weights were complex, more similar to the eigenimages which are created by the principal component analysis of the learning images rather than to the averages of the particle projections. When we set the initial learning weights according to the eigenimages in advance, the learning period was able to be shortened to less than half the time of the NN whose initial weights had been set randomly. Further, the pickup accuracy increased from 90 to 98%, and a combination of the matching filters were found to work as an integrated matching filter there. The integrated filters were amazingly similar to averaged projections and can be used directly as references for further two-dimensional averaging. Therefore, this research also presents a brand-new reference-free method for single-particle analysis.     

Patch averaging of electron images of GP3*I crystals with variable thickness.

The combination of Fourier and correlation averaging techniques with multivariate statistical analysis and classification, a method known as patch averaging, is used to analyze untilted and tilted images of negatively stained GP32*I crystals, which exhibit variable thicknesses in a single crystal. Within a single image, coherent areas of the same apparent thickness can be distinguished from areas of differing thicknesses. Analysis using the phase relationships among symmetry-related reflections from reconstituted images obtained from untilted micrographs confirms the ability of the method to classify these variable thicknesses properly. Furthermore, the phases from some of the reconstituted images obtained from both untilted and tilted micrographs were found to match well with the phases in a previously determined three-dimensional data set of this crystal with pg symmetry along the crystallographic b axis. These results indicate the utility of the patch averaging procedures in the structural determination of protein crystals with different thicknesses.     

IPET and FETR: experimental approach for studying molecular structure dynamics by cryo-electron tomography of a single-molecule structure

The dynamic personalities and structural heterogeneity of proteins are essential for proper functioning. Structural determination of dynamic/heterogeneous proteins is limited by conventional approaches of X-ray and electron microscopy (EM) of single-particle reconstruction that require an average from thousands to millions different molecules. Cryo-electron tomography (cryoET) is an approach to determine three-dimensional (3D) reconstruction of a single and unique biological object such as bacteria and cells, by imaging the object from a series of tilting angles. However, cconventional reconstruction methods use large-size whole-micrographs that are limited by reconstruction resolution (lower than 20 Å), especially for small and low-symmetric molecule (<400 kDa). In this study, we demonstrated the adverse effects from image distortion and the measuring tilt-errors (including tilt-axis and tilt-angle errors) both play a major role in limiting the reconstruction resolution. Therefore, we developed a “focused electron tomography reconstruction” (FETR) algorithm to improve the resolution by decreasing the reconstructing image size so that it contains only a single-instance protein. FETR can tolerate certain levels of image-distortion and measuring tilt-errors, and can also precisely determine the translational parameters via an iterative refinement process that contains a series of automatically generated dynamic filters and masks. To describe this method, a set of simulated cryoET images was employed; to validate this approach, the real experimental images from negative-staining and cryoET were used. Since this approach can obtain the structure of a single-instance molecule/particle, we named it individual-particle electron tomography (IPET) as a new robust strategy/approach that does not require a pre-given initial model, class averaging of multiple molecules or an extended ordered lattice, but can tolerate small tilt-errors for high-resolution single “snapshot” molecule structure determination. Thus, FETR/IPET provides a completely new opportunity for a single-molecule structure determination, and could be used to study the dynamic character and equilibrium fluctuation of macromolecules.     

UCSF tomography: an integrated software suite for real-time electron microscopic tomographic data collection, alignment, and reconstruction

A real-time alignment and reconstruction scheme for electron microscopic tomography (EMT) has been developed and integrated within our UCSF tomography data collection software. This newly integrated software suite provides full automation from data collection to real-time reconstruction by which the three-dimensional (3D) reconstructed volume is immediately made available at the end of each data collection. Real-time reconstruction is achieved by calculating a weighted back-projection on a small Linux cluster (five dual-processor compute nodes) concurrently with the UCSF tomography data collection running on the microscope's computer, and using the fiducial-marker free alignment data generated during the data collection process. The real-time reconstructed 3D volume provides users with immediate feedback to fully asses all aspects of the experiment ranging from sample choice, ice thickness, experimental parameters to the quality of specimen preparation. This information can be used to guide subsequent data collections. Access to the reconstruction is especially useful in low-dose cryo EMT where such information is very difficult to obtain due to extraordinary low signal to noise ratio in each 2D image. In our environment, we generally collect 2048 x 2048 pixel images which are subsequently computationally binned four-fold for the on-line reconstruction. Based upon experiments performed with thick and cryo specimens at various CCD magnifications (50000x-80000x), alignment accuracy is sufficient to support this reduced resolution but should be refined before calculating a full resolution reconstruction. The reduced resolution has proven to be quite adequate to assess sample quality, or to screen for the best data set for full-resolution reconstruction, significantly improving both productivity and efficiency of system resources. The total time from start of data collection to a final reconstructed volume (512 x 512 x 256 pixels) is about 50 min for a +/-70 degrees 2k x 2k pixel tilt series acquired at every 1 degrees.     

A Fast Image Alignment Approach for 2D Classification of Cryo-EM Images Using Spectral Clustering

Three-dimensional (3D) reconstruction in single-particle cryo-electron microscopy (cryo-EM) is a significant technique for recovering the 3D structure of proteins or other biological macromolecules from their two-dimensional (2D) noisy projection images taken from unknown random directions. Class averaging in single-particle cryo-EM is an important procedure for producing high-quality initial 3D structures, where image alignment is a fundamental step. In this paper, an efficient image alignment algorithm using 2D interpolation in the frequency domain of images is proposed to improve the estimation accuracy of alignment parameters of rotation angles and translational shifts between the two projection images, which can obtain subpixel and subangle accuracy. The proposed algorithm firstly uses the Fourier transform of two projection images to calculate a discrete cross-correlation matrix and then performs the 2D interpolation around the maximum value in the cross-correlation matrix. The alignment parameters are directly determined according to the position of the maximum value in the cross-correlation matrix after interpolation. Furthermore, the proposed image alignment algorithm and a spectral clustering algorithm are used to compute class averages for single-particle 3D reconstruction. The proposed image alignment algorithm is firstly tested on a Lena image and two cryo-EM datasets. Results show that the proposed image alignment algorithm can estimate the alignment parameters accurately and efficiently. The proposed method is also used to reconstruct preliminary 3D structures from a simulated cryo-EM dataset and a real cryo-EM dataset and to compare them with RELION. Experimental results show that the proposed method can obtain more high-quality class averages than RELION and can obtain higher reconstruction resolution than RELION even without iteration.     

A Monte Carlo investigation of the spatial resolution performance of a small-animal PET scanner designed for mouse brain imaging studies

Our laboratory has developed PET detectors with depth-encoding accuracy of ∼2 mm based on finely pixelated crystals with a tapered geometry, readout at both ends with position-sensitive avalanche photodiodes (PSAPDs). These detectors are currently being used in our laboratory to build a one-ring high resolution PET scanner for mouse brain imaging studies. Due to the inactive areas around the PSAPDs, large gaps exist between the detector modules which can degrade the image spatial resolution obtained using analytical reconstruction with filtered backprojection (FBP). In this work, the Geant4-based GATE Monte Carlo package was used to assist in determining whether gantry rotation was necessary and to assess the expected spatial resolution of the system. The following factors were investigated: rotating vs. static gantry modes with and without compensation of missing data using the discrete cosine transform (DCT) method, two levels of depth-encoding, and positron annihilation effects for ¹⁸F. Our results indicate that while the static scanner produces poor quality FBP images with streak and ring artifacts, the image quality was greatly improved after compensation of missing data. The simulation indicates that the expected FWHM system spatial resolution is 0.70 ± 0.05 mm, which approaches the predicted limit of 0.5 mm FWHM due to positron range, photon non-colinearity and physical detector element size effects. We conclude that excellent reconstructed resolution without gantry rotation is possible even using FBP if the gaps are appropriately handled and that this design can approach the resolution limits set by positron annihilation physics.     

It’s noisy out there! A review of denoising techniques in cryo-electron tomography

Cryo-electron tomography is the only technique that can provide sub-nanometer resolved images of cell regions or even whole cells, without the need of labeling or staining methods. Technological advances over the past decade in electron microscope stability, cameras, stage precision and software have resulted in faster acquisition speeds and considerably improved resolution. In pursuit of even better image resolution, researchers seek to reduce noise – a crucial factor affecting the reliability of the tomogram interpretation and ultimately limiting the achieved resolution. Sub-tomogram averaging is the method of choice for reducing noise in repetitive objects. However, when averaging is not applicable, a trade-off between reducing noise and conserving genuine image details must be achieved. Thus, denoising is an important process that improves the interpretability of the tomogram not only directly but also by facilitating other downstream tasks, such as segmentation and 3D visualization. Here, I review contemporary denoising techniques for cryo-electron tomography by taking into account noise-specific properties of both reconstruction and detector noise. The outcomes of different techniques are compared, in order to help researchers select the most appropriate for each dataset and to achieve better and more reliable interpretation of the tomograms.     

Reconstruction of axial tomographic high resolution data from confocal fluorescence microscopy: a method for improving 3D FISH images.

Fluorescent confocal laser scanning microscopy allows an improved imaging of microscopic objects in three dimensions. However, the resolution along the axial direction is three times worse than the resolution in lateral directions. A method to overcome this axial limitation is tilting the object under the microscope, in a way that the direction of the optical axis points into different directions relative to the sample. A new technique for a simultaneous reconstruction from a number of such axial tomographic confocal data sets was developed and used for high resolution reconstruction of 3D-data both from experimental and virtual microscopic data sets. The reconstructed images have a highly improved 3D resolution, which is comparable to the lateral resolution of a single deconvolved data set. Axial tomographic imaging in combination with simultaneous data reconstruction also opens the possibility for a more precise quantification of 3D data. The color images of this publication can be accessed from http://www.esacp.org/acp/2000/20-1/heintzmann.++ +htm. At this web address an interactive 3D viewer is additionally provided for browsing the 3D data. This java applet displays three orthogonal slices of the data set which are dynamically updated by user mouse clicks or keystrokes.     

Improvement of image classification using wavelet coefficients with structured-based neural network

Image classification is a challenging problem in organizing a large image database. However, an effective method for such an objective is still under investigation. A method based on wavelet analysis to extract features for image classification is presented in this paper. After an image is decomposed by wavelet, the statistics of its features can be obtained by the distribution of histograms of wavelet coefficients, which are respectively projected onto two orthogonal axes, i.e., x and y directions. Therefore, the nodes of tree representation of images can be represented by the distribution. The high level features are described in low dimensional space including 16 attributes so that the computational complexity is significantly decreased. 2,800 images derived from seven categories are used in experiments. Half of the images were used for training neural network and the other images used for testing. The features extracted by wavelet analysis and the conventional features are used in the experiments to prove the efficacy of the proposed method. The classification rate on the training data set with wavelet analysis is up to 91%, and the classification rate on the testing data set reaches 89%. Experimental results show that our proposed approach for image classification is more effective.     

Averaging data derived from images of helical structures with different symmetries.

There are many examples of macromolecules that form helical tubes or crystals, which are useful for structure determination by electron microscopy and image processing. Helical crystals can be thought of as two-dimensional crystals that have been rolled into a cylinder such that two lattice points are superimposed. In many real cases, helical crystals of a particular macromolecule derive from an identical two-dimensional lattice but have different lattice points superimposed, thus producing different helical symmetries which cannot be simply averaged in Fourier-space. When confronted with this situation, one can select images corresponding to one of the observed symmetries at the expense of reducing the number of images that can be used for data collection and averaging, or one can calculate separate density maps from each symmetry, then align and average them together in real-space. Here, we present a third alternative, which is based on averaging of the Fourier-Bessel coefficients, gn,l(r), and which allows the inclusion of data from all symmetry groups derived from a common two-dimensional lattice. The method is straightforward and simple in practice and is shown, through a specific example with real data, to give results comparable to real-space averaging.     

基于对象特征的山东省丘陵地区多时相遥感土地覆被自动分类

对于基于像元的土地覆被分类来说,植被的分类是难点。使用多时相面向对象分类方法可以较好的解决这个问题。以山东省烟台市丘陵地区为研究区,采用Landsat TM(Landsat Thematic Mapper remotely sensed imagery)、DEM(Digital Elevation Model)、坡度、坡位、坡向等多种数据,利用基于对象特征的多时相分类方法对研究区进行土地覆盖自动分类。首先对影像进行多尺度分割并检验分割结果选取合适的分割尺度,然后分析对象的光谱、纹理、形状特征。根据各类地物的光谱特征、地理相关性、形状、空间分布等特征,明确类别之间的差异。建立决策树使用隶属度函数进行模糊分类,借助支持向量机提高分类精度。研究结果表明,通过使用多时相影像采用面向对象分类方法,相对于传统的基于像素的分类可以明显提高分类精度,尤其是解决了乔灌草的区分问题。     

Pipeline for acquisition of quantitative data on segmentation gene expression from confocal images

We describe a data pipeline developed to extract the quantitative data on segmentation gene expression from confocal images of gene expression patterns in Drosophila. The pipeline consists of five steps: image segmentation, background removal, temporal characterization of an embryo, data registration and data averaging. This pipeline was successfully applied to obtain quantitative gene expression data at cellular resolution in space and at the 6.5-minute resolution in time, as well as to construct a spatiotemporal atlas of segmentation gene expression. Each data pipeline step can be easily adapted to process a wide range of images of gene expression patterns.     

Pipeline for acquisition of quantitative data on segmentation gene expression from confocal images

We describe a data pipeline developed to extract the quantitative data on segmentation gene expression from confocal images of gene expression patterns in Drosophila. The pipeline consists of 5 steps: image segmentation, background removal, temporal characterization of an embryo, data registration and data averaging. This pipeline was successfully applied to obtain quantitative gene expression data at cellular resolution in space and at the 6.5 minute resolution in time, as well as to construct a spatiotemporal atlas of segmentation gene expression. Each data pipeline step can be easily adapted to process a wide range of images of gene expression patterns.     

Chromatin investigation in the nucleus using a phasor approach to structured illumination microscopy

Chromatin in the nucleus is organized in functional sites at variable level of compaction. Structured illumination microscopy (SIM) can be used to generate three-dimensional super-resolution (SR) imaging of chromatin by changing in phase and in orientation a periodic line illumination pattern. The spatial frequency domain is the natural choice to process SIM raw data and to reconstruct an SR image. Using an alternative approach, we demonstrate that the additional spatial information encoded in the knowledge of the position of the illumination pattern can be efficiently decoded using a generalized version of separation of photon by lifetime tuning (SPLIT) that does not require lifetime measurements. In the resulting SPLIT-SIM, the SR image is obtained by isolating a fraction of the intensity corresponding to the center of the diffraction-limited point spread function. This extends the use of the SPLIT approach from stimulated emission depletion microscopy to SIM. The SPLIT-SIM algorithm is based only on phasor analysis and does not require deconvolution. We show that SPLIT-SIM can be used to generate SR images of chromatin organizational motifs with tunable resolution and can be a valuable tool for the imaging of functional sites in the nucleus.     

Particle-verification for single-particle, reference-based reconstruction using multivariate data analysis and classification

As collection of electron microscopy data for single-particle reconstruction becomes more efficient, due to electronic image capture, one of the principal limiting steps in a reconstruction remains particle-verification, which is especially costly in terms of user input. Recently, some algorithms have been developed to window particles automatically, but the resulting particle sets typically need to be verified manually. Here we describe a procedure to speed up verification of windowed particles using multivariate data analysis and classification. In this procedure, the particle set is subjected to multi-reference alignment before the verification. The aligned particles are first binned according to orientation and are binned further by K-means classification. Rather than selection of particles individually, an entire class of particles can be selected, with an option to remove outliers. Since particles in the same class present the same view, distinction between good and bad images becomes more straightforward. We have also developed a graphical interface, written in Python/Tkinter, to facilitate this implementation of particle-verification. For the demonstration of the particle-verification scheme presented here, electron micrographs of ribosomes are used.     

A direction‐selective local‐thresholding method,DSLT, in combination with a dye‐based method for automated three‐dimensional segmentation of cells and airspaces in developing leaves

Quantifying the anatomical data acquired from three‐dimensional (3D) images has become increasingly important in recent years. Visualization and image segmentation are essential for acquiring accurate and detailed anatomical data from images; however, plant tissues such as leaves are difficult to image by confocal or multi‐photon laser scanning microscopy because their airspaces generate optical aberrations. To overcome this problem, we established a staining method based on Nile Red in silicone‐oil solution. Our staining method enables color differentiation between lipid bilayer membranes and airspaces, while minimizing any damage to leaf development. By repeated applications of our staining method we performed time‐lapse imaging of a leaf over 5 days. To counteract the drastic decline in signal‐to‐noise ratio at greater tissue depths, we also developed a local thresholding method (direction‐selective local thresholding, DSLT) and an automated iterative segmentation algorithm. The segmentation algorithm uses the DSLT to extract the anatomical structures. Using the proposed methods, we accurately segmented 3D images of intact leaves to single‐cell resolution, and measured the airspace volumes in intact leaves.     

Automated region of interest retrieval and classification using spectral analysis

Efficient use of whole slide imaging in pathology needs automated region of interest (ROI) retrieval and classification, through the use of image analysis and data sorting tools. One possible method for data sorting uses Spectral Analysis for Dimensionality Reduction. We present some interesting results in the field of histopathology and cytohematology. In histopathology, we developed a Computer-Aided Diagnosis system applied to low-resolution images representing the totality of histological breast tumour sections. The images can be digitized directly at low resolution or be obtained from sub-sampled high-resolution virtual slides. Spectral Analysis is used (1) for image segmentation (stroma, tumour epithelium), by determining a "distance" between all the images of the database, (2) for choosing representative images and characteristic patterns of each histological type in order to index them, and (3) for visualizing images or features similar to a sample provided by the pathologist. In cytohematology, we studied a blood smear virtual slide acquired through high resolution oil scanning and Spectral Analysis is used to sort selected nucleated blood cell classes so that the pathologist may easily focus on specific classes whose morphology could then be studied more carefully or which can be analyzed through complementary instruments, like Multispectral Imaging or Raman MicroSpectroscopy.     

X-ray diffraction studies of 14-filament models of deoxygenated sickle cell hemoglobin fibers. Models based on electron micrograph reconstructions

The transforms of a large number of models of deoxygenated sickle hemoglobin fibers, related to that derived from image reconstruction of electron micrographs, have been calculated and compared with X-ray diffraction data of 15 A resolution. The model of the fiber, determined from the reconstructed image, is a helix consisting of 14 filaments that associate in a specific mode to form seven pairs, or protofilaments. Pairs were identified through the pattern of filament loss in partially disassembled fibers and by the separation between molecules, in adjacent filaments, of half a molecular diameter, along the fiber axis. An alternative mode of filament association can be derived also from the surface lattice of the reconstruction, which meets these criteria for the pairing of molecular filaments. Both pairing modes have been used in the search for structures whose transforms show the best agreement with the diffraction data. Models were generated by the systematic translation of six protofilaments, taken in symmetry related pairs, in steps of 3.5 A along the fiber axis relative to a fixed central protofilament. Each translation of a protofilament corresponds to a different fiber model, whose transform was compared with observed data. In all, over 11,000 transforms were calculated. Of all the models considered, three have been found whose residuals are minimal. At 30 A resolution, similar to that of electron micrographs, the model derived from image reconstruction and the three found through our search procedure are indistinguishable. At 15 A, however, the transforms of these models show better agreement with the observed data than the transform of the reconstructed image. Comparison of residuals shows that the model derived from the reconstructed image can be rejected with 99.5% probability relative to the model, with the same pairing scheme, found by our search procedures. The two other models, derived from the alternative pairing scheme, are also more credible than the reconstructed image, but at a lower confidence level. Each of our three models is equally acceptable. Their existence may reflect structural polymorphism of the fiber.