The ImageJ ecosystem: Open‐source software for image visualization,processing, and analysis

For decades, biologists have relied on software to visualize and interpret imaging data. As techniques for acquiring images increase in complexity, resulting in larger multidimensional datasets, imaging software must adapt. ImageJ is an open‐source image analysis software platform that has aided researchers with a variety of image analysis applications, driven mainly by engaged and collaborative user and developer communities. The close collaboration between programmers and users has resulted in adaptations to accommodate new challenges in image analysis that address the needs of ImageJ's diverse user base. ImageJ consists of many components, some relevant primarily for developers and a vast collection of user‐centric plugins. It is available in many forms, including the widely used Fiji distribution. We refer to this entire ImageJ codebase and community as the ImageJ ecosystem. Here we review the core features of this ecosystem and highlight how ImageJ has responded to imaging technology advancements with new plugins and tools in recent years. These plugins and tools have been developed to address user needs in several areas such as visualization, segmentation, and tracking of biological entities in large, complex datasets. Moreover, new capabilities for deep learning are being added to ImageJ, reflecting a shift in the bioimage analysis community towards exploiting artificial intelligence. These new tools have been facilitated by profound architectural changes to the ImageJ core brought about by the ImageJ2 project. Therefore, we also discuss the contributions of ImageJ2 to enhancing multidimensional image processing and interoperability in the ImageJ ecosystem.     

Sperm motility parameters for Steindachneridion parahybae based on open‐source software

The objective of this work was to evaluate the sperm motility of 13 Steindachneridion parahybae males using open‐source software (ImageJ/CASA plugin). The sperm activation procedure and image capture were initiated after semen collection. Four experimental phases were defined from the videos captured of each male as follows: (i) standardization of a dialogue box generated by the CASA plugin within ImageJ; (ii) frame numbers used to perform the analysis; (iii) post‐activation motility between 10 and 20 s with analysis at each 1 s; and (iv) post‐activation motility between 10 and 50 s with analysis at each 10 s. The settings used in the CASA dialogue box were satisfactory, and the results were consistent. These analyses should be performed using 50 frames immediately after sperm activation because spermatozoa quickly lose their vigor. At 10 s post‐activation, 89.1% motile sperm was observed with 107.2 μm s^?1 curvilinear velocity, 83.6 μm s^?1 average path velocity, 77.1 μm s^?1 straight line velocity; 91.6% were of straightness and 77.1% of wobble. The CASA plugin within ImageJ can be applied in sperm analysis of the study species by using the established settings.     

Fragment-based phase extension for three-dimensional structure determination of membrane proteins by electron crystallography

In electron crystallography, membrane protein structure is determined from two-dimensional crystals where the protein is embedded in a membrane. Once large and well-ordered 2D crystals are grown, one of the bottlenecks in electron crystallography is the collection of image data to directly provide experimental phases to high resolution. Here, we describe an approach to bypass this bottleneck, eliminating the need for high-resolution imaging. We use the strengths of electron crystallography in rapidly obtaining accurate experimental phase information from low-resolution images and accurate high-resolution amplitude information from electron diffraction. The low-resolution experimental phases were used for the placement of α helix fragments and extended to high resolution using phases from the fragments. Phases were further improved by density modifications followed by fragment expansion and structure refinement against the high-resolution diffraction data. Using this approach, structures of three membrane proteins were determined rapidly and accurately to atomic resolution without high-resolution image data.     

TomoJ: tomography software for three-dimensional reconstruction in transmission electron microscopy

Background  

Transmission electron tomography is an increasingly common three-dimensional electron microscopy approach that can provide new insights into the structure of subcellular components. Transmission electron tomography fills the gap between high resolution structural methods (X-ray diffraction or nuclear magnetic resonance) and optical microscopy. We developed new software for transmission electron tomography, TomoJ. TomoJ is a plug-in for the now standard image analysis and processing software for optical microscopy, ImageJ.     

Support Immersion Endoscopy in Post-Extraction Alveolar Bone Chambers: A New Window for Microscopic Bone Imaging In Vivo

Using an endoscopic approach, small intraoral bone chambers, which are routinely obtained during tooth extraction and implantation, provide visual in vivo access to internal bone structures. The aim of the present paper is to present a new method to quantify bone microstructure and vascularisation in vivo. Ten extraction sockets and 6 implant sites in 14 patients (6 men / 8 women) were examined by support immersion endoscopy (SIE). After tooth extraction or implant site preparation, microscopic bone analysis (MBA) was performed using short distance SIE video sequences of representative bone areas for off-line analysis with ImageJ. Quantitative assessment of the microstructure and vascularisation of the bone in dental extraction and implant sites in vivo was performed using ImageJ. MBA revealed bone morphology details such as unmineralised and mineralised areas, vascular canals and the presence of bleeding through vascular canals. Morphometric examination revealed that there was more unmineralised bone and less vascular canal area in the implant sites than in the extraction sockets.     

Practical considerations for using K3 cameras in CDS mode for high-resolution and high-throughput single particle cryo-EM

Detector technology plays a pivotal role in high-resolution and high-throughput cryo-EM structure determination. Compared with the first-generation, single-electron counting direct detection camera (Gatan K2), the latest K3 camera is faster, larger, and now offers a correlated-double sampling mode (CDS). Importantly this results in a higher DQE and improved throughput compared to its predecessor. In this study, we focused on optimizing camera data collection parameters for daily use within a cryo-EM facility and explored the balance between throughput and resolution. In total, eight data sets of murine heavy-chain apoferritin were collected at different dose rates and magnifications, using 9-hole image shift data collection strategies. The performance of the camera was characterized by the quality of the resultant 3D reconstructions. Our results demonstrated that the Gatan K3 operating in CDS mode outperformed standard (nonCDS) mode in terms of reconstruction resolution in all tested conditions with 8 electrons per pixel per second being the optimal dose rate. At low magnification (64kx) we were able to achieve reconstruction resolutions of 149% of the physical Nyquist limit (1.8 Å with a 1.346 Å physical pixel size). Low magnification allows more particles to be collected per image, aiding analysis of heterogeneous samples requiring large data sets. At moderate magnification (105kx, 0.834 Å physical pixel size) we achieved a resolution of 1.65 Å within 8-h of data collection, a condition optimal for achieving high-resolution on well behaved samples. Our results also show that for an optimal sample like apoferritin, one can achieve better than 2.5 Å resolution with 5 min of data collection. Together, our studies validate the most efficient ways of imaging protein complexes using the K3 direct detector and will greatly benefit the cryo-EM community.     

Perspectives in three-dimensional analysis of bone samples using synchrotron radiation microtomography.

In this paper we present a methodology based on 3D synchrotron radiation microtomography to analyze non-destructively 3D bone samples. After a technical presentation of the imaging system and the image analysis techniques, we report results on three-dimensional analysis of vertebral samples from women of different ages. The new capabilities of this technique for the investigation of bone are discussed. They include a high spatial resolution down to the micron level, a high density resolution allowing a local quantification of bone mineralization, phase contrast imaging and advances in 3D image analysis.     

A new Fiji-plugin for visualizing intra-annual density fluctuations and analyzing intra-annual theoretical volumetric flow rate fluctuations along wood cross-sections

Intra-annual analysis of wood samples by examination of xylem structure offers the possibility to reveal structural adaptations in the xylem of trees to varying climatic conditions with a high temporal resolution. Additionally, it can help closing the gap between annual tree ring width measurements and ecophysiological field studies.Typically, two approaches for intra-annual wood analyses are used: densitometry (x-ray or high frequency measurements) and analyses of microscopic images of cross-sections. While densitometry has recently become the most commonly used method, it requires expensive measurement devices, which are not necessarily part of standard laboratory equipment and accordingly not affordable for a significant number of researchers.We therefore present a new plugin for visualizing intra-annual density fluctuations (IADF) and, more specifically, for analyzing the theoretical volumetric flow rate with an adapted Hagen-Poiseuille law for elliptic conduits per pixel-column. The implemented algorithm represents an alternative for intra-annual analysis of microscopic cross-section images. The plugin has been developed for Fiji, a common package based on the open-source image processing program ImageJ to avoid the use of commercial programs for anatomical analyses.     

Practical Image Restoration of Thick Biological Specimens Using Multiple Focus Levels in Transmission Electron Microscopy

Three-dimensional electron tomographic studies of thick specimens such as cellular organelles or supramolecular structures require accurate interpretations of transmission electron micrograph intensities. In addition to microscope lens aberrations, thick specimen imaging is complicated by additional distortions resulting from multiple elastic and inelastic scattering. Extensive analysis of the mechanism of image formation using electron energy-loss spectroscopy and imaging as well as exit wavefront reconstruction demonstrated that multiple scattering does not contribute to the coherent component of the exit wave (Hanet al.,1996, 1995). Although exit wavefront restored images showed enhanced contrast and resolution, that technique, which requires the collection of more than 30 images at different focus levels, is not practical for routine data collection in 3D electron tomography, where usually over 100 projection views are required for each reconstruction. Using a 0.7-μm-thick specimen imaged at 200 keV, the accuracy of reconstructions using small numbers of defocused images and a simple linear filter (Schiske, 1968) was assessed by comparison to the complete exit wave restoration. We demonstrate that only four optimal focus levels are required to effectively restore the coherent component (deviation 5.1%). By contrast, the optimal single image (zero defocus) shows a 25.5% deviation to the exit wave restoration. Two pairs of under- and over-defocus images should be taken: one pair at quite high defocus (>10 μm) to differentiate the coherent (single elastic scattering) from the incoherent (multiple elastic and inelastic scattering) components, and the second pair to optimize information content at the highest desired resolution (e.g., 5 μm for (2.5 nm)⁻¹resolution). We also propose a new interpretation of the restored amplitude and phase components where the specimen mass-density is proportional to the logarithm of the amplitude component and linearly related to the phase component. This approach should greatly facilitate the collection of high resolution tomographic data from thick samples.     

Robotics and Dynamic Image Analysis for Studies of Gene Expression in Plant Tissues

Gene expression in plant tissues is typically studied by destructive extraction of compounds from plant tissues for in vitro analyses. The methods presented here utilize the green fluorescent protein (gfp) gene for continual monitoring of gene expression in the same pieces of tissues, over time. The gfp gene was placed under regulatory control of different promoters and introduced into lima bean cotyledonary tissues via particle bombardment. Cotyledons were then placed on a robotic image collection system, which consisted of a fluorescence dissecting microscope with a digital camera and a 2-dimensional robotics platform custom-designed to allow secure attachment of culture dishes. Images were collected from cotyledonary tissues every hour for 100 hours to generate expression profiles for each promoter. Each collected series of 100 images was first subjected to manual image alignment using ImageReady to make certain that GFP-expressing foci were consistently retained within selected fields of analysis. Specific regions of the series measuring 300 x 400 pixels, were then selected for further analysis to provide GFP Intensity measurements using ImageJ software. Batch images were separated into the red, green and blue channels and GFP-expressing areas were identified using the threshold feature of ImageJ. After subtracting the background fluorescence (subtraction of gray values of non-expressing pixels from every pixel) in the respective red and green channels, GFP intensity was calculated by multiplying the mean grayscale value per pixel by the total number of GFP-expressing pixels in each channel, and then adding those values for both the red and green channels. GFP Intensity values were collected for all 100 time points to yield expression profiles. Variations in GFP expression profiles resulted from differences in factors such as promoter strength, presence of a silencing suppressor, or nature of the promoter. In addition to quantification of GFP intensity, the image series were also used to generate time-lapse animations using ImageReady. Time-lapse animations revealed that the clear majority of cells displayed a relatively rapid increase in GFP expression, followed by a slow decline. Some cells occasionally displayed a sudden loss of fluorescence, which may be associated with rapid cell death. Apparent transport of GFP across the membrane and cell wall to adjacent cells was also observed. Time lapse animations provided additional information that could not otherwise be obtained using GFP Intensity profiles or single time point image collections.     

UCSF tomography: an integrated software suite for real-time electron microscopic tomographic data collection, alignment, and reconstruction

A real-time alignment and reconstruction scheme for electron microscopic tomography (EMT) has been developed and integrated within our UCSF tomography data collection software. This newly integrated software suite provides full automation from data collection to real-time reconstruction by which the three-dimensional (3D) reconstructed volume is immediately made available at the end of each data collection. Real-time reconstruction is achieved by calculating a weighted back-projection on a small Linux cluster (five dual-processor compute nodes) concurrently with the UCSF tomography data collection running on the microscope's computer, and using the fiducial-marker free alignment data generated during the data collection process. The real-time reconstructed 3D volume provides users with immediate feedback to fully asses all aspects of the experiment ranging from sample choice, ice thickness, experimental parameters to the quality of specimen preparation. This information can be used to guide subsequent data collections. Access to the reconstruction is especially useful in low-dose cryo EMT where such information is very difficult to obtain due to extraordinary low signal to noise ratio in each 2D image. In our environment, we generally collect 2048 x 2048 pixel images which are subsequently computationally binned four-fold for the on-line reconstruction. Based upon experiments performed with thick and cryo specimens at various CCD magnifications (50000x-80000x), alignment accuracy is sufficient to support this reduced resolution but should be refined before calculating a full resolution reconstruction. The reduced resolution has proven to be quite adequate to assess sample quality, or to screen for the best data set for full-resolution reconstruction, significantly improving both productivity and efficiency of system resources. The total time from start of data collection to a final reconstructed volume (512 x 512 x 256 pixels) is about 50 min for a +/-70 degrees 2k x 2k pixel tilt series acquired at every 1 degrees.     

A novel image-analysis toolbox enabling quantitative analysis of root system architecture

We present in this paper a novel, semiautomated image-analysis software to streamline the quantitative analysis of root growth and architecture of complex root systems. The software combines a vectorial representation of root objects with a powerful tracing algorithm that accommodates a wide range of image sources and quality. The root system is treated as a collection of roots (possibly connected) that are individually represented as parsimonious sets of connected segments. Pixel coordinates and gray level are therefore turned into intuitive biological attributes such as segment diameter and orientation as well as distance to any other segment or topological position. As a consequence, user interaction and data analysis directly operate on biological entities (roots) and are not hampered by the spatially discrete, pixel-based nature of the original image. The software supports a sampling-based analysis of root system images, in which detailed information is collected on a limited number of roots selected by the user according to specific research requirements. The use of the software is illustrated with a time-lapse analysis of cluster root formation in lupin (Lupinus albus) and an architectural analysis of the maize (Zea mays) root system. The software, SmartRoot, is an operating system-independent freeware based on ImageJ and relies on cross-platform standards for communication with data-analysis software.     

Distributed computing in image analysis using open source frameworks and application to image sharpness assessment of histological whole slide images

Background

Automated image analysis on virtual slides is evolving rapidly and will play an important role in the future of digital pathology. Due to the image size, the computational cost of processing whole slide images (WSIs) in full resolution is immense. Moreover, image analysis requires well focused images in high magnification.

Methods

We present a system that merges virtual microscopy techniques, open source image analysis software, and distributed parallel processing. We have integrated the parallel processing framework JPPF, so batch processing can be performed distributed and in parallel. All resulting meta data and image data are collected and merged. As an example the system is applied to the specific task of image sharpness assessment. ImageJ is an open source image editing and processing framework developed at the NIH having a large user community that contributes image processing algorithms wrapped as plug-ins in a wide field of life science applications. We developed an ImageJ plug-in that supports both basic interactive virtual microscope and batch processing functionality. For the application of sharpness inspection we employ an approach with non-overlapping tiles. Compute nodes retrieve image tiles of moderate size from the streaming server and compute the focus measure. Each tile is divided into small sub images to calculate an edge based sharpness criterion which is used for classification. The results are aggregated in a sharpness map.

Results

Based on the system we calculate a sharpness measure and classify virtual slides into one of the following categories - excellent, okay, review and defective. Generating a scaled sharpness map enables the user to evaluate sharpness of WSIs and shows overall quality at a glance thus reducing tedious assessment work.

Conclusions

Using sharpness assessment as an example, the introduced system can be used to process, analyze and parallelize analysis of whole slide images based on open source software.

     

Counting pollen grains using readily available, free image processing and analysis software

Background and Aims

Although many methods exist for quantifying the number of pollen grains in a sample, there are few standard methods that are user-friendly, inexpensive and reliable. The present contribution describes a new method of counting pollen using readily available, free image processing and analysis software.

Methods

Pollen was collected from anthers of two species, Carduus acanthoides and C. nutans (Asteraceae), then illuminated on slides and digitally photographed through a stereomicroscope. Using ImageJ (NIH), these digital images were processed to remove noise and sharpen individual pollen grains, then analysed to obtain a reliable total count of the number of grains present in the image. A macro was developed to analyse multiple images together. To assess the accuracy and consistency of pollen counting by ImageJ analysis, counts were compared with those made by the human eye.

Key Results and Conclusions

Image analysis produced pollen counts in 60 s or less per image, considerably faster than counting with the human eye (5–68 min). In addition, counts produced with the ImageJ procedure were similar to those obtained by eye. Because count parameters are adjustable, this image analysis protocol may be used for many other plant species. Thus, the method provides a quick, inexpensive and reliable solution to counting pollen from digital images, not only reducing the chance of error but also substantially lowering labour requirements.     

High‐resolution photo‐mosaic time‐series imagery for monitoring human use of an artificial reef

Successful marine management relies on understanding patterns of human use. However, obtaining data can be difficult and expensive given the widespread and variable nature of activities conducted. Remote camera systems are increasingly used to overcome cost limitations of conventional labour‐intensive methods. Still, most systems face trade‐offs between the spatial extent and resolution over which data are obtained, limiting their application. We trialed a novel methodology, CSIRO Ruggedized Autonomous Gigapixel System (CRAGS), for time series of high‐resolution photo‐mosaic (HRPM) imagery to estimate fine‐scale metrics of human activity at an artificial reef located 1.3 km from shore. We compared estimates obtained using the novel system to those produced with a web camera that concurrently monitored the site. We evaluated the effect of day type (weekday/weekend) and time of day on each of the systems and compared to estimates obtained from binocular observations. In general, both systems delivered similar estimates for the number of boats observed and to those obtained by binocular counts; these results were also unaffected by the type of day (weekend vs. weekday). CRAGS was able to determine additional information about the user type and party size that was not possible with the lower resolution webcam system. However, there was an effect of time of day as CRAGS suffered from poor image quality in early morning conditions as a result of fixed camera settings. Our field study provides proof of concept of use of this new cost‐effective monitoring tool for the remote collection of high‐resolution large‐extent data on patterns of human use at high temporal frequency.     

Fabrication of titanium based MALDI bacterial chips for rapid, sensitive and direct analysis of pathogenic bacteria

For the first time, we report the fabrication of a titanium bacterial chip for MALDI-MS produced from a simple, cost effective and rapid heat treatment process. This bacterial chip can be reused many times and is highly versatile. These bacterial chips serve dual roles: (1) They can be applied as MALDI-MS target plates for direct and highly sensitive bacterial analysis. (2) They can be used as bacterial sensors for direct analysis of the captured bacteria using MALDI-MS. The sensitivity of these chips when used as bacterial sensors is <10(3)cfu/mL. The lowest detectable concentration for direct MALDI-MS analysis was found to be 10(4)cfu/mL. The results were further justified by using standard plate counting method combined with Tukey-Kramer statistical analysis and fluorescence imaging followed by image processing for fluorescence quantification using ImageJ software to substantiate the MALDI-MS results.     

A new high‐resolution computed tomography (CT) segmentation method for trabecular bone architectural analysis

In the last decade, high‐resolution computed tomography (CT) and microcomputed tomography (micro‐CT) have been increasingly used in anthropological studies and as a complement to traditional histological techniques. This is due in large part to the ability of CT techniques to nondestructively extract three‐dimensional representations of bone structures. Despite prior studies employing CT techniques, no completely reliable method of bone segmentation has been established. Accurate preprocessing of digital data is crucial for measurement accuracy, especially when subtle structures such as trabecular bone are investigated. The research presented here is a new, reproducible, accurate, and fully automated computerized segmentation method for high‐resolution CT datasets of fossil and recent cancellous bone: the Ray Casting Algorithm (RCA). We compare this technique with commonly used methods of image thresholding (i.e., the half‐maximum height protocol and the automatic, adaptive iterative thresholding procedure). While the quality of the input images is crucial for conventional image segmentation, the RCA method is robust regarding the signal to noise ratio, beam hardening, ring artifacts, and blurriness. Tests with data of extant and fossil material demonstrate the superior quality of RCA compared with conventional thresholding procedures, and emphasize the need for careful consideration of optimal CT scanning parameters. Am J Phys Anthropol 2009. © 2009 Wiley‐Liss, Inc.     

Object-oriented image analysis to extract landscape elements in urban fringes, Central Japan

A method enabling the object-oriented image analysis of landscape elements incorporating topographic data was designed and tested on a Japanese countryside target area. IKONOS data (four multispectral bands with a spatial resolution of 4 m and a panchromatic band with a spatial resolution of 1 m) acquired on 23 April 2001 were used. Definiens v.5 software (Definiens AG, München, Germany) was employed for the classification. The initial segmentation was multiresolution and bottom-up, and each segment identified was considered to be one object. Two classifications employing the same landscape elements and ground truth data were implemented. One classification adopted an object-based image analysis classification method based on spectral characteristics; the other utilized an object-oriented image analysis (OOIA) that allows for a suitable scale parameter to be selected independently for each landscape element. In addition, topographic data derived from field surveys (walking surveys) and topographic maps were used to create a topographic database delineating the boundary between valley bottoms and the adjacent slopes (elevation: about 10 m). These data were then integrated into the OOIA analysis. The accuracies of the two classifications were assessed by comparing the results to a master landscape map produced directly from aerial photographs and on-site observations. The object-oriented method using the topographic data resulted in a higher overall kappa coefficients (0.63–0.47) than the object-based method. These results indicate that object-oriented image analysis of very high resolution data used in combination with topographic data can be an effective tool for landscape classification in Japan, where historical land-use patterns have resulted in finely dissected landscapes.     

Automated method for the rapid and precise estimation of adherent cell culture characteristics from phase contrast microscopy images

The quantitative determination of key adherent cell culture characteristics such as confluency, morphology, and cell density is necessary for the evaluation of experimental outcomes and to provide a suitable basis for the establishment of robust cell culture protocols. Automated processing of images acquired using phase contrast microscopy (PCM), an imaging modality widely used for the visual inspection of adherent cell cultures, could enable the non‐invasive determination of these characteristics. We present an image‐processing approach that accurately detects cellular objects in PCM images through a combination of local contrast thresholding and post hoc correction of halo artifacts. The method was thoroughly validated using a variety of cell lines, microscope models and imaging conditions, demonstrating consistently high segmentation performance in all cases and very short processing times (<1 s per 1,208 × 960 pixels image). Based on the high segmentation performance, it was possible to precisely determine culture confluency, cell density, and the morphology of cellular objects, demonstrating the wide applicability of our algorithm for typical microscopy image processing pipelines. Furthermore, PCM image segmentation was used to facilitate the interpretation and analysis of fluorescence microscopy data, enabling the determination of temporal and spatial expression patterns of a fluorescent reporter. We created a software toolbox (PHANTAST) that bundles all the algorithms and provides an easy to use graphical user interface. Source‐code for MATLAB and ImageJ is freely available under a permissive open‐source license. Biotechnol. Bioeng. 2014;111: 504–517. © 2013 Wiley Periodicals, Inc.