A text and image analysis workflow using citizen science data to extract relevant social media records: Combining red kite observations from Flickr,eBird and iNaturalist

There is an urgent need to develop new methods to monitor the state of the environment. One potential approach is to use new data sources, such as User-Generated Content, to augment existing approaches. However, to date, studies typically focus on a single date source and modality. We take a new approach, using citizen science records recording sightings of red kites (Milvus milvus) to train and validate a Convolutional Neural Network (CNN) capable of identifying images containing red kites. This CNN is integrated in a sequential workflow which also uses an off-the-shelf bird classifier and text metadata to retrieve observations of red kites in the Chilterns, England. Our workflow reduces an initial set of more than 600,000 images to just 3065 candidate images. Manual inspection of these images shows that our approach has a precision of 0.658. A workflow using only text identifies 14% less images than that including image content analysis, and by combining image and text classifiers we achieve almost perfect precision of 0.992. Images retrieved from social media records complement those recorded by citizen scientists spatially and temporally, and our workflow is sufficiently generic that it can easily be transferred to other species.     

A computational image analysis glossary for biologists

Recent advances in biological imaging have resulted in an explosion in the quality and quantity of images obtained in a digital format. Developmental biologists are increasingly acquiring beautiful and complex images, thus creating vast image datasets. In the past, patterns in image data have been detected by the human eye. Larger datasets, however, necessitate high-throughput objective analysis tools to computationally extract quantitative information from the images. These tools have been developed in collaborations between biologists, computer scientists, mathematicians and physicists. In this Primer we present a glossary of image analysis terms to aid biologists and briefly discuss the importance of robust image analysis in developmental studies.     

Face recognition from a single image per person using deep architecture neural networks

Implementing an accurate face recognition system requires images in different variations, and if our database is large, we suffer from problems such as storing cost and low speed in recognition algorithms. On the other hand, in some applications there is only one image available per person for training recognition model. In this article, we propose a neural network model inspired of bidirectional analysis and synthesis brain network which can learn nonlinear mapping between image space and components space. Using a deep neural network model, we have tried to separate pose components from person ones. After setting apart these components, we can use them to synthesis virtual images of test data in different pose and lighting conditions. These virtual images are used to train neural network classifier. The results showed that training neural classifier with virtual images gives better performance than training classifier with frontal view images.     

基于图像融合与混合像元分解的城市植被盖度提取

城市植被盖度提取对于开展城市绿色空间保护和城市规划具有重要意义。随着遥感技术的发展,混合像元分解模型被广泛用于从中等分辨率的多光谱影像提取城市植被盖度,但较低的影像空间分辨率限制了该模型的应用领域。为此,以杭州市为例,首先引入Gram-Schmidt(GS)方法对Landsat ETM+的多光谱波段和全色波段进行融合,再通过混合像元分解模型从ETM+融合影像上提取城市植被盖度,最后利用SPOT影像进行精度检验。结果发现,采用GS方法对影像进行融合后,标准差、信息熵、平均梯度提高,相对偏差小于0.07,说明在保留多光谱信息的基础上提高了其空间分辨率。与SPOT影像相比,在融合影像上75%以上样本的植被盖度值相似,误差较大的区域是市区植被特别稀疏或茂盛的像元。与源影像相比,从融合影像上提取的植被盖度的均方根误差和系统误差降低了0.01。该方法在降低城市植被监测成本、提高监测精度方面具有潜力。     

Comparative study of the enzymatic,hemorrhagic, procoagulant and anticoagulant activities of some animal venoms

1. The enzymatic, hemorrhagic, procoagulant and anticoagulant activities of venoms of some animals including snakes, lizards, toads, scorpions, spider, wasps, bees and ants were compared.2. Snake venom was the richest source of enzymes among the animal venoms. Most other animal venoms were devoid of phosphodiesterase, l-amino acid oxidase, alkaline phosphomonoesterase and acetylcholinesterase activities and only a few exhibited arginine ester hydrolase activity. These venoms, however, exhibited wide ranges of protease, 5'-nucleotidase and hyaluronidase activities. Most of the animal venoms examined exhibited some phospholipase A activity.3. Other than snake venoms, only venoms of the toad Bufo calamita and the lizards were hemorrhagic, and only venoms of the social wasps, social bees and harvester ant exhibited strong anticoagulant activity. Procoagulant activity occurs only in snake venoms.     

A microtubule RELION-based pipeline for cryo-EM image processing

Microtubules are polar filaments built from αβ-tubulin heterodimers that exhibit a range of architectures in vitro and in vivo. Tubulin heterodimers are arranged helically in the microtubule wall but many physiologically relevant architectures exhibit a break in helical symmetry known as the seam. Noisy 2D cryo-electron microscopy projection images of pseudo-helical microtubules therefore depict distinct but highly similar views owing to the high structural similarity of α- and β-tubulin. The determination of the αβ-tubulin register and seam location during image processing is essential for alignment accuracy that enables determination of biologically relevant structures. Here we present a pipeline designed for image processing and high-resolution reconstruction of cryo-electron microscopy microtubule datasets, based in the popular and user-friendly RELION image-processing package, Microtubule RELION-based Pipeline (MiRP). The pipeline uses a combination of supervised classification and prior knowledge about geometric lattice constraints in microtubules to accurately determine microtubule architecture and seam location. The presented method is fast and semi-automated, producing near-atomic resolution reconstructions with test datasets that contain a range of microtubule architectures and binding proteins.     

coupleCoC+: An information-theoretic co-clustering-based transfer learning framework for the integrative analysis of single-cell genomic data

Technological advances have enabled us to profile multiple molecular layers at unprecedented single-cell resolution and the available datasets from multiple samples or domains are growing. These datasets, including scRNA-seq data, scATAC-seq data and sc-methylation data, usually have different powers in identifying the unknown cell types through clustering. So, methods that integrate multiple datasets can potentially lead to a better clustering performance. Here we propose coupleCoC+ for the integrative analysis of single-cell genomic data. coupleCoC+ is a transfer learning method based on the information-theoretic co-clustering framework. In coupleCoC+, we utilize the information in one dataset, the source data, to facilitate the analysis of another dataset, the target data. coupleCoC+ uses the linked features in the two datasets for effective knowledge transfer, and it also uses the information of the features in the target data that are unlinked with the source data. In addition, coupleCoC+ matches similar cell types across the source data and the target data. By applying coupleCoC+ to the integrative clustering of mouse cortex scATAC-seq data and scRNA-seq data, mouse and human scRNA-seq data, mouse cortex sc-methylation and scRNA-seq data, and human blood dendritic cells scRNA-seq data from two batches, we demonstrate that coupleCoC+ improves the overall clustering performance and matches the cell subpopulations across multimodal single-cell genomic datasets. coupleCoC+ has fast convergence and it is computationally efficient. The software is available at https://github.com/cuhklinlab/coupleCoC_plus.     

CT iterative reconstruction in image space: A phantom study

Although iterative reconstruction is widely applied in SPECT/PET, its introduction in clinical CT is quite recent, in the past the demand for extensive computer power and long image reconstruction times have stopped the diffusion of this technique. Recently Iterative Reconstruction in Image Space (IRIS) has been introduced on Siemens top CT scanners. This recon method works on image data area, reducing the time-consuming loops on raw data and noise removal is obtained in subsequent iterative steps with a smoothing process. We evaluated image noise, low contrast resolution, CT number linearity and accuracy, transverse and z-axis spatial resolution using some dedicated phantoms in single, dual source and cardiac mode. We reconstructed images with a traditional filtered back-projection algorithm and with IRIS. The iterative procedure preserves spatial resolution, CT number accuracy and linearity moreover decreases image noise. These preliminary results support the idea that dose reduction with preserved image quality is possible with IRIS, even if studies on patients are necessary to confirm these data.     

Markov random field based automatic image alignment for electron tomography

We present a method for automatic full-precision alignment of the images in a tomographic tilt series. Full-precision automatic alignment of cryo electron microscopy images has remained a difficult challenge to date, due to the limited electron dose and low image contrast. These facts lead to poor signal to noise ratio (SNR) in the images, which causes automatic feature trackers to generate errors, even with high contrast gold particles as fiducial features. To enable fully automatic alignment for full-precision reconstructions, we frame the problem probabilistically as finding the most likely particle tracks given a set of noisy images, using contextual information to make the solution more robust to the noise in each image. To solve this maximum likelihood problem, we use Markov Random Fields (MRF) to establish the correspondence of features in alignment and robust optimization for projection model estimation. The resulting algorithm, called Robust Alignment and Projection Estimation for Tomographic Reconstruction, or RAPTOR, has not needed any manual intervention for the difficult datasets we have tried, and has provided sub-pixel alignment that is as good as the manual approach by an expert user. We are able to automatically map complete and partial marker trajectories and thus obtain highly accurate image alignment. Our method has been applied to challenging cryo electron tomographic datasets with low SNR from intact bacterial cells, as well as several plastic section and X-ray datasets.     

Measurement of strain in the left ventricle during diastole with cine-MRI and deformable image registration

The assessment of regional heart wall motion (local strain) can localize ischemic myocardial disease, evaluate myocardial viability, and identify impaired cardiac function due to hypertrophic or dilated cardiomyopathies. The objectives of this research were to develop and validate a technique known as hyperelastic warping for the measurement of local strains in the left ventricle from clinical cine-magnetic resonance imaging (MRI) image datasets. The technique uses differences in image intensities between template (reference) and target (loaded) image datasets to generate a body force that deforms a finite element (FE) representation of the template so that it registers with the target image. To validate the technique, MRI image datasets representing two deformation states of a left ventricle were created such that the deformation map between the states represented in the images was known. A beginning diastolic cine-MRI image dataset from a normal human subject was defined as the template. A second image dataset (target) was created by mapping the template image using the deformation results obtained from a forward FE model of diastolic filling. Fiber stretch and strain predictions from hyperelastic warping showed good agreement with those of the forward solution (R2=0.67 stretch, R2=0.76 circumferential strain, R2=0.75 radial strain, and R2=0.70 in-plane shear). The technique had low sensitivity to changes in material parameters (deltaR2= -0.023 fiber stretch, deltaR2=-0.020 circumferential strain, deltaR2=-0.005 radial strain, and deltaR2=0.0125 shear strain with little or no change in rms error), with the exception of changes in bulk modulus of the material. The use of an isotropic hyperelastic constitutive model in the warping analyses degraded the predictions of fiber stretch. Results were unaffected by simulated noise down to a signal-to-noise ratio (SNR) of 4.0 (deltaR2= -0.032 fiber stretch, deltaR2=-0.023 circumferential strain, deltaR2=-0.04 radial strain, and deltaAR2=0.0211 shear strain with little or no increase in rms error). This study demonstrates that warping in conjunction with cine-MRI imaging can be used to determine local ventricular strains during diastole.     

Automated identification of circulating tumor cells by image cytometry

Presence of circulating tumor cells (CTC), as detected by the CellSearch System, in patients with metastatic carcinomas is associated with poor survival prospects. CellTracks TDI, a dedicated image cytometer, was developed to improve the enumeration of these rare CTC. The CellSearch System was used to enumerate CTC in 7.5 mL blood of 68 patients with cancer and 9 healthy controls. Cartridges containing the fluorescently labeled CTC from this system were reanalyzed using the image cytometer, which acquires images with a TDI camera using a 40×/0.6 NA objective and lasers as light source. Automated classification of events was performed by the Random Forest method using Matlab. An automated classifier was developed to classify events into CTC, apoptotic CTC, CTC debris, leukocytes, and debris not related to CTC. A high agreement in classification was obtained between the automated classifier and five expert reviewers. Comparison of images from the same events in CellTracks TDI and CellTracks Analyzer II shows improved resolution in fluorescence images and improved classification by adding bright-field images. Improved detection efficiency for CD45-APC avoids the classification of leukocytes nonspecifically binding to cytokeratin as CTC. The correlation between number of CTC detected in CellTracks TDI and CellTracks Analyzer II is good with a slope of 1.88 and a correlation coefficient of 0.87. Automated classification of events by CellTracks TDI eliminates the operator error in classification of events as CTC and permits quantitative assessment of parameters. The clinical relevance of various CTC definitions can now be investigated.     

A phylogenetic investigation of the Neotropical genus Alphamenes van der Vecht, 1977 (Hymenoptera,Vespidae, Eumeninae)

Advances in potter wasp systematics have been achieved recently, with classificatory changes resulting from analyses based upon large scale molecular datasets. For the Neotropics, recent hypotheses point to the occurrence of an exclusive clade recognized within the tribe Eumenini. In this group, several contributions regarding taxonomy and systematics have been proposed in the last five years, including the genus Alphamenes. This taxon contains seven described species whose distribution is exclusively Neotropical. Females are morphologically homogeneous, and characters related to copulatory organs are useful in male diagnosis. This contribution forms the first phylogenetic approach to include all species of Alphamenes, hence the first to strongly test for group monophyly. Our cladistic results recovered Alphamenes as a monophyletic group supported by male genital features. Relationships among included species also rely upon genitalic characters, highlighting the importance of these attributes for eumenine systematics. Recent phylogenetic investigations applied to the Neotropical fauna of potter wasps represent desirable advancements towards a natural classification for the group.     

The ImageJ ecosystem: Open‐source software for image visualization,processing, and analysis

For decades, biologists have relied on software to visualize and interpret imaging data. As techniques for acquiring images increase in complexity, resulting in larger multidimensional datasets, imaging software must adapt. ImageJ is an open‐source image analysis software platform that has aided researchers with a variety of image analysis applications, driven mainly by engaged and collaborative user and developer communities. The close collaboration between programmers and users has resulted in adaptations to accommodate new challenges in image analysis that address the needs of ImageJ's diverse user base. ImageJ consists of many components, some relevant primarily for developers and a vast collection of user‐centric plugins. It is available in many forms, including the widely used Fiji distribution. We refer to this entire ImageJ codebase and community as the ImageJ ecosystem. Here we review the core features of this ecosystem and highlight how ImageJ has responded to imaging technology advancements with new plugins and tools in recent years. These plugins and tools have been developed to address user needs in several areas such as visualization, segmentation, and tracking of biological entities in large, complex datasets. Moreover, new capabilities for deep learning are being added to ImageJ, reflecting a shift in the bioimage analysis community towards exploiting artificial intelligence. These new tools have been facilitated by profound architectural changes to the ImageJ core brought about by the ImageJ2 project. Therefore, we also discuss the contributions of ImageJ2 to enhancing multidimensional image processing and interoperability in the ImageJ ecosystem.     

Digital image analysis of Zostera marina leaf injury

Current methods for assessing leaf injury in Zostera marina (eelgrass) utilize subjective indexes for desiccation injury and wasting disease. Because of the subjective nature of these measures, they are inherently imprecise making them difficult to use in quantifying complex leaf injuries from multiple sources. We have developed a method using color digital photography of eelgrass leaves which are then manipulated using image processing programs and analyzed using geographic digital image analysis. The resulting false color images are then assigned by the user into uninjured and injured groupings which may then be reported as a percentage of leaf area affected. If images are rectified, leaf area (cm²) of injured and uninjured leaf segments may be determined. Although this method is time consuming and still requires some subjective judgments, it does allow for precise analysis of highly complex leaf injuries and has the potential to be a substantial improvement over existing leaf injury indexes.     

Cryo-EM image alignment: From pair-wise to joint with deep unsupervised difference learning

Cryo-electron microscopy (cryo-EM) single-particle analysis is a revolutionary imaging technique to resolve and visualize biomacromolecules. Image alignment in cryo-EM is an important and basic step to improve the precision of the image distance calculation. However, it is a very challenging task due to high noise and low signal-to-noise ratio. Therefore, we propose a new deep unsupervised difference learning (UDL) strategy with novel pseudo-label guided learning network architecture and apply it to pair-wise image alignment in cryo-EM. The training framework is fully unsupervised. Furthermore, a variant of UDL called joint UDL (JUDL), is also proposed, which is capable of utilizing the similarity information of the whole dataset and thus further increase the alignment precision. Assessments on both real-world and synthetic cryo-EM single-particle image datasets suggest the new unsupervised joint alignment method can achieve more accurate alignment results. Our method is highly efficient by taking advantages of GPU devices. The source code of our methods is publicly available at “http://www.csbio.sjtu.edu.cn/bioinf/JointUDL/” for academic use.     

The Interacting Effects of Ungulate Hoofprints and Predatory Native Ants on Metamorph Cane Toads in Tropical Australia

Many invasive species exploit the disturbed habitats created by human activities. Understanding the effects of habitat disturbance on invasion success, and how disturbance interacts with other factors (such as biotic resistance to the invaders from the native fauna) may suggest new ways to reduce invader viability. In tropical Australia, commercial livestock production can facilitate invasion by the cane toad (Rhinella marina), because hoofprints left by cattle and horses around waterbody margins provide distinctive (cool, moist) microhabitats; nevertheless the same microhabitat can inhibit the success of cane toads by increasing the risks of predation or drowning. Metamorph cane toads actively select hoofprints as retreat-sites to escape dangerous thermal and hydric conditions in the surrounding landscape. However, hoofprint geometry is important: in hoofprints with steep sides the young toads are more likely to be attacked by predatory ants (Iridomyrmex reburrus) and are more likely to drown following heavy rain. Thus, anthropogenic changes to the landscape interact with predation by native taxa to affect the ability of cane toads in this vulnerable life-history stage to thrive in the harsh abiotic conditions of tropical Australia.     

Distributed computing in image analysis using open source frameworks and application to image sharpness assessment of histological whole slide images

Background

Automated image analysis on virtual slides is evolving rapidly and will play an important role in the future of digital pathology. Due to the image size, the computational cost of processing whole slide images (WSIs) in full resolution is immense. Moreover, image analysis requires well focused images in high magnification.

Methods

We present a system that merges virtual microscopy techniques, open source image analysis software, and distributed parallel processing. We have integrated the parallel processing framework JPPF, so batch processing can be performed distributed and in parallel. All resulting meta data and image data are collected and merged. As an example the system is applied to the specific task of image sharpness assessment. ImageJ is an open source image editing and processing framework developed at the NIH having a large user community that contributes image processing algorithms wrapped as plug-ins in a wide field of life science applications. We developed an ImageJ plug-in that supports both basic interactive virtual microscope and batch processing functionality. For the application of sharpness inspection we employ an approach with non-overlapping tiles. Compute nodes retrieve image tiles of moderate size from the streaming server and compute the focus measure. Each tile is divided into small sub images to calculate an edge based sharpness criterion which is used for classification. The results are aggregated in a sharpness map.

Results

Based on the system we calculate a sharpness measure and classify virtual slides into one of the following categories - excellent, okay, review and defective. Generating a scaled sharpness map enables the user to evaluate sharpness of WSIs and shows overall quality at a glance thus reducing tedious assessment work.

Conclusions

Using sharpness assessment as an example, the introduced system can be used to process, analyze and parallelize analysis of whole slide images based on open source software.

     

Pattern recognition software and techniques for biological image analysis

The increasing prevalence of automated image acquisition systems is enabling new types of microscopy experiments that generate large image datasets. However, there is a perceived lack of robust image analysis systems required to process these diverse datasets. Most automated image analysis systems are tailored for specific types of microscopy, contrast methods, probes, and even cell types. This imposes significant constraints on experimental design, limiting their application to the narrow set of imaging methods for which they were designed. One of the approaches to address these limitations is pattern recognition, which was originally developed for remote sensing, and is increasingly being applied to the biology domain. This approach relies on training a computer to recognize patterns in images rather than developing algorithms or tuning parameters for specific image processing tasks. The generality of this approach promises to enable data mining in extensive image repositories, and provide objective and quantitative imaging assays for routine use. Here, we provide a brief overview of the technologies behind pattern recognition and its use in computer vision for biological and biomedical imaging. We list available software tools that can be used by biologists and suggest practical experimental considerations to make the best use of pattern recognition techniques for imaging assays.