Double-flow convolutional neural network for rapid large field of view Fourier ptychographic reconstruction

Fourier ptychographic microscopy is a promising imaging technique which can circumvent the space-bandwidth product of the system and achieve a reconstruction result with wide field-of-view (FOV), high-resolution and quantitative phase information. However, traditional iterative-based methods typically require multiple times to get convergence, and due to the wave vector deviation in different areas, the millimeter-level full-FOV cannot be well reconstructed once and typically required to be separated into several portions with sufficient overlaps and reconstructed separately, which makes traditional methods suffer from long reconstruction time for a large-FOV (of the order of minutes) and limits the application in real-time large-FOV monitoring of live sample in vitro. Here we propose a novel deep-learning based method called DFNN which can be used in place of traditional iterative-based methods to increase the quality of single large-FOV reconstruction and reducing the processing time from 167.5 to 0.1125 second. In addition, we demonstrate that by training based on the simulation dataset with high-entropy property (Opt. Express 28, 24 152 [2020]), DFNN could has fine generalizability and little dependence on the morphological features of samples. The superior robustness of DFNN against noise is also demonstrated in both simulation and experiment. Furthermore, our model shows more robustness against the wave vector deviation. Therefore, we could achieve better results at the edge areas of a single large-FOV reconstruction. Our method demonstrates a promising way to perform real-time single large-FOV reconstructions and provides further possibilities for real-time large-FOV monitoring of live samples with sub-cellular resolution.     

Deep generative modeling for volume reconstruction in cryo-electron microscopy

Advances in cryo-electron microscopy (cryo-EM) for high-resolution imaging of biomolecules in solution have provided new challenges and opportunities for algorithm development for 3D reconstruction. Next-generation volume reconstruction algorithms that combine generative modelling with end-to-end unsupervised deep learning techniques have shown promise, but many technical and theoretical hurdles remain, especially when applied to experimental cryo-EM images. In light of the proliferation of such methods, we propose here a critical review of recent advances in the field of deep generative modelling for cryo-EM reconstruction. The present review aims to (i) provide a unified statistical framework using terminology familiar to machine learning researchers with no specific background in cryo-EM, (ii) review the current methods in this framework, and (iii) outline outstanding bottlenecks and avenues for improvements in the field.     

Registration of pedobarographic image data in the frequency domain

Image registration has been used to support pixel-level data analysis on pedobarographic image data sets. Some registration methods have focused on robustness and sacrificed speed, but a recent approach based on external contours offered both high computational processing speed and high accuracy. However, since contours can be influenced by local perturbations, we sought more global methods. Thus, we propose two new registration methods based on the Fourier transform, cross-correlation and phase correlation which offer high computational speed. We found out that both proposed methods revealed high accuracy for the similarity measures considered, using control geometric transformations. Additionally, both methods revealed high computational processing speed which, combined with their accuracy and robustness, allows their implementation in near-real-time applications. Furthermore, we found that the current methods were robust to moderate levels of noise, and consequently, do not require noise removal procedure like the contours method does.     

Tapered empirical likelihood for time series data in time and frequency domains

We investigate data tapering in two formulations of empiricallikelihood for time series. One empirical likelihood is formedfrom tapered data blocks in the time domain and a second isbased on the tapered periodogram in the frequency domain. Limitingdistributions are provided for both empirical likelihood versionsunder tapering. Theoretical and simulation evidence indicatesthat a data taper improves the coverage accuracy of empiricallikelihood confidence intervals for time series parameters,such as means and correlations.     

Protein diffusion in alginate beads monitored by confocal microscopy. The application of wavelets for data reconstruction and analysis

A microscopic technique has been developed to obtain the protein profiles inside calcium alginate gel. To do this, the diffusion of BSA, previously marked with FITC, inside calcium alginate beads was observed using confocal laser microscopy, thus obtaining the spatio-temporal evolution of the protein concentration. The technique, however, presents certain limitations and zones where it is impossible to obtain experimental data. Wavelets analysis, commonly used in signal processing and statistics, was employed to reconstruct and subsequently analyse the experimental results. Once the diffusion model was defined, the substrate profiles obtained were used to calculate a diffusivity value for BSA in alginate gel. Received 09 February 1999/ Accepted in revised form 14 May 1999     

Determination of molecular conformation and permeation in skin via IR spectroscopy, microscopy, and imaging

Skin tissue, in addition to its specific use in dermal research, provides an excellent model for developing the techniques of vibrational microscopy and imaging for biomedical applications. In addition to permitting characterization of various regions of skin, the relative paucity of major biological constituents in the stratum corneum (the outermost layer of skin), permits us to image, with microscopic resolution, conformational alterations and concentration variations in both the lipid and protein components. Thus we are able to monitor the effects of exogenous materials such as models for drug delivery agents (liposomes) and permeation enhancers (DMSO) on stratum corneum lipid organization and protein structure. In addition, we are able to monitor protein conformational changes in single corneocytes. The current article demonstrates these procedures, ranging from direct univariate measures of lipid chain conformational disorder, to factor analysis which permits us to image conformational differences between liposomes that have permeated through the stratum corneum from those which have remained on the surface in a reservoir outside the skin.     

Optical hyperspectral imaging in microscopy and spectroscopy – a review of data acquisition

Rather than simply acting as a photographic camera capturing two‐dimensional (x, y) intensity images or a spectrometer acquiring spectra (λ), a hyperspectral imager measures entire three‐dimensional (x, y, λ) datacubes for multivariate analysis, providing structural, molecular, and functional information about biological cells or tissue with unprecedented detail. Such data also gives clinical insights for disease diagnosis and treatment. We summarize the principles underpinning this technology, highlight its practical implementation, and discuss its recent applications at microscopic to macroscopic scales.

Datacube acquisition strategies in hyperspectral imaging x, y, spatial coordinates; λ, wavelength.     

Digitizing life at the level of the cell: high-performance laser-scanning microscopy and image analysis for in toto imaging of development

The field of biological imaging is progressing at an amazing rate. Advances in both laser-scanning microscopy and green fluorescent protein (GFP) technology are combining to make possible imaging-based approaches for studying developmental mechanisms that were previously impossible. Modern confocal and multi-photon microscopes are pushing the envelope of speed, sensitivity, spectral resolution, and depth resolution to allow in vivo imaging of whole, live embryos at cellular resolution over extended periods of time. In toto imaging, in which nearly every cell in an embryo or tissue can be tracked through space and time during development, may become a standard technique for small transparent embryos such as zebrafish and early stage chick and mouse embryos. GFP and its spectral variants can be used to mark a wide range of in vivo biological information for in toto imaging including gene expression patterns, mutant phenotypes, and protein subcellular localization patterns. Combining in toto imaging and GFP transgenic approaches on a large scale may usher in an explosion of in vivo, developmental data as has happened in the past several years with genomic data. There are significant challenges that must be met to reach these goals. This paper will discuss the current state-of-the-art, the challenges, and the prospects of in toto imaging in the areas of imaging, image analysis, and informatics.     

Reduction in time required for synthesis of high specific surface area silica from pyrolyzed rice husk by precipitation at low pH

Porous silica with a high specific surface area (SSA) was prepared from pyrolyzed rice husk (PRH) by adding H₃PO₄ to sodium silicate solution (SSS) until the pH values of 5.7, 5.0, 4.1 and 3.2 were achieved. The preparation process involved producing SSS from PRH, forming silica-polyethylene glycol (PEG) composites using SSS, H₃PO₄ and PEG, and calcinating the composites. The required preparation time was below 10 h, and the SSA of the sample prepared at pH 3.2 reached 1018 m²/g. Decreasing pH significantly increased the amount of PEG incorporated into the silica-PEG composites, and hence more pores were generated in the lower pH sample when the PEG was destroyed by calcination at 500 °C. The process developed in this study could lead to more efficient conversion of rice husk into high value-added porous materials that might be used for the adsorption of gas and heavy metal ions.     

Convolutional neural networks for reconstruction of undersampled optical projection tomography data applied to in vivo imaging of zebrafish

Optical projection tomography (OPT) is a 3D mesoscopic imaging modality that can utilize absorption or fluorescence contrast. 3D images can be rapidly reconstructed from tomographic data sets sampled with sufficient numbers of projection angles using the Radon transform, as is typically implemented with optically cleared samples of the mm‐to‐cm scale. For in vivo imaging, considerations of phototoxicity and the need to maintain animals under anesthesia typically preclude the acquisition of OPT data at a sufficient number of angles to avoid artifacts in the reconstructed images. For sparse samples, this can be addressed with iterative algorithms to reconstruct 3D images from undersampled OPT data, but the data processing times present a significant challenge for studies imaging multiple animals. We show here that convolutional neural networks (CNN) can be used in place of iterative algorithms to remove artifacts—reducing processing time for an undersampled in vivo zebrafish dataset from 77 to 15 minutes. We also show that using CNN produces reconstructions of equivalent quality to compressed sensing with 40% fewer projections. We further show that diverse training data classes, for example, ex vivo mouse tissue data, can be used for CNN‐based reconstructions of OPT data of other species including live zebrafish.      

Integration of diffraction phase microscopy and Raman imaging for label‐free morpho‐molecular assessment of live cells

Label‐free quantitative imaging is highly desirable for studying live cells by extracting pathophysiological information without perturbing cell functions. Here, we demonstrate a novel label‐free multimodal optical imaging system with the capability of providing comprehensive morphological and molecular attributes of live cells. Our morpho‐molecular microscopy (3M) system draws on the combined strength of quantitative phase microscopy (QPM) and Raman microscopy to probe the morphological features and molecular fingerprinting characteristics of each cell under observation. While the commonr‐path geometry of our QPM system allows for highly sensitive phase measurement, the Raman microscopy is equipped with dual excitation wavelengths and utilizes the same detection and dispersion system, making it a distinctive multi‐wavelength system with a small footprint. We demonstrate the applicability of the 3M system by investigating nucleated and nonnucleated cells. This integrated label‐free platform has a promising potential in preclinical research, as well as in clinical diagnosis in the near future.      

Climate change and the optimal flowering time of annual plants in seasonal environments

Long‐term phenology monitoring has documented numerous examples of changing flowering dates during the last century. A pivotal question is whether these phenological responses are adaptive or not under directionally changing climatic conditions. We use a classic dynamic growth model for annual plants, based on optimal control theory, to find the fitness‐maximizing flowering time, defined as the switching time from vegetative to reproductive growth. In a typical scenario of global warming, with advanced growing season and increased productivity, optimal flowering time advances less than the start of the growing season. Interestingly, increased temporal spread in production over the season may either advance or delay the optimal flowering time depending on overall productivity or season length. We identify situations where large phenological changes are necessary for flowering time to remain optimal. Such changes also indicate changed selection pressures. In other situations, the model predicts advanced phenology on a calendar scale, but no selection for early flowering in relation to the start of the season. We also show that the optimum is more sensitive to increased productivity when productivity is low than when productivity is high. All our results are derived using a general, graphical method to calculate the optimal flowering time applicable for a large range of shapes of the seasonal production curve. The model can thus explain apparent maladaptation in phenological responses in a multitude of scenarios of climate change. We conclude that taking energy allocation trade‐offs and appropriate time scales into account is critical when interpreting phenological patterns.     

Two-photon synthetic aperture microscopy for minimally invasive fast 3D imaging of native subcellular behaviors in deep tissue

1. Download : Download high-res image (261KB)
2. Download : Download full-size image

     

A class of latent Markov models for capture-recapture data allowing for time, heterogeneity, and behavior effects

We propose an extension of the latent class model for the analysis of capture-recapture data which allows us to take into account the effect of a capture on the behavior of a subject with respect to future captures. The approach is based on the assumption that the variable indexing the latent class of a subject follows a Markov chain with transition probabilities depending on the previous capture history. Several constraints are allowed on these transition probabilities and on the parameters of the conditional distribution of the capture configuration given the latent process. We also allow for the presence of discrete explanatory variables, which may affect the parameters of the latent process. To estimate the resulting models, we rely on the conditional maximum likelihood approach and for this aim we outline an EM algorithm. We also give some simple rules for point and interval estimation of the population size. The approach is illustrated by applying it to two data sets concerning small mammal populations.     

Combined Fourier transform infrared and Raman spectroscopic approach for identification of multidrug resistance phenotype in cancer cell lines

Cancer cells escape cytotoxic effects of anticancer drugs by a process known as multidrug resistance (MDR). Identification of cell status by less time-consuming methods can be extremely useful in patient management and treatment. This study aims at evaluating the potentials of vibrational spectroscopic methods to perform cell typing and to differentiate between sensitive and resistant human cancer cell lines, in particular those that exhibit the MDR phenotype. Micro-Raman and Fourier transform infrared (FTIR) spectra have been acquired from the sensitive promyelocytic HL60 leukemia cell line and two of its subclones resistant to doxorubicin (HL60/DOX) and daunorubicin (HL60/DNR), and from the sensitive MCF7 breast cancer cell line and its MDR counterpart resistant to verapamil (MCF7/VP). Principal components analysis (PCA) was employed for spectral comparison and classification. Our data show that cell typing was feasible with both methods, giving two distinct clusters for HL60- and MCF7-sensitive cells. In addition, phenotyping of HL60 cells, i.e., discriminating between the sensitive and MDR phenotypes, was attempted by both methods. FTIR could not only delineate between the sensitive and resistant HL60 cells, but also gave two distinct clusters for the resistant cells, which required a two-step procedure with Raman spectra. In the case of MCF7 cell lines, both the sensitive and resistant phenotypes could be differentiated very efficiently by PCA analysis of their FTIR and Raman point spectra. These results indicate the prospective applicability of FTIR and micro-Raman approaches in the differentiation of cell types as well as characterization of the cell status, such as the MDR phenotype exhibited in resistant leukemia cell lines like HL60 and MCF7.     

Indirect and Direct Energy Requirements of City Households in Sweden: Options for Reduction, Lessons from Modeling

The objective of this article is to explore the potential for lowering household energy use given existing local support systems, in this case in the Stockholm inner city with the aid of the Dutch energy analysis program (EAP) that was adapted to Swedish conditions and that portrays total energy use for 300 consumption categories. Previously such modeling for Sweden was carried out using only Dutch databases. Our case-study area is well equipped with food stores, local markets, public transportation, and entertainment, facilitating some energy-efficient consumption choices. With maintained expenditure levels but changed consumption patterns, current reduction potentials are on the order of 10–20%. Options concerning diet can lower food indirect energy use by up to 30%, whereas options in other areas have a lower potential. Further reductions will require enhanced local support systems, external as well as internal. The results indicate that it is risky not to use nationally adapted figures for energy efficiency in the production sectors when modeling household energy use, because potential for change may be overlooked. Future work should include foreign energy intensities when modeling imported goods; otherwise, results may be less reliable. The Swedish EAP needs further work before it can be put to use as a modeling tool for everyday behavior, but it is already generating important possibilities for producing reliable data that can be used by local energy counselors.     

Determination and correction of aberrations in full field optical coherence tomography using phase gradient autofocus by maximizing the likelihood function

A method for numerical estimation and correction of aberrations of the eye in fundus imaging with optical coherence tomography (OCT) is presented. Aberrations are determined statistically by using the estimate based on likelihood function maximization. The method can be considered as an extension of the phase gradient autofocusing algorithm in synthetic aperture radar imaging to 2D optical aberration correction. The efficacy of the proposed method has been demonstrated in OCT fundus imaging with 6λ aberrations. After correction, single photoreceptors were resolved. It is also shown that wave front distortions with high spatial frequencies can be determined and corrected.     

Soft sensor based on 2D‐fluorescence and process data enabling real‐time estimation of biomass in Escherichia coli cultivations

In bioprocesses, specific process responses such as the biomass cannot typically be measured directly on‐line, since analytical sampling is associated with unavoidable time delays. Accessing those responses in real‐time is essential for Quality by Design and process analytical technology concepts. Soft sensors overcome these limitations by indirectly measuring the variables of interest using a previously derived model and actual process data in real time. In this study, a biomass soft sensor based on 2D‐fluorescence data and process data, was developed for a comprehensive study with a 20‐L experimental design, for Escherichia coli fed‐batch cultivations. A multivariate adaptive regression splines algorithm was applied to 2D‐fluorescence spectra and process data, to estimate the biomass concentration at any time during the process. Prediction errors of 4.9% (0.99 g/L) for validation and 3.8% (0.69 g/L) for new data (external validation), were obtained. Using principal component and parallel factor analyses on the 2D‐fluorescence data, two potential chemical compounds were identified and directly linked to cell metabolism. The same wavelength pairs were also important predictors for the regression‐model performance. Overall, the proposed soft sensor is a valuable tool for monitoring the process performance on‐line, enabling Quality by Design.     

Evidence from surface microscopy for recognition of fleshy and tendinous muscle insertion in extant vertebrate femora: implications for muscle reconstruction in fossils

Recognition of muscle attachment sites and their modification has been an important tool in anthropologic and paleontologic research, but has been compromised by limited ability to recognise sites of tendinous attachments. We investigated bone–tendon (three sites) and bone–muscle (six sites) interfaces in six pairs of femora across a broad taxonomic spectrum of higher amniote archosaurs (both recent and fossil) by epi-illumination microscopy. Direct fleshy and indirect tendinous muscle attachments were identified by dissection of fresh specimens and examination of fossils and the surface microscopic changes identified at those locations. Examination revealed bone modifications specific to each type of muscle insertion, allowing them to be identified and distinguished. Application of a surface microscopy technique not only permits more confident localisation of tendinous attachments, but for the first time allows recognition of sites of direct fleshy muscle attachments – in a reproducible manner across phylogenetic lines.