Rotation-Invariant Features for Multi-Oriented Text Detection in Natural Images

Texts in natural scenes carry rich semantic information, which can be used to assist a wide range of applications, such as object recognition, image/video retrieval, mapping/navigation, and human computer interaction. However, most existing systems are designed to detect and recognize horizontal (or near-horizontal) texts. Due to the increasing popularity of mobile-computing devices and applications, detecting texts of varying orientations from natural images under less controlled conditions has become an important but challenging task. In this paper, we propose a new algorithm to detect texts of varying orientations. Our algorithm is based on a two-level classification scheme and two sets of features specially designed for capturing the intrinsic characteristics of texts. To better evaluate the proposed method and compare it with the competing algorithms, we generate a comprehensive dataset with various types of texts in diverse real-world scenes. We also propose a new evaluation protocol, which is more suitable for benchmarking algorithms for detecting texts in varying orientations. Experiments on benchmark datasets demonstrate that our system compares favorably with the state-of-the-art algorithms when handling horizontal texts and achieves significantly enhanced performance on variant texts in complex natural scenes.     

Multiscale Persistent Functions for Biomolecular Structure Characterization

In this paper, we introduce multiscale persistent functions for biomolecular structure characterization. The essential idea is to combine our multiscale rigidity functions (MRFs) with persistent homology analysis, so as to construct a series of multiscale persistent functions, particularly multiscale persistent entropies, for structure characterization. To clarify the fundamental idea of our method, the multiscale persistent entropy (MPE) model is discussed in great detail. Mathematically, unlike the previous persistent entropy (Chintakunta et al. in Pattern Recognit 48(2):391–401, 2015; Merelli et al. in Entropy 17(10):6872–6892, 2015; Rucco et al. in: Proceedings of ECCS 2014, Springer, pp 117–128, 2016), a special resolution parameter is incorporated into our model. Various scales can be achieved by tuning its value. Physically, our MPE can be used in conformational entropy evaluation. More specifically, it is found that our method incorporates in it a natural classification scheme. This is achieved through a density filtration of an MRF built from angular distributions. To further validate our model, a systematical comparison with the traditional entropy evaluation model is done. It is found that our model is able to preserve the intrinsic topological features of biomolecular data much better than traditional approaches, particularly for resolutions in the intermediate range. Moreover, by comparing with traditional entropies from various grid sizes, bond angle-based methods and a persistent homology-based support vector machine method (Cang et al. in Mol Based Math Biol 3:140–162, 2015), we find that our MPE method gives the best results in terms of average true positive rate in a classic protein structure classification test. More interestingly, all-alpha and all-beta protein classes can be clearly separated from each other with zero error only in our model. Finally, a special protein structure index (PSI) is proposed, for the first time, to describe the “regularity” of protein structures. Basically, a protein structure is deemed as regular if it has a consistent and orderly configuration. Our PSI model is tested on a database of 110 proteins; we find that structures with larger portions of loops and intrinsically disorder regions are always associated with larger PSI, meaning an irregular configuration, while proteins with larger portions of secondary structures, i.e., alpha-helix or beta-sheet, have smaller PSI. Essentially, PSI can be used to describe the “regularity” information in any systems.     

Automated Detection of Soma Location and Morphology in Neuronal Network Cultures

Automated identification of the primary components of a neuron and extraction of its sub-cellular features are essential steps in many quantitative studies of neuronal networks. The focus of this paper is the development of an algorithm for the automated detection of the location and morphology of somas in confocal images of neuronal network cultures. This problem is motivated by applications in high-content screenings (HCS), where the extraction of multiple morphological features of neurons on large data sets is required. Existing algorithms are not very efficient when applied to the analysis of confocal image stacks of neuronal cultures. In addition to the usual difficulties associated with the processing of fluorescent images, these types of stacks contain a small number of images so that only a small number of pixels are available along the z-direction and it is challenging to apply conventional 3D filters. The algorithm we present in this paper applies a number of innovative ideas from the theory of directional multiscale representations and involves the following steps: (i) image segmentation based on support vector machines with specially designed multiscale filters; (ii) soma extraction and separation of contiguous somas, using a combination of level set method and directional multiscale filters. We also present an approach to extract the soma’s surface morphology using the 3D shearlet transform. Extensive numerical experiments show that our algorithms are computationally efficient and highly accurate in segmenting the somas and separating contiguous ones. The algorithms presented in this paper will facilitate the development of a high-throughput quantitative platform for the study of neuronal networks for HCS applications.     

Multiscale characterization and micromechanical modeling of crop stem materials

An essential prerequisite for the efficient biomechanical tailoring of crops is to accurately relate mechanical behavior to compositional and morphological properties across different length scales. In this article, we develop a multiscale approach to predict macroscale stiffness and strength properties of crop stem materials from their hierarchical microstructure. We first discuss the experimental multiscale characterization based on microimaging (micro-CT, light microscopy, transmission electron microscopy) and chemical analysis, with a particular focus on oat stems. We then derive in detail a general micromechanics-based model of macroscale stiffness and strength. We specify our model for oats and validate it against a series of bending experiments that we conducted with oat stem samples. In the context of biomechanical tailoring, we demonstrate that our model can predict the effects of genetic modifications of microscale composition and morphology on macroscale mechanical properties of thale cress that is available in the literature.

     

Empirical Study of an Adaptive Multiscale Model for Simulating Cardiac Conduction

We modify and empirically study an adaptive multiscale model for simulating cardiac action potential propagation along a strand of cardiomyocytes. The model involves microscale partial differential equations posed over cells near the action potential upstroke and macroscale partial differential equations posed over the remainder of the tissue. An important advantage of the modified model of this paper is that, unlike our original model, it does not require perfect alignment between myocytes and the macroscale computational grid. We study the effects of gap-junctional coupling, ephaptic coupling, and macroscale grid spacing on the accuracy of the multiscale model. Our simulations reveal that the multiscale method accurately reproduces both the wavespeed and the waveform, including both upstroke and recovery, of fully microscale models. They also reveal that perfect alignment between myocytes and the macroscale grid is not necessary to reproduce the dynamics of a traveling action potential. Further, our simulations suggest that the macroscale grid spacing used in an adaptive multiscale model need not be much finer than the spatial width of an action potential. These results are demonstrated to hold under high, low, and zero gap-junctional coupling regimes.     

A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields

Sparse coding algorithms trained on natural images can accurately predict the features that excite visual cortical neurons, but it is not known whether such codes can be learned using biologically realistic plasticity rules. We have developed a biophysically motivated spiking network, relying solely on synaptically local information, that can predict the full diversity of V1 simple cell receptive field shapes when trained on natural images. This represents the first demonstration that sparse coding principles, operating within the constraints imposed by cortical architecture, can successfully reproduce these receptive fields. We further prove, mathematically, that sparseness and decorrelation are the key ingredients that allow for synaptically local plasticity rules to optimize a cooperative, linear generative image model formed by the neural representation. Finally, we discuss several interesting emergent properties of our network, with the intent of bridging the gap between theoretical and experimental studies of visual cortex.     

Semantic region of interest and species classification in the deep neural network feature domain

In this paper, we focus on animal object detection and species classification in camera-trap images collected in highly cluttered natural scenes. Using a deep neural network (DNN) model training for animal- background image classification, we analyze the input camera-trap images to generate a multi-level visual representation of the input image. We detect semantic regions of interest for animals from this representation using k-mean clustering and graph cut in the DNN feature domain. These animal regions are then classified into animal species using multi-class deep neural network model. According the experimental results, our method achieves 99.75% accuracy for classifying animals and background and 90.89% accuracy for classifying 26 animal species on the Snapshot Serengeti dataset, outperforming existing image classification methods.     

改良的小波变换算法在生物微粒检测中的应用

荧光图像的微粒检测已经成为了生物学研究中不可或缺的工具之一．介绍了一种改良的小波变换算法(improved wavelet transform,IWT),该方法实现简单,能够以很高的速度和精度来进行生物微粒的检测．IWT源自多尺度小波乘积算法(wavelet multiscale products,WMP),但它不仅解决了WMP算法遇到的问题,而且在处理各类图像的时候具有更强的适应性．使用人工合成的图像和真实的图像来定量地分析IWT、WMP以及多尺度方差稳定变换算法(multiscale variance stabilizing transform,MSVST)的检测效果．实验结果表明, IWT在大多数情况下的检测效果比WMP好很多,且与更为复杂的MSVST算法相当．此外,在处理相同图像时,IWT的速度比MSVST快20%．因此,IWT算法能够普遍适用于各种生物微粒的自动化检测,其简单准确的特点使之成为荧光图像分析更好的选择．     

Environments that induce synthetic microbial ecosystems

Interactions between microbial species are sometimes mediated by the exchange of small molecules, secreted by one species and metabolized by another. Both one-way (commensal) and two-way (mutualistic) interactions may contribute to complex networks of interdependencies. Understanding these interactions constitutes an open challenge in microbial ecology, with applications ranging from the human microbiome to environmental sustainability. In parallel to natural communities, it is possible to explore interactions in artificial microbial ecosystems, e.g. pairs of genetically engineered mutualistic strains. Here we computationally generate artificial microbial ecosystems without re-engineering the microbes themselves, but rather by predicting their growth on appropriately designed media. We use genome-scale stoichiometric models of metabolism to identify media that can sustain growth for a pair of species, but fail to do so for one or both individual species, thereby inducing putative symbiotic interactions. We first tested our approach on two previously studied mutualistic pairs, and on a pair of highly curated model organisms, showing that our algorithms successfully recapitulate known interactions, robustly predict new ones, and provide novel insight on exchanged molecules. We then applied our method to all possible pairs of seven microbial species, and found that it is always possible to identify putative media that induce commensalism or mutualism. Our analysis also suggests that symbiotic interactions may arise more readily through environmental fluctuations than genetic modifications. We envision that our approach will help generate microbe-microbe interaction maps useful for understanding microbial consortia dynamics and evolution, and for exploring the full potential of natural metabolic pathways for metabolic engineering applications.     

Capturing the Dynamics of a Hybrid Multiscale Cancer Model with a Continuum Model

Cancer is a complex disease involving processes at spatial scales from subcellular, like cell signalling, to tissue scale, such as vascular network formation. A number of multiscale models have been developed to study the dynamics that emerge from the coupling between the intracellular, cellular and tissue scales. Here, we develop a continuum partial differential equation model to capture the dynamics of a particular multiscale model (a hybrid cellular automaton with discrete cells, diffusible factors and an explicit vascular network). The purpose is to test under which circumstances such a continuum model gives equivalent predictions to the original multiscale model, in the knowledge that the system details are known, and differences in model results can be explained in terms of model features (rather than unknown experimental confounding factors). The continuum model qualitatively replicates the dynamics from the multiscale model, with certain discrepancies observed owing to the differences in the modelling of certain processes. The continuum model admits travelling wave solutions for normal tissue growth and tumour invasion, with similar behaviour observed in the multiscale model. However, the continuum model enables us to analyse the spatially homogeneous steady states of the system, and hence to analyse these waves in more detail. We show that the tumour microenvironmental effects from the multiscale model mean that tumour invasion exhibits a so-called pushed wave when the carrying capacity for tumour cell proliferation is less than the total cell density at the tumour wave front. These pushed waves of tumour invasion propagate by triggering apoptosis of normal cells at the wave front. Otherwise, numerical evidence suggests that the wave speed can be predicted from linear analysis about the normal tissue steady state.     

A generative bias towards average complexity in artificial cell lineages

The evolution of life on earth has been characterized by generalized long-term increases in phenotypic complexity. Although natural selection is a plausible cause for these trends, one alternative hypothesis--generative bias--has been proposed repeatedly based on theoretical considerations. Here, we introduce a computational model of a developmental system and use it to test the hypothesis that long-term increasing trends in phenotypic complexity are caused by a generative bias towards greater complexity. We use our model to generate random organisms with different levels of phenotypic complexity and analyse the distributions of mutational effects on complexity. We show that highly complex organisms are easy to generate but there are trade-offs between different measures of complexity. We also find that only the simplest possible phenotypes show a generative bias towards higher complexity, whereas phenotypes with high complexity display a generative bias towards lower complexity. These results suggest that generative biases alone are not sufficient to explain long-term evolutionary increases in phenotypic complexity. Rather, our finding of a generative bias towards average complexity argues for a critical role of selective biases in driving increases in phenotypic complexity and in maintaining high complexity once it has evolved.     

Using a multiscale image processing method to characterize the periodic growth patterns on scallop shells

The fine periodic growth patterns on shell surfaces have been widely used for studies in the ecology and evolution of scallops. Modern X‐ray CT scanners and digital cameras can provide high‐resolution image data that contain abundant information such as the shell formation rate, ontogenetic age, and life span of shellfish organisms. We introduced a novel multiscale image processing method based on matched filters with Gaussian kernels and partial differential equation (PDE) multiscale hierarchical decomposition to segment the small tubular and periodic structures in scallop shell images. The periodic patterns of structures (consisting of bifurcation points, crossover points of the rings and ribs, and the connected lines) could be found by our Space‐based Depth‐First Search (SDFS) algorithm. We created a MATLAB package to implement our method of periodic pattern extraction and pattern matching on the CT and digital scallop images available in this study. The results confirmed the hypothesis that the shell cyclic structure patterns encompass genetically specific information that can be used as an effective invariable biomarker for biological individual recognition. The package is available with a quick‐start guide and includes three examples: http://mgb.ouc.edu.cn/novegene/html/code.php .     

Statistical regularities of art images and natural scenes: spectra, sparseness and nonlinearities

Paintings are the product of a process that begins with ordinary vision in the natural world and ends with manipulation of pigments on canvas. Because artists must produce images that can be seen by a visual system that is thought to take advantage of statistical regularities in natural scenes, artists are likely to replicate many of these regularities in their painted art. We have tested this notion by computing basic statistical properties and modeled cell response properties for a large set of digitized paintings and natural scenes. We find that both representational and non-representational (abstract) paintings from our sample (124 images) show basic similarities to a sample of natural scenes in terms of their spatial frequency amplitude spectra, but the paintings and natural scenes show significantly different mean amplitude spectrum slopes. We also find that the intensity distributions of paintings show a lower skewness and sparseness than natural scenes. We account for this by considering the range of luminances found in the environment compared to the range available in the medium of paint. A painting's range is limited by the reflective properties of its materials. We argue that artists do not simply scale the intensity range down but use a compressive nonlinearity. In our studies, modeled retinal and cortical filter responses to the images were less sparse for the paintings than for the natural scenes. But when a compressive nonlinearity was applied to the images, both the paintings' sparseness and the modeled responses to the paintings showed the same or greater sparseness compared to the natural scenes. This suggests that artists achieve some degree of nonlinear compression in their paintings. Because paintings have captivated humans for millennia, finding basic statistical regularities in paintings' spatial structure could grant insights into the range of spatial patterns that humans find compelling.     

Lipid lateral organization on giant unilamellar vesicles containing lipopolysaccharides

We developed a new (to our knowledge) protocol to generate giant unilamellar vesicles (GUVs) composed of mixtures of single lipopolysaccharide (LPS) species and Escherichia coli polar lipid extracts. Four different LPSs that differed in the size of the polar headgroup (i.e., LPS smooth > LPS-Ra > LPS-Rc > LPS-Rd) were selected to generate GUVs composed of different LPS/E. coli polar lipid mixtures. Our procedure consists of two main steps: 1), generation and purification of oligolamellar liposomes containing LPSs; and 2), electroformation of GUVs using the LPS-containing oligolamellar vesicles at physiological salt and pH conditions. Analysis of LPS incorporation into the membrane models (both oligolamellar vesicles and GUVs) shows that the final concentration of LPS is lower than that expected from the initial E. coli lipids/LPS mixture. In particular, our protocol allows incorporation of no more than 15 mol % for LPS-smooth and LPS-Ra, and up to 25 mol % for LPS-Rc and LPS-Rd (with respect to total lipids). We used the GUVs to evaluate the impact of different LPS species on the lateral structure of the host membrane (i.e., E. coli polar lipid extract). Rhodamine-DPPE-labeled GUVs show the presence of elongated micrometer-sized lipid domains for GUVs containing either LPS-Rc or LPS-Rd above 10 mol %. Laurdan GP images confirm this finding and show that this particular lateral scenario corresponds to the coexistence of fluid disordered and gel (LPS-enriched)-like micron-sized domains, in similarity to what is observed when LPS is replaced with lipid A. For LPSs containing the more bulky polar headgroup (i.e., LPS-smooth and LPS-Ra), an absence of micrometer-sized domains is observed for all LPS concentrations explored in the GUVs (up to ∼15 mol %). However, fluorescence correlation spectroscopy (using fluorescently labeled LPS) and Laurdan GP experiments in these microscopically homogeneous membranes suggests the presence of LPS clusters with dimensions below our microscope's resolution (∼380 nm radial). Our results indicate that LPSs can cluster into gel-like domains in these bacterial model membranes, and that the size of these domains depends on the chemical structure and concentration of the LPSs.     

Learning receptive fields using predictive feedback.

Previously, it was suggested that feedback connections from higher- to lower-level areas carry predictions of lower-level neural activities, whereas feedforward connections carry the residual error between the predictions and the actual lower-level activities [Rao, R.P.N., Ballard, D.H., 1999. Nature Neuroscience 2, 79-87.]. A computational model implementing the hypothesis learned simple cell receptive fields when exposed to natural images. Here, we use predictive feedback to explain tuning properties in medial superior temporal area (MST). We implement the hypothesis using a new, biologically plausible, algorithm based on matching pursuit, which retains all the features of the previous implementation, including its ability to efficiently encode input. When presented with natural images, the model developed receptive field properties as found in primary visual cortex. In addition, when exposed to visual motion input resulting from movements through space, the model learned receptive field properties resembling those in MST. These results corroborate the idea that predictive feedback is a general principle used by the visual system to efficiently encode natural input.     

Stochastic ICA contrast maximisation using OJA's nonlinear PCA algorithm

Independent Component Analysis (ICA) is an important extension of linear Principal Component Analysis (PCA). PCA performs a data transformation to provide independence to second order, that is, decorrelation. ICA transforms data to provide approximate independence up to and beyond second order yielding transformed data with fully factorable probability densities. The linear ICA transformation has been applied to the classical statistical signal-processing problem of Blind Separation of Sources (BSS), that is, separating unknown original source signals from a mixture whose mode of mixing is undetermined. In this paper it is shown that Oja's Nonlinear PCA algorithm performs a general stochastic online adaptive ICA. This analysis is corroborated with three simulations. The first separates unknown mixtures of original natural images, which have sub-Gaussian densities, the second separates linear mixtures of natural speech whose densities are super-Gaussian. Finally unknown mixtures of original images, which have both sub- and super-Gaussian densities are separated.     

Integrating intracellular dynamics using CompuCell3D and Bionetsolver: applications to multiscale modelling of cancer cell growth and invasion

In this paper we present a multiscale, individual-based simulation environment that integrates CompuCell3D for lattice-based modelling on the cellular level and Bionetsolver for intracellular modelling. CompuCell3D or CC3D provides an implementation of the lattice-based Cellular Potts Model or CPM (also known as the Glazier-Graner-Hogeweg or GGH model) and a Monte Carlo method based on the metropolis algorithm for system evolution. The integration of CC3D for cellular systems with Bionetsolver for subcellular systems enables us to develop a multiscale mathematical model and to study the evolution of cell behaviour due to the dynamics inside of the cells, capturing aspects of cell behaviour and interaction that is not possible using continuum approaches. We then apply this multiscale modelling technique to a model of cancer growth and invasion, based on a previously published model of Ramis-Conde et al. (2008) where individual cell behaviour is driven by a molecular network describing the dynamics of E-cadherin and β-catenin. In this model, which we refer to as the centre-based model, an alternative individual-based modelling technique was used, namely, a lattice-free approach. In many respects, the GGH or CPM methodology and the approach of the centre-based model have the same overall goal, that is to mimic behaviours and interactions of biological cells. Although the mathematical foundations and computational implementations of the two approaches are very different, the results of the presented simulations are compatible with each other, suggesting that by using individual-based approaches we can formulate a natural way of describing complex multi-cell, multiscale models. The ability to easily reproduce results of one modelling approach using an alternative approach is also essential from a model cross-validation standpoint and also helps to identify any modelling artefacts specific to a given computational approach.     

Simple model for encoding natural images by retinal ganglion cells with nonlinear spatial integration

A central goal in sensory neuroscience is to understand the neuronal signal processing involved in the encoding of natural stimuli. A critical step towards this goal is the development of successful computational encoding models. For ganglion cells in the vertebrate retina, the development of satisfactory models for responses to natural visual scenes is an ongoing challenge. Standard models typically apply linear integration of visual stimuli over space, yet many ganglion cells are known to show nonlinear spatial integration, in particular when stimulated with contrast-reversing gratings. We here study the influence of spatial nonlinearities in the encoding of natural images by ganglion cells, using multielectrode-array recordings from isolated salamander and mouse retinas. We assess how responses to natural images depend on first- and second-order statistics of spatial patterns inside the receptive field. This leads us to a simple extension of current standard ganglion cell models. We show that taking not only the weighted average of light intensity inside the receptive field into account but also its variance over space can partly account for nonlinear integration and substantially improve response predictions of responses to novel images. For salamander ganglion cells, we find that response predictions for cell classes with large receptive fields profit most from including spatial contrast information. Finally, we demonstrate how this model framework can be used to assess the spatial scale of nonlinear integration. Our results underscore that nonlinear spatial stimulus integration translates to stimulation with natural images. Furthermore, the introduced model framework provides a simple, yet powerful extension of standard models and may serve as a benchmark for the development of more detailed models of the nonlinear structure of receptive fields.     

Morphogenesis of the branching reef coral Madracis mirabilis

Understanding external deciding factors in growth and morphology of reef corals is essential to elucidate the role of corals in marine ecosystems, and to explain their susceptibility to pollution and global climate change. Here, we extend on a previously presented model for simulating the growth and form of a branching coral and we compare the simulated morphologies to three-dimensional (3D) images of the coral species Madracis mirabilis. Simulation experiments and isotope analyses of M. mirabilis skeletons indicate that external gradients of dissolved inorganic carbon (DIC) determine the morphogenesis of branching, phototrophic corals. In the simulations we use a first principle model of accretive growth based on local interactions between the polyps. The only species-specific information in the model is the average size of a polyp. From flow tank and simulation studies it is known that a relatively large stagnant and diffusion dominated region develops within a branching colony. We have used this information by assuming in our model that growth is entirely driven by a diffusion-limited process, where DIC supply represents the limiting factor. With such model constraints it is possible to generate morphologies that are virtually indistinguishable from the 3D images of the actual colonies.