Image-based cell phenotyping with deep learning

A cell's phenotype is the culmination of several cellular processes through a complex network of molecular interactions that ultimately result in a unique morphological signature. Visual cell phenotyping is the characterization and quantification of these observable cellular traits in images. Recently, cellular phenotyping has undergone a massive overhaul in terms of scale, resolution, and throughput, which is attributable to advances across electronic, optical, and chemical technologies for imaging cells. Coupled with the rapid acceleration of deep learning–based computational tools, these advances have opened up new avenues for innovation across a wide variety of high-throughput cell biology applications. Here, we review applications wherein deep learning is powering the recognition, profiling, and prediction of visual phenotypes to answer important biological questions. As the complexity and scale of imaging assays increase, deep learning offers computational solutions to elucidate the details of previously unexplored cellular phenotypes.     

Deep learning for computational biology

Technological advances in genomics and imaging have led to an explosion of molecular and cellular profiling data from large numbers of samples. This rapid increase in biological data dimension and acquisition rate is challenging conventional analysis strategies. Modern machine learning methods, such as deep learning, promise to leverage very large data sets for finding hidden structure within them, and for making accurate predictions. In this review, we discuss applications of this new breed of analysis approaches in regulatory genomics and cellular imaging. We provide background of what deep learning is, and the settings in which it can be successfully applied to derive biological insights. In addition to presenting specific applications and providing tips for practical use, we also highlight possible pitfalls and limitations to guide computational biologists when and how to make the most use of this new technology.     

Protein therapeutics: promises and challenges for the 21st century.

Recent advances in massively parallel experimental and computational technologies are leading to radically new approaches to the early phases of the drug production pipeline. The revolution in DNA microarray technologies and the imminent emergence of its analogue for proteins, along with machine learning algorithms, promise rapid acceleration in the identification of potential drug targets, and in high-throughput screens for subpopulation-specific toxicity. Similarly, advances in structural genomics in conjunction with in vitro and in silico evolutionary methods will rapidly accelerate the number of lead drug candidates and substantially augment their target specificity. Taken collectively, these advances will usher in an era of predictive medicine, which will move medical practice from reactive therapy after disease onset, to proactive prevention.     

Stages of motor skill learning

Successful learning of a motor skill requires repetitive training. Once the skill is mastered, it can be remembered for a long period of time. The durable memory makes motor skill learning an interesting paradigm for the study of learning and memory mechanisms. To gain better understanding, one scientific approach is to dissect the process into stages and to study these as well as their interactions. This article covers the growing evidence that motor skill learning advances through stages, in which different storage mechanisms predominate. The acquisition phase is characterized by fast (within session) and slow learning (between sessions). For a short period following the initial training sessions, the skill is labile to interference by other skills and by protein synthesis inhibition, indicating that consolidation processes occur during rest periods between training sessions. During training as well as rest periods, activation in different brain regions changes dynamically. Evidence for stages in motor skill learning is provided by experiments using behavioral, electrophysiological, functional imaging, and cellular/molecular methods.     

Immediate and long-term effects of testosterone on song plasticity and learning in juvenile song sparrows

Steroid sex hormones play critical roles in the development of brain regions used for vocal learning. It has been suggested that puberty-induced increases in circulating testosterone (T) levels crystallize a bird's repertoire and inhibit future song learning. Previous studies show that early administration of T crystallizes song repertoires but have not addressed whether new songs can be learned after this premature crystallization. We brought 8 juvenile song sparrows (Melospiza melodia) into the laboratory in the late summer and implanted half of them with subcutaneous T pellets for a two week period in October. Birds treated with T tripled their singing rates and crystallized normal songs in 2 weeks. After T removal, subjects were tutored by 4 new adults. Birds previously treated with T tended toward learning fewer new songs post T, consistent with the hypothesis that T helps to close the song learning phase. However, one T-treated bird proceeded to learn several new songs in the spring, despite singing perfectly crystallized songs in the fall. His small crystallized fall repertoire and initial lag behind other subjects in song development suggest that this individual may have had limited early song learning experience. We conclude that an exposure to testosterone sufficient for crystallization of a normal song repertoire does not necessarily prevent future song learning and suggest that early social experiences might override the effects of hormones in closing song learning.     

Artificial intelligence and machine learning for medical imaging: A technology review

Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.     

Three models of song learning: Evidence from behavior

Research on avian song learning has traditionally been based on an instructional model, as exemplified by the sensorimotor model of song development. Several large-scale, species-wide field studies of learned birdsongs have revealed that variation is narrowly restricted to certain aspects of song structure. Other aspects are sufficiently stereotyped and so widely shared by species' members that they qualify as species-specific universals. The limitations on natural song variation are difficult to reconcile with a fully open, instructive model of song learning. An alternative model based on memorization by selection postulates a system of innate neural templates that facilitate the recognition and rapid memorization of conspecific song patterns. Behavioral evidence compatible with this model includes learning preferences, rapid conspecific song learning, and widespread ocurrence of species-specific song universals that are recognized innately but fail to develop in songs of social isolates. A third model combines instruction, in the memorization phase, with selection during song production. An overproduced repertoire of plastic songs previously memorized by instruction is winnowed by selection imposed during social interactions at the time of adult song crystallization. Selection during production is well established as a factor in the song development of several species, in the form of action-based learning. The possible role of selective processes in song memorization merits further neurobiological investigation. © 1997 John Wiley & Sons, Inc. J Neurobiol 33: 501–516, 1997     

Application of machine learning methods to pathogen safety evaluation in biological manufacturing processes

The production of recombinant therapeutic proteins from animal or human cell lines entails the risk of endogenous viral contamination from cell substrates and adventitious agents from raw materials and environment. One of the approaches to control such potential viral contamination is to ensure the manufacturing process can adequately clear the potential viral contaminants. Viral clearance for production of human monoclonal antibodies is achieved by dedicated unit operations, such as low pH inactivation, viral filtration, and chromatographic separation. The process development of each viral clearance step for a new antibody production requires significant effort and resources invested in wet laboratory experiments for process characterization studies. Machine learning methods have the potential to help streamline the development and optimization of viral clearance unit operations for new therapeutic antibodies. The current work focuses on evaluating the usefulness of machine learning methods for process understanding and predictive modeling for viral clearance via a case study on low pH viral inactivation.     

Sequencing enabling design and learning in synthetic biology

The ability to read and quantify nucleic acids such as DNA and RNA using sequencing technologies has revolutionized our understanding of life. With the emergence of synthetic biology, these tools are now being put to work in new ways — enabling de novo biological design. Here, we show how sequencing is supporting the creation of a new wave of biological parts and systems, as well as providing the vast data sets needed for the machine learning of design rules for predictive bioengineering. However, we believe this is only the tip of the iceberg and end by providing an outlook on recent advances that will likely broaden the role of sequencing in synthetic biology and its deployment in real-world environments.     

Domestication and stress effects on contrafreeloading and spatial learning performance in red jungle fowl (Gallus gallus) and White Leghorn layers

White Leghorn layers (WL) show modified foraging strategies, compared to their ancestor, the red jungle fowl (RJF). Birds selected for high production may invest more resources into production traits and less in other biological processes. This may affect the capacity to adapt to new or variable environments.Thirty birds of each of RJF and WL were raised in a stressful environment (unpredictable light:dark schedule) and 30 control animals of each breed in similar pens, but on a 12:12 h light:dark schedule. We studied the differences between breed and treatment with respect to contrafreeloading (CFL), spatial learning ability and the birds’ behaviour in a T-maze.WL showed less CFL, were less cautious in the test arena and showed an impaired spatial learning ability compared with RJF in both test situations. Stress impaired spatial learning in both breeds, but stressed RJF showed a more active response to the test situation than non-stressed RJF, by starting to eat faster, while stressed WL prolonged their time to start eating compared to non-stressed WL.Our results may reflect different adaptive strategies, where RJF appear better adapted to an unpredictable environment.     

At the interface of the auditory and vocal motor systems: NIf and its role in vocal processing,production and learning

Communication between auditory and vocal motor nuclei is essential for vocal learning. In songbirds, the nucleus interfacialis of the nidopallium (NIf) is part of a sensorimotor loop, along with auditory nucleus avalanche (Av) and song system nucleus HVC, that links the auditory and song systems. Most of the auditory information comes through this sensorimotor loop, with the projection from NIf to HVC representing the largest single source of auditory information to the song system. In addition to providing the majority of HVC’s auditory input, NIf is also the primary driver of spontaneous activity and premotor-like bursting during sleep in HVC. Like HVC and RA, two nuclei critical for song learning and production, NIf exhibits behavioral-state dependent auditory responses and strong motor bursts that precede song output. NIf also exhibits extended periods of fast gamma oscillations following vocal production. Based on the converging evidence from studies of physiology and functional connectivity it would be reasonable to expect NIf to play an important role in the learning, maintenance, and production of song. Surprisingly, however, lesions of NIf in adult zebra finches have no effect on song production or maintenance. Only the plastic song produced by juvenile zebra finches during the sensorimotor phase of song learning is affected by NIf lesions. In this review, we carefully examine what is known about NIf at the anatomical, physiological, and behavioral levels. We reexamine conclusions drawn from previous studies in the light of our current understanding of the song system, and establish what can be said with certainty about NIf’s involvement in song learning, maintenance, and production. Finally, we review recent theories of song learning integrating possible roles for NIf within these frameworks and suggest possible parallels between NIf and sensorimotor areas that form part of the neural circuitry for speech processing in humans.     

利用文献精读教学新模式优化遗传学教学

《遗传学》是生命科学相关专业本科阶段最重要的课程之一。近年来,随着生命科学领域研究的不断深入,新知识与新技术也在不断更新。但遗传学的教学模式目前仍以理论讲授为主,这使得抽象的原理难以被学生理解接受,直接影响了教学效果。因此探索新的教学模式尤为必要。2010年以来我校在生物技术专业《微生物遗传学》教学中开展了新教学模式——文献精读,文章从文献精读的前期课程基础,如何选择专业文献,怎样组织教学过程,开展文献精读对学生和教师的意义等方面全面分析了实施情况和应用价值,指出该教学模式体现了“前沿”和“经典”的结合,使书本的知识在实践中具体化,既提高学生的学习效果,激发学习兴趣,又开拓了学生的思路,锻炼其能力。这种教学模式为《遗传学》教学授课不断探索新的模式、在“精准医疗”时代下如何培养兼具临床与科研能力的医疗人才提供新思路。     

Evolution of protolinguistic abilities as a by-product of learning to forage in structured environments

The skills required for the learning and use of language are the focus of extensive research, and their evolutionary origins are widely debated. Using agent-based simulations in a range of virtual environments, we demonstrate that challenges of foraging for food can select for cognitive mechanisms supporting complex, hierarchical, sequential learning, the need for which arises in language acquisition. Building on previous work, where we explored the conditions under which reinforcement learning is out-competed by seldom-reinforced continuous learning that constructs a network model of the environment, we now show that realistic features of the foraging environment can select for two critical advances: (i) chunking of meaningful sequences found in the data, leading to representations composed of units that better fit the prevalent statistical patterns in the environment; and (ii) generalization across units based on their contextual similarity. Importantly, these learning processes, which in our framework evolved for making better foraging decisions, had been earlier shown to reproduce a range of findings in language learning in humans. Thus, our results suggest a possible evolutionary trajectory that may have led from basic learning mechanisms to complex hierarchical sequential learning that can support advanced cognitive abilities of the kind needed for language acquisition.     

Songbird: a unique animal model for studying the molecular basis of disorders of vocal development and communication

Like humans, songbirds are one of the few animal groups that learn vocalization. Vocal learning requires coordination of auditory input and vocal output using auditory feedback to guide one’s own vocalizations during a specific developmental stage known as the critical period. Songbirds are good animal models for understand the neural basis of vocal learning, a complex form of imitation, because they have many parallels to humans with regard to the features of vocal behavior and neural circuits dedicated to vocal learning. In this review, we will summarize the behavioral, neural, and genetic traits of birdsong. We will also discuss how studies of birdsong can help us understand how the development of neural circuits for vocal learning and production is driven by sensory input (auditory information) and motor output (vocalization).     

The promise of new ideas and new technology for improving teaching and learning

There have been enormous advances in our understanding of human learning in the past three decades. There have also been important advances in our understanding of the nature of knowledge and new knowledge creation. These advances, when combined with the explosive development of the Internet and other technologies, permit advances in educational practices at least as important as the invention of the printing press in 1460. We have built on the cognitive learning theory of David Ausubel and various sources of new ideas on epistemology. Our research program has focused on understanding meaningful learning and on developing better methods to achieve such learning and to assess progress in meaningful learning. The concept map tool developed in our program has proved to be highly effective both in promoting meaningful learning and in assessing learning outcomes. Concept mapping strategies are also proving powerful for eliciting, capturing, and archiving knowledge of experts and organizations. New technology for creating concept maps developed at the University of West Florida permits easier and better concept map construction, thus facilitating learning, knowledge capture, and local or distance creation and sharing of structured knowledge, especially when utilized with the Internet. A huge gap exists between what we now know to improve learning and use of knowledge and the practices currently in place in most schools and corporations. There are promising projects in progress that may help to achieve accelerated advances. These include projects in schools at all educational levels, including projects in Colombia, Costa Rica, Italy, Spain, and the United States, and collaborative projects with corporate organizations and distance learning projects. Results to date have been encouraging and suggest that we may be moving from the lag phase of educational innovation to a phase of exponential growth.     

Budding biology teachers: what have botanical gardens got to offer inquiry learning

Teachers conceptualise inquiry learning in science learning differently. This is particularly evident when teachers are introduced to inquiry pedagogy within a new context. This exploratory study draws on semi-structured interviews conducted with eight pre-service secondary biology teachers following a day visit with university tutors to the Royal Botanical Gardens, Kew. Emerging findings were: first, pre-service biology teachers’ views of inquiry learning range in sophistication from simple notions of ‘learning from doing’ to complex multi-notions such as student generated questions, developing curiosity and encouraging authentic scientific practices. Second, similarly their views of inquiry learning opportunities in botanical gardens ranged from simply places that offered ‘memorable experiences’ to enabling autonomous learning due to the organism diversity and multiple climates. Pre-service teachers categorised as having unsophisticated views of inquiry learning had limited expectations of botanical gardens as productive learning environments. Third, the majority of pre-service teachers were concerned about managing inquiry learning. A tension was identified between how open-ended an inquiry activity could be whilst ensuring student focus. Further, participants were concerned about the practical management of inquiry learning. We discuss implications for teacher educators and botanical garden educators and the requirement for curriculum development and promotion.     

A robust deep attention dense convolutional neural network for plant leaf disease identification and classification from smart phone captured real world images

Plant diseases cause significant food loss and hence economic loss around the globe. Therefore, automatic plant disease identification is a primary task to take proper medications for controlling the spread of the diseases. Large variety of plants species and their dissimilar phytopathological symptoms call for the implementation of supervised machine learning techniques for efficient and reliable disease identification and classification. With the development of deep learning strategies, convolutional neural network (CNN) has paved its way for classification of multiple plant diseases by extracting rich features. However, several characteristics of the input images especially captured in real world environment, viz. complex or indistinguishable background, presence of multiple leaves with the diseased leaf, small lesion area, solemnly affect the robustness and accuracy of the CNN modules. Available strategies usually applied standard CNN architectures on the images captured in the laboratory environment and very few have considered practical in-field leaf images for their studies. However, those studies are limited with very limited number of plant species. Therefore, there is need of a robust CNN module which can successfully recognize and classify the dissimilar leaf health conditions of non-identical plants from the in-field RGB images. To achieve the above goal, an attention dense learning (ADL) mechanism is proposed in this article by merging mixed sigmoid attention learning with the basic dense learning process of deep CNN. The basic dense learning process derives new features at higher layer considering all lower layer features and that provides fast and efficient training process. Further, the attention learning process amplifies the learning ability of the dense block by discriminating the meaningful lesion portions of the images from the background areas. Other than adding an extra layer for attention learning, in the proposed ADL block the output features from higher layer dense learning are used as an attention mask to the lower layers. For an effective and fast classification process, five ADL blocks are stacked to build a new CNN architecture named DADCNN-5 for obtaining classification robustness and higher testing accuracy. Initially, the proposed DADCNN-5 module is applied on publicly available extended PlantVillage dataset to classify 38 different health conditions of 14 plant species from 54,305 images. Classification accuracy of 99.93% proves that the proposed CNN module can be used for successful leaf disease identification. Further, the efficacy of the DADCNN-5 model is checked after performing stringent experiments on a new real world plant leaf database, created by the authors. The new leaf database contains 10,851 real-world RGB leaf images of 17 plant species for classifying their 44 distinguished health conditions. Experimental outcomes reveal that the proposed DADCNN-5 outperforms the existing machine learning and standard CNN architectures, and achieved 97.33% accuracy. The obtained sensitivity, specificity and false positive rate values are 96.57%, 99.94% and 0.063% respectively. The module takes approximately 3235 min for training process and achieves 99.86% of training accuracy. Visualization of Class activation mapping (CAM) depicts that DADCNN-5 is able to learn distinguishable features from semantically important regions (i.e. lesion regions) on the leaves. Further, the robustness of the DADCNN-5 is established after experimenting with augmented and noise contaminated images of the practical database.     

Mathematical properties of neuronal TD-rules and differential Hebbian learning: a comparison

A confusingly wide variety of temporally asymmetric learning rules exists related to reinforcement learning and/or to spike-timing dependent plasticity, many of which look exceedingly similar, while displaying strongly different behavior. These rules often find their use in control tasks, for example in robotics and for this rigorous convergence and numerical stability is required. The goal of this article is to review these rules and compare them to provide a better overview over their different properties. Two main classes will be discussed: temporal difference (TD) rules and correlation based (differential hebbian) rules and some transition cases. In general we will focus on neuronal implementations with changeable synaptic weights and a time-continuous representation of activity. In a machine learning (non-neuronal) context, for TD-learning a solid mathematical theory has existed since several years. This can partly be transfered to a neuronal framework, too. On the other hand, only now a more complete theory has also emerged for differential Hebb rules. In general rules differ by their convergence conditions and their numerical stability, which can lead to very undesirable behavior, when wanting to apply them. For TD, convergence can be enforced with a certain output condition assuring that the δ-error drops on average to zero (output control). Correlation based rules, on the other hand, converge when one input drops to zero (input control). Temporally asymmetric learning rules treat situations where incoming stimuli follow each other in time. Thus, it is necessary to remember the first stimulus to be able to relate it to the later occurring second one. To this end different types of so-called eligibility traces are being used by these two different types of rules. This aspect leads again to different properties of TD and differential Hebbian learning as discussed here. Thus, this paper, while also presenting several novel mathematical results, is mainly meant to provide a road map through the different neuronally emulated temporal asymmetrical learning rules and their behavior to provide some guidance for possible applications.     

Nonlinear machine learning in simulations of soft and biological materials

Abstract

Interpretable parameterisations of free energy landscapes for soft and biological materials calculated from molecular simulation require the availability of ‘good’ collective variables (CVs) capable of discriminating the metastable states of the system and the barriers between them. If these CVs are coincident with the slow collective modes governing the long-time dynamical evolution, then they also furnish good coordinates in which to perform enhanced sampling to surmount high free energy barriers and efficiently explore and recover the landscape. Non-linear manifold learning techniques provide a means to systematically extract such CVs from molecular simulation trajectories by identifying and extracting low-dimensional manifolds lying latent within the high-dimensional coordinate space. We survey recent advances in data-driven CV discovery and enhanced sampling using non-linear manifold learning, describe the mathematical and theoretical underpinnings of these techniques, and present illustrative examples to molecular folding and colloidal self-assembly. We close with our outlook and perspective on future advances in this rapidly evolving field.