Modeling of municipal waste disposal behaviors related to meteorological seasons using recurrent neural network LSTM models

The literature suggests that long short-term memory (LSTM) paired with recurrent neural network (RNN) can better express long- and short-term reliance of a data set. The study objectives are to quantify mixed waste disposal (MWD) behaviors at a Canadian landfill from 2013 to 2021, and develop separate RNN-LSTM models to predict MWD rates under four meteorological seasons. Seasonal variations are clearly presented in the historical disposal data, with higher MWD of 417.8 t/month in summer and about 289.7 t/month in winter. The variabilities of MWD are also different among the seasons. Winter experienced the least variation, probably due to similarities in inhabitants' lifestyles. All seasonal sets are negatively skewed, and the highest skewness is observed in summer. The overall model performance using the entire data range is generally satisfactory, with R² values between 0.72–0.86. Meteorological seasons appear to be a significant factor in waste disposal rate modeling. The model performances are less reliable for smaller disposal rates <200 t/day, with 0.01 < R² < 0.59. The results suggest the disposal behaviors on a quiet day can be quite different. The use of distinct time series related to seasons on MWD modeling is original. The proposed analytical approach provides an alternative waste modeling approach accounting for both short term (seasonal) and longer term (annual) effects.     

Prediction of continuous B-cell epitopes in an antigen using recurrent neural network

B-cell epitopes play a vital role in the development of peptide vaccines, in diagnosis of diseases, and also for allergy research. Experimental methods used for characterizing epitopes are time consuming and demand large resources. The availability of epitope prediction method(s) can rapidly aid experimenters in simplifying this problem. The standard feed-forward (FNN) and recurrent neural network (RNN) have been used in this study for predicting B-cell epitopes in an antigenic sequence. The networks have been trained and tested on a clean data set, which consists of 700 non-redundant B-cell epitopes obtained from Bcipep database and equal number of non-epitopes obtained randomly from Swiss-Prot database. The networks have been trained and tested at different input window length and hidden units. Maximum accuracy has been obtained using recurrent neural network (Jordan network) with a single hidden layer of 35 hidden units for window length of 16. The final network yields an overall prediction accuracy of 65.93% when tested by fivefold cross-validation. The corresponding sensitivity, specificity, and positive prediction values are 67.14, 64.71, and 65.61%, respectively. It has been observed that RNN (JE) was more successful than FNN in the prediction of B-cell epitopes. The length of the peptide is also important in the prediction of B-cell epitopes from antigenic sequences. The webserver ABCpred is freely available at www.imtech.res.in/raghava/abcpred/.     

Artificial neural network prediction of viruses in shellfish

A database was probed with artificial neural network (ANN) and multivariate logistic regression (MLR) models to investigate the efficacy of predicting PCR-identified human adenovirus (ADV), Norwalk-like virus (NLV), and enterovirus (EV) presence or absence in shellfish harvested from diverse countries in Europe (Spain, Sweden, Greece, and the United Kingdom). The relative importance of numerical and heuristic input variables to the ANN model for each country and for the combined data was analyzed with a newly defined relative strength effect, which illuminated the importance of bacteriophages as potential viral indicators. The results of this analysis showed that ANN models predicted all types of viral presence and absence in shellfish with better precision than MLR models for a multicountry database. For overall presence/absence classification accuracy, ANN modeling had a performance rate of 95.9%, 98.9%, and 95.7% versus 60.5%, 75.0%, and 64.6% for the MLR for ADV, NLV, and EV, respectively. The selectivity (prediction of viral negatives) was greater than the sensitivity (prediction of viral positives) for both models and with all virus types, with the ANN model performing with greater sensitivity than the MLR. ANN models were able to illuminate site-specific relationships between microbial indicators chosen as model inputs and human virus presence. A validation study on ADV demonstrated that the MLR and ANN models differed in sensitivity and selectivity, with the ANN model correctly identifying ADV presence with greater precision.     

Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks

Recurrent neural networks (RNNs) are widely used in computational neuroscience and machine learning applications. In an RNN, each neuron computes its output as a nonlinear function of its integrated input. While the importance of RNNs, especially as models of brain processing, is undisputed, it is also widely acknowledged that the computations in standard RNN models may be an over-simplification of what real neuronal networks compute. Here, we suggest that the RNN approach may be made computationally more powerful by its fusion with Bayesian inference techniques for nonlinear dynamical systems. In this scheme, we use an RNN as a generative model of dynamic input caused by the environment, e.g. of speech or kinematics. Given this generative RNN model, we derive Bayesian update equations that can decode its output. Critically, these updates define a 'recognizing RNN' (rRNN), in which neurons compute and exchange prediction and prediction error messages. The rRNN has several desirable features that a conventional RNN does not have, e.g. fast decoding of dynamic stimuli and robustness to initial conditions and noise. Furthermore, it implements a predictive coding scheme for dynamic inputs. We suggest that the Bayesian inversion of RNNs may be useful both as a model of brain function and as a machine learning tool. We illustrate the use of the rRNN by an application to the online decoding (i.e. recognition) of human kinematics.     

Adaptation supports short-term memory in a visual change detection task

The maintenance of short-term memories is critical for survival in a dynamically changing world. Previous studies suggest that this memory can be stored in the form of persistent neural activity or using a synaptic mechanism, such as with short-term plasticity. Here, we compare the predictions of these two mechanisms to neural and behavioral measurements in a visual change detection task. Mice were trained to respond to changes in a repeated sequence of natural images while neural activity was recorded using two-photon calcium imaging. We also trained two types of artificial neural networks on the same change detection task as the mice. Following fixed pre-processing using a pretrained convolutional neural network, either a recurrent neural network (RNN) or a feedforward neural network with short-term synaptic depression (STPNet) was trained to the same level of performance as the mice. While both networks are able to learn the task, the STPNet model contains units whose activity are more similar to the in vivo data and produces errors which are more similar to the mice. When images are omitted, an unexpected perturbation which was absent during training, mice often do not respond to the omission but are more likely to respond to the subsequent image. Unlike the RNN model, STPNet produces a similar pattern of behavior. These results suggest that simple neural adaptation mechanisms may serve as an important bottom-up memory signal in this task, which can be used by downstream areas in the decision-making process.     

Inference of genetic regulatory networks with recurrent neural network models using particle swarm optimization

Genetic regulatory network inference is critically important for revealing fundamental cellular processes, investigating gene functions, and understanding their relations. The availability of time series gene expression data makes it possible to investigate the gene activities of whole genomes, rather than those of only a pair of genes or among several genes. However, current computational methods do not sufficiently consider the temporal behavior of this type of data and lack the capability to capture the complex nonlinear system dynamics. We propose a recurrent neural network (RNN) and particle swarm optimization (PSO) approach to infer genetic regulatory networks from time series gene expression data. Under this framework, gene interaction is explained through a connection weight matrix. Based on the fact that the measured time points are limited and the assumption that the genetic networks are usually sparsely connected, we present a PSO-based search algorithm to unveil potential genetic network constructions that fit well with the time series data and explore possible gene interactions. Furthermore, PSO is used to train the RNN and determine the network parameters. Our approach has been applied to both synthetic and real data sets. The results demonstrate that the RNN/PSO can provide meaningful insights in understanding the nonlinear dynamics of the gene expression time series and revealing potential regulatory interactions between genes.     

Application of an artificial neural network (ANN) model for predicting mosquito abundances in urban areas

The mosquito species is one of most important insect vectors of several diseases, namely, malaria, filariasis, Japanese encephalitis, dengue, and so on. In particular, in recent years, as the number of people who enjoy outdoor activities in urban areas continues to increase, information about mosquito activity is in demand. Furthermore, mosquito activity prediction is crucial for managing the safety and the health of humans. However, the estimation of mosquito abundances frequently involves uncertainty because of high spatial and temporal variations, which hinders the accuracy of general mechanistic models of mosquito abundances. For this reason, it is necessary to develop a simpler and lighter mosquito abundance prediction model. In this study, we tested the efficacy of the artificial neural network (ANN), which is a popular empirical model, for mosquito abundance prediction. For comparison, we also developed a multiple linear regression (MLR) model. Both the ANN and the MLR models were applied to estimate mosquito abundances in 2-year observations in Yeongdeungpo-gu, Seoul, conducted using the Digital Mosquito Monitoring System (DMS). As input variables, we used meteorological data, including temperature, wind speed, humidity, and precipitation. The results showed that performances of the ANN model and the MLR model are almost same in terms of R and root mean square error (RMSE). The ANN model was able to predict the high variability as compared to MLR. A sensitivity analysis of the ANN model showed that the relationships between input variables and mosquito abundances were well explained. In conclusion, ANNs have the potential to predict fluctuations in mosquito numbers (especially the extreme values), and can do so better than traditional statistical techniques. But, much more work needs to be conducted to assess meaningful time delays in environmental variables and mosquito numbers.     

Modeling daily chlorophyll a dynamics in a German lowland river using artificial neural networks and multiple linear regression approaches

Phytoplankton biomass is an important indicator for water quality, and predicting its dynamics is thus regarded as one of the important issues in the domain of river ecology and management. However, the vast majority of models in river systems have focused mostly on flow prediction and water quality with very few applications to biotic parameters such as chlorophyll a (Chl a). Based on a 1.5-year measured dataset of Chl a and environmental variables, we developed two modeling approaches [artificial neural networks (ANN) and multiple linear regression (MLR)] to simulate the daily Chl a dynamics in a German lowland river. In general, the developed ANN and MLR models achieved satisfactory accuracy in predicting daily dynamics of Chl a concentrations. Although some peaks and lows were not predicted, the predicted and the observed data matched closely by the MLR model with the coefficient of determination (R ²), Nash–Sutcliffe efficiency (NS), and the root mean square error (RMSE) of 0.53, 0.53, and 2.75 for the calibration period and 0.63, 0.62, and 1.94 for the validation period, respectively. Likewise, the results of the ANN model also illustrated a good agreement between observed and predicted data during calibration and validation periods, which was demonstrated by R ², NS, and RMSE values (0.68, 0.68, and 2.27 for the calibration period, 0.55, 0.66 and 2.12 for the validation period, respectively). According to the sensitivity analysis, Chl a concentration was highly sensitive to dissolved inorganic nitrogen, nitrate–nitrogen, autoregressive Chl a, chloride, sulfate, and total phosphorus. We concluded that it was possible to predict the daily Chl a dynamics in the German lowland river based on relevant environmental factors using either ANN or MLR models. The ANN model is well suited for solving non-linear and complex problems, while the MLR model can explicitly explore the coefficients between independent and dependent variables. Further studies are still needed to improve the accuracy of the developed models.     

Design and Implementation of a Flexible Manufacturing Control System Using Neural Network

Design and implementation of a sequential controller based on the concept of artificial neural networks for a flexible manufacturing system are presented. The recurrent neural network (RNN) type is used for such a purpose. Contrary to the programmable controller, an RNN-based sequential controller is based on a definite mathematical model rather than depending on experience and trial and error techniques. The proposed controller is also more flexible because it is not limited by the restrictions of the finite state automata theory. Adequate guidelines of how to construct an RNN-based sequential controller are presented. These guidelines are applied to different case studies. The proposed controller is tested by simulations and real-time experiments. These tests prove the successfulness of the proposed controller performances. Theoretical as well as experimental results are presented and discussed indicating that the proposed design procedure using Elman's RNN can be effective in designing a sequential controller for event-based type manufacturing systems. In addition, the simulation results assure the effectiveness of the proposed controller to outperform the effect of noisy inputs.     

Exploring two methods for statistical downscaling of Central European phenological time series

In this study we set out to investigate the possibility of linking phenological phases throughout the vegetation cycle, as a local-scale biological phenomenon, directly with large-scale atmospheric variables via two different empirical downscaling techniques. In recent years a number of methods have been developed to transfer atmospheric information at coarse General Circulation Model's grid resolutions to local scales and individual points. Here multiple linear regression (MLR) and canonical correlation analysis (CCA) have been selected as downscaling methods. Different validation experiments (e.g. temporal cross-validation, split-sample tests) are used to test the performance of both approaches and compare them for time series of 17 phenological phases and air temperatures from Central Europe as microscale variables. A number of atmospheric variables over the North Atlantic and Europe are utilized as macroscale predictors. The period considered is 1951–1998. Temporal cross-validation reveals that the CCA model generally performs better than MLR, which explains 20%–50% of the phenological variances, whereas the CCA model shows a range from 40% to over 60% throughout most of the vegetation cycle. To show the validity of employing phenological observations for downscaling purposes both methods (MLR and CCA) are also applied to gridded local air temperature time series over Central Europe. In this case there is no obvious superiority of the CCA model over the MLR model. Both models show explained variances from 40% to over 70% in the temporal cross-validation experiment. The results of this study indicate that time series of phenological occurrence dates are very compatible with the needs of empirical downscaling originally developed of local-scale atmospheric variables.     

White Blood Cells Image Classification Using Deep Learning with Canonical Correlation Analysis

White Blood Cells play an important role in observing the health condition of an individual. The opinion related to blood disease involves the identification and characterization of a patient's blood sample. Recent approaches employ Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and merging of CNN and RNN models to enrich the understanding of image content. From beginning to end, training of big data in medical image analysis has encouraged us to discover prominent features from sample images. A single cell patch extraction from blood sample techniques for blood cell classification has resulted in the good performance rate. However, these approaches are unable to address the issues of multiple cells overlap. To address this problem, the Canonical Correlation Analysis (CCA) method is used in this paper. CCA method views the effects of overlapping nuclei where multiple nuclei patches are extracted, learned and trained at a time. Due to overlapping of blood cell images, the classification time is reduced, the dimension of input images gets compressed and the network converges faster with more accurate weight parameters. Experimental results evaluated using publicly available database show that the proposed CNN and RNN merging model with canonical correlation analysis determines higher accuracy compared to other state-of-the-art blood cell classification techniques.     

Towards the next generation of recurrent network models for cognitive neuroscience

Recurrent neural networks (RNNs) trained with machine learning techniques on cognitive tasks have become a widely accepted tool for neuroscientists. In this short opinion piece, we discuss fundamental challenges faced by the early work of this approach and recent steps to overcome such challenges and build next-generation RNN models for cognition. We propose several essential questions that practitioners of this approach should address to continue to build future generations of RNN models.