Voltage-stepping schemes for the simulation of spiking neural networks

The numerical simulation of spiking neural networks requires particular attention. On the one hand, time-stepping methods are generic but they are prone to numerical errors and need specific treatments to deal with the discontinuities of integrate-and-fire models. On the other hand, event-driven methods are more precise but they are restricted to a limited class of neuron models. We present here a voltage-stepping scheme that combines the advantages of these two approaches and consists of a discretization of the voltage state-space. The numerical simulation is reduced to a local event-driven method that induces an implicit activity-dependent time discretization (time-steps automatically increase when the neuron is slowly varying). We show analytically that such a scheme leads to a high-order algorithm so that it accurately approximates the neuronal dynamics. The voltage-stepping method is generic and can be used to simulate any kind of neuron models. We illustrate it on nonlinear integrate-and-fire models and show that it outperforms time-stepping schemes of Runge-Kutta type in terms of simulation time and accuracy.

Applying neural networks as software sensors for enzyme engineering

The on-line control of enzyme-production processes is difficult, owing to the uncertainties typical of biological systems and to the lack of suitable on-line sensors for key process variables. For example, intelligent methods to predict the end point of fermentation could be of great economic value. Computer-assisted control based on artificial-neural-network models offers a novel solution in such situations. Well-trained feedforward-backpropagation neural networks can be used as software sensors in enzyme-process control; their performance can be affected by a number of factors.     

Accuracy evaluation of numerical methods used in state-of-the-art simulators for spiking neural networks

With the various simulators for spiking neural networks developed in recent years, a variety of numerical solution methods for the underlying differential equations are available. In this article, we introduce an approach to systematically assess the accuracy of these methods. In contrast to previous investigations, our approach focuses on a completely deterministic comparison and uses an analytically solved model as a reference. This enables the identification of typical sources of numerical inaccuracies in state-of-the-art simulation methods. In particular, with our approach we can separate the error of the numerical integration from the timing error of spike detection and propagation, the latter being prominent in simulations with fixed timestep. To verify the correctness of the testing procedure, we relate the numerical deviations to theoretical predictions for the employed numerical methods. Finally, we give an example of the influence of simulation artefacts on network behaviour and spike-timing-dependent plasticity (STDP), underlining the importance of spike-time accuracy for the simulation of STDP.     

An STDP training algorithm for a spiking neural network with dynamic threshold neurons

This paper proposes a supervised training algorithm for Spiking Neural Networks (SNNs) which modifies the Spike Timing Dependent Plasticity (STDP)learning rule to support both local and network level training with multiple synaptic connections and axonal delays. The training algorithm applies the rule to two and three layer SNNs, and is benchmarked using the Iris and Wisconsin Breast Cancer (WBC) data sets. The effectiveness of hidden layer dynamic threshold neurons is also investigated and results are presented.     

SPAAN: a software program for prediction of adhesins and adhesin-like proteins using neural networks

MOTIVATION: The adhesion of microbial pathogens to host cells is mediated by adhesins. Experimental methods used for characterizing adhesins are time-consuming and demand large resources. The availability of specialized software can rapidly aid experimenters in simplifying this problem. We have employed 105 compositional properties and artificial neural networks to develop SPAAN, which predicts the probability of a protein being an adhesin (Pad). RESULTS: SPAAN had optimal sensitivity of 89% and specificity of 100% on a defined test set and could identify 97.4% of known adhesins at high Pad value from a wide range of bacteria. Furthermore, SPAAN facilitated improved annotation of several proteins as adhesins. Novel adhesins were identified in 17 pathogenic organisms causing diseases in humans and plants. In the severe acute respiratory syndrome (SARS) associated human corona virus, the spike glycoprotein and nsps (nsp2, nsp5, nsp6 and nsp7) were identified as having adhesin-like characteristics. These results offer new lead for rapid experimental testing. AVAILABILITY: SPAAN is freely available through ftp://203.195.151.45 CONTACT: ramu@igib.res.in.     

An improvement of extreme learning machine for compact single-hidden-layer feedforward neural networks

Recently, a novel learning algorithm called extreme learning machine (ELM) was proposed for efficiently training single-hidden-layer feedforward neural networks (SLFNs). It was much faster than the traditional gradient-descent-based learning algorithms due to the analytical determination of output weights with the random choice of input weights and hidden layer biases. However, this algorithm often requires a large number of hidden units and thus slowly responds to new observations. Evolutionary extreme learning machine (E-ELM) was proposed to overcome this problem; it used the differential evolution algorithm to select the input weights and hidden layer biases. However, this algorithm required much time for searching optimal parameters with iterative processes and was not suitable for data sets with a large number of input features. In this paper, a new approach for training SLFNs is proposed, in which the input weights and biases of hidden units are determined based on a fast regularized least-squares scheme. Experimental results for many real applications with both small and large number of input features show that our proposed approach can achieve good generalization performance with much more compact networks and extremely high speed for both learning and testing.     

An olfactory bulb slice-based biosensor for multi-site extracellular recording of neural networks

Multi-site recording is the important component for studies of the neural networks. In order to investigate the electrophysiological properties of the olfactory bulb neural networks, we developed a novel slice-based biosensor for synchronous measurement with multi-sites. In the present study, the horizontal olfactory bulb slices with legible layered structures were prepared as the sensing element to construct a tissue-based biosensor with the microelectrode array. This olfactory bulb slice-based biosensor was used to simultaneously record the extracellular potentials from multi-positions. Spike detection and cross-correlation analysis were applied to evaluate the electrophysiological activities. The spontaneous potentials as well as the induced responses by glutamic acid took on different electrophysiological characteristics and firing patterns at the different sites of the olfactory bulb slice. This slice-based biosensor can realize multi-site synchronous monitoring and is advantageous for searching after the firing patterns and synaptic connections in the olfactory bulb neural networks. It is also helpful for further probing into olfactory information encoding of the olfactory neural networks.     

An efficient model development strategy for bioprocesses based on neural networks in macroscopic balances

In the serial gray box modeling strategy, generally available knowledge, represented in the macroscopic balance, is combined naturally with neural networks, which are powerful and convenient tools to model the inaccurately known terms in the macroscopic balance. This article shows, for a typical biochemical conversion, that in the serial gray box modeling strategy the identification data only have to cover the input-output space of the inaccurately known term in the macroscopic balances and that the accurately known terms can be used to achieve reliable extrapolation. The strategy is demonstrated successfully on the modeling of the enzymatic (repeated) batch conversion of penicillin G, for which real-time results are presented. Compared with a more data-driven black box strategy, the serial gray box strategy leads to models with reliable extrapolation properties, so that with the same number of identification experiments the model can be applied to a much wider range of different conditions. Compared to a more knowledge-driven white box strategy, the serial gray box model structure is only based on readily available or easily obtainable knowledge, so that the development time of serial gray box models still may be short in a situation where there is no detailed knowledge of the system available. (c) 1997 John Wiley & Sons, Inc. Biotechnol Bioeng 53: 549-566, 1997.     

An improved method for estimating human circadian phase derived from multichannel ambulatory monitoring and artificial neural networks

Recently, we developed a novel method for estimating human circadian phase with noninvasive ambulatory measurements combined with subject-independent multiple regression models and a curve-fitting approach. With this, we were able to estimate circadian phase under real-life conditions with low subject burden, i.e., without need of constant routine (CR) laboratory conditions, and without measuring standard circadian markers, such as core body temperature (CBT) or pineal hormone melatonin rhythms. The precision of ambulatory-derived estimated circadian phase was within an error of 12?±?41?min (mean?±?SD) in comparison to melatonin phase during a CR protocol. The physiological measures could be reduced to a triple combination: skin temperatures, irradiance in the blue spectral band of ambient light, and motion acceleration. Here, we present a nonlinear regression model approach based on artificial neural networks for a larger data set (25 healthy young males), including both the original data and additional data collected in the same protocol and using the same equipment. Throughout our validation study, subjects wore multichannel ambulatory monitoring devices and went about their daily routine for 1 wk. The devices collected a large number of physiological, behavioral, and environmental variables, including CBT, skin temperatures, cardiovascular and respiratory functions, movement/posture, ambient temperature, spectral composition and intensity of light perceived at eye level, and sleep logs. After the ambulatory phase, study volunteers underwent a 32-h CR protocol in the laboratory for measuring unmasked circadian phase (i.e., "midpoint" of the nighttime melatonin rhythm). To overcome the complex masking effects of many different confounding variables during ambulatory measurements, neural network-based nonlinear regression techniques were applied in combination with the cross-validation approach to subject-independent prediction of circadian phase. The most accurate estimate of circadian phase with a prediction error of -3?±?23?min (mean?±?SD) was achieved using only two types of the measured variables: skin temperatures and irradiance for ambient light in the blue spectral band. Compared to our previous linear multiple regression modeling approach, motion acceleration data can be excluded and prediction accuracy, nevertheless, improved. Neural network regression showed statistically significant improvement of variance of prediction error over traditional approaches in determining circadian phase based on single predictors (CBT, motion acceleration, or sleep logs), even though none of these variables was included as predictor. We, therefore, have identified two sets of noninvasive measures that, combined with the prediction model, can provide researchers and clinicians with a precise measure of internal time, in spite of the masking effects of daily behavior. This method, here validated in healthy young men, requires testing in a clinical or shiftwork population suffering from circadian sleep-wake disorders. (Author correspondence: vitaliy.kolodyazhniy@sbg.ac.at ).     

An efficient model development strategy for bioprocesses based on neural networks in macroscopic balances: Part II

There is a need for efficient modeling strategies which quickly lead to reliable mathematical models that can be applied for design and optimization of (bio)-chemical processes. The serial gray box modeling strategy is potentially very efficient because no detailed knowledge is needed to construct the white box part of the model and because covenient black box modeling techniques like neural networks can be used for the black box part of the model. This paper shows for a typical biochemical conversion how the serial gray box modeling strategy can be applied efficiently to obtain a model with good frequency extrapolation properties. Models with good frequency extrapolation properties can be applied under dynamic conditions that were not present during the identification experiments. For a given application domain of a model, this property can be used to considerably reduce the number of identification experiments. The serial gray box modeling strategy is demonstrated to be successful for the modeling of the enzymatic conversion of penicillin G In the concentration range of 10-100 mM and temperature range of 298-335 K. Frequency extrapolation is shown by using only constant temperatures in the (batch) identification experiments, while the model can be used reliable with varying temperatures during the (batch) validation experiments. No reliable frequency extrapolation properties could be obtained for a black box model, and for a more knowledge-driven white box model reliable frequency extrapolation properties could only be obtained by incorporating more knowledge in the model. Copyright 1999 John Wiley & Sons, Inc.     

Applying neural networks to predict HPC-I/O bandwidth over seismic data on lustre file system for ExSeisDat

Cluster Computing - HPC or super-computing clusters are designed for executing computationally intensive operations that typically involve large scale I/O operations. This most commonly involves...     

Analysis of ethanol–glucose mixtures by two microbial sensors: application of chemometrics and artificial neural networks for data processing

Although biosensors based on whole microbial cells have many advantages in terms of convenience, cost and durability, a major limitation of these sensors is often their inability to distinguish between different substrates of interest. This paper demonstrates that it is possible to use sensors entirely based upon whole microbial cells to selectively measure ethanol and glucose in mixtures. Amperometric sensors were constructed using immobilized cells of either Gluconobacter oxydans or Pichia methanolica. The bacterial cells of G. oxydans were sensitive to both substrates, while the yeast cells of P. methanolica oxidized only ethanol. Using chemometric principles of polynomial approximation, data from both of these sensors were processed to provide accurate estimates of glucose and ethanol over a concentration range of 1.0–8.0 mM (coefficients of determination, R²=0.99 for ethanol and 0.98 for glucose). When data were processed using an artificial neural network, glucose and ethanol were accurately estimated over a range of 1.0–10.0 mM (R²=0.99 for both substrates). The described methodology extends the sphere of utility for microbial sensors.     

Assessing the applicability of fuzzy neural networks for habitat preference evaluation of Japanese medaka (Oryzias latipes)

Species–environment relationships are key information for the development of planning and management strategies for conservation or restoration of ecosystems. Artificial neural networks (ANNs) are one widely applied type of species distribution model (SDM). Fuzzy neural networks (FNNs), that is, fuzzified ANNs, have been introduced to take into account the uncertainties inherent in fish behaviour and errors in input data. Despite their high predictive ability in modelling complex systems, FNNs cannot describe habitat preference curves (HPCs), although these are the basis for habitat quality assessment. The present study therefore aimed to evaluate the applicability of FNNs for modelling habitat preference and spatial distributions of Japanese medaka (Oryzias latipes), one of the most common freshwater fish in Japan. Three independent data sets were collected during a series of field surveys and used for model development and evaluation of FNNs. A weight decay backpropagation algorithm was additionally introduced, and its effects on the FNNs were evaluated on the basis of model performance and habitat preference information retrieved from the field observation data. Modified sensitivity analysis was applied to derive HPCs of the target fish. Application of weight decay backpropagation markedly reduced the variability of the model structures, improved the generalization ability of the FNNs, and resulted in well-converged and consistent HPCs that were similar to those evaluated by fuzzy habitat preference models. These results support the applicability of FNNs to habitat preference modelling, which can provide useful information on the habitat use by the target fish. Further study should focus on the effects of sources of uncertainty, such as zero abundance, on the SDMs and the resulting habitat preference evaluation.     

An application of the segregation model for septation of Escherichia coli

A diffusion process moving in an attractive environment is segregated if it reaches one of several excited states. This segregation model is applied to the problem of septation of a mutant Escherichia coli, which shows its interesting character at high temperature (41°C) and after temperature shift down. An Escherichia coli is considered as a distribution of a diffusion process (= a molecule in the Escherichia coli), and an environment potential which produces an attractive centre in the Escherichia coli is assumed.     

An approach towards method development for untargeted urinary metabolite profiling in metabonomic research using UPLC/QToF MS

The application of LC-MS for untargeted urinary metabolite profiling in metabonomic research has gained much interest in recent years. However, the effects of varying sample pre-treatments and LC conditions on generic metabolite profiling have not been studied. We aimed to evaluate the effects of varying experimental conditions on data acquisition in untargeted urinary metabolite profiling using UPLC/QToF MS. In-house QC sample clustering was used to monitor the performance of the analytical platform. In terms of sample pre-treatment, results showed that untreated filtered urine yielded the highest number of features but dilution with methanol provided a more homogenous urinary metabolic profile with less variation in number of features and feature intensities. An increased cycle time with a lower flow rate (400mul/min vs 600mul/min) also resulted in a higher number of features with less variability. The step elution gradient yielded the highest number of features and the best chromatographic resolution among three different elution gradients tested. The maximum retention time and mass shift were only 0.03min and 0.0015Da respectively over 600 injections. The analytical platform also showed excellent robustness as evident by tight QC sample clustering. To conclude, we have investigated LC conditions by studying variability and repeatability of LC-MS data for untargeted urinary metabolite profiling.     

An integrated approach utilizing artificial neural networks and SELDI mass spectrometry for the classification of human tumours and rapid identification of potential biomarkers

MOTIVATION: MALDI mass spectrometry is able to elicit macromolecular expression data from cellular material and when used in conjunction with Ciphergen protein chip technology (also referred to as SELDI-Surface Enhanced Laser Desorption/Ionization), it permits a semi-high throughput approach to be taken with respect to sample processing and data acquisition. Due to the large array of data that is generated from a single analysis (8-10000 variables using a mass range of 2-15 kDa-this paper) it is essential to implement the use of algorithms that can detect expression patterns from such large volumes of data correlating to a given biological/pathological phenotype from multiple samples. If successful, the methodology could be extrapolated to larger data sets to enable the identification of validated biomarkers correlating strongly to disease progression. This would not only serve to enable tumours to be classified according to their molecular expression profile but could also focus attention upon a relatively small number of molecules that might warrant further biochemical/molecular characterization to assess their suitability as potential therapeutic targets. RESULTS: Using a multi-layer perceptron Artificial Neural Network (ANN) (Neuroshell 2) with a back propagation algorithm we have developed a prototype approach that uses a model system (comprising five low and seven high-grade human astrocytomas) to identify mass spectral peaks whose relative intensity values correlate strongly to tumour grade. Analyzing data derived from MALDI mass spectrometry in conjunction with Ciphergen protein chip technology we have used relative importance values, determined from the weights of trained ANNs (Balls et al., Water, Air Soil Pollut., 85, 1467-1472, 1996), to identify masses that accurately predict tumour grade. Implementing a three-stage procedure, we have screened a population of approximately 100000-120000 variables and identified two ions (m/z values of 13454 and 13457) whose relative intensity pattern was significantly reduced in high-grade astrocytoma. The data from this initial study suggests that application of ANN-based approaches can identify molecular ion patterns which strongly associate with disease grade and that its application to larger cohorts of patient material could potentially facilitate the rapid identification of validated biomarkers having significant clinical (i.e. diagnostic/prognostic) potential for the field of cancer biology. AVAILIBILITY: Neuroshell 2 is commercially available from ward systems.