Voltage-stepping schemes for the simulation of spiking neural networks

The numerical simulation of spiking neural networks requires particular attention. On the one hand, time-stepping methods are generic but they are prone to numerical errors and need specific treatments to deal with the discontinuities of integrate-and-fire models. On the other hand, event-driven methods are more precise but they are restricted to a limited class of neuron models. We present here a voltage-stepping scheme that combines the advantages of these two approaches and consists of a discretization of the voltage state-space. The numerical simulation is reduced to a local event-driven method that induces an implicit activity-dependent time discretization (time-steps automatically increase when the neuron is slowly varying). We show analytically that such a scheme leads to a high-order algorithm so that it accurately approximates the neuronal dynamics. The voltage-stepping method is generic and can be used to simulate any kind of neuron models. We illustrate it on nonlinear integrate-and-fire models and show that it outperforms time-stepping schemes of Runge-Kutta type in terms of simulation time and accuracy.

Event-driven simulation of cerebellar granule cells

Around half of the neurons of a human brain are granule cells (approximately 10(11)granule neurons) [Kandel, E.R., Schwartz, J.H., Jessell, T.M., 2000. Principles of Neural Science. McGraw-Hill Professional Publishing, New York]. In order to study in detail the functional role of the intrinsic features of this cell we have developed a pre-compiled behavioural model based on the simplified granule-cell model of Bezzi et al. [Bezzi, M., Nieus, T., Arleo, A., D'Angelo, E., Coenen, O.J.-M.D., 2004. Information transfer at the mossy fiber-granule cell synapse of the cerebellum. 34th Annual Meeting. Society for Neuroscience, San Diego, CA, USA]. We can use an efficient event-driven simulation scheme based on lookup tables (EDLUT) [Ros, E., Carrillo, R.R., Ortigosa, E.M., Barbour, B., Ags, R., 2006. Event-driven simulation scheme for spiking neural networks using lookup tables to characterize neuronal dynamics. Neural Computation 18 (12), 2959-2993]. For this purpose it is necessary to compile into tables the data obtained through a massive numerical calculation of the simplified cell model. This allows network simulations requiring minimal numerical calculation. There are three major features that are considered functionally relevant in the simplified granule cell model: bursting, subthreshold oscillations and resonance. In this work we describe how the cell model is compiled into tables keeping these key properties of the neuron model.     

Simulation of networks of spiking neurons: A review of tools and strategies

We review different aspects of the simulation of spiking neural networks. We start by reviewing the different types of simulation strategies and algorithms that are currently implemented. We next review the precision of those simulation strategies, in particular in cases where plasticity depends on the exact timing of the spikes. We overview different simulators and simulation environments presently available (restricted to those freely available, open source and documented). For each simulation tool, its advantages and pitfalls are reviewed, with an aim to allow the reader to identify which simulator is appropriate for a given task. Finally, we provide a series of benchmark simulations of different types of networks of spiking neurons, including Hodgkin–Huxley type, integrate-and-fire models, interacting with current-based or conductance-based synapses, using clock-driven or event-driven integration strategies. The same set of models are implemented on the different simulators, and the codes are made available. The ultimate goal of this review is to provide a resource to facilitate identifying the appropriate integration strategy and simulation tool to use for a given modeling problem related to spiking neural networks. Action Editor: Barry J. Richmond     

Amino acid encoding schemes from protein structure alignments: multi-dimensional vectors to describe residue types

Bioinformatic software has used various numerical encoding schemes to describe amino acid sequences. Orthogonal encoding, employing 20 numbers to describe the amino acid type of one protein residue, is often used with artificial neural network (ANN) models. However, this can increase the model complexity, thus leading to difficulty in implementation and poor performance. Here, we use ANNs to derive encoding schemes for the amino acid types from protein three-dimensional structure alignments. Each of the 20 amino acid types is characterized with a few real numbers. Our schemes are tested on the simulation of amino acid substitution matrices. These simplified schemes outperform the orthogonal encoding on small data sets. Using one of these encoding schemes, we generate a colouring scheme for the amino acids in which comparable amino acids are in similar colours. We expect it to be useful for visual inspection and manual editing of protein multiple sequence alignments.     

Simple, fast and accurate implementation of the diffusion approximation algorithm for stochastic ion channels with multiple states

BACKGROUND: The phenomena that emerge from the interaction of the stochastic opening and closing of ion channels (channel noise) with the non-linear neural dynamics are essential to our understanding of the operation of the nervous system. The effects that channel noise can have on neural dynamics are generally studied using numerical simulations of stochastic models. Algorithms based on discrete Markov Chains (MC) seem to be the most reliable and trustworthy, but even optimized algorithms come with a non-negligible computational cost. Diffusion Approximation (DA) methods use Stochastic Differential Equations (SDE) to approximate the behavior of a number of MCs, considerably speeding up simulation times. However, model comparisons have suggested that DA methods did not lead to the same results as in MC modeling in terms of channel noise statistics and effects on excitability. Recently, it was shown that the difference arose because MCs were modeled with coupled gating particles, while the DA was modeled using uncoupled gating particles. Implementations of DA with coupled particles, in the context of a specific kinetic scheme, yielded similar results to MC. However, it remained unclear how to generalize these implementations to different kinetic schemes, or whether they were faster than MC algorithms. Additionally, a steady state approximation was used for the stochastic terms, which, as we show here, can introduce significant inaccuracies. MAIN CONTRIBUTIONS: We derived the SDE explicitly for any given ion channel kinetic scheme. The resulting generic equations were surprisingly simple and interpretable--allowing an easy, transparent and efficient DA implementation, avoiding unnecessary approximations. The algorithm was tested in a voltage clamp simulation and in two different current clamp simulations, yielding the same results as MC modeling. Also, the simulation efficiency of this DA method demonstrated considerable superiority over MC methods, except when short time steps or low channel numbers were used.     

Motif-based protein sequence classification using neural networks.

We present a system for multi-class protein classification based on neural networks. The basic issue concerning the construction of neural network systems for protein classification is the sequence encoding scheme that must be used in order to feed the neural network. To deal with this problem we propose a method that maps a protein sequence into a numerical feature space using the matching scores of the sequence to groups of conserved patterns (called motifs) into protein families. We consider two alternative ways for identifying the motifs to be used for feature generation and provide a comparative evaluation of the two schemes. We also evaluate the impact of the incorporation of background features (2-grams) on the performance of the neural system. Experimental results on real datasets indicate that the proposed method is highly efficient and is superior to other well-known methods for protein classification.     

Program Code Generator for Cardiac Electrophysiology Simulation with Automatic PDE Boundary Condition Handling

Clinical and experimental studies involving human hearts can have certain limitations. Methods such as computer simulations can be an important alternative or supplemental tool. Physiological simulation at the tissue or organ level typically involves the handling of partial differential equations (PDEs). Boundary conditions and distributed parameters, such as those used in pharmacokinetics simulation, add to the complexity of the PDE solution. These factors can tailor PDE solutions and their corresponding program code to specific problems. Boundary condition and parameter changes in the customized code are usually prone to errors and time-consuming. We propose a general approach for handling PDEs and boundary conditions in computational models using a replacement scheme for discretization. This study is an extension of a program generator that we introduced in a previous publication. The program generator can generate code for multi-cell simulations of cardiac electrophysiology. Improvements to the system allow it to handle simultaneous equations in the biological function model as well as implicit PDE numerical schemes. The replacement scheme involves substituting all partial differential terms with numerical solution equations. Once the model and boundary equations are discretized with the numerical solution scheme, instances of the equations are generated to undergo dependency analysis. The result of the dependency analysis is then used to generate the program code. The resulting program code are in Java or C programming language. To validate the automatic handling of boundary conditions in the program code generator, we generated simulation code using the FHN, Luo-Rudy 1, and Hund-Rudy cell models and run cell-to-cell coupling and action potential propagation simulations. One of the simulations is based on a published experiment and simulation results are compared with the experimental data. We conclude that the proposed program code generator can be used to generate code for physiological simulations and provides a tool for studying cardiac electrophysiology.     

Accuracy evaluation of numerical methods used in state-of-the-art simulators for spiking neural networks

With the various simulators for spiking neural networks developed in recent years, a variety of numerical solution methods for the underlying differential equations are available. In this article, we introduce an approach to systematically assess the accuracy of these methods. In contrast to previous investigations, our approach focuses on a completely deterministic comparison and uses an analytically solved model as a reference. This enables the identification of typical sources of numerical inaccuracies in state-of-the-art simulation methods. In particular, with our approach we can separate the error of the numerical integration from the timing error of spike detection and propagation, the latter being prominent in simulations with fixed timestep. To verify the correctness of the testing procedure, we relate the numerical deviations to theoretical predictions for the employed numerical methods. Finally, we give an example of the influence of simulation artefacts on network behaviour and spike-timing-dependent plasticity (STDP), underlining the importance of spike-time accuracy for the simulation of STDP.     

Encoder-Decoder Optimization for Brain-Computer Interfaces

Neuroprosthetic brain-computer interfaces are systems that decode neural activity into useful control signals for effectors, such as a cursor on a computer screen. It has long been recognized that both the user and decoding system can adapt to increase the accuracy of the end effector. Co-adaptation is the process whereby a user learns to control the system in conjunction with the decoder adapting to learn the user''s neural patterns. We provide a mathematical framework for co-adaptation and relate co-adaptation to the joint optimization of the user''s control scheme ("encoding model") and the decoding algorithm''s parameters. When the assumptions of that framework are respected, co-adaptation cannot yield better performance than that obtainable by an optimal initial choice of fixed decoder, coupled with optimal user learning. For a specific case, we provide numerical methods to obtain such an optimized decoder. We demonstrate our approach in a model brain-computer interface system using an online prosthesis simulator, a simple human-in-the-loop pyschophysics setup which provides a non-invasive simulation of the BCI setting. These experiments support two claims: that users can learn encoders matched to fixed, optimal decoders and that, once learned, our approach yields expected performance advantages.     

基于神经网络的复介电常数重建算法

提出将一种神经网络的方法应用于微波断层成象,重建复的介电常数图象。微波断层成象发展了二十多年,仍然处于实验阶段,这是因为必须求解逆散射问题,它具有非适应性和非线性。为了改善其非适应性及非线性,引入适当的正则化过程并且应用神经网络的方法来求解这个问题。数值模拟实例证明了此方法的有效性。     

Computing flow pipe of embedded hybrid systems using deep group preserving scheme

In this paper, we propose a novel methodology of numerical approximation to analyze flow of a nonlinear embedded hybrid system. For proving that all trajectories of a hybrid system do not enter an unsafe region, many classic numerical approaches such as Euler, Runge–Kutta methods for ordinary differential equations (ODEs) are applied, whereas, there exist several defects, including so-called spurious solutions and ghost fixed points. Moreover, to approximate the proper solution as much as possible, step size selection becomes especially important. In comparison, integrating group preserving scheme (GPS) which calculates true circumstance getting rid of spurious solutions and ghost fixed points, with neural network model which reduces numerical errors, deep GPS (DGPS) eliminates aforementioned adverse factors and gains better numerical approximation using a large time step size. The experimental results show that the proposed method makes safety verification for an embedded hybrid system well.

     

Evolutionary games on networks and payoff invariance under replicator dynamics

The commonly used accumulated payoff scheme is not invariant with respect to shifts of payoff values when applied locally in degree-inhomogeneous population structures. We propose a suitably modified payoff scheme and we show both formally and by numerical simulation, that it leaves the replicator dynamics invariant with respect to affine transformations of the game payoff matrix. We then show empirically that, using the modified payoff scheme, an interesting amount of cooperation can be reached in three paradigmatic non-cooperative two-person games in populations that are structured according to graphs that have a marked degree inhomogeneity, similar to actual graphs found in society. The three games are the Prisoner’s Dilemma, the Hawks-Doves and the Stag-Hunt. This confirms previous important observations that, under certain conditions, cooperation may emerge in such network-structured populations, even though standard replicator dynamics for mixing populations prescribes equilibria in which cooperation is totally absent in the Prisoner’s Dilemma, and it is less widespread in the other two games.     

Multiple pulse interactions and averaging in systems of coupled neural oscillators

Oscillators coupled strongly are capable of complicated behavior which may be pathological for biological control systems. Nevertheless, strong coupling may be needed to prevent asynchrony. We discuss how some neural networks may be designed to achieve only simple locking behavior when the coupling is strong. The design is based on the fact that the method of averaging produces equations that are capable only of locking or drift, not pathological complexity. Furthermore, it is shown that oscillators that interact by means of multiple pulses per cycle, dispersed around the cycle, behave like averaged equations, even if the number of pulses is small. We discuss the biological intuition behind this scheme, and show numerically that it works when the oscillators are taken to be composites, each unit of which is governed by a well-known model of a neural oscillator. Finally, we describe numerical methods for computing from equations for coupled limit cycle oscillators the averaged coupling functions of our theory.Research partially supported by the National Science Foundation under grants DMS 8796235 and DMS 8701405 and the Air Force Office of Scientific Research under University Research Contract F 49620-C-0131 to Northeastern University     

Mass-weighted molecular dynamics simulation and conformational analysis of polypeptide.

Atomic motions in protein molecules have been studied by molecular dynamics (MD) simulations; dynamics simulation methods have also been employed in conformational studies of polypeptide molecules. It was found that when atomic masses are weighted, the molecular dynamics method can significantly increase the sampling of dihedral conformation space in such studies, compared to a conventional MD simulation of the same total simulation time length. Herein the theoretical study of molecular conformation sampling by the molecular dynamics-based simulation method in which atomic masses are weighted is reported in detail; moreover, a numerical scheme for analyzing the extensive conformational sampling in the simulation of a tetrapeptide amide molecule is presented. From numerical analyses of the mass-weighted molecular dynamics trajectories of backbone dihedral angles, low-resolution structures covering the entire backbone dihedral conformation space of the molecule were determined, and the distribution of rotationally stable conformations in this space were analyzed quantitatively. The theoretical analyses based on the computer simulation and numerical analytical methods suggest that distinctive regimes in the conformational space of the peptide molecule can be identified.     

Mean square stability of uncertain stochastic BAM neural networks with interval time-varying delays

The robust asymptotic stability analysis for uncertain BAM neural networks with both interval time-varying delays and stochastic disturbances is considered. By using the stochastic analysis approach, employing some free-weighting matrices and introducing an appropriate type of Lyapunov functional which takes into account the ranges for delays, some new stability criteria are established to guarantee the delayed BAM neural networks to be robustly asymptotically stable in the mean square. Unlike the most existing mean square stability conditions for BAM neural networks, the supplementary requirements that the time derivatives of time-varying delays must be smaller than 1 are released and the lower bounds of time varying delays are not restricted to be 0. Furthermore, in the proposed scheme, the stability conditions are delay-range-dependent and rate-dependent/independent. As a result, the new criteria are applicable to both fast and slow time-varying delays. Three numerical examples are given to illustrate the effectiveness of the proposed criteria.     

Robust exponential stabilization of a class of delayed neural networks with reaction-diffusion terms

In this paper, the problem of global robust exponential stabilization for a class of neural networks with reaction-diffusion terms and time-varying delays which covers the Hopfield neural networks and cellular neural networks is investigated. A feedback control gain matrix is derived to achieve the global robust exponential stabilization of the neural networks by using the Lyapunov stability theory, and the stabilization condition can be verified if a certain Hamiltonian matrix with no eigenvalues on the imaginary axis. This condition can avoid solving an algebraic Riccati equation. Finally, a numerical simulation illustrates the effectiveness of the results.     

Backpropagation algorithm adaptation parameters using learning automata

Despite of the many successful applications of backpropagation for training multi-layer neural networks, it has many drawbocks. For complex problems it may require a long time to train the networks, and it may not train at all. Long training time can be the result of the non-optimal parameters. It is not easy to choose appropriate value of the parameters for a particular problem. In this paper, by interconnection of fixed structure learning automata (FSLA) to the feedforward neural networks, we apply learning automata (LA) scheme for adjusting these parameters based on the observation of random response of neural networks. The main motivation in using learning automata as an adaptation algorithm is to use its capability of global optimization when dealing with multi-modal surface. The feasibility of proposed method is shown through simulations on three learning problems: exclusive-or, encoding problem, and digit recognition. The simulation results show that the adaptation of these parameters using this method not only increases the convergence rate of learning but it increases the likelihood of escaping from the local minima.     

Topography simulation for structural analysis using cell advancing method

In this paper, we propose a novel method for topography simulation in micro-electronic processes such as deposition and etching processes. The proposed scheme comprises of calculating the forward and backward movement of the surface and converting the cell structure into a tetrahedral mesh structure with topological information. Memory requirements are mitigated through a dynamic allocating scheme, which takes only the surface cells under consideration. For the removal of cells, a fixed time step is employed while volume remains in the surface cells. A spillover algorithm has also been devised in order to consider the case when more volume has to be removed from a cell during a single time step. Our proposed scheme was applied to the cases such as the construction of a TFT-LCD structure, ROM and a DRAM cell. A numerical simulator was interfaced with the topography simulator in order to investigate the successful meshing operation form the cell structure. For exemplary application, parasitic capacitances were extracted from a test wafer structure having 4 metal lines embedded in two types of non-planar dielectric layer. The simulation result exhibited about maximum 8% error, which seems to be relatively small in comparison to the planar dielectric layer.     

Studies in RF Power Communication,SAR, and Temperature Elevation in Wireless Implantable Neural Interfaces

Implantable neural interfaces are designed to provide a high spatial and temporal precision control signal implementing high degree of freedom real-time prosthetic systems. The development of a Radio Frequency (RF) wireless neural interface has the potential to expand the number of applications as well as extend the robustness and longevity compared to wired neural interfaces. However, it is well known that RF signal is absorbed by the body and can result in tissue heating. In this work, numerical studies with analytical validations are performed to provide an assessment of power, heating and specific absorption rate (SAR) associated with the wireless RF transmitting within the human head. The receiving antenna on the neural interface is designed with different geometries and modeled at a range of implanted depths within the brain in order to estimate the maximum receiving power without violating SAR and tissue temperature elevation safety regulations. Based on the size of the designed antenna, sets of frequencies between 1 GHz to 4 GHz have been investigated. As expected the simulations demonstrate that longer receiving antennas (dipole) and lower working frequencies result in greater power availability prior to violating SAR regulations. For a 15 mm dipole antenna operating at 1.24 GHz on the surface of the brain, 730 uW of power could be harvested at the Federal Communications Commission (FCC) SAR violation limit. At approximately 5 cm inside the head, this same antenna would receive 190 uW of power prior to violating SAR regulations. Finally, the 3-D bio-heat simulation results show that for all evaluated antennas and frequency combinations we reach FCC SAR limits well before 1 °C. It is clear that powering neural interfaces via RF is possible, but ultra-low power circuit designs combined with advanced simulation will be required to develop a functional antenna that meets all system requirements.     

New stability criterion of neural networks with leakage delays and impulses: a piecewise delay method

This paper analyzes the global asymptotic stability of a class of neural networks with time delay in the leakage term and time-varying delays under impulsive perturbations. Here the time-varying delays are assumed to be piecewise. In this method, the interval of the variation is divided into two subintervals by its central point. By developing a new Lyapunov–Krasovskii functional and checking its variation in between the two subintervals, respectively, and then we present some sufficient conditions to guarantee the global asymptotic stability of the equilibrium point for the considered neural network. The proposed results which do not require the boundedness, differentiability and monotonicity of the activation functions, can be easily verified via the linear matrix inequality (LMI) control toolbox in MATLAB. Finally, a numerical example and its simulation are given to show the conditions obtained are new and less conservative than some existing ones in the literature.