Fully implicit parallel simulation of single neurons

When a multi-compartment neuron is divided into subtrees such that no subtree has more than two connection points to other subtrees, the subtrees can be on different processors and the entire system remains amenable to direct Gaussian elimination with only a modest increase in complexity. Accuracy is the same as with standard Gaussian elimination on a single processor. It is often feasible to divide a 3-D reconstructed neuron model onto a dozen or so processors and experience almost linear speedup. We have also used the method for purposes of load balance in network simulations when some cells are so large that their individual computation time is much longer than the average processor computation time or when there are many more processors than cells. The method is available in the standard distribution of the NEURON simulation program.     

Neuron splitting in compute-bound parallel network simulations enables runtime scaling with twice as many processors

Neuron tree topology equations can be split into two subtrees and solved on different processors with no change in accuracy, stability, or computational effort; communication costs involve only sending and receiving two double precision values by each subtree at each time step. Splitting cells is useful in attaining load balance in neural network simulations, especially when there is a wide range of cell sizes and the number of cells is about the same as the number of processors. For compute-bound simulations load balance results in almost ideal runtime scaling. Application of the cell splitting method to two published network models exhibits good runtime scaling on twice as many processors as could be effectively used with whole-cell balancing.     

Parallel Processing of Divisible Loads on Partitionable Static Interconnection Networks

We analyze the parallel time and speedup for processing a divisible load on (1) a linear array with a corner initial processor; (2) a linear array with an interior initial processor; (3) a mesh with a corner initial processor; (4) a mesh with an interior initial processor; (5) a b-ary complete tree with the root as the initial processor; (6) a pyramid with the apex as the initial processor. Due to communication overhead and limited network connectivity, the speedup of parallel processing for a divisible load on static interconnection networks with constant node degrees is bounded from above by a quantity independent of network size. It is shown that for the above six cases, as the network size becomes large, the asymptotic speedup is approximately , 2 , ^3/4, 4^3/4, (b–1), and 3, respectively, where is the ratio of the time for computing a unit load to the time for communicating a unit load. We also investigate divisible load distribution on hypercubes. Our strategy takes advantage of the recursive structure of a hypercube. It is proven that linear speedup can be achieved as the communication cost becomes smaller and smaller.     

Timing in the absence of clocks: encoding time in neural network states

Decisions based on the timing of sensory events are fundamental to sensory processing. However, the mechanisms by which the brain measures time over ranges of milliseconds to seconds remain unclear. The dominant model of temporal processing proposes that an oscillator emits events that are integrated to provide a linear metric of time. We examine an alternate model in which cortical networks are inherently able to tell time as a result of time-dependent changes in network state. Using computer simulations we show that within this framework, there is no linear metric of time, and that a given interval is encoded in the context of preceding events. Human psychophysical studies were used to examine the predictions of the model. Our results provide theoretical and experimental evidence that, for short intervals, there is no linear metric of time, and that time may be encoded in the high-dimensional state of local neural networks.     

Block‐cyclic redistribution over heterogeneous networks

Clusters of workstations and networked parallel computing systems are emerging as promising computational platforms for HPC applications. The processors in such systems are typically interconnected by a collection of heterogeneous networks such as Ethernet, ATM, and FDDI, among others. In this paper, we develop techniques to perform block-cyclic redistribution over P processors interconnected by such a collection of heterogeneous networks. We represent the communication scheduling problem using a timing diagram formalism. Here, each interprocessor communication event is represented by a rectangle whose height denotes the time to perform this event over the heterogeneous network. The communication scheduling problem is then one of appropriately positioning the rectangles so as to minimize the completion time of all the communication events. For the important case where the block size changes by a factor of K, we develop a heuristic algorithm whose completion time is at most twice the optimal. The running time of the heuristic is O(PK ²). Our heuristic algorithm is adaptive to variations in network performance, and derives schedules at run-time, based on current information about the available network bandwidth. Our experimental results show that our schedules always have communication times that are very close to optimal. This revised version was published online in July 2006 with corrections to the Cover Date.     

Efficient and Accurate Time-Stepping Schemes for Integrate-and-Fire Neuronal Networks

To avoid the numerical errors associated with resetting the potential following a spike in simulations of integrate-and-fire neuronal networks, Hansel et al. and Shelley independently developed a modified time-stepping method. Their particular scheme consists of second-order Runge-Kutta time-stepping, a linear interpolant to find spike times, and a recalibration of postspike potential using the spike times. Here we show analytically that such a scheme is second order, discuss the conditions under which efficient, higher-order algorithms can be constructed to treat resets, and develop a modified fourth-order scheme. To support our analysis, we simulate a system of integrate-and-fire conductance-based point neurons with all-to-all coupling. For six-digit accuracy, our modified Runge-Kutta fourth-order scheme needs a time-step of t = 0.5 × 10^–3 seconds, whereas to achieve comparable accuracy using a recalibrated second-order or a first-order algorithm requires time-steps of 10^–5 seconds or 10^–9 seconds, respectively. Furthermore, since the cortico-cortical conductances in standard integrate-and-fire neuronal networks do not depend on the value of the membrane potential, we can attain fourth-order accuracy with computational costs normally associated with second-order schemes.     

A Novel Optimal Load Distribution Algorithm for Divisible Loads

A new model for divisible load problem is introduced. Its characteristics are analyzed. Optimal load distribution algorithms on the new model are presented for the tree-network and linear network. Applications that fit our model are briefly described. We show that our model outperforms the existing model such as Cheng–Robertazzi model. We show that the linear model is equivalent to a single-level tree network if the intermediate processors do not follow the store-and-forward communication model, but they follow the store-and-bypass model. This paper introduces the concept of store-and-bypass for divisible load theory.     

Systolic Loop Methods for Molecular Dynamics Simulation,Generalised for Macromolecules

Abstract

Systolic loop programs have been shown to be very efficient for molecular dynamics simulations of liquid systems on distributed memory parallel computers. The original methods address the case where the number of molecules simulated exceeds the number of processors used. Simulations of large flexible molecules often do not meet this condition, requiring the three- and four-body terms used to model chemical bonds within a molecule to be distributed over several processors. This paper discusses how the systolic loop methods may be generalised to accommodate such systems, and describes the implementation of a computer program for simulation of protein dynamics. Performance figures are given for this program running typical simulations on a Meiko Computing Surface using different number of processors.     

Improved multi-scale modeling of uterine electrical activity

ObjectiveWe aim to simulate the uterine electrical activity at the myometrium level and at the skin level where it can be recorded non-invasively.Material and methodsWe use 2D models both at the myometrium scale and for the volume conductor. The multi-scale model has been implemented in Python, as a complete package including the needed tools. To speedup the slowest step, we integrated parallel execution.ResultsWe obtain realistic simulations of EHG signals as recorded by an electrode array placed on the woman abdomen.ConclusionThese simulations are still generic, the next problem to address will be the identification of the model's parameters to obtain patient-specific simulations.     

Link contention-constrained scheduling and mapping of tasks and messages to a network of heterogeneous processors

In this paper, we consider the problem of scheduling and mapping precedence-constrained tasks to a network of heterogeneous processors. In such systems, processors are usually physically distributed, implying that the communication cost is considerably higher than in tightly coupled multiprocessors. Therefore, scheduling and mapping algorithms for such systems must schedule the tasks as well as the communication traffic by treating both the processors and communication links as equally important resources. We propose an algorithm that achieves these objectives and adapts its task scheduling and mapping decisions according to the given network topology. Just like tasks, messages are also scheduled and mapped to suitable links during the minimization of the finish times of tasks. Heterogeneity of processors is exploited by scheduling critical tasks to the fastest processors. Our experimental study has demonstrated that the proposed algorithm is efficient and robust, and yields consistent performance over a wide range of scheduling parameters. This revised version was published online in July 2006 with corrections to the Cover Date.     

Optimization and comparison of two methods for spike train estimation in an unfused tetanic contraction of low threshold motor units

BackgroundRecent findings have shown that imaging voluntarily activated motor units (MUs) by decomposing ultrasound-based displacement images provides estimates of unfused tetanic signals evoked by spinal motoneurons’ neural discharges (spikes). Two methods have been suggested to estimate its spike trains: band-pass filter (BPM) and Haar wavelet transform (HWM). However, the methods’ optimal parameters and which method performs the best are unknown. This study will answer these questions.MethodHWM and BPM were optimized using simulations. Their performance was evaluated based on simulations and 21 experimental datasets, considering their rate of agreement, spike offset, and spike offset variability to the simulated or experimental spikes.ResultsA range of parameter sets that resulted in the highest possible agreement with simulated spikes was provided. Both methods highly agreed with simulated and experimental spikes, but HWM was a better spike estimation method than BPM because it had a higher agreement, less bias, and less variation (p < 0.001).ConclusionsThe optimized HWM will be an important contributor to further developing the identification and analysis of MUs using imaging, providing indirect access to the neural drive of the spinal cord to the muscle by the unfused tetanic signals.     

Multiply accelerated ReaxFF molecular dynamics: coupling parallel replica dynamics with collective variable hyper dynamics

ABSTRACT

To tackle the time scales required to study complex chemical reactions, methods performing accelerated molecular dynamics are necessary even with the recent advancement in high-performance computing. A number of different acceleration techniques are available. Here we explore potential synergies between two popular acceleration methods – Parallel Replica Dynamics (PRD) and Collective Variable Hyperdynamics (CVHD), by analysing the speedup obtained for the pyrolysis of n-dodecane. We observe that PRD?+?CVHD provides additional speedup to CVHD by reaching the required time scales for the reaction at an earlier wall-clock time. Although some speedup is obtained with the additional replicas, we found that the effectiveness of the inclusion of PRD is depreciated for systems where there is a dramatic increase in reaction rates induced by CVHD. Similar observations were made in the simulation of ethylene-carbonate/Li system, which is inherently more reactive than pyrolysis, indicate that the speedup obtained via the combination of the two acceleration methods can be generalised to most practical chemical systems.     

The Mechanism of Discharge Pattern Formation in Crayfish Interneurons

Excitatory and inhibitory processes which result in the generation of output impulses were analyzed in single crayfish interneurons by using intracellular recording and membrane polarizing techniques. Individual spikes which are initiated orthodromically in axon branches summate temporally and spatially to generate a main axon spike; temporally dispersed branch spikes often pace repetitive discharge of the main axon. Hyperpolarizing IPSP's sometimes suppress axonal discharge to most of these inputs, but in other cases may interact selectively with some of them. The IPSP's reverse their polarity at a hyperpolarized level of membrane potential; they sometimes exhibit two discrete time courses indicating two different input sources. Outward direct current at the main axon near branches causes repetitive discharges which may last, with optimal current intensities, for 1 to 15 seconds. The relation of discharge frequency to current intensity is linear for an early spike interval, but above 100 to 200 impulses/sec. it begins to show saturation. In one unit the current-frequency curve exhibited two linear portions, suggesting the presence of two spike-generating sites in the axon. Current threshold measurements, using test stimuli of different durations, showed that both accommodation and "early" or "residual" refractoriness contribute to the determination of discharge rate at different frequencies.     

A Multistage Load Distribution Strategy for Three-Dimensional Meshes

We study the problem of scheduling a divisible load in a three-dimensional mesh of processors. The objective is to find partition of a load into shares and distribution of load shares among processors which minimize load processing time subject to communication delays involved in sending load from one processor to another. We propose a new scheduling algorithm which distributes load in a sequence of stages across the network, each stage brings load to a set of processors located at the same distance from the load source. A key feature of our solution is that sets of processors receive load in the order of decreasing processing capacities. We call this scheduling strategy Largest Layer First. A theorem about the processing time attained by the algorithm is stated. Performance of the algorithm is compared to earlier results.     

Bursting Reverberation as a Multiscale Neuronal Network Process Driven by Synaptic Depression-Facilitation

Neuronal networks can generate complex patterns of activity that depend on membrane properties of individual neurons as well as on functional synapses. To decipher the impact of synaptic properties and connectivity on neuronal network behavior, we investigate the responses of neuronal ensembles from small (5–30 cells in a restricted sphere) and large (acute hippocampal slice) networks to single electrical stimulation: in both cases, a single stimulus generated a synchronous long-lasting bursting activity. While an initial spike triggered a reverberating network activity that lasted 2–5 seconds for small networks, we found here that it lasted only up to 300 milliseconds in slices. To explain this phenomena present at different scales, we generalize the depression-facilitation model and extracted the network time constants. The model predicts that the reverberation time has a bell shaped relation with the synaptic density, revealing that the bursting time cannot exceed a maximum value. Furthermore, before reaching its maximum, the reverberation time increases sub-linearly with the synaptic density of the network. We conclude that synaptic dynamics and connectivity shape the mean burst duration, a property present at various scales of the networks. Thus bursting reverberation is a property of sufficiently connected neural networks, and can be generated by collective depression and facilitation of underlying functional synapses.     

Exact digital simulation of time-invariant linear systems with applications to neuronal modeling

An efficient new method for the exact digital simulation of time-invariant linear systems is presented. Such systems are frequently encountered as models for neuronal systems, or as submodules of such systems. The matrix exponential is used to construct a matrix iteration, which propagates the dynamic state of the system step by step on a regular time grid. A large and general class of dynamic inputs to the system, including trains of δ-pulses, can be incorporated into the exact simulation scheme. An extension of the proposed scheme presents an attractive alternative for the approximate simulation of networks of integrate-and-fire neurons with linear sub-threshold integration and non-linear spike generation. The performance of the proposed method is analyzed in comparison with a number of multi-purpose solvers. In simulations of integrate-and-fire neurons, Exact Integration systematically generates the smallest error with respect to both sub-threshold dynamics and spike timing. For the simulation of systems where precise spike timing is important, this results in a practical advantage in particular at moderate integration step sizes. Received: 3 October 1998 / Accepted in revised form: 19 March 1999     

Variability v.s. synchronicity of neuronal activity in local cortical network models with different wiring topologies

Dynamical behavior of a biological neuronal network depends significantly on the spatial pattern of synaptic connections among neurons. While neuronal network dynamics has extensively been studied with simple wiring patterns, such as all-to-all or random synaptic connections, not much is known about the activity of networks with more complicated wiring topologies. Here, we examined how different wiring topologies may influence the response properties of neuronal networks, paying attention to irregular spike firing, which is known as a characteristic of in vivo cortical neurons, and spike synchronicity. We constructed a recurrent network model of realistic neurons and systematically rewired the recurrent synapses to change the network topology, from a localized regular and a “small-world” network topology to a distributed random network topology. Regular and small-world wiring patterns greatly increased the irregularity or the coefficient of variation (Cv) of output spike trains, whereas such an increase was small in random connectivity patterns. For given strength of recurrent synapses, the firing irregularity exhibited monotonous decreases from the regular to the random network topology. By contrast, the spike coherence between an arbitrary neuron pair exhibited a non-monotonous dependence on the topological wiring pattern. More precisely, the wiring pattern to maximize the spike coherence varied with the strength of recurrent synapses. In a certain range of the synaptic strength, the spike coherence was maximal in the small-world network topology, and the long-range connections introduced in this wiring changed the dependence of spike synchrony on the synaptic strength moderately. However, the effects of this network topology were not really special in other properties of network activity. Action Editor: Xiao-Jing Wang     

Improved backpropagation learning in neural networks with windowed momentum

Backpropagation, which is frequently used in Neural Network training, often takes a great deal of time to converge on an acceptable solution. Momentum is a standard technique that is used to speed up convergence and maintain generalization performance. In this paper we present the Windowed Momentum algorithm, which increases speedup over Standard Momentum. Windowed Momentum is designed to use a fixed width history of recent weight updates for each connection in a neural network. By using this additional information, Windowed Momentum gives significant speedup over a set of applications with same or improved accuracy. Windowed Momentum achieved an average speedup of 32% in convergence time on 15 data sets, including a large OCR data set with over 500,000 samples. In addition to this speedup, we present the consequences of sample presentation order. We show that Windowed Momentum is able to overcome these effects that can occur with poor presentation order and still maintain its speedup advantages.     

Partial correlation analysis for the identification of synaptic connections

In this paper, we investigate the use of partial correlation analysis for the identification of functional neural connectivity from simultaneously recorded neural spike trains. Partial correlation analysis allows one to distinguish between direct and indirect connectivities by removing the portion of the relationship between two neural spike trains that can be attributed to linear relationships with recorded spike trains from other neurons. As an alternative to the common frequency domain approach based on the partial spectral coherence we propose a new statistic in the time domain. The new scaled partial covariance density provides additional information on the direction and the type, excitatory or inhibitory, of the connectivities. In simulation studies, we investigated the power and limitations of the new statistic. The simulations show that the detectability of various connectivity patterns depends on various parameters such as connectivity strength and background activity. In particular, the detectability decreases with the number of neurons included in the analysis and increases with the recording time. Further, we show that the method can also be used to detect multiple direct connectivities between two neurons. Finally, the methods of this paper are illustrated by an application to neurophysiological data from spinal dorsal horn neurons.     

ASSET: Analysis of Sequences of Synchronous Events in Massively Parallel Spike Trains

With the ability to observe the activity from large numbers of neurons simultaneously using modern recording technologies, the chance to identify sub-networks involved in coordinated processing increases. Sequences of synchronous spike events (SSEs) constitute one type of such coordinated spiking that propagates activity in a temporally precise manner. The synfire chain was proposed as one potential model for such network processing. Previous work introduced a method for visualization of SSEs in massively parallel spike trains, based on an intersection matrix that contains in each entry the degree of overlap of active neurons in two corresponding time bins. Repeated SSEs are reflected in the matrix as diagonal structures of high overlap values. The method as such, however, leaves the task of identifying these diagonal structures to visual inspection rather than to a quantitative analysis. Here we present ASSET (Analysis of Sequences of Synchronous EvenTs), an improved, fully automated method which determines diagonal structures in the intersection matrix by a robust mathematical procedure. The method consists of a sequence of steps that i) assess which entries in the matrix potentially belong to a diagonal structure, ii) cluster these entries into individual diagonal structures and iii) determine the neurons composing the associated SSEs. We employ parallel point processes generated by stochastic simulations as test data to demonstrate the performance of the method under a wide range of realistic scenarios, including different types of non-stationarity of the spiking activity and different correlation structures. Finally, the ability of the method to discover SSEs is demonstrated on complex data from large network simulations with embedded synfire chains. Thus, ASSET represents an effective and efficient tool to analyze massively parallel spike data for temporal sequences of synchronous activity.