Spike correlations in a songbird agree with a simple markov population model

The relationships between neural activity at the single-cell and the population levels are of central importance for understanding neural codes. In many sensory systems, collective behaviors in large cell groups can be described by pairwise spike correlations. Here, we test whether in a highly specialized premotor system of songbirds, pairwise spike correlations themselves can be seen as a simple corollary of an underlying random process. We test hypotheses on connectivity and network dynamics in the motor pathway of zebra finches using a high-level population model that is independent of detailed single-neuron properties. We assume that neural population activity evolves along a finite set of states during singing, and that during sleep population activity randomly switches back and forth between song states and a single resting state. Individual spike trains are generated by associating with each of the population states a particular firing mode, such as bursting or tonic firing. With an overall modification of one or two simple control parameters, the Markov model is able to reproduce observed firing statistics and spike correlations in different neuron types and behavioral states. Our results suggest that song- and sleep-related firing patterns are identical on short time scales and result from random sampling of a unique underlying theme. The efficiency of our population model may apply also to other neural systems in which population hypotheses can be tested on recordings from small neuron groups.     

Population oscillations in a discrete model of neural networks of the brain

A discrete model of biological neural networks is used to find out how synchronized firing of neurons emerges in a randomly connected neural population. The objective is to understand the mechanisms underlying brain waves and to find and characterize conditions which support spontaneous switching from disordered to rhythmic population activity as in case of an epileptic seizure. The model is kept as simple as possible to achieve on one hand a fast performance of computer simulations of networks with up to 10,000 neurons and to keep on the other hand an overview of parameter dependences. Dynamics of the model can be classified into different regimes: random fluctuations, rhythmic oscillations and silence. When the ratio of the inhibitory/excitatory connectivity is raised the system crosses from the fluctuating regime through the rhythmic oscillating region to the silence regime. Close to the boundary between the fluctuating and the oscillating regimes the network shows spontaneous bursting of high amplitude rhythmic oscillations, which is characteristic of epileptiform behavior. The simulation results are in agreement with recent theories saying that focal epilepsy after injury of the brain could result from axonal sprouting of GABAergic neurons in the injured region.     

Feeding and Swimming Behavior in Grazing Microzooplankton1,2

ABSTRACT A random-walk model of food-searching behavior is considered for the microzooplankton. It is suggested that in still waters a random walk of the conventional sort, modeled by a Wiener process, is less efficient than a Levy walk (a random walk whose excursions follow a Levy distribution) with Levy parameter less than two. For Levy parameter less than one, however, little advantage is gained by further reduction. In turbulent water, on the other hand, dispersion due to a random walk is dominated by the turbulent diffusion of the medium so that the Levy parameter appears to be less important. The effect of chemosensory responses is considered. It is suggested that these are most useful in still water, whereas in turbulent water their value would be less, and a non-specific filtering behavior might be more plausible.     

Learning shapes spontaneous activity itinerating over memorized states

Learning is a process that helps create neural dynamical systems so that an appropriate output pattern is generated for a given input. Often, such a memory is considered to be included in one of the attractors in neural dynamical systems, depending on the initial neural state specified by an input. Neither neural activities observed in the absence of inputs nor changes caused in the neural activity when an input is provided were studied extensively in the past. However, recent experimental studies have reported existence of structured spontaneous neural activity and its changes when an input is provided. With this background, we propose that memory recall occurs when the spontaneous neural activity changes to an appropriate output activity upon the application of an input, and this phenomenon is known as bifurcation in the dynamical systems theory. We introduce a reinforcement-learning-based layered neural network model with two synaptic time scales; in this network, I/O relations are successively memorized when the difference between the time scales is appropriate. After the learning process is complete, the neural dynamics are shaped so that it changes appropriately with each input. As the number of memorized patterns is increased, the generated spontaneous neural activity after learning shows itineration over the previously learned output patterns. This theoretical finding also shows remarkable agreement with recent experimental reports, where spontaneous neural activity in the visual cortex without stimuli itinerate over evoked patterns by previously applied signals. Our results suggest that itinerant spontaneous activity can be a natural outcome of successive learning of several patterns, and it facilitates bifurcation of the network when an input is provided.     

Probabilistic computation by neuromine networks

In this paper, we address the question, can biologically feasible neural nets compute more than can be computed by deterministic polynomial time algorithms? Since we want to maintain a claim of plausibility and reasonableness we restrict ourselves to algorithmically easy to construct nets and we rule out infinite precision in parameters and in any analog parts of the computation. Our approach is to consider the recent advances in randomized algorithms and see if such randomized computations can be described by neural nets. We start with a pair of neurons and show that by connecting them with reciprocal inhibition and some tonic input, then the steady-state will be one neuron ON and one neuron OFF, but which neuron will be ON and which neuron will be OFF will be chosen at random (perhaps, it would be better to say that microscopic noise in the analog computation will be turned into a megascale random bit). We then show that we can build a small network that uses this random bit process to generate repeatedly random bits. This random bit generator can then be connected with a neural net representing the deterministic part of randomized algorithm. We, therefore, demonstrate that these neural nets can carry out probabilistic computation and thus be less limited than classical neural nets.     

Maximum likelihood identification of neural point process systems

Using the theory of random point processes, a method is presented whereby functional relationships between neurons can be detected and modeled. The method is based on a point process characterization involving stochastic intensities and an additive rate function model. Estimates are based on the maximum likelihood (ML) principle and asymptotic properties are examined in the absence of a stationarity assumption. An iterative algorithm that computes the ML estimates is presented. It is based on the expectation/maximization (EM) procedure of Dempster et al. (1977) and makes ML identification accessible to models requiring many parameters. Examples illustrating the use of the method are also presented. These examples are derived from simulations of simple neural systems that cannot be identified using correlation techniques. It is shown that the ML method correctly identifies each of these systems.     

Photosynthesis as a power supply for (bio-)hydrogen production

Although hydrogen is considered to be one of the most promising future energy sources and the technical aspects involved in using it have advanced considerably, the future supply of hydrogen from renewable sources is still unsolved. This review focuses on the production of hydrogen from water using biological catalysts that have been optimized by nature: the process of water-splitting photosynthesis on the one hand and hydrogen production via the catalyst hydrogenase on the other. Using water as a source of electrons and sunlight as a source of energy, both engineered natural systems and biomimetic (bio-inspired) model systems can be designed as first steps towards water-splitting-based hydrogen production (biophotolytic hydrogen production).     

Hand function: peripheral and central constraints on performance.

The hand is one of the most fascinating and sophisticated biological motor systems. The complex biomechanical and neural architecture of the hand poses challenging questions for understanding the control strategies that underlie the coordination of finger movements and forces required for a wide variety of behavioral tasks, ranging from multidigit grasping to the individuated movements of single digits. Hence, a number of experimental approaches, from studies of finger movement kinematics to the recording of electromyographic and cortical activities, have been used to extend our knowledge of neural control of the hand. Experimental evidence indicates that the simultaneous motion and force of the fingers are characterized by coordination patterns that reduce the number of independent degrees of freedom to be controlled. Peripheral and central constraints in the neuromuscular apparatus have been identified that may in part underlie these coordination patterns, simplifying the control of multi-digit grasping while placing certain limitations on individuation of finger movements. We review this evidence, with a particular emphasis on how these constraints extend through the neuromuscular system from the behavioral aspects of finger movements and forces to the control of the hand from the motor cortex.     

A p-Adic model for the process of thinking disturbed by physiological and information noise

We develop a model of the process of thinking in the presence of noise (which is produced by the simultaneous action of a huge number of neurons in the brain as well as by external information and internal cognitive processes). Our model is based on Freud's idea on the splitting of cognitive processes into two (closely connected) domains: consciousness and subconsciousness. We represent the process of thinking as a random dynamical process in a space of ideas endowed with a non-Euclidean geometry (which differs extremely from the ordinary Euclidean geometry of spatial location of neurons in the brain). The so-called p-adic geometry on a space of ideas describes the ability of cognitive systems to form associations. We show that random dynamical thinking systems on a p -adic space of ideas still generate only deterministic ideas. We also study positive and negative effects of noise (in particular, creativeness and stress).     

Bioprocess model building using artificial neural networks

Artificial neural networks are made upon of highly interconnected layers of simple neuron-like nodes. The neurons act as non-linear processing elements within the network. An attractive property of artificial neural networks is that given the appropriate network topology, they are capable of learning and characterising non-linear functional relationships. Furthermore, the structure of the resulting neural network based process model may be considered generic, in the sense that little prior process knowledge is required in its determination. The methodology therefore provides a cost efficient and reliable process modelling technique. One area where such a technique could be useful is biotechnological systems. Here, for example, the use of a process model within an estimation scheme has long been considered an effective means of overcoming inherent on-line measurement problems. However, the development of an accurate process model is extremely time consuming and often results in a model of limited applicability. Artificial neural networks could therefore prove to be a useful model building tool when striving to improve bioprocess operability. Two large scale industrial fermentation systems have been considered as test cases; a fed-batch penicillin fermentation and a continuous mycelial fermentation. Both systems serve to demonstrate the utility, flexibility and potential of the artificial neural network approach to process modelling.     

Protein secondary structure prediction using three neural networks and a segmental semi Markov model

Prediction of protein secondary structure is an important step towards elucidating its three dimensional structure and its function. This is a challenging problem in bioinformatics. Segmental semi Markov models (SSMMs) are one of the best studied methods in this field. However, incorporating evolutionary information to these methods is somewhat difficult. On the other hand, the systems of multiple neural networks (NNs) are powerful tools for multi-class pattern classification which can easily be applied to take these sorts of information into account.To overcome the weakness of SSMMs in prediction, in this work we consider a SSMM as a decision function on outputs of three NNs that uses multiple sequence alignment profiles. We consider four types of observations for outputs of a neural network. Then profile table related to each sequence is reduced to a sequence of four observations. In order to predict secondary structure of each amino acid we need to consider a decision function. We use an SSMM on outputs of three neural networks. The proposed SSMM has discriminative power and weights over different dependency models for outputs of neural networks. The results show that the accuracy of our model in predictions, particularly for strands, is considerably increased.     

Physical models of neural networks

The state of art in computer modelling of neural networks with associative memory is reviewed. The available experimental data are considered on learning and memory of small neural systems, on isolated synapses and on molecular level. Computer simulations demonstrate that realistic models of neural ensembles exhibit properties which can be interpreted as image recognition, categorization, learning, prototype forming, etc. A bilayer model of associative neural network is proposed. One layer corresponds to the short-term memory, the other one to the long-term memory. Patterns are stored in terms of the synaptic strength matrix. We have studied the relaxational dynamics of neurons firing and suppression within the short-term memory layer under the influence of the long-term memory layer. The interaction among the layers has found to create a number of novel stable states which are not the learning patterns. These synthetic patterns may consist of elements belonging to different non-intersecting learning patterns. Within the framework of a hypothesis of selective and definite coding of images in brain one can interpret the observed effect as the "idea? generating" process.     

The formation of cell assemblies

A simple model is presented for the formation of functional groups in a random neural net. They show the following characteristics: 1. They can maintain autonomous activity which might serve as temporary memory traces. 2. Early in the process of formation they become resistant to contraction. 3. Later they become resistant to expansion. 4. Nearby groups inhibit one another. 5. Two groups may contain some cells in common.     

The potential of random forest and neural networks for biomass and recombinant protein modeling in Escherichia coli fed‐batch fermentations

Product quality assurance strategies in production of biopharmaceuticals currently undergo a transformation from empirical “quality by testing” to rational, knowledge‐based “quality by design” approaches. The major challenges in this context are the fragmentary understanding of bioprocesses and the severely limited real‐time access to process variables related to product quality and quantity. Data driven modeling of process variables in combination with model predictive process control concepts represent a potential solution to these problems. The selection of statistical techniques best qualified for bioprocess data analysis and modeling is a key criterion. In this work a series of recombinant Escherichia coli fed‐batch production processes with varying cultivation conditions employing a comprehensive on‐ and offline process monitoring platform was conducted. The applicability of two machine learning methods, random forest and neural networks, for the prediction of cell dry mass and recombinant protein based on online available process parameters and two‐dimensional multi‐wavelength fluorescence spectroscopy is investigated. Models solely based on routinely measured process variables give a satisfying prediction accuracy of about ± 4% for the cell dry mass, while additional spectroscopic information allows for an estimation of the protein concentration within ± 12%. The results clearly argue for a combined approach: neural networks as modeling technique and random forest as variable selection tool.     

Dynamic decorrelation as a unifying principle for explaining a broad range of brightness phenomena

The visual system is highly sensitive to spatial context for encoding luminance patterns. Context sensitivity inspired the proposal of many neural mechanisms for explaining the perception of luminance (brightness). Here we propose a novel computational model for estimating the brightness of many visual illusions. We hypothesize that many aspects of brightness can be explained by a dynamic filtering process that reduces the redundancy in edge representations on the one hand, while non-redundant activity is enhanced on the other. The dynamic filter is learned for each input image and implements context sensitivity. Dynamic filtering is applied to the responses of (model) complex cells in order to build a gain control map. The gain control map then acts on simple cell responses before they are used to create a brightness map via activity propagation. Our approach is successful in predicting many challenging visual illusions, including contrast effects, assimilation, and reverse contrast with the same set of model parameters.     

Simulating a neural cross-talk model for between-hand interference during bimanual circle drawing

Studies on drawing circles with both hands in the horizontal plane have shown that this task is easy to perform across a wide range of movement frequencies under the symmetrical mode of coordination, whereas under the asymmetrical mode (both limbs moving clockwise or counterclockwise) increases in movement frequency have a disruptive effect on trajectory control and hand coordination. To account for these interference effects, we propose a simplified computer model for bimanual circle drawing based on the assumptions that (1) circular trajectories are generated from two orthogonal oscillations coupled with a phase delay, (2) the trajectories are organized on two levels, “intention” and “motor execution”, and (3) the motor systems controlling each hand are prone to neural cross-talk. The neural cross-talk consists in dispatching some fraction of any force command sent to one limb as a mirror image to the other limb. Assuming predominating coupling influences from the dominant to the nondominant limb, the simulations successfully reproduced the main characteristics of performance during asymmetrical bimanual circle drawing with increasing movement frequencies, including disruption of the circular form drawn with the nondominant hand, increasing dephasing of the hand movements, increasing variability of the phase difference, and occasional reversals of the movement direction in the nondominant limb. The implications of these results for current theories of bimanual coordination are discussed. Received: 23 June 1998 / Accepted in revised form: 20 April 1999     

Model of autism: increased ratio of excitation/inhibition in key neural systems

Autism is a severe neurobehavioral syndrome, arising largely as an inherited disorder, which can arise from several diseases. Despite recent advances in identifying some genes that can cause autism, its underlying neurological mechanisms are uncertain. Autism is best conceptualized by considering the neural systems that may be defective in autistic individuals. Recent advances in understanding neural systems that process sensory information, various types of memories and social and emotional behaviors are reviewed and compared with known abnormalities in autism. Then, specific genetic abnormalities that are linked with autism are examined. Synthesis of this information leads to a model that postulates that some forms of autism are caused by an increased ratio of excitation/inhibition in sensory, mnemonic, social and emotional systems. The model further postulates that the increased ratio of excitation/inhibition can be caused by combinatorial effects of genetic and environmental variables that impinge upon a given neural system. Furthermore, the model suggests potential therapeutic interventions.     

Computer simulation of different types of evolutionary process

The computer model of two alternative variants of biological evolution is proposed. The first variant supposes random while the second--directed change of individual features, thus corresponding to the Darwinian and non-Darwinian evolution. The evolution of fish communities in fresh waters serves as a particular example. The model is executed using object-oriented method of programming and mathematical apparatus of fuzzy logics. The investigation of the model showed that process of Darwinian evolution is connected with significantly greater species diversity and variability of evolutionary process trajectories than non-Darwinian one. On the other hand, non-Darwinian type of evolution provides fast achievement of high individual fitness, especially under conditions of constant environment. Non-Darwinian type evolution failed in big evolutionary alteration (for example, transition to predation); while the Darwinian evolution under the same conditions can produce such alterations though it took more time and many extinct species. Phylogenetic tree of Darwinian evolution is always more complex than of non-Darwinian one under the same conditions.