Formation of a direction map by projection learning using Kohonen's self-organization map

 In this paper, we propose a modification of Kohonen's self-organization map (SOM) algorithm. When the input signal space is not convex, some reference vectors of SOM can protrude from it. The input signal space must be convex to keep all the reference vectors fixed on it for any updates. Thus, we introduce a projection learning method that fixes the reference vectors onto the input signal space. This version of SOM can be applied to a non-convex input signal space. We applied SOM with projection learning to a direction map observed in the primary visual cortex of area 17 of ferrets, and area 18 of cats. Neurons in those areas responded selectively to the orientation of edges or line segments, and their directions of motion. Some iso-orientation domains were subdivided into selective regions for the opposite direction of motion. The abstract input signal space of the direction map described in the manner proposed by Obermayer and Blasdel [(1993) J Neurosci 13: 4114–4129] is not convex. We successfully used SOM with projection learning to reproduce a direction-orientation joint map. Received: 29 September 2000 / Accepted: 7 March 2001     

Ordering process of self-organizing maps improved by asymmetric neighborhood function

The Self-organizing map (SOM) is an unsupervised learning method based on the neural computation, which has found wide applications. However, the learning process sometime takes multi-stable states, within which the map is trapped to an undesirable disordered state including topological defects on the map. These topological defects critically aggravate the performance of the SOM. In order to overcome this problem, we propose to introduce an asymmetric neighborhood function for the SOM algorithm. Compared with the conventional symmetric one, the asymmetric neighborhood function accelerates the ordering process even in the presence of the defect. However, this asymmetry tends to generate a distorted map. This can be suppressed by an improved method of the asymmetric neighborhood function. In the case of one-dimensional SOM, it is found that the required steps for perfect ordering is numerically shown to be reduced from O(N ³) to O(N ²). We also discuss the ordering process of a twisted state in two-dimensional SOM, which can not be rectified by the ordinary symmetric neighborhood function.     

An SOM-based algorithm for optimization with dynamic weight updating

The self-organizing map (SOM), as a kind of unsupervised neural network, has been used for both static data management and dynamic data analysis. To further exploit its search abilities, in this paper we propose an SOM-based algorithm (SOMS) for optimization problems involving both static and dynamic functions. Furthermore, a new SOM weight updating rule is proposed to enhance the learning efficiency; this may dynamically adjust the neighborhood function for the SOM in learning system parameters. As a demonstration, the proposed SOMS is applied to function optimization and also dynamic trajectory prediction, and its performance compared with that of the genetic algorithm (GA) due to the similar ways both methods conduct searches.     

A SOM projection technique with the growing structure for visualizing high-dimensional data

The Self-Organizing Map (SOM) is an efficient tool for visualizing high-dimensional data. In this paper, an intuitive and effective SOM projection method is proposed for mapping high-dimensional data onto the two-dimensional grid structure with a growing self-organizing mechanism. In the learning phase, a growing SOM is trained and the growing cell structure is used as the baseline framework. In the ordination phase, the new projection method is used to map the input vector so that the input data is mapped to the structure of the SOM without having to plot the weight values, resulting in easy visualization of the data. The projection method is demonstrated on four different data sets, including a 118 patent data set and a 399 checical abstract data set related to polymer cements, with promising results and a significantly reduced network size.     

Emergence of invariant-feature detectors in the adaptive-subspace self-organizing map

 A new self-organizing map (SOM) architecture called the ASSOM (adaptive-subspace SOM) is shown to create sets of translation-invariant filters when randomly displaced or moving input patterns are used as training data. No analytical functional forms for these filters are thereby postulated. Different kinds of filters are formed by the ASSOM when pictures are rotated during learning, or when they are zoomed. The ASSOM can thus act as a learning feature-extraction stage for pattern recognizers, being able to adapt to many sensory environments and to many different transformation groups of patterns. Received: 14 September 1995 / Accepted in revised form: 8 May 1996     

Classification of protein families and detection of the determinant residues with an improved self-organizing map

 Using a SOM (self-organizing map) we can classify sequences within a protein family into subgroups that generally correspond to biological subcategories. These maps tend to show sequence similarity as proximity in the map. Combining maps generated at different levels of resolution, the structure of relations in protein families can be captured that could not otherwise be represented in a single map. The underlying representation of maps enables us to retrieve characteristic sequence patterns for individual subgroups of sequences. Such patterns tend to correspond to functionally important regions. We present a modified SOM algorithm that includes a convergence test that dynamically controls the learning parameters to adapt them to the learning set instead of being fixed and externally optimized by trial and error. Given the variability of protein family size and distribution, the addition of this feature is necessary. The method is successfully tested with a number of families. The rab family of small GTPases is used to illustrate the performance of the method. Received: 25 July 1996 / Accepted in revised form: 13 February 1997     

Nonparametric discriminant analysis of phytoplankton species using data from analytical flow cytometry

BACKGROUND: Analytical flow cytometry (AFC) provides rapid and accurate measurement of particles from heterogeneous populations. AFC has been used to classify and identify phytoplankton species, but most methods of discriminant analysis of resulting data have depended on normality assumptions and outcomes have been disappointing. METHODS AND RESULTS: In this study, we consider nonparametric methods based on density estimation. In addition to the familiar kernel method, methods based on wavelets are also implemented. Full five-dimensional wavelet estimation proves to be computationally prohibitive with current workstation power, so we employ projection pursuit for reduction of dimensionality. AFC typically produces very large samples, so we also investigate data simplification through binning. Further modifications to the discrimination strategy are suggested by specific features of phytoplankton data, namely, a hierarchical group structure, the possible presence of many groups, and the likelihood of encountering an aberrant group in a test sample. CONCLUSIONS: We apply all the resultant procedures to appropriate subsets of a very large data set, demonstrate their efficacy, and compare their error rates with those of more conventional methods. We further show that incorporation of the specific features of phytoplankton data into the analysis leads to improved results and provides a general framework for analysis of such data.     

The effect of intracortical competition on the formation of topographic maps in models of Hebbian learning

Correlation-based learning (CBL) models and self-organizing maps (SOM) are two classes of Hebbian models that have both been proposed to explain the activity-driven formation of cortical maps. Both models differ significantly in the way lateral cortical interactions are treated, leading to different predictions for the formation of receptive fields. The linear CBL models predict that receptive field profiles are determined by the average values and the spatial correlations of the second order of the afferent activity patterns, wheras SOM models map stimulus features. Here, we investigate a class of models which are characterized by a variable degree of lateral competition and which have the CBL and SOM models as limit cases. We show that there exists a critical value for intracortical competition below which the model exhibits CBL properties and above which feature mapping sets in. The class of models is then analyzed with respect to the formation of topographic maps between two layers of neurons. For Gaussian input stimuli we find that localized receptive fields and topographic maps emerge above the critical value for intracortical competition, and we calculate this value as a function of the size of the input stimuli and the range of the lateral interaction function. Additionally, we show that the learning rule can be derived via the optimization of a global cost function in a framework of probabilistic output neurons which represent a set of input stimuli by a sparse code. Received: 23 June 1999 / Accepted in revised form: 05 November 1999     

Som-based class discovery exploring the ICA-reduced features of microarray expression profiles

Gene expression datasets are large and complex, having many variables and unknown internal structure. We apply independent component analysis (ICA) to derive a less redundant representation of the expression data. The decomposition produces components with minimal statistical dependence and reveals biologically relevant information. Consequently, to the transformed data, we apply cluster analysis (an important and popular analysis tool for obtaining an initial understanding of the data, usually employed for class discovery). The proposed self-organizing map (SOM)-based clustering algorithm automatically determines the number of 'natural' subgroups of the data, being aided at this task by the available prior knowledge of the functional categories of genes. An entropy criterion allows each gene to be assigned to multiple classes, which is closer to the biological representation. These features, however, are not achieved at the cost of the simplicity of the algorithm, since the map grows on a simple grid structure and the learning algorithm remains equal to Kohonen's one.     

Analysis of gene expression data using self-organizing maps.

DNA microarray technologies together with rapidly increasing genomic sequence information is leading to an explosion in available gene expression data. Currently there is a great need for efficient methods to analyze and visualize these massive data sets. A self-organizing map (SOM) is an unsupervised neural network learning algorithm which has been successfully used for the analysis and organization of large data files. We have here applied the SOM algorithm to analyze published data of yeast gene expression and show that SOM is an excellent tool for the analysis and visualization of gene expression profiles.     

Outlier Detection using Projection Quantile Regression for Mass Spectrometry Data with Low Replication

ABSTRACT: BACKGROUND: Mass spectrometry (MS) data are often generated from various biological or chemical experiments and there may exist outlying observations, which are extreme due to technical reasons. The determination of outlying observations is important in the analysis of replicated MS data because elaborate pre-processing is essential for successful analysis with reliable results and manual outlier detection as one of pre-processing steps is time-consuming. The heterogeneity of variability and low replication are often obstacles to successful analysis, including outlier detection. Existing approaches, which assume constant variability, can generate many false positives (outliers) and/or false negatives non-outliers). Thus, a more powerful and accurate approach is needed to account for the heterogeneity of variability and low replication. FINDINGS: We proposed an outlier detection algorithm using projection and quantile regression in MS data from multiple experiments. The performance of the algorithm and program was demonstrated by using both simulated and real-life data. The projection approach with linear, nonlinear, or nonparametric quantile regression was appropriate in heterogeneous high-throughput data with low replication. CONCLUSION: Various quantile regression approaches combined with projection were proposed for detecting outliers. The choice among linear, nonlinear, and nonparametric regressions is dependent on the degree of heterogeneity of the data. The proposed approach was illustrated with MS data with two or more replicates.     

Self-organizing maps: ordering,convergence properties and energy functions

We investigate the convergence properties of the self-organizing feature map algorithm for a simple, but very instructive case: the formation of a topographic representation of the unit interval [0,1] by a linear chain of neurons. We extend the proofs of convergence of Kohonen and of Cottrell and Fort to hold in any case where the neighborhood function, which is used to scale the change in the weight values at each neuron, is a monotonically decreasing function of distance from the winner neuron. We prove that the learning dynamics cannot be described by a gradient descent on a single energy function, but may be described using a set of potential functions, one for each neuron, which are independently minimized following a stochastic gradient descent. We derive the correct potential functions for the oneand multi-dimensional case, and show that the energy functions given by Tolat (1990) are an approximation which is no longer valid in the case of highly disordered maps or steep neighborhood functions.     

Self-organizing maps: stationary states,metastability and convergence rate

We investigate the effect of various types of neighborhood function on the convergence rates and the presence or absence of metastable stationary states of Kohonen's self-organizing feature map algorithm in one dimension. We demonstrate that the time necessary to form a topographic representation of the unit interval [0, 1] may vary over several orders of magnitude depending on the range and also the shape of the neighborhood function, by which the weight changes of the neurons in the neighborhood of the winning neuron are scaled. We will prove that for neighborhood functions which are convex on an interval given by the length of the Kohonen chain there exist no metastable states. For all other neighborhood functions, metastable states are present and may trap the algorithm during the learning process. For the widely-used Gaussian function there exists a threshold for the width above which metastable states cannot exist. Due to the presence or absence of metastable states, convergence time is very sensitive to slight changes in the shape of the neighborhood function. Fastest convergence is achieved using neighborhood functions which are "convex" over a large range around the winner neuron and yet have large differences in value at neighboring neurons.     

Water quality assessment using diatom assemblages and advanced modelling techniques

1. Two types of artificial neural networks procedures were used to define and predict diatom assemblage structures in Luxembourg streams using environmental data. 2. Self‐organising maps (SOM) were used to classify samples according to their diatom composition, and multilayer perceptron with a backpropagation learning algorithm (BPN) was used to predict these assemblages using environmental characteristics of each sample as input and spatial coordinates (X and Y) of the cell centres of the SOM map identified as diatom assemblages as output. Classical methods (correspondence analysis and clustering analysis) were then used to identify the relations between diatom assemblages and the SOM cell number. A canonical correspondence analysis was also used to define the relationship between these assemblages and the environmental conditions. 3. The diatom‐SOM training set resulted in 12 representative assemblages (12 clusters) having different species compositions. Comparison of observed and estimated sample positions on the SOM map were used to evaluate the performance of the BPN (correlation coefficients were 0.93 for X and 0.94 for Y). Mean square errors of 12 cells varied from 0.47 to 1.77 and the proportion of well predicted samples ranged from 37.5 to 92.9%. This study showed the high predictability of diatom assemblages using physical and chemical parameters for a small number of river types within a restricted geographical area.     

Efficient Semiparametric Marginal Estimation for the Partially Linear Additive Model for Longitudinal/Clustered Data

We consider the efficient estimation of a regression parameter in a partially linear additive nonparametric regression model from repeated measures data when the covariates are multivariate. To date, while there is some literature in the scalar covariate case, the problem has not been addressed in the multivariate additive model case. Ours represents a first contribution in this direction. As part of this work, we first describe the behavior of nonparametric estimators for additive models with repeated measures when the underlying model is not additive. These results are critical when one considers variants of the basic additive model. We apply them to the partially linear additive repeated-measures model, deriving an explicit consistent estimator of the parametric component; if the errors are in addition Gaussian, the estimator is semiparametric efficient. We also apply our basic methods to a unique testing problem that arises in genetic epidemiology; in combination with a projection argument we develop an efficient and easily computed testing scheme. Simulations and an empirical example from nutritional epidemiology illustrate our methods.     

A high-order graph generating self-organizing structure

A large class of neural network models have their units organized in a lattice with fixed topology or generate their topology during the learning process. These network models can be used as neighborhood preserving map of the input manifold, but such a structure is difficult to manage since these maps are graphs with a number of nodes that is just one or two orders of magnitude less than the number of input points (i.e., the complexity of the map is comparable with the complexity of the manifold) and some hierarchical algorithms were proposed in order to obtain a high-level abstraction of these structures. In this paper a general structure capable to extract high order information from the graph generated by a large class of self-organizing networks is presented. This algorithm will allow to build a two layers hierarchical structure starting from the results obtained by using the suitable neural network for the distribution of the input data. Moreover the proposed algorithm is also capable to build a topology preserving map if it is trained using a graph that is also a topology preserving map.     

Predicting Market Impact Costs Using Nonparametric Machine Learning Models

Market impact cost is the most significant portion of implicit transaction costs that can reduce the overall transaction cost, although it cannot be measured directly. In this paper, we employed the state-of-the-art nonparametric machine learning models: neural networks, Bayesian neural network, Gaussian process, and support vector regression, to predict market impact cost accurately and to provide the predictive model that is versatile in the number of variables. We collected a large amount of real single transaction data of US stock market from Bloomberg Terminal and generated three independent input variables. As a result, most nonparametric machine learning models outperformed a-state-of-the-art benchmark parametric model such as I-star model in four error measures. Although these models encounter certain difficulties in separating the permanent and temporary cost directly, nonparametric machine learning models can be good alternatives in reducing transaction costs by considerably improving in prediction performance.     

Variable Selection for Semiparametric Mixed Models in Longitudinal Studies

Summary .  We propose a double-penalized likelihood approach for simultaneous model selection and estimation in semiparametric mixed models for longitudinal data. Two types of penalties are jointly imposed on the ordinary log-likelihood: the roughness penalty on the nonparametric baseline function and a nonconcave shrinkage penalty on linear coefficients to achieve model sparsity. Compared to existing estimation equation based approaches, our procedure provides valid inference for data with missing at random, and will be more efficient if the specified model is correct. Another advantage of the new procedure is its easy computation for both regression components and variance parameters. We show that the double-penalized problem can be conveniently reformulated into a linear mixed model framework, so that existing software can be directly used to implement our method. For the purpose of model inference, we derive both frequentist and Bayesian variance estimation for estimated parametric and nonparametric components. Simulation is used to evaluate and compare the performance of our method to the existing ones. We then apply the new method to a real data set from a lactation study.     

Nonparametric regression using local kernel estimating equations for correlated failure time data

We study nonparametric regression for correlated failure timedata. Kernel estimating equations are used to estimate nonparametriccovariate effects. Independent and weighted-kernel estimatingequations are studied. The derivative of the nonparametric functionis first estimated and the nonparametric function is then estimatedby integrating the derivative estimator. We show that the nonparametrickernel estimator is consistent for any arbitrary working correlationmatrix and that its asymptotic variance is minimized by assumingworking independence. We evaluate the performance of the proposedkernel estimator using simulation studies, and apply the proposedmethod to the western Kenya parasitaemia data.