Reducing topological defects in self-organizing maps using multiple scale neighborhood functions

In this paper, we propose a method of reducing topological defects in self-organizing maps (SOMs) using multiple scale neighborhood functions. The multiple scale neighborhood functions are inspired by multiple scale channels in the human visual system. To evaluate the proposed method, we applied it to the traveling salesman problem (TSP), and examined two indexes: the tour length of the solution and the number of kinks in the solution. Consequently, the two indexes are lower for the proposed method. These results indicate that our proposed method has the ability to reduce topological defects.     

Deep Learning Using Havrda-Charvat Entropy for Classification of Pulmonary Optical Endomicroscopy

1) ObjectivePulmonary optical endomicroscopy (POE) is an imaging technology in real time. It allows to examine pulmonary alveoli at a microscopic level. Acquired in clinical settings, a POE image sequence can have as much as 25% of the sequence being uninformative frames (i.e. pure-noise and motion artifacts). For future data analysis, these uninformative frames must be first removed from the sequence. Therefore, the objective of our work is to develop an automatic detection method of uninformative images in endomicroscopy images.2) Material and methodsWe propose to take the detection problem as a classification one. Considering advantages of deep learning methods, a classifier based on CNN (Convolutional Neural Network) is designed with a new loss function based on Havrda-Charvat entropy which is a parametrical generalization of the Shannon entropy. We propose to use this formula to get a better hold on all sorts of data since it provides a model more stable than the Shannon entropy.3) ResultsOur method is tested on one POE dataset including 3895 distinct images and is showing better results than using Shannon entropy and behaves better with regard to the problem of overfitting. We obtain 70% of accuracy with Shannon entropy versus 77 to 79% with Havrda-Charvat.4) ConclusionWe can conclude that Havrda-Charvat entropy is better suited for restricted and or noisy datasets due to its generalized nature. It is also more suitable for classification in endomicroscopy datasets.     

Inferring Cell-Scale Signalling Networks via Compressive Sensing

Signalling network inference is a central problem in system biology. Previous studies investigate this problem by independently inferring local signalling networks and then linking them together via crosstalk. Since a cellular signalling system is in fact indivisible, this reductionistic approach may have an impact on the accuracy of the inference results. Preferably, a cell-scale signalling network should be inferred as a whole. However, the holistic approach suffers from three practical issues: scalability, measurement and overfitting. Here we make this approach feasible based on two key observations: 1) variations of concentrations are sparse due to separations of timescales; 2) several species can be measured together using cross-reactivity. We propose a method, CCELL, for cell-scale signalling network inference from time series generated by immunoprecipitation using Bayesian compressive sensing. A set of benchmark networks with varying numbers of time-variant species is used to demonstrate the effectiveness of our method. Instead of exhaustively measuring all individual species, high accuracy is achieved from relatively few measurements.     

Generalized optimal spatial filtering using a kernel approach with application to EEG classification

Common spatial patterns (CSP) has been widely used for finding the linear spatial filters which are able to extract the discriminative brain activities between two different mental tasks. However, the CSP is difficult to capture the nonlinearly clustered structure from the non-stationary EEG signals. To relax the presumption of strictly linear patterns in the CSP, in this paper, a generalized CSP (GCSP) based on generalized singular value decomposition (GSVD) and kernel method is proposed. Our method is able to find the nonlinear spatial filters which are formulated in the feature space defined by a nonlinear mapping through kernel functions. Furthermore, in order to overcome the overfitting problem, the regularized GCSP is developed by adding the regularized parameters. The experimental results demonstrate that our method is an effective nonlinear spatial filtering method.     

Hands-on training about overfitting

Overfitting is one of the critical problems in developing models by machine learning. With machine learning becoming an essential technology in computational biology, we must include training about overfitting in all courses that introduce this technology to students and practitioners. We here propose a hands-on training for overfitting that is suitable for introductory level courses and can be carried out on its own or embedded within any data science course. We use workflow-based design of machine learning pipelines, experimentation-based teaching, and hands-on approach that focuses on concepts rather than underlying mathematics. We here detail the data analysis workflows we use in training and motivate them from the viewpoint of teaching goals. Our proposed approach relies on Orange, an open-source data science toolbox that combines data visualization and machine learning, and that is tailored for education in machine learning and explorative data analysis.     

Validation of a decision support system for the cytodiagnosis of fine needle aspirates of the breast using a prospectively collected dataset from multiple observers in a working clinical environment

We have used a 692 case dataset, collected retrospectively by a single observer, to develop decision support systems for the cytodiagnosis of fine needle aspirates of breast lesions. In this study, we use a 322 case dataset that was prospectively collected by multiple observers in a working clinical environment to test two predictive systems, using logistic regression and the multilayer perceptron (MLP) type of neural network. Ten observed features and the patient age were used as input features. The systems were developed using a training set and test set from the single observer dataset and then applied to the multiple observer dataset. For the independent test cases from the single observer dataset, with a threshold set for no false positives on the training set, logistic regression produced a sensitivity of 82% (95% confidence interval 73-91) and a predictive value of a positive result (PV +) of 98% (95-99), the values for the MLP were 79% (69-89) and 100%, respectively. However the performance on the prospective multiple observer dataset was much worse, with a sensitivity of 72% (65-80), and PV + of 97% (94-99) for logistic regression and 67% (60-75) and 91% (85-97) for the MLP. These results suggest that there is considerable interobserver variability for the defined features and that this system is unsuitable for further development in the clinical environment unless this problem can be overcome.     

Online Multi-Modal Robust Non-Negative Dictionary Learning for Visual Tracking

Dictionary learning is a method of acquiring a collection of atoms for subsequent signal representation. Due to its excellent representation ability, dictionary learning has been widely applied in multimedia and computer vision. However, conventional dictionary learning algorithms fail to deal with multi-modal datasets. In this paper, we propose an online multi-modal robust non-negative dictionary learning (OMRNDL) algorithm to overcome this deficiency. Notably, OMRNDL casts visual tracking as a dictionary learning problem under the particle filter framework and captures the intrinsic knowledge about the target from multiple visual modalities, e.g., pixel intensity and texture information. To this end, OMRNDL adaptively learns an individual dictionary, i.e., template, for each modality from available frames, and then represents new particles over all the learned dictionaries by minimizing the fitting loss of data based on M-estimation. The resultant representation coefficient can be viewed as the common semantic representation of particles across multiple modalities, and can be utilized to track the target. OMRNDL incrementally learns the dictionary and the coefficient of each particle by using multiplicative update rules to respectively guarantee their non-negativity constraints. Experimental results on a popular challenging video benchmark validate the effectiveness of OMRNDL for visual tracking in both quantity and quality.     

BCDForest: a boosting cascade deep forest model towards the classification of cancer subtypes based on gene expression data

Background

The classification of cancer subtypes is of great importance to cancer disease diagnosis and therapy. Many supervised learning approaches have been applied to cancer subtype classification in the past few years, especially of deep learning based approaches. Recently, the deep forest model has been proposed as an alternative of deep neural networks to learn hyper-representations by using cascade ensemble decision trees. It has been proved that the deep forest model has competitive or even better performance than deep neural networks in some extent. However, the standard deep forest model may face overfitting and ensemble diversity challenges when dealing with small sample size and high-dimensional biology data.

Results

In this paper, we propose a deep learning model, so-called BCDForest, to address cancer subtype classification on small-scale biology datasets, which can be viewed as a modification of the standard deep forest model. The BCDForest distinguishes from the standard deep forest model with the following two main contributions: First, a named multi-class-grained scanning method is proposed to train multiple binary classifiers to encourage diversity of ensemble. Meanwhile, the fitting quality of each classifier is considered in representation learning. Second, we propose a boosting strategy to emphasize more important features in cascade forests, thus to propagate the benefits of discriminative features among cascade layers to improve the classification performance. Systematic comparison experiments on both microarray and RNA-Seq gene expression datasets demonstrate that our method consistently outperforms the state-of-the-art methods in application of cancer subtype classification.

Conclusions

The multi-class-grained scanning and boosting strategy in our model provide an effective solution to ease the overfitting challenge and improve the robustness of deep forest model working on small-scale data. Our model provides a useful approach to the classification of cancer subtypes by using deep learning on high-dimensional and small-scale biology data.

     

Sparse Support Vector Machines with L_{p} Penalty for Biomarker Identification

The development of high-throughput technology has generated a massive amount of high-dimensional data, and many of them are of discrete type. Robust and efficient learning algorithms such as LASSO [1] are required for feature selection and overfitting control. However, most feature selection algorithms are only applicable to the continuous data type. In this paper, we propose a novel method for sparse support vector machines (SVMs) with L_{p} (p ≪ 1) regularization. Efficient algorithms (LpSVM) are developed for learning the classifier that is applicable to high-dimensional data sets with both discrete and continuous data types. The regularization parameters are estimated through maximizing the area under the ROC curve (AUC) of the cross-validation data. Experimental results on protein sequence and SNP data attest to the accuracy, sparsity, and efficiency of the proposed algorithm. Biomarkers identified with our methods are compared with those from other methods in the literature. The software package in Matlab is available upon request.     

Optimized multilayer perceptrons for molecular classification and diagnosis using genomic data

MOTIVATION: Multilayer perceptrons (MLP) represent one of the widely used and effective machine learning methods currently applied to diagnostic classification based on high-dimensional genomic data. Since the dimensionalities of the existing genomic data often exceed the available sample sizes by orders of magnitude, the MLP performance may degrade owing to the curse of dimensionality and over-fitting, and may not provide acceptable prediction accuracy. RESULTS: Based on Fisher linear discriminant analysis, we designed and implemented an MLP optimization scheme for a two-layer MLP that effectively optimizes the initialization of MLP parameters and MLP architecture. The optimized MLP consistently demonstrated its ability in easing the curse of dimensionality in large microarray datasets. In comparison with a conventional MLP using random initialization, we obtained significant improvements in major performance measures including Bayes classification accuracy, convergence properties and area under the receiver operating characteristic curve (A(z)). SUPPLEMENTARY INFORMATION: The Supplementary information is available on http://www.cbil.ece.vt.edu/publications.htm     

DMP3: a dynamic multilayer perceptron construction algorithm

This paper presents DMP3 (Dynamic Multilayer Perceptron 3), a multilayer perceptron (MLP) constructive training method that constructs MLPs by incrementally adding network elements of varying complexity to the network. DMP3 differs from other MLP construction techniques in several important ways, and the motivation for these differences are given. Information gain rather than error minimization is used to guide the growth of the network, which increases the utility of newly added network elements and decreases the likelihood that a premature dead end in the growth of the network will occur. The generalization performance of DMP3 is compared with that of several other well-known machine learning and neural network learning algorithms on nine real world data sets. Simulation results show that DMP3 performs better (on average) than any of the other algorithms on the data sets tested. The main reasons for this result are discussed in detail.     

Can the false-discovery rate be misleading?

The decoy-database approach is currently the gold standard for assessing the confidence of identifications in shotgun proteomic experiments. Here, we demonstrate that what might appear to be a good result under the decoy-database approach for a given false-discovery rate could be, in fact, the product of overfitting. This problem has been overlooked until now and could lead to obtaining boosted identification numbers whose reliability does not correspond to the expected false-discovery rate. To overcome this, we are introducing a modified version of the method, termed a semi-labeled decoy approach, which enables the statistical determination of an overfitted result.     

The cardiac mechanical stretch sensor machinery involves a Z disc complex that is defective in a subset of human dilated cardiomyopathy

Muscle cells respond to mechanical stretch stimuli by triggering downstream signals for myocyte growth and survival. The molecular components of the muscle stretch sensor are unknown, and their role in muscle disease is unclear. Here, we present biophysical/biochemical studies in muscle LIM protein (MLP) deficient cardiac muscle that support a selective role for this Z disc protein in mechanical stretch sensing. MLP interacts with and colocalizes with telethonin (T-cap), a titin interacting protein. Further, a human MLP mutation (W4R) associated with dilated cardiomyopathy (DCM) results in a marked defect in T-cap interaction/localization. We propose that a Z disc MLP/T-cap complex is a key component of the in vivo cardiomyocyte stretch sensor machinery, and that defects in the complex can lead to human DCM and associated heart failure.     

A computational model of the development of separate representations of facial identity and expression in the primate visual system

Experimental studies have provided evidence that the visual processing areas of the primate brain represent facial identity and facial expression within different subpopulations of neurons. For example, in non-human primates there is evidence that cells within the inferior temporal gyrus (TE) respond primarily to facial identity, while cells within the superior temporal sulcus (STS) respond to facial expression. More recently, it has been found that the orbitofrontal cortex (OFC) of non-human primates contains some cells that respond exclusively to changes in facial identity, while other cells respond exclusively to facial expression. How might the primate visual system develop physically separate representations of facial identity and expression given that the visual system is always exposed to simultaneous combinations of facial identity and expression during learning? In this paper, a biologically plausible neural network model, VisNet, of the ventral visual pathway is trained on a set of carefully-designed cartoon faces with different identities and expressions. The VisNet model architecture is composed of a hierarchical series of four Self-Organising Maps (SOMs), with associative learning in the feedforward synaptic connections between successive layers. During learning, the network develops separate clusters of cells that respond exclusively to either facial identity or facial expression. We interpret the performance of the network in terms of the learning properties of SOMs, which are able to exploit the statistical indendependence between facial identity and expression.     

CWLy-pred: A novel cell wall lytic enzyme identifier based on an improved MRMD feature selection method

Cell wall lytic enzymes play key roles in biochemical, morphological, genetic research and industry fields. To save time and labor costs, bioinformatic methods are usually adopted to narrow the scope of in vitro experimentation. In this paper, we established a novel machine learning (support vector machine) based identifier called CWLy-pred to identify cell wall lytic enzymes. An improved MRMD feature selection method is also proposed to select the optimal training set to avoid data redundancy. CWLy-pred obtains an accuracy of 93.067%, a sensitivity of 85.3%, a specificity of 94.8%, an MCC of 0.775 and an AUC of 0.900. It outperforms the state-of-the-art identifier in terms of accuracy, sensitivity, specificity and MCC. Our proposed model is based on a feature set of only 6 dimensions; therefore, it not only can overcome overfitting problems but can also supervise biological experiments effectively. CWLy-pred is embedded in a web application at http://server.malab.cn/CWLy-pred/index.jsp, which is accessible for free.     

Multi-scale training of a large backpropagation net

The shortcomings of automatic learning in neural networks can be overcome by incorporating features of the problem into the weight space. We teach a multi-layer perceptron to recognize hand-printed characters by using a multi-scale method. This leads to better performance for learning rates and for generalization than direct use of back-propagation.     

Neural correlates for feeling-of-knowing: an fMRI parametric analysis

The "feeling-of-knowing" (FOK) is a subjective sense of knowing a word before recalling it, and the FOK provides us clues to understanding the mechanisms of human metamemory systems. We investigated neural correlates for the FOK based on the recall-judgment-recognition paradigm. Event-related functional magnetic resonance imaging with a parametric analysis was used. We found activations in left dorsolateral, left anterior, bilateral inferior, and medial prefrontal cortices that significantly increased as the FOK became greater, and the activations remained significant even when the potentially confounding factor of the response latency was removed. Furthermore, we demonstrated that the FOK region in the right inferior frontal gyrus and a subset of the FOK region in the left inferior frontal gyrus are not recruited for successful recall processes, suggesting their particular role in metamemory processing.     

Cellular and functional defects in a mouse model of heart failure

Heart failure and dilated cardiomyopathy develop in mice that lack the muscle LIM protein (MLP) gene (MLP(-/-)). The character and extent of the heart failure that occurs in MLP(-/-) mice were investigated using echocardiography and in vivo pressure-volume (P-V) loop measurements. P-V loop data were obtained with a new method for mice (sonomicrometry) using two pairs of orthogonal piezoelectric crystals implanted in the endocardial wall. Sonomicrometry revealed right-shifted P-V loops in MLP(-/-) mice, depressed systolic contractility, and additional evidence of heart failure. Cellular changes in MLP(-/-) mice were examined in isolated single cells using patch-clamp and confocal Ca(2+) concentration ([Ca(2+)]) imaging techniques. This cellular investigation revealed unchanged Ca(2+) currents and Ca(2+) spark characteristics but decreased intracellular [Ca(2+)] transients and contractile responses and a defect in excitation-contraction coupling. Normal cellular and whole heart function was restored in MLP(-/-) mice that express a cardiac-targeted transgene, which blocks the function of beta-adrenergic receptor (beta-AR) kinase-1 (betaARK1). These data suggest that, despite the persistent stimulus to develop heart failure in MLP(-/-) mice (i.e., loss of the structural protein MLP), downregulation and desensitization of the beta-ARs may play a pivotal role in the pathogenesis. Furthermore, this work suggests that the inhibition of betaARK1 action may prove an effective therapy for heart failure.     

Fundus Retinal Vessels Image Segmentation Method Based on Improved U-Net

ObjectivesAlthough the segmentation of retinal vessels in the fundus is of great significance for screening and diagnosing retinal vascular diseases, it remains difficult to detect the low contrast and the information around the lesions provided by retinal vessels in the fundus and to locate and segment micro-vessels in the fine-grained area. To overcome this problem, we propose herein an improved U-Net segmentation method NoL-UNet.Material and methodsThis work introduces NoL-UNet. First of all, the ordinary convolution block of the U-Net network is changed to random dropout convolution blocks, which can better extract the relevant features of the image and effectively alleviate the network overfitting. Next, a NoL-Block attention mechanism added to the bottom of the encoding-decoding structure expands the receptive field and enhances the correlation of pixel information without increasing the number of parameters.ResultsThe proposed method is verified by applying it to the fundus image datasets DRIVE, CHASE_DB1, and HRF. The AUC for DRIVE, CHASE_DB1 and HRF is 0.9861, 0.9891 and 0.9893, Se for DRIVE, CHASE_DB1 and HRF is 0.8489, 0.8809 and 0.8476, and the Acc for DRIVE, CHASE_DB1 and HRF is 0.9697, 0.9826 and 0.9732, respectively. The total number of parameters is 1.70M, and for DRIVE, it takes 0.050s to segment an image.ConclusionOur method is statistically significantly different from the U-Net method, and the improved method shows superior performance with better accuracy and robustness of the model, which has good practical application in auxiliary diagnosis.     

Ordering process of self-organizing maps improved by asymmetric neighborhood function

The Self-organizing map (SOM) is an unsupervised learning method based on the neural computation, which has found wide applications. However, the learning process sometime takes multi-stable states, within which the map is trapped to an undesirable disordered state including topological defects on the map. These topological defects critically aggravate the performance of the SOM. In order to overcome this problem, we propose to introduce an asymmetric neighborhood function for the SOM algorithm. Compared with the conventional symmetric one, the asymmetric neighborhood function accelerates the ordering process even in the presence of the defect. However, this asymmetry tends to generate a distorted map. This can be suppressed by an improved method of the asymmetric neighborhood function. In the case of one-dimensional SOM, it is found that the required steps for perfect ordering is numerically shown to be reduced from O(N ³) to O(N ²). We also discuss the ordering process of a twisted state in two-dimensional SOM, which can not be rectified by the ordinary symmetric neighborhood function.