Sparse Coding Models Can Exhibit Decreasing Sparseness while Learning Sparse Codes for Natural Images

The sparse coding hypothesis has enjoyed much success in predicting response properties of simple cells in primary visual cortex (V1) based solely on the statistics of natural scenes. In typical sparse coding models, model neuron activities and receptive fields are optimized to accurately represent input stimuli using the least amount of neural activity. As these networks develop to represent a given class of stimulus, the receptive fields are refined so that they capture the most important stimulus features. Intuitively, this is expected to result in sparser network activity over time. Recent experiments, however, show that stimulus-evoked activity in ferret V1 becomes less sparse during development, presenting an apparent challenge to the sparse coding hypothesis. Here we demonstrate that some sparse coding models, such as those employing homeostatic mechanisms on neural firing rates, can exhibit decreasing sparseness during learning, while still achieving good agreement with mature V1 receptive field shapes and a reasonably sparse mature network state. We conclude that observed developmental trends do not rule out sparseness as a principle of neural coding per se: a mature network can perform sparse coding even if sparseness decreases somewhat during development. To make comparisons between model and physiological receptive fields, we introduce a new nonparametric method for comparing receptive field shapes using image registration techniques.     

Balanced Sparse Model for Tight Frames in Compressed Sensing Magnetic Resonance Imaging

Compressed sensing has shown to be promising to accelerate magnetic resonance imaging. In this new technology, magnetic resonance images are usually reconstructed by enforcing its sparsity in sparse image reconstruction models, including both synthesis and analysis models. The synthesis model assumes that an image is a sparse combination of atom signals while the analysis model assumes that an image is sparse after the application of an analysis operator. Balanced model is a new sparse model that bridges analysis and synthesis models by introducing a penalty term on the distance of frame coefficients to the range of the analysis operator. In this paper, we study the performance of the balanced model in tight frame based compressed sensing magnetic resonance imaging and propose a new efficient numerical algorithm to solve the optimization problem. By tuning the balancing parameter, the new model achieves solutions of three models. It is found that the balanced model has a comparable performance with the analysis model. Besides, both of them achieve better results than the synthesis model no matter what value the balancing parameter is. Experiment shows that our proposed numerical algorithm constrained split augmented Lagrangian shrinkage algorithm for balanced model (C-SALSA-B) converges faster than previously proposed algorithms accelerated proximal algorithm (APG) and alternating directional method of multipliers for balanced model (ADMM-B).     

Modelling neural informational propagation and functional auditory sensory memory with temporal multi-scale operators

In this paper we prove that both diffusion and the leaky integrators cascade based transport mechanisms have as their inherent property the effect of temporal multi-scaling. The two transport mechanisms are modeled not as convolution based algorithms but as causal physical processes. This implies that propagation of information through a neural map may act as a mechanism for achieving temporal multi-scale analysis in the auditory system. Specifically, we are interested in the effects of such a transport process on the formation and the dynamics of auditory sensory memory. Two temporal models of information propagation are discussed and compared in terms of their ability to model auditory sensory memory effects and the biological plausibility of their structure: the causal diffusion based operator (CD) and the leaky integrator cascade based operator (LINC). We show that temporal multi-scale representations achieved by both models exhibit the effects similar to those of auditory sensory memory (filtering, time delay and binding of information). As regards higher-level functions of auditory sensory memory such as change detection, the LINC operator seems to be a biologically more plausible solution for modeling temporal cortical processing.     

Assessment of human operator functional state using a novel differential evolution optimization based adaptive fuzzy model

With the development of human–machine systems, there has been a growing concern about the consequences of operator performance breakdown under excessive level of workload, especially in safety-critical situations. Assessment and detection of the operator functional state (OFS) enable us to predict the high operational risks of operator. This paper adopts the psychophysiological signals and task performance measures to evaluate OFS under different levels of mental workload. Four indices extracted from electrocardiogram and electroencephalogram, including heart rate (HR), ratio of the standard deviation to the average of HR segment, task load indices (TLI1 and TLI2), are chosen as the inputs of the proposed model. A technique of differential evolution with ant colony search (DEACS) is developed to optimize the parameters of Adaptive-Network-based Fuzzy Inference System (ANFIS). The optimized ANFIS model is employed to estimate the OFS under a series of process control tasks on a simulated software platform of AUTOmation-enhanced Cabin Air Management System. The results showed that the proposed adaptive fuzzy model based on ANFIS and DEACS algorithm is applicable for the operator functional state assessment.     

基于最小二乘支持向量机的农田水汽通量建模

支持向量机是在统计学习理论基础上发展起来的一种新型的机器学习方法,已在模式识别、非线性建模等领域中得到了应用．本文将最小二乘支持向量机方法应用于农田水汽通量的建模中,并同前馈反向传播神经网络的建模性能进行了比较．结果表明,最小二乘支持向量机方法具有可调参数少、学习速度较快等优点,具有更好的推广能力,以更高的精度建立农田水汽通量模型．模型的敏感性分析进一步显示,用最小二乘支持向量机方法建立的农田水汽通量模型是合理可行的．     

Operations management in the home care services: a heuristic for the caregivers’ routing problem

Home Care Services (HCS) aim at providing complex coordinated health care for patients at their homes. This paper addresses the challenges of routing and scheduling HCS caregivers under precedence and coordination constraints, with patients receiving multiple caregivers. Moreover, the visits are performed simultaneously and possibly in a predefined order. The routing problem involves a fleet of vehicles to serve a number of customers at different locations. The objective is to find the minimal round for vehicle, while satisfying all the customers and without violating customers’ time windows. It has been proved that the complexity of the caregivers routing problem is linked to both (1) the number of care activities per caregiver ratio and (2) the temporal dependencies rate. Given the poor performance of the mathematical modeling based on exact approaches, a heuristic approach called the Caregivers Routing Heuristic (CRH) has been developed and tested using real size instances. In fact, the exact approaches are not able to solve real size instances. The performance of the CRH has been evaluated using real size instances. The numerical results show that the CRH is very efficient in terms of computation times. Otherwise, the CRH is less sensitive than the exact approaches to both complexity axes: the temporal dependencies constraints and the ratio of the number of care activities per caregiver.     

Identification of sparse neural functional connectivity using penalized likelihood estimation and basis functions

One key problem in computational neuroscience and neural engineering is the identification and modeling of functional connectivity in the brain using spike train data. To reduce model complexity, alleviate overfitting, and thus facilitate model interpretation, sparse representation and estimation of functional connectivity is needed. Sparsities include global sparsity, which captures the sparse connectivities between neurons, and local sparsity, which reflects the active temporal ranges of the input-output dynamical interactions. In this paper, we formulate a generalized functional additive model (GFAM) and develop the associated penalized likelihood estimation methods for such a modeling problem. A GFAM consists of a set of basis functions convolving the input signals, and a link function generating the firing probability of the output neuron from the summation of the convolutions weighted by the sought model coefficients. Model sparsities are achieved by using various penalized likelihood estimations and basis functions. Specifically, we introduce two variations of the GFAM using a global basis (e.g., Laguerre basis) and group LASSO estimation, and a local basis (e.g., B-spline basis) and group bridge estimation, respectively. We further develop an optimization method based on quadratic approximation of the likelihood function for the estimation of these models. Simulation and experimental results show that both group-LASSO-Laguerre and group-bridge-B-spline can capture faithfully the global sparsities, while the latter can replicate accurately and simultaneously both global and local sparsities. The sparse models outperform the full models estimated with the standard maximum likelihood method in out-of-sample predictions.     

考虑气候因子变化的湖泊富营养化模型研究进展

气候因子是影响湖泊营养状态和进程的主要自然因素.在全球气候变化的趋势下,将气候因子的变化纳入湖泊富营养化模型中,可以为湖泊演化趋势分析和环境管理决策提供技术支持.本文首先分析了气温、降水、光照和大气等气候因子对湖泊富营养化的影响,进而对考虑气候因子变化的数理统计与分析模型、生态动力学模型、系统生态学模型及智能算法等的研究进行了综述.在此基础上,对完善气候因子变化下湖泊营养状态变化的模型研究进行了展望：1)加强气候因子作用于湖泊营养状态的机理研究;2)选择合适的气候模拟模型,合理设置气候变化情景,在不同模型嵌套时保证时空尺度的匹配;3)以水动力学模型为基础,耦合生态模型及智能算法等,并结合良好的气候模拟模型,以精确模拟预测气候变化下湖泊富营养化的演化过程和趋势.     

Sparsity-Regularized HMAX for Visual Recognition

About ten years ago, HMAX was proposed as a simple and biologically feasible model for object recognition, based on how the visual cortex processes information. However, the model does not encompass sparse firing, which is a hallmark of neurons at all stages of the visual pathway. The current paper presents an improved model, called sparse HMAX, which integrates sparse firing. This model is able to learn higher-level features of objects on unlabeled training images. Unlike most other deep learning models that explicitly address global structure of images in every layer, sparse HMAX addresses local to global structure gradually along the hierarchy by applying patch-based learning to the output of the previous layer. As a consequence, the learning method can be standard sparse coding (SSC) or independent component analysis (ICA), two techniques deeply rooted in neuroscience. What makes SSC and ICA applicable at higher levels is the introduction of linear higher-order statistical regularities by max pooling. After training, high-level units display sparse, invariant selectivity for particular individuals or for image categories like those observed in human inferior temporal cortex (ITC) and medial temporal lobe (MTL). Finally, on an image classification benchmark, sparse HMAX outperforms the original HMAX by a large margin, suggesting its great potential for computer vision.     

Heliospheric current sheet and effects of its interaction with solar cosmic rays

The effects of interaction of solar cosmic rays (SCRs) with the heliospheric current sheet (HCS) in the solar wind are analyzed. A self-consistent kinetic model of the HCS is developed in which ions with quasiadiabatic dynamics can present. The HCS is considered an equilibrium embedded current structure in which two main plasma species with different temperatures (the low-energy background plasma of the solar wind and the higher energy SCR component) contribute to the current. The obtained results are verified by comparing with the results of numerical simulations based on solving equations of motion by the particle tracing method in the given HCS magnetic field with allowance for SCR particles. It is shown that the HCS is a relatively thin multiscale current configuration embedded in a thicker plasma layer. In this case, as a rule, the shear (tangential to the sheet current) component of the magnetic field is present in the HCS. Taking into account high-energy SCR particles in the HCS can lead to a change of its configuration and the formation of a multiscale embedded structure. Parametric family of solutions is considered in which the current balance in the HCS is provided at different SCR temperatures and different densities of the high-energy plasma. The SCR densities are determined at which an appreciable (detectable by satellites) HCS thickening can occur. Possible applications of this modeling to explain experimental observations are discussed.     

Operator functional state estimation based on EEG-data-driven fuzzy model

This paper proposed a max–min-entropy-based fuzzy partition method for fuzzy model based estimation of human operator functional state (OFS). The optimal number of fuzzy partitions for each I/O variable of fuzzy model is determined by using the entropy criterion. The fuzzy models were constructed by using Wang–Mendel method. The OFS estimation results showed the practical usefulness of the proposed fuzzy modeling approach.     

SVM-based learning control of space robots in capturing operation

In this paper, we presents a novel approach for tracking and catching operation of space robots using learning and transferring human control strategies (HCS). We firstly use an efficient support vector machine (SVM) to parametrize the model of HCS. Then we develop a new SVM-based learning structure to better implement human control strategy learning in tracking and capturing control. The approach is fundamentally valuable in dealing with some problems such as small sample data and local minima, and so on. Therefore this approach is efficient in modeling, understanding and transferring its learning process. The simulation results attest that this approach is useful and feasible in generating tracking trajectory and catching objects autonomously.     

Comparison of Modeling Methods for DoE‐Based Holistic Upstream Process Characterization

Upstream bioprocess characterization and optimization are time and resource‐intensive tasks. Regularly in the biopharmaceutical industry, statistical design of experiments (DoE) in combination with response surface models (RSMs) are used, neglecting the process trajectories and dynamics. Generating process understanding with time‐resolved, dynamic process models allows to understand the impact of temporal deviations, production dynamics, and provides a better understanding of the process variations that stem from the biological subsystem. The authors propose to use DoE studies in combination with hybrid modeling for process characterization. This approach is showcased on Escherichia coli fed‐batch cultivations at the 20L scale, evaluating the impact of three critical process parameters. The performance of a hybrid model is compared to a pure data‐driven model and the widely adopted RSM of the process endpoints. Further, the performance of the time‐resolved models to simultaneously predict biomass and titer is evaluated. The superior behavior of the hybrid model compared to the pure black‐box approaches for process characterization is presented. The evaluation considers important criteria, such as the prediction accuracy of the biomass and titer endpoints as well as the time‐resolved trajectories. This showcases the high potential of hybrid models for soft‐sensing and model predictive control.     

A Hidden Markov Model for Urban-Scale Traffic Estimation Using Floating Car Data

Urban-scale traffic monitoring plays a vital role in reducing traffic congestion. Owing to its low cost and wide coverage, floating car data (FCD) serves as a novel approach to collecting traffic data. However, sparse probe data represents the vast majority of the data available on arterial roads in most urban environments. In order to overcome the problem of data sparseness, this paper proposes a hidden Markov model (HMM)-based traffic estimation model, in which the traffic condition on a road segment is considered as a hidden state that can be estimated according to the conditions of road segments having similar traffic characteristics. An algorithm based on clustering and pattern mining rather than on adjacency relationships is proposed to find clusters with road segments having similar traffic characteristics. A multi-clustering strategy is adopted to achieve a trade-off between clustering accuracy and coverage. Finally, the proposed model is designed and implemented on the basis of a real-time algorithm. Results of experiments based on real FCD confirm the applicability, accuracy, and efficiency of the model. In addition, the results indicate that the model is practicable for traffic estimation on urban arterials and works well even when more than 70% of the probe data are missing.     

Inferential biases linked to unobservable states in complex occupancy models

Modeling of species distributions has undergone a shift from relying on equilibrium assumptions to recognizing transient system dynamics explicitly. This shift has necessitated more complex modeling techniques, but the performance of these dynamic models has not yet been assessed for systems where unobservable states exist. Our work is motivated by the impacts of the emerging infectious disease chytridiomycosis, a disease of amphibians that is associated with declines of many species worldwide. Using this host‐pathogen system as a general example, we first illustrate how misleading inferences can result from failing to incorporate pathogen dynamics into the modeling process, especially when the pathogen is difficult or impossible to survey in the absence of a host species. We found that traditional modeling techniques can underestimate the effect of a pathogen on host species occurrence and dynamics when the pathogen can only be detected in the host, and pathogen information is treated as a covariate. We propose a dynamic multistate modeling approach that is flexible enough to account for the detection structures that may be present in complex multistate systems, especially when the sampling design is limited by a species’ natural history or sampling technology. When multistate occupancy models are used and an unobservable state is present, parameter estimation can be influenced by model complexity, data sparseness, and the underlying dynamics of the system. We show that, even with large sample sizes, many models incorporating seasonal variation in vital rates may not generate reasonable estimates, indicating parameter redundancy. We found that certain types of missing data can greatly hinder inference, and we make study design recommendations to avoid these issues. Additionally, we advocate the use of time‐varying covariates to explain temporal trends in the data, and the development of sampling techniques that match the biology of the system to eliminate unobservable states when possible.     

Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis

MOTIVATION: Many practical pattern recognition problems require non-negativity constraints. For example, pixels in digital images and chemical concentrations in bioinformatics are non-negative. Sparse non-negative matrix factorizations (NMFs) are useful when the degree of sparseness in the non-negative basis matrix or the non-negative coefficient matrix in an NMF needs to be controlled in approximating high-dimensional data in a lower dimensional space. RESULTS: In this article, we introduce a novel formulation of sparse NMF and show how the new formulation leads to a convergent sparse NMF algorithm via alternating non-negativity-constrained least squares. We apply our sparse NMF algorithm to cancer-class discovery and gene expression data analysis and offer biological analysis of the results obtained. Our experimental results illustrate that the proposed sparse NMF algorithm often achieves better clustering performance with shorter computing time compared to other existing NMF algorithms. AVAILABILITY: The software is available as supplementary material.     

A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields

Computational models of primary visual cortex have demonstrated that principles of efficient coding and neuronal sparseness can explain the emergence of neurones with localised oriented receptive fields. Yet, existing models have failed to predict the diverse shapes of receptive fields that occur in nature. The existing models used a particular "soft" form of sparseness that limits average neuronal activity. Here we study models of efficient coding in a broader context by comparing soft and "bard" forms of neuronal sparseness. As a result of our analyses, we propose a novel network model for visual cortex. The model forms efficient visual representations in which the number of active neurones, rather than mean neuronal activity, is limited. This form of hard sparseness also economises cortical resources like synaptic memory and metabolic energy. Furthermore, our model accurately predicts the distribution of receptive field shapes found in the primary visual cortex of cat and monkey.     

Modeling of Sugar Crystallization through Knowledge Integration

This paper reports on the comparison of three modeling approaches that were applied to a fed batch evaporative sugar crystallization process. They are termed white box, black box, and grey box modeling strategies, which reflects the level of physical transparency and understanding of the model. White box models represent the traditional modeling approach, based on modeling by first principles. Black box models rely on recorded process data and knowledge collected during the normal process operation. Among various tools in this group an artificial neural networks (ANN) approach is adopted in this paper. The grey box model is obtained from a combination of first principles modeling, based on mass, energy and population balances, with an ANN to approximate three kinetic parameters ‐‐ crystal growth rate, nucleation rate and the agglomeration kernel. The results have shown that the hybrid modeling approach outperformed the other aforementioned modeling strategies.     

MIA-QSAR,PCA-ranking and least-squares support-vector machines in the accurate prediction of the activities of phosphodiesterase type 5 (PDE-5) inhibitors

Phosphodiesterase type-5 (PDE-5) is a key enzyme involved in the erection process. PDE-5 inhibitors, such as Sildenafil (Viagra^TM), Vardenafil (Levitra^TM) and Tadalafil (Cialis^TM), are used for the treatment of erectile dysfunction. Computer-assisted modelling of biological activities of PDE-5 inhibitors may make quantitative structure–activity relationship (QSAR) models useful for the development of safer (low side effects) and more potent drugs. The multivariate image analysis applied to QSAR (MIA-QSAR) method, coupled to partial least-squares (PLS) regression, has provided highly predictive QSAR models. Nevertheless, regression methods which take into account nonlinearity, such as least-squares support-vector machines (LS-SVMs), are supposed to predict biological activities more accurately than the usual linear methods. Thus, together with prior variable selection using principal component analysis ranking, MIA-QSAR and LS-SVM regression were applied to model the bioactivities of a series of cyclic guanine derivatives (PDE-5 inhibitors), and the results were compared with those based on linear methodologies. MIA-QSAR/LS-SVM was found to improve greatly the prediction performance when compared with MIA-QSAR/PLS, MIA-QSAR/N-PLS, CoMFA/PLS and CoMSIA/PLS models.