On problem solving with Hopfield neural networks

Hopfield and Tank have shown that neural networks can be used to solve certain computationally hard problems, in particular they studied the Traveling Salesman Problem (TSP). Based on network simulation results they conclude that analog VLSI neural nets can be promising in solving these problems. Recently, Wilson and Pawley presented the results of their simulations which contradict the original results and cast doubts on the usefulness of neural nets. In this paper we give the results of our simulations that clarify some of the discrepancies. We also investigate the scaling of TSP solutions found by neural nets as the size of the problem increases. Further, we consider the neural net solution of the Clustering Problem, also a computationally hard problem, and discuss the types of problems that appear to be well suited for a neural net approach.     

Optimal HP configurations of proteins by combining local search with elastic net algorithm

The prediction of protein conformation from its amino-acid sequence is one of the most prominent problems in computational biology. But it is NP-hard. Here, we focus on an abstraction widely studied of this problem, the two-dimensional hydrophobic-polar protein folding problem (2D HP PFP). Mathematical optimal model of free energy of protein is established. Native conformations are often sought using stochastic sampling methods, but which are slow. The elastic net (EN) algorithm is one of fast deterministic methods as travelling salesman problem (TSP) strategies. However, it cannot be applied directly to protein folding problem, because of fundamental differences in the two types of problems. In this paper, how the 2D HP protein folding problem can be framed in terms of TSP is shown. Combination of the modified elastic net algorithm and novel local search method is adopted to solve this problem. To our knowledge, this is the first application of EN algorithm to 2D HP model. The results indicate that our approach can find more optimal conformations and is simple to implement, computationally efficient and fast.     

Optimal HP configurations of proteins by combining local search with elastic net algorithm

The prediction of protein conformation from its amino-acid sequence is one of the most prominent problems in computational biology. But it is NP-hard. Here, we focus on an abstraction widely studied of this problem, the two-dimensional hydrophobic-polar protein folding problem (2D HP PFP). Mathematical optimal model of free energy of protein is established. Native conformations are often sought using stochastic sampling methods, but which are slow. The elastic net (EN) algorithm is one of fast deterministic methods as travelling salesman problem (TSP) strategies. However, it cannot be applied directly to protein folding problem, because of fundamental differences in the two types of problems. In this paper, how the 2D HP protein folding problem can be framed in terms of TSP is shown. Combination of the modified elastic net algorithm and novel local search method is adopted to solve this problem. To our knowledge, this is the first application of EN algorithm to 2D HP model. The results indicate that our approach can find more optimal conformations and is simple to implement, computationally efficient and fast.     

A study on rule extraction from several combined neural networks

The problem of rule extraction from neural networks is NP-hard. This work presents a new technique to extract "if-then-else" rules from ensembles of DIMLP neural networks. Rules are extracted in polynomial time with respect to the dimensionality of the problem, the number of examples, and the size of the resulting network. Further, the degree of matching between extracted rules and neural network responses is 100%. Ensembles of DIMLP networks were trained on four data sets in the public domain. Extracted rules were on average significantly more accurate than those extracted from C4.5 decision trees.     

Automated computer evaluation of time-varying cryomicroscopical images

A system has been developed to perform automatic computerized recognition, tracking, and quantitative morphological analysis of viable cells in freezing solutions. Cryomicroscopical image sequences of freezing cells are digitized and analyzed by computer. Image-processing techniques are used which are insensitive to contrast fluctuations from image to image, and which perform well even in noisy, cluttered images. The generalized Hough transform is used for shape detection, and a heuristic graph-search boundary completion algorithm is applied for shape extraction. The extracted cell shape may be analyzed for changes in cross-sectional area, perimeter length, shape deformation, and other metrics of interest. Knowledge from the shapeextraction phase is used to form a prediction of what shape the cell will be in the next image frame, and thus what to look for in the next shape-detection phase. This combination of knowledge-feedback with a powerful shape-detection technique produces an automatic, dynamic shape-recognition scheme capable of accurate recognition and analysis of the cells regardless of how deformed they may become during the freezing sequence. Example performance of the system is illustrated for a series of micrographs of freezing granulocytes.     

Image segmentation using a neural network

An object extraction problem based on the Gibbs Random Field model is discussed. The Maximum a'posteriori probability (MAP) estimate of a scene based on a noise-corrupted realization is found to be computationally exponential in nature. A neural network, which is a modified version of that of Hopfield, is suggested for solving the problem. A single neuron is assigned to every pixel. Each neuron is supposed to be connected only to all of its nearest neighbours. The energy function of the network is designed in such a way that its minimum value corresponds to the MAP estimate of the scene. The dynamics of the network are described. A possible hardware realization of a neuron is also suggested. The technique is implemented on a set of noisy images and found to be highly robust and immune to noise.     

Vision-based force measurement using neural networks for biological cell microinjection

This paper presents a vision-based force measurement method using an artificial neural network model. The proposed model is used for measuring the applied load to a spherical biological cell during micromanipulation process. The devised vision-based method is most useful when force measurement capability is required, but it is very challenging or even infeasible to use a force sensor. Artificial neural networks in conjunction with image processing techniques have been used to estimate the applied load to a cell. A bio-micromanipulation system capable of force measurement has also been established in order to collect the training data required for the proposed neural network model. The geometric characterization of zebrafish embryos membranes has been performed during the penetration of the micropipette prior to piercing. The geometric features are extracted from images using image processing techniques. These features have been used to describe the shape and quantify the deformation of the cell at different indentation depths. The neural network is trained by taking the visual data as the input and the measured corresponding force as the output. Once the neural network is trained with sufficient number of data, it can be used as a precise sensor in bio-micromanipulation setups. However, the proposed neural network model is applicable for indentation of any other spherical elastic object. The results demonstrate the capability of the proposed method. The outcomes of this study could be useful for measuring force in biological cell micromanipulation processes such as injection of the mouse oocyte/embryo.     

Artificial neural networks for blur identification and restoration of nonlinearly degraded images.

In this paper, an image restoration algorithm is proposed to identify noncausal blur function. Image degradation processes include both linear and nonlinear phenomena. A neural network model combining an adaptive auto-associative network with a random Gaussian process is proposed to restore the blurred image and blur function simultaneously. The noisy and blurred images are modeled as continuous associative networks, whereas auto-associative part determines the image model coefficients and the hetero-associative part determines the blur function of the system. The self-organization like structure provides the potential solution of the blind image restoration problem. The estimation and restoration are implemented by using an iterative gradient based algorithm to minimize the error function.     

Decreased astrocytic thrombospondin‐1 secretion after chronic ammonia treatment reduces the level of synaptic proteins: in vitro and in vivo studies

Chronic hepatic encephalopathy (CHE) is a major complication in patients with severe liver disease. Elevated blood and brain ammonia levels have been implicated in its pathogenesis, and astrocytes are the principal neural cells involved in this disorder. Since defective synthesis and release of astrocytic factors have been shown to impair synaptic integrity in other neurological conditions, we examined whether thrombospondin‐1 (TSP‐1), an astrocytic factor involved in the maintenance of synaptic integrity, is also altered in CHE. Cultured astrocytes were exposed to ammonia (NH₄Cl, 0.5–2.5 mM) for 1–10 days, and TSP‐1 content was measured in cell extracts and culture media. Astrocytes exposed to ammonia exhibited a reduction in intra‐ and extracellular TSP‐1 levels. Exposure of cultured neurons to conditioned media from ammonia‐treated astrocytes showed a decrease in synaptophysin, PSD95, and synaptotagmin levels. Conditioned media from TSP‐1 over‐expressing astrocytes that were treated with ammonia, when added to cultured neurons, reversed the decline in synaptic proteins. Recombinant TSP‐1 similarly reversed the decrease in synaptic proteins. Metformin, an agent known to increase TSP‐1 synthesis in other cell types, also reversed the ammonia‐induced TSP‐1 reduction. Likewise, we found a significant decline in TSP‐1 level in cortical astrocytes, as well as a reduction in synaptophysin content in vivo in a rat model of CHE. These findings suggest that TSP‐1 may represent an important therapeutic target for CHE.

     

Parameters affecting substance P measurement in heart, lung, and skin

Substance P (SP), a neuropeptide that is widely distributed both peripherally and centrally, mediates several pathophysiological processes. Among current assays for SP, enzyme-linked immunosorbent assay (ELISA) and radioimmunoassay (RIA) have been most widely used. Several previous studies, mostly performed with nerve extracts or organ perfusates, determined that acidity of the extraction buffer as well as the number extractions performed constitute factors influencing accurate measurements. We used an ELISA protocol in this study to analyze methodological aspects of SP measurement in extracts from heart, skin, and lung. The extraction procedure had two steps, an acid extraction followed by a column extraction. We could effectively measure SP with extract from as little as 10 mg of tissue. For each tissue examined, different variables influenced the SP measured. For all tissues, the weight of tissue extracted was critical; the more tissue extracted, the lower the sensitivity of the assay. This problem could be overcome in skin by omitting the column extraction. When mechanical loses were considered (e.g., loss during extraction and SP retained by the column after elution), column extraction improved SP measurements only with lung tissue. The amount of SP remaining in the sample after the first extraction also varied among tissues. The first acid extraction effectively isolated 80% of total SP from skin. In contrast, the first extraction with lung tissue recovered only 58%. Because both acid and heat effectively release SP from nerve endings, this could reflect the presence of non-neuronal SP, especially in lung. High-dose capsaicin treatment, which depletes SP in nerve endings, caused 42% loss of SP in skin independent of amount of tissue extracted Our results suggest that a second acid extraction of tissue should be performed and that column extraction is clearly detrimental with skin samples.     

Sensori-motor transformations in the brain (with a critique of the tensor theory of cerebellum)

Section 1 lists 12 points which must be addressed by neural models of sensorimotor coordination. Section 2 addresses the problem of extrapolating motor output from noisy data or from sensory input. The Pellionisz-Llinas cerebellar lookahead module addresses this problem for the noise-free case, and we suggest theoretical and experimental tests of the model; we then suggest the investigation of neural analogs of the Kalman-Bucy filter. Section 3 offers a brief exposition of mechanics in a tensor framework to provide the irreducible minimum of mathematical machinery to evaluate the Pellionisz-Llinás tensor theory of brain function and to suggest fruitful new hypotheses. Our critique of this theory in section 4 leads us to conclude that what they offer is based on metaphorical use of terminology from Euclidean tensors, not on rigorous application of the mathematics of tensor analysis. The central claim of their theory--that the input is a covariant intention vector transformed by a metric tensor encoded in the cerebellum to a contravariant execution vector--has not been substantiated and probably cannot be substantiated. However, we do point the way to further use of tensor analysis in the study of neural control of movement. The concluding section then returns to the points raised in section 1 with a highly selective survey of models of cerebellum and tectum.     

Artificial neural networks and their use in quantitative pathology

A brief general introduction to artificial neural networks is presented, examining in detail the structure and operation of a prototype net developed for the solution of a simple pattern recognition problem in quantitative pathology. The process by which a neural network learns through example and gradually embodies its knowledge as a distributed representation is discussed, using this example. The application of neurocomputer technology to problems in quantitative pathology is explored, using real-world and illustrative examples. Included are examples of the use of artificial neural networks for pattern recognition, database analysis and machine vision. In the context of these examples, characteristics of neural nets, such as their ability to tolerate ambiguous, noisy and spurious data and spontaneously generalize from known examples to handle unfamiliar cases, are examined. Finally, the strengths and deficiencies of a connectionist approach are compared to those of traditional symbolic expert system methodology. It is concluded that artificial neural networks, used in conjunction with other nonalgorithmic artificial intelligence techniques and traditional algorithmic processing, may provide useful software engineering tools for the development of systems in quantitative pathology.     

Predicting the eye fixation locations in the gray scale images in the visual scenes with different semantic contents

In recent years, there has been considerable interest in visual attention models (saliency map of visual attention). These models can be used to predict eye fixation locations, and thus will have many applications in various fields which leads to obtain better performance in machine vision systems. Most of these models need to be improved because they are based on bottom-up computation that does not consider top-down image semantic contents and often does not match actual eye fixation locations. In this study, we recorded the eye movements (i.e., fixations) of fourteen individuals who viewed images which consist natural (e.g., landscape, animal) and man-made (e.g., building, vehicles) scenes. We extracted the fixation locations of eye movements in two image categories. After extraction of the fixation areas (a patch around each fixation location), characteristics of these areas were evaluated as compared to non-fixation areas. The extracted features in each patch included the orientation and spatial frequency. After feature extraction phase, different statistical classifiers were trained for prediction of eye fixation locations by these features. This study connects eye-tracking results to automatic prediction of saliency regions of the images. The results showed that it is possible to predict the eye fixation locations by using of the image patches around subjects’ fixation points.     

A Modified Verhoeff-Van Gieson Elastin Histochemical Stain to Enable Pulmonary Arterial Hypertension Model Characterization

Optimal histochemical staining is critical to ensure excellent quality stained sections to enable light microscopic and histomorphometric image analysis. Verhoeff-van Gieson is the most widely used histochemical stain for the visualization of vascular elastic fibers. However, it is notoriously difficult to differentiate fine elastic fibers of small vasculature to enable histomorphometric image analysis, especially in organs such as the lung. A tissue fixation procedure of 10% neutral buffered formalin with subsequent fixation in 70% ethanol further compounds the problem of small vessel staining and identification. Therefore, a modified Verhoeff’s elastin stain was developed as a reliable method to optimally highlight the internal and external elastic laminae of small arteries (50-100 µm external diameter) and intra-acinar vessels (10-50 µm external diameter) in 3 µm thick lung tissue sections from models of pulmonary arterial hypertension. This modified Verhoeff’s elastin stain demonstrated well-defined staining of fine elastic fibers of pulmonary blood vessels enabling subsequent histomorphometric image analysis of vessel wall thickness in small arteries and intra-acinar vessels. In conclusion, modification of the standard Verhoeff-van Gieson histochemical stain is needed to visualize small caliber vessels’ elastic fibers especially in tissues fixed in 10% neutral buffered formalin followed by additional fixation in 70% ethanol.Key words: Histochemical stain, histomorphology, lung, Verhoeff-van Gieson, elastin     

Learning Pitch with STDP: A Computational Model of Place and Temporal Pitch Perception Using Spiking Neural Networks

Pitch perception is important for understanding speech prosody, music perception, recognizing tones in tonal languages, and perceiving speech in noisy environments. The two principal pitch perception theories consider the place of maximum neural excitation along the auditory nerve and the temporal pattern of the auditory neurons’ action potentials (spikes) as pitch cues. This paper describes a biophysical mechanism by which fine-structure temporal information can be extracted from the spikes generated at the auditory periphery. Deriving meaningful pitch-related information from spike times requires neural structures specialized in capturing synchronous or correlated activity from amongst neural events. The emergence of such pitch-processing neural mechanisms is described through a computational model of auditory processing. Simulation results show that a correlation-based, unsupervised, spike-based form of Hebbian learning can explain the development of neural structures required for recognizing the pitch of simple and complex tones, with or without the fundamental frequency. The temporal code is robust to variations in the spectral shape of the signal and thus can explain the phenomenon of pitch constancy.     

A Discrete Fruit Fly Optimization Algorithm for the Traveling Salesman Problem

The fruit fly optimization algorithm (FOA) is a newly developed bio-inspired algorithm. The continuous variant version of FOA has been proven to be a powerful evolutionary approach to determining the optima of a numerical function on a continuous definition domain. In this study, a discrete FOA (DFOA) is developed and applied to the traveling salesman problem (TSP), a common combinatorial problem. In the DFOA, the TSP tour is represented by an ordering of city indices, and the bio-inspired meta-heuristic search processes are executed with two elaborately designed main procedures: the smelling and tasting processes. In the smelling process, an effective crossover operator is used by the fruit fly group to search for the neighbors of the best-known swarm location. During the tasting process, an edge intersection elimination (EXE) operator is designed to improve the neighbors of the non-optimum food location in order to enhance the exploration performance of the DFOA. In addition, benchmark instances from the TSPLIB are classified in order to test the searching ability of the proposed algorithm. Furthermore, the effectiveness of the proposed DFOA is compared to that of other meta-heuristic algorithms. The results indicate that the proposed DFOA can be effectively used to solve TSPs, especially large-scale problems.     

Stability analysis of higher-order neural networks for combinatorial optimization

Recurrent neural networks with higher order connections, from here on referred to as higher-order neural networks (HONNs), may be used for the solution of combinatorial optimization problems. In Ref. 5 a mapping of the traveling salesman problem (TSP) onto a HONN of arbitrary order was developed, thereby creating a family of related networks that can be used to solve the TSP. In this paper, we explore the trade-off between network complexity and quality of solution that is made available by the HONN mapping of the TSP. The trade-off is investigated by undertaking an analysis of the stability of valid solutions to the TSP in a HONN of arbitrary order. The techniques used to perform the stability analysis are not new, but have been widely used elsewhere in the literature. The original contribution in this paper is the application of these techniques to a HONN of arbitrary order used to solve the TSP. The results of the stability analysis show that the quality of solution is improved by increasing the network complexity, as measured by the order of the network. Furthermore, it is shown that the Hopfield network, as the simplest network in the family of higher-order networks, is expected to produce the poorest quality of solution.     

Deep self-supervised transformation learning for leukocyte classification

The scarcity of training annotation is one of the major challenges for the application of deep learning technology in medical image analysis. Recently, self-supervised learning provides a powerful solution to alleviate this challenge by extracting useful features from a large number of unlabeled training data. In this article, we propose a simple and effective self-supervised learning method for leukocyte classification by identifying the different transformations of leukocyte images, without requiring a large batch of negative sampling or specialized architectures. Specifically, a convolutional neural network backbone takes different transformations of leukocyte image as input for feature extraction. Then, a pretext task of self-supervised transformation recognition on the extracted feature is conducted by a classifier, which helps the backbone learn useful representations that generalize well across different leukocyte types and datasets. In the experiment, we systematically study the effect of different transformation compositions on useful leukocyte feature extraction. Compared with five typical baselines of self-supervised image classification, experimental results demonstrate that our method performs better in different evaluation protocols including linear evaluation, domain transfer, and finetuning, which proves the effectiveness of the proposed method.     

LeukocyteMask: An automated localization and segmentation method for leukocyte in blood smear images using deep neural networks

Digital pathology and microscope image analysis is widely used in comprehensive studies of cell morphology. Identification and analysis of leukocytes in blood smear images, acquired from bright field microscope, are vital for diagnosing many diseases such as hepatitis, leukaemia and acquired immune deficiency syndrome (AIDS). The major challenge for robust and accurate identification and segmentation of leukocyte in blood smear images lays in the large variations of cell appearance such as size, colour and shape of cells, the adhesion between leukocytes (white blood cells, WBCs) and erythrocytes (red blood cells, RBCs), and the emergence of substantial dyeing impurities in blood smear images. In this paper, an end‐to‐end leukocyte localization and segmentation method is proposed, named LeukocyteMask, in which pixel‐level prior information is utilized for supervisor training of a deep convolutional neural network, which is then employed to locate the region of interests (ROI) of leukocyte, and finally segmentation mask of leukocyte is obtained based on the extracted ROI by forward propagation of the network. Experimental results validate the effectiveness of the propose method and both the quantitative and qualitative comparisons with existing methods indicate that LeukocyteMask achieves a state‐of‐the‐art performance for the segmentation of leukocyte in terms of robustness and accuracy .