Reinforcement Learning on Slow Features of High-Dimensional Input Streams

Humans and animals are able to learn complex behaviors based on a massive stream of sensory information from different modalities. Early animal studies have identified learning mechanisms that are based on reward and punishment such that animals tend to avoid actions that lead to punishment whereas rewarded actions are reinforced. However, most algorithms for reward-based learning are only applicable if the dimensionality of the state-space is sufficiently small or its structure is sufficiently simple. Therefore, the question arises how the problem of learning on high-dimensional data is solved in the brain. In this article, we propose a biologically plausible generic two-stage learning system that can directly be applied to raw high-dimensional input streams. The system is composed of a hierarchical slow feature analysis (SFA) network for preprocessing and a simple neural network on top that is trained based on rewards. We demonstrate by computer simulations that this generic architecture is able to learn quite demanding reinforcement learning tasks on high-dimensional visual input streams in a time that is comparable to the time needed when an explicit highly informative low-dimensional state-space representation is given instead of the high-dimensional visual input. The learning speed of the proposed architecture in a task similar to the Morris water maze task is comparable to that found in experimental studies with rats. This study thus supports the hypothesis that slowness learning is one important unsupervised learning principle utilized in the brain to form efficient state representations for behavioral learning.     

Memory Storage Fidelity in the Hippocampal Circuit: The Role of Subregions and Input Statistics

In the last decades a standard model regarding the function of the hippocampus in memory formation has been established and tested computationally. It has been argued that the CA3 region works as an auto-associative memory and that its recurrent fibers are the actual storing place of the memories. Furthermore, to work properly CA3 requires memory patterns that are mutually uncorrelated. It has been suggested that the dentate gyrus orthogonalizes the patterns before storage, a process known as pattern separation. In this study we review the model when random input patterns are presented for storage and investigate whether it is capable of storing patterns of more realistic entorhinal grid cell input. Surprisingly, we find that an auto-associative CA3 net is redundant for random inputs up to moderate noise levels and is only beneficial at high noise levels. When grid cell input is presented, auto-association is even harmful for memory performance at all levels. Furthermore, we find that Hebbian learning in the dentate gyrus does not support its function as a pattern separator. These findings challenge the standard framework and support an alternative view where the simpler EC-CA1-EC network is sufficient for memory storage.     

Finite volume analysis of temperature effects induced by active MRI implants: 2. Defects on active MRI implants causing hot spots

Background

Active magnetic resonance imaging implants, for example stents, stent grafts or vena cava filters, are constructed as wireless inductively coupled transmit and receive coils. They are built as a resonator tuned to the Larmor frequency of a magnetic resonance system. The resonator can be added to or incorporated within the implant. This technology can counteract the shielding caused by eddy currents inside the metallic implant structure. This may allow getting diagnostic information of the implant lumen (in stent stenosis or thrombosis for example). The electro magnetic rf-pulses during magnetic resonance imaging induce a current in the circuit path of the resonator. A by material fatigue provoked partial rupture of the circuit path or a broken wire with touching surfaces can set up a relatively high resistance on a very short distance, which may behave as a point-like power source, a hot spot, inside the body part the resonator is implanted to. This local power loss inside a small volume can reach ¼ of the total power loss of the intact resonating circuit, which itself is proportional to the product of the resonator volume and the quality factor and depends as well from the orientation of the resonator with respect to the main magnetic field and the imaging sequence the resonator is exposed to.

Methods

First an analytical solution of a hot spot for thermal equilibrium is described. This analytical solution with a definite hot spot power loss represents the worst case scenario for thermal equilibrium inside a homogeneous medium without cooling effects. Starting with this worst case assumptions additional conditions are considered in a numerical simulation, which are more realistic and may make the results less critical. The analytical solution as well as the numerical simulations use the experimental experience of the maximum hot spot power loss of implanted resonators with a definite volume during magnetic resonance imaging investigations. The finite volume analysis calculates the time developing temperature maps for the model of a broken linear metallic wire embedded in tissue. Half of the total hot spot power loss is assumed to diffuse into both wire parts at the location of a defect. The energy is distributed from there by heat conduction. Additionally the effect of blood perfusion and blood flow is respected in some simulations because the simultaneous appearance of all worst case conditions, especially the absence of blood perfusion and blood flow near the hot spot, is very unlikely for vessel implants.

Results

The analytical solution as worst case scenario as well as the finite volume analysis for near worst case situations show not negligible volumes with critical temperature increases for part of the modeled hot spot situations. MR investigations with a high rf-pulse density lasting below a minute can establish volumes of several cubic millimeters with temperature increases high enough to start cell destruction. Longer exposure times can involve volumes larger than 100 mm³. Even temperature increases in the range of thermal ablation are reached for substantial volumes. MR sequence exposure time and hot spot power loss are the primary factors influencing the volume with critical temperature increases. Wire radius, wire material as well as the physiological parameters blood perfusion and blood flow inside larger vessels reduce the volume with critical temperature increases, but do not exclude a volume with critical tissue heating for resonators with a large product of resonator volume and quality factor.

Conclusion

The worst case scenario assumes thermal equilibrium for a hot spot embedded in homogeneous tissue without any cooling due to blood perfusion or flow. The finite volume analysis can calculate the results for near and not close to worst case conditions. For both cases a substantial volume can reach a critical temperature increase in a short time. The analytical solution, as absolute worst case, points out that resonators with a small product of inductance volume and quality factor (Q V_ind < 2 cm³) are definitely save. Stents for coronary vessels or resonators used as tracking devices for interventional procedures therefore have no risk of high temperature increases. The finite volume analysis shows for sure that also conditions not close to the worst case reach physiologically critical temperature increases for implants with a large product of inductance volume and quality factor (Q V_ind > 10 cm³). Such resonators exclude patients from exactly the MRI investigation these devices are made for.     

Provisional stenting in the real world: results in 1058 consecutive patients undergoing percutaneous coronary angioplasty

OBJECTIVE: To study a strategy of aggressive coronary balloon angioplasty with provisional stenting in allcomers. In randomized trials, stenting has improved the outcome of patients undergoing coronary intervention. However, whether these results hold up in clinical practice is largely unknown. Furthermore, the results of balloon angioplasty have also improved dramatically. It is therefore essential to evaluate the current results of balloon angioplasty and to assess whether stents are required in all patients. METHODS: The authors prospectively studied the occurrence of death, myocardial infarction (MI) and target lesion revascularization (TLR) of a large consecutive group of patients undergoing aggressive balloon angioplasty with provisional stenting. None of the patients received a platelet glycoprotein IIb/IIIa receptor blocker. The results were compared with the outcome of routine stenting in recent randomized trials. RESULTS: Angioplasty was performed in 1058 patients of whom 369 (34.9%) received a stent. The angiographic success rate was 98.9%. During hospital stay, 4.8% of the patients suffered any cardiac event. At one-year follow-up, death occurred in 1.1%, MI in 3.3%, TLR in 12.4% and any event in 16.7% of the patients. Event-free survival at one-year was 82.3%. These results compare favorably with routine stenting in recent trials. CONCLUSIONS: Aggressive balloon angioplasty with provisional stenting yields excellent results in a general patient population.     

Evolution of cytochrome P-450 proteins [published erratum appears in Mol Biol Evol 1988 Mar;5(2):199]

Thirty-four cytochrome P-450 sequences from one bacterial and six vertebrate species have been aligned with the aid of a computer alignment algorithm. Phylogenetic trees were constructed using the unweighted-pair-group and neighbor-joining methods. The two trees differed at only a single branch point near the base of the tree. The cytochrome P-450 superfamily of proteins clustered into eight families and contained 16 gene-duplication events. The first gene duplication occurred approximately 1,360 Myr before the present (Mybp) and gave rise to cytochrome P-450s found in two different cellular organelles, the mitochondria and the endoplasmic reticulum. Both groups utilize cholesterol or its metabolites as substrates, implying that cholesterol existed greater than 1,360 Mybp. The fourth gene duplication (approximately 900 Mybp) gave rise to the drug-metabolizing P-450s. These proteins aid in the detoxification of foreign chemicals, as opposed to the metabolism of endogenous compounds. The importance of the capacity to metabolize drugs is reflected in 11 further gene duplications occurring in this lineage. The first occurred approximately 800 Mybp and gave rise to the two major P-450 families, the phenobarbital and 3-methylcholanthrene families. An apparent increase in the rate of cytochrome P-450 evolution is noted between the bird-mammal divergence (300 Mybp) and the mammalian radiation (75 Mybp).      

Natural occurrence of aflatoxin B<Subscript>1</Subscript> in some Indonesian food and feed products in Yogyakarta in year 1998–1999

A survey was conducted between 1998–1999 to evaluate the level of aflatoxin B₁ (AfB₁) contamination in some selected Indonesian food products, mainly peanuts and peanut products for sale in supermarkets or traditional markets in Yogyakarta, Indonesia. Quantitative analysis was carried out on 118 samples using the ELISA (Enzyme-Linked Immunosorbent Assay) technique. The results indicate that (61.1%) samples were contaminated with AfB₁ at range 2.0 to 249.0 μg/kg. Approximately 50% of the baby food products analysed were contaminated with AfB₁ and the maximum level found was 7.0 μg/kg. In corn products and fermented products, AfB₁ was detected in 66.7 and 50.0% of samples, respectively. A level as high as 5.6 μg/kg of AfB₁ was found in the corn and 6.0 μg/kg in fermented product. AfB₁ was also detected in all rice products, feed products, and other processed products at levels of up to 7.0, 27.0, and 26.0 μg/kg, respectively.     

Persistent directional cell migration requires ion transport proteins as direction sensors and membrane potential differences in order to maintain directedness

Background  

Ion transport proteins generate small electric fields that can induce directional cell motility; however, little is known about their mechanisms that lead to directedness. We investigated Na, K-ATPase (NaKA) and Na+/H+ exchanger isoforms (NHE1 and 3) in SaOS-2 and Calvarial osteoblasts, which present anode- and cathode- directed motility, during electrotaxis.