Stimulus sampling as an exploration mechanism for fast reinforcement learning

Reinforcement learning in neural networks requires a mechanism for exploring new network states in response to a single, nonspecific reward signal. Existing models have introduced synaptic or neuronal noise to drive this exploration. However, those types of noise tend to almost average out—precluding or significantly hindering learning —when coding in neuronal populations or by mean firing rates is considered. Furthermore, careful tuning is required to find the elusive balance between the often conflicting demands of speed and reliability of learning. Here we show that there is in fact no need to rely on intrinsic noise. Instead, ongoing synaptic plasticity triggered by the naturally occurring online sampling of a stimulus out of an entire stimulus set produces enough fluctuations in the synaptic efficacies for successful learning. By combining stimulus sampling with reward attenuation, we demonstrate that a simple Hebbian-like learning rule yields the performance that is very close to that of primates on visuomotor association tasks. In contrast, learning rules based on intrinsic noise (node and weight perturbation) are markedly slower. Furthermore, the performance advantage of our approach persists for more complex tasks and network architectures. We suggest that stimulus sampling and reward attenuation are two key components of a framework by which any single-cell supervised learning rule can be converted into a reinforcement learning rule for networks without requiring any intrinsic noise source. This work was supported by the Swiss National Science Foundation grant K-32K0-118084.     

Evidence for cryptic speciation in Carchesium polypinum Linnaeus, 1758 (Ciliophora: Peritrichia) inferred from mitochondrial, nuclear, and morphological markers

Protist diversity is currently a much debated issue in eukaryotic microbiology. Recent evidence suggests that morphological and genetic diversity might be decoupled in some groups of protists, including ciliates, and that these organisms might be much more diverse than their morphology implies. We sought to assess the genetic and morphological diversity of Carchesium polypinum, a widely distributed peritrich ciliate. The mitochondrial marker cytochrome c oxidase subunit I and the nuclear small subunit ribosomal RNA were used to examine genetic diversity. For the morphological assessment, live microscopy and Protargol staining were used. The mitochondrial marker revealed six robust, deeply diverging, and strongly supported clades, while the nuclear gene was congruent for three of these clades. There were no major differences among individuals from the different clades in any of the morphological features examined. Thus, the underlying genetic diversity in C. polypinum is greater than what its morphology suggests, indicating that morphology and genetics are not congruent in this organism. Furthermore, because the clades identified by the mitochondrial marker are so genetically diverse and are confirmed by a conserved nuclear marker in at least three cases, we propose that C. polypinum be designated as a "cryptic species complex." Our results provide another example where species diversity can be underestimated in microbial eukaryotes when using only morphological criteria to estimate species richness.     

Fatty acids derived from royal jelly are modulators of estrogen receptor functions

Royal jelly (RJ) excreted by honeybees and used as a nutritional and medicinal agent has estrogen-like effects, yet the compounds mediating these effects remain unidentified. The possible effects of three RJ fatty acids (FAs) (10-hydroxy-2-decenoic-10H2DA, 3,10-dihydroxydecanoic-3,10DDA, sebacic acid-SA) on estrogen signaling was investigated in various cellular systems. In MCF-7 cells, FAs, in absence of estradiol (E(2)), modulated the estrogen receptor (ER) recruitment to the pS2 promoter and pS2 mRNA levels via only ERβ but not ERα, while in presence of E(2) FAs modulated both ERβ and ERα. Moreover, in presence of FAs, the E(2)-induced recruitment of the EAB1 co-activator peptide to ERα is masked and the E(2)-induced estrogen response element (ERE)-mediated transactivation is inhibited. In HeLa cells, in absence of E(2), FAs inhibited the ERE-mediated transactivation by ERβ but not ERα, while in presence of E(2), FAs inhibited ERE-activity by both ERβ and ERα. Molecular modeling revealed favorable binding of FAs to ERα at the co-activator-binding site, while binding assays showed that FAs did not bind to the ligand-binding pocket of ERα or ERβ. In KS483 osteoblasts, FAs, like E(2), induced mineralization via an ER-dependent way. Our data propose a possible molecular mechanism for the estrogenic activities of RJ's components which, although structurally entirely different from E(2), mediate estrogen signaling, at least in part, by modulating the recruitment of ERα, ERβ and co-activators to target genes.     

Analogies and surprising differences between recombinant nitric oxide synthase-like proteins from Staphylococcus aureus and Bacillus anthracis in their interactions with l-arginine analogs and iron ligands

Genome sequencing has recently shown the presence of genes coding for NO-synthase (NOS)-like proteins in bacteria. The roles of these proteins remain unclear. The interactions of a series of l-arginine (l-arg) analogs and iron ligands with two recombinant NOS-like proteins from Staphylococcus aureus (saNOS) and Bacillus anthracis (baNOS) have been studied by UV–visible spectroscopy. SaNOS and baNOS in their ferric native state, as well as their complexes with l-arg analogs and with various ligands, exhibit spectral characteristics highly similar to the corresponding complexes of heme-thiolate proteins such as cytochromes P450 and NOSs. However, saNOS greatly differs from baNOS at the level of three main properties: (i) native saNOS mainly exists under an hexacoordinated low-spin ferric state whereas native baNOS is mainly high-spin, (ii) the addition of tetrahydrobiopterin (H4B) or H4B analogs leads to an increase of the affinity of l-arg for saNOS but not for baNOS, and (iii) saNOS Fe^II, contrary to baNOS, binds relatively bulky ligands such as nitrosoalkanes and tert-butylisocyanide. Thus, saNOS exhibits properties very similar to those of the oxygenase domain of inducible NOS (iNOS_oxy) not containing H4B, as expected for a NOSoxy-like protein that does not contain H4B. By contrast, the properties of baNOS which look like those of H4B-containing iNOS_oxy are unexpected for a NOS-like protein not containing H4B. The origin of these surprising properties of baNOS remains to be determined.     

Smc5/6 is required for repair at collapsed replication forks

In eukaryotes, three pairs of structural-maintenance-of-chromosome (SMC) proteins are found in conserved multisubunit protein complexes required for chromosomal organization. Cohesin, the Smc1/3 complex, mediates sister chromatid cohesion while two condensin complexes containing Smc2/4 facilitate chromosome condensation. Smc5/6 scaffolds an essential complex required for homologous recombination repair. We have examined the response of smc6 mutants to the inhibition of DNA replication. We define homologous recombination-dependent and -independent functions for Smc6 during replication inhibition and provide evidence for a Rad60-independent function within S phase, in addition to a Rad60-dependent function following S phase. Both genetic and physical data show that when forks collapse (i.e., are not stabilized by the Cds1Chk2 checkpoint), Smc6 is required for the effective repair of resulting lesions but not for the recruitment of recombination proteins. We further demonstrate that when the Rad60-dependent, post-S-phase Smc6 function is compromised, the resulting recombination-dependent DNA intermediates that accumulate following release from replication arrest are not recognized by the G2/M checkpoint.     

Host immunity increases Mycobacterium tuberculosis reliance on cytochrome bd oxidase

In order to sustain a persistent infection, Mycobacterium tuberculosis (Mtb) must adapt to a changing environment that is shaped by the developing immune response. This necessity to adapt is evident in the flexibility of many aspects of Mtb metabolism, including a respiratory chain that consists of two distinct terminal cytochrome oxidase complexes. Under the conditions tested thus far, the bc₁/aa₃ complex appears to play a dominant role, while the alternative bd oxidase is largely redundant. However, the presence of two terminal oxidases in this obligate pathogen implies that respiratory requirements might change during infection. We report that the cytochrome bd oxidase is specifically required for resisting the adaptive immune response. While the bd oxidase was dispensable for growth in resting macrophages and the establishment of infection in mice, this complex was necessary for optimal fitness after the initiation of adaptive immunity. This requirement was dependent on lymphocyte-derived interferon gamma (IFNγ), but did not involve nitrogen and oxygen radicals that are known to inhibit respiration in other contexts. Instead, we found that ΔcydA mutants were hypersusceptible to the low pH encountered in IFNγ-activated macrophages. Unlike wild type Mtb, cytochrome bd-deficient bacteria were unable to sustain a maximal oxygen consumption rate (OCR) at low pH, indicating that the remaining cytochrome bc₁/aa₃ complex is preferentially inhibited under acidic conditions. Consistent with this model, the potency of the cytochrome bc₁/aa₃ inhibitor, Q203, is dramatically enhanced at low pH. This work identifies a critical interaction between host immunity and pathogen respiration that influences both the progression of the infection and the efficacy of potential new TB drugs.     

Isolation and characterization of vitellin from the fruitfly,Dacus oleae

Vitellin was isolated from mature eggs of Dacus oleae. A combination of anion-exchange chromatography and gel filtration was used for purification of the protein. The molecular weight of isolated vitellin, as determined by Sephacryl S-300 chromatography, was approximately 300,000. Electrophoresis on SDS-polyacrylamide gels demonstrated the presence of vitellin subunits with molecular weight of 47,000 and 49,000. Isoelectric focusing on polyacrylamide gels revealed a series of polypeptides with isoelectric points covering an acidic pH region of 5.7 to 6.2. Immunodiffusion, immunoelectrophoresis, and immunoblotting were used for further characterization of vitellin.     

Cellular characteristics of long-term cultured rat parotid acinar cells

Summary We have successfully maintained and biochemically characterized differentiated rat parotid acinar cells cultured for long periods (6 mo.). The cells were cultured on a reconstituted basement membrane matrix in a medium containing a variety of agents that promote cellular proliferation and differentiation. The cultured cells retain the characteristics of the parental parotid acinar cells. They exhibit both secretory granules and abundant cellular organelles required for protein synthesis and secretion. In situ hybridization and immunocytochemistry demonstrate high levels of proline-rich protein mRNA and protein, and lower levels of amylase mRNA and protein, in their cytoplasm. These findings suggest that rat parotid acinar cells can be maintained in a differentiated state in vitro for long periods, and can serve as a useful model system for studying the regulation of exocrine secretory processes.     

Immunohistochemical localization of nerve growth factor (NGF) and NGF receptor (NGF-R) in the developing first molar tooth of the rat

Nerve growth factor (NGF) is a well established target-derived trophic factor supporting sympathetic and sensory innervation in the peripheral tissues as well as cholinergic innervation in the brain. Despite its name, NGF may have broader biological functions early in development in a wide range of non-neuronal differentiating cells. The many effects of NGF are directly dependent on initial binding of NGF to specific plasma membrane receptors on target cells. Here we use immunohistochemical methods to show that NGF and its receptor (NGF-R) are localized in a variety of embryonic epithelial and mesenchymal cells in the rat developing molar tooth. Dental cells known to play important roles in morphogenesis and inductive tissue interactions show NGF-like reactivity. Thus, labelling is seen in epithelial preameloblasts and mesenchymal odontoblasts. We also show a transient expression of NGF-R in restricted parts of the dental epithelium (inner dental epithelium) and dental mesenchyme differentiating cells (post-mitotic, polarizing odontoblasts). The expression patterns of NGF are different to those of NGF-R during embryogenesis and this is illustrated in detail in the developing tooth. The histochemical findings reported here support the notion that NGF may have multiple roles during morphogenetic and cytodifferentiation events in the tooth.     

A general modeling framework for open wildlife populations based on the Polya tree prior

Wildlife monitoring for open populations can be performed using a number of different survey methods. Each survey method gives rise to a type of data and, in the last five decades, a large number of associated statistical models have been developed for analyzing these data. Although these models have been parameterized and fitted using different approaches, they have all been designed to either model the pattern with which individuals enter and/or exit the population, or to estimate the population size by accounting for the corresponding observation process, or both. However, existing approaches rely on a predefined model structure and complexity, either by assuming that parameters linked to the entry and exit pattern (EEP) are specific to sampling occasions, or by employing parametric curves to describe the EEP. Instead, we propose a novel Bayesian nonparametric framework for modeling EEPs based on the Polya tree (PT) prior for densities. Our Bayesian nonparametric approach avoids overfitting when inferring EEPs, while simultaneously allowing more flexibility than is possible using parametric curves. Finally, we introduce the replicate PT prior for defining classes of models for these data allowing us to impose constraints on the EEPs, when required. We demonstrate our new approach using capture–recapture, count, and ring-recovery data for two different case studies.