An information-theoretic approach to curiosity-driven reinforcement learning

We provide a fresh look at the problem of exploration in reinforcement learning, drawing on ideas from information theory. First, we show that Boltzmann-style exploration, one of the main exploration methods used in reinforcement learning, is optimal from an information-theoretic point of view, in that it optimally trades expected return for the coding cost of the policy. Second, we address the problem of curiosity-driven learning. We propose that, in addition to maximizing the expected return, a learner should choose a policy that also maximizes the learner's predictive power. This makes the world both interesting and exploitable. Optimal policies then have the form of Boltzmann-style exploration with a bonus, containing a novel exploration-exploitation trade-off which emerges naturally from the proposed optimization principle. Importantly, this exploration-exploitation trade-off persists in the optimal deterministic policy, i.e., when there is no exploration due to randomness. As a result, exploration is understood as an emerging behavior that optimizes information gain, rather than being modeled as pure randomization of action choices.     

In vitro differentiation of dendritic cells in the presence of prostaglandin E2 alters the IL-12/IL-23 balance and promotes differentiation of Th17 cells

PGE2, an endogenous lipid mediator released in inflammatory conditions, affects both dendritic cell (DC) differentiation and maturation. Whereas the effect of PGE2 on fully differentiated DC was studied extensively, little is known about its effects on DC differentiation. In this study, we show that bone marrow-derived DC generated in the presence of PGE2 (DCp) acquire a proinflammatory profile; produce higher levels of proinflammatory cytokines/chemokines; express higher levels of MHC class II, costimulatory molecules, and TLRs; and exhibit increased activation of the NF-kappaB-signaling pathway. In addition, DCp exhibit a different IL-12/IL-23 profile than DC generated in the absence of PGE2. The low IL-12 and high IL-23 production in LPS-stimulated DCp is associated with the down-regulation of p35 and the up-regulation of p19 expression, respectively. In agreement with the DCp proinflammatory phenotype and especially with the altered IL-12/IL-23 balance which strongly favors IL-23, DCp also affect T cell differentiation. In contrast to DC which favor Th1 differentiation, DCp promote Th17 and inhibit Th1/Th2 differentiation, in vitro and in vivo. Previous in vivo studies indicated that PGE2 had a proinflammatory effect, especially in models of autoimmune diseases. Our results suggest that the proinflammatory effects of PGE2 could be mediated, at least partially, through effects on differentiating DC and subsequent alterations in CD4+ T cell differentiation, resulting in the preferential development of pathogenic autoimmune Th17 cells.     

Binding is not enough: flexibility is needed for photocrosslinking of Lck kinase by benzophenone photoligands

Benzophenone photophores are employed widely for photoaffinity-labeling studies. Photolabeling with benzophenone, however, is hardly a routine experiment. Even when a photoprobe binds to its target, photocrosslinking does not necessarily occur. This is because photolabeling by benzophenone is affected by many factors other than target-binding, such as conformational flexibility of photoligand. Despite the widespread recognition of such complications, there has been no systematic study to assess the relative importance of individual factors that can affect photolabeling efficiency. In order to gain an insight into this problem, we conducted a structure-activity relationship (SAR) study of benzophenone photoligands for Lck kinase, in which photoligands with varying target-binding affinity and conformational flexibility were compared. The study found that binding-affinity, as indicated by kinase inhibitory potency, did not correlate with photolabeling efficiency. Instead, conformational flexibility was found to be the determining factor for efficient photolabeling by our photoligands. Implication of the current findings, in particular, with regard to selection and optimization of benzophenone photoligands, is discussed.     

Incidence, antibiotic resistance and clonal relations of MRSA strains isolated from a Romanian university hospital

Our aim was to estimate the frequency and characteristics ofmethicillin-resistant Staphylococcus aureus (MRSA) strains occurring in a Romanian teaching hospital. We retrospectively studied isolates from infected or colonized patients treated at the intensive care and surgical units during January 2004-December 2005. The antibiotic susceptibility of MRSA strains and the presence of mecA gene were determined. Consecutively occurring strains isolated through a three-month period were typed using pulsed field gel electrophoresis. A total of 423 S. aureus strains were identified, methicillin-resistance was detected in 211 (49.9%) strains. Most of them were multiresistant. One of the MRSA genotypes identified by PFGE was commonly recovered from patients treated in the intensive care unit. According to our results, MRSA strains were frequently isolated pathogens in our hospital and there is an urgent need to enhance infection control efforts.     

Function and molecular modeling of the interaction between human interleukin 6 and its HNK-1 oligosaccharide ligands.

Interleukin 6 (IL-6) is endowed with a lectin activity for oligosaccharide ligands possessing the HNK-1 epitope (3-sulfated glucuronic acid) found on some mammalian glycoprotein N-glycans (Cebo, C., Dambrouck, T., Maes, E., Laden, C., Strecker, G., Michalski, J. C., and Zanetta, J. P. (2001) J. Biol. Chem. 276, 5685-5691). Using high affinity oligosaccharide ligands, it is demonstrated that this lectin activity is responsible for the early dephosphorylation of tyrosine residues found on specific proteins induced by interleukin 6 in human resting lymphocytes. The gp130 glycoprotein, the signal-transducing molecule of the IL-6 pathway, is itself a molecule possessing the HNK-1 epitope. This indicates that IL-6 is a bi-functional molecule able to extracellularly associate its alpha-receptor with the gp130 surface complex. Computational modeling indicates that the lower energy conformers of the high affinity ligands of IL-6 have a common structure. Docking experiments of these conformers suggest that the carbohydrate recognition domain of IL-6 is localized in the domain previously identified as site 3 of IL-6 (Somers, W., Stahl, M., and Seehra, J. S. (1997) EMBO J. 16, 989-997), already known to be involved in interactions with gp130.     

Mitochondrial biogenesis: An update

In response to the energy demand triggered by developmental signals and environmental stressors, the cells launch the mitochondrial biogenesis process. This is a self‐renewal route, by which new mitochondria are generated from the ones already existing. Recently, considerable progress has been made in deciphering mitochondrial biogenesis‐related proteins and genes that function in health and in pathology‐related circumstances. However, an outlook on the intracellular mechanisms shared by the main players that drive mitochondrial biogenesis machinery is still missing. Here, we provide such a view by focusing on the following issues: (a) the role of mitochondrial biogenesis in homeostasis of the mitochondrial mass and function, (b) the signalling pathways beyond the induction/promotion, stimulation and inhibition of mitochondrial biogenesis and (c) the therapeutic applications aiming the repair and regeneration of defective mitochondrial biogenesis (in ageing, metabolic diseases, neurodegeneration and cancer). The review is concluded by the perspectives of mitochondrial medicine and research.     

Dual Split Protein-Based Fusion Assay Reveals that Mutations to Herpes Simplex Virus (HSV) Glycoprotein gB Alter the Kinetics of Cell-Cell Fusion Induced by HSV Entry Glycoproteins

Herpes simplex virus (HSV) entry and cell-cell fusion require glycoproteins gD, gH/gL, and gB. We propose that receptor-activated changes to gD cause it to activate gH/gL, which then triggers gB into an active form. We employed a dual split-protein (DSP) assay to monitor the kinetics of HSV glycoprotein-induced cell-cell fusion. This assay measures content mixing between two cells, i.e., fusion, within the same cell population in real time (minutes to hours). Titration experiments suggest that both gD and gH/gL act in a catalytic fashion to trigger gB. In fact, fusion rates are governed by the amount of gB on the cell surface. We then used the DSP assay to focus on mutants in two functional regions (FRs) of gB, FR1 and FR3. FR1 contains the fusion loops (FL1 and FL2), and FR3 encompasses the crown at the trimer top. All FL mutants initiated fusion very slowly, if at all. However, the fusion rates caused by some FL2 mutants increased over time, so that total fusion by 8 h looked much like that of the WT. Two distinct kinetic patterns, “slow and fast,” emerged for mutants in the crown of gB (FR3), again showing differences in initiation and ongoing fusion. Of note are the fusion kinetics of the gB syn mutant (LL871/872AA). Although this mutant was originally included as an ongoing high-rate-of-fusion control, its initiation of fusion is so rapid that it appears to be on a “hair trigger.” Thus, the DSP assay affords a unique way to examine the dynamics of HSV glycoprotein-induced cell fusion.     

Selective CB2 receptor activation ameliorates EAE by reducing Th17 differentiation and immune cell accumulation in the CNS

CB2, the cannabinoid receptor expressed primarily on hematopoietic cells and activated microglia, mediates the immunoregulatory functions of cannabinoids. The involvement of CB2 in EAE has been demonstrated by using both endogenous and exogenous ligands. We showed previously that CB2 selective agonists inhibit leukocyte rolling and adhesion to CNS microvasculature and ameliorate clinical symptom in both chronic and remitting-relapsing EAE models. Here we showed that Gp1a, a highly selective CB2 agonist, with a four log higher affinity for CB2 than CB1, reduced clinical scores and facilitated recovery in EAE in conjunction with long term reduction in demyelination and axonal loss. We also established that Gp1a affected EAE through at least two different mechanisms, i.e. an early effect on Th1/Th17 differentiation in peripheral immune organs, and a later effect on the accumulation of pathogenic immune cells in the CNS, associated with reductions in the expression of CNS and T cell chemokine receptors, chemokines and adhesion molecules. This is the first report on the in vivo CB2-mediated Gp1a inhibition of Th17/Th1 differentiation. We also confirmed the Gp1a-induced inhibition of Th17/Th1 differentiation in vitro, both in non-polarizing and polarizing conditions. The CB2-induced inhibition of Th17 differentiation is highly relevant in view of recent studies emphasizing the importance of pathogenic self-reactive Th17 cells in EAE/MS. In addition, the combined effect on Th17 differentiation and immune cell accumulation into the CNS, emphasize the relevance of CB2 selective ligands as potential therapeutic agents in neuroinflammation.     

Histochemistry of the epimerases

Summary On the assumption that electron transfer is involved in the process of epimerization and by using tetrazolium salts as an indicator, a histochemical reaction for the demonstration of UDPGal-4-epimerase has been developed.By using UDPG or UDPGal as substrates it has been possible to ascertain the direction of the reaction catalysed by this enzyme in various tissues in normal physiological conditions.Biochemical tests support the concept that the histochemical reactions recorded were the result of UDPGal-4-epimerase activity.