Generating adaptive behaviour within a memory-prediction framework

The Memory-Prediction Framework (MPF) and its Hierarchical-Temporal Memory implementation (HTM) have been widely applied to unsupervised learning problems, for both classification and prediction. To date, there has been no attempt to incorporate MPF/HTM in reinforcement learning or other adaptive systems; that is, to use knowledge embodied within the hierarchy to control a system, or to generate behaviour for an agent. This problem is interesting because the human neocortex is believed to play a vital role in the generation of behaviour, and the MPF is a model of the human neocortex.We propose some simple and biologically-plausible enhancements to the Memory-Prediction Framework. These cause it to explore and interact with an external world, while trying to maximize a continuous, time-varying reward function. All behaviour is generated and controlled within the MPF hierarchy. The hierarchy develops from a random initial configuration by interaction with the world and reinforcement learning only. Among other demonstrations, we show that a 2-node hierarchy can learn to successfully play "rocks, paper, scissors" against a predictable opponent.     

Intelligence is universal in life.

Behaviorists assume that living things memorize random atoms of information (engrams), "reinforced" by success, just as in the neo-Darwinian mutation-selection process. On the contrary we have to recognize the existence of organized and systematic responses in the learning process (Krechevsky). The animals seek desperately to "understand the meaning" of the world around them, by widening its context. Intelligence is not an exclusive prerogative of human mind. The minds of insects operate in the same way as that of man. Even a cell has a sort of intelligence (Cuenot). Consciousness is a state of awareness associated with enhanced mental activity. It occurs also in other "higher" animals (Thorpe). However human themselves are non conscious of their basic underlying motivations. Unconscious or ineffable knowledge plays a great role in shaping our world-view and in determining our influence on the Gaian hierarchy.     

Autogenesis: the evolution of replicative systems

Questions concerning the nature and origin of living systems and the hierarchy of their evolutionary processes are considered, and several problems which arise in connection with formerly developed theories--the autopoiesis of Maturana & Varela, the POL theory of Haukioja and the earlier developed evolutionary theory of Csányi--are discussed. The organization of living systems, the use of informational terms and the question how reproduction can enter into their characterization, problems of autonomy and identity are included in the list. It is suggested that replication--a copying process achieved by a special network of interrelatedness of components and component-producing processes that produces the same network as that which produced them--characterizes the living organization. The information "used" in this copying process, whether it is stored by special means or distributed in the whole system, is called replicative information. A theoretical model is introduced for the spontaneous emergence of replicative organization, called autogenesis. Autogenesis commences in a system by an organized "small" subsystem, referred to as AutoGenetic System Precursor (AGSP), which conveys replicative information to the system. During autogenesis, replicative information increases in system and compartment(s) form. A compartment is the co-replicating totality of components. The end state of autogenesis is an invariantly self-replicating organization which is unable to undergo further intrinsic organizational changes. It is suggested that replicative unities--such as living organisms--evolve via autogenesis. Levels of evolution emerge as a consequence of the relative autonomy of the autogenetic unities. On the next level they can be considered as components endowed with functions and a new autogenetic process can commence. Thus evolution proceeds towards its end state through the parallel autogenesis of the various levels. In terms of applications, ontogenesis is dealt with in detail as an autogenetic process as is the autogenesis of the biosphere and the global system.     

The evolution of the human mind and logic--mathematics structures

The evolution of the human mind is discussed based on: (i) the fact that living beings interchange matter, energy and information with their environment, (ii) an ontological interpretation of the "reality" of the quantum world, of which logic-mathematics structures are considered constitutive parts, (iii) recent theories according to which living beings are considered as dynamic complex systems organized by information, and (iv) the fact that the evolution of living beings is guided by information about the environment and by intrinsic information on living systems (auto-organization). Assuming the evolution of vision as a model we observe that the driving forces that directed the evolution of the eyes, as dynamic complex systems, are the information about the environment supplied by sunlight and the intrinsic information-gaining mechanism of living organisms. Thus, there exists a convergence toward a visual system with the greatest ability to obtain light information, like the human eye, and also a divergence that leads to the development of specific qualities in some species. As in the case of vision the evolution of the human mind-brain cannot be a consequence of factors unrelated to the object of its own functioning. The human mind was structured for the acquisition from reality of the logic-mathematics structures that underlie the whole universe and consequently of an internal representation of the external world and of its own self. Thus, these structures are, together with the intrinsic capacity for auto-organization of the human brain, the predominant driving force of the human mind evolution. Both factors are complementary.     

Ukhtomski? and the nature of the human "ego"]

A. Ukhtomski? is one of those Russian naturalists of the beginning of the 20th century (K. Tsiolkovski?, V. Vernadski?, K. Timiriazev, I. Pavlov), whose scientific work is characterized by a combination of the valuable contribution to a specific field of science and deep philosophical conceptualizations. Ukhtomski? enriched modern neurophysiology by the ideas on dominant--a stable focus of enhanced excitability, determining the organism reactions to the environmental stimuli. Unlike artificial experimental models, natural dominant--vector of the goal-directed behaviour--represents the needs of immediate satisfaction, dominating at the present moment. Formation of needs hierarchy, characteristic of the given personality, occurs during the process of people communication. Namely, another man serves to the subject a mirror, looking into which he realizes himself as a human being (K. Marx). By his teaching on the dominant, Ukhtomski? has revealed the dual nature of the personality's self-consciousness in the process of its perception of the surrounding people. He showed that involuntary projection of its dominating vital and social needs (motives, intentions) to the image of another man makes the latter only the "double" of the observer, which does not promote the development and enrichment of personality. Actually productive is only the perception which is motivated by the ideal need of learning another personality, sincere interest in him, what makes another person a "well-deserved colocutor", a source of new knowledge about people and himself. The ideas of Ukhtomski? on the nature of human "self" are close to ideological searchings of L. Tolsto? and F. Dostoevski?.(ABSTRACT TRUNCATED AT 250 WORDS)     

Human evolution: humanistic selection and looking to the future

Cultural evolution has predominated over biological evolution in modern man (Homo sapiens sapiens). Cultural evolution differs from biological evolution not only by inheritance of acquired characteristics but also, as is proposed in the present essay, by another kind of selection mechanism. Whereas selection in biological evolution is executed according to a criterion of reproductive success (the natural selection), selection in cultural evolution appears to be carried out according to human and humanistic criteria (success or fitness in meeting human needs, interests and humanistic values--"humanistic selection"). Many humanistic needs or values do not seem to be prerequisite for reproductive success, yet some of them (e.g. a need for freedom) seem to be inborn. Innateness, humanistic selection (decisive at a community level) and hierarchy of some human needs, interests and values appear to give cultural evolution a generally upward trend although long periods of stagnation or even regression may occur. Modern humans appear to be still at the early stage of their cultural evolution. A further cultural evolution of man appears to be, in contrast to biological evolution, predictable (with an optimistic outlook) and testable. The problem is that the hopeful result of this test will probably be known only in the fairly remote future provided that this species will not become extinct before that.     

基于生态系统服务供需的景感尺度特征分析和应用

自然生态系统与人类感知的交互作用是景感生态学关注的焦点,与生态系统服务的供需紧密关联。生态系统、生态过程、生态系统服务以及人的需求均具有一定的时空尺度特征,因此,由这些过程产生的"景感"也具有时空尺度特征,并依赖于这个过程的尺度特征相互作用。梳理生态系统服务供需关系与景感营造规划的尺度特征,探索二者相互联系,进而提出基于景感生态学理论,通过景感营造实现生态系统服务供需平衡的多尺度实现路径。从空间维度,体现从"社区-城市-流域-区域"的多尺度景感规划框架,从应用维度,体现从生态系统服务需求探索-关键指标体系识别-多源谜码数据感知-景感营造与规划设计的具体流程。通过多尺度的联合景感营造,调整自然供给水平与人类不同尺度的生态系统服务需求以期达到平衡,从而实现人类生存环境福利与居民福祉提升的目标。在此基础上,结合雄安新区规划建设实例,从多尺度景感营造的角度为城市绿色生态建设与可持续发展提出优化建议。主要结论：（1）基于"社区-城市-流域-区域"的多尺度景感分析框架,将景感要素与生态系统服务供需思想融入城市规划,在不同尺度上趋善优化,对于实现可持续发展城市规划和管理具有重要指示意义;（2）雄安新区规划设计需更系统和细致地考虑人类感知需求,通过多尺度景感营造设计,共同实现雄安新区的可持续发展与人居环境提升。其中,在社区尺度,需重点关注社区植被配置和居民的休闲游憩;在新区尺度重点关注绿地系统网络构建;在大清河流域尺度重点关注河流健康与水文调节;在京津冀区域尺度需注重山体通风廊道与生态屏障的构建与优化。     

Ontogenetic and gender-related variation in the isotopic niche within and between three species of fur seals (genus <Emphasis Type="Italic">Arctocephalus</Emphasis>)

Differences in aggressiveness when competing for environmental resources are the main factor leading to social hierarchy in group living fish. Social status acquired is related to changes in physiological parameters, as metabolic rate. Habitat variation can interfere with aggressive behaviour and promote changes in physiological parameters associated with social status. The primary goal of our study was to investigate how differences in habitat complexity affect the relationship between resting metabolic rate (RMR) and social status in the Amazonian dwarf cichlid Apistogramma agassizii. We compared agonistic interactions between pairs of males in aquaria with different habitat enrichment levels, manipulated by adding shelters. RMR was measured before and after hierarchy establishment. Habitat enrichment promotes changes in aggressive behaviour and influences differences in metabolic rate between dominant and subordinate fish. We observed an increase in biting by dominant fish at high enrichment habitat, which could be related to the increase in territory value. We observed an increase in metabolic rate in dominant fish after hierarchy establishment. However, it occurs only in enriched habitats. We concluded that habitat structure interfere with behavioural characteristics in social hierarchies, as aggressiveness, and changes in aggressive interactions affect metabolic rate in different social ranks in the dwarf cichlid Apistogramma agassizii.     

Wounds on the jaw of an eocene mesosuchian crocodilian as possible evidence for the antiquity of crocodilian intraspecific fighting behaviour

Wounds on the lower jaw of a mesosuchian crocodilian (Tilemsisuchus lavocati, family Dyrosauridae) from the Eocene of Mali are interpreted as a result of intraspecific fighting. This behavioural feature is known in the males of living crocodilians, and related to territoriality, dominance hierarchy and mating. The occurrence of similar kinds of behaviour in the Dyrosauridae and in the living eusuchians is thus suggested, which in turn points to a considerable antiquity for this behaviour, if it was inherited from a common ancestor.     

Reconciling conflicts in a one-male society: the case of geladas (<Emphasis Type="Italic">Theropithecus gelada</Emphasis>)

Animals derive benefits from living in social groups but sociality also has its costs in that animals must compete with others for resources and mating opportunities. To cope with the conflict aftermath and social damage caused by competitive aggression, several group-living species use a variety of peace-keeping strategies. The affinitive post-conflict reunion of former opponents, defined as reconciliation, is the primary peace-keeping mechanism. In this study, we provide evidence for the occurrence of reconciliation and test some hypotheses on this post-conflict mechanism in geladas (Theropithecus gelada), a species often neglected in the study of post-conflict dynamics. The conciliatory contacts were uniformly distributed across the different sex–class combinations. Different from baboons, geladas did not show any particular kind of affinitive reconciliation behaviour. Notwithstanding the presence of a linear hierarchy, the dominance relationships did not affect the reconciliation dynamics. According to the valuable relationship hypothesis, coalitionary support seems to be a good predictor for a high level of conciliatory contacts. Finally, at an immediate level reconciliation plays a role in reducing renewed attacks by aggressors, which sought conciliatory contact more frequently than victims. In conclusion, even though the study of post-conflict behaviour in geladas needs to be continued, the patchy nature of their social network is a good model for testing some of the theoretical assumptions about primate conflict resolution.     

Comparing Older Adults' Perceived Need for Dental Care with a Normative Hierarchy of Needs1

The elderly are reported to have poor perception of their need for dental care. In a secondary analysis of data from a community interview and clinical examination survey of older adults, we set out to determine the nature and extent of the inconsistency of perceived and normative needs. The data were gathered originally to estimate the oral health status and need for treatment among adults aged 50 and older in East York, Ontario. In the interview, participants responded to four questions dealing with their satisfaction with, and need for, dental care. Dentists conducted clinical examinations following WHO methods which included oral health status and treatment needs. Data were analyzed using SPSSPC +. We placed the clinical needs in order from urgent to no need, following ADA guidelines. The perceptions of subjects were then compared to this normative hierarchy using Kruskal-Wallis analysis of variance. We found that participants' perceptions were closely associated with the normative hierarchy in all comparisons. Our results suggest that the concordance between perceived and normative needs is greater than previously reported.     

Orienting reaction, organizing for action, and emotional processes

In the first half of the sixties, general notions were formulated concerning the functional role of the orienting reaction (OR) in adaptive activity, its elicitation and habituation. These notions included the following: a) The OR is elicited only by significant changes in a situation. This implies that OR elicitation is preceded by brain processes (usually unconscious) pertaining to the evaluation of the significance of changes according to an existing hierarchy of motivations, attitudes, and goals. Therefore, the OR is of an active (vs. reactive) nature, i.e., is inevitably determined by internal factors of brain activity. b) The OR is not a unitary reaction, but a complex polyfunctional activity, different aspects of which are reflected in different OR components which can be modified rather independently. c) The OR represents the processes of organizing new (non-standard) actions: sensory, motor, or intellectual. OR habituation is a manifestation of attenuation of the active control of an action, and an increase in its automation. Thus, the emphasis in understanding the OR has shifted from a predominantly "sensualistic" platform to a predominantly "actualistic" one. In recent experiments, the role of emotional processes in the elicitation and habituation of components of the OR has been analyzed. Complex relations between the GSR and anxiety were found in a study of patients with acute alcohol withdrawal syndrome treated with different psychopharmacological agents. The study of auditory evoked potential habituation in depressive patients has shown the emotional state influence on sensory aspects of the OR with the participation of the OR brain mechanisms in perceptual defense.(ABSTRACT TRUNCATED AT 250 WORDS)     

Wanting,liking and welfare: The role of affective states in proximate control of behaviour in vertebrates

Animals choose a course of action countless times each day. To do so, they need to prioritise their behaviour within a set of alternative actions and decide which of these actions to perform at any one time and for how long, that is, determine when the behaviour has reached its desired effect. This process has classically been called the proximate behavioural control mechanism. Several aspects contribute to this process: internal and external stimuli, the emotions that they elicit, motivation (wants), behavioural goals, valuation, decision‐making and its modulation by mood, and the assessment of behavioural outcomes (liking). I will address all these aspects in the form of an integrated conceptual model. In the process of behavioural control, options need to be valued, and I will refer to evidence showing that an affective hedonic process in respect to (future) reward and punishment heavily affects this value. Moreover, I view motivation, the force that finally drives a specific behavioural output, as being primarily influenced by affective states or even corresponding fully to them. Given the feedback in behavioural control by (dis‐)liking outcomes of behaviour, I reason that in respect to welfare it is more important for animals to reach proximate goals than to avoid negative stimuli. All behavioural choices are modulated, and I show how mood, a long‐term affective state, can cause such modulation. Proximate control of behaviour takes place in the brain, and I will briefly discuss how current and future brain research may elucidate how the brain computes these processes. Furthermore, the inclusion of affective states in the conceptual model raises the question of the subjective experience of animals, and I will address some of the important open questions in this area of research. I will conclude that neural studies cannot currently provide a detailed and general theory on the algorithms of proximate behavioural control. In parallel to further developing these approaches, I propose to strengthen a refined ethological approach with a focus on the states of “wanting” and “liking” in ecologically meaningful circumstances and with a strong ontogenetic (within species) and comparative (between species) component. I consider this ethological approach to be a highly promising step in understanding proximate behavioural control.     

Sleep, emotions, and visceral control

It is known that sleep is connected with sensory isolation of the brain, inactivation of the consciousness and reorganization of the electrical activity in all cerebral cortical areas. On the other hand, sleep deprivation leads to pathology in visceral organs and finally to the death of animals, while there are no obvious changes in the brain itself. It is still unclear how the changes in the brain activity during sleep could be connected with the visceral health. We assumed that the same brain areas and the same neurons that, in wakefulness, process exteroceptive information, switch, during sleep, to the processing of the interoceptive information. Thus, the central nervous system is involved in regulating the life support functions of the body during sleep. The results of our experiments supported this hypothesis, explained many observations obtained in somnology, and offered mechanisms of several pathological states connected with sleep. However, at the present level of the visceral sleep theory, there is no understanding of the well-known link between the emotional reactions of the body and transition from wakefulness to sleep, and sleep quality. In this study, an attempt is undertaken to combine the visceral theory of sleep with the need-informational theory of emotions proposed by P. Simonov. The visceral theory of sleep assumes that in living organisms there is a constant monitoring of the correspondence of the visceral parameters to the genetically determined values. Mismatch signals evoke the feeling of tiredness and the need of sleep. This sleep need enters the competition with other actual needs of the body. In accordance with the theory of Simonov, emotions connected with a particular need play an important role in their ranking for satisfaction. We propose that emotional estimation of the sleep need based on visceral signals occurs in the same brain structures which undertake this estimation for other behavioral needs in wakefulness. During sleep, the same brain structures involved in estimating emotions continue to rank visceral needs and define their order for processing in the cortical areas and in the highest centers of visceral integration. In the context of the proposed hypothesis, we discuss the results of the studies on the link between sleep and emotions.     

Reciprocal preening and food sharing in colour‐polymorphic nestling barn owls

Barn owl (Tyto alba) siblings preen and offer food items to one another, behaviours that can be considered prosocial because they benefit a conspecific by relieving distress or need. In experimental broods, we analysed whether such behaviours were reciprocated, preferentially exchanged between specific phenotypes, performed to avoid harassment and food theft or signals of hierarchy status. Three of the results are consistent with the hypothesis of direct reciprocity. First, food sharing was reciprocated in three‐chick broods but not in pairs of siblings, that is when nestlings could choose a partner with whom to develop a reciprocating interaction. Second, a nestling was more likely to give a prey item to its sibling if the latter individual had preened the former. Third, siblings matched their investment in preening each other. Manipulation of age hierarchy showed that food stealing was directed towards older siblings but was not performed to compensate for a low level of cooperation received. Social behaviours were related to melanin‐based coloration, suggesting that animals may signal their propensity to interact socially. The most prosocial phenotype (darker reddish) was also the phenotype that stole more food, and the effect of coloration on prosocial behaviour depended upon rank and sex, suggesting that colour‐related prosociality is state dependent.     

Evidence for coevolution of sociality and relative brain size in three orders of mammals

As the brain is responsible for managing an individual's behavioral response to its environment, we should expect that large relative brain size is an evolutionary response to cognitively challenging behaviors. The "social brain hypothesis" argues that maintaining group cohesion is cognitively demanding as individuals living in groups need to be able to resolve conflicts that impact on their ability to meet resource requirements. If sociality does impose cognitive demands, we expect changes in relative brain size and sociality to be coupled over evolutionary time. In this study, we analyze data on sociality and relative brain size for 206 species of ungulates, carnivores, and primates and provide, for the first time, evidence that changes in sociality and relative brain size are closely correlated over evolutionary time for all three mammalian orders. This suggests a process of coevolution and provides support for the social brain theory. However, differences between taxonomic orders in the stability of the transition between small-brained/nonsocial and large-brained/social imply that, although sociality is cognitively demanding, sociality and relative brain size can become decoupled in some cases. Carnivores seem to have been especially prone to this.     

The why,what, where,when and how of goal-directed choice: neuronal and computational principles

The central problems that goal-directed animals must solve are: ‘What do I need and Why, Where and When can this be obtained, and How do I get it?'' or the H4W problem. Here, we elucidate the principles underlying the neuronal solutions to H4W using a combination of neurobiological and neurorobotic approaches. First, we analyse H4W from a system-level perspective by mapping its objectives onto the Distributed Adaptive Control embodied cognitive architecture which sees the generation of adaptive action in the real world as the primary task of the brain rather than optimally solving abstract problems. We next map this functional decomposition to the architecture of the rodent brain to test its consistency. Following this approach, we propose that the mammalian brain solves the H4W problem on the basis of multiple kinds of outcome predictions, integrating central representations of needs and drives (e.g. hypothalamus), valence (e.g. amygdala), world, self and task state spaces (e.g. neocortex, hippocampus and prefrontal cortex, respectively) combined with multi-modal selection (e.g. basal ganglia). In our analysis, goal-directed behaviour results from a well-structured architecture in which goals are bootstrapped on the basis of predefined needs, valence and multiple learning, memory and planning mechanisms rather than being generated by a singular computation.     

Human connectomics

Recent advances in non-invasive neuroimaging have enabled the measurement of connections between distant regions in the living human brain, thus opening up a new field of research: Human connectomics. Different imaging modalities allow the mapping of structural connections (axonal fibre tracts) as well as functional connections (correlations in time series), and individual variations in these connections may be related to individual variations in behaviour and cognition. Connectivity analysis has already led to a number of new insights about brain organization. For example, segregated brain regions may be identified by their unique patterns of connectivity, structural and functional connectivity may be compared to elucidate how dynamic interactions arise from the anatomical substrate, and the architecture of large-scale networks connecting sets of brain regions may be analysed in detail. The combined analysis of structural and functional networks has begun to reveal components or modules with distinct patterns of connections that become engaged in different cognitive tasks. Collectively, advances in human connectomics open up the possibility of studying how brain connections mediate regional brain function and thence behaviour.     

Youth and personality formation--the world belongs to the strength of the young and to the wisdom of the elderly (a point of view of the communicologist)

It is known from the pedagogic, didactic and experience coordinates that a man doesn't become better by using words but by his actions. They are matchless examples for the self-apprehension process and the perception of the action itself as well as the only worthy and accurate offer for the assessment of the range and the significance of the achieved result. From the earliest stage of their life, from the childhood and then later on in different stages of their development and growth, the human being as a social being has a double need: to feel himself in the range of his "I" and to feel himself as a member of a group, a family and of the society in general. Satisfying the sense and the level of those needs doubtlessly depends on the examples given by the life itself and the circumstances of the living within the family, the school and finally the society, that is, the surroundings where the individual is affirming himself by his individual creative capability and skills with which he realizes himself as a subject on its own as well as within the social group he lives and works in. The conditions of the life and work of the elderly are necessary for this important and precious task in the development of a young man so that the applications of their experiences could ensure reliable guidelines for the harmony and success of the life and work of the young. On this bridge of generations the experience of the old is affirmed along with the need for its aimed, creative and fruitful offer to the young in the dynamic process of their development to the maturity and creativity.