Fly-like visuomotor responses of a robot using aVLSI motion-sensitive chips

 We explore the use of continuous-time analog very-large-scale-integrated (aVLSI) neuromorphic visual preprocessors together with a robotic platform in generating bio-inspired behaviors. Both the aVLSI motion sensors and the robot behaviors described in this work are inspired by the motion computation in the fly visual system and two different fly behaviors. In most robotic systems, the visual information comes from serially scanned imagers. This restricts the form of computation of the visual image and slows down the input rate to the controller system of the robot, hence increasing the reaction time of the robot. These aVLSI neuromorphic sensors reduce the computational load and power consumption of the robot, thus making it possible to explore continuous-time visuomotor control systems that react in real-time to the environment. The motion sensor provides two outputs: one for the preferred direction and the other for the null direction. These motion outputs are created from the aggregation of six elementary motion detectors that implement a variant of Reichardt's correlation algorithm. The four analog continuous-time outputs from the motion chips go to the control system on the robot which generates a mixture of two behaviors – course stabilization and fixation – from the outputs of these sensors. Since there are only four outputs, the amount of information transmitted to the controller is reduced (as compared to using a CCD sensor), and the reaction time of the robot is greatly decreased. In this work, the robot samples the motion sensors every 3.3 ms during the behavioral experiments. Received: 4 October 1999 / Accepted in revised form: 26 April 2001     

Schema design and implementation of the grasp-related mirror neuron system

 Mirror neurons within a monkey's premotor area F5 fire not only when the monkey performs a certain class of actions but also when the monkey observes another monkey (or the experimenter) perform a similar action. It has thus been argued that these neurons are crucial for understanding of actions by others. We offer the hand-state hypothesis as a new explanation of the evolution of this capability: the basic functionality of the F5 mirror system is to elaborate the appropriate feedback – what we call the hand state– for opposition-space based control of manual grasping of an object. Given this functionality, the social role of the F5 mirror system in understanding the actions of others may be seen as an exaptation gained by generalizing from one's own hand to an other's hand. In other words, mirror neurons first evolved to augment the “canonical” F5 neurons (active during self-movement based on observation of an object) by providing visual feedback on “hand state,” relating the shape of the hand to the shape of the object. We then introduce the MNS1 (mirror neuron system 1) model of F5 and related brain regions. The existing Fagg–Arbib–Rizzolatti–Sakata model represents circuitry for visually guided grasping of objects, linking the anterior intraparietal area (AIP) with F5 canonical neurons. The MNS1 model extends the AIP visual pathway by also modeling pathways, directed toward F5 mirror neurons, which match arm–hand trajectories to the affordances and location of a potential target object. We present the basic schemas for the MNS1 model, then aggregate them into three “grand schemas”– visual analysis of hand state, reach and grasp, and the core mirror circuit – for each of which we present a useful implementation (a non-neural visual processing system, a multijoint 3-D kinematics simulator, and a learning neural network, respectively). With this implementation we show how the mirror system may learnto recognize actions already in the repertoire of the F5 canonical neurons. We show that the connectivity pattern of mirror neuron circuitry can be established through training, and that the resultant network can exhibit a range of novel, physiologically interesting behaviors during the process of action recognition. We train the system on the basis of final grasp but then observe the whole time course of mirror neuron activity, yielding predictions for neurophysiological experiments under conditions of spatial perturbation, altered kinematics, and ambiguous grasp execution which highlight the importance of the timingof mirror neuron activity. Received: 6 August 2001 / Accepted in revised form: 5 February 2002     

Development of an Anthropomorphic Robotic Arm and Hand for Interactive Humanoids

Humanoid robots are designed and built to mimic human form and movement.Ultimately,they are meant to resemble the size and physical abilities of a human in order to function in human-oriented environments and to work autonomously but to pose no physical threat to humans.Here,a humanoid robot that resembles a human in appearance and movement is built using powerful actuators paired with gear trains,joint mechanisms,and motor drivers that are all encased in a package no larger than that of the human physique.In this paper,we propose the construction of a humanoid-applicable anthropomorphic 7-DoF arm complete with an 8-DoF hand.The novel mechanical design of this humanoid arm makes it sufficiently compact to be compatible with currently available narrating-model humanoids,and to be sufficiently powerful and flexible to be functional; the number of degrees of freedom endowed in this robotic arm is sufficient for executing a wide range of tasks,including dexterous hand movements.The developed humanoid arm and hand are capable of sensing and interpreting incoming external force using the motor in each joint current without conventional torque sensors.The humanoid arm adopts an algorithm to avoid obstacles and the dexterous hand is capable of grasping objects.The developed robotic arm is suitable for use in an interactive humanoid robot.     

Object Grasping using the Minimum Variance Model

Reaching-to-grasp has generally been classified as the coordination of two separate visuomotor processes: transporting the hand to the target object and performing the grip. An alternative view has recently been formed that grasping can be explained as pointing movements performed by the digits of the hand to target positions on the object. We have previously implemented the minimum variance model of human movement as an optimal control scheme suitable for control of a robot arm reaching to a target. Here, we extend that scheme to perform grasping movements with a hand and arm model. Since the minimum variance model requires that signal-dependent noise be present on the motor commands to the actuators of the movement, our approach is to plan the reach and the grasp separately, in line with the classical view, but using the same computational model for pointing, in line with the alternative view. We show that our model successfully captures some of the key characteristics of human grasping movements, including the observations that maximum grip size increases with object size (with a slope of approximately 0.8) and that this maximum grip occurs at 60–80% of the movement time. We then use our model to analyse contributions to the digit end-point variance from the two components of the grasp (the transport and the grip). We also briefly discuss further areas of investigation that are prompted by our model.     

An octopus-bioinspired solution to movement and manipulation for soft robots

Soft robotics is a challenging and promising branch of robotics. It can drive significant improvements across various fields of traditional robotics, and contribute solutions to basic problems such as locomotion and manipulation in unstructured environments. A challenging task for soft robotics is to build and control soft robots able to exert effective forces. In recent years, biology has inspired several solutions to such complex problems. This study aims at investigating the smart solution that the Octopus vulgaris adopts to perform a crawling movement, with the same limbs used for grasping and manipulation. An ad hoc robot was designed and built taking as a reference a biological hypothesis on crawling. A silicone arm with cables embedded to replicate the functionality of the arm muscles of the octopus was built. This novel arm is capable of pushing-based locomotion and object grasping, mimicking the movements that octopuses adopt when crawling. The results support the biological observations and clearly show a suitable way to build a more complex soft robot that, with minimum control, can perform diverse tasks.     

Distinct haptic cues do not reduce interference when learning to reach in multiple force fields

Background

Previous studies of learning to adapt reaching movements in the presence of novel forces show that learning multiple force fields is prone to interference. Recently it has been suggested that force field learning may reflect learning to manipulate a novel object. Within this theoretical framework, interference in force field learning may be the result of static tactile or haptic cues associated with grasp, which fail to indicate changing dynamic conditions. The idea that different haptic cues (e.g. those associated with different grasped objects) signal motor requirements and promote the learning and retention of multiple motor skills has previously been unexplored in the context of force field learning.

Methodology/Principle Findings

The present study tested the possibility that interference can be reduced when two different force fields are associated with differently shaped objects grasped in the hand. Human subjects were instructed to guide a cursor to targets while grasping a robotic manipulandum, which applied two opposing velocity-dependent curl fields to the hand. For one group of subjects the manipulandum was fitted with two different handles, one for each force field. No attenuation in interference was observed in these subjects relative to controls who used the same handle for both force fields.

Conclusions/Significance

These results suggest that in the context of the present learning paradigm, haptic cues on their own are not sufficient to reduce interference and promote learning multiple force fields.     

Modelling natural and artificial hands with synergies

We report on recent work in modelling the process of grasping and active touch by natural and artificial hands. Starting from observations made in human hands about the correlation of degrees of freedom in patterns of more frequent use (postural synergies), we consider the implications of a geometrical model accounting for such data, which is applicable to the pre-grasping phase occurring when shaping the hand before actual contact with the grasped object. To extend applicability of the synergy model to study force distribution in the actual grasp, we introduce a modified model including the mechanical compliance of the hand's musculotendinous system. Numerical results obtained by this model indicate that the same principal synergies observed from pre-grasp postural data are also fundamental in achieving proper grasp force distribution. To illustrate the concept of synergies in the dual domain of haptic sensing, we provide a review of models of how the complexity and heterogeneity of sensory information from touch can be harnessed in simplified, tractable abstractions. These abstractions are amenable to fast processing to enable quick reflexes as well as elaboration of high-level percepts. Applications of the synergy model to the design and control of artificial hands and tactile sensors are illustrated.     

Grasping without Sight: Insights from the Congenitally Blind

We reach for and grasp different sized objects numerous times per day. Most of these movements are visually-guided, but some are guided by the sense of touch (i.e. haptically-guided), such as reaching for your keys in a bag, or for an object in a dark room. A marked right-hand preference has been reported during visually-guided grasping, particularly for small objects. However, little is known about hand preference for haptically-guided grasping. Recently, a study has shown a reduction in right-hand use in blindfolded individuals, and an absence of hand preference if grasping was preceded by a short haptic experience. These results suggest that vision plays a major role in hand preference for grasping. If this were the case, then one might expect congenitally blind (CB) individuals, who have never had a visual experience, to exhibit no hand preference. Two novel findings emerge from the current study: first, the results showed that contrary to our expectation, CB individuals used their right hand during haptically-guided grasping to the same extent as visually-unimpaired (VU) individuals did during visually-guided grasping. And second, object size affected hand use in an opposite manner for haptically- versus visually-guided grasping. Big objects were more often picked up with the right hand during haptically-guided, but less often during visually-guided grasping. This result highlights the different demands that object features pose on the two sensory systems. Overall the results demonstrate that hand preference for grasping is independent of visual experience, and they suggest a left-hemisphere specialization for the control of grasping that goes beyond sensory modality.     

Workability and productivity of robotic plug transplanting workcell

Summary Transplanting is a necessary operation in transplant production systems. Transplanting operation is labor-intensive and automation can reduce labor costs. Plugs are actively growing young transplants with two well-defined morphologic parts: the stem-leaf portion and the root-growth medium portion. They may be grown in regularly situated cells on traylike containers. This regularity makes plugs suitable for automated transplanting operations. It is, therefore, beneficial for in vitro plant propagation systems to include plugs as intermediate products before they are delivered to the greenhouses. Flexible automation and robotics technologies have been applied to develop a robotic workcell for transplanting plugs from plug trays to growing flats. Main components of the workcell include a robot, an end-effector, and two conveyer belts for transporting trays and flats. The end-effector for extracting, holding, and planting plugs is a “sliding-needles-with-sensor” gripper. The sensor signals the robot to complete a transplanting cycle only when a plug is properly held by the gripper. Systems analysis and computer simulation were conducted to study factors affecting workability and productivity of various workcell designs. These factors included: dimensions and kinematics of the robot and its peripheral equipment, layout and materials flow, fullness of plug trays, and successful extraction rate of plugs. The analysis also indicated that machine vision systems could add valuable capabilities to the workcell, such as robot guidance and plug quality evaluation. Engineering economic analysis was performed to investigate the interaction of workcell technical feasibility and economic viability. Presented in the Session-in-Depth Robotics in Tissue Culture at the 1991 World Congress on Cell and Tissue Culture, Anaheim, California, June 16–20, 1991.     

Order Parameter Dynamics of Body-scaled Hysteresis and Mode Transitions in Grasping Behavior

Several experimental studies have shown that human grasping behavior exhibits a transition from one-handed to two-handed grasping when to-be-grasped objects become larger and larger. The transition point depends on the relative size of objects measured in terms of human body-scales. Most strikingly, the transitions between the two different behavioral ‘modes’ of grasping exhibit hysteresis. That is, one-to-two hand transitions and two-to-one hand transitions occur at different relative object sizes when objects are scaled up or down in size. In our study we approach body-scaled hysteresis and mode transitions in grasping by exploiting the notion that human behavior in general results from self-organization and satisfies appropriately-defined order parameter equations. To this end, grasping transitions and grasping hysteresis are discussed from a theoretical perspective in analogy to cognitive processes defined by Haken’s neural network model for pattern recognition. In doing so, issues such as the exclusivity of grasping modes, biomechanical constraints, mode-mode interactions, single subject behavior and population behavior are explored.     

Learning to control a brain-machine interface for reaching and grasping by primates

Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain–machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e., hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.     

Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates

Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain–machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e., hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.     

Grasping Objects with Environmentally Induced Position Uncertainty

Due to noisy motor commands and imprecise and ambiguous sensory information, there is often substantial uncertainty about the relative location between our body and objects in the environment. Little is known about how well people manage and compensate for this uncertainty in purposive movement tasks like grasping. Grasping objects requires reach trajectories to generate object-fingers contacts that permit stable lifting. For objects with position uncertainty, some trajectories are more efficient than others in terms of the probability of producing stable grasps. We hypothesize that people attempt to generate efficient grasp trajectories that produce stable grasps at first contact without requiring post-contact adjustments. We tested this hypothesis by comparing human uncertainty compensation in grasping objects against optimal predictions. Participants grasped and lifted a cylindrical object with position uncertainty, introduced by moving the cylinder with a robotic arm over a sequence of 5 positions sampled from a strongly oriented 2D Gaussian distribution. Preceding each reach, vision of the object was removed for the remainder of the trial and the cylinder was moved one additional time. In accord with optimal predictions, we found that people compensate by aligning the approach direction with covariance angle to maintain grasp efficiency. This compensation results in higher probability to achieve stable grasps at first contact than non-compensation strategies in grasping objects with directional position uncertainty, and the results provide the first demonstration that humans compensate for uncertainty in a complex purposive task.     

Active Sensing System with In Situ Adjustable Sensor Morphology

Background

Despite the widespread use of sensors in engineering systems like robots and automation systems, the common paradigm is to have fixed sensor morphology tailored to fulfill a specific application. On the other hand, robotic systems are expected to operate in ever more uncertain environments. In order to cope with the challenge, it is worthy of note that biological systems show the importance of suitable sensor morphology and active sensing capability to handle different kinds of sensing tasks with particular requirements.

Methodology

This paper presents a robotics active sensing system which is able to adjust its sensor morphology in situ in order to sense different physical quantities with desirable sensing characteristics. The approach taken is to use thermoplastic adhesive material, i.e. Hot Melt Adhesive (HMA). It will be shown that the thermoplastic and thermoadhesive nature of HMA enables the system to repeatedly fabricate, attach and detach mechanical structures with a variety of shape and size to the robot end effector for sensing purposes. Via active sensing capability, the robotic system utilizes the structure to physically probe an unknown target object with suitable motion and transduce the arising physical stimuli into information usable by a camera as its only built-in sensor.

Conclusions/Significance

The efficacy of the proposed system is verified based on two results. Firstly, it is confirmed that suitable sensor morphology and active sensing capability enables the system to sense different physical quantities, i.e. softness and temperature, with desirable sensing characteristics. Secondly, given tasks of discriminating two visually indistinguishable objects with respect to softness and temperature, it is confirmed that the proposed robotic system is able to autonomously accomplish them. The way the results motivate new research directions which focus on in situ adjustment of sensor morphology will also be discussed.     

The neuroscience of grasping

People have always been fascinated by the exquisite precision and flexibility of the human hand. When hand meets object, we confront the overlapping worlds of sensorimotor and cognitive functions. We reach for objects, grasp and lift them, manipulate them and use them to act on other objects. This review examines one of these actions--grasping. Recent research in behavioural neuroscience, neuroimaging and electrophysiology has the potential to reveal where in the brain the process of grasping is organized, but has yet to address several questions about the sensorimotor transformations that relate to the control of the hands.     

Biomechanical Characteristics of Hand Coordination in Grasping Activities of Daily Living

Hand coordination can allow humans to have dexterous control with many degrees of freedom to perform various tasks in daily living. An important contributing factor to this important ability is the complex biomechanical architecture of the human hand. However, drawing a clear functional link between biomechanical architecture and hand coordination is challenging. It is not understood which biomechanical characteristics are responsible for hand coordination and what specific effect each biomechanical characteristic has. To explore this link, we first inspected the characteristics of hand coordination during daily tasks through a statistical analysis of the kinematic data, which were collected from thirty right-handed subjects during a multitude of grasping tasks. Then, the functional link between biomechanical architecture and hand coordination was drawn by establishing the clear corresponding causality between the tendinous connective characteristics of the human hand and the coordinated characteristics during daily grasping activities. The explicit functional link indicates that the biomechanical characteristic of tendinous connective architecture between muscles and articulations is the proper design by the Creator to perform a multitude of daily tasks in a comfortable way. The clear link between the structure and the function of the human hand also suggests that the design of a multifunctional robotic hand should be able to better imitate such basic architecture.     

Design of a cybernetic hand for perception and action

Strong motivation for developing new prosthetic hand devices is provided by the fact that low functionality and controllability—in addition to poor cosmetic appearance—are the most important reasons why amputees do not regularly use their prosthetic hands. This paper presents the design of the CyberHand, a cybernetic anthropomorphic hand intended to provide amputees with functional hand replacement. Its design was bio-inspired in terms of its modular architecture, its physical appearance, kinematics, sensorization, and actuation, and its multilevel control system. Its underactuated mechanisms allow separate control of each digit as well as thumb–finger opposition and, accordingly, can generate a multitude of grasps. Its sensory system was designed to provide proprioceptive information as well as to emulate fundamental functional properties of human tactile mechanoreceptors of specific importance for grasp-and-hold tasks. The CyberHand control system presumes just a few efferent and afferent channels and was divided in two main layers: a high-level control that interprets the user’s intention (grasp selection and required force level) and can provide pertinent sensory feedback and a low-level control responsible for actuating specific grasps and applying the desired total force by taking advantage of the intelligent mechanics. The grasps made available by the high-level controller include those fundamental for activities of daily living: cylindrical, spherical, tridigital (tripod), and lateral grasps. The modular and flexible design of the CyberHand makes it suitable for incremental development of sensorization, interfacing, and control strategies and, as such, it will be a useful tool not only for clinical research but also for addressing neuroscientific hypotheses regarding sensorimotor control.     

Evolving locomotion for a 12-DOF quadruped robot in simulated environments

We demonstrate the power of evolutionary robotics (ER) by comparing to a more traditional approach its performance and cost on the task of simulated robot locomotion. A novel quadruped robot is introduced, the legs of which – each having three non-coplanar degrees of freedom – are very maneuverable. Using a simplistic control architecture and a physics simulation of the robot, gaits are designed both by hand and using a highly parallel evolutionary algorithm (EA). It is found that the EA produces, in a small fraction of the time that takes to design by hand, gaits that travel at two to four times the speed of the hand-designed one. The flexibility of this approach is demonstrated by applying it across a range of differently configured simulators.     

Grasp modelling with a biomechanical model of the hand

The use of a biomechanical model for human grasp modelling is presented. A previously validated biomechanical model of the hand has been used. The equilibrium of the grasped object was added to the model through the consideration of a soft contact model. A grasping posture generation algorithm was also incorporated into the model. All the geometry was represented using a spherical extension of polytopes (s-topes) for efficient collision detection. The model was used to simulate an experiment in which a subject was asked to grasp two cylinders of different diameters and weights. Different objective functions were checked to solve the indeterminate problem. The normal finger forces estimated by the model were compared to those experimentally measured. The popular objective function sum of the squared muscle stresses was shown not suitable for the grasping simulation, requiring at least being complemented by task-dependent grasp quality measures.     

Do Postures of Distal Effectors Affect the Control of Actions of Other Distal Effectors? Evidence for a System of Interactions between Hand and Mouth

The present study aimed at determining whether, in healthy humans, postures assumed by distal effectors affect the control of the successive grasp executed with other distal effectors. In experiments 1 and 2, participants reached different objects with their head and grasped them with their mouth, after assuming different hand postures. The postures could be implicitly associated with interactions with large or small objects. The kinematics of lip shaping during grasp varied congruently with the hand posture, i.e. it was larger or smaller when it could be associated with the grasping of large or small objects, respectively. In experiments 3 and 4, participants reached and grasped different objects with their hand, after assuming the postures of mouth aperture or closure (experiment 3) and the postures of toe extension or flexion (experiment 4). The mouth postures affected the kinematics of finger shaping during grasp, that is larger finger shaping corresponded with opened mouth and smaller finger shaping with closed mouth. In contrast, the foot postures did not influence the hand grasp kinematics. Finally, in experiment 5 participants reached-grasped different objects with their hand while pronouncing opened and closed vowels, as verified by the analysis of their vocal spectra. Open and closed vowels induced larger and smaller finger shaping, respectively. In all experiments postures of the distal effectors induced no effect, or only unspecific effects on the kinematics of the reach proximal/axial component. The data from the present study support the hypothesis that there exists a system involved in establishing interactions between movements and postures of hand and mouth. This system might have been used to transfer a repertoire of hand gestures to mouth articulation postures during language evolution and, in modern humans, it may have evolved a system controlling the interactions existing between speech and gestures.