Simulation of an iterative learning control system for fed-batch cell culture processes

This paper describes an iterative learning control scheme for fed-batch operation where repetitive trajectory tracking tasks are required. The proposed learning strategy is model-independent, and it takes advantage of the repetitive feature of system operations with a certain degree of intelligence and requires only small size of dynamic database for the learning process. The convergence of the learning process is proven. An example of simultaneously tracking two predefined trajectories by iterative learning control with two control inputs is given to illustrate the methodology. Satisfactory performance of the learning system can be observed from the simulation results.     

Motion control of musculoskeletal systems with redundancy

Motion control of musculoskeletal systems for functional electrical stimulation (FES) is a challenging problem due to the inherent complexity of the systems. These include being highly nonlinear, strongly coupled, time-varying, time-delayed, and redundant. The redundancy in particular makes it difficult to find an inverse model of the system for control purposes. We have developed a control system for multiple input multiple output (MIMO) redundant musculoskeletal systems with little prior information. The proposed method separates the steady-state properties from the dynamic properties. The dynamic control uses a steady-state inverse model and is implemented with both a PID controller for disturbance rejection and an artificial neural network (ANN) feedforward controller for fast trajectory tracking. A mechanism to control the sum of the muscle excitation levels is also included. To test the performance of the proposed control system, a two degree of freedom ankle–subtalar joint model with eight muscles was used. The simulation results show that separation of steady-state and dynamic control allow small output tracking errors for different reference trajectories such as pseudo-step, sinusoidal and filtered random signals. The proposed control method also demonstrated robustness against system parameter and controller parameter variations. A possible application of this control algorithm is FES control using multiple contact cuff electrodes where mathematical modeling is not feasible and the redundancy makes the control of dynamic movement difficult.     

Novel Hybrid Adaptive Controller for Manipulation in Complex Perturbation Environments

In this paper we present a hybrid control scheme, combining the advantages of task-space and joint-space control. The controller is based on a human-like adaptive design, which minimises both control effort and tracking error. Our novel hybrid adaptive controller has been tested in extensive simulations, in a scenario where a Baxter robot manipulator is affected by external disturbances in the form of interaction with the environment and tool-like end-effector perturbations. The results demonstrated improved performance in the hybrid controller over both of its component parts. In addition, we introduce a novel method for online adaptation of learning parameters, using the fuzzy control formalism to utilise expert knowledge from the experimenter. This mechanism of meta-learning induces further improvement in performance and avoids the need for tuning through trial testing.     

On-line monitoring and control of substrate concentrations in biological processes by flow injection analysis systems

Concentrations of substrates, glucose, and ammionia in biological processes have been on-line monitored by using glucose-flow injection (FIA) and ammonia-FIA systems. Based on the on-line monitored data the concentrations of substrates have been controlled by an on-off controller, a PID controller, and a neural network (NN) based controller. A simulation program has been developed to test the control quality of each controller and to estimate the control parameters. The on-off controller often produced high oscillations at the set point due to its low robustness. The control quality of a PID controller could have been improved by a high analysis frequency and by a short residence time of sample in a FIA system. A NN-based controller with 3 layers has been developed, and a 3(input)-2(hidden)-1(output) network structure has been found to be optimal for the NN-based controller. The performance of the three controllers has been tested in a simulated process as well as in a cultivation process ofSaccharomyces cerevisiae, and the performance has also been compared to simulation results. The NN-based controller with the 3-2-1 network structure was robust and stable against some disturbances, such as a sudden injection of distilled water into a biological process.     

Towards Active Tracking of Beating Heart Motion in the Presence of Arrhythmia for Robotic Assisted Beating Heart Surgery

In robotic assisted beating heart surgery, the control architecture for heart motion tracking has stringent requirements in terms of bandwidth of the motion that needs to be tracked. In order to achieve sufficient tracking accuracy, feed-forward control algorithms, which rely on estimations of upcoming heart motion, have been proposed in the literature. However, performance of these feed-forward motion control algorithms under heart rhythm variations is an important concern. In their past work, the authors have demonstrated the effectiveness of a receding horizon model predictive control-based algorithm, which used generalized adaptive predictors, under constant and slowly varying heart rate conditions. This paper extends these studies to the case when the heart motion statistics change abruptly and significantly, such as during arrhythmias. A feasibility study is carried out to assess the motion tracking capabilities of the adaptive algorithms in the occurrence of arrhythmia during beating heart surgery. Specifically, the tracking performance of the algorithms is evaluated on prerecorded motion data, which is collected in vivo and includes heart rhythm irregularities. The algorithms are tested using both simulations and bench experiments on a three degree-of-freedom robotic test bed. They are also compared with a position-plus-derivative controller as well as a receding horizon model predictive controller that employs an extended Kalman filter algorithm for predicting future heart motion.     

Control of constraint forces and trajectories in a rich sensory and actuation environment

A simple control strategy is proposed and applied to a class of non-linear systems that have abundant sensory and actuation channels as in living systems. The main objective is the independent control of constrained trajectories of motion, and control of the corresponding constraint forces. The peripheral controller is a proportional, derivative and integral (PID) controller. A central controller produces, via pattern generators, reference signals that are the desired constrained position and velocity trajectories, and the desired constraint forces. The basic tenet of the this hybrid control strategy is the use of two mechanisms: 1. linear state and force feedback, and 2. non-linear constraint velocity feedback - sliding mode feedback. The first mechanism can be envisioned as a high gain feedback systems. The high gain attribute imitates the agonist-antagonist co-activation in natural systems. The strategy is applied to the control of the force and trajectory of a two-segment thigh-leg planar biped leg with a mass-less foot cranking a pedal that is analogous to a bicycle pedal. Five computational experiments are presented to show the effectiveness of the strategy and the performance of the controller. The findings of this paper are applicable to the design of orthoses and prostheses to supplement functional electrical stimulation for support purposes in the spinally injured cases.     

Full-state tracking control of a mobile robot using neural networks

In this paper a nonholonomic mobile robot with completely unknown dynamics is discussed. A mathematical model has been considered and an efficient neural network is developed, which ensures guaranteed tracking performance leading to stability of the system. The neural network assumes a single layer structure, by taking advantage of the robot regressor dynamics that expresses the highly nonlinear robot dynamics in a linear form in terms of the known and unknown robot dynamic parameters. No assumptions relating to the boundedness is placed on the unmodeled disturbances. It is capable of generating real-time smooth and continuous velocity control signals that drive the mobile robot to follow the desired trajectories. The proposed approach resolves speed jump problem existing in some previous tracking controllers. Further, this neural network does not require offline training procedures. Lyapunov theory has been used to prove system stability. The practicality and effectiveness of the proposed tracking controller are demonstrated by simulation and comparison results.     

The study of neural network-based controller for controlling dissolved oxygen concentration in a sequencing batch reactor

The design and development of the neural network (NN)-based controller performance for the activated sludge process in sequencing batch reactor (SBR) is presented in this paper. Here we give a comparative study of various neural network (NN)-based controllers such as the direct inverse control, internal model control (IMC) and hybrid NN control strategies to maintain the dissolved oxygen (DO) level of an activated sludge system by manipulating the air flow rate. The NN inverse model-based controller with the model-based scheme represents the controller, which relies solely upon the simple NN inverse model. In the IMC, both the forward and inverse models are used directly as elements within the feedback loop. The hybrid NN control consists of a basic NN controller in parallel with a proportional integral (PI) controller. Various simulation tests involving multiple set-point changes, disturbances rejection and noise effects were performed to review the performances of these various controllers. From the results it can be seen that hybrid controller gives the best results in tracking set-point changes under disturbances and noise effects.     

Tracking the acetate threshold using DO-transient control during medium and high cell density cultivation of recombinant Escherichia coli in complex media

DO-transient nutrient controllers use the dissolved oxygen signal to attempt acetate threshold tracking during fed-batch cultivation of recombinant E. coli. Here we apply DO-transient control to the production of Jembrana disease virus protein in complex Super Luria medium and compare performance against a high-limit pH-stat controller. For induction at medium cell density (harvest between 31 and 32.5 g dcw L) a total productivity of 0.27 g L h was achieved as compared to 0.24 g L h with the high-limit pH-stat. For induction at high cell density (harvest at 60 g dcw L), decreased productivity (0.12 g L h) was attributed to the effect of acetate accumulation on recombinant protein formation and a concomitant lowering of the critical growth rate. Our results suggest that complex media provides a difficult environment for the application of acetate threshold tracking DO-transient control because of difficulties in re-oxidizing acetate, and apparent localized production of acetate below the production threshold (as detected by the DO-transient controller as SPOUR(crit)). Configuring the DO-transient controller to avoid aggressive threshold probing is suggested as a means to improve performance and reduce acetate accumulation in complex media.     

Dynamic fuzzy model based predictive controller for a biochemical reactor

The kinetics of bioreactions often involve some uncertainties and the dynamics of the process vary during the course of fermentation. For such processes, conventional control schemes may not provide satisfactory control performance and demands extra effort to design advanced control schemes. In this study, a dynamic fuzzy model based predictive controller (DFMBPC) is presented for the control of a biochemical reactor. The DFMBPC incorporates an adaptive fuzzy modeling framework into a model based predictive control scheme to derive analytical controller output. The DFMBPC has the flexibility to opt with various types of fuzzy models whose choice also lead to improve the control performance. The performance of DFMBPC is evaluated by comparing with a fuzzy model based predictive controller (FMBPC) with no model adaptation and a conventional PI controller. The results show that DFMBPC provides better performance for tracking setpoint changes and rejecting unmeasured disturbances in the biochemical reactor.     

On‐line yeast propagation process monitoring and control using an intelligent automatic control system

The monitoring and control of bioprocesses is a challenging task. This applies particularly if the actions to the process have to be carried out in real‐time. This work presents a system for on‐line monitoring and control of batch yeast propagation under limiting conditions based on a virtual plant operator, which uses the concept of intelligent control algorithms by means of fuzzy logic theory. Process information is provided on‐line using a sensor array comprising the measurement of OD, operating temperature, pressure, density, dissolved oxygen, and pH value. In this context practical problems arising through on‐line sensing and signal processing are addressed. The preprocessed sensor data are fed to a neural network for on‐line biomass estimation. The root mean squared error of prediction is 4 × 10⁶ cells/mL. The proposed system then triggers temperature and aeration by usage of a temperature dependent metabolic growth model and sensor data. The deviation of the predicted biomass from that of the reference trajectory as modeled by the metabolic growth model and its temporal derivative are used as inputs for the fuzzy temperature controller. The inputs used by the fuzzy aeration controller are the deviation of measured extract from that of the reference trajectory, the predicted cell count, and the dissolved oxygen concentration. The fuzzy‐based expert system allows to provide the desired yeast cell concentration of 100–120 × 10⁶ cells/mL at a minimum residual extract limit of 6.0 g/100 g at the required point of time. Thus, a dynamic adjustment of the propagation process to the overall production schedule is possible in order to produce the required amount of biomass at the right time.     

Comparison of Modeling Methods for DoE‐Based Holistic Upstream Process Characterization

Upstream bioprocess characterization and optimization are time and resource‐intensive tasks. Regularly in the biopharmaceutical industry, statistical design of experiments (DoE) in combination with response surface models (RSMs) are used, neglecting the process trajectories and dynamics. Generating process understanding with time‐resolved, dynamic process models allows to understand the impact of temporal deviations, production dynamics, and provides a better understanding of the process variations that stem from the biological subsystem. The authors propose to use DoE studies in combination with hybrid modeling for process characterization. This approach is showcased on Escherichia coli fed‐batch cultivations at the 20L scale, evaluating the impact of three critical process parameters. The performance of a hybrid model is compared to a pure data‐driven model and the widely adopted RSM of the process endpoints. Further, the performance of the time‐resolved models to simultaneously predict biomass and titer is evaluated. The superior behavior of the hybrid model compared to the pure black‐box approaches for process characterization is presented. The evaluation considers important criteria, such as the prediction accuracy of the biomass and titer endpoints as well as the time‐resolved trajectories. This showcases the high potential of hybrid models for soft‐sensing and model predictive control.     

Intermittent control: a computational theory of human control

The paradigm of continuous control using internal models has advanced understanding of human motor control. However, this paradigm ignores some aspects of human control, including intermittent feedback, serial ballistic control, triggered responses and refractory periods. It is shown that event-driven intermittent control provides a framework to explain the behaviour of the human operator under a wider range of conditions than continuous control. Continuous control is included as a special case, but sampling, system matched hold, an intermittent predictor and an event trigger allow serial open-loop trajectories using intermittent feedback. The implementation here may be described as ??continuous observation, intermittent action??. Beyond explaining unimodal regulation distributions in common with continuous control, these features naturally explain refractoriness and bimodal stabilisation distributions observed in double stimulus tracking experiments and quiet standing, respectively. Moreover, given that human control systems contain significant time delays, a biological-cybernetic rationale favours intermittent over continuous control: intermittent predictive control is computationally less demanding than continuous predictive control. A standard continuous-time predictive control model of the human operator is used as the underlying design method for an event-driven intermittent controller. It is shown that when event thresholds are small and sampling is regular, the intermittent controller can masquerade as the underlying continuous-time controller and thus, under these conditions, the continuous-time and intermittent controller cannot be distinguished. This explains why the intermittent control hypothesis is consistent with the continuous control hypothesis for certain experimental conditions.     

Human learning characteristics in the tracking tasks of iterative nature

In this paper, human learning characteristics in the tracking tasks of iterative nature are investigated. Various linear and nonlinear systems are used as plant, and a human operator has to generate the proper control inputs to force these systems in tracking the desired trajectory. The learning behaviour of the human operator in modifying his control actions is studied and it is observed that the human operator can improve his performance quite efficiently despite the unavailability of any information about the system or the desired trajectories. It is concluded from the experiments that the human operator not only use the information that is directly available to him (error in this case), but also extracts some useful information (e.g. error rate) that he feels is necessary to generate a good control action. The limitation of the human performance is studied in frequency domain, and the performance of the human operator against the frequency bandwidth of error and error rate signals are highlighted. Analysis of the results revealed that a human operator gives more importance to the error rate in generating his control actions and, accordingly, it is observed that his limitation in term of performance is more sensitive to the frequency bandwidth of the error rate as compared to the error. The human operator cannot improve his performance once the frequency components of the error or error rates shift to the higher frequencies, say above 1.0 Hz.     

A balance control model of quiet upright stance based on an optimal control strategy

Models of balance control can aid in understanding the mechanisms by which humans maintain balance. A balance control model of quiet upright stance based on an optimal control strategy is presented here. In this model, the human body was represented by a simple single-segment inverted pendulum during upright stance, and the neural controller was assumed to be an optimal controller that generates ankle control torques according to a certain performance criterion. This performance criterion was defined by several physical quantities relevant to sway. In order to accurately simulate existing experimental data, an optimization procedure was used to specify the set of model parameters to minimize the scalar error between experimental and simulated sway measures. Thirty-two independent simulations were performed for both younger and older adults. The model's capabilities, in terms of reflecting sway behaviors and identifying aging effects, were then analyzed based on the simulation results. The model was able to accurately predict center-of-pressure-based sway measures, and identify potential changes in balance control mechanisms caused by aging. Correlations between sway measures and model parameters are also discussed.     

Control and fault diagnosis in bottom fermentation systems using parity space approach

The main objective of this note is to describe a real-time fault diagnosis and control for a cylindroconical fermenter to laboratory scale as an alternative to the classic systems. Development of a good controller for a fed-batch reactor is not enough without a fault diagnosis system. We are working to expand this idea. The failure detection system is based on the parity space approach joined with a fuzzy controller with more robustness and of superior performance in MIMO systems compared to conventional strategies. A 2-week period is reported.     

Postprandial blood glucose control using a hybrid adaptive PD controller with insulin-on-board limitation

This paper addresses the design of blood glucose control during the postprandial period for Type 1 diabetes patients. An artificial pancreas for ambulatory purposes has to deal with the delays inherent to the subcutaneous route, the carbohydrate intakes, the metabolic changes, the glucose sensor errors and noise, and the insulin pump constraints. A time response typically obtained in closed-loop insulin delivery shows hyperglycemia in the early postprandial period caused by the lag in the insulin absorbtion, followed by hypoglycemia caused by control over-reaction. A hybrid control system is proposed in this paper to overcome these problems. An insulin bolus is administered prior to the meals like in open-loop control, whereas a PD controller is used for robust glucose regulation. The controller gain is progressively increased after the bolus from zero up to its nominal value as function of the insulin on board, so that the PD controller becomes fully operational just when the insulin on board falls below a prescribed value. An excessive accumulation of active insulin is avoided in this way, drastically reducing the risk of hypoglycemia. The controller gain is adapted by means of a variable structure algorithm, allowing a very simple software implementation. The robust performance of the control algorithm is intensively assessed in silico on a cohort of virtual patients under challenging realistic scenarios considering mixed meals, circadian variations, time-varying uncertainties, discrete measurement and actuation, sensor errors and other disturbances.     

Analysis of an optimal control model of multi-joint arm movements

 In this paper, we propose a model of biological motor control for generation of goal-directed multi-joint arm movements, and study the formation of muscle control inputs and invariant kinematic features of movements. The model has a hierarchical structure that can determine the control inputs for a set of redundant muscles without any inverse computation. Calculation of motor commands is divided into two stages, each of which performs a transformation of motor commands from one coordinate system to another. At the first level, a central controller in the brain accepts instructions from higher centers, which represent the motor goal in the Cartesian space. The controller computes joint equilibrium trajectories and excitation signals according to a minimum effort criterion. At the second level, a neural network in the spinal cord translates the excitation signals and equilibrium trajectories into control commands to three pairs of antagonist muscles which are redundant for a two-joint arm. No inverse computation is required in the determination of individual muscle commands. The minimum effort controller can produce arm movements whose dynamic and kinematic features are similar to those of voluntary arm movements. For fast movements, the hand approaches a target position along a near-straight path with a smooth bell-shaped velocity. The equilibrium trajectories in X and Y show an ‘N’ shape, but the end-point equilibrium path zigzags around the hand path. Joint movements are not always smooth. Joint reversal is found in movements in some directions. The excitation signals have a triphasic (or biphasic) pulse pattern, which leads to stereotyped triphasic (or biphasic) bursts in muscle control inputs, and a dynamically modulated joint stiffness. There is a fixed sequence of muscle activation from proximal muscles to distal muscles. The order is preserved in all movements. For slow movements, it is shown that a constant joint stiffness is necessary to produce a smooth movement with a bell-shaped velocity. Scaled movements can be reproduced by varying the constraints on the maximal level of excitation signals according to the speed of movement. When the inertial parameters of the arm are altered, movement trajectories can be kept invariant by adjusting the pulse height values, showing the ability to adapt to load changes. These results agree with a wide range of experimental observations on human voluntary movements. Received: 4 December 1995 / Accepted in revised form: 17 September 1996     

A method for measurement of joint kinematics in vivo by registration of 3-D geometric models with cine phase contrast magnetic resonance imaging data

A new method is presented for measuring joint kinematics by optimally matching modeled trajectories of geometric surface models of bones with cine phase contrast (cine-PC) magnetic resonance imaging data. The incorporation of the geometric bone models (GBMs) allows computation of kinematics based on coordinate systems placed relative to full 3-D anatomy, as well as quantification of changes in articular contact locations and relative velocities during dynamic motion. These capabilities are additional to those of cine-PC based techniques that have been used previously to measure joint kinematics during activity. Cine-PC magnitude and velocity data are collected on a fixed image plane prescribed through a repetitively moved skeletal joint. The intersection of each GBM with a simulated image plane is calculated as the model moves along a computed trajectory, and cine-PC velocity data are sampled from the regions of the velocity images within the area of this intersection. From the sampled velocity data, the instantaneous linear and angular velocities of a coordinate system fixed to the GBM are estimated, and integration of the linear and angular velocities is used to predict updated trajectories. A moving validation phantom that produces motions and velocity data similar to those observed in an experiment on human knee kinematics was designed. This phantom was used to assess cine-PC rigid body tracking performance by comparing the kinematics of the phantom measured by this method to similar measurements made using a magnetic tracking system. Average differences between the two methods were measured as 2.82 mm rms for anterior/posterior tibial position, and 2.63 deg rms for axial rotation. An intertrial repeatability study of human knee kinematics using the new method produced rms differences in anterior/posterior tibial position and axial rotation of 1.44 mm and 2.35 deg. The performance of the method is concluded to be sufficient for the effective study of kinematic changes caused to knees by soft tissue injuries.     

Modelling 3D control of upright stance using an optimal control strategy

A 3D balance control model of quiet upright stance is presented, based on an optimal control strategy, and evaluated in terms of its ability to simulate postural sway in both the anterior–posterior and medial–lateral directions. The human body was represented as a two-segment inverted pendulum. Several assumptions were made to linearise body dynamics, for example, that there was no transverse rotation during upright stance. The neural controller was presumed to be an optimal controller that generates ankle control torque and hip control torque according to certain performance criteria. An optimisation procedure was used to determine the values of unspecified model parameters including random disturbance gains and sensory delay times. This model was used to simulate postural sway behaviours characterised by centre-of-pressure (COP)-based measures. Confidence intervals for all normalised COP-based measures contained unity, indicating no significant differences between any of the simulated COP-based measures and corresponding experimental references. In addition, mean normalised errors for the traditional measures were < 8%, and those for most statistical mechanics measures were ～3–66%. On the basis these results, the proposed 3D balance control model appears to have the ability to accurately simulate 3D postural sway behaviours.