Personalized Tuning of a Reinforcement Learning Control Algorithm for Glucose Regulation

Artificial pancreas is in the forefront of research towards the automatic insulin infusion for patients with type 1 diabetes. Due to the high inter- and intra-variability of the diabetic population, the need for personalized approaches has been raised. This study presents an adaptive, patient-specific control strategy for glucose regulation based on reinforcement learning and more specifically on the Actor-Critic (AC) learning approach. The control algorithm provides daily updates of the basal rate and insulin-to-carbohydrate (IC) ratio in order to optimize glucose regulation. A method for the automatic and personalized initialization of the control algorithm is designed based on the estimation of the transfer entropy (TE) between insulin and glucose signals. The algorithm has been evaluated in silico in adults, adolescents and children for 10 days. Three scenarios of initialization to i) zero values, ii) random values and iii) TE-based values have been comparatively assessed. The results have shown that when the TE-based initialization is used, the algorithm achieves faster learning with 98%, 90% and 73% in the A+B zones of the Control Variability Grid Analysis for adults, adolescents and children respectively after five days compared to 95%, 78%, 41% for random initialization and 93%, 88%, 41% for zero initial values. Furthermore, in the case of children, the daily Low Blood Glucose Index reduces much faster when the TE-based tuning is applied. The results imply that automatic and personalized tuning based on TE reduces the learning period and improves the overall performance of the AC algorithm.

Pulsatile control of rotary blood pumps: Does the modulation waveform matter?

Mechanical support of a failing heart is typically performed with rotary blood pumps running at constant speed, which results in a limited control on cardiac workload and nonpulsatile hemodynamics. A potential solution to overcome these limitations is to modulate the pump speed to create pulses. This study aims at developing a pulsatile control algorithm for rotary pumps, while investigating its effect on left ventricle unloading and the hemodynamics.

A Physiological Controller for Turbodynamic Ventricular Assist Devices Based on a Measurement of the Left Ventricular Volume

The current article presents a novel physiological control algorithm for ventricular assist devices (VADs), which is inspired by the preload recruitable stroke work. This controller adapts the hydraulic power output of the VAD to the end-diastolic volume of the left ventricle. We tested this controller on a hybrid mock circulation where the left ventricular volume (LVV) is known, i.e., the problem of measuring the LVV is not addressed in the current article. Experiments were conducted to compare the response of the controller with the physiological and with the pathological circulation, with and without VAD support. A sensitivity analysis was performed to analyze the influence of the controller parameters and the influence of the quality of the LVV signal on the performance of the control algorithm. The results show that the controller induces a response similar to the physiological circulation and effectively prevents over- and underpumping, i.e., ventricular suction and backflow from the aorta to the left ventricle, respectively. The same results are obtained in the case of a disturbed LVV signal. The results presented in the current article motivate the development of a robust, long-term stable sensor to measure the LVV.

Adaptive Algorithms for Personalized Diabetes Treatment

Dynamic systems, especially in real-life applications, are often determined by inter-/intra-variability, uncertainties and time-varying components. Physiological systems are probably the most representative example in which population variability, vital signal measurement noise and uncertain dynamics render their explicit representation and optimization a rather difficult task. Systems characterized by such challenges often require the use of adaptive algorithmic solutions able to perform an iterative structural and/or parametrical update process towards optimized behavior. Adaptive optimization presents the advantages of (i) individualization through learning of basic system characteristics, (ii) ability to follow time-varying dynamics and (iii) low computational cost. In this chapter, the use of online adaptive algorithms is investigated in two basic research areas related to diabetes management: (i) real-time glucose regulation and (ii) real-time prediction of hypo-/hyperglycemia. The applicability of these methods is illustrated through the design and development of an adaptive glucose control algorithm based on reinforcement learning and optimal control and an adaptive, personalized early-warning system for the recognition and alarm generation against hypo- and hyperglycemic events.

GoCARB in the Context of an Artificial Pancreas

Background: In an artificial pancreas (AP), the meals are either manually announced or detected and their size estimated from the blood glucose level. Both methods have limitations, which result in suboptimal postprandial glucose control. The GoCARB system is designed to provide the carbohydrate content of meals and is presented within the AP framework. Method: The combined use of GoCARB with a control algorithm is assessed in a series of 12 computer simulations. The simulations are defined according to the type of the control (open or closed loop), the use or not-use of GoCARB and the diabetics’ skills in carbohydrate estimation. Results: For bad estimators without GoCARB, the percentage of the time spent in target range (70-180 mg/dl) during the postprandial period is 22.5% and 66.2% for open and closed loop, respectively. When the GoCARB is used, the corresponding percentages are 99.7% and 99.8%. In case of open loop, the time spent in severe hypoglycemic events (<50 mg/dl) is 33.6% without the GoCARB and is reduced to 0.0% when the GoCARB is used. In case of closed loop, the corresponding percentage is 1.4% without the GoCARB and is reduced to 0.0% with the GoCARB. Conclusion: The use of GoCARB improves the control of postprandial response and glucose profiles especially in the case of open loop. However, the most efficient regulation is achieved by the combined use of the control algorithm and the GoCARB.

An insulin infusion advisory system based on autotuning nonlinear model-predictive control

This paper aims at the development and evaluation of a personalized insulin infusion advisory system (IIAS), able to provide real-time estimations of the appropriate insulin infusion rate for type 1 diabetes mellitus (T1DM) patients using continuous glucose monitors and insulin pumps. The system is based on a nonlinear model-predictive controller (NMPC) that uses a personalized glucose-insulin metabolism model, consisting of two compartmental models and a recurrent neural network. The model takes as input patient's information regarding meal intake, glucose measurements, and insulin infusion rates, and provides glucose predictions. The predictions are fed to the NMPC, in order for the latter to estimate the optimum insulin infusion rates. An algorithm based on fuzzy logic has been developed for the on-line adaptation of the NMPC control parameters. The IIAS has been in silico evaluated using an appropriate simulation environment (UVa T1DM simulator). The IIAS was able to handle various meal profiles, fasting conditions, interpatient variability, intraday variation in physiological parameters, and errors in meal amount estimations.

Rapid parallel adaptive radiations from a single hybridogenic ancestral population

The Alpine lake whitefish (Coregonus lavaretus) species complex is a classic example of a recent radiation, associated with colonization of the Alpine lakes following the glacial retreat (less than 15 kyr BP). They have formed a unique array of endemic lake flocks, each with one to six described sympatric species differing in morphology, diet and reproductive ecology. Here, we present a genomic investigation of the relationships between and within the lake flocks. Comparing the signal between over 1000 AFLP loci and mitochondrial control region sequence data, we use phylogenetic tree-based and population genetic methods to reconstruct the phylogenetic history of the group and to delineate the principal centres of genetic diversity within the radiation. We find significant cytonuclear discordance showing that the genomically monophyletic Alpine whitefish clade arose from a hybrid swarm of at least two glacial refugial lineages. Within this radiation, we find seven extant genetic clusters centred on seven lake systems. Most interestingly, we find evidence of sympatric speciation within and parallel evolution of equivalent phenotypes among these lake systems. However, we also find the genetic signature of human-mediated gene flow and diversity loss within many lakes, highlighting the fragility of recent radiations.

Subcortical electrostimulation to identify network subserving motor control

Objectives: Recent anatomical-functional studies have transformed our understanding of cerebral motor control away from a hierarchical structure and toward parallel and interconnected specialized circuits. Subcortical electrical stimulation during awake surgery provides a unique opportunity to identify white matter tracts involved in motor control. For the first time, this study reports the findings on motor modulatory responses evoked by subcortical stimulation and investigates the cortico-subcortical connectivity of cerebral motor control. Experimental design: Twenty-one selected patients were operated while awake for frontal, insular, and parietal diffuse low-grade gliomas. Subcortical electrostimulation mapping was used to search for interference with voluntary movements. The corresponding stimulation sites were localized on brain schemas using the anterior and posterior commissures method. Principal observations: Subcortical negative motor responses were evoked in 20/21 patients, whereas acceleration of voluntary movements and positive motor responses were observed in three and five patients, respectively. The majority of the stimulation sites were detected rostral of the corticospinal tract near the vertical anterior-commissural line, and additional sites were seen in the frontal and parietal white matter. Conclusions: The diverse interferences with motor function resulting in inhibition and acceleration imply a modulatory influence of the detected fiber network. The subcortical stimulation sites were distributed veil-like, anterior to the primary motor fibers, suggesting descending pathways originating from premotor areas known for negative motor response characteristics. Further stimulation sites in the parietal white matter as well as in the anterior arm of the internal capsule indicate a large-scale fronto-parietal motor control network. Hum Brain Mapp, 2012. © 2012 Wiley Periodicals, Inc.

Parallel evolution of facial stripe patterns in the Neolamprologus brichardilpulcher species complex endemic to Lake Tanganyika

Colour pattern diversity can be due to random processes or to natural or sexual selection. Consequently, similarities in colour patterns are not always correlated with common ancestry, but may result from convergent evolution under shared selection pressures or drift. Neolamprologus brichardi and Neolamprologus pulcher have been described as two distinct species based on differences in the arrangement of two dark bars on the operculum. Our study uses DNA sequences of the mitochondrial control region to show that relatedness of haplotypes disagrees with species assignment based on head colour pattern. This suggests repeated parallel evolution of particular stripe patterns. The complete lack of shared haplotypes between populations of the same or different phenotypes reflects strong philopatric behaviour, possibly induced by the cooperative breeding mode in which offspring remain in their natal territory and serve as helpers until they disperse to nearby territories or take over a breeding position. Concordant phylogeographic patterns between N. brichardi/N. pulcher populations and other rock-dwelling cichlids suggest that the same colonization routes have been taken by sympatric species and that these routes were affected by lake level fluctuations in the past. (C) 2007 Elsevier Inc. All rights reserved.

An integral conservative gridding--algorithm using Hermitian curve interpolation

The problem of re-sampling spatially distributed data organized into regular or irregular grids to finer or coarser resolution is a common task in data processing. This procedure is known as 'gridding' or 're-binning'. Depending on the quantity the data represents, the gridding-algorithm has to meet different requirements. For example, histogrammed physical quantities such as mass or energy have to be re-binned in order to conserve the overall integral. Moreover, if the quantity is positive definite, negative sampling values should be avoided. The gridding process requires a re-distribution of the original data set to a user-requested grid according to a distribution function. The distribution function can be determined on the basis of the given data by interpolation methods. In general, accurate interpolation with respect to multiple boundary conditions of heavily fluctuating data requires polynomial interpolation functions of second or even higher order. However, this may result in unrealistic deviations (overshoots or undershoots) of the interpolation function from the data. Accordingly, the re-sampled data may overestimate or underestimate the given data by a significant amount. The gridding-algorithm presented in this work was developed in order to overcome these problems. Instead of a straightforward interpolation of the given data using high-order polynomials, a parametrized Hermitian interpolation curve was used to approximate the integrated data set. A single parameter is determined by which the user can control the behavior of the interpolation function, i.e. the amount of overshoot and undershoot. Furthermore, it is shown how the algorithm can be extended to multidimensional grids. The algorithm was compared to commonly used gridding-algorithms using linear and cubic interpolation functions. It is shown that such interpolation functions may overestimate or underestimate the source data by about 10-20%, while the new algorithm can be tuned to significantly reduce these interpolation errors. The accuracy of the new algorithm was tested on a series of x-ray CT-images (head and neck, lung, pelvis). The new algorithm significantly improves the accuracy of the sampled images in terms of the mean square error and a quality index introduced by Wang and Bovik (2002 IEEE Signal Process. Lett. 9 81-4).

Liraglutide once a day versus exenatide twice a day for type 2 diabetes: a 26-week randomised, parallel-group, multinational, open-label trial (LEAD-6)

BACKGROUND: Unlike most antihyperglycaemic drugs, glucagon-like peptide-1 (GLP-1) receptor agonists have a glucose-dependent action and promote weight loss. We compared the efficacy and safety of liraglutide, a human GLP-1 analogue, with exenatide, an exendin-based GLP-1 receptor agonist. METHODS: Adults with inadequately controlled type 2 diabetes on maximally tolerated doses of metformin, sulphonylurea, or both, were stratified by previous oral antidiabetic therapy and randomly assigned to receive additional liraglutide 1.8 mg once a day (n=233) or exenatide 10 microg twice a day (n=231) in a 26-week open-label, parallel-group, multinational (15 countries) study. The primary outcome was change in glycosylated haemoglobin (HbA(1c)). Efficacy analyses were by intention to treat. The trial is registered with ClinicalTrials.gov, number NCT00518882. FINDINGS: Mean baseline HbA(1c) for the study population was 8.2%. Liraglutide reduced mean HbA(1c) significantly more than did exenatide (-1.12% [SE 0.08] vs -0.79% [0.08]; estimated treatment difference -0.33; 95% CI -0.47 to -0.18; p<0.0001) and more patients achieved a HbA(1c) value of less than 7% (54%vs 43%, respectively; odds ratio 2.02; 95% CI 1.31 to 3.11; p=0.0015). Liraglutide reduced mean fasting plasma glucose more than did exenatide (-1.61 mmol/L [SE 0.20] vs -0.60 mmol/L [0.20]; estimated treatment difference -1.01 mmol/L; 95% CI -1.37 to -0.65; p<0.0001) but postprandial glucose control was less effective after breakfast and dinner. Both drugs promoted similar weight losses (liraglutide -3.24 kg vs exenatide -2.87 kg). Both drugs were well tolerated, but nausea was less persistent (estimated treatment rate ratio 0.448, p<0.0001) and minor hypoglycaemia less frequent with liraglutide than with exenatide (1.93 vs 2.60 events per patient per year; rate ratio 0.55; 95% CI 0.34 to 0.88; p=0.0131; 25.5%vs 33.6% had minor hypoglycaemia). Two patients taking both exenatide and a sulphonylurea had a major hypoglycaemic episode. INTERPRETATION: Liraglutide once a day provided significantly greater improvements in glycaemic control than did exenatide twice a day, and was generally better tolerated. The results suggest that liraglutide might be a treatment option for type 2 diabetes, especially when weight loss and risk of hypoglycaemia are major considerations.

An insulin infusion advisory system for type 1 diabetes patients based on non-linear model predictive control methods

In this paper, an Insulin Infusion Advisory System (IIAS) for Type 1 diabetes patients, which use insulin pumps for the Continuous Subcutaneous Insulin Infusion (CSII) is presented. The purpose of the system is to estimate the appropriate insulin infusion rates. The system is based on a Non-Linear Model Predictive Controller (NMPC) which uses a hybrid model. The model comprises a Compartmental Model (CM), which simulates the absorption of the glucose to the blood due to meal intakes, and a Neural Network (NN), which simulates the glucose-insulin kinetics. The NN is a Recurrent NN (RNN) trained with the Real Time Recurrent Learning (RTRL) algorithm. The output of the model consists of short term glucose predictions and provides input to the NMPC, in order for the latter to estimate the optimum insulin infusion rates. For the development and the evaluation of the IIAS, data generated from a Mathematical Model (MM) of a Type 1 diabetes patient have been used. The proposed control strategy is evaluated at multiple meal disturbances, various noise levels and additional time delays. The results indicate that the implemented IIAS is capable of handling multiple meals, which correspond to realistic meal profiles, large noise levels and time delays.

Verbal instructions and top-down saccade control.

Few studies have addressed the interaction between instruction content and saccadic eye movement control. To assess the impact of instructions on top-down control, we instructed 20 healthy volunteers to deliberately delay saccade triggering, to make inaccurate saccades or to redirect saccades--i.e. to glimpse towards and then immediately opposite to the target. Regular pro- and antisaccade tasks were used for comparison. Bottom-up visual input remained unchanged and was a gap paradigm for all instructions. In the inaccuracy and delay tasks, both latencies and accuracies were detrimentally impaired by either type of instruction and the variability of latency and accuracy was increased. The intersaccadic interval (ISI) required to correct erroneous antisaccades was shorter than the ISI for instructed direction changes in the redirection task. The word-by-word instruction content interferes with top-down saccade control. Top-down control is a time consuming process, which may override bottom-up processing only during a limited time period. It is questionable whether parallel processing is possible in top-down control, since the long ISI for instructed direction changes suggests sequential planning.

Numerical solution of optimal control problems with constant control delays

We investigate a class of optimal control problems that exhibit constant exogenously given delays in the control in the equation of motion of the differential states. Therefore, we formulate an exemplary optimal control problem with one stock and one control variable and review some analytic properties of an optimal solution. However, analytical considerations are quite limited in case of delayed optimal control problems. In order to overcome these limits, we reformulate the problem and apply direct numerical methods to calculate approximate solutions that give a better understanding of this class of optimization problems. In particular, we present two possibilities to reformulate the delayed optimal control problem into an instantaneous optimal control problem and show how these can be solved numerically with a stateof- the-art direct method by applying Bock’s direct multiple shooting algorithm. We further demonstrate the strength of our approach by two economic examples.

Low-frequency drug-resistant HIV-1 and risk of virological failure to first-line NNRTI-based ART: a multicohort European case-control study using centralized ultrasensitive 454 pyrosequencing.

OBJECTIVES It is still debated if pre-existing minority drug-resistant HIV-1 variants (MVs) affect the virological outcomes of first-line NNRTI-containing ART. METHODS This Europe-wide case-control study included ART-naive subjects infected with drug-susceptible HIV-1 as revealed by population sequencing, who achieved virological suppression on first-line ART including one NNRTI. Cases experienced virological failure and controls were subjects from the same cohort whose viraemia remained suppressed at a matched time since initiation of ART. Blinded, centralized 454 pyrosequencing with parallel bioinformatic analysis in two laboratories was used to identify MVs in the 1%-25% frequency range. ORs of virological failure according to MV detection were estimated by logistic regression. RESULTS Two hundred and sixty samples (76 cases and 184 controls), mostly subtype B (73.5%), were used for the analysis. Identical MVs were detected in the two laboratories. 31.6% of cases and 16.8% of controls harboured pre-existing MVs. Detection of at least one MV versus no MVs was associated with an increased risk of virological failure (OR = 2.75, 95% CI = 1.35-5.60, P = 0.005); similar associations were observed for at least one MV versus no NRTI MVs (OR = 2.27, 95% CI = 0.76-6.77, P = 0.140) and at least one MV versus no NNRTI MVs (OR = 2.41, 95% CI = 1.12-5.18, P = 0.024). A dose-effect relationship between virological failure and mutational load was found. CONCLUSIONS Pre-existing MVs more than double the risk of virological failure to first-line NNRTI-based ART.