The Policy Iteration Algorithm for Average Continuous Control of Piecewise Deterministic Markov Processes

The main goal of this paper is to apply the so-called policy iteration algorithm (PIA) for the long run average continuous control problem of piecewise deterministic Markov processes (PDMP`s) taking values in a general Borel space and with compact action space depending on the state variable. In order to do that we first derive some important properties for a pseudo-Poisson equation associated to the problem. In the sequence it is shown that the convergence of the PIA to a solution satisfying the optimality equation holds under some classical hypotheses and that this optimal solution yields to an optimal control strategy for the average control problem for the continuous-time PDMP in a feedback form.

THE VANISHING DISCOUNT APPROACH FOR THE AVERAGE CONTINUOUS CONTROL OF PIECEWISE DETERMINISTIC MARKOV PROCESSES

This work is concerned with the existence of an optimal control strategy for the long-run average continuous control problem of piecewise-deterministic Markov processes (PDMPs). In Costa and Dufour (2008), sufficient conditions were derived to ensure the existence of an optimal control by using the vanishing discount approach. These conditions were mainly expressed in terms of the relative difference of the alpha-discount value functions. The main goal of this paper is to derive tractable conditions directly related to the primitive data of the PDMP to ensure the existence of an optimal control. The present work can be seen as a continuation of the results derived in Costa and Dufour (2008). Our main assumptions are written in terms of some integro-differential inequalities related to the so-called expected growth condition, and geometric convergence of the post-jump location kernel associated to the PDMP. An example based on the capacity expansion problem is presented, illustrating the possible applications of the results developed in the paper.

Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes

This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP`s) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space a""e (n) . The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter epsilon > 0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem as epsilon goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as epsilon goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.

Closed-loop model re-identification of processes under MPC with zone control

This work deals with a procedure for model re-identification of a process in closed loop with ail already existing commercial MPC. The controller considered here has a two-layer structure where the upper layer performs a target calculation based on a simplified steady-state optimization of the process. Here, it is proposed a methodology where a test signal is introduced in a tuning parameter of the target calculation layer. When the outputs are controlled by zones instead of at fixed set points, the approach allows the continuous operation of the process without an excessive disruption of the operating objectives as process constraints and product specifications remain satisfied during the identification test. The application of the method is illustrated through the simulation of two processes of the oil refining industry. (c) 2008 Elsevier Ltd. All rights reserved.

SELF-SIMILARITY AND LAMPERTI CONVERGENCE FOR FAMILIES OF STOCHASTIC PROCESSES

We define a new type of self-similarity for one-parameter families of stochastic processes, which applies to certain important families of processes that are not self-similar in the conventional sense. This includes Hougaard Levy processes such as the Poisson processes, Brownian motions with drift and the inverse Gaussian processes, and some new fractional Hougaard motions defined as moving averages of Hougaard Levy process. Such families have many properties in common with ordinary self-similar processes, including the form of their covariance functions, and the fact that they appear as limits in a Lamperti-type limit theorem for families of stochastic processes.

Deciphering the weathering processes using environmental mineralogy and geochemistry: Towards an integrated model of laterite and podzol genesis in the Upper Amazon Basin

This article reports major results from collaborative research between France and Brazil on soil and water systems, carried out in the Upper Amazon Basin. It reveals the weathering processes acting in the partly inundated, low elevation plateaus of the Basin, mostly covered by evergreen forest. Our findings are based on geochemical data and mineral spectroscopy that probe the crystal chemistry of Fe and Al in mineral phases (mainly kaolinite, Al- and Fe-(hydr)oxides) of tropical soils (laterites). These techniques reveal crystal alterations in mineral populations of different ages and changes of metal speciation associated with mineral or organic phases. These results provide an integrated model of soil formation and changes (from laterites to podzols) in distinct hydrological compartments of the Amazon landscapes and under altered water regimes. (C) 2010 Academie des sciences. Published by Elsevier Masson SAS. All rights reserved.

Biochemistry of the Envenomation Response-a Generator Theme for Interdisciplinary Integration

The understanding of complex physiological processes requires information from many different areas of knowledge. To meet this interdisciplinary scenario, the ability of integrating and articulating information is demanded. The difficulty of such approach arises because, more often than not, information is fragmented through under graduation education in Health Sciences. Shifting from a fragmentary and deep view of many topics to joining them horizontally in a global view is not a trivial task for teachers to implement. To attain that objective we proposed a course herein described Biochemistry of the envenomation response aimed at integrating previous contents of Health Sciences courses, following international recommendations of interdisciplinary model. The contents were organized by modules with increasing topic complexity. The full understanding of the envenoming pathophysiology of each module would be attained by the integration of knowledge from different disciplines. Active-learning strategy was employed focusing concept map drawing. Evaluation was obtained by a 30-item Likert-type survey answered by ninety students; 84% of the students considered that the number of relations that they were able to establish as seen by concept maps increased throughout the course. Similarly, 98% considered that both the theme and the strategy adopted in the course contributed to develop an interdisciplinary view.

Nuclear Quadrupole Resonance: A Technique to Control Hydration Processes in the Pharmaceutical Industry

Pharmaceuticals can exist in many solid forms, which can have different physical and chemical properties. These solid forms include polymorphs, solvates, amorphous, and hydrates. Particularly, hydration process can be quite common since pharmaceutical solids can be in contact with water during manufacturing process and can also be exposed to water during storage. In the present work, it is proved that NQR technique is capable of detecting different hydrated forms not only in the pure raw material but also in the final product (tablets), being in this way a useful technique for quality control. This technique was also used to study the dehydration process from pentahydrate to trihydrate.

Information Society, Teaching, and Learning: Education Reforms in Brazil

This article examines the subject matter of learning within the context of information society, through an inquiry concerning both the reforms in education adopted in Brazil in the last thirty years and their results. It provides a revision on the explanations of school failure based on assumptions of learning problems due to cognitive and linguistic deficits. From the guidelines related with written school forms as well as the constant cultural oppression accomplished inside the school, the article claims the necessity of changing the psychological and pedagogic views that, under the label of democratic practices, determine school institutions and its daily life, by means of instrumental relations with knowledge that disregard the reading practices which are congenial to popular culture.

Effects of postnatal protein malnutrition on learning and memory procedures

Protein malnutrition induces structural, neurochemical and functional changes in the central nervous system leading to alterations in cognitive and behavioral development of rats. The aim of this work was to investigate the effects of postnatal protein malnutrition on learning and memory tasks. Previously malnourished (6% protein) and well-nourished rats (16% protein) were tested in three experiments: working memory tasks in the Morris water maze (Experiment I), recognition memory of objects (Experiment II), and working memory in the water T-maze (Experiment III). The results showed higher escape latencies in malnourished animals in Experiment I, lower recognition indexes of malnourished animals in Experiment II, and no differences due to diet in Experiment III. It is suggested that protein malnutrition imposed on early life of rats can produce impairments on both working memory in the Morris maze and recognition memory in the open field tests.

Occasion setting in Pavlovian ambiguous target discriminations

Rats were trained in a Pavlovian serial ambiguous target discrimination, in which a target cue was reinforced if it was preceded by one stimulus (P -> T+) but was not reinforced if it was preceded by another stimulus (N -> T-). Test performance indicated that stimulus control by these features was weaker than that acquired by features trained within separate serial feature positive (P -> T+, T-) and serial feature negative (N -> W-, W+) discriminations. The form of conditioned responding and the patterns of transfer observed suggested that the serial ambiguous target discrimination was solved by occasion setting. The data are discussed in terms of the use of retrospective coding strategies when solving Pavlovian serial conditional discriminations, and the acquisition of special properties by both feature and target stimuli. (C) 2008 Published by Elsevier B.V.

Interaction of adrenocorticotropin peptides with microheterogeneous systems - A fluorescence study

Adrenocorticotropin (ACM) and alpha-melanocyte stimulating hormone (alpha-MSH) are peptides which present many physiological effects related to pigmentation, motor and sexual behavior, learning and memory, analgesia, anti-inflammatory and antipyretic processes. The 13 amino acid residues of alpha-MSH are the same initial sequence of ACM and due to the presence of a tryptophan residue in position 9 of the peptide chain, fluorescence techniques could be used to investigate the conformational properties of the hormones in different environments and the mechanisms of interaction with biomimetic systems like sodium dodecyl sulphate (SDS) micelles, sodium dodecyl sulphate-poly(ethylene oxide) (SDS-PEO) aggregates and neutral polymeric micelles. In buffer solution, fluorescence parameters were typical of peptides containing tryptophan exposed to the aqueous medium and upon addition of surfactant and polymer molecules, the gradual change of those parameters demonstrated the interaction of the peptides with the microheterogeneous systems. From time-resolved experiments it was shown that the interaction proceeded with conformational changes in both peptides, and further information was obtained from quenching of Trp fluorescence by a family of N-alkylpyridinium ions, which possess affinity to the microheterogeneous systems dependent on the length of the alkyl chain. The quenching of Trp fluorescence was enhanced in the presence of charged micelles, compared to the buffer solution and the accessibility of the fluorophore to the quencher was dependent on the peptide and the alkylpyridinium: in ACTH(1-21) highest collisional constants were obtained using ethylpyridinium as quencher, indicating a location of the residue in the surface of the micelle, while in alpha-MSH the best quencher was hexylpyridinium, indicating insertion of the residue into the non-polar region of the micelles. The results had shown that the interaction between the peptides and the biomimetic systems where driven by combined electrostatic and hydrophobic effects: in ACTH(1-24) the electrostatic interaction between highly positively charged C-terminal and negatively charged surface of micelles; and aggregates predominates over hydrophobic interactions involving residues in the central region of the peptide; in alpha-MSH, which presents one residual positive charge, the hydrophobic interactions are relevant to position the Trp residue in the non-polar region of the microheterogeneous systems. (C) 2008 Elsevier B.V. All rights reserved.

Improving Alzheimer`s Disease Diagnosis with Machine Learning Techniques

There is not a specific test to diagnose Alzheimer`s disease (AD). Its diagnosis should be based upon clinical history, neuropsychological and laboratory tests, neuroimaging and electroencephalography (EEG). Therefore, new approaches are necessary to enable earlier and more accurate diagnosis and to follow treatment results. In this study we used a Machine Learning (ML) technique, named Support Vector Machine (SVM), to search patterns in EEG epochs to differentiate AD patients from controls. As a result, we developed a quantitative EEG (qEEG) processing method for automatic differentiation of patients with AD from normal individuals, as a complement to the diagnosis of probable dementia. We studied EEGs from 19 normal subjects (14 females/5 males, mean age 71.6 years) and 16 probable mild to moderate symptoms AD patients (14 females/2 males, mean age 73.4 years. The results obtained from analysis of EEG epochs were accuracy 79.9% and sensitivity 83.2%. The analysis considering the diagnosis of each individual patient reached 87.0% accuracy and 91.7% sensitivity.

Ligaments of the Posterior and Lateral Talar Processes: MRI and MR Arthrography of the Ankle and Posterior Subtalar Joint with Anatomic and Histologic Correlation

OBJECTIVE. MRI and combined ankle and posterior subtalar MR arthrography in cadavers were used to evaluate the ligaments of the posterior and lateral talar processes. Subsequent anatomic and histologic correlation was performed. MATERIALS AND METHODS. Ten cadaveric ankles were used. Routine radiography and MRI were initially performed. Ankle and posterior subtalar MR arthrography, followed by anatomic and histologic analysis, was then performed to allow better assessment of the ligaments of the lateral and posterior talar process. RESULTS. In all subjects, MR arthrography provided superior delineation of the articular and periarticular structures, as well as the ligaments. The lateral talocalcaneal and medial talocalcaneal ligaments were best seen in the axial and coronal planes, respectively. The axial plane was best for visualizing the fibulotalocalcaneal ligament, and the sagittal plane was best for evaluating the posterior talocalcaneal ligament. The anterior and posterior talofibular ligaments and the posterior tibiotalar ligament (superficial and deep portions) were best seen in the axial plane. Histologic analysis was correlated to anatomic sectioning and showed the attachment sites of these ligaments. CONCLUSION. Combined ankle and posterior subtalar MR arthrography enhances visualization of the ligaments attaching to the posterior and lateral talar processes, including the posterior, lateral, and medial talocalcaneal and fibulotalocalcaneal ligaments.

The impact of functional connectivity changes on support vector machines mapping of fMRI data

Functional magnetic resonance imaging (fMRI) is currently one of the most widely used methods for studying human brain function in vivo. Although many different approaches to fMRI analysis are available, the most widely used methods employ so called ""mass-univariate"" modeling of responses in a voxel-by-voxel fashion to construct activation maps. However, it is well known that many brain processes involve networks of interacting regions and for this reason multivariate analyses might seem to be attractive alternatives to univariate approaches. The current paper focuses on one multivariate application of statistical learning theory: the statistical discrimination maps (SDM) based on support vector machine, and seeks to establish some possible interpretations when the results differ from univariate `approaches. In fact, when there are changes not only on the activation level of two conditions but also on functional connectivity, SDM seems more informative. We addressed this question using both simulations and applications to real data. We have shown that the combined use of univariate approaches and SDM yields significant new insights into brain activations not available using univariate methods alone. In the application to a visual working memory fMRI data, we demonstrated that the interaction among brain regions play a role in SDM`s power to detect discriminative voxels. (C) 2008 Elsevier B.V. All rights reserved.