Modeling Stock Order Flows and Learning Market-Making from Data

Stock markets employ specialized traders, market-makers, designed to provide liquidity and volume to the market by constantly supplying both supply and demand. In this paper, we demonstrate a novel method for modeling the market as a dynamic system and a reinforcement learning algorithm that learns profitable market-making strategies when run on this model. The sequence of buys and sells for a particular stock, the order flow, we model as an Input-Output Hidden Markov Model fit to historical data. When combined with the dynamics of the order book, this creates a highly non-linear and difficult dynamic system. Our reinforcement learning algorithm, based on likelihood ratios, is run on this partially-observable environment. We demonstrate learning results for two separate real stocks.

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

An artificial economy based on reinforcement learning and agent based modeling

In this paper, we employ techniques from artificial intelligence such as reinforcement learning and agent based modeling as building blocks of a computational model for an economy based on conventions. First we model the interaction among firms in the private sector. These firms behave in an information environment based on conventions, meaning that a firm is likely to behave as its neighbors if it observes that their actions lead to a good pay off. On the other hand, we propose the use of reinforcement learning as a computational model for the role of the government in the economy, as the agent that determines the fiscal policy, and whose objective is to maximize the growth of the economy. We present the implementation of a simulator of the proposed model based on SWARM, that employs the SARSA(λ) algorithm combined with a multilayer perceptron as the function approximation for the action value function.

Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Dynamics of Aeromonas species isolated from wastewater treatment system

Aeromonas are widely distributed in the aquatic environment, and are considered to be emerging organisms that can produce a series of virulence factors. The present study was carried out in a sanitary sewage stabilization pond treatment system, located in Lins, State of Sao Paulo, Brazil. Most probable number was applied for estimation of the genus Aeromonas. Colony isolation was carried out on blood agar ampicillin and confirmed by biochemical characterization. Aeromonas species were isolated in 72.4% of influent samples, and in 55.2 and 48.3% of effluent from anaerobic and facultative lagoons, respectively. Thirteen Aeromonas species were isolated, representing most of the recognized species of these organisms. Even though it was possible to observe a tendency of decrease, total elimination of these organisms from the studied system was not achieved. Understanding of the pathogenic organism`s dynamics in wastewater treatment systems with a reuse potential is especially important because of the risk it represents.

Increasing consumption of ultra-processed foods and likely impact on human health: evidence from Brazil

Objective: To assess time trends in the contribution of processed foods to food purchases made by Brazilian households and to explore the potential impact on the overall quality of the diet. Design: Application of a new classification of foodstuffs based on extent and purpose of food processing to data collected by comparable probabilistic household budget surveys. The classification assigns foodstuffs to the following groups: unprocessed/minimally processed foods (Group 1); processed culinary ingredients (Group 2); or ultra-processed ready-to-eat or ready-to-heat food products (Group 3). Setting: Eleven metropolitan areas of Brazil. Subjects: Households; n 13 611 in 1987-8, n 16 014 in 1995-5 and n 13 848 in 2002-3. Results: Over the last three decades, the household consumption of Group 1 and Group 2 foods has been steadily replaced by consumption of Group 3 ultra-processed food products, both overall and in lower- and upper-income groups. In the 2002-3 survey, Group 3 items represented more than one-quarter of total energy (more than one-third for higher-income households). The overall nutrient profile of Group 3 items, compared with that of Group 1 and Group 2 items, revealed more added sugar, more saturated fat, more sodium, less fibre and much higher energy density. Conclusions: The high energy density and the unfavourable nutrition profiling of Group 3 food products, and also their potential harmful effects on eating and drinking behaviours, indicate that governments and health authorities should use all possible methods, including legislation and statutory regulation, to halt and reverse the replacement of minimally processed foods and processed culinary ingredients by ultra-processed food products.

Surface properties of eucalyptus pulp fibres as reinforcement of cement-based composites

The objective of the present work is to evaluate the effects of the surface properties of unrefined eucalyptus pulp fibres concerning their performance in cement-based composites. The influence of the fibre surface on the microstructure of fibre-cement composites was evaluated after accelerated ageing cycles, which simulate natural weathering. The surface of unbleached pulp is a thin layer that is rich in cellulose, lignin, hemicelluloses, and extractives. Such a layer acts as a physical and chemical barrier to the penetration of low molecular components of cement. The unbleached fibres are less hydrophilic than the bleached ones. Bleaching removes the amorphous lignin and extractives from the surface and renders it more permeable to liquids. Atomic force microscopy (AFM) helps in understanding the fibre-cement interface. Bleaching improved the fibre- cement interfacial bonding, whereas fibres in the unbleached pulp were less susceptible to the re-precipitation of cement hydration products into the fibre cavities (lumens). Therefore, unbleached fibres can improve the long-term performance of the fibre-cement composite owing to their delayed mineralization.

Risk of Giardia infection for drinking water and bathing in a peri-urban area in Sao Paulo, Brazil

A high incidence of waterborne diseases is observed worldwide and in order to address contamination problems prior to an outbreak, quantitative microbial risk assessment is a useful tool for estimating the risk of infection. The objective of this paper was to assess the probability of Giardia infection from consuming water from shallow wells in a peri-urban area. Giardia has been described as an important waterborne pathogen and reported in several water sources, including ground waters. Sixteen water samples were collected and examined according to the US EPA (1623, 2005). A Monte Carlo method was used to address the potential risk as described by the exponential dose response model. Giardia cysts occurred in 62.5% of the samples (0.1-36.1 cysts/l). A median risk of 10-1 for the population was estimated and the adult ingestion was the highest risk driver. This study illustrates the vulnerability of shallow well water supply systems in peri-urban areas.

Electrocombustion of humic acid and removal of algae from aqueous solutions

This paper reports experiments involving the electrochemical combustion of humic acid (HA) and removal of algae from pond water. An electrochemical flow reactor with a boron-doped diamond film anode was used and constant current experiments were conducted in batch recirculation mode. The mass transfer characteristics of the electrochemical device were determined by voltammetric experiments in the potential region of water stability, followed by a controlled current experiment in the potential region of oxygen evolution. The average mass transfer coefficient was 5.2 x 10(-5) m s(-1). The pond water was then processed to remove HA and algae in the conditions in which the reaction combustion occurred under mass transfer control. To this end, the mass transfer coefficient was used to estimate the initial limiting current density applied in the electrolytic experiments. As expected, all the parameters analyzed here-solution absorbance at 270 nm, total phenol concentration and total organic carbon concentration-decayed according to first-order kinetics. Since the diamond film anode successfully incinerated organic matter, the electrochemical system proved to be predictable and programmable.

Biobased composites from tannin-phenolic polymers reinforced with coir fibers

Tannin-phenolic polymers prepared using tannin, a macromolecule obtained from natural sources, were used in the preparation of composites reinforced with coir fibers. The composites based on tannin-phenolic polymers (50% (w/w) of tannin as substitute of the phenol) were prepared using the coir fibers as reinforcement (30-70% (w/w), 3.0-6.0 cm, randomly distributed). The Izod impact strength of the composites showed an improvement in this property due to the incorporation of coir fibers in the tannin-phenolic matrices. The SEM images showed excellent adhesion at the fiber/matrix interface. The coir fiber had bundles regularly spaced, which enhanced the diffusion of the resin into the fiber. In addition, the high lignin content of this fiber results in a high concentration of aromatic rings, which increased the compatibility with the matrix. The values of the diffusion coefficient of water, determined using Fick`s laws, show that there was no correlation between the fiber percentage and the water diffusion. The DMTA curves showed that the storage moduli of the composites reinforced with coir fibers were considerably higher than that of the thermoset, and the increase in the proportion of fibers led to a proportional increase in the storage moduli of these materials. The biobased composites obtained have potential for non-structural applications, such as in the internal parts of automotives vehicles. To our knowledge, this is the first study on this kind of biobased composites. (C) 2010 Elsevier B.V. All rights reserved.

Biobased composites from glyoxal-phenolic resins and sisal fibers

Lignocellulosic materials can significantly contribute to the development of biobased composites. In this work, glyoxal-phenolic resins for composites were prepared using glyoxal, which is a dialdehyde obtained from several natural resources. The resins were characterized by (1)H, (13)C, (2)D, and (31)P NMR spectroscopies. Resorcinol (10%) was used as an accelerator for curing the glyoxal-phenol resins in order to obtain the thermosets. The impact-strength measurement showed that regardless of the cure cycle used, the reinforcement of thermosets by 30% (w/w) sisal fibers improved the impact strength by one order of magnitude. Curing with cycle 1 (150 degrees C) induced a high diffusion coefficient for water absorption in composites, due to less interaction between the sisal fibers and water. The composites cured with cycle 2 (180 degrees C) had less glyoxal resin coverage of the cellulosic fibers, as observed by images of the fractured interface observed by SEM. This study shows that biobased composites with good properties can be prepared using a high proportion of materials obtained from natural resources. (C) 2009 Elsevier Ltd. All rights reserved.

The use of polyacrylamides (PAMs) for removing natural organic matter (NOM) from Paraiba do Sul River (Brazil)

This work assesses the efficiency of polyacrylamides for natural organic matter (NOM) removal from Paraiba do Sul River (Brazil) raw water for drinking purposes. Jar tests were performed following an experimental design protocol. Three kinds of polyacrylamides (anionic, cationic, and non-ionic) at 0.2 mg L(-1) were tested. After coagulation, turbidity, DOC, UVA(254) and SCAN (UV-absorbing material) were determined. Color and pH were also measured. It was found that polyacrylamides did not reduce the amounts of alum and lime needed in the process and that the amount of alum alone for removing UV-absorbing organic matter is significantly higher. Efficiency of the coagulation process decreased as follows: non-ionic -> cationic -> anionic -> no polyacrylamide. Removal efficiencies for the best case were: 100%, 90%, 83%, and 68% for turbidity, DOC, UVA(254), and SCAN, respectively.

Risk assessment of trihalomethanes from tap water in Fortaleza, Brazil

The cancer risks (CR) by oral ingestion, dermal absorption, and inhalation exposure of trihalomethanes (THM) from tap water of ten districts in Fortaleza, Brazil were estimated. The mean levels of THM compounds were obtained in Fortaleza tap water as follow: 63.9 mu g L(-1) for chloroform (CHCl(3)), 40.0 mu g L(-1) for bromodichloromethane (CHBrCl(2)), and 15.6 mu g L(-1) for dibromochloromethane (CHBr(2)Cl). Bromoform (CHBr(3)) was not detected. The mean CR for THMs in tap water is 3.96 x 10(-4). The results indicate that Fortaleza residents have a higher CR by inhalation than dermal absorption and oral ingestion. The CR for CHCl(3) contributes with 68% as compared with the total CR, followed by CHBrCl(2) (21%), and CHBr(2)Cl (11%). The hazard index (HI) is about ten times lower than unity, not indicating non-cancer effects.

Influence of chlorine added to drinking water during the preslaughter feed withdrawal on microbiology and morphology of the broiler gastrointestinal tract

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Hypolipemic and antioxidant activities from Tamarindus indica L. pulp fruit extract in hypercholesterolemic hamsters

Dietary modifications may significantly reduce cardiovascular disease (CVD) risk factors, including cholesterol and atherosclerosis. The present study addressed the effects of the crude extract from the pulp fruit of Tamarindus indica L. on lipid serum levels and early atherosclerotic lesions in hypercholesterolemic hamsters in vivo, and the extract's antioxidant action, in vitro. Animals were fed on either chow or atherogenic diet during 10 weeks and concomitantly received either water or T indica L. extract for drinking. Treatment of hypercholesterolemic hamsters with the T. indica pulp fruit extract (5%) led to a decrease in the levels of serum total cholesterol (50%), non-HDL cholesterol (73%) and triglyceride (60%), and to an increase of high-density lipoprotein (HDL) cholesterol levels (61%). In vitro, the extract presented radical scavenging ability, as assessed by the 2,2-diphenyl-1-picrylhydrazyl (DPPH) and superoxide radicals assays, and led to decreased lipid peroxidation in serum, as assessed by the thiobarbituric acid reactive substances (TBARS) assay. In vivo, the extract improved the efficiency of the antioxidant defense system, as assessed by the superoxide dismutase, catalase and glutathione peroxidase activities. Together these results indicate the potential of tamarind extracts in diminishing the risk of atherosclerosis development in humans. (c) 2005 Elsevier Ltd. All rights reserved.