Hierarchical Reinforcement Learning in Behavior and the Brain

Dissertation presented to obtain the Ph.D degree in Biology, Neuroscience

UNCERTAINTY AND DECISION-MAKING IN THE HUMAN BRAIN

Dissertation presented to obtain the Ph.D degree in Biology, Neuroscience

Thermo-hydro-mechanical analysis of CO2 injection into deep aquifers

One of the biggest challenges for humanity is global warming and consequently, climate changes. Even though there has been increasing public awareness and investments from numerous countries concerning renewable energies, fossil fuels are and will continue to be in the near future, the main source of energy. Carbon capture and storage (CCS) is believed to be a serious measure to mitigate CO2 concentration. CCS briefly consists of capturing CO2 from the atmosphere or stationary emission sources and transporting and storing it via mineral carbonation, in oceans or geological media. The latter is referred to as carbon capture and geological storage (CCGS) and is considered to be the most promising of all solutions. Generally it consists of a storage (e.g. depleted oil reservoirs and deep saline aquifers) and sealing (commonly termed caprock in the oil industry) formations. The present study concerns the injection of CO2 into deep aquifers and regardless injection conditions, temperature gradients between carbon dioxide and the storage formation are likely to occur. Should the CO2 temperature be lower than the storage formation, a contractive behaviour of the reservoir and caprock is expected. The latter can result in the opening of new paths or re-opening of fractures, favouring leakage and compromising the CCGS project. During CO2 injection, coupled thermo-hydro-mechanical phenomena occur, which due to their complexity, hamper the assessment of each relative influence. For this purpose, several analyses were carried out in order to evaluate their influences but focusing on the thermal contractive behaviour. It was finally concluded that depending on mechanical and thermal properties of the pair aquifer-seal, the sealing caprock can undergo significant decreases in effective stress.

Modulation of electrical stimulation applied to human physiology and clinical diagnostic

The use, manipulation and application of electrical currents, as a controlled interference mechanism in the human body system, is currently a strong source of motivation to researchers in areas such as clinical, sports, neuroscience, amongst others. In electrical stimulation (ES), the current applied to tissue is traditionally controlled concerning stimulation amplitude, frequency and pulse-width. The main drawbacks of the transcutaneous ES are the rapid fatigue induction and the high discomfort induced by the non-selective activation of nervous fibers. There are, however, electrophysiological parameters whose response, like the response to different stimulation waveforms, polarity or a personalized charge control, is still unknown. The study of the following questions is of great importance: What is the physiological effect of the electric pulse parametrization concerning charge, waveform and polarity? Does the effect change with the clinical condition of the subjects? The parametrization influence on muscle recruitment can retard fatigue onset? Can parametrization enable fiber selectivity, optimizing the motor fibers recruitment rather than the nervous fibers, reducing contraction discomfort? Current hardware solutions lack flexibility at the level of stimulation control and physiological response assessment. To answer these questions, a miniaturized, portable and wireless controlled device with ES functions and full integration with a generic biosignals acquisition platform has been created. Hardware was also developed to provide complete freedom for controlling the applied current with respect to the waveform, polarity, frequency, amplitude, pulse-width and duration. The impact of the methodologies developed is successfully applied and evaluated in the contexts of fundamental electrophysiology, psycho-motor rehabilitation and neuromuscular disorders diagnosis. This PhD project was carried out in the Physics Department of Faculty of Sciences and Technology (FCT-UNL), in straight collaboration with PLUX - Wireless Biosignals S.A. company and co-funded by the Foundation for Science and Technology.

Knowledge management framework based on brain models and human physiology

The life of humans and most living beings depend on sensation and perception for the best assessment of the surrounding world. Sensorial organs acquire a variety of stimuli that are interpreted and integrated in our brain for immediate use or stored in memory for later recall. Among the reasoning aspects, a person has to decide what to do with available information. Emotions are classifiers of collected information, assigning a personal meaning to objects, events and individuals, making part of our own identity. Emotions play a decisive role in cognitive processes as reasoning, decision and memory by assigning relevance to collected information. The access to pervasive computing devices, empowered by the ability to sense and perceive the world, provides new forms of acquiring and integrating information. But prior to data assessment on its usefulness, systems must capture and ensure that data is properly managed for diverse possible goals. Portable and wearable devices are now able to gather and store information, from the environment and from our body, using cloud based services and Internet connections. Systems limitations in handling sensorial data, compared with our sensorial capabilities constitute an identified problem. Another problem is the lack of interoperability between humans and devices, as they do not properly understand human’s emotional states and human needs. Addressing those problems is a motivation for the present research work. The mission hereby assumed is to include sensorial and physiological data into a Framework that will be able to manage collected data towards human cognitive functions, supported by a new data model. By learning from selected human functional and behavioural models and reasoning over collected data, the Framework aims at providing evaluation on a person’s emotional state, for empowering human centric applications, along with the capability of storing episodic information on a person’s life with physiologic indicators on emotional states to be used by new generation applications.

Study of gold nanoparticle assisted neuron stimulation

The unique proprieties exhibited by nanoscale particles compared to their macro size counterparts allow for the creation of novel neural activity manipula-tion procedures. In this sense, gold nanoparticles (AuNPs) can be used to stimu-late the electrical activity of neuron by converting light into heat. During this dissertation, AuNPs are synthesized by the citrate reduction method, resulting in a hydrodynamic diameter of approximately 16 nm and an absorbance peak of 530 nm. A system to control a 532 nm laser and measure the temperature variation was custom built from scratch specifically for this project. Temperature is then measured with recourse to a thermocouple and through changes in impedance. The built system had in consideration the necessities pre-sented by in vivo tests. Trials were performed by measuring the temperature rise of colloidal AuNP solutions, having the temperature variation reached a maximum of ap-proximately 18 ºC relative to control trials; successfully showing that light is ef-fectively transduced into heat when AuNPs are present. This novel approach enables an alternative to optogenetics, which require the animal to be genetically modified in order to allow neuron stimulation.