Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

View-based approaches to spatial representation in human vision

In an immersive virtual environment, observers fail to notice the expansion of a room around them and consequently make gross errors when comparing the size of objects. This result is difficult to explain if the visual system continuously generates a 3-D model of the scene based on known baseline information from interocular separation or proprioception as the observer walks. An alternative is that observers use view-based methods to guide their actions and to represent the spatial layout of the scene. In this case, they may have an expectation of the images they will receive but be insensitive to the rate at which images arrive as they walk. We describe the way in which the eye movement strategy of animals simplifies motion processing if their goal is to move towards a desired image and discuss dorsal and ventral stream processing of moving images in that context. Although many questions about view-based approaches to scene representation remain unanswered, the solutions are likely to be highly relevant to understanding biological 3-D vision.

Contagion in financial networks: a network theory and agent-based approaches to modeling the spread of risk in financial systems

Esta dissertação estuda a propagação de crises sobre o sistema financeiro. Mais especi- ficamente, busca-se desenvolver modelos que permitam simular como um determinado choque econômico atinge determinados agentes do sistema financeiro e apartir dele se propagam, transformando-se em um problema sistêmico. A dissertação é dividida em dois capítulos,além da introdução. O primeiro capítulo desenvolve um modelo de propa- gação de crises em fundos de investimento baseado em ciência das redes.Combinando dois modelos de propagação em redes financeiras, um simulando a propagação de perdas em redes bipartites de ativos e agentes financeiros e o outro simulando a propagação de perdas em uma rede de investimentos diretos em quotas de outros agentes, desenvolve-se um algoritmo para simular a propagação de perdas através de ambos os mecanismos e utiliza-se este algoritmo para simular uma crise no mercado brasileiro de fundos de investimento. No capítulo 2,desenvolve-se um modelo de simulação baseado em agentes, com agentes financeiros, para simular propagação de um choque que afeta o mercado de operações compromissadas.Criamos também um mercado artificial composto por bancos, hedge funds e fundos de curto prazo e simulamos a propagação de um choque de liquidez sobre um ativo de risco securitizando utilizado para colateralizar operações compromissadas dos bancos.

Analysis of fish assemblages in sectors along a salinity gradient based on species, families and functional groups

Neste trabalho foi testado o efeito do gradiente de salinidade do eixo-leste oeste do sistema subtropical Complexo Estuarino da Baía de Paranaguá na estrutura dos peixes de águas rasas, determinado de acordo com as métricas taxonômica (famílias e espécies) e de composição funcional. Um total de 152 espécies foi registrado. As famílias com maior número de espécies foram Sciaenidae, Carangidae, Haemulidae e Gobiidae. As espécies mais abundantes foram A. brasiliensis, H. clupeola, A. januaria e A. tricolor. Os visitantes marinhos dominaram em número de espécies, seguidos pelos migrantes marinhos e estuarinos. A maioria das espécies são zoobentívoras, seguidas pelas piscívoras e zooplanctívoras. As famílias e espécies mais relacionadas com condições estuarinas dominaram no setor mesohalino e aquelas mais relacionadas com condições marinhas dominaram no setor euhalino. A métrica taxonômica foi mais eficiente na caracterização das assembleias de peixes ao longo do gradiente estuarino de salinidade do que a funcional. Isso ocorreu principalmente porque indivíduos de todos os grupos funcionais estiveram presentes ao longo de todos os setores de salinidade, invalidando o emprego dessa métrica na diferenciação das assembleias nos diversos setores. Nosso resultado foi diferente do encontrado em outros estuários tropicais e subtropicais, que enfatizaram a importância dos grupos funcionais na estruturação das assembleias de peixes ao longo de um gradiente de salinidade.

Insights in the maturation of pathogenic bacteria vaccine candidates using mass spectrometry based approaches

The study of the maturation process that occurs to a protein is of pivotal importance for the understanding of its function. This is true also in the vaccine field but in this case is also important to evaluate if inappropriate protein conformation and maturation play roles in the impairment of the functional immunogenicity of protein vaccines. Mass spectrometry (MS) is the method of choice for the study of the maturation process since each modification that occurs during the maturation will lead to a change in the mass of the entire protein. Therefore the aim of my thesis is the development of mass spectrometry-based approaches to study the maturation of proteins and the application of these methods to proteic vaccine candidates. The thesis is divided in two main parts. In the first part, I focused my attention on the study of the maturation of different vaccine candidates using native mass spectrometry. The analyses in this case have been performed using recombinant proteins produced in E. coli. In the second part I applied different MS strategies for the identification of unknown PTMs on pathogenic bacteria surface proteins since modified surface proteins are now considered for vaccine candidate selection.

Analysis and development of ecologically based approaches to coastal defense

Climate-change related impacts, notably coastal erosion, inundation and flooding from sea level rise and storms, will increase in the coming decades enhancing the risks for coastal populations. Further recourse to coastal armoring and other engineered defenses to address risk reduction will exacerbate threats to coastal ecosystems. Alternatively, protection services provided by healthy ecosystems is emerging as a key element in climate adaptation and disaster risk management. I examined two distinct approaches to coastal defense on the base of their ecological and ecosystem conservation values. First, I analyzed the role of coastal ecosystems in providing services for hazard risk reduction. The value in wave attenuation of coral reefs was quantitatively demonstrated using a meta-analysis approach. Results indicate that coral reefs can provide wave attenuation comparable to hard engineering artificial defenses and at lower costs. Conservation and restoration of existing coral reefs are cost-effective management options for disaster risk reduction. Second, I evaluated the possibility to enhance the ecological value of artificial coastal defense structures (CDS) as habitats for marine communities. I documented the suitability of CDS to support native, ecologically relevant, habitat-forming canopy algae exploring the feasibility of enhancing CDS ecological value by promoting the growth of desired species. Juveniles of Cystoseira barbata can be successfully transplanted at both natural and artificial habitats and not affected by lack of surrounding adult algal individuals nor by substratum orientation. Transplantation success was limited by biotic disturbance from macrograzers on CDS compared to natural habitats. Future work should explore the reasons behind the different ecological functioning of artificial and natural habitats unraveling the factors and mechanisms that cause it. The comprehension of the functioning of systems associated with artificial habitats is the key to allow environmental managers to identify proper mitigation options and to forecast the impact of alternative coastal development plans.

[Stem-cell-based approaches for treating inner ear diseases]

The capacity of stem cells to regenerate lost tissue cells has gained recognition among physicians. Despite the successful use of blood stem cells for treating blood cancers, other stem cell types have not yet been widely introduced into clinical practice. Therapy options involving stem cells for inner ear diseases consequently have not been implemented. Nonetheless, several reports have recently been published describing the generation of morphologically and immunologically distinctive inner ear cell types-such as hair cells, supporting cells, and spiral ganglion neurons-from stem cells. Although promising, all of these studies still lack functional results regarding hearing restoration or vestibular function.