Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Assessment of virtual design and manufacturing techniques for fibre reinforced composite materials

Virtual tools are commonly used nowadays to optimize product design and manufacturing process of fibre reinforced composite materials. The present work focuses on two areas of interest to forecast the part performance and the production process particularities. The first part proposes a multi-physical optimization tool to support the concept stage of a composite part. The strategy is based on the strategic handling of information and, through a single control parameter, is able to evaluate the effects of design variations throughout all these steps in parallel. The second part targets the resin infusion process and the impact of thermal effects. The numerical and experimental approach allowed the identificationof improvement opportunities regarding the implementation of algorithms in commercially available simulation software.

In vitro cumulative gas production techniques: History, methodological considerations and challenges

Methodology used to measure in vitro gas production is reviewed to determine impacts of sources of variation on resultant gas production profiles (GPP). Current methods include measurement of gas production at constant pressure (e.g., use of gas tight syringes), a system that is inexpensive, but may be less sensitive than others thereby affecting its suitability in some situations. Automated systems that measure gas production at constant volume allow pressure to accumulate in the bottle, which is recorded at different times to produce a GPP, and may result in sufficiently high pressure that solubility of evolved gases in the medium is affected, thereby resulting in a recorded volume of gas that is lower than that predicted from stoichiometric calculations. Several other methods measure gas production at constant pressure and volume with either pressure transducers or sensors, and these may be manual, semi-automated or fully automated in operation. In these systems, gas is released as pressure increases, and vented gas is recorded. Agitating the medium does not consistently produce more gas with automated systems, and little or no effect of agitation was observed with manual systems. The apparatus affects GPP, but mathematical manipulation may enable effects of apparatus to be removed. The amount of substrate affects the volume of gas produced, but not rate of gas production, provided there is sufficient buffering capacity in the medium. Systems that use a very small amount of substrate are prone to experimental error in sample weighing. Effect of sample preparation on GPP has been found to be important, but further research is required to determine the optimum preparation that mimics animal chewing. Inoculum is the single largest source of variation in measuring GPP, as rumen fluid is variable and sampling schedules, diets fed to donor animals and ratios of rumen fluid/medium must be selected such that microbial activity is sufficiently high that it does not affect rate and extent of fermentation. Species of donor animal may also cause differences in GPP. End point measures can be mathematically manipulated to account for species differences, but rates of fermentation are not related. Other sources of inocula that have been used include caecal fluid (primarily for investigating hindgut fermentation in monogastrics), effluent from simulated rumen fermentation (e.g., 'Rusitec', which was as variable as rumen fluid), faeces, and frozen or freeze-dried rumen fluid (which were both less active than fresh rumen fluid). Use of mixtures of cell-free enzymes, or pure cultures of bacteria, may be a way of increasing GPP reproducibility, while reducing reliance on surgically modified animals. However, more research is required to develop these inocula. A number of media have been developed which buffer the incubation and provide relevant micro-nutrients to the microorganisms. To date, little research has been completed on relationships between the composition of the medium and measured GPP. However, comparing GPP from media either rich in N or N-free, allows assessment of contributions of N containing compounds in the sample. (c) 2005 Published by Elsevier B.V.

Theoretical and experimental approaches towards the determination of solute effective diffusivities in foods

The aim of this review paper is to present experimental methodologies and the mathematical approaches used to determine effective diffusivities of solutes in food materials. The paper commences by describing the diffusion phenomena related to solute mass transfer in foods and effective diffusivities. It then focuses on the mathematical formulation for the calculation of effective diffusivities considering different diffusion models based on Fick's second law of diffusion. Finally, experimental considerations for effective diffusivity determination are elucidated primarily based on the acquirement of a series of solute content versus time curves appropriate to the equation model chosen. Different factors contributing to the determination of the effective diffusivities such as the structure of food material, temperature, diffusion solvent, agitation, sampling, concentration and different techniques used are considered. (c) 2005 Elsevier Inc. All rights reserved.

Review of current in vivo measurement techniques for quantifying enteric methane emission from ruminants

Ruminant husbandry is a major source of anthropogenic greenhouse gases (GHG). Filling knowledge gaps and providing expert recommendation are important for defining future research priorities, improving methodologies and establishing science-based GHG mitigation solutions to government and non-governmental organisations, advisory/extension networks, and the ruminant livestock sector. The objectives of this review is to summarize published literature to provide a detailed assessment of the methodologies currently in use for measuring enteric methane (CH4) emission from individual animals under specific conditions, and give recommendations regarding their application. The methods described include respiration chambers and enclosures, sulphur hexafluoride tracer (SF6) technique, and techniques based on short-term measurements of gas concentrations in samples of exhaled air. This includes automated head chambers (e.g. the GreenFeed system), the use of carbon dioxide (CO2) as a marker, and (handheld) laser CH4 detection. Each of the techniques are compared and assessed on their capability and limitations, followed by methodology recommendations. It is concluded that there is no ‘one size fits all’ method for measuring CH4 emission by individual animals. Ultimately, the decision as to which method to use should be based on the experimental objectives and resources available. However, the need for high throughput methodology e.g. for screening large numbers of animals for genomic studies, does not justify the use of methods that are inaccurate. All CH4 measurement techniques are subject to experimental variation and random errors. Many sources of variation must be considered when measuring CH4 concentration in exhaled air samples without a quantitative or at least regular collection rate, or use of a marker to indicate (or adjust) for the proportion of exhaled CH4 sampled. Consideration of the number and timing of measurements relative to diurnal patterns of CH4 emission and respiratory exchange are important, as well as consideration of feeding patterns and associated patterns of rumen fermentation rate and other aspects of animal behaviour. Regardless of the method chosen, appropriate calibrations and recovery tests are required for both method establishment and routine operation. Successful and correct use of methods requires careful attention to detail, rigour, and routine self-assessment of the quality of the data they provide.

Genetic Algorithm for the Determination of Linear Viscoelastic Relaxation Spectrum from Experimental Data

Conventional procedures employed in the modeling of viscoelastic properties of polymer rely on the determination of the polymer`s discrete relaxation spectrum from experimentally obtained data. In the past decades, several analytical regression techniques have been proposed to determine an explicit equation which describes the measured spectra. With a diverse approach, the procedure herein introduced constitutes a simulation-based computational optimization technique based on non-deterministic search method arisen from the field of evolutionary computation. Instead of comparing numerical results, this purpose of this paper is to highlight some Subtle differences between both strategies and focus on what properties of the exploited technique emerge as new possibilities for the field, In oder to illustrate this, essayed cases show how the employed technique can outperform conventional approaches in terms of fitting quality. Moreover, in some instances, it produces equivalent results With much fewer fitting parameters, which is convenient for computational simulation applications. I-lie problem formulation and the rationale of the highlighted method are herein discussed and constitute the main intended contribution. (C) 2009 Wiley Periodicals, Inc. J Appl Polym Sci 113: 122-135, 2009

Experimental characterization of nonlinear systems: a real-time evaluation of the analogous Chua`s circuit behavior

This paper presents an experimental characterization of the behavior of an analogous version of the Chua`s circuit. The electronic circuit signals are captured using a data acquisition board (DAQ) and processed using LabVIEW environment. The following aspects of the time series analysis are analyzed: time waveforms, phase portraits, frequency spectra, Poincar, sections, and bifurcation diagram. The circuit behavior is experimentally mapped with the parameter variations, where are identified equilibrium points, periodic and chaotic attractors, and bifurcations. These analysis techniques are performed in real-time and can be applied to characterize, with precision, several nonlinear systems.

Structural effects and hydrogen bonds on N,N `-di(methoxycarbonylsulfenyl)urea, [CH(3)OC(O)SNH](2)CO, studied by experimental and theoretical methods

Pure N,N`-di(methoxycarbonylsulfenyl)urea, [CH(3)OC(O)SNH](2)CO, is quantitatively prepared by the hydrolysis reaction of CH(3)OC(O)SNCO and characterized by (1)H NMR, GC-MS and FTIR spectroscopy techniques. Structural and conformational properties are analyzed using a combined approach with data obtained from X-ray diffraction, vibrational spectra and theoretical calculation methods. The IR and Raman spectra for normal and deuterated species are reported. The crystal structure of [CH(3)OC(O)SNH](2)CO was determined by X-ray diffraction methods. The substance crystallizes in the orthorhombic P2(1)2(1)2 space group with a = 9.524(2), b = 12.003(1), c = 4.481 (1) angstrom, and Z = 2 moieties in the unit cell. The molecule is sited on a twofold crystallographic axis (C(2)) parallel to c and shows the anti-anti conformation (S-N single bonds antiperiplanar with respect to the opposite C-N single bonds in sulfenyl-urea-sic group). Neighboring molecules are arranged in a chain motif that extends along the C(2)-axis and is held by bifurcated NH center dot center dot center dot O center dot center dot center dot HN intermolecular bonds. A local planar symmetry is observed in the crystal for the central -SN(H)C(O)N(H)S- skeleton. Experimental and calculated data allow to trace this structural feature to the occurrence of N-H center dot center dot center dot O=C hydrogen bonding interactions. Calculated vibrational and structural properties are in good agreement with the experimentally determined features. (C) 2008 Elsevier B.V. All rights reserved.

DIAGNOSTIC TECHNIQUES OF LOCAL INFLUENCE IN SPATIAL ANALYSIS OF SOYBEAN YIELD

Modeling of spatial dependence structure, concerning geoestatistics approach, is an indispensable tool for fixing parameters that define this structure, applied on interpolation of values in places that are not sampled, by kriging techniques. However, the estimation of parameters can be greatly affected by the presence of atypical observations on sampled data. Thus, this trial aimed at using diagnostics techniques of local influence in spatial linear Gaussians models, applied at geoestatistics in order to evaluate sensitivity of maximum likelihood estimators and restrict maximum likelihood to small perturbations in these data. So, studies with simulated and experimental data were performed. Those results, obtained from the study of real data, allowed us to conclude that the presence of atypical values among the sampled data can have a strong influence on thematic maps, changing, therefore, the spatial dependence. The application of diagnostics techniques of local influence should be part of any geoestatistic analysis, ensuring that the information from thematic maps has better quality and can be used with greater security by farmers.

Photovoltaic Process Development and innovative Techniques

Photovoltaic processing is one of the processes that have significance in semiconductor process line. It is complicated due to the no. of elements involved that directly or indirectly affect the processing and final yield. So mathematically or empirically we can’t say assertively about the results specially related with diffusion, antireflective coating and impurity poisoning. Here I have experimented and collected data on the mono-crystal silicon wafers with varying properties and outputs. Then by using neural network with available experimental data output required can be estimated which is further tested by the test data for authenticity. One can say that it’s a kind of process simulation with varying input of raw wafers to get desired yield of photovoltaic mono-crystal cells.

Design and validation of a sheet metal shearing experimental procedure

Throughout the industrial processes of sheet metal manufacturing and refining, shear cutting is widely used for its speed and cost advantages over competing cutting methods. Industrial shears may include some force measurement possibilities, but the force is most likely influenced by friction losses between shear tool and the point of measurement, and are in general not showing the actual force applied to the sheet. Well defined shears and accurate measurements of force and shear tool position are important for understanding the influence of shear parameters. Accurate experimental data are also necessary for calibration of numerical shear models. Here, a dedicated laboratory set-up with well defined geometry and movement in the shear, and high measurability in terms of force and geometry is designed, built and verified. Parameters important to the shear process are studied with perturbation analysis techniques and requirements on input parameter accuracy are formulated to meet experimental output demands. Input parameters in shearing are mostly geometric parameters, but also material properties and contact conditions. Based on the accuracy requirements, a symmetric experiment with internal balancing of forces is constructed to avoid guides and corresponding friction losses. Finally, the experimental procedure is validated through shearing of a medium grade steel. With the obtained experimental set-up performance, force changes as result of changes in studied input parameters are distinguishable down to a level of 1%.

O exame direto do lavado broncoalveolar para o diagnóstico precoce de pneumonia nosocomial : um modelo experimental em ratos

A pneumonia nosocomial é a principal causa de infecção nosocomial em unidades de tratamento intensivo e possui alta morbi/mortalidade. A incidência cumulativa varia, conforme os autores, entre limites amplos desde 8% até 51%, dependendo do tipo de paciente e do uso de instrumentação das vias respiratórias. Nos casos específicos das pneumonias de má resolução e da pneumonia associada à ventilação mecânica, o diagnóstico é problemático devido à ausência de uma padronização definitiva, decorrente não só da grande variabilidade fisiopatológica como também da pouca acurácia dos critérios clínicos, microbiológicos e radiológicos. Estes fatos ensejaram a utilização progressiva de técnicas invasivas sofisticadas para coleta de amostras respiratórias endobrônquicas. Entretanto, a validação dessas técnicas para uso sistemático depende ainda de estudos que avaliem não só o seu custo/benefício em cenários clínicos diversos como também a comparação de seu desempenho para o diagnóstico com um padrão-ouro histopatológico. Além disso, o rendimento das técnicas invasivas é freqüentemente limitado quando são aplicadas em pacientes sob antibioticoterapia, que constituem a maioria dos casos em unidades de cuidados intensivos. A otimização desses testes, discutindo suas indicações e avaliando sua capacidade técnica para a confirmação ou exclusão de pneumonia, é justificada pela necessidade da instituição precoce e correta do tratamento, pois disto resulta menor mortalidade, redução de custos e permanência no hospital. Entre os testes que podem auxiliar no diagnóstico encontra-se o exame direto do lavado broncoalveolar, que proporciona resultados precoces e úteis para a tomada de decisão, mas não suficientemente testados em diferentes situações clínicas ou experimentais. Com o objetivo de avaliar o rendimento diagnóstico do exame direto precoce e das culturas quantitativas obtido por lavado broncoalveolar, estudou-se sua utilização em um modelo experimental de pneumonia provocada através de inoculação bacteriana intrabrônquica em ratos. O estudo comparou a acurácia do exame direto e dos exames culturais em três grupos de animais: Grupo A com pneumonia pneumocócica (37 animais); Grupo P com pneumonia por P. aeruginosa (26 animais) e Grupo B controle (10 animais), utilizando a histopatologia dos pulmões como teste de referência para o diagnóstico. Os Grupos A e P foram ainda randomizados em dois subgrupos cada, para tratamento ou não com antibióticos, usando penicilina no grupo pneumococo e amicacina no grupo Pseudomonas. Como resultado, observou-se que nos animais com pneumonia e ausência de antibióticos a pesquisa de bactéria fagocitada (BIC) no exame direto mostrou um rendimento elevado para o diagnóstico, sendo superior ao das culturas quantitativas. No grupo com pneumonia pneumocócica a BIC mostrou: S:94,4% a 100%, E:100%, VPP:100% e VPN:100%; e as culturas quantitativas mostraram: S:77,8%, E:100%, VPP:100%, VPN:40%. Nos com pneumonia por Pseudomonas a BIC obteve: S: 69%, E:100%; VPP:100% e VPN:71,4%); e as culturas quantitativas mostraram valores muito baixos: S:28,6%, E:100%, VPP:100% e VPN:50%). Nos animais com pneumonia pneumocócica sob tratamento antibiótico havia uma queda acentuada de sensibilidade das culturas quantitativas (S:21%) e, em menor grau da BIC (S:57,9%), mas sem perda da especificidade de ambos os exames. Ao contrário, nos animais com pneumonias por Pseudomonas sob tratamento não havia alteração no rendimento dos dois exames, cujos resultados foram semelhantes aos dos animais sem tratamento. Não havia diferenças de leitura da BIC para o diagnóstico, contando a sua positividade em macrófagos ou em neutrófilos infectados. A avaliação global dos casos estudados, reunindo todos os grupos (tratados e não-tratados) em ambos os modelos de pneumonia, mostrou que a acurácia do exame direto, representado pela pesquisa da BIC, foi superior (66%) ao das culturas quantitativas (53%). As conclusões principais do estudo foram: 1) o exame direto do lavado broncoalveolar (LBA) mostrou-se um teste útil e de alto rendimento para o diagnóstico de pneumonia quando realizado na ausência de antibióticos; 2) o LBA feito na vigência de antibióticos efetivos para a pneumonia perde mais de 50% de sua acurácia, mas não é afetado quando o antibiótico for ineficaz ou houver resistência ao mesmo; 3) a pesquisa de BIC no LBA é um exame de resultado precoce, de alta especificidade e com melhor rendimento do que as culturas quantitativas.

Rotinas para a otimizacao experimental de misturas

No cenário atual, onde a globalização, aliada a um maior nível de exigência por parte do cliente, impõem às empresas um maior empenho por competitividade, a agdidade no desenvolvimento e otimização de produtos torna-se crucial para a sobrevivência das mesmas no mercado. Neste contexto, procurou-se compilar várias técnicas utilizadas em E n g e h dd Qd& em um método integrado para a Ot+o Expmmid de MWtwa. Essas técnicas fornecem resultados muito mais rápidos e econômicos do que a tradicional prática de variar um componente de cada vez na mistura, devido ao menor número de ensaios necessários. Entretanto, apesar de não serem tão recentes, as ferramentas aplicáveis à otimização de misturas não têm sido utilizadas pelo seu maior beneficiário (a indústria), provavelmente por falta de divulgação de sua existência, ou, principalmente, devido à complexidade dos cálculos envolvidos. Dessa forma, além do método proposto, desenvolveu-se também um sofiwa~q ue irnplementa todas os passos sugeridos, com o intuito de facilitar ainda mais a aplicação dos mesmos por pessoas não especializadas em técnicas estatísticas. Através do software (OptiMix), o método foi testado em uma situação real e em um estudo comparativo com um relato da literatura, a fim de testar sua validade, necessidade de adaptações e consistência dos resultados. A avaliaçio dos estudos de caso demonstrou que o método proposto fornece resultados coerentes com os de outras técnicas alternativas, com a vantagem de o usuário não precisar realizar cálculos, evitando assim, erros e agilizando o processo de otimização.

Influência da aboragem cirúrgica (laparotomia versus vedeolaparoscopia na gestação : estudo experimental em coelhas

Objetivo: Estudar a influência de duas abordagens cirúrgicas na evolução da gestação de coelhas prenhes. Delineamento: Estudo experimental e controlado. Método: Sessenta coelhas brancas, da raça Nova Zelândia (Oryctolagus cuniculus), prenhes, foram divididas em três grupos de estudo: controle (n=20), laparotomia (n=20) e videolaparoscopia (n=20). Os três grupos, após anestesia intravenosa e intubação orotraqueal, foram submetidos a duas diferentes abordagens cirúrgicas (laparotomia exploradora e videolaparoscopia diagnóstica) e acompanhados até o momento do parto. Foram feitas observações referentes à duração da gestação, da mortalidade fetal e do peso dos láparos vivos. Foram coletadas amostras de sangue arterial, no período pré e pós-operatório, para análise gasométrica e medidas do hematócrito e da hemoglobina das coelhas. Resultados: A duração da gestação (31,6 ± 0,99 vs. 31,8 ± 1,8 vs. 31,3 ± 2,24 dias), a taxa de mortalidade fetal (1,0 ± 2,5 vs. 1,9 ± 2,7 vs. 1,4 ± 2,0) e o peso dos láparos vivos no primeiro dia de vida (48,7 ± 11,3 vs. 51,5 ± 11,9 vs. 48,3 ± 8,2 g) nos grupos C, L e V, respectivamente, não apresentaram diferenças estatísticas significativas entre os grupos de estudo (p>0,05). Nas análises das amostras sangüíneas, quando comparado as diferenças entre o pré e o pós-operatório entre os grupos L e V, respectivamente, foram encontradas diferenças com relevância estatística (p>0,05) em relação às medidas do hematócrito (34,4 ± 3,1 e 33,1 ± 2,8 vs. 34,2 ± 3,2 e 30,3 ± 3,7), do pH (7,4 ± 0,1 e 7,4 ± 0,1 vs. 7,5 ± 0 e 7,3 ± 0,1), do paCO2 ( 30,8 ± 5,1 e 40,7 ± 8,2 vs. 32 ± 3,7 e 53,5 ± 18,4), mas sem relevância clínica. Conclusão: A evolução da gestação das coelhas prenhes submetidas a abordagens videolaparoscópica e laparotômica não mostrou diferença entre os grupos, estando ambas as técnicas indicadas no período gestacional de coelhas quando se fizer necessário.

Low cost BIST techniques for linear and non-linear analog circuits

With the ever increasing demands for high complexity consumer electronic products, market pressures demand faster product development and lower cost. SoCbased design can provide the required design flexibility and speed by allowing the use of IP cores. However, testing costs in the SoC environment can reach a substantial percent of the total production cost. Analog testing costs may dominate the total test cost, as testing of analog circuits usually require functional verification of the circuit and special testing procedures. For RF analog circuits commonly used in wireless applications, testing is further complicated because of the high frequencies involved. In summary, reducing analog test cost is of major importance in the electronic industry today. BIST techniques for analog circuits, though potentially able to solve the analog test cost problem, have some limitations. Some techniques are circuit dependent, requiring reconfiguration of the circuit being tested, and are generally not usable in RF circuits. In the SoC environment, as processing and memory resources are available, they could be used in the test. However, the overhead for adding additional AD and DA converters may be too costly for most systems, and analog routing of signals may not be feasible and may introduce signal distortion. In this work a simple and low cost digitizer is used instead of an ADC in order to enable analog testing strategies to be implemented in a SoC environment. Thanks to the low analog area overhead of the converter, multiple analog test points can be observed and specific analog test strategies can be enabled. As the digitizer is always connected to the analog test point, it is not necessary to include muxes and switches that would degrade the signal path. For RF analog circuits, this is specially useful, as the circuit impedance is fixed and the influence of the digitizer can be accounted for in the design phase. Thanks to the simplicity of the converter, it is able to reach higher frequencies, and enables the implementation of low cost RF test strategies. The digitizer has been applied successfully in the testing of both low frequency and RF analog circuits. Also, as testing is based on frequency-domain characteristics, nonlinear characteristics like intermodulation products can also be evaluated. Specifically, practical results were obtained for prototyped base band filters and a 100MHz mixer. The application of the converter for noise figure evaluation was also addressed, and experimental results for low frequency amplifiers using conventional opamps were obtained. The proposed method is able to enhance the testability of current mixed-signal designs, being suitable for the SoC environment used in many industrial products nowadays.