Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Case based reasoning as an extension of fault dictionary methods for linear electronic analog circuits diagnosis

El test de circuits és una fase del procés de producció que cada vegada pren més importància quan es desenvolupa un nou producte. Les tècniques de test i diagnosi per a circuits digitals han estat desenvolupades i automatitzades amb èxit, mentre que aquest no és encara el cas dels circuits analògics. D'entre tots els mètodes proposats per diagnosticar circuits analògics els més utilitzats són els diccionaris de falles. En aquesta tesi se'n descriuen alguns, tot analitzant-ne els seus avantatges i inconvenients. Durant aquests últims anys, les tècniques d'Intel·ligència Artificial han esdevingut un dels camps de recerca més importants per a la diagnosi de falles. Aquesta tesi desenvolupa dues d'aquestes tècniques per tal de cobrir algunes de les mancances que presenten els diccionaris de falles. La primera proposta es basa en construir un sistema fuzzy com a eina per identificar. Els resultats obtinguts son força bons, ja que s'aconsegueix localitzar la falla en un elevat tant percent dels casos. Per altra banda, el percentatge d'encerts no és prou bo quan a més a més s'intenta esbrinar la desviació. Com que els diccionaris de falles es poden veure com una aproximació simplificada al Raonament Basat en Casos (CBR), la segona proposta fa una extensió dels diccionaris de falles cap a un sistema CBR. El propòsit no és donar una solució general del problema sinó contribuir amb una nova metodologia. Aquesta consisteix en millorar la diagnosis dels diccionaris de falles mitjançant l'addició i l'adaptació dels nous casos per tal d'esdevenir un sistema de Raonament Basat en Casos. Es descriu l'estructura de la base de casos així com les tasques d'extracció, de reutilització, de revisió i de retenció, fent èmfasi al procés d'aprenentatge. En el transcurs del text s'utilitzen diversos circuits per mostrar exemples dels mètodes de test descrits, però en particular el filtre biquadràtic és l'utilitzat per provar les metodologies plantejades, ja que és un dels benchmarks proposats en el context dels circuits analògics. Les falles considerades son paramètriques, permanents, independents i simples, encara que la metodologia pot ser fàcilment extrapolable per a la diagnosi de falles múltiples i catastròfiques. El mètode es centra en el test dels components passius, encara que també es podria extendre per a falles en els actius.

Learning and memory in holocaust survivors with posttraumatic stress disorder

Background: Impairments in explicit memory have been observed in Holocaust survivors with posttraumatic stress disorder. Methods: To evaluate which memory components are preferentially affected, the California Verbal Learning Test was administered to Holocaust survivors with (n = 36) and without (n = 26) posttraumatic stress disorder, and subjects not exposed to the Holocaust (n = 40). Results: Posttraumatic stress disorder subjects showed impairments in learning and short-term and delayed retention compared to nonexposed subjects; survivors without posttraumatic stress disorder did not. Impairments in learning, but not retention, were retained after controlling fir intelligence quotient. Older age was associated with poorer learning and memory performance in the posttraumatic stress disorder group only. Conclusions: The most robust impairment observed in posttraumatic stress disorder was in verbal learning, which may be a risk factor for or consequence of chronic posttraumatic stress disorder. The negative association between performance and age may reflect accelerated cognitive decline in posttraumatic stress disorder.

Do children with Williams syndrome really have good vocabulary knowledge? Methods for comparing cognitive and linguistic abilities in developmental disorders

The comparison of cognitive and linguistic skills in individuals with developmental disorders is fraught with methodological and psychometric difficulties. In this paper, we illustrate some of these issues by comparing the receptive vocabulary knowledge and non-verbal reasoning abilities of 41 children with Williams syndrome, a genetic disorder in which language abilities are often claimed to be relatively strong. Data from this group were compared with data from typically developing children, children with Down syndrome, and children with non-specific learning difficulties using a number of approaches including comparison of age-equivalent scores, matching, analysis of covariance, and regression-based standardization. Across these analyses children with Williams syndrome consistently demonstrated relatively good receptive vocabulary knowledge, although this effect appeared strongest in the oldest children.

An instructional design model for constructivist learning

The emergent requirements for effective e-learning calls for a paradigm shift for instructional design. Constructivist theory and semiotics offer a sound underpinning to enable such revolutionary change by employing the concepts of Learning Objects. E-learning guidelines adopted by the industry have led successfully to the development of training materials. Inadequacy and deficiency of those methods for Higher Education have been identified in this paper. Based on the best practice in industry and our empirical research, we present an instructional design model with practical templates for constructivist learning.

Neurofuzzy mixture of experts network parallel learning and model construction algorithms

A connection between a fuzzy neural network model with the mixture of experts network (MEN) modelling approach is established. Based on this linkage, two new neuro-fuzzy MEN construction algorithms are proposed to overcome the curse of dimensionality that is inherent in the majority of associative memory networks and/or other rule based systems. The first construction algorithm employs a function selection manager module in an MEN system. The second construction algorithm is based on a new parallel learning algorithm in which each model rule is trained independently, for which the parameter convergence property of the new learning method is established. As with the first approach, an expert selection criterion is utilised in this algorithm. These two construction methods are equivalent in their effectiveness in overcoming the curse of dimensionality by reducing the dimensionality of the regression vector, but the latter has the additional computational advantage of parallel processing. The proposed algorithms are analysed for effectiveness followed by numerical examples to illustrate their efficacy for some difficult data based modelling problems.

The automatic layout model in the sequencing of construction activities of multi-objective diaphragm wall unit by using self-learning neural network

Foundation construction process has been an important key point in a successful construction engineering. The frequency of using diaphragm wall construction method among many deep excavation construction methods in Taiwan is the highest in the world. The traditional view of managing diaphragm wall unit in the sequencing of construction activities is to establish each phase of the sequencing of construction activities by heuristics. However, it conflicts final phase of engineering construction with unit construction and effects planning construction time. In order to avoid this kind of situation, we use management of science in the study of diaphragm wall unit construction to formulate multi-objective combinational optimization problem. Because the characteristic (belong to NP-Complete problem) of problem mathematic model is multi-objective and combining explosive, it is advised that using the 2-type Self-Learning Neural Network (SLNN) to solve the N=12, 24, 36 of diaphragm wall unit in the sequencing of construction activities program problem. In order to compare the liability of the results, this study will use random researching method in comparison with the SLNN. It is found that the testing result of SLNN is superior to random researching method in whether solution-quality or Solving-efficiency.

Evoked prior learning experience and approach to learning as predictors of academic achievement

Background. In separate studies and research from different perspectives, five factors are found to be among those related to higher quality outcomes of student learning (academic achievement). Those factors are higher self-efficacy, deeper approaches to learning, higher quality teaching, students’ perceptions that their workload is appropriate, and greater learning motivation. University learning improvement strategies have been built on these research results. Aim. To investigate how students’ evoked prior experience, perceptions of their learning environment, and their approaches to learning collectively contribute to academic achievement. This is the first study to investigate motivation and self-efficacy in the same educational context as conceptions of learning, approaches to learning and perceptions of the learning environment. Sample. Undergraduate students (773) from the full range of disciplines were part of a group of over 2,300 students who volunteered to complete a survey of their learning experience. On completing their degrees 6 and 18 months later, their academic achievement was matched with their learning experience survey data. Method. A 77-item questionnaire was used to gather students’ self-report of their evoked prior experience (self-efficacy, learning motivation, and conceptions of learning), perceptions of learning context (teaching quality and appropriate workload), and approaches to learning (deep and surface). Academic achievement was measured using the English honours degree classification system. Analyses were conducted using correlational and multi-variable (structural equation modelling) methods. Results. The results from the correlation methods confirmed those found in numerous earlier studies. The results from the multi-variable analyses indicated that surface approach to learning was the strongest predictor of academic achievement, with self-efficacy and motivation also found to be directly related. In contrast to the correlation results, a deep approach to learning was not related to academic achievement, and teaching quality and conceptions of learning were only indirectly related to achievement. Conclusions. Research aimed at understanding how students experience their learning environment and how that experience relates to the quality of their learning needs to be conducted using a wider range of variables and more sophisticated analytical methods. In this study of one context, some of the relations found in earlier bivariate studies, and on which learning intervention strategies have been built, are not confirmed when more holistic teaching–learning contexts are analysed using multi-variable methods.

Home, meaning and identity: learning from the English model of shared ownership

This article explores the problematic nature of the label “home ownership” through a case study of the English model of shared ownership, one of the methods used by the UK government to make home ownership affordable. Adopting a legal and socio-legal analysis, the article considers whether shared ownership is capable of fulfilling the aspirations households have for home ownership. To do so, the article considers the financial and nonfinancial meanings attached to home ownership and suggests that the core expectation lies in ownership of the value. The article demonstrates that the rights and responsibilities of shared owners are different in many respects from those of traditional home owners, including their rights as regards ownership of the value. By examining home ownership through the lens of shared ownership the article draws out lessons of broader significance to housing studies. In particular, it is argued that shared ownership shows the limitations of two dichotomies commonly used in housing discourse: that between private and social housing; and the classification of tenure between owner-occupiers and renters. The article concludes that a much more nuanced way of referring to home ownership is required, and that there is a need for a change of expectations amongst consumers as to what sharing ownership means.

Language learning in the public eye: an analysis of newspapers and official documents in England

This article considers the issue of low levels of motivation for foreign language learning in England by exploring how language learning is conceptualised by different key voices in that country through the examination of written data: policy documents and reports on the UK's language needs, curriculum documents, and press articles. The extent to which this conceptualisation has changed over time is explored, through the consideration of documents from two time points, before and after a change in government in the UK. The study uses corpus analysis methods in this exploration. The picture that emerges is a complex one regarding how the 'problems' and 'solutions' surrounding language learning in that context are presented in public discourse. This, we conclude, has implications for the likely success of measures adopted to increase language learning uptake in that context.

Complex extreme learning machine applications in terahertz pulsed signals feature sets

This paper presents a novel approach to the automatic classification of very large data sets composed of terahertz pulse transient signals, highlighting their potential use in biochemical, biomedical, pharmaceutical and security applications. Two different types of THz spectra are considered in the classification process. Firstly a binary classification study of poly-A and poly-C ribonucleic acid samples is performed. This is then contrasted with a difficult multi-class classification problem of spectra from six different powder samples that although have fairly indistinguishable features in the optical spectrum, they also possess a few discernable spectral features in the terahertz part of the spectrum. Classification is performed using a complex-valued extreme learning machine algorithm that takes into account features in both the amplitude as well as the phase of the recorded spectra. Classification speed and accuracy are contrasted with that achieved using a support vector machine classifier. The study systematically compares the classifier performance achieved after adopting different Gaussian kernels when separating amplitude and phase signatures. The two signatures are presented as feature vectors for both training and testing purposes. The study confirms the utility of complex-valued extreme learning machine algorithms for classification of the very large data sets generated with current terahertz imaging spectrometers. The classifier can take into consideration heterogeneous layers within an object as would be required within a tomographic setting and is sufficiently robust to detect patterns hidden inside noisy terahertz data sets. The proposed study opens up the opportunity for the establishment of complex-valued extreme learning machine algorithms as new chemometric tools that will assist the wider proliferation of terahertz sensing technology for chemical sensing, quality control, security screening and clinic diagnosis. Furthermore, the proposed algorithm should also be very useful in other applications requiring the classification of very large datasets.

Performing knowledge requirements analysis for public organisations in a Virtual Learning Environment: a social network analysis approach

This paper describes an application of Social Network Analysis methods for identification of knowledge demands in public organisations. Affiliation networks established in a postgraduate programme were analysed. The course was executed in a distance education mode and its students worked on public agencies. Relations established among course participants were mediated through a virtual learning environment using Moodle. Data available in Moodle may be extracted using knowledge discovery in databases techniques. Potential degrees of closeness existing among different organisations and among researched subjects were assessed. This suggests how organisations could cooperate for knowledge management and also how to identify their common interests. The study points out that closeness among organisations and research topics may be assessed through affiliation networks. This opens up opportunities for applying knowledge management between organisations and creating communities of practice. Concepts of knowledge management and social network analysis provide the theoretical and methodological basis.

Methods for the modulation and analysis of NF-κB-dependent adult neurogenesis

The hippocampus plays a pivotal role in the formation and consolidation of episodic memories, and in spatial orientation. Historically, the adult hippocampus has been viewed as a very static anatomical region of the mammalian brain. However, recent findings have demonstrated that the dentate gyrus of the hippocampus is an area of tremendous plasticity in adults, involving not only modifications of existing neuronal circuits, but also adult neurogenesis. This plasticity is regulated by complex transcriptional networks, in which the transcription factor NF-κB plays a prominent role. To study and manipulate adult neurogenesis, a transgenic mouse model for forebrain-specific neuronal inhibition of NF-κB activity can be used. In this study, methods are described for the analysis of NF-κB-dependent neurogenesis, including its structural aspects, neuronal apoptosis and progenitor proliferation, and cognitive significance, which was specifically assessed via a dentate gyrus (DG)-dependent behavioral test, the spatial pattern separation-Barnes maze (SPS-BM). The SPS-BM protocol could be simply adapted for use with other transgenic animal models designed to assess the influence of particular genes on adult hippocampal neurogenesis. Furthermore, SPS-BM could be used in other experimental settings aimed at investigating and manipulating DG-dependent learning, for example, using pharmacological agents.