Regularized model learning in EDAs for continuous and multi-objective optimization

Probabilistic modeling is the de�ning characteristic of estimation of distribution algorithms (EDAs) which determines their behavior and performance in optimization. Regularization is a well-known statistical technique used for obtaining an improved model by reducing the generalization error of estimation, especially in high-dimensional problems. `1-regularization is a type of this technique with the appealing variable selection property which results in sparse model estimations. In this thesis, we study the use of regularization techniques for model learning in EDAs. Several methods for regularized model estimation in continuous domains based on a Gaussian distribution assumption are presented, and analyzed from di�erent aspects when used for optimization in a high-dimensional setting, where the population size of EDA has a logarithmic scale with respect to the number of variables. The optimization results obtained for a number of continuous problems with an increasing number of variables show that the proposed EDA based on regularized model estimation performs a more robust optimization, and is able to achieve signi�cantly better results for larger dimensions than other Gaussian-based EDAs. We also propose a method for learning a marginally factorized Gaussian Markov random �eld model using regularization techniques and a clustering algorithm. The experimental results show notable optimization performance on continuous additively decomposable problems when using this model estimation method. Our study also covers multi-objective optimization and we propose joint probabilistic modeling of variables and objectives in EDAs based on Bayesian networks, speci�cally models inspired from multi-dimensional Bayesian network classi�ers. It is shown that with this approach to modeling, two new types of relationships are encoded in the estimated models in addition to the variable relationships captured in other EDAs: objectivevariable and objective-objective relationships. An extensive experimental study shows the e�ectiveness of this approach for multi- and many-objective optimization. With the proposed joint variable-objective modeling, in addition to the Pareto set approximation, the algorithm is also able to obtain an estimation of the multi-objective problem structure. Finally, the study of multi-objective optimization based on joint probabilistic modeling is extended to noisy domains, where the noise in objective values is represented by intervals. A new version of the Pareto dominance relation for ordering the solutions in these problems, namely �-degree Pareto dominance, is introduced and its properties are analyzed. We show that the ranking methods based on this dominance relation can result in competitive performance of EDAs with respect to the quality of the approximated Pareto sets. This dominance relation is then used together with a method for joint probabilistic modeling based on `1-regularization for multi-objective feature subset selection in classi�cation, where six di�erent measures of accuracy are considered as objectives with interval values. The individual assessment of the proposed joint probabilistic modeling and solution ranking methods on datasets with small-medium dimensionality, when using two di�erent Bayesian classi�ers, shows that comparable or better Pareto sets of feature subsets are approximated in comparison to standard methods.

An interactive web-based learning assistant for structural analysis of continuous beams

There are significant levels of concern about the relevance and the difficulty of learning some issues on Strength of Materials and Structural Analysis. Most students of Continuum Mechanics and Structural Analysis in Civil Engineering usually point out some key learning aspects as especially difficult for acquiring specific skills. These key concepts entail comprehension difficulties but ease access and applicability to structural analysis in more advanced subjects. Likewise, some elusive but basic structural concepts, such as flexibility, stiffness or influence lines, are paramount for developing further skills required for advanced structural design: tall buildings, arch-type structures as well as bridges. As new curricular itineraries are currently being implemented, it appears appropriate to devise a repository of interactive web-based applications for training in those basic concepts. That will hopefully train the student to understand the complexity of such concepts, to develop intuitive knowledge on actual structural response and to improve their preparation for exams. In this work, a web-based learning assistant system for influence lines on continuous beams is presented. It consists of a collection of interactive user-friendly applications accessible via Web. It is performed in both Spanish and English languages. Rather than a “black box” system, the procedure involves open interaction with the student, who can simulate and virtually envisage the structural response. Thus, the student is enabled to set the geometric, topologic and mechanic layout of a continuous beam and to change or shift the loading and the support conditions. Simultaneously, the changes in the beam response prompt on the screen, so that the effects of the several issues involved in structural analysis become apparent. The system is performed through a set of web pages which encompasses interactive exercises and problems, written in JavaScript under JQuery and DyGraphs frameworks, given that their efficiency and graphic capabilities are renowned. Students can freely boost their self-study on this subject in order to face their exams more confidently. Besides, this collection is expected to be added to the "Virtual Lab of Continuum Mechanics" of the UPM, launched in 2013 (http://serviciosgate.upm.es/laboratoriosvirtuales/laboratorios/medios-continuos-en-construcci%C3%B3n)

Playing in continuous spaces: Some analysis and extension of population-based incremental learning

As an alternative to traditional evolutionary algorithms (EAs), population-based incremental learning (PBIL) maintains a probabilistic model of the best individual(s). Originally, PBIL was applied in binary search spaces. Recently, some work has been done to extend it to continuous spaces. In this paper, we review two such extensions of PBIL. An improved version of the PBIL based on Gaussian model is proposed that combines two main features: a new updating rule that takes into account all the individuals and their fitness values and a self-adaptive learning rate parameter. Furthermore, a new continuous PBIL employing a histogram probabilistic model is proposed. Some experiments results are presented that highlight the features of the new algorithms.

Bayesian online algorithms for learning in discrete Hidden Markov Models

We propose and analyze two different Bayesian online algorithms for learning in discrete Hidden Markov Models and compare their performance with the already known Baldi-Chauvin Algorithm. Using the Kullback-Leibler divergence as a measure of generalization we draw learning curves in simplified situations for these algorithms and compare their performances.

Building a knowledge and learning society in Portugal: adult students in technological schools and higher education institutions

Lifelong learning (LLL) has received increasing attention in recent years. It implies that learning should take place at all stages of the “life cycle and it should be life-wide, that is embedded in all life contexts from the school to the work place, the home and the community” (Green, 2002, p.613). The ‘learning society’, is the vision of a society where there are recognized opportunities for learning for every person, wherever they are and however old they happen to be. Globalization and the rise of new information technologies are some of the driving forces that cause depreciation of specialised competences. This happens very quickly in terms of economic value; consequently, workers of all skills levels, during their working life, must have the opportunity to update “their technical skills and enhance general skills to keep pace with continuous technological change and new job requirements” (Fahr, 2005, p. 75). It is in this context that LLL tops the policy agenda of international bodies, national governments and non-governmental organizations, in the field of education and training, to justify the need for LLL opportunities for the population as they face contemporary employability challenges. It is in this context that the requirement and interest to analyse the behaviour patterns of adult learners has developed over the last few years

Vantagens e desvantagens no desenvolvimento de recursos educativos para dispositivos móveis. CV Learning Mobile - Software educativo sobre “Organização Administrativa de Cabo Verde”

Não é recente a contribuição das tecnologias de informação e comunicação em processos de ensino/aprendizagem, no sentido da proliferação de conhecimento, de forma fácil e rápida. Com a contínua evolução tecnológica, surgem novos conceitos relativamente a processos de ensino/aprendizagem assentes nessas tecnologias. A aprendizagem por meio de dispositivos móveis, o m-Learning, é um exemplo, sendo um campo de investigação educacional em franca evolução, que explora essencialmente a mobilidade e a interactividade. No âmbito desta dissertação, pretende-se analisar a tecnologia m-Learning, fazendo referência as principais vantagens e desvantagens desta tecnologia. Neste sentido, e por pretendermos dar o nosso contributo ao ensino cabo-verdiano, onde a utilização de tal tecnologia é ainda inexistente, desenvolveu-se a aplicação CV Learning Mobile, um software educativo sobre a “Organização Administrativa de Cabo Verde”, como resultado do estudo efectuado.

Implementação de cursos em regime de e-learning em ambiente hospitalar. A MEDUCA como suporte

A sociedade moderna encontra-se numa evolução progressiva e constante no que respeita às novas tecnologias. Independentemente da área de conhecimento, é de senso comum, que cada vez mais é necessária formação sólida, sendo fundamental a preparação e a consolidação das futuras gerações na utilização das novas tecnologias. As plataformas de e-learning são hoje em dia uma realidade mais que afirmada, e com aplicação em todos os sectores de actividade. A área da saúde não foge à regra, verificando-se que os seus profissionais evidenciam falta de disponibilidade para participação nas formações presenciais, fundamentais para o seu processo de formação contínua (LLL – Long Liffe Learning). Estes profissionais necessitam de estar continuamente actualizados, de forma a melhor poderem contribuir para o desempenho das suas funções, como o aconselhamento dos utentes, acompanhamento de doentes crónicos, uso correcto dos medicamentos, entre outros. O presente trabalho pretende implementar em ambiente hospitalar o modelo de formação à distância em regime de e-learning ou b-learning, identificando potenciais vantagens e constrangimentos inerentes ao processo. Para o efeito, será utilizada a plataforma MEDUCA criada pelo GILT-ISEP e baseada em Moodle, como uma plataforma destinada á formação para profissionais da área da Saúde. Esta dissertação apresenta uma investigação sobre a implementação da dita plataforma em ambiente hospitalar. Esta dissertação apresenta todo o estudo/trabalho desenvolvido para a implementação da plataforma MEDUCA nalgumas entidades contactadas da área da saúde. Espera-se que esta ferramenta ofereça uma alternativa interactiva de educação e aprendizagem, visando melhorar constantemente o nível formativo de cada profissional de saúde.

Lean learning academy: an innovative framework for lean manufacturing training

Worldwide competitiveness poses enormous challenges on managers, demanding a continuous quest to increase rationality in the use of resources. As a management philosophy, Lean Manufacturing focuses on the elimination of activities that do not create any type of value and therefore are considered waste. For companies to successfully implement the Lean Manufacturing philosophy it is crucial that the human resources of the organization have the necessary training, for which proper tools are required. At the same time, higher education institutions need innovative tools to increase the attractiveness of engineering curricula and develop a higher level of knowledge among students, improving their employability. This paper describes how Lean Learning Academy, an international collaboration project between five EU universities and five companies, from SME to Multinational/Global companies, developed and applied an innovative training programme for Engineers on Lean Manufacturing, a successful alternative to the traditional teaching methods in engineering courses.

Innovations in technology: a friendly math project and the learning analytics “challenge”

Currently the world around us "reboots" every minute and “staying at the forefront” seems to be a very arduous task. The continuous and “speeded” progress of society requires, from all the actors, a dynamic and efficient attitude both in terms progress monitoring and moving adaptation. With regard to education, no matter how updated we are in relation to the contents, the didactic strategies and technological resources, we are inevitably compelled to adapt to new paradigms and rethink the traditional teaching methods. It is in this context that the contribution of e-learning platforms arises. Here teachers and students have at their disposal new ways to enhance the teaching and learning process, and these platforms are seen, at the present time, as significant virtual teaching and learning supporting environments. This paper presents a Project and attempts to illustrate the potential that new technologies present as a “backing” tool in different stages of teaching and learning at different levels and areas of knowledge, particularly in Mathematics. We intend to promote a constructive discussion moment, exposing our actual perception - that the use of the Learning Management System Moodle, by Higher Education teachers, as supplementary teaching-learning environment for virtual classroom sessions can contribute for greater efficiency and effectiveness of teaching practice and to improve student achievement. Regarding the Learning analytics experience we will present a few results obtained with some assessment Learning Analytics tools, where we profoundly felt that the assessment of students’ performance in online learning environments is a challenging and demanding task.

Long-term outcome after cognitive and behavioral regression in nonlesional epilepsy with continuous spike-waves during slow-wave sleep.

PURPOSE: To present the long-term follow-up of 10 adolescents and young adults with documented cognitive and behavioral regression as children due to nonlesional focal, mainly frontal, epilepsy with continuous spike-waves during slow wave sleep (CSWS). METHODS: Past medical and electroencephalography (EEG) data were reviewed and neuropsychological tests exploring main cognitive functions were administered. KEY FINDINGS: After a mean duration of follow-up of 15.6 years (range, 8-23 years), none of the 10 patients had recovered fully, but four regained borderline to normal intelligence and were almost independent. Patients with prolonged global intellectual regression had the worst outcome, whereas those with more specific and short-lived deficits recovered best. The marked behavioral disorders resolved in all but one patient. Executive functions were neither severely nor homogenously affected. Three patients with a frontal syndrome during the active phase (AP) disclosed only mild residual executive and social cognition deficits. The main cognitive gains occurred shortly after the AP, but qualitative improvements continued to occur. Long-term outcome correlated best with duration of CSWS. SIGNIFICANCE: Our findings emphasize that cognitive recovery after cessation of CSWS depends on the severity and duration of the initial regression. None of our patients had major executive and social cognition deficits with preserved intelligence, as reported in adults with early destructive lesions of the frontal lobes. Early recognition of epilepsy with CSWS and rapid introduction of effective therapy are crucial for a best possible outcome.

Hybrid coordination of reinforcement learning-based behaviors for AUV control

This paper proposes a hybrid coordination method for behavior-based control architectures. The hybrid method takes advantages of the robustness and modularity in competitive approaches as well as optimized trajectories in cooperative ones. This paper shows the feasibility of applying this hybrid method with a 3D-navigation to an autonomous underwater vehicle (AUV). The behaviors are learnt online by means of reinforcement learning. A continuous Q-learning implemented with a feed-forward neural network is employed. Realistic simulations were carried out. The results obtained show the good performance of the hybrid method on behavior coordination as well as the convergence of the behaviors

Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task