Reinforcement Learning solution for Unit Commitment Problem through pursuit method

Unit commitment is an optimization task in electric power generation control sector. It involves scheduling the ON/OFF status of the generating units to meet the load demand with minimum generation cost satisfying the different constraints existing in the system. Numerical solutions developed are limited for small systems and heuristic methodologies find difficulty in handling stochastic cost functions associated with practical systems. This paper models Unit Commitment as a multi stage decision task and Reinforcement Learning solution is formulated through one efficient exploration strategy: Pursuit method. The correctness and efficiency of the developed solutions are verified for standard test systems

Studies on arbuscular mycorrhiza (AM) in the Alentejo (Portugal) using pea mutants resistant to AM fungi as a control tool for field conditions

The utilization and management of arbuscular mycorrhiza (AM) symbiosis may improve production and sustainability of the cropping system. For this purpose, native AM fungi (AMF) were sought and tested for their efficiency to increase plant growth by enhanced P uptake and by alleviation of drought stress. Pot experiments with safflower (Carthamus tinctorius) and pea (Pisum sativum) in five soils (mostly sandy loamy Luvisols) and field experiments with peas were carried out during three years at four different sites. Host plants were grown in heated soils inoculated with AMF or the respective heat sterilized inoculum. In the case of peas, mutants resistant to AMF colonization were used as non-mycorrhizal controls. The mycorrhizal impact on yields and its components, transpiration, and P and N uptake was studied in several experiments, partly under varying P and N levels and water supply. Screening of native AMF by most probable number bioassays was not very meaningful. Soil monoliths were placed in the open to simulate field conditions. Inoculation with a native AMF mix improved grain yield, shoot and leaf growth variables as compared to control. Exposed to drought, higher soil water depletion of mycorrhizal plants resulted in a haying-off effect. The growth response to this inoculum could not be significantly reproduced in a subsequent open air pot experiment at two levels of irrigation and P fertilization, however, safflower grew better at higher P and water supply by multiples. The water use efficiency concerning biomass was improved by the AMF inoculum in the two experiments. Transpiration rates were not significantly affected by AM but as a tendency were higher in non-mycorrhizal safflower. A fundamental methodological problem in mycorrhiza field research is providing an appropriate (negative) control for the experimental factor arbuscular mycorrhiza. Soil sterilization or fungicide treatment have undesirable side effects in field and greenhouse settings. Furthermore, artificial rooting, temperature and light conditions in pot experiments may interfere with the interpretation of mycorrhiza effects. Therefore, the myc- pea mutant P2 was tested as a non-mycorrhizal control in a bioassay to evaluate AMF under field conditions in comparison to the symbiotic isogenetic wild type of var. FRISSON as a new integrative approach. However, mutant P2 is also of nod- phenotype and therefore unable to fix N2. A 3-factorial experiment was carried out in a climate chamber at high NPK fertilization to examine the two isolines under non-symbiotic and symbiotic conditions. P2 achieved the same (or higher) biomass as wild type both under good and poor water supply. However, inoculation with the AMF Glomus manihot did not improve plant growth. Differences of grain and straw yields in field trials were large (up to 80 per cent) between those isogenetic pea lines mainly due to higher P uptake under P and water limited conditions. The lacking N2 fixation in mutants was compensated for by high mineral N supply as indicated by the high N status of the pea mutant plants. This finding was corroborated by the results of a major field experiment at three sites with two levels of N fertilization. The higher N rate did not affect grain or straw yields of the non-fixing mutants. Very efficient AMF were detected in a Ferric Luvisol on pasture land as revealed by yield levels of the evaluation crop and by functional vital staining of highly colonized roots. Generally, levels of grain yield were low, at between 40 and 980 kg ha-1. An additional pot trial was carried out to elucidate the strong mycorrhizal effect in the Ferric Luvisol. A triplication of the plant equivalent field P fertilization was necessary to compensate for the mycorrhizal benefit which was with five times higher grain yield very similar to that found in the field experiment. However, the yield differences between the two isolines were not always plausible as the evaluation variable because they were also found in (small) field test trials with apparently sufficient P and N supply and in a soil of almost no AMF potential. This similarly occurred for pea lines of var. SPARKLE and its non-fixing mycorrhizal (E135) and non-symbiotic (R25) isomutants, which were tested in order to exclude experimentally undesirable benefits by N2 fixation. In contrast to var. FRISSON, SPARKLE was not a suitable variety for Mediterranean field conditions. This raises suspicion putative genetic defects other than symbiotic ones may be effective under field conditions, which would conflict with the concept of an appropriate control. It was concluded that AMF resistant plants may help to overcome fundamental problems of present research on arbuscular mycorrhiza, but may create new ones.

Identification of Problem-based learning components and their effects on graduates' competencies

This study investigated the relationship between higher education and the requirement of the world of work with an emphasis on the effect of problem-based learning (PBL) on graduates' competencies. The implementation of full PBL method is costly (Albanese & Mitchell, 1993; Berkson, 1993; Finucane, Shannon, & McGrath, 2009). However, the implementation of PBL in a less than curriculum-wide mode is more achievable in a broader context (Albanese, 2000). This means higher education institutions implement only a few PBL components in the curriculum. Or a teacher implements a few PBL components at the courses level. For this kind of implementation there is a need to identify PBL components and their effects on particular educational outputs (Hmelo-Silver, 2004; Newman, 2003). So far, however there has been little research about this topic. The main aims of this study were: (1) to identify each of PBL components which were manifested in the development of a valid and reliable PBL implementation questionnaire and (2) to determine the effect of each identified PBL component to specific graduates' competencies. The analysis was based on quantitative data collected in the survey of medicine graduates of Gadjah Mada University, Indonesia. A total of 225 graduates responded to the survey. The result of confirmatory factor analysis (CFA) showed that all individual constructs of PBL and graduates' competencies had acceptable GOFs (Goodness-of-fit). Additionally, the values of the factor loadings (standardize loading estimates), the AVEs (average variance extracted), CRs (construct reliability), and ASVs (average shared squared variance) showed the proof of convergent and discriminant validity. All values indicated valid and reliable measurements. The investigation of the effects of PBL showed that each PBL component had specific effects on graduates' competencies. Interpersonal competencies were affected by Student-centred learning (β = .137; p < .05) and Small group components (β = .078; p < .05). Problem as stimulus affected Leadership (β = .182; p < .01). Real-world problems affected Personal and organisational competencies (β = .140; p < .01) and Interpersonal competencies (β = .114; p < .05). Teacher as facilitator affected Leadership (β = 142; p < .05). Self-directed learning affected Field-related competencies (β = .080; p < .05). These results can help higher education institution and educator to have informed choice about the implementation of PBL components. With this information higher education institutions and educators could fulfil their educational goals and in the same time meet their limited resources. This study seeks to improve prior studies' research method in four major ways: (1) by indentifying PBL components based on theory and empirical data; (2) by using latent variables in the structural equation modelling instead of using a variable as a proxy of a construct; (3) by using CFA to validate the latent structure of the measurement, thus providing better evidence of validity; and (4) by using graduate survey data which is suitable for analysing PBL effects in the frame work of the relationship between higher education and the world of work.

The Human in the Loop: User Participation in Self-Adaptive Software

Self-adaptive software provides a profound solution for adapting applications to changing contexts in dynamic and heterogeneous environments. Having emerged from Autonomic Computing, it incorporates fully autonomous decision making based on predefined structural and behavioural models. The most common approach for architectural runtime adaptation is the MAPE-K adaptation loop implementing an external adaptation manager without manual user control. However, it has turned out that adaptation behaviour lacks acceptance if it does not correspond to a user’s expectations – particularly for Ubiquitous Computing scenarios with user interaction. Adaptations can be irritating and distracting if they are not appropriate for a certain situation. In general, uncertainty during development and at run-time causes problems with users being outside the adaptation loop. In a literature study, we analyse publications about self-adaptive software research. The results show a discrepancy between the motivated application domains, the maturity of examples, and the quality of evaluations on the one hand and the provided solutions on the other hand. Only few publications analysed the impact of their work on the user, but many employ user-oriented examples for motivation and demonstration. To incorporate the user within the adaptation loop and to deal with uncertainty, our proposed solutions enable user participation for interactive selfadaptive software while at the same time maintaining the benefits of intelligent autonomous behaviour. We define three dimensions of user participation, namely temporal, behavioural, and structural user participation. This dissertation contributes solutions for user participation in the temporal and behavioural dimension. The temporal dimension addresses the moment of adaptation which is classically determined by the self-adaptive system. We provide mechanisms allowing users to influence or to define the moment of adaptation. With our solution, users can have full control over the moment of adaptation or the self-adaptive software considers the user’s situation more appropriately. The behavioural dimension addresses the actual adaptation logic and the resulting run-time behaviour. Application behaviour is established during development and does not necessarily match the run-time expectations. Our contributions are three distinct solutions which allow users to make changes to the application’s runtime behaviour: dynamic utility functions, fuzzy-based reasoning, and learning-based reasoning. The foundation of our work is a notification and feedback solution that improves intelligibility and controllability of self-adaptive applications by implementing a bi-directional communication between self-adaptive software and the user. The different mechanisms from the temporal and behavioural participation dimension require the notification and feedback solution to inform users on adaptation actions and to provide a mechanism to influence adaptations. Case studies show the feasibility of the developed solutions. Moreover, an extensive user study with 62 participants was conducted to evaluate the impact of notifications before and after adaptations. Although the study revealed that there is no preference for a particular notification design, participants clearly appreciated intelligibility and controllability over autonomous adaptations.

Analysis and Control of Robot Manipulators with Kinematic Redundancy

A closed-form solution formula for the kinematic control of manipulators with redundancy is derived, using the Lagrangian multiplier method. Differential relationship equivalent to the Resolved Motion Method has been also derived. The proposed method is proved to provide with the exact equilibrium state for the Resolved Motion Method. This exactness in the proposed method fixes the repeatability problem in the Resolved Motion Method, and establishes a fixed transformation from workspace to the joint space. Also the method, owing to the exactness, is demonstrated to give more accurate trajectories than the Resolved Motion Method. In addition, a new performance measure for redundancy control has been developed. This measure, if used with kinematic control methods, helps achieve dexterous movements including singularity avoidance. Compared to other measures such as the manipulability measure and the condition number, this measure tends to give superior performances in terms of preserving the repeatability property and providing with smoother joint velocity trajectories. Using the fixed transformation property, Taylor's Bounded Deviation Paths Algorithm has been extended to the redundant manipulators.

Explorations of the Practical Issues of Learning Prediction-Control Tasks Using Temporal Difference Learning Methods

There has been recent interest in using temporal difference learning methods to attack problems of prediction and control. While these algorithms have been brought to bear on many problems, they remain poorly understood. It is the purpose of this thesis to further explore these algorithms, presenting a framework for viewing them and raising a number of practical issues and exploring those issues in the context of several case studies. This includes applying the TD(lambda) algorithm to: 1) learning to play tic-tac-toe from the outcome of self-play and of play against a perfectly-playing opponent and 2) learning simple one-dimensional segmentation tasks.

Modeling, Estimation, and Control of Robot-Soil Interactions

This thesis presents the development of hardware, theory, and experimental methods to enable a robotic manipulator arm to interact with soils and estimate soil properties from interaction forces. Unlike the majority of robotic systems interacting with soil, our objective is parameter estimation, not excavation. To this end, we design our manipulator with a flat plate for easy modeling of interactions. By using a flat plate, we take advantage of the wealth of research on the similar problem of earth pressure on retaining walls. There are a number of existing earth pressure models. These models typically provide estimates of force which are in uncertain relation to the true force. A recent technique, known as numerical limit analysis, provides upper and lower bounds on the true force. Predictions from the numerical limit analysis technique are shown to be in good agreement with other accepted models. Experimental methods for plate insertion, soil-tool interface friction estimation, and control of applied forces on the soil are presented. In addition, a novel graphical technique for inverting the soil models is developed, which is an improvement over standard nonlinear optimization. This graphical technique utilizes the uncertainties associated with each set of force measurements to obtain all possible parameters which could have produced the measured forces. The system is tested on three cohesionless soils, two in a loose state and one in a loose and dense state. The results are compared with friction angles obtained from direct shear tests. The results highlight a number of key points. Common assumptions are made in soil modeling. Most notably, the Mohr-Coulomb failure law and perfectly plastic behavior. In the direct shear tests, a marked dependence of friction angle on the normal stress at low stresses is found. This has ramifications for any study of friction done at low stresses. In addition, gradual failures are often observed for vertical tools and tools inclined away from the direction of motion. After accounting for the change in friction angle at low stresses, the results show good agreement with the direct shear values.

Virtual Model Control of a Hexapod Walking Robot

Since robots are typically designed with an individual actuator at each joint, the control of these systems is often difficult and non-intuitive. This thesis explains a more intuitive control scheme called Virtual Model Control. This thesis also demonstrates the simplicity and ease of this control method by using it to control a simulated walking hexapod. Virtual Model Control uses imagined mechanical components to create virtual forces, which are applied through the joint torques of real actuators. This method produces a straightforward means of controlling joint torques to produce a desired robot behavior. Due to the intuitive nature of this control scheme, the design of a virtual model controller is similar to the design of a controller with basic mechanical components. The ease of this control scheme facilitates the use of a high level control system which can be used above the low level virtual model controllers to modulate the parameters of the imaginary mechanical components. In order to apply Virtual Model Control to parallel mechanisms, a solution to the force distribution problem is required. This thesis uses an extension of Gardner`s Partitioned Force Control method which allows for the specification of constrained degrees of freedom. This virtual model control technique was applied to a simulated hexapod robot. Although the hexapod is a highly non-linear, parallel mechanism, the virtual models allowed text-book control solutions to be used while the robot was walking. Using a simple linear control law, the robot walked while simultaneously balancing a pendulum and tracking an object.

Thermal control of electronics: Perspectives and prospects

One of the most prominent industrial applications of heat transfer science and engineering has been electronics thermal control. Driven by the relentless increase in spatial density of microelectronic devices, integrated circuit chip powers have risen by a factor of 100 over the past twenty years, with a somewhat smaller increase in heat flux. The traditional approaches using natural convection and forced-air cooling are becoming less viable as power levels increase. This paper provides a high-level overview of the thermal management problem from the perspective of a practitioner, as well as speculation on the prospects for electronics thermal engineering in years to come.

Defects in Self Assembled Colloidal Crystals

Colloidal self assembly is an efficient method for making 3-D ordered nanostructures suitable for materials such as photonic crystals and macroscopic solids for catalysis and sensor applications. Colloidal crystals grown by convective methods exhibit defects on two different scales. Macro defects such as cracks and void bands originate from the dynamics of meniscus motion during colloidal crystal growth while micro defects like vacancies, dislocation and stacking faults are indigenous to the colloidal crystalline structure. This paper analyses the crystallography and energetics of the microscopic defects from the point of view of classical thermodynamics and discusses the strategy for the control of the macroscopic defects through optimization of the liquid-vapor interface.

Using self organizing maps on compositional data

Self-organizing maps (Kohonen 1997) is a type of artificial neural network developed to explore patterns in high-dimensional multivariate data. The conventional version of the algorithm involves the use of Euclidean metric in the process of adaptation of the model vectors, thus rendering in theory a whole methodology incompatible with non-Euclidean geometries. In this contribution we explore the two main aspects of the problem: 1. Whether the conventional approach using Euclidean metric can shed valid results with compositional data. 2. If a modification of the conventional approach replacing vectorial sum and scalar multiplication by the canonical operators in the simplex (i.e. perturbation and powering) can converge to an adequate solution. Preliminary tests showed that both methodologies can be used on compositional data. However, the modified version of the algorithm performs poorer than the conventional version, in particular, when the data is pathological. Moreover, the conventional ap- proach converges faster to a solution, when data is \well-behaved". Key words: Self Organizing Map; Artificial Neural networks; Compositional data

Implementation of training programs in self-regulated learning strategies in Moodle format : Results of a experience in higher education

Resumen tomado de la publicaci??n

Gestión de la actividad no presencial y autoevaluación de competencias = Management of student learning outside the classroom and the self-evaluation of competences

La asignatura troncal “Evaluación Psicológica” de los estudios de Psicología y del estudio de grado “Desarrollo humano en la sociedad de la información” de la Universidad de Girona consta de 12 créditos según la Ley Orgánica de Universidades. Hasta el año académico 2004-05 el trabajo no presencial del alumno consistía en la realización de una evaluación psicológica que se entregaba por escrito a final de curso y de la cual el estudiante obtenía una calificación y revisión si se solicitaba. En el camino hacia el Espacio Europeo de Educación Superior, esta asignatura consta de 9 créditos que equivalen a un total de 255 horas de trabajo presencial y no presencial del estudiante. En los años académicos 2005-06 y 2006-07 se ha creado una guía de trabajo para la gestión de la actividad no presencial con el objetivo de alcanzar aprendizajes a nivel de aplicación y solución de problemas/pensamiento crítico (Bloom, 1975) siguiendo las recomendaciones de la Agencia para la Calidad del Sistema Universitario de Cataluña (2005). La guía incorpora: los objetivos de aprendizaje, los criterios de evaluación, la descripción de las actividades, el cronograma semanal de trabajos para todo el curso, la especificación de las tutorías programadas para la revisión de los diversos pasos del proceso de evaluación psicológica y el uso del foro para el conocimiento, análisis y crítica constructiva de las evaluaciones realizadas por los compañeros

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Experimental Modeling and Control Strategies on an Open Mobile Robot Platform PRIM

This research work deals with the problem of modeling and design of low level speed controller for the mobile robot PRIM. The main objective is to develop an effective educational tool. On one hand, the interests in using the open mobile platform PRIM consist in integrating several highly related subjects to the automatic control theory in an educational context, by embracing the subjects of communications, signal processing, sensor fusion and hardware design, amongst others. On the other hand, the idea is to implement useful navigation strategies such that the robot can be served as a mobile multimedia information point. It is in this context, when navigation strategies are oriented to goal achievement, that a local model predictive control is attained. Hence, such studies are presented as a very interesting control strategy in order to develop the future capabilities of the system