Dynamic equilibrium through reinforcement learning

Reinforcement Learning is an area of Machine Learning that deals with how an agent should take actions in an environment such as to maximize the notion of accumulated reward. This type of learning is inspired by the way humans learn and has led to the creation of various algorithms for reinforcement learning. These algorithms focus on the way in which an agent’s behaviour can be improved, assuming independence as to their surroundings. The current work studies the application of reinforcement learning methods to solve the inverted pendulum problem. The importance of the variability of the environment (factors that are external to the agent) on the execution of reinforcement learning agents is studied by using a model that seeks to obtain equilibrium (stability) through dynamism – a Cart-Pole system or inverted pendulum. We sought to improve the behaviour of the autonomous agents by changing the information passed to them, while maintaining the agent’s internal parameters constant (learning rate, discount factors, decay rate, etc.), instead of the classical approach of tuning the agent’s internal parameters. The influence of changes on the state set and the action set on an agent’s capability to solve the Cart-pole problem was studied. We have studied typical behaviour of reinforcement learning agents applied to the classic BOXES model and a new form of characterizing the environment was proposed using the notion of convergence towards a reference value. We demonstrate the gain in performance of this new method applied to a Q-Learning agent.

Magma flow, exsolution processes and rock metasomatism in the Great Messejana-Plasencia dyke (Iberian Peninsula)

Magma flow in dykes is still not well understood; some reported magnetic fabrics are contradictory and the potential effects of exsolution and metasomatism processes on the magnetic properties are issues open to debate. Therefore, a long dyke made of segments with different thickness, which record distinct degrees of metasomatism, the Messejana-Plasencia dyke (MPD), was studied. Oriented dolerite samples were collected along several cross-sections and characterized by means of microscopy and magnetic analyses. The results obtained show that the effects of metasomatism on rock mineralogy are important, and that the metasomatic processes can greatly influence anisotropy degree and mean susceptibility only when rocks are strongly affected by metasomatism. Petrography, scanning electron microscopy (SEM) and bulk magnetic analyses show a high-temperature oxidation-exsolution event, experienced by the very early Ti-spinels, during the early stages of magma cooling, which was mostly observed in central domains of the thick dyke segments. Exsolution reduced the grain size of the magnetic carrier (multidomain to single domain transformation), thus producing composite fabrics involving inverse fabrics. These are likely responsible for a significant number of the 'abnormal' fabrics, which make the interpretation of magma flow much more complex. By choosing to use only the 'normal' fabric for magma flow determination, we have reduced by 50 per cent the number of relevant sites. In these sites, the imbrication angle of the magnetic foliation relative to dyke wall strongly suggests flow with end-members indicating vertical-dominated flow (seven sites) and horizontal-dominated flow (three sites).

Paleomagnetic study of the Great Foum Zguid dyke (southern Morocco): A positive contact test related to metasomatic processes

When a paleomagnetic pole is sought for in an igneous body, the host rocks should be subjected to a contact test to assure that the determined paleopole has the age of the intrusion. If the contact test is positive, it precludes the possibility that the measured magnetization is a later effect. Therefore, we investigated the variations of the remanent magnetization along cross-sections of rocks hosting the Foum Zguid dyke (southern Morocco) and the dyke itself. A positive contact test was obtained, but it is mainly related with Chemical/Crystalline Remanent Magnetization due to metasomatic processes in the host-rocks during magma intrusion and cooling, and not only with Thermo-Remanent Magnetization as ordinarily assumed in standard studies. Paleomagnetic data obtained within the dyke then reflect the Earth magnetic field during emplacement of this well-dated (196.9 +/- 1.8 Ma) intrusion.

Large area image sensing structures based on a-SiC : H: a dynamic characterization

In recent works large area hydrogenated amorphous silicon p-i-n structures with low conductivity doped layers were proposed as single element image sensors. The working principle of this type of sensor is based on the modulation, by the local illumination conditions, of the photocurrent generated by a light beam scanning the active area of the device. In order to evaluate the sensor capabilities is necessary to perform a response time characterization. This work focuses on the transient response of such sensor and on the influence of the carbon contents of the doped layers. In order to evaluate the response time a set of devices with different percentage of carbon incorporation in the doped layers is analyzed by measuring the scanner-induced photocurrent under different bias conditions.

Constructing good deals in discrete time arbitrage-free dynamic pricing models

Recent literature has proved that many classical pricing models (Black and Scholes, Heston, etc.) and risk measures (V aR, CV aR, etc.) may lead to “pathological meaningless situations”, since traders can build sequences of portfolios whose risk leveltends to −infinity and whose expected return tends to +infinity, i.e., (risk = −infinity, return = +infinity). Such a sequence of strategies may be called “good deal”. This paper focuses on the risk measures V aR and CV aR and analyzes this caveat in a discrete time complete pricing model. Under quite general conditions the explicit expression of a good deal is given, and its sensitivity with respect to some possible measurement errors is provided too. We point out that a critical property is the absence of short sales. In such a case we first construct a “shadow riskless asset” (SRA) without short sales and then the good deal is given by borrowing more and more money so as to invest in the SRA. It is also shown that the SRA is interested by itself, even if there are short selling restrictions.

Virtual communities of practice: investigating motivations and constraints in the processes of knowledge creation and transfer

With accelerated market volatility, faster response times and increased globalization, business environments are going through a major transformation and firms have intensified their search for strategies which can give them competitive advantage. This requires that companies continuously innovate, to think of new ideas that can be transformed or implemented as products, processes or services, generating value for the firm. Innovative solutions and processes are usually developed by a group of people, working together. A grouping of people that share and create new knowledge can be considered as a Community of Practice (CoP). CoP’s are places which provide a sound basis for organizational learning and encourage knowledge creation and acquisition. Virtual Communities of Practice (VCoP's) can perform a central role in promoting communication and collaboration between members who are dispersed in both time and space. Nevertheless, it is known that not all CoP's and VCoP's share the same levels of performance or produce the same results. This means that there are factors that enable or constrain the process of knowledge creation. With this in mind, we developed a case study in order to identify both the motivations and the constraints that members of an organization experience when taking part in the knowledge creating processes of VCoP's. Results show that organizational culture and professional and personal development play an important role in these processes. No interviewee referred to direct financial rewards as a motivation factor for participation in VCoPs. Most identified the difficulty in aligning objectives established by the management with justification for the time spent in the VCoP. The interviewees also said that technology is not a constraint.

Knowledge sharing and learning processes: case study of portuguese technological schools and higher education institutions

Paper to be presented at the ESREA Conference Learning to Change? The Role of Identity and Learning Careers in Adult Education, 7-8 December, 2006, Université Catholique Louvain, Louvain–la-Neuve, Belgium

Comparison of deposited surface area of airborne ultrafine particles generated from two welding processes

This article describes work performed on the assessment of the levels of airborne ultrafine particles emitted in two welding processes metal-active gas (MAG) of carbon steel and friction-stir welding (FSW) of aluminium in terms of deposited area in alveolar tract of the lung using a nanoparticle surface area monitor analyser. The obtained results showed the dependence from process parameters on emitted ultrafine particles and clearly demonstrated the presence of ultrafine particles, when compared with background levels. The obtained results showed that the process that results on the lower levels of alveolar-deposited surface area is FSW, unlike MAG. Nevertheless, all the tested processes resulted in important doses of ultrafine particles that are to be deposited in the human lung of exposed workers.

Ankle dynamic in stroke patients: agonist vs. antagonist muscle relations

O documento em anexo encontra-se na versão post-print (versão corrigida pelo editor).

Dynamic artificial neural network for electricity market prices forecast

This paper presents an artificial neural network applied to the forecasting of electricity market prices, with the special feature of being dynamic. The dynamism is verified at two different levels. The first level is characterized as a re-training of the network in every iteration, so that the artificial neural network can able to consider the most recent data at all times, and constantly adapt itself to the most recent happenings. The second level considers the adaptation of the neural network’s execution time depending on the circumstances of its use. The execution time adaptation is performed through the automatic adjustment of the amount of data considered for training the network. This is an advantageous and indispensable feature for this neural network’s integration in ALBidS (Adaptive Learning strategic Bidding System), a multi-agent system that has the purpose of providing decision support to the market negotiating players of MASCEM (Multi-Agent Simulator of Competitive Electricity Markets).

Self-optimization for dynamic scheduling in manufacturing systems

Scheduling is a critical function that is present throughout many industries and applications. A great need exists for developing scheduling approaches that can be applied to a number of different scheduling problems with significant impact on performance of business organizations. A challenge is emerging in the design of scheduling support systems for manufacturing environments where dynamic adaptation and optimization become increasingly important. In this paper, we describe a Self-Optimizing Mechanism for Scheduling System through Nature Inspired Optimization Techniques (NIT).

A hybrid intelligent system for distributed dynamic scheduling

This chapter addresses the resolution of dynamic scheduling by means of meta-heuristic and multi-agent systems. Scheduling is an important aspect of automation in manufacturing systems. Several contributions have been proposed, but the problem is far from being solved satisfactorily, especially if scheduling concerns real world applications. The proposed multi-agent scheduling system assumes the existence of several resource agents (which are decision-making entities based on meta-heuristics) distributed inside the manufacturing system that interact with other agents in order to obtain optimal or near-optimal global performances.

MASDSheGATS – Scheduling System for Dynamic Manufacturing Environments

This chapter addresses the resolution of scheduling in manufacturing systems subject to perturbations. The planning of Manufacturing Systems involves frequently the resolution of a huge amount and variety of combinatorial optimisation problems with an important impact on the performance of manufacturing organisations. Examples of those problems are the sequencing and scheduling problems in manufacturing management, routing and transportation, layout design and timetabling problems.

A study on dynamic state information (DSI) around users for safe urban life

To select each node by devices and by contexts in urban computing, users have to put their plan information and their requests into a computing environment (ex. PDA, Smart Devices, Laptops, etc.) in advance and they will try to keep the optimized states between users and the computing environment. However, because of bad contexts, users may get the wrong decision, so, one of the users’ demands may be requesting the good server which has higher security. To take this issue, we define the structure of Dynamic State Information (DSI) which takes a process about security including the relevant factors in sending/receiving contexts, which select the best during user movement with server quality and security states from DSI. Finally, whenever some information changes, users and devices get the notices including security factors, then an automatic reaction can be possible; therefore all users can safely use all devices in urban computing.

MASDScheGATS: a prototype system for dynamic scheduling

A manufacturing system has a natural dynamic nature observed through several kinds of random occurrences and perturbations on working conditions and requirements over time. For this kind of environment it is important the ability to efficient and effectively adapt, on a continuous basis, existing schedules according to the referred disturbances, keeping performance levels. The application of Meta-Heuristics and Multi-Agent Systems to the resolution of this class of real world scheduling problems seems really promising. This paper presents a prototype for MASDScheGATS (Multi-Agent System for Distributed Manufacturing Scheduling with Genetic Algorithms and Tabu Search).