SINGULARLY PERTURBED DISCOUNTED MARKOV CONTROL PROCESSES IN A GENERAL STATE SPACE

This paper studies the asymptotic optimality of discrete-time Markov decision processes (MDPs) with general state space and action space and having weak and strong interactions. By using a similar approach as developed by Liu, Zhang, and Yin [Appl. Math. Optim., 44 (2001), pp. 105-129], the idea in this paper is to consider an MDP with general state and action spaces and to reduce the dimension of the state space by considering an averaged model. This formulation is often described by introducing a small parameter epsilon > 0 in the definition of the transition kernel, leading to a singularly perturbed Markov model with two time scales. Our objective is twofold. First it is shown that the value function of the control problem for the perturbed system converges to the value function of a limit averaged control problem as epsilon goes to zero. In the second part of the paper, it is proved that a feedback control policy for the original control problem defined by using an optimal feedback policy for the limit problem is asymptotically optimal. Our work extends existing results of the literature in the following two directions: the underlying MDP is defined on general state and action spaces and we do not impose strong conditions on the recurrence structure of the MDP such as Doeblin's condition.

Characterization of Stability Region for General Autonomous Nonlinear Dynamical Systems

The existing characterization of stability regions was developed under the assumption that limit sets on the stability boundary are exclusively composed of hyperbolic equilibrium points and closed orbits. The characterizations derived in this technical note are a generalization of existing results in the theory of stability regions. A characterization of the stability boundary of general autonomous nonlinear dynamical systems is developed under the assumption that limit sets on the stability boundary are composed of a countable number of disjoint and indecomposable components, which can be equilibrium points, closed orbits, quasi-periodic solutions and even chaotic invariant sets.

Diseño e implementación de un prototipo de enchufe inteligente mediante la placa Arduino

[ES] Este trabajo detalla las diferentes etapas del desarrollo de construcción del prototipo Clever Socket, abarcando desde su análisis, a su diseño e implementación.
Clever Socket es un conjunto hardware y software que facilita el control del encendido y el apagado de los aparatos electrónicos conectados a él a través de redes de comunicación LAN y WAN. Para lograr este propósito, el artefacto se apoya en la placa de desarrollo electrónico Arduino UNO y la placa Arduino WiFi Shield –que posibilita su conexión inalámbrica a la red–.
El dispositivo cuenta con cuatro bases de enchufe controladas a través de relés, que permiten la gestión de hasta cuatro dispositivos electrónicos de manera simultánea.
 En el lado del software se presentan tres componentes:
Un servicio web de bajo nivel destinado a la gestión de la placa Arduino. Permite la comunicación con la placa, la lectura y escritura de sus pines.
 Un servicio web específico del dispositivo Clever Socket. Facilita la comunicación con el prototipo hardware, permitiendo así la gestión de diferentes operaciones que se llevan a cabo sobre los aparatos conectados a sus enchufes.
 Una aplicación web. Actúa como interfaz entre el usuario y el servicio web de Clever Socket, permitiendo la gestión del dispositivo. El diseño de la aplicación es responsive, lo que posibilita su correcta visualización en todo tipo de dispositivos, tanto móviles como de escritorio.

Structure learning of graphical models for task-oriented robot grasping

In the collective imaginaries a robot is a human like machine as any androids in science fiction. However the type of robots that you will encounter most frequently are machinery that do work that is too dangerous, boring or onerous. Most of the robots in the world are of this type. They can be found in auto, medical, manufacturing and space industries. Therefore a robot is a system that contains sensors, control systems, manipulators, power supplies and software all working together to perform a task. The development and use of such a system is an active area of research and one of the main problems is the development of interaction skills with the surrounding environment, which include the ability to grasp objects. To perform this task the robot needs to sense the environment and acquire the object informations, physical attributes that may influence a grasp. Humans can solve this grasping problem easily due to their past experiences, that is why many researchers are approaching it from a machine learning perspective finding grasp of an object using information of already known objects. But humans can select the best grasp amongst a vast repertoire not only considering the physical attributes of the object to grasp but even to obtain a certain effect. This is why in our case the study in the area of robot manipulation is focused on grasping and integrating symbolic tasks with data gained through sensors. The learning model is based on Bayesian Network to encode the statistical dependencies between the data collected by the sensors and the symbolic task. This data representation has several advantages. It allows to take into account the uncertainty of the real world, allowing to deal with sensor noise, encodes notion of causality and provides an unified network for learning. Since the network is actually implemented and based on the human expert knowledge, it is very interesting to implement an automated method to learn the structure as in the future more tasks and object features can be introduced and a complex network design based only on human expert knowledge can become unreliable. Since structure learning algorithms presents some weaknesses, the goal of this thesis is to analyze real data used in the network modeled by the human expert, implement a feasible structure learning approach and compare the results with the network designed by the expert in order to possibly enhance it.

Semantic coordination through programmable Tuple spaces

Two of the main features of today complex software systems like pervasive computing systems and Internet-based applications are distribution and openness. Distribution revolves around three orthogonal dimensions: (i) distribution of control|systems are characterised by several independent computational entities and devices, each representing an autonomous and proactive locus of control; (ii) spatial distribution|entities and devices are physically distributed and connected in a global (such as the Internet) or local network; and (iii) temporal distribution|interacting system components come and go over time, and are not required to be available for interaction at the same time. Openness deals with the heterogeneity and dynamism of system components: complex computational systems are open to the integration of diverse components, heterogeneous in terms of architecture and technology, and are dynamic since they allow components to be updated, added, or removed while the system is running. The engineering of open and distributed computational systems mandates for the adoption of a software infrastructure whose underlying model and technology could provide the required level of uncoupling among system components. This is the main motivation behind current research trends in the area of coordination middleware to exploit tuple-based coordination models in the engineering of complex software systems, since they intrinsically provide coordinated components with communication uncoupling and further details in the references therein. An additional daunting challenge for tuple-based models comes from knowledge-intensive application scenarios, namely, scenarios where most of the activities are based on knowledge in some form|and where knowledge becomes the prominent means by which systems get coordinated. Handling knowledge in tuple-based systems induces problems in terms of syntax - e.g., two tuples containing the same data may not match due to differences in the tuple structure - and (mostly) of semantics|e.g., two tuples representing the same information may not match based on a dierent syntax adopted. Till now, the problem has been faced by exploiting tuple-based coordination within a middleware for knowledge intensive environments: e.g., experiments with tuple-based coordination within a Semantic Web middleware (surveys analogous approaches). However, they appear to be designed to tackle the design of coordination for specic application contexts like Semantic Web and Semantic Web Services, and they result in a rather involved extension of the tuple space model. The main goal of this thesis was to conceive a more general approach to semantic coordination. In particular, it was developed the model and technology of semantic tuple centres. It is adopted the tuple centre model as main coordination abstraction to manage system interactions. A tuple centre can be seen as a programmable tuple space, i.e. an extension of a Linda tuple space, where the behaviour of the tuple space can be programmed so as to react to interaction events. By encapsulating coordination laws within coordination media, tuple centres promote coordination uncoupling among coordinated components. Then, the tuple centre model was semantically enriched: a main design choice in this work was to try not to completely redesign the existing syntactic tuple space model, but rather provide a smooth extension that { although supporting semantic reasoning { keep the simplicity of tuple and tuple matching as easier as possible. By encapsulating the semantic representation of the domain of discourse within coordination media, semantic tuple centres promote semantic uncoupling among coordinated components. The main contributions of the thesis are: (i) the design of the semantic tuple centre model; (ii) the implementation and evaluation of the model based on an existent coordination infrastructure; (iii) a view of the application scenarios in which semantic tuple centres seem to be suitable as coordination media.

Sviluppo di metodologie per la Stima in Tempo Reale delle Grandezze Indicate in Motori a Combustione Interna

Modern Internal Combustion Engines are becoming increasingly complex in terms of their control systems and strategies. The growth of the algorithms’ complexity results in a rise of the number of on-board quantities for control purposes. In order to improve combustion efficiency and, simultaneously, limit the amount of pollutant emissions, the on-board evaluation of two quantities in particular has become essential; namely indicated torque produced by the engine and the angular position where 50% of fuel mass injected over an engine cycle is burned (MFB50). The above mentioned quantities can be evaluated through the measurement of in-cylinder pressure. Nonetheless, at the time being, the installation of in-cylinder pressure sensors on vehicles is extremely uncommon mainly because of measurement reliability and costs. This work illustrates a methodological approach for the estimation of indicated torque and MFB50 that is based on the engine speed fluctuation measurement. This methodology is compatible with the typical on-board application restraints. Moreover, it requires no additional costs since speed can be measured using the system already mounted on the vehicle, which is made of a magnetic pick-up faced to a toothed wheel. The estimation algorithm consists of two main parts: first, the evaluation of indicated torque fluctuation based on speed measurement and secondly, the evaluation of the mean value of the indicated torque (over an engine cycle) and MFB50 by using the relationship with the indicated torque harmonic and other engine quantities. The procedure has been successfully applied to an L4 turbocharged Diesel engine mounted on-board a vehicle.

Nonlinear Control of Magnetically Actuated Spacecraft

The topic of this thesis is the feedback stabilization of the attitude of magnetically actuated spacecraft. The use of magnetic coils is an attractive solution for the generation of control torques on small satellites flying inclined low Earth orbits, since magnetic control systems are characterized by reduced weight and cost, higher reliability, and require less power with respect to other kinds of actuators. At the same time, the possibility of smooth modulation of control torques reduces coupling of the attitude control system with flexible modes, thus preserving pointing precision with respect to the case when pulse-modulated thrusters are used. The principle based on the interaction between the Earth's magnetic field and the magnetic field generated by the set of coils introduces an inherent nonlinearity, because control torques can be delivered only in a plane that is orthogonal to the direction of the geomagnetic field vector. In other words, the system is underactuated, because the rotational degrees of freedom of the spacecraft, modeled as a rigid body, exceed the number of independent control actions. The solution of the control issue for underactuated spacecraft is also interesting in the case of actuator failure, e.g. after the loss of a reaction-wheel in a three-axes stabilized spacecraft with no redundancy. The application of well known control strategies is no longer possible in this case for both regulation and tracking, so that new methods have been suggested for tackling this particular problem. The main contribution of this thesis is to propose continuous time-varying controllers that globally stabilize the attitude of a spacecraft, when magneto-torquers alone are used and when a momentum-wheel supports magnetic control in order to overcome the inherent underactuation. A kinematic maneuver planning scheme, stability analyses, and detailed simulation results are also provided, with new theoretical developments and particular attention toward application considerations.

Characterization of optical transduction-based molecular systems and nanoparticles for the development of chemical sensors

With the increasing importance that nanotechnologies have in everyday life, it is not difficult to realize that also a single molecule, if properly designed, can be a device able to perform useful functions: such a chemical species is called chemosensor, that is a molecule of abiotic origin that signals the presence of matter or energy. Signal transduction is the mechanism by which an interaction of a sensor with an analyte yields a measurable form of energy. When dealing with the design of a chemosensor, we need to take into account a “communication requirement” between its three component: the receptor unit, responsible for the selective analyte binding, the spacer, which controls the geometry of the system and modulates the electronic interaction between the receptor and the signalling unit, whose physico-chemical properties change upon complexation. A luminescent chemosensor communicates a variation of the physico-chemical properties of the receptor unit with a luminescence output signal. This thesis work consists in the characterization of new molecular and nanoparticle-based system which can be used as sensitive materials for the construction of new optical transduction devices able to provide information about the concentration of analytes in solution. In particular two direction were taken. The first is to continue in the development of new chemosensors, that is the first step for the construction of reliable and efficient devices, and in particular the work will be focused on chemosensors for metal ions for biomedical and environmental applications. The second is to study more efficient and complex organized systems, such as derivatized silica nanoparticles. These system can potentially have higher sensitivity than molecular systems, and present many advantages, like the possibility to be ratiometric, higher Stokes shifts and lower signal-to-noise ratio.

Type Systems for Distributed Programs: Components and Sessions

Modern software systems, in particular distributed ones, are everywhere around us and are at the basis of our everyday activities. Hence, guaranteeing their cor- rectness, consistency and safety is of paramount importance. Their complexity makes the verification of such properties a very challenging task. It is natural to expect that these systems are reliable and above all usable. i) In order to be reliable, compositional models of software systems need to account for consistent dynamic reconfiguration, i.e., changing at runtime the communication patterns of a program. ii) In order to be useful, compositional models of software systems need to account for interaction, which can be seen as communication patterns among components which collaborate together to achieve a common task. The aim of the Ph.D. was to develop powerful techniques based on formal methods for the verification of correctness, consistency and safety properties related to dynamic reconfiguration and communication in complex distributed systems. In particular, static analysis techniques based on types and type systems appeared to be an adequate methodology, considering their success in guaranteeing not only basic safety properties, but also more sophisticated ones like, deadlock or livelock freedom in a concurrent setting. The main contributions of this dissertation are twofold. i) On the components side: we design types and a type system for a concurrent object-oriented calculus to statically ensure consistency of dynamic reconfigurations related to modifications of communication patterns in a program during execution time. ii) On the communication side: we study advanced safety properties related to communication in complex distributed systems like deadlock-freedom, livelock- freedom and progress. Most importantly, we exploit an encoding of types and terms of a typical distributed language, session π-calculus, into the standard typed π- calculus, in order to understand their expressive power.

EPR spectroscopy as a tool for the elucidation of non-covalent structuring principles in complex soft matter

This thesis aims at connecting structural and functional changes of complex soft matter systems due to external stimuli with non-covalent molecular interaction profiles. It addresses the problem of elucidating non-covalent forces as structuring principle of mainly polymer-based systems in solution. The structuring principles of a wide variety of complex soft matter types are analyzed. In many cases this is done by exploring conformational changes upon the exertion of external stimuli. The central question throughout this thesis is how a certain non-covalent interaction profile leads to solution condition-dependent structuring of a polymeric system.rnTo answer this question, electron paramagnetic resonance (EPR) spectroscopy is chosen as the main experimental method for the investigation of the structure principles of polymers. With EPR one detects only the local surroundings or environments of molecules that carry an unpaired electron. Non-covalent forces are normally effective on length scales of a few nanometers and below. Thus, EPR is excellently suited for their investigations. It allows for detection of interactions on length scales ranging from approx. 0.1 nm up to 10 nm. However, restriction to only one experimental technique likely leads to only incomplete pictures of complex systems. Therefore, the presented studies are frequently augmented with further experimental and computational methods in order to yield more comprehensive descriptions of the systems chosen for investigation.rnElectrostatic correlation effects in non-covalent interaction profiles as structuring principles in colloid-like ionic clusters and DNA condensation are investigated first. Building on this it is shown how electrostatic structuring principles can be combined with hydrophobic ones, at the example of host-guest interactions in so-called dendronized polymers (denpols).rnSubsequently, the focus is shifted from electrostatics in dendronized polymers to thermoresponsive alkylene oxide-based materials, whose structuring principles are based on hydrogen bonds and counteracting hydrophobic interactions. The collapse mechanism in dependence of hydrophilic-hydrophobic balance and topology of these polymers is elucidated. Complementarily the temperature-dependent phase behavior of elastin-like polypeptides (ELPs) is investigated. ELPs are the first (and so far only) class of compounds that is shown to feature a first-order inverse phase transition on nanoscopic length scales.rnFinally, this thesis addresses complex biological systems, namely intrinsically disordered proteins (IDPs). It is shown that the conformational space of the IDPs Osteopontin (OPN), a cytokine involved in metastasis of several kinds of cancer, and BASP1 (brain acid soluble protein one), a protein associated with neurite outgrowth, is governed by a subtle interplay between electrostatic forces, hydrophobic interaction, system entropy and hydrogen bonds. Such, IDPs can even sample cooperatively folded structures, which have so far only been associated with globular proteins.

Systematic functionalization of molecules for molecular self-assembly

This thesis describes the investigation of systematically varied organic molecules for use in molecular self-assembly processes. All experiments were performed using high-resolution non-contact atomic force microscopy under UHV conditions and at room temperature. Using this technique, three different approaches for influencing intermolecular and molecule-surface interaction on the insulating calcite(10.4) surface were investigated by imaging the structure formation at the molecular scale. I first demonstrated the functionalization of shape-persistent oligo(p-benzamide)s that was engineered by introducing different functional groups and investigating their effect on the structural formation on the sample surface. The molecular core was designed to provide significant electrostatic anchoring towards the surface, while at the same time maintaining the flexibility to fine-tune the resulting structure by adjusting the intermolecular cohesion energy. The success of this strategy is based on a clear separation of the molecule-substrate interaction from the molecule-molecule interaction. My results show that sufficient molecule-surface anchoring can be achieved without restricting the structural flexibility that is needed for the design of complex molecular systems. Three derivatives of terephthalic acid (TPA) were investigated in chapter 7. Here, the focus was on changing the adhesion to the calcite surface by introducing different anchor functionalities to the TPA backbone. For all observed molecules, the strong substrate templating effect results in molecular structures that are strictly oriented along the calcite main crystal directions. This templating is especially pronounced in the case of 2-ATPA where chain formation on the calcite surface is observed in contrast to the formation of molecular layers in the bulk. At the same time, the amino group of 2-ATPA proved an efficient anchor functionality, successfully stabilizing the molecular chains on the sample surface. These findings emphasizes, once again, the importance of balancing and fine-tuning molecule-molecule and molecule-surface interactions in order to achieve stable, yet structurally flexible molecular arrangements on the sample surface. In the last chapter, I showed how the intrinsic property of molecular chirality decisively influences the structure formation in molecular self-assembly. This effect is especially pronounced in the case of the chiral heptahelicene-2-carboxylic acid. Deposition of the enantiopure molecules results in the formation of homochiral islands on the sample surface which is in sharp contrast to the formation of uni-directional double rows upon deposition of the racemate onto the same surface. While it remained uncertain from these previous experiments whether the double rows are composed of hetero- or homochiral molecules, I could clearly answer that question here and demonstrate that the rows are of heterochiral origin. Chirality, thus, proves to be another important parameter to steer the intermolecular interaction on surfaces. Altogether, the results of this thesis demonstrate that, in order to successfully control the structure formation in molecular self-assembly, the correct combination of molecule and surface properties is crucial. This is of special importance when working on substrates that exhibit a strong influence on the structure formation, such as the calcite(10.4) surface. Through the systematic variation of functional groups several important parameters that influence the balance between molecule-surface and molecule-molecule interaction were identified here, and the results of this thesis can, thus, act as a guideline for the rational design of molecules for use in molecular self-assembly.

Dynamics and control issues of multi-rotor platforms

This thesis deals with the analytic study of dynamics of Multi--Rotor Unmanned Aerial Vehicles. It is conceived to give a set of mathematical instruments apt to the theoretical study and design of these flying machines. The entire work is organized in analogy with classical academic texts about airplane flight dynamics. First, the non--linear equations of motion are defined and all the external actions are modeled, with particular attention to rotors aerodynamics. All the equations are provided in a form, and with personal expedients, to be directly exploitable in a simulation environment. This has requited an answer to questions like the trim of such mathematical systems. All the treatment is developed aiming at the description of different multi--rotor configurations. Then, the linearized equations of motion are derived. The computation of the stability and control derivatives of the linear model is carried out. The study of static and dynamic stability characteristics is, thus, addressed, showing the influence of the various geometric and aerodynamic parameters of the machine and in particular of the rotors. All the theoretic results are finally utilized in two interesting cases. One concerns the design of control systems for attitude stabilization. The linear model permits the tuning of linear controllers gains and the non--linear model allows the numerical testing. The other case is the study of the performances of an innovative configuration of quad--rotor aircraft. With the non--linear model the feasibility of maneuvers impossible for a traditional quad--rotor is assessed. The linear model is applied to the controllability analysis of such an aircraft in case of actuator block.

Development of a system measuring adhesion forces in powder collectives

Fine powders commonly have poor flowability and dispersibility due to interparticle adhesion that leads to formation of agglomerates. Knowing about adhesion in particle collectives is indispensable to gain a deeper fundamental understanding of particle behavior in powders. Especially in pharmaceutical industry a control of adhesion forces in powders is mandatory to improve the performance of inhalation products. Typically the size of inhalable particles is in the range of 1 - 5 µm. In this thesis, a new method was developed to measure adhesion forces of particles as an alternative to the established colloidal probe and centrifuge technique, which are both experimentally demanding, time consuming and of limited practical applicability. The new method is based on detachment of individual particles from a surface due to their inertia. The required acceleration in the order of 500 000 g is provided by a Hopkinson bar shock excitation system and measured via laser vibrometry. Particle detachment events are detected on-line by optical video microscopy. Subsequent automated data evaluation allows obtaining a statistical distribution of particle adhesion forces. To validate the new method, adhesion forces for ensembles of single polystyrene and silica microspheres on a polystyrene coated steel surface were measured under ambient conditions. It was possible to investigate more than 150 individual particles in one experiment and obtain adhesion values of particles in a diameter range of 3 - 13 µm. This enables a statistical evaluation while measuring effort and time are considerably lower compared to the established techniques. Measured adhesion forces of smaller particles agreed well with values from colloidal probe measurements and theoretical predictions. However, for the larger particles a stronger increase of adhesion with diameter was observed. This discrepancy might be induced by surface roughness and heterogeneity that influence small and large particles differently. By measuring adhesion forces of corrugated dextran particles with sizes down to 2 µm it was demonstrated that the Hopkinson bar method can be used to characterize more complex sample systems as well. Thus, the new device will be applicable to study a broad variety of different particle-surface combinations on a routine basis, including strongly cohesive powders like pharmaceutical drugs for inhalation.

Modelling and simulating in systems biology: an approach based on multi-agent systems

Systems Biology is an innovative way of doing biology recently raised in bio-informatics contexts, characterised by the study of biological systems as complex systems with a strong focus on the system level and on the interaction dimension. In other words, the objective is to understand biological systems as a whole, putting on the foreground not only the study of the individual parts as standalone parts, but also of their interaction and of the global properties that emerge at the system level by means of the interaction among the parts. This thesis focuses on the adoption of multi-agent systems (MAS) as a suitable paradigm for Systems Biology, for developing models and simulation of complex biological systems. Multi-agent system have been recently introduced in informatics context as a suitabe paradigm for modelling and engineering complex systems. Roughly speaking, a MAS can be conceived as a set of autonomous and interacting entities, called agents, situated in some kind of nvironment, where they fruitfully interact and coordinate so as to obtain a coherent global system behaviour. The claim of this work is that the general properties of MAS make them an effective approach for modelling and building simulations of complex biological systems, following the methodological principles identified by Systems Biology. In particular, the thesis focuses on cell populations as biological systems. In order to support the claim, the thesis introduces and describes (i) a MAS-based model conceived for modelling the dynamics of systems of cells interacting inside cell environment called niches. (ii) a computational tool, developed for implementing the models and executing the simulations. The tool is meant to work as a kind of virtual laboratory, on top of which kinds of virtual experiments can be performed, characterised by the definition and execution of specific models implemented as MASs, so as to support the validation, falsification and improvement of the models through the observation and analysis of the simulations. A hematopoietic stem cell system is taken as reference case study for formulating a specific model and executing virtual experiments.

Development of Dynamic Constraint Models for a Model Based Transient Calibration Process

Model based calibration has gained popularity in recent years as a method to optimize increasingly complex engine systems. However virtually all model based techniques are applied to steady state calibration. Transient calibration is by and large an emerging technology. An important piece of any transient calibration process is the ability to constrain the optimizer to treat the problem as a dynamic one and not as a quasi-static process. The optimized air-handling parameters corresponding to any instant of time must be achievable in a transient sense; this in turn depends on the trajectory of the same parameters over previous time instances. In this work dynamic constraint models have been proposed to translate commanded to actually achieved air-handling parameters. These models enable the optimization to be realistic in a transient sense. The air handling system has been treated as a linear second order system with PD control. Parameters for this second order system have been extracted from real transient data. The model has been shown to be the best choice relative to a list of appropriate candidates such as neural networks and first order models. The selected second order model was used in conjunction with transient emission models to predict emissions over the FTP cycle. It has been shown that emission predictions based on air-handing parameters predicted by the dynamic constraint model do not differ significantly from corresponding emissions based on measured air-handling parameters.