Interacting software-agents to support a kind of expert supervision systems

Expert supervision systems are software applications specially designed to automate process monitoring. The goal is to reduce the dependency on human operators to assure the correct operation of a process including faulty situations. Construction of this kind of application involves an important task of design and development in order to represent and to manipulate process data and behaviour at different degrees of abstraction for interfacing with data acquisition systems connected to the process. This is an open problem that becomes more complex with the number of variables, parameters and relations to account for the complexity of the process. Multiple specialised modules tuned to solve simpler tasks that operate under a co-ordination provide a solution. A modular architecture based on concepts of software agents, taking advantage of the integration of diverse knowledge-based techniques, is proposed for this purpose. The components (software agents, communication mechanisms and perception/action mechanisms) are based on ICa (Intelligent Control architecture), software middleware supporting the build-up of applications with software agent features

Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Fusing data-mining and knowledge elicitation for the supervisory control of an aluminium plate mill

Knowledge-elicitation is a common technique used to produce rules about the operation of a plant from the knowledge that is available from human expertise. Similarly, data-mining is becoming a popular technique to extract rules from the data available from the operation of a plant. In the work reported here knowledge was required to enable the supervisory control of an aluminium hot strip mill by the determination of mill set-points. A method was developed to fuse knowledge-elicitation and data-mining to incorporate the best aspects of each technique, whilst avoiding known problems. Utilisation of the knowledge was through an expert system, which determined schedules of set-points and provided information to human operators. The results show that the method proposed in this paper was effective in producing rules for the on-line control of a complex industrial process.

A situation awareness interface for a bi-wheeled industrial hovercraft: design, development and evaluation

The intention of this thesis is to develop a prototype interface that enables an operator to control a bi-wheeled industrial hovercraft that will work within a fusion power plant if the automation system fails. This fusion power plant is part of the ITER project a conjoint effort of various industrialized countries to develop cleaner sources of energy. The development of the interface prototype will be based on situation awareness concepts, which provide a means to understand how human operators perceive the world around, then process that information and make decisions based on the knowledge that they already have and the projected knowledge of the reactions that will occur in the world in response to the actions the operator makes. Two major situation awareness methods will be used, GDTA as a means to discover the requirements the interface needs to solve, and SAGAT to conduct the evaluation on the three interfaces. This technique can isolate the differences an operator has in situation awareness when presented with relevant information given by each of the three interfaces that were built for this thesis. Where the first interface presents the information within the operator’s focal point of view in a pictorial style, the second interface shows the same information within the same point of view has the first interface but only shows it in a textual manner. While the third interface shows the relevant information in the operator’s peripheral field of view. Also SAGAT can provide insight on the question to know if providing the operator with feed-forward information about the stoppage distances of the bi-wheeled industrial hovercraft has any effect on the operator’s decision making.

Redes neurais aplicadas em processos de usinagem da madeira

Pós-graduação em Engenharia Mecânica - FEG

Influence of the deposition parameters on the morphology and electrical conductivity of PANI/PSS self-assembled films

The influence of deposition parameters, namely polymer concentration and pH of the deposition solution, cleaning, and drying steps on the morphology and electrical characteristics of polyaniline and sulfonated polystyrene (PANI/PSS) nanostructured films deposited by the self-assembly technique is evaluated by UV-Vis spectroscopy, optical and atomic force microscopy, and electrical resistance measurements. It is found that stirring the cleaning solution during the cleaning step is crucial for obtaining homogenous films. Stirring of the cleaning solution also influences the amount of PANI adsorbed in the films. In this regard, the drying process seems to be less critical since PANI amount and film thickness are similar in films dried with N-2 flow or with an absorbent tissue. It is observed, however, that drying with N-2 flow results in rougher films. As an additional point, an assessment of the influence of the deposition method (manual versus mechanical) on the film characteristics was carried out. A significant difference on the amount of PANI and film thickness between films prepared by different human operators and by a homemade mechanical device was observed. The variability in film thickness and PANI adsorbed amount is smaller in films mechanically assembled. (c) 2007 Elsevier B.V. All rights reserved.

Sistemi riconfigurabili a basso consumo per applicazioni di monitoraggio distribuito

The term Ambient Intelligence (AmI) refers to a vision on the future of the information society where smart, electronic environment are sensitive and responsive to the presence of people and their activities (Context awareness). In an ambient intelligence world, devices work in concert to support people in carrying out their everyday life activities, tasks and rituals in an easy, natural way using information and intelligence that is hidden in the network connecting these devices. This promotes the creation of pervasive environments improving the quality of life of the occupants and enhancing the human experience. AmI stems from the convergence of three key technologies: ubiquitous computing, ubiquitous communication and natural interfaces. Ambient intelligent systems are heterogeneous and require an excellent cooperation between several hardware/software technologies and disciplines, including signal processing, networking and protocols, embedded systems, information management, and distributed algorithms. Since a large amount of fixed and mobile sensors embedded is deployed into the environment, the Wireless Sensor Networks is one of the most relevant enabling technologies for AmI. WSN are complex systems made up of a number of sensor nodes which can be deployed in a target area to sense physical phenomena and communicate with other nodes and base stations. These simple devices typically embed a low power computational unit (microcontrollers, FPGAs etc.), a wireless communication unit, one or more sensors and a some form of energy supply (either batteries or energy scavenger modules). WNS promises of revolutionizing the interactions between the real physical worlds and human beings. Low-cost, low-computational power, low energy consumption and small size are characteristics that must be taken into consideration when designing and dealing with WSNs. To fully exploit the potential of distributed sensing approaches, a set of challengesmust be addressed. Sensor nodes are inherently resource-constrained systems with very low power consumption and small size requirements which enables than to reduce the interference on the physical phenomena sensed and to allow easy and low-cost deployment. They have limited processing speed,storage capacity and communication bandwidth that must be efficiently used to increase the degree of local ”understanding” of the observed phenomena. A particular case of sensor nodes are video sensors. This topic holds strong interest for a wide range of contexts such as military, security, robotics and most recently consumer applications. Vision sensors are extremely effective for medium to long-range sensing because vision provides rich information to human operators. However, image sensors generate a huge amount of data, whichmust be heavily processed before it is transmitted due to the scarce bandwidth capability of radio interfaces. In particular, in video-surveillance, it has been shown that source-side compression is mandatory due to limited bandwidth and delay constraints. Moreover, there is an ample opportunity for performing higher-level processing functions, such as object recognition that has the potential to drastically reduce the required bandwidth (e.g. by transmitting compressed images only when something ‘interesting‘ is detected). The energy cost of image processing must however be carefully minimized. Imaging could play and plays an important role in sensing devices for ambient intelligence. Computer vision can for instance be used for recognising persons and objects and recognising behaviour such as illness and rioting. Having a wireless camera as a camera mote opens the way for distributed scene analysis. More eyes see more than one and a camera system that can observe a scene from multiple directions would be able to overcome occlusion problems and could describe objects in their true 3D appearance. In real-time, these approaches are a recently opened field of research. In this thesis we pay attention to the realities of hardware/software technologies and the design needed to realize systems for distributed monitoring, attempting to propose solutions on open issues and filling the gap between AmI scenarios and hardware reality. The physical implementation of an individual wireless node is constrained by three important metrics which are outlined below. Despite that the design of the sensor network and its sensor nodes is strictly application dependent, a number of constraints should almost always be considered. Among them: • Small form factor to reduce nodes intrusiveness. • Low power consumption to reduce battery size and to extend nodes lifetime. • Low cost for a widespread diffusion. These limitations typically result in the adoption of low power, low cost devices such as low powermicrocontrollers with few kilobytes of RAMand tenth of kilobytes of program memory with whomonly simple data processing algorithms can be implemented. However the overall computational power of the WNS can be very large since the network presents a high degree of parallelism that can be exploited through the adoption of ad-hoc techniques. Furthermore through the fusion of information from the dense mesh of sensors even complex phenomena can be monitored. In this dissertation we present our results in building several AmI applications suitable for a WSN implementation. The work can be divided into two main areas:Low Power Video Sensor Node and Video Processing Alghoritm and Multimodal Surveillance . Low Power Video Sensor Nodes and Video Processing Alghoritms In comparison to scalar sensors, such as temperature, pressure, humidity, velocity, and acceleration sensors, vision sensors generate much higher bandwidth data due to the two-dimensional nature of their pixel array. We have tackled all the constraints listed above and have proposed solutions to overcome the current WSNlimits for Video sensor node. We have designed and developed wireless video sensor nodes focusing on the small size and the flexibility of reuse in different applications. The video nodes target a different design point: the portability (on-board power supply, wireless communication), a scanty power budget (500mW),while still providing a prominent level of intelligence, namely sophisticated classification algorithmand high level of reconfigurability. We developed two different video sensor node: The device architecture of the first one is based on a low-cost low-power FPGA+microcontroller system-on-chip. The second one is based on ARM9 processor. Both systems designed within the above mentioned power envelope could operate in a continuous fashion with Li-Polymer battery pack and solar panel. Novel low power low cost video sensor nodes which, in contrast to sensors that just watch the world, are capable of comprehending the perceived information in order to interpret it locally, are presented. Featuring such intelligence, these nodes would be able to cope with such tasks as recognition of unattended bags in airports, persons carrying potentially dangerous objects, etc.,which normally require a human operator. Vision algorithms for object detection, acquisition like human detection with Support Vector Machine (SVM) classification and abandoned/removed object detection are implemented, described and illustrated on real world data. Multimodal surveillance: In several setup the use of wired video cameras may not be possible. For this reason building an energy efficient wireless vision network for monitoring and surveillance is one of the major efforts in the sensor network community. Energy efficiency for wireless smart camera networks is one of the major efforts in distributed monitoring and surveillance community. For this reason, building an energy efficient wireless vision network for monitoring and surveillance is one of the major efforts in the sensor network community. The Pyroelectric Infra-Red (PIR) sensors have been used to extend the lifetime of a solar-powered video sensor node by providing an energy level dependent trigger to the video camera and the wireless module. Such approach has shown to be able to extend node lifetime and possibly result in continuous operation of the node.Being low-cost, passive (thus low-power) and presenting a limited form factor, PIR sensors are well suited for WSN applications. Moreover techniques to have aggressive power management policies are essential for achieving long-termoperating on standalone distributed cameras needed to improve the power consumption. We have used an adaptive controller like Model Predictive Control (MPC) to help the system to improve the performances outperforming naive power management policies.

IDT-3D: Identification and tracking in controlled environments using a 3D unified user interface

Identification and tracking of objects in specific environments such as harbors or security areas is a matter of great importance nowadays. With this purpose, numerous systems based on different technologies have been developed, resulting in a great amount of gathered data displayed through a variety of interfaces. Such amount of information has to be evaluated by human operators in order to take the correct decisions, sometimes under highly critical situations demanding both speed and accuracy. In order to face this problem we describe IDT-3D, a platform for identification and tracking of vessels in a harbour environment able to represent fused information in real time using a Virtual Reality application. The effectiveness of using IDT-3D as an integrated surveillance system is currently under evaluation. Preliminary results point to a significant decrease in the times of reaction and decision making of operators facing up a critical situation. Although the current application focus of IDT-3D is quite specific, the results of this research could be extended to the identification and tracking of targets in other controlled environments of interest as coastlines, borders or even urban areas.

Desarrollo de un proceso de razonamiento visual para la valoración del patrimonio

El tema de la presente tesis es la valoración del patrimonio y en ella se considera que el patrimonio es un proceso cultural interesado en negociar, crear y recrear recuerdos, valores y significados culturales. Actualmente el patrimonio como proceso se está consolidando en la literatura científica, aunque la idea de que es una ‘cosa’ es dominante en el debate internacional y está respaldada tanto por políticas como prácticas de la UNESCO. El considerar el patrimonio como un proceso permite una mirada crítica, que subraya la significación. Es decir, supone el correlato que conlleva definir algo como ‘patrimonio’, o hacer que lo vaya siendo. Esta visión del concepto permite la posibilidad de comprender no sólo lo que se ha valorado, sino también lo que se ha olvidado y el porqué. El principal objetivo de esta investigación es explorar las características de un proceso de razonamiento visual para aplicarlo en el de valoración del patrimonio. Éste que se presenta, implica la creación de representaciones visuales y sus relaciones, además su meta no está centrada en producir un ambiente que sea indiferenciado de la realidad física. Con él se pretende ofrecer la posibilidad de comunicar la dimensión ‘poliédrica’ del patrimonio. Para que este nuevo proceso que propongo sea viable y sostenible, existe la necesidad de tener en cuenta el fin que se quiere lograr: la valoración. Es importante considerar que es un proceso en el cual las dinámicas de aprendizaje, comportamientos y exploración del patrimonio están directamente relacionadas con su valoración. Por lo tanto, hay que saber cómo se genera la valoración del patrimonio, con el fin de ser capaces de desarrollar el proceso adaptado a estas dinámicas. La hipótesis de esta tesis defiende que un proceso de razonamiento visual para la valoración del patrimonio permite que las personas involucradas en el proceso inicien un proceso de interacción con un elemento patrimonial y su imagen mental para llegar a ciertas conclusiones con respecto a su valor y significado. El trabajo describe la metodología que da lugar al proceso de razonamiento visual para el patrimonio, que ha sido concebido sobre un modelado descriptivo de procesos, donde se han caracterizado tres niveles: meta-nivel, de análisis y operacional. En el modelado del proceso los agentes, junto con el patrimonio, son los protagonistas. El enfoque propuesto no es sólo sobre el patrimonio, sino sobre la compleja relación entre las personas y el patrimonio. Los agentes humanos dan valor a los testimonios de la vida pasada y les imbuyen de significado. Por lo tanto, este enfoque de un proceso de razonamiento visual sirve para detectar los cambios en el valor del patrimonio, además de su dimensión poliédrica en términos espaciales y temporales. Además se ha propuesto una nueva tipología de patrimonio necesaria para sustentar un proceso de razonamiento visual para su valoración. Esta tipología está apoyada en la usabilidad del patrimonio y dentro de ella se encuentran los siguientes tipos de patrimonio: accesible, cautivo, contextualizado, descontextualizado, original y vicarial. El desarrollo de un proceso de razonamiento visual para el patrimonio es una propuesta innovadora porque integra el proceso para su valoración, contemplando la dimensión poliédrica del patrimonio y explotando la potencialidad del razonamiento visual. Además, los posibles usuarios del proceso propuesto van a tener interacción de manera directa con el patrimonio e indirecta con la información relativa a él, como por ejemplo, con los metadatos. Por tanto, el proceso propuesto posibilita que los posibles usuarios se impliquen activamente en la propia valoración del patrimonio. ABSTRACT The subject of this thesis is heritage valuation and it argues that heritage is a cultural process that is inherited, transmitted, and transformed by individuals who are interested in negotiating, creating and recreating memories and cultural meanings. Recently heritage as a process has seen a consolidation in the research, although the idea that heritage is a ‘thing’ is dominant in the international debate and is supported by policies and practice of UNESCO. Seeing heritage as a process enables a critical view, underscoring the significance. That is, it is the correlate involved in defining something as ‘heritage’, or converting it into heritage. This view of the concept allows the possibility to understand not only what has been valued, but also what has been forgotten and why. The main objective of this research is to explore the characteristics of a visual reasoning process in order to apply it to a heritage valuation. The goal of the process is not centered on producing an environment that is undifferentiated from physical reality. Thus, the objective of the process is to provide the ability to communicate the ‘polyhedral’ dimension of heritage. For this new process to be viable and sustainable, it is necessary to consider what is to be achieved: heritage valuation. It is important to note that it is a process in which the dynamics of learning, behavior and exploration heritage are directly related to its valuation. Therefore, we need to know how this valuation takes place in order to be able to develop a process that is adapted to these dynamic. The hypothesis of this thesis argues that a visual reasoning process for heritage valuation allows people involved in the process to initiate an interaction with a heritage and to build its mental image to reach certain conclusions regarding its value and meaning. The thesis describes the methodology that results in a visual reasoning process for heritage valuation, which has been based on a descriptive modeling process and have characterized three levels: meta, analysis and operational -level. The agents are the protagonists in the process, along with heritage. The proposed approach is not only about heritage but the complex relationship between people and heritage. Human operators give value to the testimonies of past life and imbue them with meaning. Therefore, this approach of a visual reasoning process is used to detect changes in the value of heritage and its multifaceted dimension in spatial and temporal terms. A new type of heritage required to support a visual reasoning process for heritage valuation has also been proposed. This type is supported by its usability and it covers the following types of heritage: available, captive, contextualized, decontextualized, original and vicarious. The development of a visual reasoning process for heritage valuation is innovative because it integrates the process for valuation of heritage, considering the multifaceted dimension of heritage and exploiting the potential of visual reasoning. In addition, potential users of the proposed process will have direct interaction with heritage and indirectly with the information about it, such as the metadata. Therefore, the proposed process enables potential users to be actively involved in their own heritage valuation.

Designing a Computer Vision System for Underwater Robotic Interventions

This thesis deals with the challenging problem of designing systems able to perceive objects in underwater environments. In the last few decades research activities in robotics have advanced the state of art regarding intervention capabilities of autonomous systems. State of art in fields such as localization and navigation, real time perception and cognition, safe action and manipulation capabilities, applied to ground environments (both indoor and outdoor) has now reached such a readiness level that it allows high level autonomous operations. On the opposite side, the underwater environment remains a very difficult one for autonomous robots. Water influences the mechanical and electrical design of systems, interferes with sensors by limiting their capabilities, heavily impacts on data transmissions, and generally requires systems with low power consumption in order to enable reasonable mission duration. Interest in underwater applications is driven by needs of exploring and intervening in environments in which human capabilities are very limited. Nowadays, most underwater field operations are carried out by manned or remotely operated vehicles, deployed for explorations and limited intervention missions. Manned vehicles, directly on-board controlled, expose human operators to risks related to the stay in field of the mission, within a hostile environment. Remotely Operated Vehicles (ROV) currently represent the most advanced technology for underwater intervention services available on the market. These vehicles can be remotely operated for long time but they need support from an oceanographic vessel with multiple teams of highly specialized pilots. Vehicles equipped with multiple state-of-art sensors and capable to autonomously plan missions have been deployed in the last ten years and exploited as observers for underwater fauna, seabed, ship wrecks, and so on. On the other hand, underwater operations like object recovery and equipment maintenance are still challenging tasks to be conducted without human supervision since they require object perception and localization with much higher accuracy and robustness, to a degree seldom available in Autonomous Underwater Vehicles (AUV). This thesis reports the study, from design to deployment and evaluation, of a general purpose and configurable platform dedicated to stereo-vision perception in underwater environments. Several aspects related to the peculiar environment characteristics have been taken into account during all stages of system design and evaluation: depth of operation and light conditions, together with water turbidity and external weather, heavily impact on perception capabilities. The vision platform proposed in this work is a modular system comprising off-the-shelf components for both the imaging sensors and the computational unit, linked by a high performance ethernet network bus. The adopted design philosophy aims at achieving high flexibility in terms of feasible perception applications, that should not be as limited as in case of a special-purpose and dedicated hardware. Flexibility is required by the variability of underwater environments, with water conditions ranging from clear to turbid, light backscattering varying with daylight and depth, strong color distortion, and other environmental factors. Furthermore, the proposed modular design ensures an easier maintenance and update of the system over time. Performance of the proposed system, in terms of perception capabilities, has been evaluated in several underwater contexts taking advantage of the opportunity offered by the MARIS national project. Design issues like energy power consumption, heat dissipation and network capabilities have been evaluated in different scenarios. Finally, real-world experiments, conducted in multiple and variable underwater contexts, including open sea waters, have led to the collection of several datasets that have been publicly released to the scientific community. The vision system has been integrated in a state of the art AUV equipped with a robotic arm and gripper, and has been exploited in the robot control loop to successfully perform underwater grasping operations.

Data Reduction in Intrusion Alert Correlation

Network intrusion detection sensors are usually built around low level models of network traffic. This means that their output is of a similarly low level and as a consequence, is difficult to analyze. Intrusion alert correlation is the task of automating some of this analysis by grouping related alerts together. Attack graphs provide an intuitive model for such analysis. Unfortunately alert flooding attacks can still cause a loss of service on sensors, and when performing attack graph correlation, there can be a large number of extraneous alerts included in the output graph. This obscures the fine structure of genuine attacks and makes them more difficult for human operators to discern. This paper explores modified correlation algorithms which attempt to minimize the impact of this attack.

Formalizing operator task analysis

Human operators are unique in their decision making capability, judgment and nondeterminism. Their sense of judgment, unpredictable decision procedures, susceptibility to environmental elements can cause them to erroneously execute a given task description to operate a computer system. Usually, a computer system is protected against some erroneous human behaviors by having necessary safeguard mechanisms in place. But some erroneous human operator behaviors can lead to severe or even fatal consequences especially in safety critical systems. A generalized methodology that can allow modeling and analyzing the interactions between computer systems and human operators where the operators are allowed to deviate from their prescribed behaviors will provide a formal understanding of the robustness of a computer system against possible aberrant behaviors by its human operators. We provide several methodology for assisting in modeling and analyzing human behaviors exhibited while operating computer systems. Every human operator is usually given a specific recommended set of guidelines for operating a system. We first present process algebraic methodology for modeling and verifying recommended human task execution behavior. We present how one can perform runtime monitoring of a computer system being operated by a human operator for checking violation of temporal safety properties. We consider the concept of a protection envelope giving a wider class of behaviors than those strictly prescribed by a human task that can be tolerated by a system. We then provide a framework for determining whether a computer system can maintain its guarantees if the human operators operate within their protection envelopes. This framework also helps to determine the robustness of the computer system under weakening of the protection envelopes. In this regard, we present a tool called Tutela that assists in implementing the framework. We then examine the ability of a system to remain safe under broad classes of variations of the prescribed human task. We develop a framework for addressing two issues. The first issue is: given a human task specification and a protection envelope, will the protection envelope properties still hold under standard erroneous executions of that task by the human operators? In other words how robust is the protection envelope? The second issue is: in the absence of a protection envelope, can we approximate a protection envelope encompassing those standard erroneous human behaviors that can be safely endured by the system? We present an extension of Tutela that implements this framework. The two frameworks mentioned above use Concurrent Game Structures (CGS) as models for both computer systems and their human operators. However, there are some shortcomings of this formalism for our uses. We add incomplete information concepts in CGSs to achieve better modularity for the players. We introduce nondeterminism in both the transition system and strategies of players and in the modeling of human operators and computer systems. Nondeterministic action strategies for players in \emph{i}ncomplete information \emph{N}ondeterministic CGS (iNCGS) is a more precise formalism for modeling human behaviors exhibited while operating a computer system. We show how we can reason about a human behavior satisfying a guarantee by providing a semantics of Alternating Time Temporal Logic based on iNCGS player strategies. In a nutshell this dissertation provides formal methodology for modeling and analyzing system robustness against both expected and erroneous human operator behaviors.

Supporting Scientific Research Through Machine and Deep Learning: Fluorescence Microscopy and Operational Intelligence Use Cases

Although the debate of what data science is has a long history and has not reached a complete consensus yet, Data Science can be summarized as the process of learning from data. Guided by the above vision, this thesis presents two independent data science projects developed in the scope of multidisciplinary applied research. The first part analyzes fluorescence microscopy images typically produced in life science experiments, where the objective is to count how many marked neuronal cells are present in each image. Aiming to automate the task for supporting research in the area, we propose a neural network architecture tuned specifically for this use case, cell ResUnet (c-ResUnet), and discuss the impact of alternative training strategies in overcoming particular challenges of our data. The approach provides good results in terms of both detection and counting, showing performance comparable to the interpretation of human operators. As a meaningful addition, we release the pre-trained model and the Fluorescent Neuronal Cells dataset collecting pixel-level annotations of where neuronal cells are located. In this way, we hope to help future research in the area and foster innovative methodologies for tackling similar problems. The second part deals with the problem of distributed data management in the context of LHC experiments, with a focus on supporting ATLAS operations concerning data transfer failures. In particular, we analyze error messages produced by failed transfers and propose a Machine Learning pipeline that leverages the word2vec language model and K-means clustering. This provides groups of similar errors that are presented to human operators as suggestions of potential issues to investigate. The approach is demonstrated on one full day of data, showing promising ability in understanding the message content and providing meaningful groupings, in line with previously reported incidents by human operators.

Integration of the computer controlled screwdriver for Robotic manufacturing in the ROS environment

The computer controlled screwdriver is a modern technique to perform automatic screwing/unscrewing operations.The main focus is to study the integration of the computer controlled screwdriver for Robotic manufacturing in the ROS environment.This thesis describes a concept of automatic screwing mechanism composed by universal robots, in which one arm of the robot is for inserting cables and the other is for screwing the cables on the control panel switch gear box. So far this mechanism is carried out by human operators and is a fairly complex one to perform, due to the multiple cables and connections involved. It's for this reason that an automatic cabling and screwing process would be highly preferred within automotive/automation industries. A study is carried out to analyze the difficulties currently faced and a controller based algorithm is developed to replace the manual human efforts using universal robots, thereby allowing robot arms to insert the cables and screw them onto the control panel switch gear box. Experiments were conducted to evaluate the insertion and screwing strategy, which shows the result of inserting and screwing cables on the control panel switch gearbox precisely.

HYBRID AND INCREMENTAL FUZZY LEARNING FOR HUMAN SKIN DETECTION

In this paper, a framework for detection of human skin in digital images is proposed. This framework is composed of a training phase and a detection phase. A skin class model is learned during the training phase by processing several training images in a hybrid and incremental fuzzy learning scheme. This scheme combines unsupervised-and supervised-learning: unsupervised, by fuzzy clustering, to obtain clusters of color groups from training images; and supervised to select groups that represent skin color. At the end of the training phase, aggregation operators are used to provide combinations of selected groups into a skin model. In the detection phase, the learned skin model is used to detect human skin in an efficient way. Experimental results show robust and accurate human skin detection performed by the proposed framework.