Subsuming methods: finding new optimisation opportunities in object-oriented software

The majority of existing application profiling techniques ag- gregate and report performance costs by method or call- ing context. Modern large-scale object-oriented applications consist of thousands of methods with complex calling pat- terns. Consequently, when profiled, their performance costs tend to be thinly distributed across many thousands of loca- tions with few easily identifiable optimisation opportunities. However experienced performance engineers know that there are repeated patterns of method calls in the execution of an application that are induced by the libraries, design patterns and coding idioms used in the software. Automati- cally identifying and aggregating costs over these patterns of method calls allows us to identify opportunities to improve performance based on optimising these patterns. We have developed an analysis technique that is able to identify the entry point methods, which we call subsuming methods, of such patterns. Our ofiine analysis runs over previously collected runtime performance data structured in a calling context tree, such as produced by a large number of existing commercial and open source profilers. We have evaluated our approach on the DaCapo bench- mark suite, showing that our analysis significantly reduces the size and complexity of the runtime performance data set, facilitating its comprehension and interpretation. We also demonstrate, with a collection of case studies, that our analysis identifies new optimisation opportunities that can lead to significant performance improvements (from 20% to over 50% improvement in our case studies).

A Framework supporting collaboration on the distributed design of integrated systems

The work described in this thesis aims to support the distributed design of integrated systems and considers specifically the need for collaborative interaction among designers. Particular emphasis was given to issues which were only marginally considered in previous approaches, such as the abstraction of the distribution of design automation resources over the network, the possibility of both synchronous and asynchronous interaction among designers and the support for extensible design data models. Such issues demand a rather complex software infrastructure, as possible solutions must encompass a wide range of software modules: from user interfaces to middleware to databases. To build such structure, several engineering techniques were employed and some original solutions were devised. The core of the proposed solution is based in the joint application of two homonymic technologies: CAD Frameworks and object-oriented frameworks. The former concept was coined in the late 80's within the electronic design automation community and comprehends a layered software environment which aims to support CAD tool developers, CAD administrators/integrators and designers. The latter, developed during the last decade by the software engineering community, is a software architecture model to build extensible and reusable object-oriented software subsystems. In this work, we proposed to create an object-oriented framework which includes extensible sets of design data primitives and design tool building blocks. Such object-oriented framework is included within a CAD Framework, where it plays important roles on typical CAD Framework services such as design data representation and management, versioning, user interfaces, design management and tool integration. The implemented CAD Framework - named Cave2 - followed the classical layered architecture presented by Barnes, Harrison, Newton and Spickelmier, but the possibilities granted by the use of the object-oriented framework foundations allowed a series of improvements which were not available in previous approaches: - object-oriented frameworks are extensible by design, thus this should be also true regarding the implemented sets of design data primitives and design tool building blocks. This means that both the design representation model and the software modules dealing with it can be upgraded or adapted to a particular design methodology, and that such extensions and adaptations will still inherit the architectural and functional aspects implemented in the object-oriented framework foundation; - the design semantics and the design visualization are both part of the object-oriented framework, but in clearly separated models. This allows for different visualization strategies for a given design data set, which gives collaborating parties the flexibility to choose individual visualization settings; - the control of the consistency between semantics and visualization - a particularly important issue in a design environment with multiple views of a single design - is also included in the foundations of the object-oriented framework. Such mechanism is generic enough to be also used by further extensions of the design data model, as it is based on the inversion of control between view and semantics. The view receives the user input and propagates such event to the semantic model, which evaluates if a state change is possible. If positive, it triggers the change of state of both semantics and view. Our approach took advantage of such inversion of control and included an layer between semantics and view to take into account the possibility of multi-view consistency; - to optimize the consistency control mechanism between views and semantics, we propose an event-based approach that captures each discrete interaction of a designer with his/her respective design views. The information about each interaction is encapsulated inside an event object, which may be propagated to the design semantics - and thus to other possible views - according to the consistency policy which is being used. Furthermore, the use of event pools allows for a late synchronization between view and semantics in case of unavailability of a network connection between them; - the use of proxy objects raised significantly the abstraction of the integration of design automation resources, as either remote or local tools and services are accessed through method calls in a local object. The connection to remote tools and services using a look-up protocol also abstracted completely the network location of such resources, allowing for resource addition and removal during runtime; - the implemented CAD Framework is completely based on Java technology, so it relies on the Java Virtual Machine as the layer which grants the independence between the CAD Framework and the operating system. All such improvements contributed to a higher abstraction on the distribution of design automation resources and also introduced a new paradigm for the remote interaction between designers. The resulting CAD Framework is able to support fine-grained collaboration based on events, so every single design update performed by a designer can be propagated to the rest of the design team regardless of their location in the distributed environment. This can increase the group awareness and allow a richer transfer of experiences among them, improving significantly the collaboration potential when compared to previously proposed file-based or record-based approaches. Three different case studies were conducted to validate the proposed approach, each one focusing one a subset of the contributions of this thesis. The first one uses the proxy-based resource distribution architecture to implement a prototyping platform using reconfigurable hardware modules. The second one extends the foundations of the implemented object-oriented framework to support interface-based design. Such extensions - design representation primitives and tool blocks - are used to implement a design entry tool named IBlaDe, which allows the collaborative creation of functional and structural models of integrated systems. The third case study regards the possibility of integration of multimedia metadata to the design data model. Such possibility is explored in the frame of an online educational and training platform.

A logical approach to the management of object identifiers in non-conventional database management systems

Non-conventional database management systems are used to achieve a better performance when dealing with complex data. One fundamental concept of these systems is object identity (OID), because each object in the database has a unique identifier that is used to access and reference it in relationships to other objects. Two approaches can be used for the implementation of OIDs: physical or logical OIDs. In order to manage complex data, was proposed the Multimedia Data Manager Kernel (NuGeM) that uses a logical technique, named Indirect Mapping. This paper proposes an improvement to the technique used by NuGeM, whose original contribution is management of OIDs with a fewer number of disc accesses and less processing, thus reducing management time from the pages and eliminating the problem with exhaustion of OIDs. Also, the technique presented here can be applied to others OODBMSs. © 2011 IEEE.

Managing object identity with less I/O operations

Non-conventional database management systems are used to achieve a better performance when dealing with complex data. One fundamental concept of these systems is object identity (OID). Two techniques can be used for the implementation of OIDs: physical or logical. A logical implementation of OIDs, based on an Indirection Table, is used by NuGeM, a multimedia data manager kernel which is described in this paper. NuGeM Indirection Table allows the relocation of all pages in a database. The proposed strategy modifies the workings of this table so that it is possible to reduce considerably the number of I/O operations during the request and release of pages containing objects and their OIDs. Tests show a reduction of 84% in reading operations and a 67% reduction in writing operations when pages are requested. Although no changes were observed in writing operations during the release of pages, a 100% of reduction in reading operations was obtained. © 2012 IEEE.

Design of a Wireless Vision Sensor for Object Tracking in Wireless Vision Sensor Networks

The integration of CMOS cameras with embedded processors and wireless communication devices has enabled the development of distributed wireless vision systems. Wireless Vision Sensor Networks (WVSNs), which consist of wirelessly connected embedded systems with vision and sensing capabilities, provide wide variety of application areas that have not been possible to realize with the wall-powered vision systems with wired links or scalar-data based wireless sensor networks. In this paper, the design of a middleware for a wireless vision sensor node is presented for the realization of WVSNs. The implemented wireless vision sensor node is tested through a simple vision application to study and analyze its capabilities, and determine the challenges in distributed vision applications through a wireless network of low-power embedded devices. The results of this paper highlight the practical concerns for the development of efficient image processing and communication solutions for WVSNs and emphasize the need for cross-layer solutions that unify these two so-far-independent research areas.

Type Systems for Distributed Programs: Components and Sessions

Modern software systems, in particular distributed ones, are everywhere around us and are at the basis of our everyday activities. Hence, guaranteeing their cor- rectness, consistency and safety is of paramount importance. Their complexity makes the verification of such properties a very challenging task. It is natural to expect that these systems are reliable and above all usable. i) In order to be reliable, compositional models of software systems need to account for consistent dynamic reconfiguration, i.e., changing at runtime the communication patterns of a program. ii) In order to be useful, compositional models of software systems need to account for interaction, which can be seen as communication patterns among components which collaborate together to achieve a common task. The aim of the Ph.D. was to develop powerful techniques based on formal methods for the verification of correctness, consistency and safety properties related to dynamic reconfiguration and communication in complex distributed systems. In particular, static analysis techniques based on types and type systems appeared to be an adequate methodology, considering their success in guaranteeing not only basic safety properties, but also more sophisticated ones like, deadlock or livelock freedom in a concurrent setting. The main contributions of this dissertation are twofold. i) On the components side: we design types and a type system for a concurrent object-oriented calculus to statically ensure consistency of dynamic reconfigurations related to modifications of communication patterns in a program during execution time. ii) On the communication side: we study advanced safety properties related to communication in complex distributed systems like deadlock-freedom, livelock- freedom and progress. Most importantly, we exploit an encoding of types and terms of a typical distributed language, session π-calculus, into the standard typed π- calculus, in order to understand their expressive power.

Investigating human audio-visual object perception with a combination of hypothesis-generating and hypothesis-testing fMRI analysis tools

Primate multisensory object perception involves distributed brain regions. To investigate the network character of these regions of the human brain, we applied data-driven group spatial independent component analysis (ICA) to a functional magnetic resonance imaging (fMRI) data set acquired during a passive audio-visual (AV) experiment with common object stimuli. We labeled three group-level independent component (IC) maps as auditory (A), visual (V), and AV, based on their spatial layouts and activation time courses. The overlap between these IC maps served as definition of a distributed network of multisensory candidate regions including superior temporal, ventral occipito-temporal, posterior parietal and prefrontal regions. During an independent second fMRI experiment, we explicitly tested their involvement in AV integration. Activations in nine out of these twelve regions met the max-criterion (A < AV > V) for multisensory integration. Comparison of this approach with a general linear model-based region-of-interest definition revealed its complementary value for multisensory neuroimaging. In conclusion, we estimated functional networks of uni- and multisensory functional connectivity from one dataset and validated their functional roles in an independent dataset. These findings demonstrate the particular value of ICA for multisensory neuroimaging research and using independent datasets to test hypotheses generated from a data-driven analysis.

Distributed Repositories for Educational Content

As education providers increasingly integrate digital learning media into their education processes, the need for the systematic management of learning materials and learning arrangements becomes clearer. Digital repositories, often called Learning Object Repositories (LOR), promise to provide an answer to this challenge. This article is composed of two parts. In this part, we derive technological and pedagogical requirements for LORs from a concretization of information quality criteria for e-learning technology. We review the evolution of learning object repositories and discuss their core features in the context of pedagogical requirements, information quality demands, and e-learning technology standards. We conclude with an outlook in Part 2, which presents concrete technical solutions, in particular networked repository architectures.

Distributed Repositories for Educational Content

In Part 1 of this article we discussed the need for information quality and the systematic management of learning materials and learning arrangements. Digital repositories, often called Learning Object Repositories (LOR), were introduced as a promising answer to this challenge. We also derived technological and pedagogical requirements for LORs from a concretization of information quality criteria for e-learning technology. This second part presents technical solutions that particularly address the demands of open education movements, which aspire to a global reuse and sharing culture. From this viewpoint, we develop core requirements for scalable network architectures for educational content management. We then present edu-sharing, an advanced example of a network of homogeneous repositories for learning resources, and discuss related technology. We conclude with an outlook in terms of emerging developments towards open and networked system architectures in e-learning.

User and Machine Authentication and Authorization Infrastructure for Distributed Wireless Sensor Network Testbeds

The intention of an authentication and authorization infrastructure (AAI) is to simplify and unify access to different web resources. With a single login, a user can access web applications at multiple organizations. The Shibboleth authentication and authorization infrastructure is a standards-based, open source software package for web single sign-on (SSO) across or within organizational boundaries. It allows service providers to make fine-grained authorization decisions for individual access of protected online resources. The Shibboleth system is a widely used AAI, but only supports protection of browser-based web resources. We have implemented a Shibboleth AAI extension to protect web services using Simple Object Access Protocol (SOAP). Besides user authentication for browser-based web resources, this extension also provides user and machine authentication for web service-based resources. Although implemented for a Shibboleth AAI, the architecture can be easily adapted to other AAIs.

Rule Representation in Distributed Environments with Accepting Networks of Splicing Processors.

This paper presents the model named Accepting Networks of Evolutionary Processors as NP-problem solver inspired in the biological DNA operations. A processor has a rules set, splicing rules in this model,an object multiset and a filters set. Rules can be applied in parallel since there exists a large number of copies of objects in the multiset. Processors can form a graph in order to solve a given problem. This paper shows the network configuration in order to solve the SAT problem using linear resources and time. A rule representation arquitecture in distributed environments can be easily implemented using these networks of processors, such as decision support systems, as shown in the paper.

Distributed control strategies for photovoltaic systems in urban environments

Este trabajo es una contribución a los sistemas fotovoltaicos (FV) con seguimiento distribuido del punto de máxima potencia (DMPPT), una topología que se caracteriza porque lleva a cabo el MPPT a nivel de módulo, al contrario de las topologías más tradicionales que llevan a cabo el MPPT para un número más elevado de módulos, pudiendo ser hasta cientos de módulos. Las dos tecnologías DMPPT que existen en el mercado son conocidos como microinversores y optimizadores de potencia, y ofrecen ciertas ventajas sobre sistemas de MPPT central como: mayor producción en situaciones de mismatch, monitorización individual de cada módulo, flexibilidad de diseño, mayor seguridad del sistema, etc. Aunque los sistemas DMPPT no están limitados a los entornos urbanos, se ha enfatizado en el título ya que es su mercado natural, siendo difícil una justificación de su sobrecoste en grandes huertas solares en suelo. Desde el año 2010 el mercado de estos sistemas ha incrementado notablemente y sigue creciendo de una forma continuada. Sin embargo, todavía falta un conocimiento profundo de cómo funcionan estos sistemas, especialmente en el caso de los optimizadores de potencia, de las ganancias energéticas esperables en condiciones de mismatch y de las posibilidades avanzadas de diagnóstico de fallos. El principal objetivo de esta tesis es presentar un estudio completo de cómo funcionan los sistemas DMPPT, sus límites y sus ventajas, así como experimentos varios que verifican la teoría y el desarrollo de herramientas para valorar las ventajas de utilizar DMPPT en cada instalación. Las ecuaciones que modelan el funcionamiento de los sistemas FVs con optimizadores de potencia se han desarrollado y utilizado para resaltar los límites de los mismos a la hora de resolver ciertas situaciones de mismatch. Se presenta un estudio profundo sobre el efecto de las sombras en los sistemas FVs: en la curva I-V y en los algoritmos MPPT. Se han llevado a cabo experimentos sobre el funcionamiento de los algoritmos MPPT en situaciones de sombreado, señalando su ineficiencia en estas situaciones. Un análisis de la ventaja del uso de DMPPT frente a los puntos calientes es presentado y verificado. También se presenta un análisis sobre las posibles ganancias en potencia y energía con el uso de DMPPT en condiciones de sombreado y este también es verificado experimentalmente, así como un breve estudio de su viabilidad económica. Para ayudar a llevar a cabo todos los análisis y experimentos descritos previamente se han desarrollado una serie de herramientas software. Una siendo un programa en LabView para controlar un simulador solar y almacenar las medidas. También se ha desarrollado un programa que simula curvas I-V de módulos y generador FVs afectados por sombras y este se ha verificado experimentalmente. Este mismo programa se ha utilizado para desarrollar un programa todavía más completo que estima las pérdidas anuales y las ganancias obtenidas con DMPPT en instalaciones FVs afectadas por sombras. Finalmente, se han desarrollado y verificado unos algoritmos para diagnosticar fallos en sistemas FVs con DMPPT. Esta herramienta puede diagnosticar los siguientes fallos: sombras debido a objetos fijos (con estimación de la distancia al objeto), suciedad localizada, suciedad general, posible punto caliente, degradación de módulos y pérdidas en el cableado de DC. Además, alerta al usuario de las pérdidas producidas por cada fallo y no requiere del uso de sensores de irradiancia y temperatura. ABSTRACT This work is a contribution to photovoltaic (PV) systems with distributed maximum power point tracking (DMPPT), a system topology characterized by performing the MPPT at module level, instead of the more traditional topologies which perform MPPT for a larger number of modules. The two DMPPT technologies available at the moment are known as microinverters and power optimizers, also known as module level power electronics (MLPE), and they provide certain advantages over central MPPT systems like: higher energy production in mismatch situations, monitoring of each individual module, system design flexibility, higher system safety, etc. Although DMPPT is not limited to urban environments, it has been emphasized in the title as it is their natural market, since in large ground-mounted PV plants the extra cost is difficult to justify. Since 2010 MLPE have increased their market share steadily and continuing to grow steadily. However, there still lacks a profound understanding of how they work, especially in the case of power optimizers, the achievable energy gains with their use and the possibilities in failure diagnosis. The main objective of this thesis is to provide a complete understanding of DMPPT technologies: how they function, their limitations and their advantages. A series of equations used to model PV arrays with power optimizers have been derived and used to point out limitations in solving certain mismatch situation. Because one of the most emphasized benefits of DMPPT is their ability to mitigate shading losses, an extensive study on the effects of shadows on PV systems is presented; both on the I-V curve and on MPPT algorithms. Experimental tests have been performed on the MPPT algorithms of central inverters and MLPE, highlighting their inefficiency in I-V curves with local maxima. An analysis of the possible mitigation of hot-spots with DMPPT is discussed and experimentally verified. And a theoretical analysis of the possible power and energy gains is presented as well as experiments in real PV systems. A short economic analysis of the benefits of DMPPT has also been performed. In order to aide in the previous task, a program which simulates I-V curves under shaded conditions has been developed and experimentally verified. This same program has been used to develop a software tool especially designed for PV systems affected by shading, which estimates the losses due to shading and the energy gains obtained with DMPPT. Finally, a set of algorithms for diagnosing system faults in PV systems with DMPPT has been developed and experimentally verified. The tool can diagnose the following failures: fixed object shading (with distance estimation), localized dirt, generalized dirt, possible hot-spots, module degradation and excessive losses in DC cables. In addition, it alerts the user of the power losses produced by each failure and classifies the failures by their severity and it does not require the use of irradiance or temperature sensors.

Distributed control strategies for photovoltaic systems in urban environments : Versión definitiva

Este trabajo es una contribución a los sistemas fotovoltaicos (FV) con seguimiento distribuido del punto de máxima potencia (DMPPT), una topología que se caracteriza porque lleva a cabo el MPPT a nivel de módulo, al contrario de las topologías más tradicionales que llevan a cabo el MPPT para un número más elevado de módulos, pudiendo ser hasta cientos de módulos. Las dos tecnologías DMPPT que existen en el mercado son conocidos como microinversores y optimizadores de potencia, y ofrecen ciertas ventajas sobre sistemas de MPPT central como: mayor producción en situaciones de mismatch, monitorización individual de cada módulo, flexibilidad de diseño, mayor seguridad del sistema, etc. Aunque los sistemas DMPPT no están limitados a los entornos urbanos, se ha enfatizado en el título ya que es su mercado natural, siendo difícil una justificación de su sobrecoste en grandes huertas solares en suelo. Desde el año 2010 el mercado de estos sistemas ha incrementado notablemente y sigue creciendo de una forma continuada. Sin embargo, todavía falta un conocimiento profundo de cómo funcionan estos sistemas, especialmente en el caso de los optimizadores de potencia, de las ganancias energéticas esperables en condiciones de mismatch y de las posibilidades avanzadas de diagnóstico de fallos. El principal objetivo de esta tesis es presentar un estudio completo de cómo funcionan los sistemas DMPPT, sus límites y sus ventajas, así como experimentos varios que verifican la teoría y el desarrollo de herramientas para valorar las ventajas de utilizar DMPPT en cada instalación. Las ecuaciones que modelan el funcionamiento de los sistemas FVs con optimizadores de potencia se han desarrollado y utilizado para resaltar los límites de los mismos a la hora de resolver ciertas situaciones de mismatch. Se presenta un estudio profundo sobre el efecto de las sombras en los sistemas FVs: en la curva I-V y en los algoritmos MPPT. Se han llevado a cabo experimentos sobre el funcionamiento de los algoritmos MPPT en situaciones de sombreado, señalando su ineficiencia en estas situaciones. Un análisis de la ventaja del uso de DMPPT frente a los puntos calientes es presentado y verificado. También se presenta un análisis sobre las posibles ganancias en potencia y energía con el uso de DMPPT en condiciones de sombreado y este también es verificado experimentalmente, así como un breve estudio de su viabilidad económica. Para ayudar a llevar a cabo todos los análisis y experimentos descritos previamente se han desarrollado una serie de herramientas software. Una siendo un programa en LabView para controlar un simulador solar y almacenar las medidas. También se ha desarrollado un programa que simula curvas I-V de módulos y generador FVs afectados por sombras y este se ha verificado experimentalmente. Este mismo programa se ha utilizado para desarrollar un programa todavía más completo que estima las pérdidas anuales y las ganancias obtenidas con DMPPT en instalaciones FVs afectadas por sombras. Finalmente, se han desarrollado y verificado unos algoritmos para diagnosticar fallos en sistemas FVs con DMPPT. Esta herramienta puede diagnosticar los siguientes fallos: sombras debido a objetos fijos (con estimación de la distancia al objeto), suciedad localizada, suciedad general, posible punto caliente, degradación de módulos y pérdidas en el cableado de DC. Además, alerta al usuario de las pérdidas producidas por cada fallo y no requiere del uso de sensores de irradiancia y temperatura. ABSTRACT This work is a contribution to photovoltaic (PV) systems with distributed maximum power point tracking (DMPPT), a system topology characterized by performing the MPPT at module level, instead of the more traditional topologies which perform MPPT for a larger number of modules. The two DMPPT technologies available at the moment are known as microinverters and power optimizers, also known as module level power electronics (MLPE), and they provide certain advantages over central MPPT systems like: higher energy production in mismatch situations, monitoring of each individual module, system design flexibility, higher system safety, etc. Although DMPPT is not limited to urban environments, it has been emphasized in the title as it is their natural market, since in large ground-mounted PV plants the extra cost is difficult to justify. Since 2010 MLPE have increased their market share steadily and continuing to grow steadily. However, there still lacks a profound understanding of how they work, especially in the case of power optimizers, the achievable energy gains with their use and the possibilities in failure diagnosis. The main objective of this thesis is to provide a complete understanding of DMPPT technologies: how they function, their limitations and their advantages. A series of equations used to model PV arrays with power optimizers have been derived and used to point out limitations in solving certain mismatch situation. Because one of the most emphasized benefits of DMPPT is their ability to mitigate shading losses, an extensive study on the effects of shadows on PV systems is presented; both on the I-V curve and on MPPT algorithms. Experimental tests have been performed on the MPPT algorithms of central inverters and MLPE, highlighting their inefficiency in I-V curves with local maxima. An analysis of the possible mitigation of hot-spots with DMPPT is discussed and experimentally verified. And a theoretical analysis of the possible power and energy gains is presented as well as experiments in real PV systems. A short economic analysis of the benefits of DMPPT has also been performed. In order to aide in the previous task, a program which simulates I-V curves under shaded conditions has been developed and experimentally verified. This same program has been used to develop a software tool especially designed for PV systems affected by shading, which estimates the losses due to shading and the energy gains obtained with DMPPT. Finally, a set of algorithms for diagnosing system faults in PV systems with DMPPT has been developed and experimentally verified. The tool can diagnose the following failures: fixed object shading (with distance estimation), localized dirt, generalized dirt, possible hot-spots, module degradation and excessive losses in DC cables. In addition, it alerts the user of the power losses produced by each failure and classifies the failures by their severity and it does not require the use of irradiance or temperature sensors.

Memory for object features versus memory for object location: a positron-emission tomography study of encoding and retrieval processes.

Regional cerebral blood flow was measured with positron-emission tomography during two encoding and two retrieval tasks that were designed to compare memory for object features with memory for object locations. Bilateral increases in regional cerebral blood flow were observed in both anterior and posterior regions of inferior temporal cortex and in ventral regions of prestriate cortex, when the condition that required retrieval of object locations was subtracted from the condition that required retrieval of object features. During encoding, these changes were less pronounced and were restricted to the left inferior temporal cortex and right ventral prestriate cortex. In contrast, both encoding and retrieval of object location were associated with bilateral changes in dorsal prestriate and posterior parietal cortex. Finally, the two encoding conditions activated left frontal lobe regions preferentially, whereas the two retrieval conditions activated right frontal lobe regions. These findings confirm that, in human subjects, memory for object features is mediated by a distributed system that includes ventral prestriate cortex and both anterior and posterior regions of the inferior temporal gyrus. In contrast, memory for the locations of objects appears to be mediated by an anatomically distinct system that includes more dorsal regions of prestriate cortex and posterior regions of the parietal lobe.

3D Visual Data-Driven Spatiotemporal Deformations for Non-Rigid Object Grasping Using Robot Hands

Sensing techniques are important for solving problems of uncertainty inherent to intelligent grasping tasks. The main goal here is to present a visual sensing system based on range imaging technology for robot manipulation of non-rigid objects. Our proposal provides a suitable visual perception system of complex grasping tasks to support a robot controller when other sensor systems, such as tactile and force, are not able to obtain useful data relevant to the grasping manipulation task. In particular, a new visual approach based on RGBD data was implemented to help a robot controller carry out intelligent manipulation tasks with flexible objects. The proposed method supervises the interaction between the grasped object and the robot hand in order to avoid poor contact between the fingertips and an object when there is neither force nor pressure data. This new approach is also used to measure changes to the shape of an object’s surfaces and so allows us to find deformations caused by inappropriate pressure being applied by the hand’s fingers. Test was carried out for grasping tasks involving several flexible household objects with a multi-fingered robot hand working in real time. Our approach generates pulses from the deformation detection method and sends an event message to the robot controller when surface deformation is detected. In comparison with other methods, the obtained results reveal that our visual pipeline does not use deformations models of objects and materials, as well as the approach works well both planar and 3D household objects in real time. In addition, our method does not depend on the pose of the robot hand because the location of the reference system is computed from a recognition process of a pattern located place at the robot forearm. The presented experiments demonstrate that the proposed method accomplishes a good monitoring of grasping task with several objects and different grasping configurations in indoor environments.