Software for "Realistic limits for subpixel movement detection"

Object tracking with subpixel accuracy is of fundamental importance in many fields since it provides optimal performance at relatively low-cost. Although there are many theoretical proposals that lead to resolution increments of several orders of magnitude, in practice, this resolution is limited by the imaging systems. In this paper we propose and demonstrate through numerical models a realistic limit for subpixel accuracy. The final result is that maximum achievable resolution enhancement is connected with the dynamic range of the image, i.e. the detection limit is 1/2^(nr.bits). Results here presented may help to proper design of superresolution experiments in microscopy, surveillance, defense and other fields.

Realistic limits for subpixel movement detection

Object tracking with subpixel accuracy is of fundamental importance in many fields since it provides optimal performance at relatively low cost. Although there are many theoretical proposals that lead to resolution increments of several orders of magnitude, in practice this resolution is limited by the imaging systems. In this paper we propose and demonstrate through simple numerical models a realistic limit for subpixel accuracy. The final result is that maximum achievable resolution enhancement is connected with the dynamic range of the image, i.e., the detection limit is 1/2∧(nr.bits). The results here presented may aid in proper design of superresolution experiments in microscopy, surveillance, defense, and other fields.

Using psilocybin to investigate the relationship between attention, working memory, and the serotonin 1A and 2A receptors

Increasing evidence suggests a link between attention, working memory, serotonin (5-HT), and prefrontal cortex activity. In an attempt to tease out the relationship between these elements, this study tested the effects of the hallucinogenic mixed 5-HT1A/2A receptor agonist psilocybin alone and after pretreatment with the 5-HT2A antagonist ketanserin. Eight healthy human volunteers were rested on a multiple-object tracking task and spatial working memory task under the four conditions: placebo, psilocybin (215 mu g/kg), ketanserin (50 mg), and psilocybin and ketanserin. Psilocybin significantly reduced attentional tracking ability, but had no significant effect on spatial working memory, suggesting a functional dissociation between the two tasks. Pretreatment with ketanserin did not attenuate the effect of psilocybin on attentional performance, suggestinga primary involvement of the 5-HT1A receptor in the observed defecit. Based on physiological and pharmacological data,we speculate that this impaired attentional performance may reflect a reduced ability to suppress or ignore distracting stimuli rather than reduced attentional capacity. The clinical relevance of these results is also discussed.

Socio-economic vision graph generation and handover in distributed smart camera networks

In this article we present an approach to object tracking handover in a network of smart cameras, based on self-interested autonomous agents, which exchange responsibility for tracking objects in a market mechanism, in order to maximise their own utility. A novel ant-colony inspired mechanism is used to learn the vision graph, that is, the camera neighbourhood relations, during runtime, which may then be used to optimise communication between cameras. The key benefits of our completely decentralised approach are on the one hand generating the vision graph online, enabling efficient deployment in unknown scenarios and camera network topologies, and on the other hand relying only on local information, increasing the robustness of the system. Since our market-based approach does not rely on a priori topology information, the need for any multicamera calibration can be avoided. We have evaluated our approach both in a simulation study and in network of real distributed smart cameras.

Improved adaptivity and robustness in decentralised multi-camera networks

In this paper we present increased adaptivity and robustness in distributed object tracking by multi-camera networks using a socio-economic mechanism for learning the vision graph. To build-up the vision graph autonomously within a distributed smart-camera network, we use an ant-colony inspired mechanism, which exchanges responsibility for tracking objects using Vickrey auctions. Employing the learnt vision graph allows the system to optimise its communication continuously. Since distributed smart camera networks are prone to uncertainties in individual cameras, such as failures or changes in extrinsic parameters, the vision graph should be sufficiently robust and adaptable during runtime to enable seamless tracking and optimised communication. To better reflect real smart-camera platforms and networks, we consider that communication and handover are not instantaneous, and that cameras may be added, removed or their properties changed during runtime. Using our dynamic socio-economic approach, the network is able to continue tracking objects well, despite all these uncertainties, and in some cases even with improved performance. This demonstrates the adaptivity and robustness of our approach.

Learning to be different:heterogeneity and efficiency in distributed smart camera networks

In this paper we study the self-organising behaviour of smart camera networks which use market-based handover of object tracking responsibilities to achieve an efficient allocation of objects to cameras. Specifically, we compare previously known homogeneous configurations, when all cameras use the same marketing strategy, with heterogeneous configurations, when each camera makes use of its own, possibly different marketing strategy. Our first contribution is to establish that such heterogeneity of marketing strategies can lead to system wide outcomes which are Pareto superior when compared to those possible in homogeneous configurations. However, since the particular configuration required to lead to Pareto efficiency in a given scenario will not be known in advance, our second contribution is to show how online learning of marketing strategies at the individual camera level can lead to high performing heterogeneous configurations from the system point of view, extending the Pareto front when compared to the homogeneous case. Our third contribution is to show that in many cases, the dynamic behaviour resulting from online learning leads to global outcomes which extend the Pareto front even when compared to static heterogeneous configurations. Our evaluation considers results obtained from an open source simulation package as well as data from a network of real cameras. © 2013 IEEE.

Self-organising zooms for decentralised redundancy management in visual sensor networks

When visual sensor networks are composed of cameras which can adjust the zoom factor of their own lens, one must determine the optimal zoom levels for the cameras, for a given task. This gives rise to an important trade-off between the overlap of the different cameras’ fields of view, providing redundancy, and image quality. In an object tracking task, having multiple cameras observe the same area allows for quicker recovery, when a camera fails. In contrast having narrow zooms allow for a higher pixel count on regions of interest, leading to increased tracking confidence. In this paper we propose an approach for the self-organisation of redundancy in a distributed visual sensor network, based on decentralised multi-objective online learning using only local information to approximate the global state. We explore the impact of different zoom levels on these trade-offs, when tasking omnidirectional cameras, having perfect 360-degree view, with keeping track of a varying number of moving objects. We further show how employing decentralised reinforcement learning enables zoom configurations to be achieved dynamically at runtime according to an operator’s preference for maximising either the proportion of objects tracked, confidence associated with tracking, or redundancy in expectation of camera failure. We show that explicitly taking account of the level of overlap, even based only on local knowledge, improves resilience when cameras fail. Our results illustrate the trade-off between maintaining high confidence and object coverage, and maintaining redundancy, in anticipation of future failure. Our approach provides a fully tunable decentralised method for the self-organisation of redundancy in a changing environment, according to an operator’s preferences.

A multimedia indexing and retrieval framework for multimedia database systems

The main challenges of multimedia data retrieval lie in the effective mapping between low-level features and high-level concepts, and in the individual users' subjective perceptions of multimedia content. ^ The objectives of this dissertation are to develop an integrated multimedia indexing and retrieval framework with the aim to bridge the gap between semantic concepts and low-level features. To achieve this goal, a set of core techniques have been developed, including image segmentation, content-based image retrieval, object tracking, video indexing, and video event detection. These core techniques are integrated in a systematic way to enable the semantic search for images/videos, and can be tailored to solve the problems in other multimedia related domains. In image retrieval, two new methods of bridging the semantic gap are proposed: (1) for general content-based image retrieval, a stochastic mechanism is utilized to enable the long-term learning of high-level concepts from a set of training data, such as user access frequencies and access patterns of images. (2) In addition to whole-image retrieval, a novel multiple instance learning framework is proposed for object-based image retrieval, by which a user is allowed to more effectively search for images that contain multiple objects of interest. An enhanced image segmentation algorithm is developed to extract the object information from images. This segmentation algorithm is further used in video indexing and retrieval, by which a robust video shot/scene segmentation method is developed based on low-level visual feature comparison, object tracking, and audio analysis. Based on shot boundaries, a novel data mining framework is further proposed to detect events in soccer videos, while fully utilizing the multi-modality features and object information obtained through video shot/scene detection. ^ Another contribution of this dissertation is the potential of the above techniques to be tailored and applied to other multimedia applications. This is demonstrated by their utilization in traffic video surveillance applications. The enhanced image segmentation algorithm, coupled with an adaptive background learning algorithm, improves the performance of vehicle identification. A sophisticated object tracking algorithm is proposed to track individual vehicles, while the spatial and temporal relationships of vehicle objects are modeled by an abstract semantic model. ^

An incremental multilinear system for human face learning and recognition

This dissertation establishes a novel system for human face learning and recognition based on incremental multilinear Principal Component Analysis (PCA). Most of the existing face recognition systems need training data during the learning process. The system as proposed in this dissertation utilizes an unsupervised or weakly supervised learning approach, in which the learning phase requires a minimal amount of training data. It also overcomes the inability of traditional systems to adapt to the testing phase as the decision process for the newly acquired images continues to rely on that same old training data set. Consequently when a new training set is to be used, the traditional approach will require that the entire eigensystem will have to be generated again. However, as a means to speed up this computational process, the proposed method uses the eigensystem generated from the old training set together with the new images to generate more effectively the new eigensystem in a so-called incremental learning process. In the empirical evaluation phase, there are two key factors that are essential in evaluating the performance of the proposed method: (1) recognition accuracy and (2) computational complexity. In order to establish the most suitable algorithm for this research, a comparative analysis of the best performing methods has been carried out first. The results of the comparative analysis advocated for the initial utilization of the multilinear PCA in our research. As for the consideration of the issue of computational complexity for the subspace update procedure, a novel incremental algorithm, which combines the traditional sequential Karhunen-Loeve (SKL) algorithm with the newly developed incremental modified fast PCA algorithm, was established. In order to utilize the multilinear PCA in the incremental process, a new unfolding method was developed to affix the newly added data at the end of the previous data. The results of the incremental process based on these two methods were obtained to bear out these new theoretical improvements. Some object tracking results using video images are also provided as another challenging task to prove the soundness of this incremental multilinear learning method.

Expertise sportive et entraînement perceptivo-cognitif de l’athlète

Pour être performant au plus haut niveau, les athlètes doivent posséder une capacité perceptivo-cognitive supérieure à la moyenne. Cette faculté, reflétée sur le terrain par la vision et l’intelligence de jeu des sportifs, permet d’extraire l’information clé de la scène visuelle. La science du sport a depuis longtemps observé l’expertise perceptivo-cognitive au sein de l’environnement sportif propre aux athlètes. Récemment, des études ont rapporté que l’expertise pouvait également se refléter hors de ce contexte, lors d’activités du quotidien par exemple. De plus, les récentes théories entourant la capacité plastique du cerveau ont amené les chercheurs à développer des outils pour entraîner les capacités perceptivo-cognitives des athlètes afin de les rendre plus performants sur le terrain. Ces méthodes sont la plupart du temps contextuelles à la discipline visée. Cependant, un nouvel outil d’entraînement perceptivo-cognitif, nommé 3-Dimensional Multiple Object Tracking (3D-MOT) et dénué de contexte sportif, a récemment vu le jour et a fait l’objet de nos recherches. Un de nos objectifs visait à mettre en évidence l’expertise perceptivo-cognitive spécifique et non-spécifique chez des athlètes lors d’une même étude. Nous avons évalué la perception du mouvement biologique chez des joueurs de soccer et des non-athlètes dans une salle de réalité virtuelle. Les sportifs étaient systématiquement plus performants en termes d’efficacité et de temps de réaction que les novices pour discriminer la direction du mouvement biologique lors d’un exercice spécifique de soccer (tir) mais également lors d’une action issue du quotidien (marche). Ces résultats signifient que les athlètes possèdent une meilleure capacité à percevoir les mouvements biologiques humains effectués par les autres. La pratique du soccer semble donc conférer un avantage fondamental qui va au-delà des fonctions spécifiques à la pratique d’un sport. Ces découvertes sont à mettre en parallèle avec la performance exceptionnelle des athlètes dans le traitement de scènes visuelles dynamiques et également dénuées de contexte sportif. Des joueurs de soccer ont surpassé des novices dans le test de 3D-MOT qui consiste à suivre des cibles en mouvement et stimule les capacités perceptivo-cognitives. Leur vitesse de suivi visuel ainsi que leur faculté d’apprentissage étaient supérieures. Ces résultats confirmaient des données obtenues précédemment chez des sportifs. Le 3D-MOT est un test de poursuite attentionnelle qui stimule le traitement actif de l’information visuelle dynamique. En particulier, l’attention sélective, dynamique et soutenue ainsi que la mémoire de travail. Cet outil peut être utilisé pour entraîner les fonctions perceptivo-cognitives des athlètes. Des joueurs de soccer entraînés au 3D-MOT durant 30 sessions ont montré une amélioration de la prise de décision dans les passes de 15% sur le terrain comparés à des joueurs de groupes contrôles. Ces données démontrent pour la première fois un transfert perceptivo-cognitif du laboratoire au terrain suivant un entraînement perceptivo-cognitif non-contextuel au sport de l’athlète ciblé. Nos recherches aident à comprendre l’expertise des athlètes par l’approche spécifique et non-spécifique et présentent également les outils d’entraînements perceptivo-cognitifs, en particulier le 3D-MOT, pour améliorer la performance dans le sport de haut-niveau.

Expertise sportive et entraînement perceptivo-cognitif de l’athlète

Pour être performant au plus haut niveau, les athlètes doivent posséder une capacité perceptivo-cognitive supérieure à la moyenne. Cette faculté, reflétée sur le terrain par la vision et l’intelligence de jeu des sportifs, permet d’extraire l’information clé de la scène visuelle. La science du sport a depuis longtemps observé l’expertise perceptivo-cognitive au sein de l’environnement sportif propre aux athlètes. Récemment, des études ont rapporté que l’expertise pouvait également se refléter hors de ce contexte, lors d’activités du quotidien par exemple. De plus, les récentes théories entourant la capacité plastique du cerveau ont amené les chercheurs à développer des outils pour entraîner les capacités perceptivo-cognitives des athlètes afin de les rendre plus performants sur le terrain. Ces méthodes sont la plupart du temps contextuelles à la discipline visée. Cependant, un nouvel outil d’entraînement perceptivo-cognitif, nommé 3-Dimensional Multiple Object Tracking (3D-MOT) et dénué de contexte sportif, a récemment vu le jour et a fait l’objet de nos recherches. Un de nos objectifs visait à mettre en évidence l’expertise perceptivo-cognitive spécifique et non-spécifique chez des athlètes lors d’une même étude. Nous avons évalué la perception du mouvement biologique chez des joueurs de soccer et des non-athlètes dans une salle de réalité virtuelle. Les sportifs étaient systématiquement plus performants en termes d’efficacité et de temps de réaction que les novices pour discriminer la direction du mouvement biologique lors d’un exercice spécifique de soccer (tir) mais également lors d’une action issue du quotidien (marche). Ces résultats signifient que les athlètes possèdent une meilleure capacité à percevoir les mouvements biologiques humains effectués par les autres. La pratique du soccer semble donc conférer un avantage fondamental qui va au-delà des fonctions spécifiques à la pratique d’un sport. Ces découvertes sont à mettre en parallèle avec la performance exceptionnelle des athlètes dans le traitement de scènes visuelles dynamiques et également dénuées de contexte sportif. Des joueurs de soccer ont surpassé des novices dans le test de 3D-MOT qui consiste à suivre des cibles en mouvement et stimule les capacités perceptivo-cognitives. Leur vitesse de suivi visuel ainsi que leur faculté d’apprentissage étaient supérieures. Ces résultats confirmaient des données obtenues précédemment chez des sportifs. Le 3D-MOT est un test de poursuite attentionnelle qui stimule le traitement actif de l’information visuelle dynamique. En particulier, l’attention sélective, dynamique et soutenue ainsi que la mémoire de travail. Cet outil peut être utilisé pour entraîner les fonctions perceptivo-cognitives des athlètes. Des joueurs de soccer entraînés au 3D-MOT durant 30 sessions ont montré une amélioration de la prise de décision dans les passes de 15% sur le terrain comparés à des joueurs de groupes contrôles. Ces données démontrent pour la première fois un transfert perceptivo-cognitif du laboratoire au terrain suivant un entraînement perceptivo-cognitif non-contextuel au sport de l’athlète ciblé. Nos recherches aident à comprendre l’expertise des athlètes par l’approche spécifique et non-spécifique et présentent également les outils d’entraînements perceptivo-cognitifs, en particulier le 3D-MOT, pour améliorer la performance dans le sport de haut-niveau.

Computational Foundations for Safe and Efficient Human-Robot Collaboration in Assembly Cells

Human and robots have complementary strengths in performing assembly operations. Humans are very good at perception tasks in unstructured environments. They are able to recognize and locate a part from a box of miscellaneous parts. They are also very good at complex manipulation in tight spaces. The sensory characteristics of the humans, motor abilities, knowledge and skills give the humans the ability to react to unexpected situations and resolve problems quickly. In contrast, robots are very good at pick and place operations and highly repeatable in placement tasks. Robots can perform tasks at high speeds and still maintain precision in their operations. Robots can also operate for long periods of times. Robots are also very good at applying high forces and torques. Typically, robots are used in mass production. Small batch and custom production operations predominantly use manual labor. The high labor cost is making it difficult for small and medium manufacturers to remain cost competitive in high wage markets. These manufactures are mainly involved in small batch and custom production. They need to find a way to reduce the labor cost in assembly operations. Purely robotic cells will not be able to provide them the necessary flexibility. Creating hybrid cells where humans and robots can collaborate in close physical proximities is a potential solution. The underlying idea behind such cells is to decompose assembly operations into tasks such that humans and robots can collaborate by performing sub-tasks that are suitable for them. Realizing hybrid cells that enable effective human and robot collaboration is challenging. This dissertation addresses the following three computational issues involved in developing and utilizing hybrid assembly cells: - We should be able to automatically generate plans to operate hybrid assembly cells to ensure efficient cell operation. This requires generating feasible assembly sequences and instructions for robots and human operators, respectively. Automated planning poses the following two challenges. First, generating operation plans for complex assemblies is challenging. The complexity can come due to the combinatorial explosion caused by the size of the assembly or the complex paths needed to perform the assembly. Second, generating feasible plans requires accounting for robot and human motion constraints. The first objective of the dissertation is to develop the underlying computational foundations for automatically generating plans for the operation of hybrid cells. It addresses both assembly complexity and motion constraints issues. - The collaboration between humans and robots in the assembly cell will only be practical if human safety can be ensured during the assembly tasks that require collaboration between humans and robots. The second objective of the dissertation is to evaluate different options for real-time monitoring of the state of human operator with respect to the robot and develop strategies for taking appropriate measures to ensure human safety when the planned move by the robot may compromise the safety of the human operator. In order to be competitive in the market, the developed solution will have to include considerations about cost without significantly compromising quality. - In the envisioned hybrid cell, we will be relying on human operators to bring the part into the cell. If the human operator makes an error in selecting the part or fails to place it correctly, the robot will be unable to correctly perform the task assigned to it. If the error goes undetected, it can lead to a defective product and inefficiencies in the cell operation. The reason for human error can be either confusion due to poor quality instructions or human operator not paying adequate attention to the instructions. In order to ensure smooth and error-free operation of the cell, we will need to monitor the state of the assembly operations in the cell. The third objective of the dissertation is to identify and track parts in the cell and automatically generate instructions for taking corrective actions if a human operator deviates from the selected plan. Potential corrective actions may involve re-planning if it is possible to continue assembly from the current state. Corrective actions may also involve issuing warning and generating instructions to undo the current task.

The neuroscience of cognitive enhancement : enhanced attention, working memory and visual information processing speed using 3D-MOT

Des interventions ciblant l’amélioration cognitive sont de plus en plus à l’intérêt dans nombreux domaines, y compris la neuropsychologie. Bien qu'il existe de nombreuses méthodes pour maximiser le potentiel cognitif de quelqu’un, ils sont rarement appuyé par la recherche scientifique. D’abord, ce mémoire examine brièvement l'état des interventions d'amélioration cognitives. Il décrit premièrement les faiblesses observées dans ces pratiques et par conséquent il établit un modèle standard contre lequel on pourrait et devrait évaluer les diverses techniques ciblant l'amélioration cognitive. Une étude de recherche est ensuite présenté qui considère un nouvel outil de l'amélioration cognitive, une tâche d’entrainement perceptivo-cognitive : 3-dimensional multiple object tracking (3D-MOT). Il examine les preuves actuelles pour le 3D-MOT auprès du modèle standard proposé. Les résultats de ce projet démontrent de l’augmentation dans les capacités d’attention, de mémoire de travail visuel et de vitesse de traitement d’information. Cette étude représente la première étape dans la démarche vers l’établissement du 3D-MOT comme un outil d’amélioration cognitive.

The neuroscience of cognitive enhancement : enhanced attention, working memory and visual information processing speed using 3D-MOT

Des interventions ciblant l’amélioration cognitive sont de plus en plus à l’intérêt dans nombreux domaines, y compris la neuropsychologie. Bien qu'il existe de nombreuses méthodes pour maximiser le potentiel cognitif de quelqu’un, ils sont rarement appuyé par la recherche scientifique. D’abord, ce mémoire examine brièvement l'état des interventions d'amélioration cognitives. Il décrit premièrement les faiblesses observées dans ces pratiques et par conséquent il établit un modèle standard contre lequel on pourrait et devrait évaluer les diverses techniques ciblant l'amélioration cognitive. Une étude de recherche est ensuite présenté qui considère un nouvel outil de l'amélioration cognitive, une tâche d’entrainement perceptivo-cognitive : 3-dimensional multiple object tracking (3D-MOT). Il examine les preuves actuelles pour le 3D-MOT auprès du modèle standard proposé. Les résultats de ce projet démontrent de l’augmentation dans les capacités d’attention, de mémoire de travail visuel et de vitesse de traitement d’information. Cette étude représente la première étape dans la démarche vers l’établissement du 3D-MOT comme un outil d’amélioration cognitive.

On the motion planning & control of nonlinear robotic systems

In the last decades, we saw a soaring interest in autonomous robots boosted not only by academia and industry, but also by the ever in- creasing demand from civil users. As a matter of fact, autonomous robots are fast spreading in all aspects of human life, we can see them clean houses, navigate through city traffic, or harvest fruits and vegetables. Almost all commercial drones already exhibit unprecedented and sophisticated skills which makes them suitable for these applications, such as obstacle avoidance, simultaneous localisation and mapping, path planning, visual-inertial odometry, and object tracking. The major limitations of such robotic platforms lie in the limited payload that can carry, in their costs, and in the limited autonomy due to finite battery capability. For this reason researchers start to develop new algorithms able to run even on resource constrained platforms both in terms of computation capabilities and limited types of endowed sensors, focusing especially on very cheap sensors and hardware. The possibility to use a limited number of sensors allowed to scale a lot the UAVs size, while the implementation of new efficient algorithms, performing the same task in lower time, allows for lower autonomy. However, the developed robots are not mature enough to completely operate autonomously without human supervision due to still too big dimensions (especially for aerial vehicles), which make these platforms unsafe for humans, and the high probability of numerical, and decision, errors that robots may make. In this perspective, this thesis aims to review and improve the current state-of-the-art solutions for autonomous navigation from a purely practical point of view. In particular, we deeply focused on the problems of robot control, trajectory planning, environments exploration, and obstacle avoidance.