Real time motion estimation using a neural architecture implemented on GPUs

This work describes a neural network based architecture that represents and estimates object motion in videos. This architecture addresses multiple computer vision tasks such as image segmentation, object representation or characterization, motion analysis and tracking. The use of a neural network architecture allows for the simultaneous estimation of global and local motion and the representation of deformable objects. This architecture also avoids the problem of finding corresponding features while tracking moving objects. Due to the parallel nature of neural networks, the architecture has been implemented on GPUs that allows the system to meet a set of requirements such as: time constraints management, robustness, high processing speed and re-configurability. Experiments are presented that demonstrate the validity of our architecture to solve problems of mobile agents tracking and motion analysis.

Active Visual Features Based on Events to Guide Robot Manipulators in Tracking Tasks

Traditional visual servoing systems do not deal with the topic of moving objects tracking. When these systems are employed to track a moving object, depending on the object velocity, visual features can go out of the image, causing the fail of the tracking task. This occurs specially when the object and the robot are both stopped and then the object starts the movement. In this work, we have employed a retina camera based on Address Event Representation (AER) in order to use events as input in the visual servoing system. The events launched by the camera indicate a pixel movement. Event visual information is processed only at the moment it occurs, reducing the response time of visual servoing systems when they are used to track moving objects.

Self-organising zooms for decentralised redundancy management in visual sensor networks

When visual sensor networks are composed of cameras which can adjust the zoom factor of their own lens, one must determine the optimal zoom levels for the cameras, for a given task. This gives rise to an important trade-off between the overlap of the different cameras’ fields of view, providing redundancy, and image quality. In an object tracking task, having multiple cameras observe the same area allows for quicker recovery, when a camera fails. In contrast having narrow zooms allow for a higher pixel count on regions of interest, leading to increased tracking confidence. In this paper we propose an approach for the self-organisation of redundancy in a distributed visual sensor network, based on decentralised multi-objective online learning using only local information to approximate the global state. We explore the impact of different zoom levels on these trade-offs, when tasking omnidirectional cameras, having perfect 360-degree view, with keeping track of a varying number of moving objects. We further show how employing decentralised reinforcement learning enables zoom configurations to be achieved dynamically at runtime according to an operator’s preference for maximising either the proportion of objects tracked, confidence associated with tracking, or redundancy in expectation of camera failure. We show that explicitly taking account of the level of overlap, even based only on local knowledge, improves resilience when cameras fail. Our results illustrate the trade-off between maintaining high confidence and object coverage, and maintaining redundancy, in anticipation of future failure. Our approach provides a fully tunable decentralised method for the self-organisation of redundancy in a changing environment, according to an operator’s preferences.

Controle de mobilidade infraestruturado e orientado à qualidade para maximização de admissões em sistemas WiNeMO: uma abordagem definida por software

The spread of wireless networks and growing proliferation of mobile devices require the development of mobility control mechanisms to support the different demands of traffic in different network conditions. A major obstacle to developing this kind of technology is the complexity involved in handling all the information about the large number of Moving Objects (MO), as well as the entire signaling overhead required to manage these procedures in the network. Despite several initiatives have been proposed by the scientific community to address this issue they have not proved to be effective since they depend on the particular request of the MO that is responsible for triggering the mobility process. Moreover, they are often only guided by wireless medium statistics, such as Received Signal Strength Indicator (RSSI) of the candidate Point of Attachment (PoA). Thus, this work seeks to develop, evaluate and validate a sophisticated communication infrastructure for Wireless Networking for Moving Objects (WiNeMO) systems by making use of the flexibility provided by the Software-Defined Networking (SDN) paradigm, where network functions are easily and efficiently deployed by integrating OpenFlow and IEEE 802.21 standards. For purposes of benchmarking, the analysis was conducted in the control and data planes aspects, which demonstrate that the proposal significantly outperforms typical IPbased SDN and QoS-enabled capabilities, by allowing the network to handle the multimedia traffic with optimal Quality of Service (QoS) transport and acceptable Quality of Experience (QoE) over time.

Practical 3D reconstruction based on photometric stereo

Photometric Stereo is a powerful image based 3D reconstruction technique that has recently been used to obtain very high quality reconstructions. However, in its classic form, Photometric Stereo suffers from two main limitations: Firstly, one needs to obtain images of the 3D scene under multiple different illuminations. As a result the 3D scene needs to remain static during illumination changes, which prohibits the reconstruction of deforming objects. Secondly, the images obtained must be from a single viewpoint. This leads to depth-map based 2.5 reconstructions, instead of full 3D surfaces. The aim of this Chapter is to show how these limitations can be alleviated, leading to the derivation of two practical 3D acquisition systems: The first one, based on the powerful Coloured Light Photometric Stereo method can be used to reconstruct moving objects such as cloth or human faces. The second, permits the complete 3D reconstruction of challenging objects such as porcelain vases. In addition to algorithmic details, the Chapter pays attention to practical issues such as setup calibration, detection and correction of self and cast shadows. We provide several evaluation experiments as well as reconstruction results. © 2010 Springer-Verlag Berlin Heidelberg.

Spic'ing into Existence: Epitaph, Epithet, and the Ethnopoetic Imagination

Quién Es, Quién Somos? Spic’ing into Existence claims a four-fold close-reading: first, analysis of texts: from theoretical meditations to (prison) memoir and film. Second, a half dozen central figures appear, largely Latinx and black American. They cut across a score of registers, socio-economics, ideological reservations, but all are, as Carl Carlton sang, poetry in motion. Writers, poets, theologians, pathologists, artists, comedians, actors, students whose vocation is invocation, the inner surge of their calling. Third, the manuscript draws from a series of historical moments—from radical liberation of the late 60s, to contemporary student activism. Finally, this body of work is movement, in all its social, gestural, and kinesthetic viscera. From this last heading, we peel away layers of what I call the ethnopoet, the fascia undoing that reveals its bio-political anatomy, dressing its bare life with kinship speech. First, the social revolutions of the Civil Rights, Black Power, abolitionism, the Black Panthers and Young Lords, boycotts and jarring artistic performances. These events are superficial not in vain sense, but key epicenters of underground murmurings, the workings of a cunning assailant. She robs not lavish estates, but another day to breathe. Gesturally, as perhaps the interlocutor, lies this author, interspersing his own diatribes to conjure her presence. The final branch is admittedly the most intangible. Kinesthetically, we map the nimbleness, footwork lígera of what I call the ethnopoet. Ethnopoet is no mere aggregate of ethnicity and poetry, but like chemical reaction, the descriptor for its behavior under certain pressures, temperatures, and elements. Elusive and resisting confinement, and therefore definition, the ethnopoet is a shapeshifting figure of how racialized bodies [people of color] respond to hegemonic powers. She is, at bottom, however, a native translator, the plural-lensed subject whose loyalty is only to the imagination of a different world, one whose survival is not contingent upon her exploitation. The native translator’s constant re-calibrations of oppressive power apparatuses seem taxing at best, and near-impossible, at worst. To effectively navigate through these polarized loci, she must identify ideologies that in turn seek “affective liberatory sances” in relation to the dominant social order (43). In a kind of performative contradiction, she must marshall the knowledge necessary to “break with ideology” while speaking within it. Chicana Studies scholar, Chela Sandoval, describes this dual movement as “meta-ideologizing”: the appropriation of hegemonic ideological forms in order to transform them (82). Nuestros padres se subieron encima de La Bestia, y por eso somos pasageros a ese tren. Y ya, dentro su pansa, tenemos que ser vigilantes cuando plantamos las bombas. In Methodology of the Oppressed, Sandoval schematizes this oppositional consciousness around five principle categories: “equal rights,” “revolutionary,” “supremacist,” “separatist,” and “differential.” Taken by themselves, the first four modes appear mutually exclusive, incapable of occupying the same plane, until a fifth pillar emerges. Cinematographic in nature, differential consciousness, as Sandoval defines it, is “a kinetic motion that maneuvers, poetically transfigures, and orchestrates while demanding alienation, perversion, and reformation in both spectators and practitioners” (44). For Sandoval, then, differential consciousness is a methodology that privileges an incredible sense mobility, one reaching artistic sensibilities. Our fourth and final analytic of movement serves an apt example of this dual meaning. Lexically speaking, ‘movement’ may be regarded as a political mobilization of aggrieved populations (through sustained efforts), or the process of moving objects (people or otherwise) from one location to another. Praxis-wise, it is both action and ideal, content and form. Thus, an ethnic poetics must be regarded less as a series of stanzas, shortened lyric, or even arrangement of language, but as a lens through which peripheralized peoples kaleidecope ideological positions in an “original, eccentric, and queer sight” (43). Taking note of the advantages of postponing identifications, the thesis stands its ground on the term ethnopoet. Its abstraction is not dewey-eyed philosophy, but an anticipation of poetic justice, of what’s to come from callused hands. This thesis is divided into 7.5 chapters. The first maps out the ethnopoet’s cartographies of struggle. By revisiting that alleged Tío Tomas, Richard Rodriguez, we unearth the tensions that negatively, deny citizenship to one silo, but on the flipside, engender manifold ways of seeing, hearing, and moving . The second, through George Jackson’s prison memoirs, pans out from this ethnography of power, groping for an apparatus that feigns an impervious prestige: ‘the aesthetic regime of coercion.’ In half-way cut, the thesis sidesteps to spic into existence, formally announcing, through Aime Cesaire, myself, and Pedro Pietri, the poeticization of trauma. Such uplift denies New Age transcendence of self, but a rehearsal of our entrapment in these mortal envelopes. Thirdly, conscious of the bleeding ethnic body, we cut open the incipient corpse to observe her pathologist. Her native autopsies offer the ethnic body’s posthumous recognition, the ethnopoetics ability to speak for and through the dead. Chapter five examines prolific black artists—Beyonce and Kendrick Lamar—to elide the circumvention of their consumption via invoking radical black hi/her-stories, ones fragmenting the black body. Sixth, the paper compares the Black Power Salute of the 1968 Mexico City Olympics to Duke’s Mi Gente Boycott of their Latino Student Recruitment Weekend. Both wielded “silent gestures,” that shrewdly interfered with white noise of numbed negligence. Finally, ‘taking the mask off’ that are her functionalities, the CODA expounds on ethnopoet’s interiority, particularly after the rapid re-calibration of her politics. Through a rerun of El Chavo del Ocho, one of Mexican television’s most cherished shows, we tune into the heart-breaking indigence of barrio residents, only to marvel at the power of humor to, as Friday’s John Witherspoon put it, “fight another day.” This thesis is the tip of my tongue. Y por una vez, déjala que cante.

Real-time tracking system using C80 DSPs and a binocular robotic head

[EN]An active vision system to perform tracking of moving objects in real time is described. The main goal is to obtain a system integrating off-the-self components. These components includes a stereoscopic robotic-head, as active perception hardware; a DSP based board SDB C80, as massive data processor and image acquisition board; and finally, a Pentium PC running Windows NT that interconnects and manages the whole system. Real-time is achieved taking advantage of the special architecture of DSP. An evaluation of the performance is included.

Modelling human behaviour and event detection from videos

Automatic analysis of human behaviour in large collections of videos is gaining interest, even more so with the advent of file sharing sites such as YouTube. However, challenges still exist owing to several factors such as inter- and intra-class variations, cluttered backgrounds, occlusion, camera motion, scale, view and illumination changes. This research focuses on modelling human behaviour for action recognition in videos. The developed techniques are validated on large scale benchmark datasets and applied on real-world scenarios such as soccer videos. Three major contributions are made. The first contribution is in the area of proper choice of a feature representation for videos. This involved a study of state-of-the-art techniques for action recognition, feature extraction processing and dimensional reduction techniques so as to yield the best performance with optimal computational requirements. Secondly, temporal modelling of human behaviour is performed. This involved frequency analysis and temporal integration of local information in the video frames to yield a temporal feature vector. Current practices mostly average the frame information over an entire video and neglect the temporal order. Lastly, the proposed framework is applied and further adapted to real-world scenario such as soccer videos. A dataset consisting of video sequences depicting events of players falling is created from actual match data to this end and used to experimentally evaluate the proposed framework.

Vacant parking area estimation through background subtraction and transience map analysis

A method for estimating the dimensions of non-delimited free parking areas by using a static surveillance camera is proposed. The proposed method is specially designed to tackle the main challenges of urban scenarios (multiple moving objects, outdoor illumination conditions and occlusions between vehicles) with no training. The core of this work is the temporal analysis of the video frames to detect the occupancy variation of the parking areas. Two techniques are combined: background subtraction using a mixture of Gaussians to detect and track vehicles and the creation of a transience map to detect the parking and leaving of vehicles. The authors demonstrate that the proposed method yields satisfactory estimates on three real scenarios while being a low computational cost solution that can be applied in any kind of parking area covered by a single camera.

Wireless video-based sensor networks for surveillance of residential districts

Compared to traditional wired video sensor networks to supervise a residential district, Wireless Video-based Sensor Networks (WVSN) can provide more detail and precise information while reduce the cost. However, state-of-the-art low cost wireless video-based sensors have very constrained resources such as low bandwidth, small storage, limited processing capability, and limited energy resource. Also, due to the special sensing range of video-based sensors, cluster-based routing is not as effective as it apply to traditional sensor networks. This paper provides a novel real-time change mining algorithm based on an extracted profile model of moving objects learnt from frog's eyes. Example analysis shows the extracted profile would not miss any important semantic images to send to the Base Station for further hazards detection, while efficiently reducing futile video stream data to the degree that nowadays wireless video sensor can realize. Thus it makes WVSN available to surveillance of residential districts.

Towards simulation of flapping wings using immersed boundary method

In this work the immersed boundary method is applied to simulate incompressible turbulent flows around stationary and moving objects. The goal is to demonstrate that the immersed boundary technique along with a large eddy simulation approach is capable of simulating the effect of the so-called leading edge vortex (LEV), which can be found in flapping wing aerodynamics. A Lagrangian method is used to approximatethe solutions in the freshly cleared cells that lay within solid objects at one time step and emerge into fluid domain at the next time step. Flow around a stationary cylinder at ReD D 20, 40, and 3900 (based oncylinder diameter D) is first studied to validate the immersed boundary solver based on the finite volume scheme using a staggered grid. Then, a harmonically oscillating cylinder at ReD D 10 000 is considered to test the solver after the Lagrangian method is implemented to interpolate the solution in the freshly cleared cells. Finally, this approach is used to study flows around a stationary flat-plate at several angles of attack and fast pitching flat-plate. The rapidly pitching plate creates a dynamic LEV that can be used to improve the efficiency of flapping wings of micro air vehicle and to determine the optimum flapping frequency.

Disparity-defined objects moving in depth do not elicit three-dimensional shape constancy

Observers generally fail to recover three-dimensional shape accurately from binocular disparity. Typically, depth is overestimated at near distances and underestimated at far distances [Johnston, E. B. (1991). Systematic distortions of shape from stereopsis. Vision Research, 31, 1351–1360]. A simple prediction from this is that disparity-defined objects should appear to expand in depth when moving towards the observer, and compress in depth when moving away. However, additional information is provided when an object moves from which 3D Euclidean shape can be recovered, be this through the addition of structure from motion information [Richards, W. (1985). Structure from stereo and motion. Journal of the Optical Society of America A, 2, 343–349], or the use of non-generic strategies [Todd, J. T., & Norman, J. F. (2003). The visual perception of 3-D shape from multiple cues: Are observers capable of perceiving metric structure? Perception and Psychophysics, 65, 31–47]. Here, we investigated shape constancy for objects moving in depth. We found that to be perceived as constant in shape, objects needed to contract in depth when moving toward the observer, and expand in depth when moving away, countering the effects of incorrect distance scaling (Johnston, 1991). This is a striking example of the failure of shape con- stancy, but one that is predicted if observers neither accurately estimate object distance in order to recover Euclidean shape, nor are able to base their responses on a simpler processing strategy.

Improved detection and tracking of objects in surveillance video

Surveillance networks are typically monitored by a few people, viewing several monitors displaying the camera feeds. It is then very difficult for a human operator to effectively detect events as they happen. Recently, computer vision research has begun to address ways to automatically process some of this data, to assist human operators. Object tracking, event recognition, crowd analysis and human identification at a distance are being pursued as a means to aid human operators and improve the security of areas such as transport hubs. The task of object tracking is key to the effective use of more advanced technologies. To recognize an event people and objects must be tracked. Tracking also enhances the performance of tasks such as crowd analysis or human identification. Before an object can be tracked, it must be detected. Motion segmentation techniques, widely employed in tracking systems, produce a binary image in which objects can be located. However, these techniques are prone to errors caused by shadows and lighting changes. Detection routines often fail, either due to erroneous motion caused by noise and lighting effects, or due to the detection routines being unable to split occluded regions into their component objects. Particle filters can be used as a self contained tracking system, and make it unnecessary for the task of detection to be carried out separately except for an initial (often manual) detection to initialise the filter. Particle filters use one or more extracted features to evaluate the likelihood of an object existing at a given point each frame. Such systems however do not easily allow for multiple objects to be tracked robustly, and do not explicitly maintain the identity of tracked objects. This dissertation investigates improvements to the performance of object tracking algorithms through improved motion segmentation and the use of a particle filter. A novel hybrid motion segmentation / optical flow algorithm, capable of simultaneously extracting multiple layers of foreground and optical flow in surveillance video frames is proposed. The algorithm is shown to perform well in the presence of adverse lighting conditions, and the optical flow is capable of extracting a moving object. The proposed algorithm is integrated within a tracking system and evaluated using the ETISEO (Evaluation du Traitement et de lInterpretation de Sequences vidEO - Evaluation for video understanding) database, and significant improvement in detection and tracking performance is demonstrated when compared to a baseline system. A Scalable Condensation Filter (SCF), a particle filter designed to work within an existing tracking system, is also developed. The creation and deletion of modes and maintenance of identity is handled by the underlying tracking system; and the tracking system is able to benefit from the improved performance in uncertain conditions arising from occlusion and noise provided by a particle filter. The system is evaluated using the ETISEO database. The dissertation then investigates fusion schemes for multi-spectral tracking systems. Four fusion schemes for combining a thermal and visual colour modality are evaluated using the OTCBVS (Object Tracking and Classification in and Beyond the Visible Spectrum) database. It is shown that a middle fusion scheme yields the best results and demonstrates a significant improvement in performance when compared to a system using either mode individually. Findings from the thesis contribute to improve the performance of semi-automated video processing and therefore improve security in areas under surveillance.