Spatiotemporal Gesture Segmentation

Spotting patterns of interest in an input signal is a very useful task in many different fields including medicine, bioinformatics, economics, speech recognition and computer vision. Example instances of this problem include spotting an object of interest in an image (e.g., a tumor), a pattern of interest in a time-varying signal (e.g., audio analysis), or an object of interest moving in a specific way (e.g., a human's body gesture). Traditional spotting methods, which are based on Dynamic Time Warping or hidden Markov models, use some variant of dynamic programming to register the pattern and the input while accounting for temporal variation between them. At the same time, those methods often suffer from several shortcomings: they may give meaningless solutions when input observations are unreliable or ambiguous, they require a high complexity search across the whole input signal, and they may give incorrect solutions if some patterns appear as smaller parts within other patterns. In this thesis, we develop a framework that addresses these three problems, and evaluate the framework's performance in spotting and recognizing hand gestures in video. The first contribution is a spatiotemporal matching algorithm that extends the dynamic programming formulation to accommodate multiple candidate hand detections in every video frame. The algorithm finds the best alignment between the gesture model and the input, and simultaneously locates the best candidate hand detection in every frame. This allows for a gesture to be recognized even when the hand location is highly ambiguous. The second contribution is a pruning method that uses model-specific classifiers to reject dynamic programming hypotheses with a poor match between the input and model. Pruning improves the efficiency of the spatiotemporal matching algorithm, and in some cases may improve the recognition accuracy. The pruning classifiers are learned from training data, and cross-validation is used to reduce the chance of overpruning. The third contribution is a subgesture reasoning process that models the fact that some gesture models can falsely match parts of other, longer gestures. By integrating subgesture reasoning the spotting algorithm can avoid the premature detection of a subgesture when the longer gesture is actually being performed. Subgesture relations between pairs of gestures are automatically learned from training data. The performance of the approach is evaluated on two challenging video datasets: hand-signed digits gestured by users wearing short sleeved shirts, in front of a cluttered background, and American Sign Language (ASL) utterances gestured by ASL native signers. The experiments demonstrate that the proposed method is more accurate and efficient than competing approaches. The proposed approach can be generally applied to alignment or search problems with multiple input observations, that use dynamic programming to find a solution.

Biologically Inspired Approaches to Automated Feature Extraction and Target Recognition

Ongoing research at Boston University has produced computational models of biological vision and learning that embody a growing corpus of scientific data and predictions. Vision models perform long-range grouping and figure/ground segmentation, and memory models create attentionally controlled recognition codes that intrinsically cornbine botton-up activation and top-down learned expectations. These two streams of research form the foundation of novel dynamically integrated systems for image understanding. Simulations using multispectral images illustrate road completion across occlusions in a cluttered scene and information fusion from incorrect labels that are simultaneously inconsistent and correct. The CNS Vision and Technology Labs (cns.bu.edulvisionlab and cns.bu.edu/techlab) are further integrating science and technology through analysis, testing, and development of cognitive and neural models for large-scale applications, complemented by software specification and code distribution.

Brain Learning and Recognition: The Large and the Small of It in Inferotemporal Cortex

Anterior inferotemporal cortex (ITa) plays a key role in visual object recognition. Recognition is tolerant to object position, size, and view changes, yet recent neurophysiological data show ITa cells with high object selectivity often have low position tolerance, and vice versa. A neural model learns to simulate both this tradeoff and ITa responses to image morphs using large-scale and small-scale IT cells whose population properties may support invariant recognition.

ART-EMAP: A Neural Network Architecture for Object Recognition by Evidence Accumulation

A new neural network architecture is introduced for the recognition of pattern classes after supervised and unsupervised learning. Applications include spatio-temporal image understanding and prediction and 3-D object recognition from a series of ambiguous 2-D views. The architecture, called ART-EMAP, achieves a synthesis of adaptive resonance theory (ART) and spatial and temporal evidence integration for dynamic predictive mapping (EMAP). ART-EMAP extends the capabilities of fuzzy ARTMAP in four incremental stages. Stage 1 introduces distributed pattern representation at a view category field. Stage 2 adds a decision criterion to the mapping between view and object categories, delaying identification of ambiguous objects when faced with a low confidence prediction. Stage 3 augments the system with a field where evidence accumulates in medium-term memory (MTM). Stage 4 adds an unsupervised learning process to fine-tune performance after the limited initial period of supervised network training. Each ART-EMAP stage is illustrated with a benchmark simulation example, using both noisy and noise-free data. A concluding set of simulations demonstrate ART-EMAP performance on a difficult 3-D object recognition problem.

A What-and-Where Neural Network for Invariant Image Preprocessing

A feedforward neural network for invariant image preprocessing is proposed that represents the position1 orientation and size of an image figure (where it is) in a multiplexed spatial map. This map is used to generate an invariant representation of the figure that is insensitive to position1 orientation, and size for purposes of pattern recognition (what it is). A multiscale array of oriented filters followed by competition between orientations and scales is used to define the Where filter.

The double bind for non-traditional families in post-primary settings; to tell or not to tell their family identity

This thesis involved researching normative family discourses which are mediated through educational settings. The traditional family, consisting of father, mother and children all living together in one house is no longer reflective of the home situation of many Irish students (Lunn and Fahey, 2011). My study problematizes the dominant discourses which reflect how family differences are managed and recognised in schools. A framework using Foucauldian post structural critical analysis traces family stratification through the organisation of institutional and interpersonal relations at micro level in four post-primary schools. Standardising procedures such as the suppression of intimate relations between and among teacher and student, as well as the linear ordering of intergenerational relations, such as teacher/student and adult/child are critiqued. Normalising discourses operate in practices such as notes home which presume two parents together. Teacher assumptions about heterosexual two-parent families make it difficult for students to be open about a family setup that is constructed as different to the rest of the schools'. The management of family difference and deficit through pastoral care structures suggests a school-based politics of family adjustment. These practices beg the question whether families are better off not telling the school about their family identity. My thesis will be of interest to educational research and educational policy because it highlights how changing demographics such as family compositions are mis-conceptualised in schools, as well as revealing the changing forms of family governance through regimes such as pastoral care. This analysis allows for the existence of, and a valuing for, alternative modes of family existence, so that future curricular and legal discourses can be challenged in the interest of equity and social justice.

3D refraction correction and extraction of clinical parameters from spectral domain optical coherence tomography of the cornea.

Capable of three-dimensional imaging of the cornea with micrometer-scale resolution, spectral domain-optical coherence tomography (SDOCT) offers potential advantages over Placido ring and Scheimpflug photography based systems for accurate extraction of quantitative keratometric parameters. In this work, an SDOCT scanning protocol and motion correction algorithm were implemented to minimize the effects of patient motion during data acquisition. Procedures are described for correction of image data artifacts resulting from 3D refraction of SDOCT light in the cornea and from non-idealities of the scanning system geometry performed as a pre-requisite for accurate parameter extraction. Zernike polynomial 3D reconstruction and a recursive half searching algorithm (RHSA) were implemented to extract clinical keratometric parameters including anterior and posterior radii of curvature, central cornea optical power, central corneal thickness, and thickness maps of the cornea. Accuracy and repeatability of the extracted parameters obtained using a commercial 859nm SDOCT retinal imaging system with a corneal adapter were assessed using a rigid gas permeable (RGP) contact lens as a phantom target. Extraction of these parameters was performed in vivo in 3 patients and compared to commercial Placido topography and Scheimpflug photography systems. The repeatability of SDOCT central corneal power measured in vivo was 0.18 Diopters, and the difference observed between the systems averaged 0.1 Diopters between SDOCT and Scheimpflug photography, and 0.6 Diopters between SDOCT and Placido topography.

Modeling the evolution of regulatory elements by simultaneous detection and alignment with phylogenetic pair HMMs.

The computational detection of regulatory elements in DNA is a difficult but important problem impacting our progress in understanding the complex nature of eukaryotic gene regulation. Attempts to utilize cross-species conservation for this task have been hampered both by evolutionary changes of functional sites and poor performance of general-purpose alignment programs when applied to non-coding sequence. We describe a new and flexible framework for modeling binding site evolution in multiple related genomes, based on phylogenetic pair hidden Markov models which explicitly model the gain and loss of binding sites along a phylogeny. We demonstrate the value of this framework for both the alignment of regulatory regions and the inference of precise binding-site locations within those regions. As the underlying formalism is a stochastic, generative model, it can also be used to simulate the evolution of regulatory elements. Our implementation is scalable in terms of numbers of species and sequence lengths and can produce alignments and binding-site predictions with accuracy rivaling or exceeding current systems that specialize in only alignment or only binding-site prediction. We demonstrate the validity and power of various model components on extensive simulations of realistic sequence data and apply a specific model to study Drosophila enhancers in as many as ten related genomes and in the presence of gain and loss of binding sites. Different models and modeling assumptions can be easily specified, thus providing an invaluable tool for the exploration of biological hypotheses that can drive improvements in our understanding of the mechanisms and evolution of gene regulation.

Neural mechanisms of context effects on face recognition: automatic binding and context shift decrements.

Although people do not normally try to remember associations between faces and physical contexts, these associations are established automatically, as indicated by the difficulty of recognizing familiar faces in different contexts ("butcher-on-the-bus" phenomenon). The present fMRI study investigated the automatic binding of faces and scenes. In the face-face (F-F) condition, faces were presented alone during both encoding and retrieval, whereas in the face/scene-face (FS-F) condition, they were presented overlaid on scenes during encoding but alone during retrieval (context change). Although participants were instructed to focus only on the faces during both encoding and retrieval, recognition performance was worse in the FS-F than in the F-F condition ("context shift decrement" [CSD]), confirming automatic face-scene binding during encoding. This binding was mediated by the hippocampus as indicated by greater subsequent memory effects (remembered > forgotten) in this region for the FS-F than the F-F condition. Scene memory was mediated by right parahippocampal cortex, which was reactivated during successful retrieval when the faces were associated with a scene during encoding (FS-F condition). Analyses using the CSD as a regressor yielded a clear hemispheric asymmetry in medial temporal lobe activity during encoding: Left hippocampal and parahippocampal activity was associated with a smaller CSD, indicating more flexible memory representations immune to context changes, whereas right hippocampal/rhinal activity was associated with a larger CSD, indicating less flexible representations sensitive to context change. Taken together, the results clarify the neural mechanisms of context effects on face recognition.

Characterization of Image Quality for 3D Scatter Corrected Breast CT Images.

The goal of this study was to characterize the image quality of our dedicated, quasi-monochromatic spectrum, cone beam breast imaging system under scatter corrected and non-scatter corrected conditions for a variety of breast compositions. CT projections were acquired of a breast phantom containing two concentric sets of acrylic spheres that varied in size (1-8mm) based on their polar position. The breast phantom was filled with 3 different concentrations of methanol and water, simulating a range of breast densities (0.79-1.0g/cc); acrylic yarn was sometimes included to simulate connective tissue of a breast. For each phantom condition, 2D scatter was measured for all projection angles. Scatter-corrected and uncorrected projections were then reconstructed with an iterative ordered subsets convex algorithm. Reconstructed image quality was characterized using SNR and contrast analysis, and followed by a human observer detection task for the spheres in the different concentric rings. Results show that scatter correction effectively reduces the cupping artifact and improves image contrast and SNR. Results from the observer study indicate that there was no statistical difference in the number or sizes of lesions observed in the scatter versus non-scatter corrected images for all densities. Nonetheless, applying scatter correction for differing breast conditions improves overall image quality.

Informatic Opacity: Biometric Facial Recognition and the Aesthetics and Politics of Defacement

Confronting the rapidly increasing, worldwide reliance on biometric technologies to surveil, manage, and police human beings, my dissertation Informatic Opacity: Biometric Facial Recognition and the Aesthetics and Politics of Defacement charts a series of queer, feminist, and anti-racist concepts and artworks that favor opacity as a means of political struggle against surveillance and capture technologies in the 21st century. Utilizing biometric facial recognition as a paradigmatic example, I argue that today's surveillance requires persons to be informatically visible in order to control them, and such visibility relies upon the production of technical standardizations of identification to operate globally, which most vehemently impact non- normative, minoritarian populations. Thus, as biometric technologies turn exposures of the face into sites of governance, activists and artists strive to make the face biometrically illegible and refuse the political recognition biometrics promises through acts of masking, escape, and imperceptibility. Although I specifically describe tactics of making the face unrecognizable as "defacement," I broadly theorize refusals to visually cohere to digital surveillance and capture technologies' gaze as "informatic opacity," an aesthetic-political theory and practice of anti- normativity at a global, technical scale whose goal is maintaining the autonomous determination of alterity and difference by evading the quantification, standardization, and regulation of identity imposed by biometrics and the state. My dissertation also features two artworks: Facial Weaponization Suite, a series of masks and public actions, and Face Cages, a critical, dystopic installation that investigates the abstract violence of biometric facial diagramming and analysis. I develop an interdisciplinary, practice-based method that pulls from contemporary art and aesthetic theory, media theory and surveillance studies, political and continental philosophy, queer and feminist theory, transgender studies, postcolonial theory, and critical race studies.

Baby on board: olfactory cues indicate pregnancy and fetal sex in a non-human primate.

Olfactory cues play an integral, albeit underappreciated, role in mediating vertebrate social and reproductive behaviour. These cues fluctuate with the signaller's hormonal condition, coincident with and informative about relevant aspects of its reproductive state, such as pubertal onset, change in season and, in females, timing of ovulation. Although pregnancy dramatically alters a female's endocrine profiles, which can be further influenced by fetal sex, the relationship between gestation and olfactory cues is poorly understood. We therefore examined the effects of pregnancy and fetal sex on volatile genital secretions in the ring-tailed lemur (Lemur catta), a strepsirrhine primate possessing complex olfactory mechanisms of reproductive signalling. While pregnant, dams altered and dampened their expression of volatile chemicals, with compound richness being particularly reduced in dams bearing sons. These changes were comparable in magnitude with other, published chemical differences among lemurs that are salient to conspecifics. Such olfactory 'signatures' of pregnancy may help guide social interactions, potentially promoting mother-infant recognition, reducing intragroup conflict or counteracting behavioural mechanisms of paternity confusion; cues that also advertise fetal sex may additionally facilitate differential sex allocation.

On the problems of object restoration and image extrapolation in optics

In this paper we consider the problems of object restoration and image extrapolation, according to the regularization theory of improperly posed problems. In order to take into account the stochastic nature of the noise and to introduce the main concepts of information theory, great attention is devoted to the probabilistic methods of regularization. The kind of the restored continuity is investigated in detail; in particular we prove that, while the image extrapolation presents a Hölder type stability, the object restoration has only a logarithmic continuity. © 1979 American Institute of Physics.

Concise introduction to image processing using C++

A Concise Intro to Image Processing using C++ presents state-of-the-art image processing methodology, including current industrial practices for image compression, image de-noising methods based on partial differential equations, and new image compression methods such as fractal image compression and wavelet compression. It includes elementary concepts of image processing and related fundamental tools with coding examples as well as exercises. With a particular emphasis on illustrating fractal and wavelet compression algorithms, the text covers image segmentation, object recognition, and morphology. An accompanying CD-ROM contains code for all algorithms.

Through the portal – improving the flow of information on non-native marine species in Great Britain.

Abstract: The UK Government funded, GB Non-Native Species Information Portal (GBNNSIP) collects and collates data on non-native species in Great Britain making information available online. Resources include a comprehensive register of non-native species and detailed fact sheets for a sub-set, significant to humans or the environment. Reporting of species records are linked to risk analyses, rapid responses and horizon scanning to support the early recognition of threats (Figure 12). The portal has improved flow of new and existing distributional data to the National Biodiversity Network (NBN) to generate distribution maps for the portal. The project is led by the Biological Records Centre and the Marine Biological Association is responsible for marine non-native species within this scheme. The INTERREG IV funded project Marinexus has included professional research and citizen science work, which has fed directly into the portal. The portal outputs and the work of Marinexus have a range of marine governance applications, including supporting work towards MSFD compliance.