Algorithms for super-resolution of images based on Sparse Representation and Manifolds

lmage super-resolution is defined as a class of techniques that enhance the spatial resolution of images. Super-resolution methods can be subdivided in single and multi image methods. This thesis focuses on developing algorithms based on mathematical theories for single image super­ resolution problems. lndeed, in arder to estimate an output image, we adopta mixed approach: i.e., we use both a dictionary of patches with sparsity constraints (typical of learning-based methods) and regularization terms (typical of reconstruction-based methods). Although the existing methods already per- form well, they do not take into account the geometry of the data to: regularize the solution, cluster data samples (samples are often clustered using algorithms with the Euclidean distance as a dissimilarity metric), learn dictionaries (they are often learned using PCA or K-SVD). Thus, state-of-the-art methods still suffer from shortcomings. In this work, we proposed three new methods to overcome these deficiencies. First, we developed SE-ASDS (a structure tensor based regularization term) in arder to improve the sharpness of edges. SE-ASDS achieves much better results than many state-of-the- art algorithms. Then, we proposed AGNN and GOC algorithms for determining a local subset of training samples from which a good local model can be computed for recon- structing a given input test sample, where we take into account the underlying geometry of the data. AGNN and GOC methods outperform spectral clustering, soft clustering, and geodesic distance based subset selection in most settings. Next, we proposed aSOB strategy which takes into account the geometry of the data and the dictionary size. The aSOB strategy outperforms both PCA and PGA methods. Finally, we combine all our methods in a unique algorithm, named G2SR. Our proposed G2SR algorithm shows better visual and quantitative results when compared to the results of state-of-the-art methods.

Neural Network Evidence for the Coupling of Presaccadic Visual Remapping to Predictive Eye Position Updating.

As we look around a scene, we perceive it as continuous and stable even though each saccadic eye movement changes the visual input to the retinas. How the brain achieves this perceptual stabilization is unknown, but a major hypothesis is that it relies on presaccadic remapping, a process in which neurons shift their visual sensitivity to a new location in the scene just before each saccade. This hypothesis is difficult to test in vivo because complete, selective inactivation of remapping is currently intractable. We tested it in silico with a hierarchical, sheet-based neural network model of the visual and oculomotor system. The model generated saccadic commands to move a video camera abruptly. Visual input from the camera and internal copies of the saccadic movement commands, or corollary discharge, converged at a map-level simulation of the frontal eye field (FEF), a primate brain area known to receive such inputs. FEF output was combined with eye position signals to yield a suitable coordinate frame for guiding arm movements of a robot. Our operational definition of perceptual stability was "useful stability," quantified as continuously accurate pointing to a visual object despite camera saccades. During training, the emergence of useful stability was correlated tightly with the emergence of presaccadic remapping in the FEF. Remapping depended on corollary discharge but its timing was synchronized to the updating of eye position. When coupled to predictive eye position signals, remapping served to stabilize the target representation for continuously accurate pointing. Graded inactivations of pathways in the model replicated, and helped to interpret, previous in vivo experiments. The results support the hypothesis that visual stability requires presaccadic remapping, provide explanations for the function and timing of remapping, and offer testable hypotheses for in vivo studies. We conclude that remapping allows for seamless coordinate frame transformations and quick actions despite visual afferent lags. With visual remapping in place for behavior, it may be exploited for perceptual continuity.

Unsettling Histories: Representation and Indigenous Creative Art Praxis in Official Indian Residential School Redress

This dissertation offers an investigation of the role of visual strategies, art, and representation in reconciling Indian Residential School history in Canada. This research builds upon theories of biopolitics, settler colonialism, and race to examine the project of redress and reconciliation as nation and identity building strategies engaged in the ongoing structural invasion of settler colonialism. It considers the key policy moments and expressions of the federal government—from RCAP to the IRSSA and subsequent apology—as well as the visual discourse of reconciliation as it works through archival photography, institutional branding, and commissioned works. These articulations are read alongside the creative and critical work of Indigenous artists and knowledge producers working within and outside of hegemonic structures on the topics of Indian Residential School history and redress. In particular the works of Jeff Thomas, Adrian Stimson, Krista Belle Stewart, Christi Belcourt, Luke Marston, Peter Morin, and Carey Newman are discussed in this dissertation. These works must be understood in relationship to the normative discourse of reconciliation as a legitimizing mechanism of settler colonial hegemony. Beyond the binary of cooptation and autonomous resistance, these works demonstrate the complexity of representing Indigeneity: as an ongoing site of settler colonial encounter and simultaneously the forum for the willful refusal of contingency or containment.

Fast 2D/3D object representation with growing neural gas

This work presents the design of a real-time system to model visual objects with the use of self-organising networks. The architecture of the system addresses multiple computer vision tasks such as image segmentation, optimal parameter estimation and object representation. We first develop a framework for building non-rigid shapes using the growth mechanism of the self-organising maps, and then we define an optimal number of nodes without overfitting or underfitting the network based on the knowledge obtained from information-theoretic considerations. We present experimental results for hands and faces, and we quantitatively evaluate the matching capabilities of the proposed method with the topographic product. The proposed method is easily extensible to 3D objects, as it offers similar features for efficient mesh reconstruction.

Picturing Gentrification: Representation of Urban Change, Pendleton Salford

The paper examines how visual representations of urban regeneration contribute to the gentrification process. It asks can alternative photographic and textual strategies provide a meaningful counter narrative to resist persuasive corporate discourses on urban revitalization? Focusing on the gentrification of social housing in Pendleton, Salford (Greater Manchester) the paper debates the role of visual imagery in fostering perceptions about urban change by evaluating fieldwork undertaken by the authors in the site since 2004. The paper will question whether such an in-depth longitudinal project and its consequent archive can be utilized as a political tool to highlight the wider processes involved in such regimes of disinvestment and accumulation. Through the combination of photography and site writing in the environment can certain economic and political processes be made legible if not fully visible to highlight causation and effect?

Between representation and figuration. Nouvelle Vague’s cinema: a history revision

50 years after the birth of the Nouvelle Vague, the inheritance that the contemporary cinema receives from it is inevitable. Figures and visual motifs; stories, themes, faces or common places; aesthetic and language devices. The echoes appear in various ways, each affiliation involves a different relationship and therefore a dissimilar approximation to its analysis. And yet, both the academy and the film critics maintain their will to think the Nouvelle Vague as a whole, a universe, a stream or an aesthetic trend. However, does a Nouvelle Vague’s aesthetic exist? And if so: why and how to address their historical revision? Taking Deleuze’s thesis on the time-image and Serge Daney’s assertion according to which 50 years after the birth of the Nouvelle Vague, the inheritance that the contemporary cinema receives from it is inevitable. Figures and visual motifs; stories, themes, faces or common places; aesthetic and language devices. The echoes appear in various ways, each affiliation involves a different relationship and therefore a dissimilar approximation to its analysis. And yet, both the academy and the film critics maintain their will to think the Nouvelle Vague as a whole, a universe, a stream or an aesthetic trend. However, does a Nouvelle Vague’s aesthetic exist? And if so: why and how to address their historical revision? Taking Deleuze’s thesis on the time-image and Serge Daney’s assertion according to which

A biologically inspired appearance model for robust visual tracking

In this work, we propose a biologically inspired appearance model for robust visual tracking. Motivated in part by the success of the hierarchical organization of the primary visual cortex (area V1), we establish an architecture consisting of five layers: whitening, rectification, normalization, coding and polling. The first three layers stem from the models developed for object recognition. In this paper, our attention focuses on the coding and pooling layers. In particular, we use a discriminative sparse coding method in the coding layer along with spatial pyramid representation in the pooling layer, which makes it easier to distinguish the target to be tracked from its background in the presence of appearance variations. An extensive experimental study shows that the proposed method has higher tracking accuracy than several state-of-the-art trackers.

Monitorização e gestão em turismo : uma proposta de dashboard visual

A presente investigação propõe-se a atuar no sector turístico, uma vez que este é bombardeado diariamente por uma quantidade considerável de dados e informações. Atualmente, usufrui-se significativamente mais da tecnologia com a finalidade de promover e vender os produtos/serviços disponíveis no mercado. A par da evolução tecnológica, os utilizadores/clientes conseguem comprar, cada vez mais, à distancia de um clique os produtos turísticos que desejam. No entanto, há um variado leque de aplicações sobre o turismo que permitem entender os gostos e as necessidades dos turistas assim como a sua atitude para com o mesmo. Porém, nem as entidades nem os gestores turísticos usufruem inteligentemente dos dados que lhes são facultados. Estes tendem normalmente a prender-se pelo turismo em Portugal e de que forma é que a sua entidade é apresentada acabando por esquecer que os dados podem e devem ser utilizados para expandir o mercado assim como entender/conhecer potenciais mercados. Deste modo, o fundamento principal desta investigação remete para a criação de uma plataforma infocomunicacional que analise na totalidade os dados obtidos, assim como fornecer as ferramentas pertinentes para que se consiga fazer esta análise, nomeadamente através de uma representação infográfica adequada e estratégias de a comunicar aos stakeholders.. Para tal foi aplicada no âmbito desta dissertação a metodologia investigação/ação, vista como um processo cíclico que para além de incluir simultaneamente estas duas vertentes, vai alternando entre a ação e a reflexão critica sendo sustentada por bases teóricas. A criação do protótipo da plataforma Smart Tourism, resultou num sistema inovador que tenta responder aos indicadores escolhidos no Dashbord e ao problema infocomunicacional, tentando criar as bases necessárias para que as entidades consigam analisar de forma mais integrada/sistematizada e racional a atividade turística. Foi por isso, desenvolvido e avaliado qualitativamente um protótipo de base infocomunicacional visual (dashboard visual) que para além do que para além do que já foi referido, consegue proporcionar a gestão dos produtos, clientes, staff e parceiros, aumentando assim o valor deste sector.

A three-dimensional representation method for noisy point clouds based on growing self-organizing maps accelerated on GPUs

The research described in this thesis was motivated by the need of a robust model capable of representing 3D data obtained with 3D sensors, which are inherently noisy. In addition, time constraints have to be considered as these sensors are capable of providing a 3D data stream in real time. This thesis proposed the use of Self-Organizing Maps (SOMs) as a 3D representation model. In particular, we proposed the use of the Growing Neural Gas (GNG) network, which has been successfully used for clustering, pattern recognition and topology representation of multi-dimensional data. Until now, Self-Organizing Maps have been primarily computed offline and their application in 3D data has mainly focused on free noise models, without considering time constraints. It is proposed a hardware implementation leveraging the computing power of modern GPUs, which takes advantage of a new paradigm coined as General-Purpose Computing on Graphics Processing Units (GPGPU). The proposed methods were applied to different problem and applications in the area of computer vision such as the recognition and localization of objects, visual surveillance or 3D reconstruction.

Contributions to 3D Data Registration and Representation

Nowadays, new computers generation provides a high performance that enables to build computationally expensive computer vision applications applied to mobile robotics. Building a map of the environment is a common task of a robot and is an essential part to allow the robots to move through these environments. Traditionally, mobile robots used a combination of several sensors from different technologies. Lasers, sonars and contact sensors have been typically used in any mobile robotic architecture, however color cameras are an important sensor due to we want the robots to use the same information that humans to sense and move through the different environments. Color cameras are cheap and flexible but a lot of work need to be done to give robots enough visual understanding of the scenes. Computer vision algorithms are computational complex problems but nowadays robots have access to different and powerful architectures that can be used for mobile robotics purposes. The advent of low-cost RGB-D sensors like Microsoft Kinect which provide 3D colored point clouds at high frame rates made the computer vision even more relevant in the mobile robotics field. The combination of visual and 3D data allows the systems to use both computer vision and 3D processing and therefore to be aware of more details of the surrounding environment. The research described in this thesis was motivated by the need of scene mapping. Being aware of the surrounding environment is a key feature in many mobile robotics applications from simple robotic navigation to complex surveillance applications. In addition, the acquisition of a 3D model of the scenes is useful in many areas as video games scene modeling where well-known places are reconstructed and added to game systems or advertising where once you get the 3D model of one room the system can add furniture pieces using augmented reality techniques. In this thesis we perform an experimental study of the state-of-the-art registration methods to find which one fits better to our scene mapping purposes. Different methods are tested and analyzed on different scene distributions of visual and geometry appearance. In addition, this thesis proposes two methods for 3d data compression and representation of 3D maps. Our 3D representation proposal is based on the use of Growing Neural Gas (GNG) method. This Self-Organizing Maps (SOMs) has been successfully used for clustering, pattern recognition and topology representation of various kind of data. Until now, Self-Organizing Maps have been primarily computed offline and their application in 3D data has mainly focused on free noise models without considering time constraints. Self-organising neural models have the ability to provide a good representation of the input space. In particular, the Growing Neural Gas (GNG) is a suitable model because of its flexibility, rapid adaptation and excellent quality of representation. However, this type of learning is time consuming, specially for high-dimensional input data. Since real applications often work under time constraints, it is necessary to adapt the learning process in order to complete it in a predefined time. This thesis proposes a hardware implementation leveraging the computing power of modern GPUs which takes advantage of a new paradigm coined as General-Purpose Computing on Graphics Processing Units (GPGPU). Our proposed geometrical 3D compression method seeks to reduce the 3D information using plane detection as basic structure to compress the data. This is due to our target environments are man-made and therefore there are a lot of points that belong to a plane surface. Our proposed method is able to get good compression results in those man-made scenarios. The detected and compressed planes can be also used in other applications as surface reconstruction or plane-based registration algorithms. Finally, we have also demonstrated the goodness of the GPU technologies getting a high performance implementation of a CAD/CAM common technique called Virtual Digitizing.

ViDRILO: The Visual and Depth Robot Indoor Localization with Objects information dataset

In this article we describe a semantic localization dataset for indoor environments named ViDRILO. The dataset provides five sequences of frames acquired with a mobile robot in two similar office buildings under different lighting conditions. Each frame consists of a point cloud representation of the scene and a perspective image. The frames in the dataset are annotated with the semantic category of the scene, but also with the presence or absence of a list of predefined objects appearing in the scene. In addition to the frames and annotations, the dataset is distributed with a set of tools for its use in both place classification and object recognition tasks. The large number of labeled frames in conjunction with the annotation scheme make this dataset different from existing ones. The ViDRILO dataset is released for use as a benchmark for different problems such as multimodal place classification and object recognition, 3D reconstruction or point cloud data compression.

Efficient representation of texture details in medical images by fusion of Ripplet and DDCT transformed images

Purpose: To evaluate and compare the performance of Ripplet Type-1 transform and directional discrete cosine transform (DDCT) and their combinations for improved representation of MRI images while preserving its fine features such as edges along the smooth curves and textures. Methods: In a novel image representation method based on fusion of Ripplet type-1 and conventional/directional DCT transforms, source images were enhanced in terms of visual quality using Ripplet and DDCT and their various combinations. The enhancement achieved was quantified on the basis of peak signal to noise ratio (PSNR), mean square error (MSE), structural content (SC), average difference (AD), maximum difference (MD), normalized cross correlation (NCC), and normalized absolute error (NAE). To determine the attributes of both transforms, these transforms were combined to represent the entire image as well. All the possible combinations were tested to present a complete study of combinations of the transforms and the contrasts were evaluated amongst all the combinations. Results: While using the direct combining method (DDCT) first and then the Ripplet method, a PSNR value of 32.3512 was obtained which is comparatively higher than the PSNR values of the other combinations. This novel designed technique gives PSNR value approximately equal to the PSNR’s of parent techniques. Along with this, it was able to preserve edge information, texture information and various other directional image features. The fusion of DDCT followed by the Ripplet reproduced the best images. Conclusion: The transformation of images using Ripplet followed by DDCT ensures a more efficient method for the representation of images with preservation of its fine details like edges and textures.

Veteranness : Representations of Combat-related PTSD in U.S. Popular Visual Media

Posttraumatic stress and PTSD are becoming familiar terms to refer to what we often call the invisible wounds of war, yet these are recent additions to a popular discourse in which images of and ideas about combat-affected veterans have long circulated. A legacy of ideas about combat veterans and war trauma thus intersects with more recent clinical information about PTSD to become part of a discourse of visual media that has defined and continues to redefine veteran for popular audiences. In this dissertation I examine realist combat veteran representations in selected films and other visual media from three periods: during and after World Wars I and II (James Allen from I Am a Fugitive from a Chain Gang, Fred Derry and Al Stephenson from The Best Years of Our Lives); after the Vietnam War (Michael from The Deer Hunter, Eriksson from Casualties of War), and post 9/11 (Will James from The Hurt Locker, a collection of veterans from Wartorn: 1861-2010.) Employing a theoretical framework informed by visual media studies, Barthes’ concept of myth, and Foucault’s concept ofdiscursive unity, I analyze how these veteran representations are endowed with PTSD symptom-like behaviors and responses that seem reasonable and natural within the narrative arc. I contend that veteran myths appear through each veteran representation as the narrative develops and resolves. I argue that these veteran myths are many and varied but that they crystallize in a dominant veteran discourse, a discursive unity that I term veteranness. I further argue that veteranness entangles discrete categories such as veteran, combat veteran, and PTSD with veteran myths, often tying dominant discourse about combat-related PTSD to outdated or outmoded notions that significantly affect our attitudes about and treatment of veterans. A basic premise of my research is that unless and until we learn about the lasting effects of the trauma inherent to combat, we hinder our ability to fulfill our responsibilities to war veterans. A society that limits its understanding of posttraumatic stress, PTSD and post-war experiences of actual veterans affected by war trauma to veteranness or veteran myths risks normalizing or naturalizing an unexamined set of sociocultural expectations of all veterans, rendering them voice-less, invisible, and, ultimately disposable.

Detection of Glass pattern configurations by the primary visual cortex

The neurons in the primary visual cortex that respond to the orientation of visual stimuli were discovered in the late 1950s (Hubel, D.H. & Wiesel, T.N. 1959. J. Physiol. 148:574-591) but how they achieve this response is poorly understood. Recently, experiments have demonstrated that the visual cortex may use the image processing techniques of cross or auto-correlation to detect the streaks in random dot patterns (Barlow, H. & Berry, D.L. 2010. Proc. R. Soc. B. 278: 2069-2075). These experiments made use of sinusoidally modulated random dot patterns and of the so-called Glass patterns - where randomly positioned dot pairs are oriented in a parallel configuration (Glass, L. 1969. Nature. 223: 578-580). The image processing used by the visual cortex could be inferred from how the threshold of detection of these patterns in the presence of random noise varied as a function of the dot density in the patterns. In the present study, the detection thresholds have been measured for other types of patterns including circular, hyperbolic, spiral and radial Glass patterns and an indication of the type of image processing (cross or auto-correlation) by the visual cortex is presented. As a result, it is hoped that this study will contribute to an understanding of what David Marr called the ‘computational goal’ of the primary visual cortex (Marr, D. 1982. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. New York: Freeman.)

Visual Navigation In The Neotropical Ant Odontomachus Hastatus (formicidae, Ponerinae), A Predominantly Nocturnal, Canopy-dwelling Predator Of The Atlantic Rainforest.

The arboreal ant Odontomachus hastatus nests among roots of epiphytic bromeliads in the sandy forest at Cardoso Island (Brazil). Crepuscular and nocturnal foragers travel up to 8m to search for arthropod prey in the canopy, where silhouettes of leaves and branches potentially provide directional information. We investigated the relevance of visual cues (canopy, horizon patterns) during navigation in O. hastatus. Laboratory experiments using a captive ant colony and a round foraging arena revealed that an artificial canopy pattern above the ants and horizon visual marks are effective orientation cues for homing O. hastatus. On the other hand, foragers that were only given a tridimensional landmark (cylinder) or chemical marks were unable to home correctly. Navigation by visual cues in O. hastatus is in accordance with other diurnal arboreal ants. Nocturnal luminosity (moon, stars) is apparently sufficient to produce contrasting silhouettes from the canopy and surrounding vegetation, thus providing orientation cues. Contrary to the plain floor of the round arena, chemical cues may be important for marking bifurcated arboreal routes. This experimental demonstration of the use of visual cues by a predominantly nocturnal arboreal ant provides important information for comparative studies on the evolution of spatial orientation behavior in ants. This article is part of a Special Issue entitled: Neotropical Behaviour.