Efficient Image and Video Representations for Retrieval

Image (Video) retrieval is an interesting problem of retrieving images (videos) similar to the query. Images (Videos) are represented in an input (feature) space and similar images (videos) are obtained by finding nearest neighbors in the input representation space. Numerous input representations both in real valued and binary space have been proposed for conducting faster retrieval. In this thesis, we present techniques that obtain improved input representations for retrieval in both supervised and unsupervised settings for images and videos. Supervised retrieval is a well known problem of retrieving same class images of the query. We address the practical aspects of achieving faster retrieval with binary codes as input representations for the supervised setting in the first part, where binary codes are used as addresses into hash tables. In practice, using binary codes as addresses does not guarantee fast retrieval, as similar images are not mapped to the same binary code (address). We address this problem by presenting an efficient supervised hashing (binary encoding) method that aims to explicitly map all the images of the same class ideally to a unique binary code. We refer to the binary codes of the images as `Semantic Binary Codes' and the unique code for all same class images as `Class Binary Code'. We also propose a new class­ based Hamming metric that dramatically reduces the retrieval times for larger databases, where only hamming distance is computed to the class binary codes. We also propose a Deep semantic binary code model, by replacing the output layer of a popular convolutional Neural Network (AlexNet) with the class binary codes and show that the hashing functions learned in this way outperforms the state­ of ­the art, and at the same time provide fast retrieval times. In the second part, we also address the problem of supervised retrieval by taking into account the relationship between classes. For a given query image, we want to retrieve images that preserve the relative order i.e. we want to retrieve all same class images first and then, the related classes images before different class images. We learn such relationship aware binary codes by minimizing the similarity between inner product of the binary codes and the similarity between the classes. We calculate the similarity between classes using output embedding vectors, which are vector representations of classes. Our method deviates from the other supervised binary encoding schemes as it is the first to use output embeddings for learning hashing functions. We also introduce new performance metrics that take into account the related class retrieval results and show significant gains over the state­ of­ the art. High Dimensional descriptors like Fisher Vectors or Vector of Locally Aggregated Descriptors have shown to improve the performance of many computer vision applications including retrieval. In the third part, we will discuss an unsupervised technique for compressing high dimensional vectors into high dimensional binary codes, to reduce storage complexity. In this approach, we deviate from adopting traditional hyperplane hashing functions and instead learn hyperspherical hashing functions. The proposed method overcomes the computational challenges of directly applying the spherical hashing algorithm that is intractable for compressing high dimensional vectors. A practical hierarchical model that utilizes divide and conquer techniques using the Random Select and Adjust (RSA) procedure to compress such high dimensional vectors is presented. We show that our proposed high dimensional binary codes outperform the binary codes obtained using traditional hyperplane methods for higher compression ratios. In the last part of the thesis, we propose a retrieval based solution to the Zero shot event classification problem - a setting where no training videos are available for the event. To do this, we learn a generic set of concept detectors and represent both videos and query events in the concept space. We then compute similarity between the query event and the video in the concept space and videos similar to the query event are classified as the videos belonging to the event. We show that we significantly boost the performance using concept features from other modalities.

The Promise of Access: Hope and Inequality in the Information Economy

In 2013, a series of posters began appearing in Washington, DC’s Metro system. Each declared “The internet: Your future depends on it” next to a photo of a middle-aged black Washingtonian, and an advertisement for the municipal government’s digital training resources. This hopeful discourse is familiar but where exactly does it come from? And how are our public institutions reorganized to approach the problem of poverty as a problem of technology? The Clinton administration’s ‘digital divide’ policy program popularized this hopeful discourse about personal computing powering social mobility, positioned internet startups as the ‘right’ side of the divide, and charged institutions of social reproduction such as schools and libraries with closing the gap and upgrading themselves in the image of internet startups. After introducing the development regime that builds this idea into the urban landscape through what I call the ‘political economy of hope’, and tracing the origin of the digital divide frame, this dissertation draws on three years of comparative ethnographic fieldwork in startups, schools, and libraries to explore how this hope is reproduced in daily life, becoming the common sense that drives our understanding of and interaction with economic inequality and reproduces that inequality in turn. I show that the hope in personal computing to power social mobility becomes a method of securing legitimacy and resources for both white émigré technologists and institutions of social reproduction struggling to understand and manage the persistent poverty of the information economy. I track the movement of this common sense between institutions, showing how the political economy of hope transforms them as part of a larger development project. This dissertation models a new, relational direction for digital divide research that grounds the politics of economic inequality with an empirical focus on technologies of poverty management. It demands a conceptual shift that sees the digital divide not as a bug within the information economy, but a feature of it.