Efficient Image and Video Representations for Retrieval

Image (Video) retrieval is an interesting problem of retrieving images (videos) similar to the query. Images (Videos) are represented in an input (feature) space and similar images (videos) are obtained by finding nearest neighbors in the input representation space. Numerous input representations both in real valued and binary space have been proposed for conducting faster retrieval. In this thesis, we present techniques that obtain improved input representations for retrieval in both supervised and unsupervised settings for images and videos. Supervised retrieval is a well known problem of retrieving same class images of the query. We address the practical aspects of achieving faster retrieval with binary codes as input representations for the supervised setting in the first part, where binary codes are used as addresses into hash tables. In practice, using binary codes as addresses does not guarantee fast retrieval, as similar images are not mapped to the same binary code (address). We address this problem by presenting an efficient supervised hashing (binary encoding) method that aims to explicitly map all the images of the same class ideally to a unique binary code. We refer to the binary codes of the images as `Semantic Binary Codes' and the unique code for all same class images as `Class Binary Code'. We also propose a new class­ based Hamming metric that dramatically reduces the retrieval times for larger databases, where only hamming distance is computed to the class binary codes. We also propose a Deep semantic binary code model, by replacing the output layer of a popular convolutional Neural Network (AlexNet) with the class binary codes and show that the hashing functions learned in this way outperforms the state­ of ­the art, and at the same time provide fast retrieval times. In the second part, we also address the problem of supervised retrieval by taking into account the relationship between classes. For a given query image, we want to retrieve images that preserve the relative order i.e. we want to retrieve all same class images first and then, the related classes images before different class images. We learn such relationship aware binary codes by minimizing the similarity between inner product of the binary codes and the similarity between the classes. We calculate the similarity between classes using output embedding vectors, which are vector representations of classes. Our method deviates from the other supervised binary encoding schemes as it is the first to use output embeddings for learning hashing functions. We also introduce new performance metrics that take into account the related class retrieval results and show significant gains over the state­ of­ the art. High Dimensional descriptors like Fisher Vectors or Vector of Locally Aggregated Descriptors have shown to improve the performance of many computer vision applications including retrieval. In the third part, we will discuss an unsupervised technique for compressing high dimensional vectors into high dimensional binary codes, to reduce storage complexity. In this approach, we deviate from adopting traditional hyperplane hashing functions and instead learn hyperspherical hashing functions. The proposed method overcomes the computational challenges of directly applying the spherical hashing algorithm that is intractable for compressing high dimensional vectors. A practical hierarchical model that utilizes divide and conquer techniques using the Random Select and Adjust (RSA) procedure to compress such high dimensional vectors is presented. We show that our proposed high dimensional binary codes outperform the binary codes obtained using traditional hyperplane methods for higher compression ratios. In the last part of the thesis, we propose a retrieval based solution to the Zero shot event classification problem - a setting where no training videos are available for the event. To do this, we learn a generic set of concept detectors and represent both videos and query events in the concept space. We then compute similarity between the query event and the video in the concept space and videos similar to the query event are classified as the videos belonging to the event. We show that we significantly boost the performance using concept features from other modalities.

Are Videogames Art?

My dissertation defends a positive answer to the question: “Can a videogame be a work of art? ” To achieve this goal I develop definitions of several concepts, primarily ‘art’, ‘games’, and ‘videogames’, and offer arguments about the compatibility of these notions. In Part One, I defend a definition of art from amongst several contemporary and historical accounts. This definition, the Intentional-Historical account, requires, among other things, that an artwork have the right kind of creative intentions behind it, in short that the work be intended to be regarded in a particular manner. This is a leading account that has faced several recent objections that I address, particular the buck-passing theory, the objection against non-failure theories of art, and the simultaneous creation response to the ur-art problem, while arguing that it is superior to other theories in its ability to answer the question of videogames’ art status. Part Two examines whether games can exhibit the art-making kind of creative intention. Recent literature has suggested that they can. To verify this a definition of games is needed. I review and develop the most promising account of games in the literature, the over-looked account from Bernard Suits. I propose and defend a modified version of this definition against other accounts. Interestingly, this account entails that games cannot be successfully intended to be works of art because games are goal-directed activities that require a voluntary selection of inefficient means and that is incompatible with the proper manner of regarding that is necessary for something to be an artwork. While the conclusions of Part One and Part Two may appear to suggest that videogames cannot be works of art, Part Three proposes and defends a new account of videogames that, contrary to first appearances, implies that not all videogames are games. This Intentional-Historical Formalist account allows for non-game videogames to be created with an art-making intention, though not every non-ludic videogame will have an art-making intention behind it. I then discuss examples of videogames that are good candidates for being works of art. I conclude that a videogame can be a work of art, but that not all videogames are works of art. The thesis is significant in several respects. It is a continuation of academic work that has focused on the definition and art status of videogames. It clarifies the current debate and provides a positive account of the central issues that has so far been lacking. It also defines videogames in a way that corresponds better with the actual practice of videogame making and playing than other definitions in the literature. It offers further evidence in defense of certain theories of art over others, providing a close examination of videogames as a new case study for potential art objects and for aesthetic and artistic theory in general. Finally, it provides a compelling answer to the question of whether videogames can be art. This project also provides the groundwork for new evaluative, critical, and appreciative tools for engagement with videogames as they develop as a medium. As videogames mature, more people, both inside and outside academia, have increasing interest in what they are and how to understand them. One place many have looked is to the practice of art appreciation. My project helps make sense of which appreciative and art-critical tools and methods are applicable to videogames.