Efficient Image and Video Representations for Retrieval

Image (Video) retrieval is an interesting problem of retrieving images (videos) similar to the query. Images (Videos) are represented in an input (feature) space and similar images (videos) are obtained by finding nearest neighbors in the input representation space. Numerous input representations both in real valued and binary space have been proposed for conducting faster retrieval. In this thesis, we present techniques that obtain improved input representations for retrieval in both supervised and unsupervised settings for images and videos. Supervised retrieval is a well known problem of retrieving same class images of the query. We address the practical aspects of achieving faster retrieval with binary codes as input representations for the supervised setting in the first part, where binary codes are used as addresses into hash tables. In practice, using binary codes as addresses does not guarantee fast retrieval, as similar images are not mapped to the same binary code (address). We address this problem by presenting an efficient supervised hashing (binary encoding) method that aims to explicitly map all the images of the same class ideally to a unique binary code. We refer to the binary codes of the images as `Semantic Binary Codes' and the unique code for all same class images as `Class Binary Code'. We also propose a new class­ based Hamming metric that dramatically reduces the retrieval times for larger databases, where only hamming distance is computed to the class binary codes. We also propose a Deep semantic binary code model, by replacing the output layer of a popular convolutional Neural Network (AlexNet) with the class binary codes and show that the hashing functions learned in this way outperforms the state­ of ­the art, and at the same time provide fast retrieval times. In the second part, we also address the problem of supervised retrieval by taking into account the relationship between classes. For a given query image, we want to retrieve images that preserve the relative order i.e. we want to retrieve all same class images first and then, the related classes images before different class images. We learn such relationship aware binary codes by minimizing the similarity between inner product of the binary codes and the similarity between the classes. We calculate the similarity between classes using output embedding vectors, which are vector representations of classes. Our method deviates from the other supervised binary encoding schemes as it is the first to use output embeddings for learning hashing functions. We also introduce new performance metrics that take into account the related class retrieval results and show significant gains over the state­ of­ the art. High Dimensional descriptors like Fisher Vectors or Vector of Locally Aggregated Descriptors have shown to improve the performance of many computer vision applications including retrieval. In the third part, we will discuss an unsupervised technique for compressing high dimensional vectors into high dimensional binary codes, to reduce storage complexity. In this approach, we deviate from adopting traditional hyperplane hashing functions and instead learn hyperspherical hashing functions. The proposed method overcomes the computational challenges of directly applying the spherical hashing algorithm that is intractable for compressing high dimensional vectors. A practical hierarchical model that utilizes divide and conquer techniques using the Random Select and Adjust (RSA) procedure to compress such high dimensional vectors is presented. We show that our proposed high dimensional binary codes outperform the binary codes obtained using traditional hyperplane methods for higher compression ratios. In the last part of the thesis, we propose a retrieval based solution to the Zero shot event classification problem - a setting where no training videos are available for the event. To do this, we learn a generic set of concept detectors and represent both videos and query events in the concept space. We then compute similarity between the query event and the video in the concept space and videos similar to the query event are classified as the videos belonging to the event. We show that we significantly boost the performance using concept features from other modalities.

OPERA IN AMERICA: MUSIC OF, BY, AND FOR THE PEOPLE­: A PERFORMANCE DISSERTATION

Opera in America: Music of, by, and for the people is a study of the relationship between American popular culture and opera in the United States. Four performance projects demonstrate the on-going exchange between the operatic community-including its composer, singers, and patrons-and the country's popular entertainment industry with its broad audience base. Numerous examples of artistic cross pollination between lowbrow and highbrow music will illustrate the artistic and social consequences created by this artistic amalgamation. Program #1, By George! By Ira! By Gershwin!, is a retrospective of Gershwin's vocal music representing a blending of popular and serious music in both style and form. The concert includes selections from Porgy and Bess, a work considered by many musicologists as the first American opera. Program #2, Shadowboxer, is a premiere performance of an opera by Frank Proto and John Chenault. For this newly commissioned work, I serve as Assistant Director to Leon Major. Shadowboxer provides a clear example of opera utilizing popular culture both musically and dramatically to tell the true story of American hero and legendary boxer, Joe Louis. Program #3, Just a Song at Twilight, is an original theatrical music piece featuring music, letters, diaries, and journals of the Gilded Age, an era when opera was synonymous with popular entertainment. Special attention is focused on tum-of­ the-century singers who performed in both opera and vaudeville. Program #4 is a presentation of Dominick Argento's Miss Manners on Music and illustrates the strong relationship that can exist between opera and American popular entertainment. Originally conceived as a song cycle, I have staged the work as a one-act opera sung and acted by soprano Carmen Balthrop. This piece is based on the writings of pop icon and newspaper columnist Judith Martin, otherwise known as Miss Manners. All four performances are recorded in audio and video formats.

MUSICAL IMPROVISATION AND THE CREATIVE PROCESS

Musical improvisation combines technical proficiency and musical intuition. Due to its interactive nature, improvisation provides an avenue of communication among all art forms. This dissertation project explores the collaborative aspects of improvisation involving a musician, visual artist, a small group of dancers, and videographer. Video footage from two separate recording sessions provided hours of visual materials which were studied and edited. The first session was a live performance recorded in front of a studio audience. The second session was a two-day collaboration between musician and dancers in a studio space. The process of editing and compiling images with audio-an important element in this project-presented many unforeseeable challenges and lessons. This recorded dissertation is comprised of seven music videos that demonstrate my ability as an artist in collaboration with visual artist-professor Richard Klank, dancers David Yates, Jamie Garcia, Raha Behnam, Rachel Wolfe and Adrian Galvin, and video artist Nguyen Nguyen. Each video represents an individual creative process involving musical performance, studio lighting, sound recording, and video editing.

The Legacy of Oboist and Master Teacher, Robert Bloom

Robert Bloom (1908-1994) was legendary in the education and performance world. Often hailed as one of the last performers of the Golden Era of classical music and a favorite of conductors ranging from Stokowski to Stravinsky to Shaw, Bloom was an orchestral oboist and English hornist, oboe soloist, chamber musician, teacher (Eastman, Yale, Hartt, Manhattan School of Music, Juilliard and Philadelphia's University of the Arts), composer, conductor, editor of masterworks of the 18th century, and, as a founding member of the Bach Aria group, a seminal influence in the post-WWII revival of Baroque music in America. In The Robert Bloom Collection and the Art of Robert Bloom CD and video archives, we see what his musical ideals were in 1)18th-century performance practices, 2) writing new music for the instrument and commissioning new works, and 3) and transcribing music for the oboe and English horn. As an oboist, I believe it is important that Bloom's teachings, historical performance practices and ideas for expanding repertoire are propagated. Therefore, the works chosen for this dissertation illustrated this legacy. My recitals included 1) some of Bloom's published 18th-century baroque elaborations (his term for ornamentation), as well Baroque works which I have elaborated, 2) works written by him and by other oboists/composers (Labate, Roseman) as well as a flute/oboe duo that I commissioned by Dr. Marcus Maroney and 3) transcriptions by both Bloom and myself (Bach, Donizetti, Mendelssohn, Mozart, Handel, Schumann and Telemann). In these three dissertation recitals, I hope to have illustrated some of Robert Bloom's lasting contributions and impact on the oboe world, and to have demonstrated the potential for carrying forward this legacy by studying his teaching and emulating his example.

THE EFFECTIVENESS OF POINT-OF-VIEW VIDEO MODELING IN TEACHING SOCIAL INITIATION SKILLS TO CHILDREN WITH AUTISM SPECTRUM DISORDERS

Deficits in social communication and interaction have been identified as distinguishing impairments for individuals with an autism spectrum disorder (ASD). As a pivotal skill, the successful development of social communication and interaction in individuals with ASD is a lifelong objective. Point-of-view video modeling has the potential to address these deficits. This type of video involves filming the completion of a targeted skill or behavior from a first-person perspective. By presenting only what a person might see from his or her viewpoint, it has been identified to be more effective in limiting irrelevant stimuli by providing a clear frame of reference to facilitate imitation. The current study investigated the use of point-of-view video modeling in teaching social initiations (e.g., greetings). Using a multiple baseline across participants design, five kindergarten participants were taught social initiations using point-of-view video modeling and video priming. Immediately before and after viewing the entire point-of-view video model, the participants were evaluated on their social initiations with a trained, typically developing peer serving as a communication partner. Specifically, the social initiations involved participants’ abilities to shift their attention toward the peer who entered the classroom, maintain attention toward the peer, and engage in an appropriate social initiation (e.g., hi, hello). Both generalization and maintenance were tested. Overall, the data suggest point-of-view video modeling is an effective intervention for increasing social initiations in young students with ASD. However, retraining was necessary for acquisition of skills in the classroom environment. Generalization in novel environments and with a novel communication partner, and generalization to other social initiation skills was limited. Additionally, maintenance of gained social initiation skills only occurred in the intervention room. Despite the limitations of the study and variable results, there are a number of implications moving forward for both practitioners and future researchers examining point-of-view modeling and its potential impact on the social initiation skills of individuals with ASD.

Viral Bodies: Uncontrollable Blackness in Popular Culture and Everyday Life

Viral Bodies: Uncontrollable Blackness in Popular Culture and Everyday Life maps rapidly circulated performances of Blackness across visual media that collapse Black bodies into ubiquitous “things.” Throughout my dissertation, I use viral performance to describe the uncontrollable discursive circulation of bodies, their behaviors, and the ideas around them. In particular, viral performance is employed to describe the complicated ways that (mis)understandings of Black bodies spread and are often transformed into common-sense beliefs. As viral performances, Black bodies are often made more visible, while simultaneously becoming more opaque. This dissertation examines the recurrence of viral performances of Blackness in viral videos online, film, and photography/images. I argue that viral performances make products that reinscribe stereotypical notions of Blackness while also generating paths of alterity—which contradict the normalized clichés and provide desirable possibilities for Black performance. Viral Bodies forges a new dialogue between visual and aural technologies, performance, and larger historic discourses that script Black bodies as visually (and sonically) deviant subjects. I am interested in how technologies complicate the re-presentation of images, ideas, and ideologies—producing a necessity for new decipherings of performances of Blackness in popular culture and everyday life.

Move, for string quartet, piano, percussion and electronics

MOVE is a composition for string quartet, piano, percussion and electronics of approximately 15-16 minutes duration in three movements. The work incorporates electronic samples either synthesized electronically by the composer or recorded from acoustic instruments. The work aims to use electronic sounds as an expansion of the tonal palette of the chamber group (rather like an extended percussion setup) as opposed to a dominating sonic feature of the music. This is done by limiting the use of electronics to specific sections of the work, and by prioritizing blend and sonic coherence in the synthesized samples. The work uses fixed electronics in such a way that allows for tempo variations in the music. Generally, a difficulty arises in that fixed “tape” parts don’t allow tempo variations; while truly “live” software algorithms sacrifice rhythmic accuracy. Sample pads, such as the Roland SPD-SX, provide an elegant solution. The latency of such a device is close enough to zero that individual samples can be triggered in real time at a range of tempi. The percussion setup in this work (vibraphone and sample pad) allows one player to cover both parts, eliminating the need for an external musician to trigger the electronics. Compositionally, momentum is used as a constructing principle. The first movement makes prominent use of ostinato and shifting meter. The second is a set of variations on a repeated harmonic pattern, with a polymetric middle section. The third is a type of passacaglia, wherein the bassline is not introduced right away, but becomes more significant later in the movement. Given the importance of visual presentation in the Internet age, the final goal of the project was to shoot HD video of a studio performance of the work for publication online. The composer recorded audio and video in two separate sessions and edited the production using Logic X and Adobe Premiere Pro. The final video presentation can be seen at geoffsheil.com/move.

Heat Transfer Measurements in a Supersonic Film Flow

This thesis presents measurements of wall heat flux and flow structure in a canonical film cooling configuration with Mach 2.3 core flow in which the coolant is injected parallel to the wall through a two-dimensional louver. Four operating conditions are investigated: no film (i.e. flow over a rearward-facing step), subsonic film, pressure-matched film, and supersonic film. The overall objective is to provide a set of experimental data with well characterized boundary conditions that can be used for code validation. The results are compared to RANS and LES simulations which overpredict heat transfer in the subsonic film cases and underpredict heat transfer in supersonic cases after film breakdown. The thesis also describes a number of improvements that were made to the experimental facility including new Schlieren optics, a better film heater, more data at more locations, and a verification of the heat flux measurement hardware and data reduction methods.

EXAMINING SECONDARY MATHEMATICS TEACHER CANDIDATES’ LEARNING AND ENACTMENT OF MATHEMATICS TEACHING PRACTICES: A MULTIPLE CASE STUDY

This qualitative case study explored three teacher candidates’ learning and enactment of discourse-focused mathematics teaching practices. Using audio and video recordings of their teaching practice this study aimed to identify the shifts in the way in which the teacher candidates enacted the following discourse practices: elicited and used evidence of student thinking, posed purposeful questions, and facilitated meaningful mathematical discourse. The teacher candidates’ written reflections from their practice-based coursework as well as interviews were examined to see how two mathematics methods courses influenced their learning and enactment of the three discourse focused mathematics teaching practices. These data sources were also used to identify tensions the teacher candidates encountered. All three candidates in the study were able to successfully enact and reflect on these discourse-focused mathematics teaching practices at various time points in their preparation programs. Consistency of use and areas of improvement differed, however, depending on various tensions experienced by each candidate. Access to quality curriculum materials as well as time to formulate and enact thoughtful lesson plans that supported classroom discourse were tensions for these teacher candidates. This study shows that teacher candidates are capable of enacting discourse-focused teaching practices early in their field placements and with the support of practice-based coursework they can analyze and reflect on their practice for improvement. This study also reveals the importance of assisting teacher candidates in accessing rich mathematical tasks and collaborating during lesson planning. More research needs to be explored to identify how specific aspects of the learning cycle impact individual teachers and how this can be used to improve practice-based teacher education courses.