SMILE: Simulator for Maryland Imitation Learning Environment

As robot imitation learning is beginning to replace conventional hand-coded approaches in programming robot behaviors, much work is focusing on learning from the actions of demonstrators. We hypothesize that in many situations, procedural tasks can be learned more effectively by observing object behaviors while completely ignoring the demonstrator's motions. To support studying this hypothesis and robot imitation learning in general, we built a software system named SMILE that is a simulated 3D environment. In this virtual environment, both a simulated robot and a user-controlled demonstrator can manipulate various objects on a tabletop. The demonstrator is not embodied in SMILE, and therefore a recorded demonstration appears as if the objects move on their own. In addition to recording demonstrations, SMILE also allows programing the simulated robot via Matlab scripts, as well as creating highly customizable objects for task scenarios via XML. This report describes the features and usages of SMILE.

Lament for String Quartet

This piece explores the changing nature of emotion focusing especially on the feeling of sorrow. The opening and ending parts of the first movement represent the overall motive of sorrow. The first movement opens with an augmented chord G-C#-F-B and from this chord the first violin expands upwards while the cello moves downwards towards the C chord (p.2). As the melody alternates between each part, there is a subtle change in harmony which creates tension and release and changes the sound color. In addition, ornamentation in each part reinforces the movement towards the C chord. This progression represents the inner emotion of lament. Sostenuto e largamente section (p.2) uses heterophony in order to express a feeling of chaos. Section Scherzando (p.4) uses the interval relationship M7 and m2, and is a respite from the overwhelming feeling of sorrow. The ending of the first movement (p.12) returns to create a second tension by every instrument ascending slowly, and the viola produces a distinctive melody derived from the previous chaotic section that ends on an Ab. The second movement contrasts with the first movement in order to express a concealed, not explicit, sorrow, and differs in both tempo and texture. The tempo is a waltz that is faster than the first movement. This produces a light, playful figure and a simple melody without much ornamentation. Imitation and canonic structure emphasize the individuality of the strings. The third movement merges material from the first movement rhythmic figure and the second movement pizzicato (p.17). It shows timbral change through con sordino, pizzicato arpeggio, and sul ponticello to display string techniques. An Allegro section (p.19) especially contrasts with Misterioso in rhythm and dynamics. In the Grazioso (p.22), random beats are accentuated by pizzicato arpeggio to de-emphasize the meter. Finally, there is a return to the ending figure of the first movement with con sordino (p.23) and sul ponticello in viola that articulates the internal tension and the timbral change to return to a voice of sorrow.

THE EFFECTIVENESS OF POINT-OF-VIEW VIDEO MODELING IN TEACHING SOCIAL INITIATION SKILLS TO CHILDREN WITH AUTISM SPECTRUM DISORDERS

Deficits in social communication and interaction have been identified as distinguishing impairments for individuals with an autism spectrum disorder (ASD). As a pivotal skill, the successful development of social communication and interaction in individuals with ASD is a lifelong objective. Point-of-view video modeling has the potential to address these deficits. This type of video involves filming the completion of a targeted skill or behavior from a first-person perspective. By presenting only what a person might see from his or her viewpoint, it has been identified to be more effective in limiting irrelevant stimuli by providing a clear frame of reference to facilitate imitation. The current study investigated the use of point-of-view video modeling in teaching social initiations (e.g., greetings). Using a multiple baseline across participants design, five kindergarten participants were taught social initiations using point-of-view video modeling and video priming. Immediately before and after viewing the entire point-of-view video model, the participants were evaluated on their social initiations with a trained, typically developing peer serving as a communication partner. Specifically, the social initiations involved participants’ abilities to shift their attention toward the peer who entered the classroom, maintain attention toward the peer, and engage in an appropriate social initiation (e.g., hi, hello). Both generalization and maintenance were tested. Overall, the data suggest point-of-view video modeling is an effective intervention for increasing social initiations in young students with ASD. However, retraining was necessary for acquisition of skills in the classroom environment. Generalization in novel environments and with a novel communication partner, and generalization to other social initiation skills was limited. Additionally, maintenance of gained social initiation skills only occurred in the intervention room. Despite the limitations of the study and variable results, there are a number of implications moving forward for both practitioners and future researchers examining point-of-view modeling and its potential impact on the social initiation skills of individuals with ASD.

Essays in Market Design

In this dissertation, I study three problems in market design: the allocation of resources to schools using deferred acceptance algorithms, the demand reduction of employees on centralized labor markets, and the alleviation of traffic congestion. I show how institutional and behavioral considerations specific to each problem can alleviate several practical limitations faced by current solutions. For the case of traffic congestion, I show experimentally that the proposed solution is effective. In Chapter 1, I investigate how school districts could assign resources to schools when it is desirable to provide stable assignments. An assignment is stable if there is no student currently assigned to a school that would prefer to be assigned to a different school that would admit him if it had the resources. Current assignment algorithms assume resources are fixed. I show how simple modifications to these algorithms produce stable allocations of resources and students to schools. In Chapter 2, I show how the negotiation of salaries within centralized labor markets using deferred acceptance algorithms eliminates the incentives of the hiring firms to strategically reduce their demand. It is well-known that it is impossible to eliminate these incentives for the hiring firms in markets without negotiation of salaries. Chapter 3 investigates how to achieve an efficient distribution of traffic congestion on a road network. Traffic congestion is the product of an externality: drivers do not consider the cost they impose on other drivers by entering a road. In theory, Pigouvian prices would solve the problem. In practice, however, these prices face two important limitations: i) the information required to calculate these prices is unavailable to policy makers and ii) these prices would effectively be new taxes that would transfer resources from the public to the government. I show how to construct congestion prices that retrieve the required information from the drivers and do not transfer resources to the government. I circumvent the limitations of Pigouvian prices by assuming that individuals make some mistakes when selecting routes and have a tendency towards truth-telling. Both assumptions are very robust observations in experimental economics.

SEQUENTIAL DECISIONS AND PREDICTIONS IN NATURAL LANGUAGE PROCESSING

Natural language processing has achieved great success in a wide range of ap- plications, producing both commercial language services and open-source language tools. However, most methods take a static or batch approach, assuming that the model has all information it needs and makes a one-time prediction. In this disser- tation, we study dynamic problems where the input comes in a sequence instead of all at once, and the output must be produced while the input is arriving. In these problems, predictions are often made based only on partial information. We see this dynamic setting in many real-time, interactive applications. These problems usually involve a trade-off between the amount of input received (cost) and the quality of the output prediction (accuracy). Therefore, the evaluation considers both objectives (e.g., plotting a Pareto curve). Our goal is to develop a formal understanding of sequential prediction and decision-making problems in natural language processing and to propose efficient solutions. Toward this end, we present meta-algorithms that take an existent batch model and produce a dynamic model to handle sequential inputs and outputs. Webuild our framework upon theories of Markov Decision Process (MDP), which allows learning to trade off competing objectives in a principled way. The main machine learning techniques we use are from imitation learning and reinforcement learning, and we advance current techniques to tackle problems arising in our settings. We evaluate our algorithm on a variety of applications, including dependency parsing, machine translation, and question answering. We show that our approach achieves a better cost-accuracy trade-off than the batch approach and heuristic-based decision- making approaches. We first propose a general framework for cost-sensitive prediction, where dif- ferent parts of the input come at different costs. We formulate a decision-making process that selects pieces of the input sequentially, and the selection is adaptive to each instance. Our approach is evaluated on both standard classification tasks and a structured prediction task (dependency parsing). We show that it achieves similar prediction quality to methods that use all input, while inducing a much smaller cost. Next, we extend the framework to problems where the input is revealed incremen- tally in a fixed order. We study two applications: simultaneous machine translation and quiz bowl (incremental text classification). We discuss challenges in this set- ting and show that adding domain knowledge eases the decision-making problem. A central theme throughout the chapters is an MDP formulation of a challenging problem with sequential input/output and trade-off decisions, accompanied by a learning algorithm that solves the MDP.