Understanding information use in multiattribute decision making

An inference task in one in which some known set of information is used to produce an estimate about an unknown quantity. Existing theories of how humans make inferences include specialized heuristics that allow people to make these inferences in familiar environments quickly and without unnecessarily complex computation. Specialized heuristic processing may be unnecessary, however; other research suggests that the same patterns in judgment can be explained by existing patterns in encoding and retrieving memories. This dissertation compares and attempts to reconcile three alternate explanations of human inference. After justifying three hierarchical Bayesian version of existing inference models, the three models are com- pared on simulated, observed, and experimental data. The results suggest that the three models capture different patterns in human behavior but, based on posterior prediction using laboratory data, potentially ignore important determinants of the decision process.

SEQUENTIAL DECISIONS AND PREDICTIONS IN NATURAL LANGUAGE PROCESSING

Natural language processing has achieved great success in a wide range of ap- plications, producing both commercial language services and open-source language tools. However, most methods take a static or batch approach, assuming that the model has all information it needs and makes a one-time prediction. In this disser- tation, we study dynamic problems where the input comes in a sequence instead of all at once, and the output must be produced while the input is arriving. In these problems, predictions are often made based only on partial information. We see this dynamic setting in many real-time, interactive applications. These problems usually involve a trade-off between the amount of input received (cost) and the quality of the output prediction (accuracy). Therefore, the evaluation considers both objectives (e.g., plotting a Pareto curve). Our goal is to develop a formal understanding of sequential prediction and decision-making problems in natural language processing and to propose efficient solutions. Toward this end, we present meta-algorithms that take an existent batch model and produce a dynamic model to handle sequential inputs and outputs. Webuild our framework upon theories of Markov Decision Process (MDP), which allows learning to trade off competing objectives in a principled way. The main machine learning techniques we use are from imitation learning and reinforcement learning, and we advance current techniques to tackle problems arising in our settings. We evaluate our algorithm on a variety of applications, including dependency parsing, machine translation, and question answering. We show that our approach achieves a better cost-accuracy trade-off than the batch approach and heuristic-based decision- making approaches. We first propose a general framework for cost-sensitive prediction, where dif- ferent parts of the input come at different costs. We formulate a decision-making process that selects pieces of the input sequentially, and the selection is adaptive to each instance. Our approach is evaluated on both standard classification tasks and a structured prediction task (dependency parsing). We show that it achieves similar prediction quality to methods that use all input, while inducing a much smaller cost. Next, we extend the framework to problems where the input is revealed incremen- tally in a fixed order. We study two applications: simultaneous machine translation and quiz bowl (incremental text classification). We discuss challenges in this set- ting and show that adding domain knowledge eases the decision-making problem. A central theme throughout the chapters is an MDP formulation of a challenging problem with sequential input/output and trade-off decisions, accompanied by a learning algorithm that solves the MDP.

DEVELOPMENT OF A DECISION/DEPLOYMENT SUPPORT TOOL FOR VARIABLE SPEED LIMITS IN RECURRENT CONGESTION

Traffic demand increases are pushing aging ground transportation infrastructures to their theoretical capacity. The result of this demand is traffic bottlenecks that are a major cause of delay on urban freeways. In addition, the queues associated with those bottlenecks increase the probability of a crash while adversely affecting environmental measures such as emissions and fuel consumption. With limited resources available for network expansion, traffic professionals have developed active traffic management systems (ATMS) in an attempt to mitigate the negative consequences of traffic bottlenecks. Among these ATMS strategies, variable speed limits (VSL) and ramp metering (RM) have been gaining international interests for their potential to improve safety, mobility, and environmental measures at freeway bottlenecks. Though previous studies have shown the tremendous potential of variable speed limit (VSL) and VSL paired with ramp metering (VSLRM) control, little guidance has been developed to assist decision makers in the planning phase of a congestion mitigation project that is considering VSL or VSLRM control. To address this need, this study has developed a comprehensive decision/deployment support tool for the application of VSL and VSLRM control in recurrently congested environments. The decision tool will assist practitioners in deciding the most appropriate control strategy at a candidate site, which candidate sites have the most potential to benefit from the suggested control strategy, and how to most effectively design the field deployment of the suggested control strategy at each implementation site. To do so, the tool is comprised of three key modules, (1) Decision Module, (2) Benefits Module, and (3) Deployment Guidelines Module. Each module uses commonly known traffic flow and geometric parameters as inputs to statistical models and empirically based procedures to provide guidance on the application of VSL and VSLRM at each candidate site. These models and procedures were developed from the outputs of simulated experiments, calibrated with field data. To demonstrate the application of the tool, a list of real-world candidate sites were selected from the Maryland State Highway Administration Mobility Report. Here, field data from each candidate site was input into the tool to illustrate the step-by-step process required for efficient planning of VSL or VSLRM control. The output of the tool includes the suggested control system at each site, a ranking of the sites based on the expected benefit-to-cost ratio, and guidelines on how to deploy the VSL signs, ramp meters, and detectors at the deployment site(s). This research has the potential to assist traffic engineers in the planning of VSL and VSLRM control, thus enhancing the procedure for allocating limited resources for mobility and safety improvements on highways plagued by recurrent congestion.

SCHOOL DISTRICT GOVERNANCE AND KNOWLEDGE-FIT IN DECISION RIGHTS: HOW DISTRICTS RECRUIT AND HIRE SCHOOL LIBRARIANS

This study examines the organizational structures and decision-making processes used by school districts to recruit and hire school librarians. For students to acquire the information and technology literacy education they need, school libraries must be staffed with qualified individuals who can fulfill the librarian’s role as leader, teacher, instructional partner, information specialist, and program administrator. Principals are typically given decision rights for hiring staff, including school librarians. Research shows that principals have limited knowledge of the skills and abilities of the school librarian or the specific needs and functions of the library program. Research also indicates that those with specific knowledge of school library programs, namely school district library supervisors, are only consulted on recruiting and hiring about half the time. School districts entrust library supervisors with responsibilities such as professional development of school librarians only after they are hired. This study uses a theoretical lens from research on IT governance, which focuses on the use of knowledge-fit in applying decision rights in an organization. This framework is appropriate because of its incorporation of a specialist with a specific knowledge set in determining the placement of input and decision rights in the decision-making processes. The method used in this research was a multiple-case study design using five school districts as cases, varying by the involvement of the supervisors and other individuals in the hiring process. The data collected from each school district were interviews about the district’s recruiting and hiring practices with principals, an individual in HR, library supervisors, and recently hired school librarians. Data analysis was conducted through iterative coding from themes in the research questions, with continuous adjustments as new themes developed. Results from the study indicate that governance framework is applicable to evaluating the decision-making processes used in recruiting and hiring school librarians. However, a district’s use of governance did not consistently use knowledge-fit in the determination of input and decision rights. In the hiring process, governance was more likely to be based on placing decision rights at a certain level of the district hierarchy rather than the location of specific knowledge, most often resulting in site-based governance for decision rights at the school-building level. The governance of the recruiting process was most affected by the shortage or surplus of candidates available to the district to fill positions. Districts struggling with a shortage of candidates typically placed governance for the decision-making process on recruiting at the district level, giving the library supervisor more opportunity for input and collaboration with human resources. In districts that use site-based governance and that place all input and decision rights at the building level, some principals use their autonomy to eliminate the school library position in the allotment phase or hire librarians that, while certified through testing, do not have the same level of expertise as those who achieve certification through LIS programs. The principals in districts who use site-based governance for decision rights but call on the library supervisor for advisement stated how valuable they found the supervisor’s expertise in evaluating candidates for hire. In no district was a principal or school required to involve the library supervisor in the hiring of school librarians. With a better understanding of the tasks involved, the effect of district governance on decision-making, and the use of knowledge to assign input and decision rights, it is possible to look at how all of these factors affect the outcome in the quality of the hire. A next step is to look at the hiring process that school librarians went through and connect those with the measurable outcomes of hiring: school librarian success, retention, and attrition; the quality of school library program services, outreach, and involvement in a school; and the perceptions of the success of the school librarian and the library program as seen from students, teachers, administrators, parents, and other community stakeholders.