An Electronic Market-Maker

This paper presents an adaptive learning model for market-making under the reinforcement learning framework. Reinforcement learning is a learning technique in which agents aim to maximize the long-term accumulated rewards. No knowledge of the market environment, such as the order arrival or price process, is assumed. Instead, the agent learns from real-time market experience and develops explicit market-making strategies, achieving multiple objectives including the maximizing of profits and minimization of the bid-ask spread. The simulation results show initial success in bringing learning techniques to building market-making algorithms.

Modeling Stock Order Flows and Learning Market-Making from Data

Stock markets employ specialized traders, market-makers, designed to provide liquidity and volume to the market by constantly supplying both supply and demand. In this paper, we demonstrate a novel method for modeling the market as a dynamic system and a reinforcement learning algorithm that learns profitable market-making strategies when run on this model. The sequence of buys and sells for a particular stock, the order flow, we model as an Input-Output Hidden Markov Model fit to historical data. When combined with the dynamics of the order book, this creates a highly non-linear and difficult dynamic system. Our reinforcement learning algorithm, based on likelihood ratios, is run on this partially-observable environment. We demonstrate learning results for two separate real stocks.

Workforce Issues in the Greater Boston Health Care Industry: Implications for Work and Family

Interviews with more than 40 leaders in the Boston area health care industry have identified a range of broadly-felt critical problems. This document synthesizes these problems and places them in the context of work and family issues implicit in the organization of health care workplaces. It concludes with questions about possible ways to address such issues. The defining circumstance for the health care industry nationally as well as regionally at present is an extraordinary reorganization, not yet fully negotiated, in the provision and financing of health care. Hoped-for controls on increased costs of medical care – specifically the widespread replacement of indemnity insurance by market-based managed care and business models of operation--have fallen far short of their promise. Pressures to limit expenditures have produced dispiriting conditions for the entire healthcare workforce, from technicians and aides to nurses and physicians. Under such strains, relations between managers and workers providing care are uneasy, ranging from determined efforts to maintain respectful cooperation to adversarial negotiation. Taken together, the interviews identify five key issues affecting a broad cross-section of occupational groups, albeit in different ways: Staffing shortages of various kinds throughout the health care workforce create problems for managers and workers and also for the quality of patient care. Long work hours and inflexible schedules place pressure on virtually every part of the healthcare workforce, including physicians. Degraded and unsupportive working conditions, often the result of workplace "deskilling" and "speed up," undercut previous modes of clinical practice. Lack of opportunities for training and advancement exacerbate workforce problems in an industry where occupational categories and terms of work are in a constant state of flux. Professional and employee voices are insufficiently heard in conditions of rapid institutional reorganization and consolidation. Interviewees describe multiple impacts of these issues--on the operation of health care workplaces, on the well being of the health care workforce, and on the quality of patient care. Also apparent in the interviews, but not clearly named and defined, is the impact of these issues on the ability of workers to attend well to the needs of their families--and the reciprocal impact of workers' family tensions on workplace performance. In other words, the same things that affect patient care also affect families, and vice versa. Some workers describe feeling both guilty about raising their own family issues when their patients' needs are at stake, and resentful about the exploitation of these feelings by administrators making workplace policy. The different institutions making up the health care system have responded to their most pressing issues with a variety of specific stratagems but few that address the complexities connecting relations between work and family. The MIT Workplace Center proposes a collaborative exploration of next steps to probe these complications and to identify possible locations within the health care system for workplace experimentation with outcomes benefiting all parties.