Modeling Stock Order Flows and Learning Market-Making from Data

Stock markets employ specialized traders, market-makers, designed to provide liquidity and volume to the market by constantly supplying both supply and demand. In this paper, we demonstrate a novel method for modeling the market as a dynamic system and a reinforcement learning algorithm that learns profitable market-making strategies when run on this model. The sequence of buys and sells for a particular stock, the order flow, we model as an Input-Output Hidden Markov Model fit to historical data. When combined with the dynamics of the order book, this creates a highly non-linear and difficult dynamic system. Our reinforcement learning algorithm, based on likelihood ratios, is run on this partially-observable environment. We demonstrate learning results for two separate real stocks.

An Extension to the Tactical Planning Model for a Job Shop: Continuous-Time Control

We develop an extension to the tactical planning model (TPM) for a job shop by the third author. The TPM is a discrete-time model in which all transitions occur at the start of each time period. The time period must be defined appropriately in order for the model to be meaningful. Each period must be short enough so that a job is unlikely to travel through more than one station in one period. At the same time, the time period needs to be long enough to justify the assumptions of continuous workflow and Markovian job movements. We build an extension to the TPM that overcomes this restriction of period sizing by permitting production control over shorter time intervals. We achieve this by deriving a continuous-time linear control rule for a single station. We then determine the first two moments of the production level and queue length for the workstation.

Corner Flows in Free Liquid Films

A lubrication-flow model for a free film in a corner is presented. The model, written in the hyperbolic coordinate system ξ = x² – y², η = 2xy, applies to films that are thin in the η direction. The lubrication approximation yields two coupled evolution equations for the film thickness and the velocity field which, to lowest order, describes plug flow in the hyperbolic coordinates. A free film in a corner evolving under surface tension and gravity is investigated. The rate of thinning of a free film is compared to that of a film evolving over a solid substrate. Viscous shear and normal stresses are both captured in the model and are computed for the entire flow domain. It is shown that normal stress dominates over shear stress in the far field, while shear stress dominates close to the corner.