Exact Solution of Bayes and Minimax Change-Detection Problems

The challenge of detecting a change in the distribution of data is a sequential decision problem that is relevant to many engineering solutions, including quality control and machine and process monitoring. This dissertation develops techniques for exact solution of change-detection problems with discrete time and discrete observations. Change-detection problems are classified as Bayes or minimax based on the availability of information on the change-time distribution. A Bayes optimal solution uses prior information about the distribution of the change time to minimize the expected cost, whereas a minimax optimal solution minimizes the cost under the worst-case change-time distribution. Both types of problems are addressed. The most important result of the dissertation is the development of a polynomial-time algorithm for the solution of important classes of Markov Bayes change-detection problems. Existing techniques for epsilon-exact solution of partially observable Markov decision processes have complexity exponential in the number of observation symbols. A new algorithm, called constellation induction, exploits the concavity and Lipschitz continuity of the value function, and has complexity polynomial in the number of observation symbols. It is shown that change-detection problems with a geometric change-time distribution and identically- and independently-distributed observations before and after the change are solvable in polynomial time. Also, change-detection problems on hidden Markov models with a fixed number of recurrent states are solvable in polynomial time. A detailed implementation and analysis of the constellation-induction algorithm are provided. Exact solution methods are also established for several types of minimax change-detection problems. Finite-horizon problems with arbitrary observation distributions are modeled as extensive-form games and solved using linear programs. Infinite-horizon problems with linear penalty for detection delay and identically- and independently-distributed observations can be solved in polynomial time via epsilon-optimal parameterization of a cumulative-sum procedure. Finally, the properties of policies for change-detection problems are described and analyzed. Simple classes of formal languages are shown to be sufficient for epsilon-exact solution of change-detection problems, and methods for finding minimally sized policy representations are described.

Gait Regulation for Bipedal Locomotion

This work explores regulation of forward speed, step length, and slope walking for the passive-dynamic class of bipedal robots. Previously, an energy-shaping control for regulating forward speed has appeared in the literature; here we show that control to be a special case of a more general time-scaling control that allows for speed transitions in arbitrary time. As prior work has focused on potential energy shaping for fully actuated bipeds, we study in detail the shaping of kinetic energy for bipedal robots, giving special treatment to issues of underactuation. Drawing inspiration from features of human walking, an underactuated kinetic-shaping control is presented that provides efficient regulation of walking speed while adjusting step length. Previous results on energetic symmetries of bipedal walking are also extended, resulting in a control that allows regulation of speed and step length while walking on any slope. Finally we formalize the optimal gait regulation problem and propose a dynamic programming solution seeded with passive-dynamic limit cycles. Observations of the optimal solutions generated by this method reveal further similarities between passive dynamic walking and human locomotion and give insight into the structure of minimum-effort controls for walking.

Quantification of the human postural control system to perturbations

Human standing posture is inherently unstable. The postural control system (PCS), which maintains standing posture, is composed of the sensory, musculoskeletal, and central nervous systems. Together these systems integrate sensory afferents and generate appropriate motor efferents to adjust posture. The PCS maintains the body center of mass (COM) with respect to the base of support while constantly resisting destabilizing forces from internal and external perturbations. To assess the human PCS, postural sway during quiet standing or in response to external perturbation have frequently been examined descriptively. Minimal work has been done to understand and quantify the robustness of the PCS to perturbations. Further, there have been some previous attempts to assess the dynamical systems aspects of the PCS or time evolutionary properties of postural sway. However those techniques can only provide summary information about the PCS characteristics; they cannot provide specific information about or recreate the actual sway behavior. This dissertation consists of two parts: part I, the development of two novel methods to assess the human PCS and, part II, the application of these methods. In study 1, a systematic method for analyzing the human PCS during perturbed stance was developed. A mild impulsive perturbation that subjects can easily experience in their daily lives was used. A measure of robustness of the PCS, 1/MaxSens that was based on the inverse of the sensitivity of the system, was introduced. 1/MaxSens successfully quantified the reduced robustness to external perturbations due to age-related degradation of the PCS. In study 2, a stochastic model was used to better understand the human PCS in terms of dynamical systems aspect. This methodology also has the advantage over previous methods in that the sway behavior is captured in a model that can be used to recreate the random oscillatory properties of the PCS. The invariant density which describes the long-term stationary behavior of the center of pressure (COP) was computed from a Markov chain model that was applied to postural sway data during quiet stance. In order to validate the Invariant Density Analysis (IDA), we applied the technique to COP data from different age groups. We found that older adults swayed farther from the centroid and in more stochastic and random manner than young adults. In part II, the tools developed in part I were applied to both occupational and clinical situations. In study 3, 1/MaxSens and IDA were applied to a population of firefighters to investigate the effects of air bottle configuration (weight and size) and vision on the postural stability of firefighters. We found that both air bottle weight and loss of vision, but not size of air bottle, significantly decreased balance performance and increased fall risk. In study 4, IDA was applied to data collected on 444 community-dwelling elderly adults from the MOBILIZE Boston Study. Four out of five IDA parameters were able to successfully differentiate recurrent fallers from non-fallers, while only five out of 30 more common descriptive and stochastic COP measures could distinguish the two groups. Fall history and the IDA parameter of entropy were found to be significant risk factors for falls. This research proposed a new measure for the PCS robustness (1/MaxSens) and a new technique for quantifying the dynamical systems aspect of the PCS (IDA). These new PCS analysis techniques provide easy and effective ways to assess the PCS in occupational and clinical environments.

Coordinated pricing and inventory management

This dissertation mainly focuses on coordinated pricing and inventory management problems, where the related background is provided in Chapter 1. Several periodic-review models are then discussed in Chapters 2,3,4 and 5, respectively. Chapter 2 analyzes a deterministic single-product model, where a price adjustment cost incurs if the current selling price is changed from the previous period. We develop exact algorithms for the problem under different conditions and find out that computation complexity varies significantly associated with the cost structure. %Moreover, our numerical study indicates that dynamic pricing strategies may outperform static pricing strategies even when price adjustment cost accounts for a significant portion of the total profit. Chapter 3 develops a single-product model in which demand of a period depends not only on the current selling price but also on past prices through the so-called reference price. Strongly polynomial time algorithms are designed for the case without no fixed ordering cost, and a heuristic is proposed for the general case together with an error bound estimation. Moreover, our illustrates through numerical studies that incorporating reference price effect into coordinated pricing and inventory models can have a significant impact on firms' profits. Chapter 4 discusses the stochastic version of the model in Chapter 3 when customers are loss averse. It extends the associated results developed in literature and proves that the reference price dependent base-stock policy is proved to be optimal under a certain conditions. Instead of dealing with specific problems, Chapter 5 establishes the preservation of supermodularity in a class of optimization problems. This property and its extensions include several existing results in the literature as special cases, and provide powerful tools as we illustrate their applications to several operations problems: the stochastic two-product model with cross-price effects, the two-stage inventory control model, and the self-financing model.