Coordinating Inventory Control and Pricing Strategies with Random Demand and Fixed Ordering Cost: The Finite Horizon Case

We analyze a finite horizon, single product, periodic review model in which pricing and production/inventory decisions are made simultaneously. Demands in different periods are random variables that are independent of each other and their distributions depend on the product price. Pricing and ordering decisions are made at the beginning of each period and all shortages are backlogged. Ordering cost includes both a fixed cost and a variable cost proportional to the amount ordered. The objective is to find an inventory policy and a pricing strategy maximizing expected profit over the finite horizon. We show that when the demand model is additive, the profit-to-go functions are k-concave and hence an (s,S,p) policy is optimal. In such a policy, the period inventory is managed based on the classical (s,S) policy and price is determined based on the inventory position at the beginning of each period. For more general demand functions, i.e., multiplicative plus additive functions, we demonstrate that the profit-to-go function is not necessarily k-concave and an (s,S,p) policy is not necessarily optimal. We introduce a new concept, the symmetric k-concave functions and apply it to provide a characterization of the optimal policy.

Risk Aversion in Inventory Management

Traditional inventory models focus on risk-neutral decision makers, i.e., characterizing replenishment strategies that maximize expected total profit, or equivalently, minimize expected total cost over a planning horizon. In this paper, we propose a framework for incorporating risk aversion in multi-period inventory models as well as multi-period models that coordinate inventory and pricing strategies. In each case, we characterize the optimal policy for various measures of risk that have been commonly used in the finance literature. In particular, we show that the structure of the optimal policy for a decision maker with exponential utility functions is almost identical to the structure of the optimal risk-neutral inventory (and pricing) policies. Computational results demonstrate the importance of this approach not only to risk-averse decision makers, but also to risk-neutral decision makers with limited information on the demand distribution.

Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Should educational policies be regressive?

In this paper, we show that when the government is able to transfer wealth between generations, regressive policies are no longer optimal. The optimal educational policy can be decentralized through appropriate Pigouvian taxes and credit provision, is not regressive, and provides equality of opportunities in education (in the sense of irrelevance of parental income for the amount of education). Moreover, in the presence of default, the optimal policy can be implemented through income-contingent payments.

A note on growth and inflation

A simple exercise on growth and inflationary financing of public expenditures is presented in this note. In a parameterized overlapping generations mode1 where government expenses positivc1y affects the growth rate of human capital, steady state capital and output increase with inflation, reproducing the so called Tobin effect. For large inflation rates, however, government authorities cannot affect real variables and there are only nominal effects. It is also shown that the optimal policy implies some inflation but not growth maximization.

Inflation targeting, credibility and coordination aspects

There are plenty of economic studies pointing out some requirements, like the inexistence of fiscal dominance, for inflation targeting framework be implemented in successful (credible) way. Essays on how public targets could be used in the absence of such requirements are unusual. In this papel' we appraise how central banks could use inflation targeting before soundness economic fundamentaIs have been achieved. First, based on concise framework, where confidence crises and imperfect information are neglected, we conclude that less ambitious (greater) target for inflation increases the credibility in the precommitment. Optimal target is higher than the one obtained using the Cukierman-Liviatan [7] model, where increasing credibility effect is not considered. Second, extending the model to make confidence crises possible, multiple equilibria solutions becomes possible too. In this case, to set greater targets for inflation may stimulate confidence crises and reduce the policymaker credibility. On the other hand, multiple (bad) equilibria may be avoided. The optimal target depends on the likelihood of each equilibrium be selected. Finally, when perturbing common knowledge uniqueness is restored even considering confidence crises, as in Morris-Shin[ 14]. The first result, i.e. less ambitious target for inflation increases credibility in precommitment, is also recovered. Adding a precise public signal, cOOl'dinated self-fulfilling actions and equilibrium multiplicity may still exist for some lack of common knowledge (as in Angeleto and Weming[l]). In this case, to set greater targets for inflation may stimulate confidence crisis again, reducing the policymaker credibility. From another aspect, multiple (bad) equilibria may be avoided. Optimal policy prescriptions depend on the likelihood of each equilibrium be selected. Results also indicate that more precise public information may open the door for bad equilibrium, contrary to the conventional wisdom that more central oank transparency is always good when considering inflation targeting framework.

Processo markoviano de decisão para alocação dinâmica de recursos e controle de admissão de conexão em redes IEEE 802.16

Este trabalho apresenta uma solução para o problema de controle admissão de conexão e alocação dinâmica de recursos em redes IEEE 802.16 através da modelagem de um Processo Markoviano de Decisão (PMD) utilizando o conceito de degradação de largura de banda, o qual é baseado nos requisitos diferenciados de largura de banda das classes de serviço do IEEE 802.16. Para o critério de desempenho do PMD é feita a atribuição de diferentes retornos a cada classe de serviço, fazendo assim o tratamento diferenciado de cada fluxo. Nesse sentido, é possível avaliar a política ótima, obtida através de um algoritmo de iteração de valores, considerando aspectos como o nível de degradação médio das classes de serviço, utilização dos recursos e probabilidades de bloqueios de cada classe de serviço em relação à carga do sistema. Resultados obtidos mostram que o método de controle markoviano proposto é capaz de priorizar as classes de serviço consideradas mais relevantes para o sistema.

Green-Markov models - new optimization strategies: a case study for user allocation in co-channel macro/femto networks

ABSTRACT: The femtocell concept aims to combine fixed-line broadband access with mobile telephony using the deployment of low-cost, low-power third and fourth generation base stations in the subscribers' homes. While the self-configuration of femtocells is a plus, it can limit the quality of service (QoS) for the users and reduce the efficiency of the network, based on outdated allocation parameters such as signal power level. To this end, this paper presents a proposal for optimized allocation of users on a co-channel macro-femto network, that enable self-configuration and public access, aiming to maximize the quality of service of applications and using more efficiently the available energy, seeking the concept of Green networking. Thus, when the user needs to connect to make a voice or a data call, the mobile phone has to decide which network to connect, using the information of number of connections, the QoS parameters (packet loss and throughput) and the signal power level of each network. For this purpose, the system is modeled as a Markov Decision Process, which is formulated to obtain an optimal policy that can be applied on the mobile phone. The policy created is flexible, allowing different analyzes, and adaptive to the specific characteristics defined by the telephone company. The results show that compared to traditional QoS approaches, the policy proposed here can improve energy efficiency by up to 10%.

Emissions Control and the Regulation of Product Markets: The Case of Automobiles

The paper investigates alternative policies to regulate emissions from polluting product markets, specifically considering the case of the automobiles market. The two policies we consider are: a quota that limits the quantity produced of the polluting model and a more flexible average efficiency standard that requires a minimum energy efficiency across all models produced by a firm, similar to the US Corporate Average Fuel Economy (CAFE) standards. We use a duopoly model of vertical differentiation where firms produce both an economy (i.e., low polluting) version and a luxury (i.e., high polluting) version of a given product. We show that while a quota can raise firm profit over a certain range, CAFE always reduces firm profit relative to the pre-regulation. We also show that while the quota reduces emissions, it is possible that emissions increase under CAFE. The optimal policy choice will depend on the magnitude of unit damages. We show that when unit damages are sufficiently high, the quota policy is more efficient than the average efficiency standard. This suggests that instead of tightening CAFE to limit damages from emissions, policy makers can shift to a quota policy which is both welfare enhancing and more profitable for firms.

ANALISANDO A RELAÇÃO DOS VOLUMES DE EXPORTAÇÃO E DE IMPORTAÇÃO, PIB, TAXAS DE CÂMBIO E INFLAÇÃO NO PERÍODO DE 2004 A 2014.

Esta dissertação visa deslumbrar uma análise macroeconômica do Brasil, especialmente no que se refere à relação dos índices mensais dos volumes das exportações e das importações com os volumes mensais do PIB, da Taxa SELIC e as Taxas de Câmbio, conforme dados coletados no período de janeiro de 2004 a dezembro de 2014, através de pesquisa literária referente aos históricos sobre cada conceito envolvido no âmbito da macroeconomia das varáveis estudadas. Foi realizado um estudo de caso embasado em dados de sites governamentais, no período delimitado, empregando-se o método de regressão linear, com base na Teoria da correlação de Pearson, demonstrando os resultados obtidos no período do estudo para as varáveis estudadas. Desta maneira, conseguiu-se estudar e analisar como as variáveis dependentes (resposta): volume das exportações e volume das importações estão relacionadas com as varáveis independentes (explicativas): PIB, Taxa Selic e taxa de Câmbio. Os resultados apurados no presente estudo permitem identificar que existe correlação moderada e negativa, quando analisadas a Taxa Selic e a Taxa de Câmbio com os volumes das exportações e das importações, enquanto o PIB apresenta correlação forte e positiva na análise com os volumes das exportações e das importações

Uma formulação por média-variância multi-período para o erro de rastreamento em carteiras de investimento.

Neste trabalho, deriva-se uma política de escolha ótima baseada na análise de média-variância para o Erro de Rastreamento no cenário Multi-período - ERM -. Referindo-se ao ERM como a diferença entre o capital acumulado pela carteira escolhida e o acumulado pela carteira de um benchmark. Assim, foi aplicada a metodologia abordada por Li-Ng em [24] para a solução analítica, obtendo-se dessa maneira uma generalização do caso uniperíodo introduzido por Roll em [38]. Em seguida, selecionou-se um portfólio do mercado de ações brasileiro baseado no fator de orrelação, e adotou-se como benchmark o índice da bolsa de valores do estado de São Paulo IBOVESPA, além da taxa básica de juros SELIC como ativo de renda fixa. Dois casos foram abordados: carteira composta somente de ativos de risco, caso I, e carteira com um ativo sem risco indexado à SELIC - e ativos do caso I (caso II).

Optimization of Discrete-Time, Stochastic Systems

* This research was supported by a grant from the Greek Ministry of Industry and Technology.

Sensitivity Analysis of Some Applied Probability Models

2000 Mathematics Subject Classi cation: Primary 90C31. Secondary 62C12, 62P05, 93C41.

Técnicas de conformação de feixe em arranjo de antenas utilizando aprendizagem por reforço

Beamforming is a technique widely used in various fields. With the aid of an antenna array, the beamforming aims to minimize the contribution of unknown interferents directions, while capturing the desired signal in a given direction. In this thesis are proposed beamforming techniques using Reinforcement Learning (RL) through the Q-Learning algorithm in antennas array. One proposal is to use RL to find the optimal policy selection between the beamforming (BF) and power control (PC) in order to better leverage the individual characteristics of each of them for a certain amount of Signal to Interference plus noise Ration (SINR). Another proposal is to use RL to determine the optimal policy between blind beamforming algorithm of CMA (Constant Modulus Algorithm) and DD (Decision Direct) in multipath environments. Results from simulations showed that the RL technique could be effective in achieving na optimal of switching between different techniques.

Essays on Omnichannel Sale

Thesis (Ph.D.)--University of Washington, 2016-08