Dialog Systems based on Markov decision processes over two real tasks

In this work the state of the art of the automatic dialogue strategy management using Markov decision processes (MDP) with reinforcement learning (RL) is described. Partially observable Markov decision processes (POMDP) are also described. To test the validity of these methods, two spoken dialogue systems have been developed. The first one is a spoken dialogue system for weather forecast providing, and the second one is a more complex system for train information. With the first system, comparisons between a rule-based system and an automatically trained system have been done, using a real corpus to train the automatic strategy. In the second system, the scalability of these methods when used in larger systems has been tested.

Multilinear and Integer Programming for Markov Decision Processes with Imprecise Probabilities

Markov Decision Processes (MDPs) are extensively used to encode sequences of decisions with probabilistic effects. Markov Decision Processes with Imprecise Probabilities (MDPIPs) encode sequences of decisions whose effects are modeled using sets of probability distributions. In this paper we examine the computation of Γ-maximin policies for MDPIPs using multilinear and integer programming. We discuss the application of our algorithms to “factored” models and to a recent proposal, Markov Decision Processes with Set-valued Transitions (MDPSTs), that unifies the fields of probabilistic and “nondeterministic” planning in artificial intelligence research. 

Supporting the online consumer decision process : electronic commerce in a small Australian retailer

The paper explores ways in which online retailers of physical products can utilise the Internet to support their consumers’ decision making processes. The approach followed by a small Australian retailer in selling boots online is examined. The case is analysed using theoretical constructs from consumer behaviour. It is concluded that it is indeed feasible even for small online retailers to successfully sell “look and feel” type goods such as footwear over the Internet. However, for this to occur, high levels of customer support is essential and online retailers need to engender consumer trust in terms of payment processing and product delivery.

Corporate governance and its influence on the investment decision process of equity market's professionals: An inside view of how corporate governance issues influence the decision-making process of sell-side and buy-side professionals when analyzing Brazilian listed companies

Corporate governance has been in the spotlight for the past two decades, being subject of numerous researches all over the world. Governance is pictured as a broad and diverse theme, evolving through different routes to form distinct systems. This scenario together with 2 types of agency problems (investor vs. management and minorities vs. controlling shareholders) produce different definitions for governance. Usually, studies investigate whether corporate governance structures influence firm performance, and company valuation. This approach implies investors can identify those impacts and later take them into consideration when making investment decisions. However, behavioral finance theory shows that not always investors take rational decisions, and therefore the modus operandi of those professionals needs to be understood. So, this research aimed to investigate to what extent Brazilian corporate governance standards and practices influence the investment decision-making process of equity markets' professionals from the sell-side and buy-side. This exploratory study was carried out through qualitative and quantitative approaches. In the qualitative phase, 8 practitioners were interviewed and 3 dimensions emerged: understanding, pertinence and practice. Based on the interviews’ findings, a questionnaire was formulated and distributed to buy-siders and sell-siders that cover Brazilian stocks. 117 respondents from all over the world contributed to the study. The data obtained were analyzed through structural equation modeling and descriptive statistics. The 3 dimensions became 5 constructs: definition (institutionalized governance, informal governance), pertinence (relevance), practice (valuation process, structured governance assessment) The results of this thesis suggest there is no definitive answer, as the extent to which governance will influence an investment decision process will depend on a number of circumstances which compose the context. The only certainty is the need to present a “corporate governance behavior”, rather than simply establishing rules and regulations at firm and country level.

Green-Markov models - new optimization strategies: a case study for user allocation in co-channel macro/femto networks

ABSTRACT: The femtocell concept aims to combine fixed-line broadband access with mobile telephony using the deployment of low-cost, low-power third and fourth generation base stations in the subscribers' homes. While the self-configuration of femtocells is a plus, it can limit the quality of service (QoS) for the users and reduce the efficiency of the network, based on outdated allocation parameters such as signal power level. To this end, this paper presents a proposal for optimized allocation of users on a co-channel macro-femto network, that enable self-configuration and public access, aiming to maximize the quality of service of applications and using more efficiently the available energy, seeking the concept of Green networking. Thus, when the user needs to connect to make a voice or a data call, the mobile phone has to decide which network to connect, using the information of number of connections, the QoS parameters (packet loss and throughput) and the signal power level of each network. For this purpose, the system is modeled as a Markov Decision Process, which is formulated to obtain an optimal policy that can be applied on the mobile phone. The policy created is flexible, allowing different analyzes, and adaptive to the specific characteristics defined by the telephone company. The results show that compared to traditional QoS approaches, the policy proposed here can improve energy efficiency by up to 10%.

Green markov - uma abordagem híbrida de política markoviana e simulação discreta para planejamento de alocação de usuários em redes macro-femto

O uso da comunicação de voz e dados através de dispositivos móveis vem aumentando significativamente nos últimos anos. Tal expansão traz algumas dificuldades inerentes, tais como: ampliação constante de capacidade das redes e eficiência energética. Neste contexto, vem se consolidando o conceito de Green networks, que se concentra no esforço para economia de energia e redução de CO2. Neste sentido, este trabalho propõe validar um modelo de uma política baseado em processo markoviano de decisão, visando a otimizar o consumo de energia, QoS e QoE, na alocação de usuários em redes macrocell e femtocell. Para isso o modelo foi inserido no simulador NS-2, aliando a solução analítica markoviana à flexibilidade característica da simulação discreta. A partir dos resultados apresentados na simulação, a política obteve uma economia significativa no consumo energético, melhorando a eficiência energética em até 4%, além de melhorar a qualidade de serviço em relação às redes macrocell e femtocell, demonstrando-se eficaz, de modo a alterar diretamente as métricas de QoS e de QoE.

Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

Strategic decision making in international firms: effect of top management team internationalization on international strategic decision process

In this research project, I have integrated two research streams on international strategic decisions making in international firms: upper echelons or top management teams (TMT) internationalization research and international strategic decision making process research. Both research streams in international business literature have evolved independently, but there is a potential in combining these two streams of research. The first empirical paper “TMT internationalization and international strategic decision making process: a decision level analysis of rationality, speed, and performance” explores the influence of TMT internationalization on strategic decision rationality and speed and, subsequently, their effect on international strategic decision effectiveness (performance). The results show that the internationalization of TMT is positively related to decision effectiveness and this relationship is mediated by decision rationality while the hypotheses regarding the association between TMT internationalization and decision speed, and the mediating effect of speed were not supported. The second paper “TMT internationalization and international strategic decision rationality: the mediating role of international information” of my thesis is a simple but logical extension of first paper. The first paper showed that TMT Internationalization has a significant positive effect on international strategic decision rationality. The second paper explicitly showed that TMT internationalization affect on international strategic decision rationality comes from two sources: international experience (personal international knowledge and information) and international information collected from managerial international contacts. For this research project, I have collected data from international software firms in Pakistan. My research contributes to the literature on upper echelons theory and strategic decision making in context of international business and international firms by explicitly examining the link between TMT internationalization and characteristics of strategic decisions making process (i.e. rationality and speed) in international firms and their possible mediating effect on performance.

Cooperative off-policy prediction of markov decision processes in adaptive networks

We apply diffusion strategies to propose a cooperative reinforcement learning algorithm, in which agents in a network communicate with their neighbors to improve predictions about their environment. The algorithm is suitable to learn off-policy even in large state spaces. We provide a mean-square-error performance analysis under constant step-sizes. The gain of cooperation in the form of more stability and less bias and variance in the prediction error, is illustrated in the context of a classical model. We show that the improvement in performance is especially significant when the behavior policy of the agents is different from the target policy under evaluation.