Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

On optimal computation of MPLS label binding for multipoint-to-point connections

Most network operators have considered reducing Label Switched Routers (LSR) label spaces (i.e. the number of labels that can be used) as a means of simplifying management of underlaying Virtual Private Networks (VPNs) and, hence, reducing operational expenditure (OPEX). This letter discusses the problem of reducing the label spaces in Multiprotocol Label Switched (MPLS) networks using label merging - better known as MultiPoint-to-Point (MP2P) connections. Because of its origins in IP, MP2P connections have been considered to have tree- shapes with Label Switched Paths (LSP) as branches. Due to this fact, previous works by many authors affirm that the problem of minimizing the label space using MP2P in MPLS - the Merging Problem - cannot be solved optimally with a polynomial algorithm (NP-complete), since it involves a hard- decision problem. However, in this letter, the Merging Problem is analyzed, from the perspective of MPLS, and it is deduced that tree-shapes in MP2P connections are irrelevant. By overriding this tree-shape consideration, it is possible to perform label merging in polynomial time. Based on how MPLS signaling works, this letter proposes an algorithm to compute the minimum number of labels using label merging: the Full Label Merging algorithm. As conclusion, we reclassify the Merging Problem as Polynomial-solvable, instead of NP-complete. In addition, simulation experiments confirm that without the tree-branch selection problem, more labels can be reduced

Procedimiento estratégico de gestión de inversiones para personas naturales en el mercado bursátil colombiano

El mundo de las finanzas se encuentra dividido por gestión financiera, economía financiera, inversiones, finanzas personales y finanzas públicas. Aunque los temas que se estudiaran con mayor profundidad serán; gestión financiera, economías financieras e inversiones (Brigham, 2009). Partiendo de lo anterior, a lo largo de este documento se hará una relación entre la gestión de inversiones y la estrategia empresarial teniendo en cuenta que el objetivo final es desarrollar una metodología útil que desde la estrategia sirva como guía para la conformación de portafolios en activos que cotizan en la Bolsa de Valores de Colombia teniendo en cuenta los tres perfiles de riesgo (Amante, Moderado, Adverso al riesgo) y que además en el proceso de toma de decisiones de inversión funcione como herramienta para mejorar las posibilidades en la toma de decisiones de inversión para personas naturales en su rol como inversionistas.

Finding the optimal cardinality value for information bottleneck method

Information Bottleneck method can be used as a dimensionality reduction approach by grouping “similar” features together [1]. In application, a natural question is how many “features groups” will be appropriate. The dependency on prior knowledge restricts the applications of many Information Bottleneck algorithms. In this paper we alleviate this dependency by formulating the parameter determination as a model selection problem, and solve it using the minimum message length principle. An efficient encoding scheme is designed to describe the information bottleneck solutions and the original data, then the minimum message length principle is incorporated to automatically determine the optimal cardinality value. Empirical results in the documentation clustering scenario indicates that the proposed method works well for the determination of the optimal parameter value for information bottleneck method.

Parameter optimization of kernel-based one-class classifier on imbalance learning

Compared with conventional two-class learning schemes, one-class classification simply uses a single class in the classifier training phase. Applying one-class classification to learn from unbalanced data set is regarded as the recognition based learning and has shown to have the potential of achieving better performance. Similar to twoclass learning, parameter selection is a significant issue, especially when the classifier is sensitive to the parameters. For one-class learning scheme with the kernel function, such as one-class Support Vector Machine and Support Vector Data Description, besides the parameters involved in the kernel, there is another one-class specific parameter: the rejection rate v. In this paper, we proposed a general framework to involve the majority class in solving the parameter selection problem. In this framework, we first use the minority target class for training in the one-class classification stage; then we use both minority and majority class for estimating the generalization performance of the constructed classifier. This generalization performance is set as the optimization criteria. We employed the Grid search and Experiment Design search to attain various parameter settings. Experiments on UCI and Reuters text data show that the parameter optimized one-class classifiers outperform all the standard one-class learning schemes we examined.

Unicast and broadcast throughput maximization in amplify-and-forward relay networks

Cooperative communication (CC) offers an efficient and low-cost way to achieve spatial diversity by forming a virtual antenna array among single-antenna nodes that cooperatively share their antennas. It has been well recognized that the selection of relay nodes plays a critical role in the performance of CC. Most existing relay selection strategies focus on optimizing the outage probability or energy consumption. To fill in the vacancy of research on throughput improvement via CC, we study the relay selection problem with the objective of optimizing the throughput in this paper. For unicast, it is a P problem, and an optimal relay selection algorithm is provided with a correctness proof. For broadcast, we show the challenge of relay selection by proving it nonprobabilistic hard (NP-hard). A greedy heuristic algorithm is proposed to effectively choose a set of relay nodes that maximize the broadcast throughput. Simulation results show that the proposed algorithms can achieve high throughput under various network settings.

Political budget cycles in a fiscal federation : the effect of partisan voluntary transfers

This article first presents an econometric study suggesting that intergovernmental transfers to Brazilian municipalities are strongly partisan motivated. In light of that stylized fact, it develops an extension to Rogoff (1990)’s model to analyze the effect of partisan motivated transfers into sub-national electoral and fiscal equilibria. The main finding is that important partisan transfers may undo the positive selection aspect of political budget cycles. Indeed, partisan transfers may, on one hand, eliminate the political budget cycle, solving a moral hazard problem, but, on the other hand, they may retain an incompetent incumbent in office, bringing about an adverse selection problem.

Do insurance defrauders want to be punished?

We analyze a Principal-Agent model of an insurer who faces an adverse selection problem. He is unable to observe if his client has a high risk or a low risk of having an accident. At the underwriting of the contract, the insurer requests the client to declare his risk. After that, the former can costly audit the truthfulness of this announcement. If the audit confirms a false declaration, the insurer is legally allowed to punish the defrauder. We characterize the efRcient contracts when this punishment is bounded from above by a legal restriction. Then, we do some comparative statics on the efRcient contracts and on the agent's utility. The most important result of this paper concerns the legal limit to a defrauder's punishment. We prove that there exists a uni que value of this legal limit that maximizes the expected utility of a high risk type. Facing this particular value of the legal limit to a defrauder's punishment, the insurer will effectively audit a low risk reporto We also show that this particular value increases with the probability of facing a high risk policyholder. Therefore, when this probability is sufRciently high, the nullity of the contract is not enough. From the point of view of a potential defrauder, the law should allow harder sanctions. This is an striking result because the nullity of the contract is a common sanction for this kind of fraud in the USA and in some European countries.

Três ensaios em economia da educação

Essa tese é composta por três artigos na área de Economia da Educação que buscam investigar a questão do ensino médio técnico no Brasil. O primeiro e segundo artigos se inserem na literatura de avaliação de impacto dos tipos de ensino médio realizado, o de formação geral ou técnica, sobre os resultados do mercado de trabalho. Esse é um tema cada vez mais explorado em estudos empíricos internacionais, mas que ainda merece atenção dos pesquisadores no Brasil, seja para preencher lacunas no que se refere à identificação do efeito causal dessa política, seja porque o ensino técnico profissionalizante tenha se tornado alvo de maiores investimentos públicos nos últimos anos. Dessa forma, os dois artigos têm o propósito de estimar o efeito causal das habilidades técnicas adquiridas nos cursos técnicos de nível médio sobre os salários dos trabalhadores, no entanto se diferem pela fonte de dados utilizada. No primeiro artigo são utilizados os dados da Pesquisa Nacional por Amostra de Domicílios (IBGE) de 2007 e nossa contribuição está na estratégia de identificação do efeito causal, ao resolver o problema de auto seleção com o método de duplas diferenças (DD). Diversos testes de robustez foram realizados para dar suporte ao impacto positivo e estatisticamente significante que encontramos das habilidades técnicas sobre os salários, embora este seja restrito ao subgrupo de trabalhadores com no máximo o ensino médio completo. No segundo artigo foram utilizadas duas fontes de dados ainda não exploradas para esse fim, os microdados do ENEM (Exame Nacional do Ensino Médio) entre os anos de 2004 e 2006 e os dados da RAIS (Relação Anual de Informações Sociais) de 2009 a 2012 identificados por indivíduo (CPF). Por ser possível observar as notas dos jovens ao final do ensino médio, conseguimos separar os impactos do ensino médio técnico nos salários relacionados às habilidades técnicas específicas obtidas nos cursos técnicos, daqueles relacionados às habilidades gerais dos indivíduos (cognitivas ou não). Implementamos a estratégia de duplas diferenças e mais uma vez encontramos impacto positivo sobre os salários anuais dos jovens que concluíram o ensino médio técnico. Dessa vez nossos resultados são positivos tanto para os jovens que pararam seus estudos no ensino médio como para aqueles que concluíram o ensino superior. Verificamos a existência de heterogeneidade do efeito por região do país e nos quartis de distribuição de notas médias por escola no ENEM, indicando que os efeitos são crescentes com a qualidade oferecida pelas escolas. O terceiro artigo tem por objetivo estudar o mercado de trabalho no que diz respeito às ocupações tipicamente preenchidas por profissionais com nível médio técnico e analisar os diferenciais de salários dessas ocupações. A partir das ideias desenvolvidas no estudo, almejamos que ele seja orientador de políticas que relacionem educação técnica com demandas do mercado, com a finalidade de melhorar o matching entre formação e emprego. Entre as contribuições deste artigo estão a identificação e classificação das ocupações típicas de nível médio técnico pelo diferencial de salários, a correspondência entre as ocupações e as formações técnicas, a análise da atual oferta de vagas nos cursos correspondentes e o destaque para os cursos que parecem necessitar de maiores investimentos para o suprir as necessidades de mais profissionais.

Uma abordagem híbrida para o problema de seleção de fornecedores em cadeia de suprimentos

This work presents a hybrid approach for the supplier selection problem in Supply Chain Management. We joined decision-making philosophy by researchers from business school and researchers from engineering in order to deal with the problem more extensively. We utilized traditional multicriteria decision-making methods, like AHP and TOPSIS, in order to evaluate alternatives according decision maker s preferences. The both techiniques were modeled by using definitions from the Fuzzy Sets Theory to deal with imprecise data. Additionally, we proposed a multiobjetive GRASP algorithm to perform an order allocation procedure between all pre-selected alternatives. These alternatives must to be pre-qualified on the basis of the AHP and TOPSIS methods before entering the LCR. Our allocation procedure has presented low CPU times for five pseudorandom instances, containing up to 1000 alternatives, as well as good values for all considered objectives. This way, we consider the proposed model as appropriate to solve the supplier selection problem in the SCM context. It can be used to help decision makers in reducing lead times, cost and risks in their supply chain. The proposed model can also improve firm s efficiency in relation to business strategies, according decision makers, even when a large number of alternatives must be considered, differently from classical models in purchasing literature

Optimal design of smart structures using bonded plezoelectrics for vibration control

Smart material technology has become an area of increasing interest for the development of lighter and stronger structures which are able to incorporate actuator and sensor capabilities for collocated control. In the design of actively controlled structures, the determination of the actuator locations and the controller gains, is a very important issue. For that purpose, smart material modelling, modal analysis methods, control and optimization techniques are the most important ingredients to be taken into account. The optimization problem to be solved in this context presents two interdependent aspects. The first one is related to the discrete optimal actuator location selection problem which is solved in this paper using genetic algorithms. The second is represented by a continuous variable optimization problem, through which the control gains are determined using classical techniques. A cantilever Euler-Bernoulli beam is used to illustrate the presented methodology.

Optimal placement of piezoelectric sensor/actuators for smart structures vibration control

Smart material technology has become an area of increasing interest for the development of lighter and stronger structures that are able to incorporate actuator and sensor capabilities for collocated control. In the design of actively controlled structures, the determination of the actuator locations and the controller gains is a very important issue. For that purpose, smart material modeling, modal analysis methods, and control and optimization techniques are the most important ingredients to be taken into account. The optimization problem to be solved in this context presents two interdependent aspects. The first is related to the discrete optimal actuator location selection problem, which is solved in this paper using genetic algorithms. The second is represented by a continuous variable optimization problem, through which the control gains are determined using classical techniques. A cantilever Euler-Bernoulli beam is used to illustrate the presented methodology.

Optimal design of smart structures using bonded piezoelectrics for vibration control

Smart material technology has become an area of increasing interest for the development of lighter and stronger structures which are able to incorporate actuator and sensor capabilities for collocated control. In the design of actively controlled structures, the determination of the actuator locations and the controller gains, is a very important issue. For that purpose, smart material modelling, modal analysis methods, control and optimization techniques are the most important ingredients to be taken into account. The optimization problem to be solved in this context presents two interdependent aspects. The first one is related to the discrete optimal actuator location selection problem, which is solved in this paper using genetic algorithms. The second is represented by a continuous variable optimization problem, through which the control gains are determined using classical techniques. A cantilever Euler-Bernoulli beam is used to illustrate the presented methodology.

Optimal mean-variance control for discrete-time linear systems with Markovian jumps and multiplicative noises

In this paper, we consider the stochastic optimal control problem of discrete-time linear systems subject to Markov jumps and multiplicative noises under two criteria. The first one is an unconstrained mean-variance trade-off performance criterion along the time, and the second one is a minimum variance criterion along the time with constraints on the expected output. We present explicit conditions for the existence of an optimal control strategy for the problems, generalizing previous results in the literature. We conclude the paper by presenting a numerical example of a multi-period portfolio selection problem with regime switching in which it is desired to minimize the sum of the variances of the portfolio along the time under the restriction of keeping the expected value of the portfolio greater than some minimum values specified by the investor. (C) 2011 Elsevier Ltd. All rights reserved.