Modeling Stock Order Flows and Learning Market-Making from Data

Stock markets employ specialized traders, market-makers, designed to provide liquidity and volume to the market by constantly supplying both supply and demand. In this paper, we demonstrate a novel method for modeling the market as a dynamic system and a reinforcement learning algorithm that learns profitable market-making strategies when run on this model. The sequence of buys and sells for a particular stock, the order flow, we model as an Input-Output Hidden Markov Model fit to historical data. When combined with the dynamics of the order book, this creates a highly non-linear and difficult dynamic system. Our reinforcement learning algorithm, based on likelihood ratios, is run on this partially-observable environment. We demonstrate learning results for two separate real stocks.

Learning and adaptation in physical heterogeneous teams of robots

In this paper we present a novel approach to assigning roles to robots in a team of physical heterogeneous robots. Its members compete for these roles and get rewards for them. The rewards are used to determine each agent’s preferences and which agents are better adapted to the environment. These aspects are included in the decision making process. Agent interactions are modelled using the concept of an ecosystem in which each robot is a species, resulting in emergent behaviour of the whole set of agents. One of the most important features of this approach is its high adaptability. Unlike some other learning techniques, this approach does not need to start a whole exploitation process when the environment changes. All this is exemplified by means of experiments run on a simulator. In addition, the algorithm developed was applied as applied to several teams of robots in order to analyse the impact of heterogeneity in these systems

Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Curriculum sequencing for an e-learning system based on learning styles

This work shows the use of adaptation techniques involved in an e-learning system that considers students' learning styles and students' knowledge states. The mentioned e-learning system is built on a multiagent framework designed to examine opportunities to improve the teaching and to motivate the students to learn what they want in a user-friendly and assisted environment

EdShare: collaboration in a University for learning and teaching

Poster for the School of Electronics and Computer Science, Learning Societies Lab Open Day, 27 February 2008 at the University of Southampton. Profile and presentation of the EdShare resource. The poster illustrates the philosophy of EdShare, how it relates to the Web 2.0 environment and its relationship to the education agenda in a University.

Aula de música : desarrollo de contenidos en una herramienta de e-learning para la asignatura de música en Educación Primaria.

Aula de música es una herramienta e-learning para el desarrollo del aprendizaje de la música para niños con edades comprendidas entre los 6 y 12 años, edades correspondientes a las de los alumnos de la etapa de la Educación Primaria. En esta herramienta destaca el uso de estándares y especificaciones como LOM, IMS, etc. que van a facilitar la tarea de reutilizar la documentación incluida para compartir conocimiento. El proceso de elaboración del contenido ha sido fundamental y en relación con el entorno de trabajo debe mencionarse que se ha primado la construcción de una GUI que sirva para aprender y que motive a los alumnos a aprender música de una forma diferente, en contraposición a realizar una diseño estético que fuera incapaz de adaptarse a las capacidades de cada tipo de usuario, para lo que se han tenido en cuenta criterios de usabilidad y accesibilidad (WAI).

A User-Centric Approach to eCertificate for Electronic Identities (eIDs) Management in Mobile Environment

The use of electronic documents is constantly growing and the necessity to implement an ad-hoc eCertificate which manages access to private information is not only required but also necessary. This paper presents a protocol for the management of electronic identities (eIDs), meant as a substitute for the paper-based IDs, in a mobile environment with a user-centric approach. Mobile devices have been chosen because they provide mobility, personal use and high computational complexity. The inherent user-centricity also allows the user to personally manage the ID information and to display only what is required. The chosen path to develop the protocol is to migrate the existing eCert technologies implemented by the Learning Societies Laboratory in Southampton. By comparing this protocol with the analysis of the eID problem domain, a new solution has been derived which is compatible with both systems without loss of features.

Impact on the quality of learning as a result of the use of computers in schools

The educational software and computer assisted learning has been used in schools to promote the interest of students in new ways of thinking and learning so it can be useful in the reading learning process. Experimental studies performed in preschool and school age population have shown a better yield and a positive effect in reading, mathematics and cognitive skills in children who use educative software for fi fteen to twenty minutes a day periods. The goal of this study was to evaluate the progression in verbal, visual-motor integration and reading skills in children who were using educational software to compare them with a group in traditional pedagogic methodology. Results: All children were evaluated before using any kind of pedagogic approach. Initial evaluation revealed a lower–age score in all applied test. 11% of them were at high risk for learning disorders. There was a second evaluation that showed a significant positive change compared with the fi rst one. Nevertheless, despite some items, there were no general differences comparing the groups according if they were using or not a computer. In conclusion, policies on using educational software and computers must be revaluated due to the fact that children in our public schools come from a deprived environment with a lack of opportunities to use technologies.

An artificial economy based on reinforcement learning and agent based modeling

In this paper, we employ techniques from artificial intelligence such as reinforcement learning and agent based modeling as building blocks of a computational model for an economy based on conventions. First we model the interaction among firms in the private sector. These firms behave in an information environment based on conventions, meaning that a firm is likely to behave as its neighbors if it observes that their actions lead to a good pay off. On the other hand, we propose the use of reinforcement learning as a computational model for the role of the government in the economy, as the agent that determines the fiscal policy, and whose objective is to maximize the growth of the economy. We present the implementation of a simulator of the proposed model based on SWARM, that employs the SARSA(λ) algorithm combined with a multilayer perceptron as the function approximation for the action value function.

Tomorrow's geography for Edexcel GCSE specification A : revision guide. Unit 2 : the natural environment.

Esta guía de revisión ofrece el contenido completo de la unidad 2 de la especificación A para el examen del General Certificate Secondary Education (GCSE) por el organismo Edexcel. Incluye consejos de los examinadores sobre el estilo de preguntas y sobre la preparación del examen.

Dialogue Systems for Language Learning

En este artículo se presenta el caso de Milao, un entorno virtual que ofrece a los estudiantes de idiomas extranjeros la oportunidad de desarrollar y mejorar sus habilidades comunicativas dialogando en escenarios de conversación predefinidos que simulan la interacción con un nativo. Esta tecnología propone una solución a uno de los mayores retos en el aprendizaje de lenguas extranjeras: la falta de oportunidades para poner en práctica la gramática y el vocabulario recién adquiridos. Combinando la investigación sobre la lingüística y el aprendizaje de lenguas con los avances tecnológicos en el campo del Procesamiento del Lenguaje Natural (NPL), particularmente sobre sistemas de diálogo, hemos creado oportunidades en la demanda de los estudiantes a conversar en la lengua que tratan de aprender.

A teaching/learning support tool for introductory programming courses

In This work we present a Web-based tool developed with the aim of reinforcing teaching and learning of introductory programming courses. This tool provides support for teaching and learning. From the teacher's perspective the system introduces important gains with respect to the classical teaching methodology. It reinforces lecture and laboratory sessions, makes it possible to give personalized attention to the student, assesses the degree of participation of the students and most importantly, performs a continuous assessment of the student's progress. From the student's perspective it provides a learning framework, consisting in a help environment and a correction environment, which facilitates their personal work. With this tool students are more motivated to do programming

Social Networking Sites as Technology Enhanced Learning Communities towards autonomous language learning

The following contribution pretends to cope with the demands of a globalised, post-modern environment through the design and implementation of an online international project where an SNS is used in order to join English as Second Language (ESL) students from different parts of the world. The design of the project appears around the implementation of the Bologna process in the Faculty of Education from the University of Girona where the basic prerequisite of all students to acquire English at the level B1 of the Common European Portfolio makes English a compulsory competence for communication among its higher education candidates in order to develop in the world. Together with the University of Girona, there is the International Educational and Resources Network (iEARN) which promotes the participation of schools around the world in online international projects

Socio-Cultural Transformation and the Promotion of Learning

The main premise of Vygotsky’s cultural-historical theory is that to promote learning, and thus development, educators must intervene in, and change, the students’ socio-cultural context. Vygotsky’s theory, however, has been misinterpreted and the opposite approach has been accepted: the teaching is adapted, according to the context. The result is widespread failure in schools. This article reclaims the true transformative meaning of Vygotskian theory and shows how successful schools in several countries implement various actions to transform their social and cultural environment. Data is presented from six case studies of successful schools conducted in five European countries. The analysis shows that these actions improve instrumental learning and, consequently, cognitive development. All these efforts focus on teaching methods that aim to increase the amount that students learn