Application of reinforcement learning in an open railway access market price negotiation

In an open railway access market price negotiation, it is feasible to achieve higher cost recovery by applying the principles of price discrimination. The price negotiation can be modeled as an optimization problem of revenue intake. In this paper, we present the pricing negotiation based on reinforcement learning model. A negotiated-price setting technique based on agent learning is introduced, and the feasible applications of the proposed method for open railway access market simulation are discussed.

Simulation model for driving dynamics, energy use and power supply

Dissertação para obtenção do grau de Mestre em Engenharia Electrotécnica Ramo de Energia

Computer simulation and modeling in railway applications

An electrified railway system includes complex interconnections and interactions of several subsystems. Computer simulation is the only viable means for system evaluation and analysis. This paper discusses the difficulties and requirements of effective simulation models for this specialized industrial application; and the development of a general-purpose multi-train simulator.

Parallel linear system solution and its application to railway power network simulation

The Streaming SIMD extension (SSE) is a special feature embedded in the Intel Pentium III and IV classes of microprocessors. It enables the execution of SIMD type operations to exploit data parallelism. This article presents improving computation performance of a railway network simulator by means of SSE. Voltage and current at various points of the supply system to an electrified railway line are crucial for design, daily operation and planning. With computer simulation, their time-variations can be attained by solving a matrix equation, whose size mainly depends upon the number of trains present in the system. A large coefficient matrix, as a result of congested railway line, inevitably leads to heavier computational demand and hence jeopardizes the simulation speed. With the special architectural features of the latest processors on PC platforms, significant speed-up in computations can be achieved.

A review of simulation models for railway systems

With the advances in computer hardware and software development techniques in the past 25 years, digital computer simulation of train movement and traction systems has been widely adopted as a standard computer-aided engineering tool [1] during the design and development stages of existing and new railway systems. Simulators of different approaches and scales are used extensively to investigate various kinds of system studies. Simulation is now proven to be the cheapest means to carry out performance predication and system behaviour characterisation. When computers were first used to study railway systems, they were mainly employed to perform repetitive but time-consuming computational tasks, such as matrix manipulations for power network solution and exhaustive searches for optimal braking trajectories. With only simple high-level programming languages available at the time, full advantage of the computing hardware could not be taken. Hence, structured simulations of the whole railway system were not very common. Most applications focused on isolated parts of the railway system. It is more appropriate to regard those applications as primarily mechanised calculations rather than simulations. However, a railway system consists of a number of subsystems, such as train movement, power supply and traction drives, which inevitably contains many complexities and diversities. These subsystems interact frequently with each other while the trains are moving; and they have their special features in different railway systems. To further complicate the simulation requirements, constraints like track geometry, speed restrictions and friction have to be considered, not to mention possible non-linearities and uncertainties in the system. In order to provide a comprehensive and accurate account of system behaviour through simulation, a large amount of data has to be organised systematically to ensure easy access and efficient representation; the interactions and relationships among the subsystems should be defined explicitly. These requirements call for sophisticated and effective simulation models for each component of the system. The software development techniques available nowadays allow the evolution of such simulation models. Not only can the applicability of the simulators be largely enhanced by advanced software design, maintainability and modularity for easy understanding and further development, and portability for various hardware platforms are also encouraged. The objective of this paper is to review the development of a number of approaches to simulation models. Attention is, in particular, given to models for train movement, power supply systems and traction drives. These models have been successfully used to enable various ‘what-if’ issues to be resolved effectively in a wide range of applications, such as speed profiles, energy consumption, run times etc.

Parallel solution for railway power network simulation

The Streaming SIMD extension (SSE) is a special feature that is available in the Intel Pentium III and P4 classes of microprocessors. As its name implies, SSE enables the execution of SIMD (Single Instruction Multiple Data) operations upon 32-bit floating-point data therefore, performance of floating-point algorithms can be improved. In electrified railway system simulation, the computation involves the solving of a huge set of simultaneous linear equations, which represent the electrical characteristic of the railway network at a particular time-step and a fast solution for the equations is desirable in order to simulate the system in real-time. In this paper, we present how SSE is being applied to the railway network simulation.

Numerical simulation of railway track supporting system using finite-infinite and thin layer elements under impulsive loads

The present study deals with two dimensional, numerical simulation of railway track supporting system subjected to dynamic excitation force. Under plane strain condition, the coupled finite-infinite elements to represent the near and far field stress distribution and thin layer interface element was employed to model the interfacial behavior between sleepers and ballast. To account for the relative debonding, slipping and crushing that could take place in the contact area between the sleepers and ballast, modified Mohr-Coulomb criterion was adopted. Furthermore an attempt has been made to consider the elasto-plastic material non-linearity of the railway track supporting media by employing different constitutive models to represent steel, concrete and supporting materials. Based on the proposed physical and constitutive modeling a code has been developed for dynamic loads. The applicability of the developed F.E code has been demonstrated by analyzing a real railway supporting structure.

Optimal track access rights allocation for agent negotiation in an open railway market

In open railway access markets, a train service provider (TSP) negotiates with an infrastructure provider (IP) for track access rights. This negotiation has been modeled by a multi-agent system (MAS) in which the IP and TSP are represented by separate software agents. One task of the IP agent is to generate feasible (and preferably optimal) track access rights, subject to the constraints submitted by the TSP agent. This paper formulates an IP-TSP transaction and proposes a branch-and-bound algorithm for the IP agent to identify the optimal track access rights. Empirical simulation results show that the model is able to emulate rational agent behaviors. The simulation results also show good consistency between timetables attained from the proposed methods and those derived by the scheduling principles adopted in practice.

Service bid comparisons by fuzzy ranking in open railway market timetabling

In an open railway access market, the Infrastructure Provider (IP), upon the receipts of service bids from the Train Service Providers (TSPs), assigns track access rights according to its own business objectives and the merits of the bids; and produces the train service timetable through negotiations. In practice, IP chooses to negotiate with the TSPs one by one in such a sequence that IP optimizes its objectives. The TSP bids are usually very complicated, containing a large number of parameters in different natures. It is a difficult task even for an expert to give a priority sequence for negotiations from the contents of the bids. This study proposes the application of fuzzy ranking method to compare and prioritize the TSP bids in order to produce a negotiation sequence. The results of this study allow investigations on the behaviors of the stakeholders in bid preparation and negotiation, as well as evaluation of service quality in the open railway market.

Intelligent negotiation behaviour model for an open railway access market

In an open railway access market, the provisions of railway infrastructures and train services are separated and independent. Negotiations between the track owner and train service providers are thus required for the allocation of the track capacity and the formulation of the services timetables, in which each party, i.e. a stakeholder, exhibits intelligence from the previous negotiation experience to obtain the favourable terms and conditions for the track access. In order to analyse the realistic interacting behaviour among the stakeholders in the open railway access market schedule negotiations, intelligent learning capability should be included in the behaviour modelling. This paper presents a reinforcement learning approach on modelling the intelligent negotiation behaviour. The effectiveness of incorporating learning capability in the stakeholder negotiation behaviour is then demonstrated through simulation.

Conflict resolution through negotiation in a railway open access market : a multi-agent system approach

Open access reforms to railway regulations allow multiple train operators to provide rail services on a common infrastructure. As railway operations are now independently managed by different stakeholders, conflicts in operations may arise, and there have been attempts to derive an effective access charge regime so that these conflicts may be resolved. One approach is by direct negotiation between the infrastructure manager and the train service providers. Despite the substantial literature on the topic, few consider the benefits of employing computer simulation as an evaluation tool for railway operational activities such as access pricing. This article proposes a multi-agent system (MAS) framework for the railway open market and demonstrates its feasibility by modelling the negotiation between an infrastructure provider and a train service operator. Empirical results show that the model is capable of resolving operational conflicts according to market demand.

Optimal traffic control at railway junctions

Conflict occurs when two or more trains approach the same junction within a specified time. Such conflicts result in delays. Current practices to assign the right of way at junctions achieve orderly and safe passage of the trains, but do not attempt to reduce the delays. A traffic controller developed in the paper assigns right of way to impose minimum total weighted delay on the trains. The traffic flow model and the optimisation technique used in this controller are described. Simulation studies of the performance of the controller are given.