Learning Multirobot Hose Transportation and Deployment by Distributed Round-Robin Q-Learning

Multi-Agent Reinforcement Learning (MARL) algorithms face two main difficulties: the curse of dimensionality, and environment non-stationarity due to the independent learning processes carried out by the agents concurrently. In this paper we formalize and prove the convergence of a Distributed Round Robin Q-learning (D-RR-QL) algorithm for cooperative systems. The computational complexity of this algorithm increases linearly with the number of agents. Moreover, it eliminates environment non sta tionarity by carrying a round-robin scheduling of the action selection and execution. That this learning scheme allows the implementation of Modular State-Action Vetoes (MSAV) in cooperative multi-agent systems, which speeds up learning convergence in over-constrained systems by vetoing state-action pairs which lead to undesired termination states (UTS) in the relevant state-action subspace. Each agent's local state-action value function learning is an independent process, including the MSAV policies. Coordination of locally optimal policies to obtain the global optimal joint policy is achieved by a greedy selection procedure using message passing. We show that D-RR-QL improves over state-of-the-art approaches, such as Distributed Q-Learning, Team Q-Learning and Coordinated Reinforcement Learning in a paradigmatic Linked Multi-Component Robotic System (L-MCRS) control problem: the hose transportation task. L-MCRS are over-constrained systems with many UTS induced by the interaction of the passive linking element and the active mobile robots.

Impact of postponement on transportation: An environmental perspective

Yang, Ying, Yang, Biao, and Wijngaard, Jacob, ' Impact of postponement on transportation: An environmental perspective', International Journal of Logistics Management (2005) 16(2) pp.192-204 RAE2008

Two-machine open shop scheduling with special transportation times

The paper considers a problem of scheduling n jobs in a two-machine open shop to minimise the makespan, provided that preemption is not allowed and the interstage transportation times are involved. In general, this problem is known to be NP-hard. We present a linear time algorithm that finds an optimal schedule if no transportation time exceeds the smallest of the processing times. We also describe an algorithm that creates a heuristic solution to the problem with job-independent transportation times. Our algorithm provides a worst-case performance ratio of 8/5 if the transportation time of a job depends on the assigned processing route. The ratio reduces to 3/2 if all transportation times are equal.

Determining optimal duration of seed translocation periods for benthic mussel (Mytilus edulis) cultivation using physiological and behavioural measures of stress

During the benthic cultivation process of Mytilus edulis (blue mussels), wild mussel seed is often transplanted from naturally occurring subtidal beds to sheltered in-shore waters to be grown to a commercial size. The survival of these relaid mussels is ultimately a function of their quality and physiological condition upon relaying and it has been recognised that mussels can suffer from a loss in condition following transportation. We investigated whether the process of being transported to ongrowing plots had a negative effect on the physiological health and resultant behaviour of mussels by simulating transportation conditions in a controlled experiment. Mussels were kept, out of water, in plastic piping to recreate translocation conditions and further, we tested if depth held in a ship hold (0, 1.5 and 3 m) and length of time emersed (12, 24 and 48 h) affected mussel condition and behaviour. Physiological condition was assessed by quantifying mussel tissue pH and whole tissue glucose, glycogen, succinate and propionate concentrations. The rate of byssogenesis was also quantified to estimate recovery following a period of re-immersion. The depth at which mussels were held did not affect any of the physiological indicators of mussel stress but short-term byssus production was affected. Mussels held at 3 m produced fewer byssus threads during the first 72 h following re-immersion compared with mussels at 0 m (i.e. not buried) suggesting that depth held can impede recovery following transportation. Duration of emersion affected all stress indicators. Specifically, mussels held out of water for 48 h had a reduced physiological condition compared with those emersed for just 12 h. This work has practical implications for the benthic cultivation industry and based on these results we recommend that mussels are held out of water for less than 24 h prior to relaying to ensure physiological health and resultant condition is preserved.

A Reliability Assessment Approach For Integrated Transportation and Electrical Power Systems Incorporating Electric Vehicles

With the increasing utilization of electric vehicles (EVs), transportation systems and electrical power systems are becoming increasingly coupled. However, the interaction between these two kinds of systems are not well captured, especially from the perspective of transportation systems. This paper studies the reliability of integrated transportation and electrical power system (ITES). A bidirectional EV charging control strategy is first demonstrated to model the interaction between the two systems. Thereafter, a simplified transportation system model is developed, whose high efficiency makes the reliability assessment of the ITES realizable with an acceptable accuracy. Novel transportation system reliability indices are then defined from the view point of EV’s driver. Based on the charging control model and the transportation simulation method, a daily periodic quasi sequential reliability assessment method is proposed for the ITES system. Case studies based on RBTS system demonstrate that bidirectional charging controls of EVs will benefit the reliability of power systems, while decrease the reliability of EVs travelling. Also, the optimal control strategy can be obtained based on the proposed method. Finally, case studies are performed based on a large scale test system to verify the practicability of the proposed method.

MODELING THE EVOLUTION OF COMPLEX NETWORKS THROUGH THE PATH-STAR TRANSFORMATION AND OPTIMAL MULTIVARIATE METHODS

The topology of real-world complex networks, such as in transportation and communication, is always changing with time. Such changes can arise not only as a natural consequence of their growth, but also due to major modi. cations in their intrinsic organization. For instance, the network of transportation routes between cities and towns ( hence locations) of a given country undergo a major change with the progressive implementation of commercial air transportation. While the locations could be originally interconnected through highways ( paths, giving rise to geographical networks), transportation between those sites progressively shifted or was complemented by air transportation, with scale free characteristics. In the present work we introduce the path-star transformation ( in its uniform and preferential versions) as a means to model such network transformations where paths give rise to stars of connectivity. It is also shown, through optimal multivariate statistical methods (i.e. canonical projections and maximum likelihood classification) that while the US highways network adheres closely to a geographical network model, its path-star transformation yields a network whose topological properties closely resembles those of the respective airport transportation network.

Simulation model using meta heuristic algorithms for achieving optimal arrangement of storage bins in a sawmill yard

Bin planning (arrangements) is a key factor in the timber industry. Improper planning of the storage bins may lead to inefficient transportation of resources, which threaten the overall efficiency and thereby limit the profit margins of sawmills. To address this challenge, a simulation model has been developed. However, as numerous alternatives are available for arranging bins, simulating all possibilities will take an enormous amount of time and it is computationally infeasible. A discrete-event simulation model incorporating meta-heuristic algorithms has therefore been investigated in this study. Preliminary investigations indicate that the results achieved by GA based simulation model are promising and better than the other meta-heuristic algorithm. Further, a sensitivity analysis has been done on the GA based optimal arrangement which contributes to gaining insights and knowledge about the real system that ultimately leads to improved and enhanced efficiency in sawmill yards. It is expected that the results achieved in the work will support timber industries in making optimal decisions with respect to arrangement of storage bins in a sawmill yard.

Draft Proposed Public Transportation Routes for Waterville, ME

Currently, there is a public bus transportation route in Waterville, Maine. However, this system could be improved. Our goal was to use GIS to find optimal public transportation routes throughout the city based on given points of interest and high population density areas. Three different groups of points of interest were created in the North, West, and South sections of Waterville. Using the Network Analyst tool, which calculates optimal routes, using existing street data, based on the input of stops, barriers, and impedance, we ran an analysis of what we thought would be the routes that best served the greatest number of people. Two different sets of routes were found: one with length as the impedance (the shortest length between the selected stops was favored), and one with population density as the impedance (the roads with the highest population density were favored). Finally, the times of the resulting routes (given a constant speed limit of 25 mph) were calculated and evaluated.

A mass-transportation approach to a one dimensional fluid mechanics model with nonlocal velocity

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Aggregation in the generalized transportation problem

Aggregation disaggregation is used to reduce the analysis of a large generalized transportation problem to a smaller one. Bounds for the actual difference between the aggregated objective and the original optimal value are used to quantify the error due to aggregation and estimate the quality of the aggregation. The bounds can be calculated either before optimization of the aggregated problem (a priori) or after (a posteriori). Both types of the bounds are derived and numerically compared. A computational experiment was designed to (a) study the correlation between the bounds and the actual error and (b) quantify the difference of the error bounds from the actual error. The experiment shows a significant correlation between some a priori bounds, the a posteriori bounds and the actual error. These preliminary results indicate that calculating the a priori error bound is a useful strategy to select the appropriate aggregation level, since the a priori bound varies in the same way that the actual error does. After the aggregated problem has been selected and optimized, the a posteriori bound provides a good quantitative measure for the error due to aggregation.

Branch and bound algorithm for transmission system expansion planning using a transportation model

A method for optimal transmission network expansion planning is presented. The transmission network is modelled as a transportation network. The problem is solved using hierarchical Benders decomposition in which the problem is decomposed into master and slave subproblems. The master subproblem models the investment decisions and is solved using a branch-and-bound algorithm. The slave subproblem models the network operation and is solved using a specialised linear program. Several alternative implementations of the branch-and-bound algorithm have been rested. Special characteristics of the transmission expansion problem have been taken into consideration in these implementations. The methods have been tested on various test systems available in the literature.

Optimal and Long-Term Dynamic Transport Policy Design: Seeking Maximum Social Welfare through a Pricing Scheme.

This article presents an alternative approach to the decision-making process in transport strategy design. The study explores the possibility of integrating forecasting, assessment and optimization procedures in support of a decision-making process designed to reach the best achievable scenario through mobility policies. Long-term evaluation, as required by a dynamic system such as a city, is provided by a strategic Land-Use and Transport Interaction (LUTI) model. The social welfare achieved by implementing mobility LUTI model policies is measured through a cost-benefit analysis and maximized through an optimization process throughout the evaluation period. The method is tested by optimizing a pricing policy scheme in Madrid on a cordon toll in a context requiring system efficiency, social equity and environmental quality. The optimized scheme yields an appreciable increase in social surplus through a relatively low rate compared to other similar pricing toll schemes. The results highlight the different considerations regarding mobility impacts on the case study area, as well as the major contributors to social welfare surplus. This leads the authors to reconsider the cost-analysis approach, as defined in the study, as the best option for formulating sustainability measures.

Analysis of alternative tendering mechanisms for transportation Public-Private Partnerships

This paper describes a theoretical model based primarily on transaction costs, for comparing the various tendering mechanisms used for transportation Public-Private Partnership (PPP) projects. In particular, the model contrasts negotiated procedures with the open procedure, as defined by the current European Union legislation on public tendering. The model includes both ex ante transaction costs (borne during the tendering stage) and ex post transaction costs (such as enforcement costs, re-negotiation costs, and costs arising from litigation between partners), explaining the trade-off between them. Generally speaking, it is assumed that the open procedure implies lower transaction costs ex ante, while the negotiated procedure reduces the probability of the appearance of new contingencies not foreseen in the contract, hence diminishing the expected value of transaction costs ex post. Therefore, the balance between ex ante and ex post transaction costs is the main criterion for deciding whether the open or negotiated procedure would be optimal. Notwithstanding, empirical evidence currently exists only on ex ante transaction costs in transportation infrastructure projects. This evidence has shown a relevant difference between the two procedures as far as ex ante costs are concerned, favouring the open procedure. The model developed in this paper also demonstrates that a larger degree of complexity in a contract does not unequivocally favour the use of a negotiated procedure. Only in those cases dealing with very innovative projects, where important dimensions of the quality of the asset or service are not verifiable, may we observe an advantage in favour of the negotiated procedure. The bottom line is that we find it difficult to justify the employment of negotiated procedures in most transportation PPP contracts, especially in the field of roads. Nevertheless, the field remains open for future empirical work and research on the levels of transaction costs borne ex post in PPP contracts, as well as on the probabilities of such costs appearing under any of the procurement procedures.

How varies optimal welfare pricing with income distribution? The case of the untolled alternative.

In some countries, such as Spain, it is very common that in the same corridor there are two roads with the same origin and destination but with some differences. The most important contrast is that one is a toll highway which offers a better quality than the parallel road in exchange of a price. The users decide if the price of the toll is worth to pay for the advantages offered. This problem is known as the untolled alternative and it has been largely studied in the academic literature, particularly related to economic efficiency and the optimal welfare toll. However, there is a gap in the literature academic regarding how it affects income distribution to the optimal toll. The main objective of the paper is to fill this gap. In this paper a theoretical model in order to obtain the optimal welfare price in a toll highway that competes for capturing the traffic with a conventional road is developed. This model is done for non-usual users who decide over the expectation of free flow conditions. This model is finally applied to the variables we want to focus on: average value of travel time (VTT) which is strongly related with income, dispersion of this VTT and traffic levels, from free flow to congestion. Derived from the results, we conclude that the higher the average VTT the higher the optimal price, the higher the dispersion of this VTT the lower the optimal price and finally, the more the traffic the higher the optimal toll.