Optimistic linear programming gives logarithmic regret for irreducible MDPs

We present an algorithm called Optimistic Linear Programming (OLP) for learning to optimize average reward in an irreducible but otherwise unknown Markov decision process (MDP). OLP uses its experience so far to estimate the MDP. It chooses actions by optimistically maximizing estimated future rewards over a set of next-state transition probabilities that are close to the estimates, a computation that corresponds to solving linear programs. We show that the total expected reward obtained by OLP up to time T is within C(P) log T of the reward obtained by the optimal policy, where C(P) is an explicit, MDP-dependent constant. OLP is closely related to an algorithm proposed by Burnetas and Katehakis with four key differences: OLP is simpler, it does not require knowledge of the supports of transition probabilities, the proof of the regret bound is simpler, but our regret bound is a constant factor larger than the regret of their algorithm. OLP is also similar in flavor to an algorithm recently proposed by Auer and Ortner. But OLP is simpler and its regret bound has a better dependence on the size of the MDP.

Linear programming for large-scale Markov decision problems

We consider the problem of controlling a Markov decision process (MDP) with a large state space, so as to minimize average cost. Since it is intractable to compete with the optimal policy for large scale problems, we pursue the more modest goal of competing with a low-dimensional family of policies. We use the dual linear programming formulation of the MDP average cost problem, in which the variable is a stationary distribution over state-action pairs, and we consider a neighborhood of a low-dimensional subset of the set of stationary distributions (defined in terms of state-action features) as the comparison class. We propose a technique based on stochastic convex optimization and give bounds that show that the performance of our algorithm approaches the best achievable by any policy in the comparison class. Most importantly, this result depends on the size of the comparison class, but not on the size of the state space. Preliminary experiments show the effectiveness of the proposed algorithm in a queuing application.

Linear stern waves in finite depth channels

This paper formulates an analytically tractable problem for the wake generated by a long flat bottom ship by considering the steady free surface flow of an inviscid, incompressible fluid emerging from beneath a semi-infinite rigid plate. The flow is considered to be irrotational and two-dimensional so that classical potential flow methods can be exploited. In addition, it is assumed that the draft of the plate is small compared to the depth of the channel. The linearised problem is solved exactly using a Fourier transform and the Wiener-Hopf technique, and it is shown that there is a family of subcritical solutions characterised by a train of sinusoidal waves on the downstream free surface. The amplitude of these waves decreases as the Froude number increases. Supercritical solutions are also obtained, but, in general, these have infinite vertical velocities at the trailing edge of the plate. Consideration of further terms in the expansions suggests a way of canceling the singularity for certain values of the Froude number.

Metaheuristic approaches to the hybrid flow shop scheduling problem with a cost-related criterion

In the paper, the flow-shop scheduling problem with parallel machines at each stage (machine center) is studied. For each job its release and due date as well as a processing time for its each operation are given. The scheduling criterion consists of three parts: the total weighted earliness, the total weighted tardiness and the total weighted waiting time. The criterion takes into account the costs of storing semi-manufactured products in the course of production and ready-made products as well as penalties for not meeting the deadlines stated in the conditions of the contract with customer. To solve the problem, three constructive algorithms and three metaheuristics (based one Tabu Search and Simulated Annealing techniques) are developed and experimentally analyzed. All the proposed algorithms operate on the notion of so-called operation processing order, i.e. the order of operations on each machine. We show that the problem of schedule construction on the base of a given operation processing order can be reduced to the linear programming task. We also propose some approximation algorithm for schedule construction and show the conditions of its optimality.

Two-dimensional bow and stern flows in a finite-depth fluid with constant vorticity

This thesis is concerned with two-dimensional free surface flows past semi-infinite surface-piercing bodies in a fluid of finite-depth. Throughout the study, it is assumed that the fluid in question is incompressible, and that the effects of viscosity and surface tension are negligible. The problems considered are physically important, since they can be used to model the flow of water near the bow or stern of a wide, blunt ship. Alternatively, the solutions can be interpreted as describing the flow into, or out of, a horizontal slot. In the past, all research conducted on this topic has been dedicated to the situation where the flow is irrotational. The results from such studies are extended here, by allowing the fluid to have constant vorticity throughout the flow domain. In addition, new results for irrotational flow are also presented. When studying the flow of a fluid past a surface-piercing body, it is important to stipulate in advance the nature of the free surface as it intersects the body. Three different possibilities are considered in this thesis. In the first of these possibilities, it is assumed that the free surface rises up and meets the body at a stagnation point. For this configuration, the nonlinear problem is solved numerically with the use of a boundary integral method in the physical plane. Here the semi-infinite body is assumed to be rectangular in shape, with a rounded corner. Supercritical solutions which satisfy the radiation condition are found for various values of the Froude number and the dimensionless vorticity. Subcritical solutions are also found; however these solutions violate the radiation condition and are characterised by a train of waves upstream. In the limit that the height of the body above the horizontal bottom vanishes, the flow approaches that due to a submerged line sink in a $90^\circ$ corner. This limiting problem is also examined as a special case. The second configuration considered in this thesis involves the free surface attaching smoothly to the front face of the rectangular shaped body. For this configuration, nonlinear solutions are computed using a similar numerical scheme to that used in the stagnant attachment case. It is found that these solution exist for all supercritical Froude numbers. The related problem of the cusp-like flow due to a submerged sink in a corner is also considered. Finally, the flow of a fluid emerging from beneath a semi-infinite flat plate is examined. Here the free surface is assumed to detach from the trailing edge of the plate horizontally. A linear problem is formulated under the assumption that the elevation of the plate is close to the undisturbed free surface level. This problem is solved exactly using the Wiener-Hopf technique, and subcritical solutions are found which are characterised by a train of sinusoidal waves in the far field. The nonlinear problem is also considered. Exact relations between certain parameters for supercritical flow are derived using conservation of mass and momentum arguments, and these are confirmed numerically. Nonlinear subcritical solutions are computed, and the results are compared to those predicted by the linear theory.

Engaging operations management students by designing the unit with industry focus

Operations management is an area concerned with the production of goods and services ensuring that business operations are efficient in utilizing resource and effective to meet customer requirements. It deals with the design and management of products, processes, services and supply chains and considers the acquisition, development, and effective and efficient utilization of resources. Unlike other engineering subjects, content of these units could be very wide and vast. It is therefore necessary to cover the content that is most related to the contemporary industries. It is also necessary to understand what engineering management skills are critical for engineers working in the contemporary organisations. Most of the operations management books contain traditional Operations Management techniques. For example ‘inventory management’ is an important topic in operations management. All OM books deal with effective method of inventory management. However, new trend in OM is Just in time (JIT) delivery or minimization of inventory. It is therefore important to decide whether to emphasise on keeping inventory (as suggested by most books) or minimization of inventory. Similarly, for OM decisions like forecasting, optimization and linear programming most organisations now a day’s use software. Now it is important for us to determine whether some of these software need to be introduced in tutorial/ lab classes. If so, what software? It is established in the Teaching and Learning literature that there must be a strong alignment between unit objectives, assessment and learning activities to engage students in learning. Literature also established that engaging students is vital for learning. However, engineering units (more specifically Operations management) is quite different from other majors. Only alignment between objectives, assessment and learning activities cannot guarantee student engagement. Unit content must be practical oriented and skills to be developed should be those demanded by the industry. Present active learning research, using a multi-method research approach, redesigned the operations management content based on latest developments in Engineering Management area and the necessity of Australian industries. The redesigned unit has significantly helped better student engagement and better learning. It was found that students are engaged in the learning if they find the contents are helpful in developing skills that are necessary in their practical life.

Bow and stern flows with constant vorticity

Free surface flows of a rotational fluid past a two-dimensional semi-infinite body are considered. The fluid is assumed to be inviscid, incompressible, and of finite depth. A boundary integral method is used to solve the problem for the case where the free surface meets the body at a stagnation point. Supercritical solutions which satisfy the radiation condition are found for various values of the Froude number and the dimensionless vorticity. Subcritical solutions are also found; however these solutions violate the radiation condition and are characterized by a train of waves upstream. It is shown numerically that the amplitude of these waves increases as each of the Froude number, vorticity and height of the body above the bottom increases.

Smoothly attaching bow flows with constant vorticity

The free surface flow of a finite depth fluid past a semi-infinite body is considered. The fluid is assumed to have constant vorticity throughout and the free surface is assumed to attach smoothly to the front face of the body. Numerical solutions are found using a boundary integral method in the physical plane and it is shown that solutions exist for all supercritical Froude numbers. The related problem of the cusp-like flow due to a submerged sink in a corner is also considered. Vorticity is included in the flow and it is shown that the behaviour of the solutions is qualitatively the same as that found in the problem described above.

Minimising wave drag for free surface flow past a two-dimensional stern

Free surface flow past a two-dimensional semi-infinite curved plate is considered, with emphasis given to solving for the shape of the resulting wave train that appears downstream on the surface of the fluid. This flow configuration can be interpreted as applying near the stern of a wide blunt ship. For steady flow in a fluid of finite depth, we apply the Wiener-Hopf technique to solve a linearised problem, valid for small perturbations of the uniform stream. Weakly nonlinear results found using a forced KdV equation are also presented, as are numerical solutions to the fully nonlinear problem, computed using a conformal mapping and a boundary integral technique. By considering different families of shapes for the semi-infinite plate, it is shown how the amplitude of the waves can be minimised. For plates that increase in height as a function of the direction of flow, reach a local maximum, and then point slightly downwards at the point at which the free surface detaches, it appears the downstream wavetrain can be eliminated entirely.

A control and surveillance ITS location model to improve safety

Traffic safety in rural highways can be considered as a constant source of concern in many countries. Nowadays, transportation professionals widely use Intelligent Transportation Systems (ITS) to address safety issues. However, compared to metropolitan applications, the rural highway (non-urban) ITS applications are still not well defined. This paper provides a comprehensive review on the existing ITS safety solutions for rural highways. This research is mainly focused on the infrastructure-based control and surveillance ITS technology, such as Crash Prevention and Safety, Road Weather Management and other applications, that is directly related to the reduction of frequency and severity of accidents. The main outcome of this research is the development of a ‘ITS control and surveillance device locating model’ to achieve the maximum safety benefit for rural highways. Using cost and benefits databases of ITS, an integer linear programming method is utilized as an optimization technique to choose the most suitable set of ITS devices. Finally, computational analysis is performed on an existing highway in Iran, to validate the effectiveness of the proposed locating model.

On the natural convection boundary layer adjacent to an inclined flat plate subject to ramp heating

An investigation of the natural convection boundary layer adjacent to an inclined semi-infinite plate subject to a temperature boundary condition which follows a ramp function up until some specified time and then remains constant is reported. The development of the flow from start-up to a steadystate has been described based on scaling analyses and verified by numerical simulations. Attention in this study has been given to fluids having a Prandtl number Pr less than unity. The boundary layer flow depends on the comparison of the time at which the ramp heating is completed and the time at which the boundary layer completes its growth. If the ramp time is long compared with the steady state time, the layer reaches a quasi steady mode in which the growth of the layer is governed solely by the thermal balance between convection and conduction. On the other hand, if the ramp is completed before the layer becomes steady; the subsequent growth is governed by the balance between buoyancy and inertia, as for the case of instantaneous heating.

Scaling analysis of unsteady natural convection boundary layer for ramp heating

A scaling analysis for the natural convection boundary layer adjacent to an inclined semi-infinite plate subject to a non-instantaneous heating in the form of an imposed wall temperature which increases linearly up to a prescribed steady value over a prescribed time is reported. The development of the flow from start-up to a steady-state has been described based on scaling analyses and verified by numerical simulations. The analysis reveals that, if the period of temperature growth on the wall is sufficiently long, the boundary layer reaches a quasisteady mode before the growth of the temperature is completed. In this mode the thermal boundary layer at first grows in thickness and then contracts with increasing time. However, if the imposed wall temperature growth period is sufficiently short, the boundary layer develops differently, but after the wall temperature growth is completed, the boundary layer develops as though the start up had been instantaneous. The steady state values of the boundary layer for both cases are ultimately the same.

An improved boundary layer scaling with ramp heating on a sloping plate

A scaling analysis for the natural convection boundary layer adjacent to an inclined semi-infinite plate subject to a non-instantaneous heating in the form of an imposed wall temperature which increases linearly up to a prescribed steady value over a prescribed time is reported. The development of the boundary layer flow from start-up to a steady-state has been described based on scaling analyses and verified by numerical simulations. The analysis reveals that, if the period of temperature growth on the wall is sufficiently long, the boundary layer reaches a quasi-steady mode before the growth of the temperature is completed. In this mode the thermal boundary layer at first grows in thickness and then contracts with increasing time. However, if the imposed wall temperature growth period is sufficiently short, the boundary layer develops differently, but after the wall temperature growth is completed, the boundary layer develops as though the startup had been instantaneous. The steady state values of the boundary layer for both cases are ultimately the same.

Prandtl number scaling of the unsteady natural convection boundary layer adjacent to a vertical flat plate for Pr > 1 subject to ramp surface heat flux

It is found in the literature that the existing scaling results for the boundary layer thickness, velocity and steady state time for the natural convection flow over an evenly heated plate provide a very poor prediction of the Prandtl number dependency of the flow. However, those scalings provide a good prediction of two other governing parameters’ dependency, the Rayleigh number and the aspect ratio. Therefore, an improved scaling analysis using a triple-layer integral approach and direct numerical simulations have been performed for the natural convection boundary layer along a semi-infinite flat plate with uniform surface heat flux. This heat flux is a ramp function of time, where the temperature gradient on the surface increases with time up to some specific time and then remains constant. The growth of the boundary layer strongly depends on the ramp time. If the ramp time is sufficiently long, the boundary layer reaches a quasi steady mode before the growth of the temperature gradient is completed. In this mode, the thermal boundary layer at first grows in thickness and then contracts with increasing time. However, if the ramp time is sufficiently short, the boundary layer develops differently, but after the wall temperature gradient growth is completed, the boundary layer develops as though the startup had been instantaneous.