Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

Stability of non-parabolic flow in a flexible tube

Flows with velocity profiles very different from the parabolic velocity profile can occur in the entrance region of a tube as well as in tubes with converging/diverging cross-sections. In this paper, asymptotic and numerical studies are undertaken to analyse the temporal stability of such 'non-parabolic' flows in a flexible tube in the limit of high Reynolds numbers. Two specific cases are considered: (i) developing flow in a flexible tube; (ii) flow in a slightly converging flexible tube. Though the mean velocity profile contains both axial and radial components, the flow is assumed to be locally parallel in the stability analysis. The fluid is Newtonian and incompressible, while the flexible wall is modelled as a viscoelastic solid. A high Reynolds number asymptotic analysis shows that the non-parabolic velocity profiles can become unstable in the inviscid limit. This inviscid instability is qualitatively different from that observed in previous studies on the stability of parabolic flow in a flexible tube, and from the instability of developing flow in a rigid tube. The results of the asymptotic analysis are extended numerically to the moderate Reynolds number regime. The numerical results reveal that the developing flow could be unstable at much lower Reynolds numbers than the parabolic flow, and hence this instability can be important in destabilizing the fluid flow through flexible tubes at moderate and high Reynolds number. For flow in a slightly converging tube, even small deviations from the parabolic profile are found to be sufficient for the present instability mechanism to be operative. The dominant non-parallel effects are incorporated using an asymptotic analysis, and this indicates that non-parallel effects do not significantly affect the neutral stability curves. The viscosity of the wall medium is found to have a stabilizing effect on this instability.

Persistent passive hopping and juggling is possible even with plastic collisions

We describe simple one-dimensional models of passive (no energy input, no control), generally dissipative, vertical hopping and one-ball juggling. The central observation is that internal passive system motions can conspire to eliminate collisions in these systems. For hopping, two point masses are connected by a spring and the lower mass has inelastic collisions with the ground. For juggling, a lower point-mass hand is connected by a spring to the ground and an upper point-mass ball is caught with an inelastic collision and then re-thrown into gravitational free flight. The two systems have identical dynamics. Despite inelastic collisions between non-zero masses, these systems have special symmetric energy-conserving periodic motions where the collision is at zero relative velocity. Additionally, these special periodic motions have a non-zero sized, one-sided region of attraction on the higher-energy side. For either very large or very small mass ratios, the one-sided region of attraction is large. These results persist for mildly non-linear springs and non-constant gravity. Although non-collisional damping destroys the periodic motions, small energy injection makes the periodic motions stable, with a two-sided region of attraction. The existence of such special energy conserving solutions for hopping and juggling points to possibly useful strategies for both animals and robots. The lossless motions are demonstrated with a table-top experiment.

Shock tunnel study of spiked aerodynamic bodies flying at hypersonic Mach numbers

A three-component accelerometer balance system is used to study the drag reduction effect of an aerodisc on large angle blunt cones flying at hypersonic Mach numbers. Measurements in a hypersonic shock tunnel at a freestream Mach number of 5.75 indicate more than 50% reduction in the drag coefficient for a 120degrees apex angle blunt cone with a forward facing aerospike having a flat faced aerodisc at moderate angles of attack. Enhancement of drag has been observed for higher angles of attack due to the impingement of the flow separation shock on the windward side of the cone. The flowfields around the large angle blunt cone with aerospike assembly flying at hypersonic Mach numbers are also simulated numerically using a commercial CFD code. The pressure and density levels on the model surface, which is under the aerodynamic shadow of the flat disc tipped spike, are found very low and a drag reduction of 64.34% has been deduced numerically.

Thermal instability in a three-dimensional rigid container with prescribed heat flux at lower boundary

The Bénard–Marangoni convection is studied in a three-dimensional container with thermally insulated lateral walls and prescribed heat flux at lower boundary. The upper surface of the incompressible, viscous fluid is assumed to be flat with temperature dependent surface tension. A Galerkin–Tau method with odd and even trial functions satisfying all the essential boundary conditions except the natural boundary conditions at the free surface has been used to solve the problem. The critical Marangoni and Rayleigh numbers are determined for the onset of steady convection as a function of aspect ratios x0 and y0 for the cases of Bénard–Marangoni, pure Marangoni and pure Bénard convections. It is observed that critical parameters are decreasing with an increase in aspect ratios. The flow structures corresponding to the values of the critical parameters are presented in all the cases. It is observed that the critical parameters are higher for case with heat flux prescribed than those corresponding to the case with prescribed temperature. The critical Marangoni number for pure Marangoni convection is higher than critical Rayleigh number corresponding to pure Bénard convection for a given aspect ratio whereas the reverse was observed for two-dimensional infinite layer.

Almost quarter-pinched Kähler metrics and Chern numbers

Given n is an element of Z(+) and epsilon > 0, we prove that there exists delta = delta(epsilon, n) > 0 such that the following holds: If (M(n),g) is a compact Kahler n-manifold whose sectional curvatures K satisfy -1 -delta <= K <= -1/4 and c(I)(M), c(J)(M) are any two Chern numbers of M, then |c(I)(M)/c(J)(M) - c(I)(0)/c(J)(0)| < epsilon, where c(I)(0), c(J)(0) are the corresponding characteristic numbers of a complex hyperbolic space form. It follows that the Mostow-Siu surfaces and the threefolds of Deraux do not admit Kahler metrics with pinching close to 1/4.

Transition to turbulence in the free convection boundary layers on an inclined heated plate

An experimental study has been made of transition to turbulence in the free convective flows on a heated plate. Observations have been made with the plate vertical and inclined at angles up to about 50° to the vertical, both above and below the plate. A fibre anemometer was used to survey the region of intermittent turbulence. Information has thus been obtained about the range of Grashof numbers over which transition takes place. Even when the plate is vertical the region of intermittent turbulence is long. When it is inclined, this region becomes still longer in the flow below the plate as a result of the stabilizing stratification, a Richardson number effect. It is possible to have a whole flow such that it should be described as transitional, not laminar or turbulent. It was noticed that in this flow and the vertical plate one, the velocity during the laminar periods could be either of two characteristic values, one of them close to zero. The behaviour above an inclined plate could be interpreted largely as a trend towards the behaviour described in a preceding paper.

On cavity flow at high Reynolds numbers

The flow in a square cavity is studied by solving the full Navier–Stokes and energy equations numerically, employing finite-difference techniques. Solutions are obtained over a wide range of Reynolds numbers from 0 to 50000. The solutions show that only at very high Reynolds numbers (Re [gt-or-equal, slanted] 30000) does the flow in the cavity completely correspond to that assumed by Batchelor's model for separated flows. The flow and thermal fields at such high Reynolds numbers clearly exhibit a boundary-layer character. For the first time, it is demonstrated that the downstream secondary eddy grows and decays in a manner similar to the upstream one. The upstream and downstream secondary eddies remain completely viscous throughout the range of Reynolds numbers of their existence. It is suggested that the behaviour of the secondary eddies may be characteristic of internal separated flows.

Fluid Mechanics of Aquatic Locomotion at Large Reynolds Numbers

Abstract | There exist a huge range of fish species besides other aquatic organisms like squids and salps that locomote in water at large Reynolds numbers, a regime of flow where inertial forces dominate viscous forces. In the present review, we discuss the fluid mechanics governing the locomotion of such organisms. Most fishes propel themselves by periodic undulatory motions of the body and tail, and the typical classification of their swimming modes is based on the fraction of their body that undergoes such undulatory motions. In the angulliform mode, or the eel type, the entire body undergoes undulatory motions in the form of a travelling wave that goes from head to tail, while in the other extreme case, the thunniform mode, only the rear tail (caudal fin) undergoes lateral oscillations. The thunniform mode of swimming is essentially based on the lift force generated by the airfoil like crosssection of the fish tail as it moves laterally through the water, while the anguilliform mode may be understood using the “reactive theory” of Lighthill. In pulsed jet propulsion, adopted by squids and salps, there are two components to the thrust; the first due to the familiar ejection of momentum and the other due to an over-pressure at the exit plane caused by the unsteadiness of the jet. The flow immediately downstream of the body in all three modes consists of vortex rings; the differentiating point being the vastly different orientations of the vortex rings. However, since all the bodies are self-propelling, the thrust force must be equal to the drag force (at steady speed), implying no net force on the body, and hence the wake or flow downstream must be momentumless. For such bodies, where there is no net force, it is difficult to directly define a propulsion efficiency, although it is possible to use some other very different measures like “cost of transportation” to broadly judge performance.

Stability numbers for an unsupported vertical circular excavation in c-phi soil

By using an axisymmetric lower bound finite element limit analysis formulation, the stability numbers (gamma H/C) for an unsupported vertical circular excavation in a cohesive-frictional soil have been generated. The numerical results are obtained for values of normalized excavation height (H/b) and friction angle (phi) greater than those considered previously in the literature. The results compare well with those available in literature. The stability numbers presented in this note would be beneficial from a design point of view. (C) 2011 Elsevier Ltd. All rights reserved.

Numerical analysis of transpiration cooling in carbon dioxide atmosphere at hypersonic Mach numbers

The numerical solutions are obtained for skin friction, heat transfer to the wall and growth of boundary layer along the flat plate by employing two dimensional Navier-Stokes equations governing the hypersonic flow coupled with species continuity equations. Flow fields have been computed along the flat plate in CO2 atmosphere in the presence of transpiration cooling using air and carbon dioxide.

Badly approximable numbers and vectors in Cantor-like sets

We show that a large class of Cantor-like sets of R-d, d >= 1, contains uncountably many badly approximable numbers, respectively badly approximable vectors, when d >= 2. An analogous result is also proved for subsets of R-d arising in the study of geodesic flows corresponding to (d+1)-dimensional manifolds of constant negative curvature and finite volume, generalizing the set of badly approximable numbers in R. Furthermore, we describe a condition on sets, which is fulfilled by a large class, ensuring a large intersection with these Cantor-like sets.

Cumulus clouds and convective boundary layers: a tropical perspective on two turbulent shear flows

This paper is a review prepared for the second Marseille Colloquium on the mechanics of turbulence, held in 2011, 50 years after the first. The review covers recent developments in our understanding of the large-scale dynamics of cumulus cloud flows and of the atmospheric boundary layer in the low-wind convective regime that is often encountered in the tropics. It has recently been shown that a variety of cumulus cloud forms and life cycles can be experimentally realized in the laboratory, with the transient diabatic plume taken as the flow model for a cumulus cloud. The plume is subjected to diabatic heating scaled to be dynamically similar to heat release from phase changes in clouds. The experiments are complemented by exact numerical solutions of the Navier-Stokes-Boussinesq equations for plumes with scaled off-source heating. The results show that the Taylor entrainment coefficient first increases with heating, reaches a positive maximum and then drops rapidly to zero or even negative values. This reduction in entrainment is a consequence of structural changes in the flow, smoothing out the convoluted boundaries in the non-diabatic plume, including the tongues engulfing the ambient flow. This is accompanied by a greater degree of mixedness in the core flow because of lower dilution by the ambient fluid. The cloud forms generated depend strongly on the history of the diabatic heating profile in the vertical direction. The striking effects of heating on the flow are attributable to the operation of the baroclinic torque due to the temperature field. The mean baroclinic torque is shown to peak around a quasi-cylindrical sheet situated midway between the axis of the flow and the edges. This torque is shear-enhancing and folds down the engulfment tongues. The increase in mixedness can be traced to an explosive growth in the enstrophy, triggered by a strong fluctuating baroclinic torque that acts as a source, especially at the higher wave numbers, thus enhancing the mixedness. In convective boundary layers field measurements show that, under conditions prevailing in the tropics, the eddy fluxes of momentum and energy do not follow the Monin-Obukhov similarity. Instead, the eddy momentum flux is found to be linear in the wind speed at low winds; and the eddy heat flux is, to a first approximation, governed by free convection laws, with wind acting as a small perturbation on a regime of free convection. A new boundary layer code, based on heat flux scaling rather than wall-stress scaling, shows promising improvements in predictive skills of a general circulation model.