Linear programming for large-scale Markov decision problems

We consider the problem of controlling a Markov decision process (MDP) with a large state space, so as to minimize average cost. Since it is intractable to compete with the optimal policy for large scale problems, we pursue the more modest goal of competing with a low-dimensional family of policies. We use the dual linear programming formulation of the MDP average cost problem, in which the variable is a stationary distribution over state-action pairs, and we consider a neighborhood of a low-dimensional subset of the set of stationary distributions (defined in terms of state-action features) as the comparison class. We propose a technique based on stochastic convex optimization and give bounds that show that the performance of our algorithm approaches the best achievable by any policy in the comparison class. Most importantly, this result depends on the size of the comparison class, but not on the size of the state space. Preliminary experiments show the effectiveness of the proposed algorithm in a queuing application.

Linear and weakly nonlinear stability analyses of cooperative car-following models

Stability analyses have been widely used to better understand the mechanism of traffic jam formation. In this paper, we consider the impact of cooperative systems (a.k.a. connected vehicles) on traffic dynamics and, more precisely, on flow stability. Cooperative systems are emerging technologies enabling communication between vehicles and/or with the infrastructure. In a distributed communication framework, equipped vehicles are able to send and receive information to/from other equipped vehicles. Here, the effects of cooperative traffic are modeled through a general bilateral multianticipative car-following law that improves cooperative drivers' perception of their surrounding traffic conditions within a given communication range. Linear stability analyses are performed for a broad class of car-following models. They point out different stability conditions in both multianticipative and nonmultianticipative situations. To better understand what happens in unstable conditions, information on the shock wave structure is studied in the weakly nonlinear regime by the mean of the reductive perturbation method. The shock wave equation is obtained for generic car-following models by deriving the Korteweg de Vries equations. We then derive traffic-state-dependent conditions for the sign of the solitary wave (soliton) amplitude. This analytical result is verified through simulations. Simulation results confirm the validity of the speed estimate. The variation of the soliton amplitude as a function of the communication range is provided. The performed linear and weakly nonlinear analyses help justify the potential benefits of vehicle-integrated communication systems and provide new insights supporting the future implementation of cooperative systems.

Improved linear cryptanalysis of reduced-round SIMON-32 and SIMON-48

In this paper we analyse two variants of SIMON family of light-weight block ciphers against variants of linear cryptanalysis and present the best linear cryptanalytic results on these variants of reduced-round SIMON to date. We propose a time-memory trade-off method that finds differential/linear trails for any permutation allowing low Hamming weight differential/linear trails. Our method combines low Hamming weight trails found by the correlation matrix representing the target permutation with heavy Hamming weight trails found using a Mixed Integer Programming model representing the target differential/linear trail. Our method enables us to find a 17-round linear approximation for SIMON-48 which is the best current linear approximation for SIMON-48. Using only the correlation matrix method, we are able to find a 14-round linear approximation for SIMON-32 which is also the current best linear approximation for SIMON-32. The presented linear approximations allow us to mount a 23-round key recovery attack on SIMON-32 and a 24-round Key recovery attack on SIMON-48/96 which are the current best results on SIMON-32 and SIMON-48. In addition we have an attack on 24 rounds of SIMON-32 with marginal complexity.

Modelling of the evaporation of a droplet suspended in a binary atmosphere

The process of spray drying is applied in a number of contexts. One such application is the production of a synthetic rock used for storage of nuclear waste. To establish a framework for a model of the spray drying process for this application, we here develop a model describing evaporation from droplets of pure water, such that the model may be extended to account for the presence of colloid within the droplet. We develop a spherically-symmetric model and formulate continuum equations describing mass, momentum, and energy balance in both the liquid and gas phases from first principles. We establish appropriate boundary conditions at the surface of the droplet, including a generalised Clapeyron equation that accurately describes the temperature at the surface of the droplet. To account for experiment design, we introduce a simplified platinum ball and wire model into the system using a thin wire problem. The resulting system of equations is transformed in order to simplify a finite volume solution scheme. The results from numerical simulation are compared with data collected for validation, and the sensitivity of the model to variations in key parameters, and to the use of Clausius–Clapeyron and generalised Clapeyron equations, is investigated. Good agreement is found between the model and experimental data, despite the simplicity of the platinum phase model.

Nonlinear Enzyme Kinetics Can Lead to High Metabolic Flux Control Coefficients: Implications for the Evolution of Dominance

In a classic study, Kacser & Burns (1981, Genetics 97, 639-666) demonstrated that given certain plausible assumptions, the flux in a metabolic pathway was more or less indifferent to the activity of any of the enzymes in the pathway taken singly. It was inferred from this that the observed dominance of most wild-type alleles with respect to loss-of-function mutations did not require an adaptive, meaning selectionist, explanation. Cornish-Bowden (1987, J. theor. Biol. 125, 333-338) showed that the Kacser-Burns inference was not valid when substrate concentrations were large relative to the relevant Michaelis constants. We find that in a randomly constructed functional pathway, even when substrate levels are small, one can expect high values of control coefficients for metabolic flux in the presence of significant nonlinearities as exemplified by enzymes with Hill coefficients ranging from two to six, or by the existence of oscillatory loops. Under these conditions the flux can be quite sensitive to changes in enzyme activity as might be caused by inactivating one of the two alleles in a diploid. Therefore, the phenomenon of dominance cannot be a trivial ''default'' consequence of physiology but must be intimately linked to the manner in which metabolic networks have been moulded by natural selection.

Ergodic control of partially observed Markov chains

The ergodic or long-run average cost control problem for a partially observed finite-state Markov chain is studied via the associated fully observed separated control problem for the nonlinear filter. Dynamic programming equations for the latter are derived, leading to existence and characterization of optimal stationary policies.

Reservoir Operation for Hydropower Optimization: A Chance-Constrained Approach

This paper presents a chance-constrained linear programming formulation for reservoir operation of a multipurpose reservoir. The release policy is defined by a chance constraint that the probability of irrigation release in any period equalling or exceeding the irrigation demand is at least equal to a specified value P (called reliability level). The model determines the maximum annual hydropower produced while meeting the irrigation demand at a specified reliability level. The model considers variation in reservoir water level elevation and also the operating range within which the turbine operates. A linear approximation for nonlinear power production function is assumed and the solution obtained within a specified tolerance limit. The inflow into the reservoir is considered random. The chance constraint is converted into its deterministic equivalent using a linear decision rule and inflow probability distribution. The model application is demonstrated through a case study.

Mixed convection in non-Newtonian fluids along a vertical plate in a porous medium

The problem of mixed convection from vertical surfaces in a porous medium saturated with a power-law type non-Newtonian fluid is investigated. The transformed conservation laws are solved numerically for the case of variable wall temperature conditions. Results for the details of the velocity and temperature fields as well as the Nusselt number have been presented. The viscosity index ranged from 0.5-2.0.

Mixed Convection Boundary Layer Flow Past a Horizontal Circular Cylinder Embedded in a Bidisperse Porous Medium

Adopting a two-temperature and two-velocity model, appropriate to a bidisperse porous medium (BDPM) proposed by Nield and Kuznetsov (2008), the classical steady, mixed convection boundary layer flow about a horizontal, isothermal circular cylinder embedded in a porous medium has been theoretically studied in this article. It is shown that the boundary layer analysis leads to expressions for the flow and heat transfer characteristics in terms of an inter-phase momentum parameter, a thermal diffusivity ratio, a thermal conductivity ratio, a permeability ratio, a modified thermal capacity ratio, and a buoyancy or mixed convection parameter. The transformed partial differential equations governing the flow and heat transfer in the f-phase (the macro-pores) and the p-phase (the remainder of the structure) are solved numerically using a very efficient implicit finite-difference technique known as Keller-box method. A good agreement is observed between the present results and those known from the open literature in the special case of a traditional Darcy formulation (monodisperse system).

Einstein Relation in n-Channel Inversion Layers of Nonlinear Optical, Optoelectronic and Related Materials: Simplified Theory, Relative Comparison and Suggestion for an Experimental Determination

In this paper, we study the Einstein relation for the diffusivity to mobility ratio (DMR) in n-channel inversion layers of non-linear optical materials on the basis of a newly formulated electron dispersion relation by considering their special properties within the frame work of k.p formalism. The results for the n-channel inversion layers of III-V, ternary and quaternary materials form a special case of our generalized analysis. The DMR for n-channel inversion layers of II-VI, IV-VI and stressed materials has been investigated by formulating the respective 2D electron dispersion laws. It has been found, taking n-channel inversion layers of CdGeAs2, Cd(3)AS(2), InAs, InSb, Hg1-xCdxTe, In1-xGaxAsyP1-y lattice matched to InP, CdS, PbTe, PbSnTe, Pb1-xSnxSe and stressed InSb as examples, that the DMR increases with the increasing surface electric field with different numerical values and the nature of the variations are totally band structure dependent. The well-known expression of the DMR for wide gap materials has been obtained as a special case under certain limiting conditions and this compatibility is an indirect test for our generalized formalism. Besides, an experimental method of determining the 2D DMR for n-channel inversion layers having arbitrary dispersion laws has been suggested.

Different phenomenological theories and their abilities to describe the interdiffusion process in a binary system during multiphase growth

There are essentially two different phenomenological models available to describe the interdiffusion process in binary systems in the olid state. The first of these, which is used more frequently, is based on the theory of flux partitioning. The second model, developed much more recently, uses the theory of dissociation and reaction. Although the theory of flux partitioning has been widely used, we found that this theory does not account for the mobility of both species and therefore is not suitable for use in most interdiffusion systems. We have first modified this theory to take into account the mobility of both species and then further extended it to develop relations or the integrated diffusion coefficient and the ratio of diffusivities of the species. The versatility of these two different models is examined in the Co-Si system with respect to different end-member compositions. From our analysis, we found that the applicability of the theory of flux partitioning is rather limited but the theory of dissociation and reaction can be used in any binary system.

Field imperfection induced axial secular frequency shifts in nonlinear ion traps

This article develops a simple analytical expression that relates ion axial secular frequency to field aberration in ion trap mass spectrometers. Hexapole and octopole aberrations have been considered in the present computations. The equation of motion of the ions in a pseudopotential well with these superpositions has the form of a Duffing-like equation and a perturbation method has been used to obtain the expression for ion secular frequency as a function of field imperfections. The expression indicates that the frequency shift is sensitive to the sign of the octopole superposition and insensitive to the sign of the hexapole superposition. Further, for weak multipole superposition of the same magnitude, octopole superposition causes a larger frequency shift in comparison to hexapole superposition.

Finite Element Solution Of Surface-Tension Driven Flows In Laser Surface-Melting

Lasers are very efficient in heating localized regions and hence they find a wide application in surface treatment processes. The surface of a material can be selectively modified to give superior wear and corrosion resistance. In laser surface-melting and welding problems, the high temperature gradient prevailing in the free surface induces a surface-tension gradient which is the dominant driving force for convection (known as thermo-capillary or Marangoni convection). It has been reported that the surface-tension driven convection plays a dominant role in determining the melt pool shape. In most of the earlier works on laser-melting and related problems, the finite difference method (FDM) has been used to solve the Navier Stokes equations [1]. Since the Reynolds number is quite high in these cases, upwinding has been used. Though upwinding gives physically realistic solutions even on a coarse grid, the results are inaccurate. McLay and Carey have solved the thermo-capillary flow in welding problems by an implicit finite element method [2]. They used the conventional Galerkin finite element method (FEM) which requires that the pressure be interpolated by one order lower than velocity (mixed interpolation). This restricts the choice of elements to certain higher order elements which need numerical integration for evaluation of element matrices. The implicit algorithm yields a system of nonlinear, unsymmetric equations which are not positive definite. Computations would be possible only with large mainframe computers.Sluzalec [3] has modeled the pulsed laser-melting problem by an explicit method (FEM). He has used the six-node triangular element with mixed interpolation. Since he has considered the buoyancy induced flow only, the velocity values are small. In the present work, an equal order explicit FEM is used to compute the thermo-capillary flow in the laser surface-melting problem. As this method permits equal order interpolation, there is no restriction in the choice of elements. Even linear elements such as the three-node triangular elements can be used. As the governing equations are solved in a sequential manner, the computer memory requirement is less. The finite element formulation is discussed in this paper along with typical numerical results.

A probabilistic constrained nonlinear optimization framework to optimize RED parameters

The random early detection (RED) technique has seen a lot of research over the years. However, the functional relationship between RED performance and its parameters viz,, queue weight (omega(q)), marking probability (max(p)), minimum threshold (min(th)) and maximum threshold (max(th)) is not analytically availa ble. In this paper, we formulate a probabilistic constrained optimization problem by assuming a nonlinear relationship between the RED average queue length and its parameters. This problem involves all the RED parameters as the variables of the optimization problem. We use the barrier and the penalty function approaches for its Solution. However (as above), the exact functional relationship between the barrier and penalty objective functions and the optimization variable is not known, but noisy samples of these are available for different parameter values. Thus, for obtaining the gradient and Hessian of the objective, we use certain recently developed simultaneous perturbation stochastic approximation (SPSA) based estimates of these. We propose two four-timescale stochastic approximation algorithms based oil certain modified second-order SPSA updates for finding the optimum RED parameters. We present the results of detailed simulation experiments conducted over different network topologies and network/traffic conditions/settings, comparing the performance of Our algorithms with variants of RED and a few other well known adaptive queue management (AQM) techniques discussed in the literature.

Examining the profile and experiences of older sub-acute patients who present to public hospital emergency departments in South East Queensland: A mixed methods approach

There is a scarcity of research that informs Interface Health Service (IHS) development. This research applied a mixed methods approach to profile older emergency department patients and patterns of health service use and to explore their ED experiences in public hospital EDs in South-East Queensland. IHS was under-utilised by older people with complex co-morbidities. Lack of communication and need identification were factors that undermined the effectiveness of IHS in reaching this cohort which highlighted a need for change.