A reinforcement learning based algorithm for finite horizon Markov decision processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

Nonadiabatic charge pumping by oscillating potentials in one dimension: Results for infinite system and finite ring

We study charge pumping when a combination of static potentials and potentials oscillating with a time period T is applied in a one-dimensional system of noninteracting electrons. We consider both an infinite system using the Dirac equation in the continuum approximation and a periodic ring with a finite number of sites using the tight-binding model. The infinite system is taken to be coupled to reservoirs on the two sides which are at the same chemical potential and temperature. We consider a model in which oscillating potentials help the electrons to access a transmission resonance produced by the static potentials and show that nonadiabatic pumping violates the simple sin phi rule which is obeyed by adiabatic two-site pumping. For the ring, we do not introduce any reservoirs, and we present a method for calculating the current averaged over an infinite time using the time evolution operator U(T) assuming a purely Hamiltonian evolution. We analytically show that the averaged current is zero if the Hamiltonian is real and time-reversal invariant. Numerical studies indicate another interesting result, namely, that the integrated current is zero for any time dependence of the potential if it is applied to only one site. Finally we study the effects of pumping at two sites on a ring at resonant and nonresonant frequencies, and show that the pumped current has different dependences on the pumping amplitude in the two cases.

Infinite dimensional slow modulations in a well known delayed model for cutting tool vibrations

We apply the method of multiple scales (MMS) to a well-known model of regenerative cutting vibrations in the large delay regime. By ``large'' we mean the delay is much larger than the timescale of typical cutting tool oscillations. The MMS up to second order, recently developed for such systems, is applied here to study tool dynamics in the large delay regime. The second order analysis is found to be much more accurate than the first order analysis. Numerical integration of the MMS slow flow is much faster than for the original equation, yet shows excellent accuracy in that plotted solutions of moderate amplitudes are visually near-indistinguishable. The advantages of the present analysis are that infinite dimensional dynamics is retained in the slow flow, while the more usual center manifold reduction gives a planar phase space; lower-dimensional dynamical features, such as Hopf bifurcations and families of periodic solutions, are also captured by the MMS; the strong sensitivity of the slow modulation dynamics to small changes in parameter values, peculiar to such systems with large delays, is seen clearly; and though certain parameters are treated as small (or, reciprocally, large), the analysis is not restricted to infinitesimal distances from the Hopf bifurcation.

Cooling of an infinite slab in a two-fluid medium

A mixed boundary-valued problem associated with the diffusion equation, that involves the physical problem of cooling of an infinite slab in a two-fluid medium, is solved completely by using the Wiener-Hopf technique. An analytical solution is derived for the temperature distribution at the quench fronts being created by two different layers of cold fluids having different cooling abilities moving on the upper surface of the slab at constant speed. Simple expressions are derived for the values of the sputtering temperatures of the slab at the points of contact with the respective layers, assuming one layer of the fluid to be of finite extent and the other of infinite extent. The main problem is solved through a three-part Wiener - Hopf problem of a special type, and the numerical results under certain special circumstances are obtained and presented in the form of a table.

Random sequential multilayer deposition of different‐sized k mers on a one dimensional infinite substrate

Kinetics of random sequential, irreversible multilayer deposition of macromolecules of two different sizes on a one dimensional infinite lattice is analyzed at the mean field level. A formal solution for the corresponding rate equation is obtained. The Jamming limits and the distribution of gaps of exact sizes are discussed. In the absence of screening, the jamming limits are shown to be the same for all the layers. A detailed analysis for the components differing by one monomer unit is presented. The small and large time behaviors and the dependence of the individual jamming limits of the k mers and (k−1) mers on k and the rate parameters are analyzed.

Transient Flow of a Compressible Viscous Fluid Near an Infinite Disk

We have consider ed the transient motion of art electrically conducting viscous compressible fluid which is in contact with an insulated infinite disk. The initial motion is considered to be due to the uniform rotation of the disk in an otherwise stationary fluid or due to the uniform rigid rotation of the fluid over a stationary disk. Different cases of transient motion due to finite impulse imparted either to the disk or to the distant fluid have been investigated. Effects of the imposed axial magnetic field and the disk temperature on the transient flow are included. The nonlinear partial differential equations governing the motion are solved numerically using an implicit finite-difference scheme along with the Newton's linearisation technique.

A transfer matrix approach for evaluation of the response of a multi-layer infinite plate to a two-dimensional pressure excitation

A 6 X 6 transfer matrix is presented to evaluate the response of a multi-layer infinite plate to a given two-dimensional pressure excitation on one of its faces or, alternatively, to evaluate the acoustic pressure distribution excited by the normal velocity components of the radiating surfaces. It is shown that the present transfer matrix is a general case embodying the transfer matrices of normal excitation and one-dimensional pressure excitation due to an oblique incident wave. It is also shown that the present transfer matrix obeys the necessary checks to categorize the physically symmetric multi-layer plate as dynamically symmetric. Expressions are derived to obtain the wave propagation parameters, such as the transmission, absorption and reflection coefficients, in terms of the elements of the transfer matrix presented. Numerical results for transmission loss and reflection coefficients of a two-layer configuration are presented to illustrate the effect of angles of incidence, layer characteristics and ambient media.

Spin distributions in antiferromagnetically coupled Mn2+Cu2+ systems: from the pair to the infinite chain

Probably the most informative description of the ground slate of a magnetic molecular species is provided by the spin density map. Such a map may be experimentally obtained from polarized neutron diffraction (PND) data or theoretically calculated using quantum chemical approaches. Density functional theory (DFT) methods have been proved to be well-adapted for this. Spin distributions in one-dimensional compounds may also be computed using the density matrix renormalization group (DMRG) formalism. These three approaches, PND, DFT, and DMRG, have been utilized to obtain new insights on the ground state of two antiferromagnetically coupled Mn2+Cu2+ compounds, namely [Mn(Me-6-[14]ane-N-4)Cu(oxpn)](CF3SO3)(2) and MnCu(pba)(H2O)(3) . 2H(2)O, with Me-6-[14]ane-N-4 = (+/-)-5,7,7,12,14,14-hexamethyl-1,4,8,11-tetraazacyclotetradecane, oxpn = N,N'-bis(3-aminopropyl)oxamido and pba = 1,3-propylenebis(oxamato). Three problems in particular have been investigated: the spin distribution in the mononuclear precursors [Cu(oxpn)] and [Cu(pba)](2-), the spin density maps in the two Mn2+Cu2+ compounds, and the evolution of the spin distributions on the Mn2+ and Cu2+ sites when passing from a pair to a one-dimensional ferrimagnet.

Optimal control of a stochastic hybrid system with discounted cost

We address the optimal control problem of a very general stochastic hybrid system with both autonomous and impulsive jumps. The planning horizon is infinite and we use the discounted-cost criterion for performance evaluation. Under certain assumptions, we show the existence of an optimal control. We then derive the quasivariational inequalities satisfied by the value function and establish well-posedness. Finally, we prove the usual verification theorem of dynamic programming.

Unsteady flow and heat transfer on a semi-infinite flat plate with an aligned magnetic field

The unsteady laminar boundary layer flow of an electrically conducting fluid past a semi-infinite flat plate with an aligned magnetic field has been studied when at time t > 0 the plate is impulsively moved with a constant velocity which is in the same or opposite direction to that of free stream velocity. The effect of the induced magnetic field has been included in the analysis. The non-linear partial differential equations have been solved numerically using an implicit finite-difference method. The effect of the impulsive motion of the surface is found to be more pronounced on the skin friction but its effect on the x-component of the induced magnetic field and heat transfer is small. Velocity defect occurs near the surface when the plate is impulsively moved in the same direction as that of the free stream velocity. The surface shear stress, x-component of the induced magnetic field on the surface and the surface heat transfer decrease with an increasing magnetic field, but they increase with the reciprocal of the magnetic Prandtl number. However, the effect of the reciprocal of the magnetic Prandtl number is more pronounced on the x-component of the induced magnetic field. (C) 1999 Elsevier Science Ltd. All rights reserved.

An inverse problem for the Helmholtz equation involving two semi-infinite fluids

An analytical method is developed for solving an inverse problem for Helmholtz's equation associated with two semi-infinite incompressible fluids of different variable refractive indices, separated by a plane interface. The unknowns of the inverse problem are: (i) the refractive indices of the two fluids, (ii) the ratio of the densities of the two fluids, and (iii) the strength of an acoustic source assumed to be situated at the interface of the two fluids. These are determined from the pressure on the interface produced by the acoustic source. The effect of the surface tension force at the interface is taken into account in this paper. The application of the proposed analytical method to solve the inverse problem is also illustrated with several examples. In particular, exact solutions of two direct problems are first derived using standard classical methods which are then used in our proposed inverse method to recover the unknowns of the corresponding inverse problems. The results are found to be in excellent agreement.