Non-Stationary Semi-Markov Decision Processes on a Finite Horizon

We introduce and study a class of non-stationary semi-Markov decision processes on a finite horizon. By constructing an equivalent Markov decision process, we establish the existence of a piecewise open loop relaxed control which is optimal for the finite horizon problem.

A novel Q-learning algorithm with function approximation for constrained Markov decision processes

We present a novel multi-timescale Q-learning algorithm for average cost control in a Markov decision process subject to multiple inequality constraints. We formulate a relaxed version of this problem through the Lagrange multiplier method. Our algorithm is different from Q-learning in that it updates two parameters - a Q-value parameter and a policy parameter. The Q-value parameter is updated on a slower time scale as compared to the policy parameter. Whereas Q-learning with function approximation can diverge in some cases, our algorithm is seen to be convergent as a result of the aforementioned timescale separation. We show the results of experiments on a problem of constrained routing in a multistage queueing network. Our algorithm is seen to exhibit good performance and the various inequality constraints are seen to be satisfied upon convergence of the algorithm.

Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.

Ergodic control of partially observed Markov chains

The ergodic or long-run average cost control problem for a partially observed finite-state Markov chain is studied via the associated fully observed separated control problem for the nonlinear filter. Dynamic programming equations for the latter are derived, leading to existence and characterization of optimal stationary policies.

Study of the processes occurring when sputtering YBa2Cu3O7-x in pure oxygen

The sputter deposition of YBa2Cu3O7-x in a de-diode was performed in pure oxygen medium and an optical spectroscopic study of the resultant discharge revealed strong emissions from both metal atoms and oxygen ions. Emission intensities were studied in pressure range from 0.5 to 3 mbar, with substrate temperatures from 150 to 850 degrees C. Raising the substrate temperature to 850 degrees C increased the number of positive ions and excited neutral atoms. Raising the pressure decreased the emission intensities of excited neutral and ionic species. The results have been compared with those obtained from Langmuir probe measurements. The rise in emission intensities of excited neutrals and ions with temperature suggested the possibility of chemically enhanced physical sputtering of YBa2Cu3O7-x. The effect of process conditions on film composition and quality is also discussed.

Influence of friction during forming processes-a study using a numerical simulation technique

Friction has an important influence in metal forming operations, as it contributes to the success or otherwise of the process. In the present investigation, the effect of friction on metal forming was studied by simulating compression tests on cylindrical Al-Mg alloy using the finite element method (FEM) technique. Three kinds of compression tests were considered wherein a constant coefficient of friction was employed at the upper die-work-piece interface. However, the coefficient of friction between the lower die-work-piece interfaces was varied in the tests. The simulation results showed that a difference in metal flow occurs near the interfaces owing to the differences in the coefficient of friction. It was concluded that the variations in the coefficient of friction between the dies and the work-piece directly affect the stress distribution and shape of the work-piece, having implications on the microstructure of the material being processed.

Thermal Degradation Processes in Poly(xylylene sulfides) Investigated by Comparative Direct Pyrolysis MS and Flash Pyrolysis GC/MS Experiments

The thermal degradation processes of two sulfur polymers, poly(xylylene sulfide) (PXM) and poly(xylylene disulfide) (PXD), were investigated in parallel by direct pyrolysis mass spectrometry (DPMS) and flash pyrolysis GC/MS (Py-GC/MS). Thermogravimetric data showed that these polymers decompose with two separate steps in the temperature ranges of 250-280 and 600-650 degrees C, leaving a high amount of residue (about 50% at 800 degrees C). The pyrolysis products detected by DPMS in the first degradation step of PXM and PXD were terminated by three types of end groups, -CH3, -CH2SH, and -CH=S, originating from thermal cleavage reactions involving a series of homolytic chain scissions followed by hydrogen transfer reactions, generating several oligomers containing some intact xylylene sulfide repeating units. The presence of pyrolysis compounds containing some stilbene-like units in the first degradation step has also been observed. Their formation has been accounted for with a parallel cleavage involving the elimination of H2S from the PXM main chains. These unsaturated units can undergo cross-linking at higher temperatures, producing the high amount of char residue observed. The thermal degradation compounds detected by DPMS in the second decomposition step at about 600-650 degrees C were constituted of condensed aromatic molecules containing dihydrofenanthrene and fenanthrene units. These compounds might be generated from the polymer chains containing stilbene units, by isomerization and dehydrogenation reactions. The pyrolysis products obtained in the Py-GC/MS of PXM and PXD at 610 degrees C are almost identical. The relative abundance in the pyrolysate and the spectral properties of the main pyrolysis products were found to be in generally good agreement with those obtained by DPMS. Polycyclic aromatic hydrocarbons (PAHs) were also detected by Py-GC/MS but in minor amounts with respect to DPMS. This apparent discrepancy was due to the simultaneous detection of PAHs together with all pyrolysis products in the Py-GC/MS, whereas in DPMS they were detected in the second thermal degradation step without the greatest part of pyrolysis compounds generated in the first degradation step. The results obtained by DPMS and PSI-GC/MS experiments showed complementary data for the degradation of PXM and PXD and, therefore, allowed the unequivocal formulation of the thermal degradation mechanism for these sulfur-containing polymers.

Risk Minimizing Option Pricing in a Semi-Markov Modulated Market

We address risk minimizing option pricing in a semi-Markov modulated market where the floating interest rate depends on a finite state semi-Markov process. The growth rate and the volatility of the stock also depend on the semi-Markov process. Using the Föllmer–Schweizer decomposition we find the locally risk minimizing price for European options and the corresponding hedging strategy. We develop suitable numerical methods for computing option prices.

Portfolio Optimization in a Semi-Markov Modulated Market

We address a portfolio optimization problem in a semi-Markov modulated market. We study both the terminal expected utility optimization on finite time horizon and the risk-sensitive portfolio optimization on finite and infinite time horizon. We obtain optimal portfolios in relevant cases. A numerical procedure is also developed to compute the optimal expected terminal utility for finite horizon problem.

Structural origin of set-reset processes in Ge15Te83Si2 glass investigated using in situ Raman scattering and transmission electron microscopy

We report here that the structural origin of an easily reversible Ge15Te83Si2 glass can be a promising candidate for phase change random access memories. In situ Raman scattering studies on Ge15Te83Si2 sample, undertaken during the amorphous set and reset processes, indicate that the degree of disorder in the glass is reduced from off to set state. It is also found that the local structure of the sample under reset condition is similar to that in the amorphous off state. Electron microscopic studies on switched samples indicate the formation of nanometric sized particles of c-SiTe2 structure. ©2009 American Institute of Physics

Curing And Thermal Cycling Processes In Pvc Graphite Thick-Films

Abstract is not available.

On a generalized dynamic model of bi-state social interaction processes

A non-linear model, construed as a generalized version of the models put forth earlier for the study of bi-state social interaction processes, is proposed in this study. The feasibility of deriving the dynamics of such processes is demonstrated by establishing equivalence between the non-linear model and a higher order linear model.

Optimal maintenance and replacement via a semi-Markov decision model

Optimal bang-coast maintenance policies for a machine, subject to failure, are considered. The approach utilizes a semi-Markov model for the system. A simplified model for modifying the probability of machine failure with maintenance is employed. A numerical example is presented to illustrate the procedure and results.

An analytical approach to the study of a class of bi-state social interaction processes

Sufficient conditions for obtaining an equivalent linear model to classes of non-linear, bi-state, social interaction processes are derived. These parametric constraints, when satisfied, permit analytical determination of the dynamics of the non-linear process of social interaction.

Unitary Processes with Independent Increments and Representations of Hilbert Tensor Algebras

The aim of this article is to characterize unitary increment process by a quantum stochastic integral representation on symmetric Fock space. Under certain assumptions we have proved its unitary equivalence to a Hudson-Parthasarathy flow.