The actor-critic algorithm as multi-time-scale stochastic approximation

The actor-critic algorithm of Barto and others for simulation-based optimization of Markov decision processes is cast as a two time Scale stochastic approximation. Convergence analysis, approximation issues and an example are studied.

Regarding the effect of aging heat treatment on the failure behaviour of a SiC reinforced Al based metal matrix composite

In the present study, 6061 Al metallic matrix was reinforced by 12.2 wt% df SiC particulates using liquid metallurgy route. The composite material thus obtained was extruded and characterized in the as-solutionized and peak aged conditions in order to delineate the effect of aging associated precipitation of secondary phases on the tensile fracture behavior of the composite samples. The results' of microstructural characterization studies carried out using scanning electron microscope revealed the increased presence of precipitated secondary phases in the metallic matrix and a more pronounced interfacial segregation of alloying elements in case of peak aged samples when compared to the as-solutionized samples. The results of the fractographic studies conducted on the as-solutionized samples revealed that the failure was dominated by the SiC particulates cracking while for the peak aged samples the fracture surface revealed a comparatively more pronounced SiC/6061 Al debonding and reduced SiC particulates cracking. This change in the failure behavior was rationalized in terms of embrittlement of the interfacial region brought about by the aging heat treatment and is correlated, in addition, with the mechanical properties of the composite samples in as-solutionized and peak aged conditions.

Microstructure and properties of squeeze cast Cu-carbon fibre metal matrix composite

There have been reported attempts of producing Cu based MMCs employing solid phase routes. In this work, copper was reinforced with short carbon fibres by pressure infiltration (squeeze casting) of molten metal through dry-separated carbon fibres. The resulting MMC's microstructure revealed uniform distribution of fibres with minimum amount of clustering. Hardness values are considerably higher than that for the unreinforced matrix. Addition of carbon fibres has brought in strain in the crystal lattice of the matrix, resulting in higher microhardness of MMCs and improved wear resistance. Tensile strength values of MMCs at elevated temperatures are considerably higher than that of the unreinforced matrix processed under identical conditions. (C) 1999 Kluwer Academic Publishers.

Optimal control of a stochastic hybrid system with discounted cost

We address the optimal control problem of a very general stochastic hybrid system with both autonomous and impulsive jumps. The planning horizon is infinite and we use the discounted-cost criterion for performance evaluation. Under certain assumptions, we show the existence of an optimal control. We then derive the quasivariational inequalities satisfied by the value function and establish well-posedness. Finally, we prove the usual verification theorem of dynamic programming.

Size effect on the superconducting transition of embedded lead particles in an Al-Cu-V amorphous matrix

We have synthesized specimens of nanometric lead dispersion in a glassy Al-Cu-V matrix by rapid solidification of the corresponding melt. The microstructure has been designed to avoid superconducting percolation due to coupling of the neighboring particles by the proximity effect. Using these specimens, we have determined quantitatively the effect of size of the ultrafine lead particles on the superconducting transition. (C) 1999 American Institute of Physics. [S0003-6951(99)02037-9].

Stochastic dynamics modeling of the protein sequence length distribution in genomes: implications for microbial evolution

In this paper, we report an analysis of the protein sequence length distribution for 13 bacteria, four archaea and one eukaryote whose genomes have been completely sequenced, The frequency distribution of protein sequence length for all the 18 organisms are remarkably similar, independent of genome size and can be described in terms of a lognormal probability distribution function. A simple stochastic model based on multiplicative processes has been proposed to explain the sequence length distribution. The stochastic model supports the random-origin hypothesis of protein sequences in genomes. Distributions of large proteins deviate from the overall lognormal behavior. Their cumulative distribution follows a power-law analogous to Pareto's law used to describe the income distribution of the wealthy. The protein sequence length distribution in genomes of organisms has important implications for microbial evolution and applications. (C) 1999 Elsevier Science B.V. All rights reserved.

The melting and solidification of nanoscale Bi particles embedded in a glassy and crystalline matrix

We report the formation of an amorphous phase in nanosized Pi particles embedded in an Al-based glassy alloy matrix. High-resolution electron microscopy (HREM) has been used to show that the particles contain crystalline and amorphous portions. A depression of the melting point by more than 100 K of the crystalline portion of the Pi particles was found by differential scanning calorimetric studies and by in-situ electron microscopy using a heating stage. The same techniques established the absence of an amorphous phase in the particles when the matrix is crystallized. It is shown that the formation of the amorphous phase and the depression of the melting point cannot be explained by the pressure developed by the volume change during solidification in this constrained system.

The studies on micro-structural defects in deformed composite matrix of Al-Si-Mg and SiC

Detailed Fourier line shape analysis has been performed on three different compositions of the composite matrix of Al-Si-Mg and SiC. The alloy composition in wt% is Al-7%Si, 0.35%Mg, 0.14%Fe and traces of copper and titanium (similar to 0.01%) with SiC varying from 0 to 30wt% in three steps i.e., 0, 10 and 30wt%. The line shift analysis has been performed by considering 111, 200, 220, 311 and 222 reflections after estimating their relative shift. Peak asymmetry analysis has been performed considering neighbouring 111 and 200 reflections and Fourier line shape analysis has been performed after considering the multiple orders 111 and 222, 200 and 400 reflections. Combining all these three analyses it has been found that the deformation stacking faults both intrinsic alpha' and extrinsic alpha " are absent in this alloy system whereas the deformation twin beta has been found to be positive and increases with the increase of SiC concentration. So, like other Al-base alloys this ternary alloy also shows high stacking fault energy, and the addition of SiC introduces deformation twin which increases with its concentration in the deformed lattices.

Providing quality of service via opportunistic splitting using stochastic approximation

We consider the problem of wireless channel allocation to multiple users. A slot is given to a user with a highest metric (e.g., channel gain) in that slot. The scheduler may not know the channel states of all the users at the beginning of each slot. In this scenario opportunistic splitting is an attractive solution. However this algorithm requires that the metrics of different users form independent, identically distributed (iid) sequences with same distribution and that their distribution and number be known to the scheduler. This limits the usefulness of opportunistic splitting. In this paper we develop a parametric version of this algorithm. The optimal parameters of the algorithm are learnt online through a stochastic approximation scheme. Our algorithm does not require the metrics of different users to have the same distribution. The statistics of these metrics and the number of users can be unknown and also vary with time. Each metric sequence can be Markov. We prove the convergence of the algorithm and show its utility by scheduling the channel to maximize its throughput while satisfying some fairness and/or quality of service constraints.

Opportunistic splitting for scheduling via stochastic approximation

We consider the problem of scheduling a wireless channel among multiple users. A slot is given to a user with a highest metric (e.g., channel gain) in that slot. The scheduler may not know the channel states of all the users at the beginning of each slot. In this scenario opportunistic splitting is an attractive solution. However this algorithm requires that the metrics of different users form independent, identically distributed (iid) sequences with same distribution and that their distribution and number be known to the scheduler. This limits the usefulness of opportunistic splitting. In this paper we develop a parametric version of this algorithm. The optimal parameters of the algorithm are learnt online through a stochastic approximation scheme. Our algorithm does not require the metrics of different users to have the same distribution. The statistics of these metrics and the number of users can be unknown and also vary with time. We prove the convergence of the algorithm and show its utility by scheduling the channel to maximize its throughput while satisfying some fairness and/or quality of service constraints.

Density-matrix renormalization-group study of low-lying excitations of polyacene within a Pariser-Parr-Pople model

We have carried out symmetrized density-matrix renormalization-group calculations to study the nature of excited states of long polyacene oligomers within a Pariser-Parr-Pople Hamiltonian. We have used the C-2 symmetry, the electron-hole symmetry, and the spin parity of the system in our calculations. We find that there is a crossover in the lowest dipole forbidden two-photon state and the lowest dipole allowed excited state with size of the oligomer. In the long system limit, the two-photon state lies below the lowest dipole allowed excited state. The triplet state lies well below the two-photon state and energetically does not correspond to its description as being made up of two triplets. These results are in agreement with the general trends in linear conjugated polymers. However, unlike in linear polyenes wherein the two-photon state is a localized excitation, we find that in polyacenes, the two-photon excitation is spread out over the system. We have doped the systems with a hole and an electron and have calculated the charge excitation gap. Using the charge gap and the optical gap, we estimate the binding energy of the 1(1)B(-) exciton to be 2.09 eV. We have also studied doubly doped polyacenes and find that the bipolaron in these systems, to be composed of two separated polarons, as indicated by the calculated charge-density profile and charge-charge correlation function. We have studied bond orders in various states in order to get an idea of the excited state geometry of the system. We find that the ground state, the triplet state, the dipole allowed state, and the polaron excitations correspond to lengthening of the rung bonds in the interior of the oligomer while the two-photon excitation corresponds to the rung bond lengths having two maxima in the system.

In situ synthesis of Al2O3-ZrO2-SiCW ceramic matrix composites by carbothermal reduction of natural silicates

In situ formations of Al2O3 + ZrO2 + SiCW ternary composite powders have been obtained by carbothermal reduction of a mixture of Sillimanite. Kaolin and Zircon using two different carbon sources. Products formed were mixtures of alumina and zirconia along with silicon carbide in the form of whiskers. The effects of composition of the reactants, the role of fineness of the starting precursors and the nature of the carbon Source on the final product powder obtained are presented. XRD and SEM analyses indicate complete reaction of the precursors to yield Al2O3 + ZrO2 + SiCW as product powders, with the SiC having whisker morphology. It is also seen that zirconia could be stabilised to some extent in the tetragonal form without any stabilising agent by tailoring the starting materials and their composition. (C) 2002 Published by Elsevier Science B.V.

Study of embedded lead-tin nanoparticles in aluminum matrix

Nanoembedded lead-tin alloys in aluminum matrix were synthesized by rapid solidification processing. These melt-spun aluminum alloys were then investigated using XRD, EDX and TEM. The XRD study reveals that the melt-spun samples contain elemental aluminum, lead and tin. The TEM analysis shows that embedded particles in aluminium matrix have a distinct two-phase contrast of lead and tin. The lead and tin in these nanoalloys exhibit an orientation relationship with the matrix aluminum and with each other. DSC studies were conducted to reveal the melting and solidification characteristics of these embedded nanoalloys. DSC thermograms exhibit features of multiple solidification exotherms on thermal cycling, which can be attributed to sequential melting and solidification of lead and tin in the respective alloys.