A novel Q-learning algorithm with function approximation for constrained Markov decision processes

We present a novel multi-timescale Q-learning algorithm for average cost control in a Markov decision process subject to multiple inequality constraints. We formulate a relaxed version of this problem through the Lagrange multiplier method. Our algorithm is different from Q-learning in that it updates two parameters - a Q-value parameter and a policy parameter. The Q-value parameter is updated on a slower time scale as compared to the policy parameter. Whereas Q-learning with function approximation can diverge in some cases, our algorithm is seen to be convergent as a result of the aforementioned timescale separation. We show the results of experiments on a problem of constrained routing in a multistage queueing network. Our algorithm is seen to exhibit good performance and the various inequality constraints are seen to be satisfied upon convergence of the algorithm.

Analytical Expression for RMS DC Link Capacitor Current in a Three-Level Inverter

An analytical expression is derived for calculating the rms current through the DC link capacitor in a three level inverter. The output current of the inverter is assumed to sinusoidal. Variations in the capacitor rms current with modulation index as well as line side power factor are studied. The worst case current stress on the capacitor is determined. This is required for sizing the capacitor and is useful for predicting the capacitor losses and life. The analytical expression derived is validated through simulations and experimental results at a number of operating points.

SEP-optimal adaptive gain and transmit power amplify-and-forward relaying

Amplify-and-forward (AF) relay based cooperation has been investigated in the literature given its simplicity and practicality. Two models for AF, namely, fixed gain and fixed power relaying, have been extensively studied. In fixed gain relaying, the relay gain is fixed but its transmit power varies as a function of the source-relay (SR) channel gain. In fixed power relaying, the relay's instantaneous transmit power is fixed, but its gain varies. We propose a general AF cooperation model in which an average transmit power constrained relay jointly adapts its gain and transmit power as a function of the channel gains. We derive the optimal AF gain policy that minimizes the fading- averaged symbol error probability (SEP) of MPSK and present insightful and tractable lower and upper bounds for it. We then analyze the SEP of the optimal policy. Our results show that the optimal scheme is up to 39.7% and 47.5% more energy-efficient than fixed power relaying and fixed gain relaying, respectively. Further, the weaker the direct source-destination link, the greater are the energy-efficiency gains.

On the zeta function of a periodic-finite-type shift

Periodic-finite-type shifts (PFT's) are sofic shifts which forbid the appearance of finitely many pre-specified words in a periodic manner. The class of PFT's strictly includes the class of shifts of finite type (SFT's). The zeta function of a PET is a generating function for the number of periodic sequences in the shift. For a general sofic shift, there exists a formula, attributed to Manning and Bowen, which computes the zeta function of the shift from certain auxiliary graphs constructed from a presentation of the shift. In this paper, we derive an interesting alternative formula computable from certain ``word-based graphs'' constructed from the periodically-forbidden word description of the PET. The advantages of our formula over the Manning-Bowen formula are discussed.

Cl-35 NQR frequency and spin lattice relaxation time in 3,4-dichlorophenol as a function of pressure and temperature

The pressure dependences of Cl-35 nuclear quadrupole resonance (NQR) frequency, temperature and pressure variation of spin lattice relaxation time (T-1) were investigated in 3,4-dichlorophenol. T-1 was measured in the temperature range 77-300 K. Furthermore, the NQR frequency and T-1 for these compounds were measured as a function of pressure up to 5 kbar at 300 K. The temperature dependence of the average torsional lifetimes of the molecules and the transition probabilities W-1 and W-2 for the Delta m = +/- 1 and Delta m = +/- 2 transitions were also obtained. A nonlinear variation of NQR frequency with pressure has been observed and the pressure coefficients were observed to be positive. A thermodynamic analysis of the data was carried out to determine the constant volume temperature coefficients of the NQR frequency. An attempt is made to compare the torsional frequencies evaluated from NQR data with those obtained by IR spectra. On selecting the appropriate mode from IR spectra, a good agreement with torsional frequency obtained from NQR data is observed. The previously mentioned approach is a good illustration of the supplementary nature of the data from IR studies, in relation to NQR studies of compounds in solid state.

Synthesis and application of weight function for edge cracked semicircular disk (ECSD)

We analyze the utility of edge cracked semicircular disk (ECSD) for rapid assessment of fracture toughness using compressive loading. Continuing our earlier work on ECSD, a theoretical examination here leads to a novel way for synthesizing weight functions using two distinct form factors. The efficacy of ECSD mode-I weight function synthesized using displacement and form factor methods is demonstrated by comparing with finite element results. Theory of elasticity in conjunction with finite element method is utilized to analyze crack opening potency of ECSD under eccentric compression to explore newer configurations of ECSD for fracture testing.

Case for multiple views of function in design based on a common definition

Functions are important in designing. However, several issues hinder progress with the understanding and usage of functions: lack of a clear and overarching definition of function, lack of overall justifications for the inevitability of the multiple views of function, and scarcity of systematic attempts to relate these views with one another. To help resolve these, the objectives of this research are to propose a common definition of function that underlies the multiple views in literature and to identify and validate the views of function that are logically justified to be present in designing. Function is defined as a change intended by designers between two scenarios: before and after the introduction of the design. A framework is proposed that comprises the above definition of function and an empirically validated model of designing, extended generate, evaluate, modify, and select of state-change, and an action, part, phenomenon, input, organ, and effect model of causality (Known as GEMS of SAPPhIRE), comprising the views of activity, outcome, requirement-solution-information, and system-environment. The framework is used to identify the logically possible views of function in the context of designing and is validated by comparing these with the views of function in the literature. Describing the different views of function using the proposed framework should enable comparisons and determine relationships among the various views, leading to better understanding and usage of functions in designing.

Transverse single spin asymmetry in e plus p(up arrow) -> e plus J/psi plus X and transverse momentum dependent evolution of the Sivers function

We extend our analysis of transverse single spin asymmetry in electroproduction of J/psi to include the effect of the scale evolution of the transverse momentum dependent (TMD) parton distribution functions and gluon Sivers function. We estimate single spin asymmetry for JLab, HERMES, COMPASS, and eRHIC energies using the color evaporation model of charmonium production, using an analytically obtained approximate solution of TMD evolution equations discussed in the literature. We find that there is a reduction in the asymmetry compared with our predictions for the earlier case considered by us, wherein the Q(2) dependence came only from DGLAP evolution of the unpolarized gluon densities and a different parametrization of the TMD Sivers function was used.

Expansions of tau hadronic spectral function moments in a nonpower QCD perturbation theory with tamed large order behavior

The moments of the hadronic spectral functions are of interest for the extraction of the strong coupling alpha(s) and other QCD parameters from the hadronic decays of the tau lepton. Motivated by the recent analyses of a large class of moments in the standard fixed-order and contour-improved perturbation theories, we consider the perturbative behavior of these moments in the framework of a QCD nonpower perturbation theory, defined by the technique of series acceleration by conformal mappings, which simultaneously implements renormalization-group summation and has a tame large-order behavior. Two recently proposed models of the Adler function are employed to generate the higher-order coefficients of the perturbation series and to predict the exact values of the moments, required for testing the properties of the perturbative expansions. We show that the contour-improved nonpower perturbation theories and the renormalization-group-summed nonpower perturbation theories have very good convergence properties for a large class of moments of the so-called ``reference model,'' including moments that are poorly described by the standard expansions. The results provide additional support for the plausibility of the description of the Adler function in terms of a small number of dominant renormalons.

Single electron transfer-driven multi-dimensional signal read-out function of TCNQ as an ``off-the-shelf'' detector for cyanide

Herein we report the first applications of TCNQ as a rapid and highly sensitive off-the-shelf cyanide detector. As a proof-of-concept, we have applied a kinetically selective single-electron transfer (SET) from cyanide to deep-lying LUMO orbitals of TCNQ to generate a persistently stable radical anion (TCNQ(center dot-)), under ambient condition. In contrast to the known cyanide sensors that operate with limited signal outputs, TCNQ(center dot-) offers a unique multiple signaling platform. The signal readability is facilitated through multichannel absorption in the UV-vis-NIR region and scattering-based spectroscopic methods like Raman spectroscopy and hyper Rayleigh scattering techniques. Particularly notable is the application of the intense 840 nm NIR absorption band to detect cyanide. This can be useful for avoiding background interference in the UV-vis region predominant in biological samples. We also demonstrate the fabrication of a practical electronic device with TCNQ as a detector. The device generates multiorder enhancement in current with cyanide because of the formation of the conductive TCNQ(center dot-).

Scale-dependent relationships between tree species richness and ecosystem function in forests

1. The relationship between species richness and ecosystem function, as measured by productivity or biomass, is of long-standing theoretical and practical interest in ecology. This is especially true for forests, which represent a majority of global biomass, productivity and biodiversity. 2. Here, we conduct an analysis of relationships between tree species richness, biomass and productivity in 25 forest plots of area 8-50ha from across the world. The data were collected using standardized protocols, obviating the need to correct for methodological differences that plague many studies on this topic. 3. We found that at very small spatial grains (0.04ha) species richness was generally positively related to productivity and biomass within plots, with a doubling of species richness corresponding to an average 48% increase in productivity and 53% increase in biomass. At larger spatial grains (0.25ha, 1ha), results were mixed, with negative relationships becoming more common. The results were qualitatively similar but much weaker when we controlled for stem density: at the 0.04ha spatial grain, a doubling of species richness corresponded to a 5% increase in productivity and 7% increase in biomass. Productivity and biomass were themselves almost always positively related at all spatial grains. 4. Synthesis. This is the first cross-site study of the effect of tree species richness on forest biomass and productivity that systematically varies spatial grain within a controlled methodology. The scale-dependent results are consistent with theoretical models in which sampling effects and niche complementarity dominate at small scales, while environmental gradients drive patterns at large scales. Our study shows that the relationship of tree species richness with biomass and productivity changes qualitatively when moving from scales typical of forest surveys (0.04ha) to slightly larger scales (0.25 and 1ha). This needs to be recognized in forest conservation policy and management.

MPK-09, a small molecule inspired from bioactive styryllactone restores the wild-type function of mutant p53

In the search for more efficacious and less toxic cancer drugs, the tumor suppressor p53 protein has long been a desirable therapeutic target. In the recent past, few independent studies have demonstrated that the antitumor activity of wild-type p53 can be restored in cancer cells harboring mutant form of p53 using small molecule activators. In this study, we describe a novel small molecule MPK-09, which is selective and highly potent against allele specific p53 mutations mainly, R175H, R249S, R273H, R273C, and E285K. Except E285K, all other mutations tested are among the six ``hot spot'' p53 mutations reported in majority of human cancer. Furthermore, our study conclusively demonstrates that the apoptotic activity of the small molecule MPK-09 against cancer cells harboring R273C and E285K mutations is due to restoration of the wild-type conformation to the corresponding mutant form of p53.

Shape-function-relationship (SFR) framework for semantic interoperability of product model

The problem of semantic interoperability arises while integrating applications in different task domains across the product life cycle. A new shape-function-relationship (SFR) framework is proposed as a taxonomy based on which an ontology is developed. Ontology based on the SFR framework, that captures explicit definition of terminology and knowledge relationships in terms of shape, function and relationship descriptors, offers an attractive approach for solving semantic interoperability issue. Since all instances of terms are based on single taxonomy with a formal classification, mapping of terms requires a simple check on the attributes used in the classification. As a preliminary study, the framework is used to develop ontology of terms used in the aero-engine domain and the ontology is used to resolve the semantic interoperability problem in the integration of design and maintenance. Since the framework allows a single term to have multiple classifications, handling context dependent usage of terms becomes possible. Automating the classification of terms and establishing the completeness of the classification scheme are being addressed presently.

The Lovasz θ function, SVMs and finding large dense subgraphs

The Lovasz θ function of a graph, is a fundamental tool in combinatorial optimization and approximation algorithms. Computing θ involves solving a SDP and is extremely expensive even for moderately sized graphs. In this paper we establish that the Lovasz θ function is equivalent to a kernel learning problem related to one class SVM. This interesting connection opens up many opportunities bridging graph theoretic algorithms and machine learning. We show that there exist graphs, which we call SVM−θ graphs, on which the Lovasz θ function can be approximated well by a one-class SVM. This leads to a novel use of SVM techniques to solve algorithmic problems in large graphs e.g. identifying a planted clique of size Θ(n√) in a random graph G(n,12). A classic approach for this problem involves computing the θ function, however it is not scalable due to SDP computation. We show that the random graph with a planted clique is an example of SVM−θ graph, and as a consequence a SVM based approach easily identifies the clique in large graphs and is competitive with the state-of-the-art. Further, we introduce the notion of a ''common orthogonal labeling'' which extends the notion of a ''orthogonal labelling of a single graph (used in defining the θ function) to multiple graphs. The problem of finding the optimal common orthogonal labelling is cast as a Multiple Kernel Learning problem and is used to identify a large common dense region in multiple graphs. The proposed algorithm achieves an order of magnitude scalability compared to the state of the art.

Joint Evaluation of Reduced Feedback Scheme, Scheduling, and Rate Adaptation in OFDMA Systems with Feedback Delays

Orthogonal frequency division multiple access (OFDMA) systems exploit multiuser diversity and frequency-selectivity to achieve high spectral efficiencies. However, they require considerable feedback for scheduling and rate adaptation, and are sensitive to feedback delays. We develop a comprehensive analysis of the OFDMA system throughput as a function of the feedback scheme, frequency-domain scheduler, and discrete rate adaptation rule in the presence of feedback delays. We analyze the popular best-n and threshold-based feedback schemes. We show that for both the greedy and round-robin schedulers, the throughput degradation, given a feedback delay, depends primarily on the fraction of feedback reduced by the feedback scheme and not the feedback scheme itself. Even small feedback delays at low vehicular speeds are shown to significantly degrade the throughput. We also show that optimizing the link adaptation thresholds as a function of the feedback delay can effectively counteract the detrimental effect of delays.