Parametrized actor-critic algorithms for finite-horizon MDPs

Due to their non-stationarity, finite-horizon Markov decision processes (FH-MDPs) have one probability transition matrix per stage. Thus the curse of dimensionality affects FH-MDPs more severely than infinite-horizon MDPs. We propose two parametrized 'actor-critic' algorithms to compute optimal policies for FH-MDPs. Both algorithms use the two-timescale stochastic approximation technique, thus simultaneously performing gradient search in the parametrized policy space (the 'actor') on a slower timescale and learning the policy gradient (the 'critic') via a faster recursion. This is in contrast to methods where critic recursions learn the cost-to-go proper. We show w.p 1 convergence to a set with the necessary condition for constrained optima. The proposed parameterization is for FHMDPs with compact action sets, although certain exceptions can be handled. Further, a third algorithm for stochastic control of stopping time processes is presented. We explain why current policy evaluation methods do not work as critic to the proposed actor recursion. Simulation results from flow-control in communication networks attest to the performance advantages of all three algorithms.

An Actor-Critic Algorithm for Finite Horizon Markov Decision Processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

Leveraging coherent distributed space-time codes for noncoherent communication in relay networks via training

For point to point multiple input multiple output systems, Dayal-Brehler-Varanasi have proved that training codes achieve the same diversity order as that of the underlying coherent space time block code (STBC) if a simple minimum mean squared error estimate of the channel formed using the training part is employed for coherent detection of the underlying STBC. In this letter, a similar strategy involving a combination of training, channel estimation and detection in conjunction with existing coherent distributed STBCs is proposed for noncoherent communication in Amplify-and-Forward (AF) relay networks. Simulation results show that the proposed simple strategy outperforms distributed differential space-time coding for AF relay networks. Finally, the proposed strategy is extended to asynchronous relay networks using orthogonal frequency division multiplexing.

Implementation of Art1 and Art2 Artificial Neural Networks on Ring and Mesh Architectures

The Artificial Neural Networks (ANNs) are being used to solve a variety of problems in pattern recognition, robotic control, VLSI CAD and other areas. In most of these applications, a speedy response from the ANNs is imperative. However, ANNs comprise a large number of artificial neurons, and a massive interconnection network among them. Hence, implementation of these ANNs involves execution of computer-intensive operations. The usage of multiprocessor systems therefore becomes necessary. In this article, we have presented the implementation of ART1 and ART2 ANNs on ring and mesh architectures. The overall system design and implementation aspects are presented. The performance of the algorithm on ring, 2-dimensional mesh and n-dimensional mesh topologies is presented. The parallel algorithm presented for implementation of ART1 is not specific to any particular architecture. The parallel algorithm for ARTE is more suitable for a ring architecture.

Energy-Efficient Decentralized Cooperative Routing in Wireless Networks

Wireless adhoc networks transmit information from a source to a destination via multiple hops in order to save energy and, thus, increase the lifetime of battery-operated nodes. The energy savings can be especially significant in cooperative transmission schemes, where several nodes cooperate during one hop to forward the information to the next node along a route to the destination. Finding the best multi-hop transmission policy in such a network which determines nodes that are involved in each hop, is a very important problem, but also a very difficult one especially when the physical wireless channel behavior is to be accounted for and exploited. We model the above optimization problem for randomly fading channels as a decentralized control problem - the channel observations available at each node define the information structure, while the control policy is defined by the power and phase of the signal transmitted by each node. In particular, we consider the problem of computing an energy-optimal cooperative transmission scheme in a wireless network for two different channel fading models: (i) slow fading channels, where the channel gains of the links remain the same for a large number of transmissions, and (ii) fast fading channels, where the channel gains of the links change quickly from one transmission to another. For slow fading, we consider a factored class of policies (corresponding to local cooperation between nodes), and show that the computation of an optimal policy in this class is equivalent to a shortest path computation on an induced graph, whose edge costs can be computed in a decentralized manner using only locally available channel state information (CSI). For fast fading, both CSI acquisition and data transmission consume energy. Hence, we need to jointly optimize over both these; we cast this optimization problem as a large stochastic optimization problem. We then jointly optimize over a set of CSI functions of the local channel states, and a c- - orresponding factored class of control poli.

Functional correlation of bacterial LuxS with their quaternary associations: interface analysis of the structure networks

Background The genome of a wide variety of prokaryotes contains the luxS gene homologue, which encodes for the protein S-ribosylhomocysteinelyase (LuxS). This protein is responsible for the production of the quorum sensing molecule, AI-2 and has been implicated in a variety of functions such as flagellar motility, metabolic regulation, toxin production and even in pathogenicity. A high structural similarity is present in the LuxS structures determined from a few species. In this study, we have modelled the structures from several other species and have investigated their dimer interfaces. We have attempted to correlate the interface features of LuxS with the phenotypic nature of the organisms. Results The protein structure networks (PSN) are constructed and graph theoretical analysis is performed on the structures obtained from X-ray crystallography and on the modelled ones. The interfaces, which are known to contain the active site, are characterized from the PSNs of these homodimeric proteins. The key features presented by the protein interfaces are investigated for the classification of the proteins in relation to their function. From our analysis, structural interface motifs are identified for each class in our dataset, which showed distinctly different pattern at the interface of LuxS for the probiotics and some extremophiles. Our analysis also reveals potential sites of mutation and geometric patterns at the interface that was not evident from conventional sequence alignment studies. Conclusion The structure network approach employed in this study for the analysis of dimeric interfaces in LuxS has brought out certain structural details at the side-chain interaction level, which were elusive from the conventional structure comparison methods. The results from this study provide a better understanding of the relation between the luxS gene and its functional role in the prokaryotes. This study also makes it possible to explore the potential direction towards the design of inhibitors of LuxS and thus towards a wide range of antimicrobials.

Co-ordinate Interleaved Distributed Space-Time Coding for Two-Antenna-Relays Networks

Distributed space time coding for wireless relay networks when the source, the destination and the relays have multiple antennas have been studied by Jing and Hassibi. In this set-up, the transmit and the receive signals at different antennas of the same relay are processed and designed independently, even though the antennas are colocated. In this paper, a wireless relay network with single antenna at the source and the destination and two antennas at each of the R relays is considered. A new class of distributed space time block codes called Co-ordinate Interleaved Distributed Space-Time Codes (CIDSTC) are introduced where, in the first phase, the source transmits a T-length complex vector to all the relays;and in the second phase, at each relay, the in-phase and quadrature component vectors of the received complex vectors at the two antennas are interleaved and processed before forwarding them to the destination. Compared to the scheme proposed by Jing-Hassibi, for T >= 4R, while providing the same asymptotic diversity order of 2R, CIDSTC scheme is shown to provide asymptotic coding gain with the cost of negligible increase in the processing complexity at the relays. However, for moderate and large values of P, CIDSTC scheme is shown to provide more diversity than that of the scheme proposed by Jing-Hassibi. CIDSTCs are shown to be fully diverse provided the information symbols take value from an appropriate multidimensional signal set.

Single-Symbol ML Decodable Distributed STBCs for Partially-Coherent Cooperative Networks

A relay network with N relays and a single source-destination pair is called a partially-coherent relay channel (PCRC) if the destination has perfect channel state information (CSI) of all the channels and the relays have only the phase information of the source-to-relay channels. In this paper, first, a new set of necessary and sufficient conditions for a space-time block code (STBC) to be single-symbol decodable (SSD) for colocated multiple antenna communication is obtained. Then, this is extended to a set of necessary and sufficient conditions for a distributed STBC (DSTBC) to be SSD for. a PCRC. Using this, several SSD DSTBCs for PCRC are identified. It is proved that even if a SSD STBC for a co-located MIMO channel does not satisfy the additional conditions for the code to be SSD for a PCRC, single-symbol decoding of it in a PCRC gives full-diversity and only coding gain is lost. It is shown that when a DSTBC is SSD for a PCRC, then arbitrary coordinate interleaving of the in-phase and quadrature-phase components of the variables does not disturb its SSD property for PCRC. Finally, it is shown that the possibility of channel phase compensation operation at the relay nodes using partial CSI at the relays increases the possible rate of SSD DSTBCs from (2)/(N) when the relays do not have CSI to(1)/(2), which is independent of N.

Allostery in tRNA Synthetases Elucidated from MD Simulations and Protein Structure Networks

tRNA synthetases (aaRS) are enzymes crucial in the translation of genetic code. The enzyme accylates the acceptor stem of tRNA by the congnate amino acid bound at the active site, when the anti-codon is recognized by the anti-codon site of aaRS. In a typical aaRS, the distance between the anti-codon region and the amino accylation site is approximately 70 Å. We have investigated this allosteric phenomenon at molecular level by MD simulations followed by the analysis of protein structure networks (PSN) of non-covalent interactions. Specifically, we have generated conformational ensembles by performing MD simulations on different liganded states of methionyl tRNA synthetase (MetRS) from Escherichia coli and tryptophenyl tRNA synthetase (TrpRS) from Human. The correlated residues during the MD simulations are identified by cross correlation maps. We have identified the amino acids connecting the correlated residues by the shortest path between the two selected members of the PSN. The frequencies of paths have been evaluated from the MD snapshots[1]. The conformational populations in different liganded states of the protein have been beautifully captured in terms of network parameters such as hubs, cliques and communities[2]. These parameters have been associated with the rigidity and plasticity of the protein conformations and can be associated with free energy landscape. A comparison of allosteric communication in MetRS and TrpRS [3] elucidated in this study highlights diverse means adopted by different enzymes to perform a similar function. The computational method described for these two enzymes can be applied to the investigation of allostery in other systems.

Self-Assembly of Bile Steroid Analogues: Molecules, Fibers, and Networks

Structural and rheological features of a series of molecular hydrogels formed by synthetic bile salt analogues have been scrutinized. Among seven gelators, two are neutral compounds, while the others are cationic systems among which one is a tripodal steroid derivative. Despite the fact that the chemical structures are closely related, the variety of physical characteristics is extremely large in the structures of the connected fibers (either plain cylinders or ribbons), in the dynamical modes for stress relaxation of the associated SAFINs, in the scaling laws of the shear elasticity (typical of either cellular solids or fractal floc-like assemblies), in the micron-scale texture and the distribution of ordered domains (spherulites, crystallites) embedded in a random mesh, in the type of nodal zones (either crystalline-like, fiber entanglements, or bundles), in the evolution of the distribution and morphology of fibers and nodes, and in the sensitivity to added salt. SANS appears to be a suitable technique to infer all geometrical parameters defining the fibers, their interaction modes, and the volume fraction of nodes in a SAFIN. The tripodal system is particularly singular in the series and exhibits viscosity overshoots at the startup of shear flows, an “umbrella-like” molecular packing mode involving three molecules per cross section of fiber, and scattering correlation peaks revealing the ordering and overlap of 1d self-assembled polyelectrolyte species.

Soft Functional Materials Induced by Fibrillar Networks of Small Molecular Photochromic Gelators

Low-molecular-mass organogelators (LMOGs) based on photochromic molecules aggregate in selected solvents to form gels through various spatio-temporal interactions. The factors that control the mode of aggregation of the chromophoric core in the LMOGs during gelation, gelation-induced changes in fluorescence, the formation of stacked superstructures of extended pi-conjugated systems, and so forth are discussed with selected examples. Possible ways of generating various light-harvesting assemblies are proposed, and some unresolved questions, future challenges, and their possible solutions on this topic are presented.

Distributed Scheduling for Multi-Hop Wireless Networks

We study the performance of greedy scheduling in multihop wireless networks where the objective is aggregate utility maximization. Following standard approaches, we consider the dual of the original optimization problem. Optimal scheduling requires selecting independent sets of maximum aggregate price, but this problem is known to be NP-hard. We propose and evaluate a simple greedy heuristic. We suggest how the greedy heuristic can be implemented in a distributed manner. We evaluate an analytical bound in detail, for the special case of a line graph and also provide a loose bound on the greedy heuristic for the case of an arbitrary graph.

Protection and Switching-Aid Networks for Transistor Bridge Inverters

The present trend in the industry is towards the use of power transistors in the development of efficient Pulsewidth Modulated (PWM) inverters, because of their operation at high frequency, simplicity of turn-off, and low commutation losses compared to the technology using thyristors. But the protection of power transistors, minimization of switching power loss, and design of base drive circuit are very important for a reliable operation of the system. The requirements, analysis, and a simplified procedure for calculation of the switching-aid network components are presented. The transistor is protected against short circuit using a modified autoregulated and autoprotection drive circuit. The experimental results show that the switching power loss and voltage stress in the device can be reduced by suitable choice of the switching-aid network component values.

Diversity Multiplexing Tradeoff of Asynchronous Cooperative Relay Networks

The assumption of nodes in a cooperative communication relay network operating in synchronous fashion is often unrealistic. In the present paper we consider two different models of asynchronous operation in cooperative-diversity networks experiencing slow fading and examine the corresponding diversity-multiplexing tradeoffs (DMT). For both models, we propose protocols and distributed space-time codes that asymptotically achieve the transmit diversity bound for all multiplexing gains and for number of relays N >= 2.

Efficient Energy Management Policies for Networks with Energy Harvesting Sensor Nodes

We study sensor networks with energy harvesting nodes. The generated energy at a node can be stored in a buffer. A sensor node periodically senses a random field and generates a packet. These packets are stored in a queue and transmitted using the energy available at that time at the node. For such networks we develop efficient energy management policies. First, for a single node, we obtain policies that are throughput optimal, i.e., the data queue stays stable for the largest possible data rate. Next we obtain energy management policies which minimize the mean delay in the queue. We also compare performance of several easily implementable suboptimal policies. A greedy policy is identified which, in low SNR regime, is throughput optimal and also minimizes mean delay. Next using the results for a single node, we develop efficient MAC policies.