Two-timescale Q-learning Algorithms with an Application to Routing in Networks

We propose two variants of the Q-learning algorithm that (both) use two timescales. One of these updates Q-values of all feasible state-action pairs at each instant while the other updates Q-values of states with actions chosen according to the ‘current ’ randomized policy updates. A sketch of convergence of the algorithms is shown. Finally, numerical experiments using the proposed algorithms for routing on different network topologies are presented and performance comparisons with the regular Q-learning algorithm are shown.

Three-Component Chiral Derivatizing Protocols for NMR Spectroscopic Enantiodiscrimination of Hydroxy Acids and Primary Amines

The novel three-component chiral derivatization protocols have been derived for (1)H and (19)F NMR spectroscopic discrimination of a series of chiral hydroxy acids by their coordination and self-assembly with optically active a-methylbenzylamine and 2-formylphenylboronic acid. In addition, the optically pure (S)-mandelic acid in combination with 2-formylphenylboronic acid permits visualization of enantiomers of primary amines. These protocols have been demonstrated on enantiodiscrimination of chiral amines and hydroxy acids.

An Efficient Distributed Scheme for Source Routing Protocol in Communication Networks

In this paper, we propose an efficient source routing algorithm for unicast flows, which addresses the scalability problem associated with the basic source routing technique. Simulation results indicate that the proposed algorithm indeed helps in reducing the message overhead considerably, and at the same time it gives comparable performance in terms of resource utilization across a wide range of workloads.

Smoothed functional and Quasi-Newton algorithms for routing in multi-stage queueing network with constraints

We consider the problem of optimal routing in a multi-stage network of queues with constraints on queue lengths. We develop three algorithms for probabilistic routing for this problem using only the total end-to-end delays. These algorithms use the smoothed functional (SF) approach to optimize the routing probabilities. In our model all the queues are assumed to have constraints on the average queue length. We also propose a novel quasi-Newton based SF algorithm. Policies like Join Shortest Queue or Least Work Left work only for unconstrained routing. Besides assuming knowledge of the queue length at all the queues. If the only information available is the expected end-to-end delay as with our case such policies cannot be used. We also give simulation results showing the performance of the SF algorithms for this problem.

A trust model for routing in MANETs: a cognitive agents based approach

Mobile ad hoc networks (MANETs) is one of the successful wireless network paradigms which offers unrestricted mobility without depending on any underlying infrastructure. MANETs have become an exciting and im- portant technology in recent years because of the rapid proliferation of variety of wireless devices, and increased use of ad hoc networks in various applications. Like any other networks, MANETs are also prone to variety of attacks majorly in routing side, most of the proposed secured routing solutions based on cryptography and authentication methods have greater overhead, which results in latency problems and resource crunch problems, especially in energy side. The successful working of these mechanisms also depends on secured key management involving a trusted third authority, which is generally difficult to implement in MANET environ-ment due to volatile topology. Designing a secured routing algorithm for MANETs which incorporates the notion of trust without maintaining any trusted third entity is an interesting research problem in recent years. This paper propose a new trust model based on cognitive reasoning,which associates the notion of trust with all the member nodes of MANETs using a novel Behaviors-Observations- Beliefs(BOB) model. These trust values are used for detec- tion and prevention of malicious and dishonest nodes while routing the data. The proposed trust model works with the DTM-DSR protocol, which involves computation of direct trust between any two nodes using cognitive knowledge. We have taken care of trust fading over time, rewards, and penalties while computing the trustworthiness of a node and also route. A simulator is developed for testing the proposed algorithm, the results of experiments shows incorporation of cognitive reasoning for computation of trust in routing effectively detects intrusions in MANET environment, and generates more reliable routes for secured routing of data.

Analysis of MAC protocols with correlation receiver for optical CDMA networks – part I

In this paper optical code-division multiple-access (O-CDMA) packet network is considered. Two types of random access protocols are proposed for packet transmission. In protocol 1, all distinct codes and in protocol 2, distinct codes as well as shifted versions of all these codes are used. O-CDMA network performance using optical orthogonal codes (OOCs) 1-D and twodimensional (2-D) wavelength/time single-pulse-per-row (W/TSPR) codes are analyzed. The main advantage of using 2-D codes instead of one-dimensional (1-D) codes is to reduce the errors due to multiple access interference among different users. In this paper, correlation receiver is considered in the analysis. Using analytical model, we compute and compare packet-success probability for 1-D and 2-D codes in an O-CDMA network and the analysis shows improved performance with 2-D codes as compared to 1-D codes.

Membership models and the design of authentication protocols for MANETs

Authentication protocols are very much essential for secure communication in mobile ad hoc networks (MANETs). A number of authentication protocols for MANETs have been proposed in the literature which provide the basic authentication service while trying to optimize their performance and resource consumption parameters. A problem with most of these protocols is that the underlying networking environment on which they are applicable have been left unspecified. As a result, lack of specifications about the networking environments applicable to an authentication protocol for MANETs can mislead about the performance and the applicability of the protocol. In this paper, we first characterize networking environment for a MANET as its 'Membership Model' which is defined as a set of specifications related to the 'Membership Granting Server' (MGS) and the 'Membership Set Pattern' (MSP) of the MANET. We then identify various types of possible membership models for a MANET. In order to illustrate that while designing an authentication protocol for a MANET, it is very much necessary to consider the underlying membership model of the MANET, we study a set of six representative authentication protocols, and analyze their applicability for the membership models as enumerated in this paper. The analysis shows that the same protocol may not perform equally well in all membership models. In addition, there may be membership models which are important from the point of view of users, but for which no authentication protocol is available.

Joint Routing, Scheduling and Power Control in Multihop MIMO Networks with MAC and Broadcast Links

We consider the problem of joint routing, scheduling and power control in a multihop wireless network when the nodes have multiple antennas. We focus on exploiting the multiple degrees-of-freedom available at each transmitter and receiver due to multiple antennas. Specifically we use multiple antennas at each node to form multiple access and broadcast links in the network rather than just point to point links. We show that such a generic transmission model improves the system performance significantly. Since the complexity of the resulting optimization problem is very high, we also develop efficient suboptimal solutions for joint routing, scheduling and power control in this setup.