A tutorial survey of reinforcement learning

This paper gives a compact, self-contained tutorial survey of reinforcement learning, a tool that is increasingly finding application in the development of intelligent dynamic systems. Research on reinforcement learning during the past decade has led to the development of a variety of useful algorithms. This paper surveys the literature and presents the algorithms in a cohesive framework.

A Tutorial on Integrating CDS/ISIS Databases with the World Wide Web

With the emergence of Internet, the global connectivity of computers has become a reality. Internet has progressed to provide many user-friendly tools like Gopher, WAIS, WWW etc. for information publishing and access. The WWW, which integrates all other access tools, also provides a very convenient means for publishing and accessing multimedia and hypertext linked documents stored in computers spread across the world. With the emergence of WWW technology, most of the information activities are becoming Web-centric. Once the information is published on the Web, a user can access this information from any part of the world. A Web browser like Netscape or Internet Explorer is used as a common user interface for accessing information/databases. This will greatly relieve a user from learning the search syntax of individual information systems. Libraries are taking advantage of these developments to provide access to their resources on the Web. CDS/ISIS is a very popular bibliographic information management software used in India. In this tutorial we present details of integrating CDS/ISIS with the WWW. A number of tools are now available for making CDS/ISIS database accessible on the Internet/Web. Some of these are 1) the WAIS_ISIS Server. 2) the WWWISIS Server 3) the IQUERY Server. In this tutorial, we have explained in detail the steps involved in providing Web access to an existing CDS/ISIS database using the freely available software, WWWISIS. This software is developed, maintained and distributed by BIREME, the Latin American & Caribbean Centre on Health Sciences Information. WWWISIS acts as a server for CDS/ISIS databases in a WWW client/server environment. It supports functions for searching, formatting and data entry operations over CDS/ISIS databases. WWWISIS is available for various operating systems. We have tested this software on Windows '95, Windows NT and Red Hat Linux release 5.2 (Appolo) Kernel 2. 0. 36 on an i686. The testing was carried out using IISc's main library's OPAC containing more than 80,000 records and Current Contents issues (bibliographic data) containing more than 25,000 records. WWWISIS is fully compatible with CDS/ISIS 3.07 file structure. However, on a system running Unix or its variant, there is no guarantee of this compatibility. It is therefore safe to recreate the master and the inverted files, using utilities provided by BIREME, under Unix environment.

Consistency tests of Ampcalculator and chiral amplitudes in SU(3) Chiral Perturbation Theory: A tutorial-based approach

Ampcalculator (AMPC) is a Mathematica (c) based program that was made publicly available some time ago by Unterdorfer and Ecker. It enables the user to compute several processes at one loop (upto O(p(4))) in SU(3) chiral perturbation theory. They include computing matrix elements and form factors for strong and non-leptonic weak processes with at most six external states. It was used to compute some novel processes and was tested against well-known results by the original authors. Here we present the results of several thorough checks of the package. Exhaustive checks performed by the original authors are not publicly available, and hence the present effort. Some new results are obtained from the software especially in the kaon odd-intrinsic parity non-leptonic decay sector involving the coupling G(27). Another illustrative set of amplitudes at tree level we provide is in the context of tau-decays with several mesons including quark mass effects, of use to the BELLE experiment. All eight meson-meson scattering amplitudes have been checked. The Kaon-Compton amplitude has been checked and a minor error in the published results has been pointed out. This exercise is a tutorial-based one, wherein several input and output notebooks are also being made available as ancillary files on the arXiv. Some of the additional notebooks we provide contain explicit expressions that we have used for comparison with established results. The purpose is to encourage users to apply the software to suit their specific needs. An automatic amplitude generator of this type can provide error-free outputs that could be used as inputs for further simplification, and in varied scenarios such as applications of chiral perturbation theory at finite temperature, density and volume. This can also be used by students as a learning aid in low-energy hadron dynamics.

Tutorial: implementing a pedestrian tracker using inertial sensors

Shoe-mounted inertial sensors offer a convenient way to track pedestrians in situations where other localization systems fail. This tutorial outlines a simple yet effective approach for implementing a reasonably accurate tracker. This Web extra presents the Matlab implementation and a few sample recordings for implementing the pedestrian inertial tracking system using an error-state Kalman filter for zero-velocity updates (ZUPTs) and orientation estimation.

Metal-organic framework structures - how closely are they related to classical inorganic structures?

Metal-organic frameworks (MOFs) have emerged as an important family of compounds for which new properties are increasingly being found. The potential for such compounds appears to be immense, especially in catalysis, sorption and separation processes. In order to appreciate the properties and to design newer frameworks it is necessary to understand the structures from a fundamental perspective. The use of node, net and vertex symbols has helped in simplifying some of the complex MOF structures. Many MOF structures are beginning to be described as derived from inorganic structures. In this tutorial review, we have provided the basics of the node, the net and the vertex symbols and have explained some of the MOF structures. In addition, we have also attempted to provide some leads towards designing newer structures/topologies.

Solvation dynamics in dipolar liquids

The time dependent response of a polar solvent to a changing charge distribution is studied in solvation dynamics. The change in the energy of the solute is measured by a time domain Stokes shift in the fluorescence spectrum of the solute. Alternatively, one can use sophisticated non-linear optical spectroscopic techniques to measure the energy fluctuation of the solute at equilibrium. In both methods, the measured dynamic response is expressed by the normalized solvation time correlation function, S(t). The latter is found to exhibit uniquefeatures reflecting both the static and dynamic characteristics of each solvent. For water, S(t) consists of a dominant sub-50 fs ultrafast component, followed by a multi-exponential decay. Acetonitrile exhibitsa sub-100 fs ultrafast component, followed by an exponential decay. Alcohols and amides show features unique to each solvent and solvent series. However, understanding and interpretation of these results have proven to be difficult, and often controversial. Theoretical studiesand computer simulations have greatly facilitated the understanding ofS(t) in simple systems. Recently solvation dynamics has been used extensively to explore dynamics of complex systems, like micelles and reverse micelles, protein and DNA hydration layers, sol-gel mixtures and polymers. In each case one observes rich dynamical features, characterized again by multi-exponential decays but the initial and final time constants are now widely separated. In this tutorial review, we discuss the difficulties in interpreting the origin of the observed behaviour in complex systems.

Dynamic scheduling in manufacturing systems using brownian approximations

Recently, Brownian networks have emerged as an effective stochastic model to approximate multiclass queueing networks with dynamic scheduling capability, under conditions of balanced heavy loading. This paper is a tutorial introduction to dynamic scheduling in manufacturing systems using Brownian networks. The article starts with motivational examples. It then provides a review of relevant weak convergence concepts, followed by a description of the limiting behaviour of queueing systems under heavy traffic. The Brownian approximation procedure is discussed in detail and generic case studies are provided to illustrate the procedure and demonstrate its effectiveness. This paper places emphasis only on the results and aspires to provide the reader with an up-to-date understanding of dynamic scheduling based on Brownian approximations.

Integrated optics � technology and applications

The emergence of optoelectronics and photonics as viable alternatives to electronics in many key areas of engineering relevance is indeed significant. This paper presents a tutorial review of integrated optics � a technologically important development in photonics. Materials, processes, device technology and applications are highlighted.

DMT of Wireless Networks: An overview

The e�cient operation of single-source, single-sink wireless network is considered with the diversity-multiplexing gain tradeo� (DMT) as the measure of performance. Whereas in the case of a point-to-point MIMO channel the DMT is determined by the fading statistics, in the case of a network, the DMT is additionally, a function of the time schedule according to which the network is operated, as well as the protocol that dictates the mode of operation of the intermediate relays.In general, it is only possible at present, to provide upper bounds on the DMT of the network in terms of the DMT of the MIMO channel appearing across cuts in the network. This paper presents a tutorial overview on the DMT of half-duplex multi-hop wireless networks that also attempts to identify where possible, codes that achieve the DMT.For example, it is shown how one can construct codes that achieve the DMT of a network under a given schedule and either an amplify-and-forward or decode-and-forward protocol. Also contained in the paper,are discussions on the DMT of the multiple-access channel as well as the impact of feedback on the DMT of a MIMO channel.

“Post silicon debug of SOC designs”

Continuous advances in VLSI technology have made implementation of very complicated systems possible. Modern System-on -Chips (SoCs) have many processors, IP cores and other functional units. As a result, complete verification of whole systems before implementation is becoming infeasible; hence it is likely that these systems may have some errors after manufacturing. This increases the need to find design errors in chips after fabrication. The main challenge for post-silicon debug is the observability of the internal signals. Post-silicon debug is the problem of determining what's wrong when the fabricated chip of a new design behaves incorrectly. This problem now consumes over half of the overall verification effort on large designs, and the problem is growing worse.Traditional post-silicon debug methods concentrate on functional parts of systems and provide mechanisms to increase the observability of internal state of systems. Those methods may not be sufficient as modern SoCs have lots of blocks (processors, IP cores, etc.) which are communicating with one another and communication is another source of design errors. This tutorial will be provide an insight into various observability enhancement techniques, on chip instrumentation techniques and use of high level models to support the debug process targeting both inside blocks and communication among them. It will also cover the use of formal methods to help debug process.

Algebraic Approaches to Space-Time Code Construction for Multiple-Antenna Communication

A major challenge in wireless communications is overcoming the deleterious effects of fading, a phenomenon largely responsible for the seemingly inevitable dropped call. Multiple-antennas communication systems, commonly referred to as MIMO systems, employ multiple antennas at both transmitter and receiver, thereby creating a multitude of signalling pathways between transmitter and receiver. These multiple pathways give the signal a diversity advantage with which to combat fading. Apart from helping overcome the effects of fading, MIMO systems can also be shown to provide a manyfold increase in the amount of information that can be transmitted from transmitter to receiver. Not surprisingly,MIMO has played, and continues to play, a key role in the advancement of wireless communication.Space-time codes are a reference to a signalling format in which information about the message is dispersed across both the spatial (or antenna) and time dimension. Algebraic techniques drawing from algebraic structures such as rings, fields and algebras, have been extensively employed in the construction of optimal space-time codes that enable the potential of MIMO communication to be realized, some of which have found their way into the IEEE wireless communication standards. In this tutorial article, reflecting the authors’interests in this area, we survey some of these techniques.

Multivalent glycoliposomes and micelles to study carbohydrate-protein and carbohydrate-carbohydrate interactions

This tutorial review describes multivalent carbohydrate-protein and carbohydrate-carbohydrate interaction studies that utilize self-assembled aggregates of thermodynamically stable liposomes and micelles. Strategies to prepare multivalent glycoliposomes and micelles include: (i) insertion of synthetic glycolipids into matrix lipids; (ii) preparation of glycolipids that aggregate to liposomes and micelles and (iii) modification of the hydrophilic surfaces with desired sugars. Several design strategies have been developed in order to obtain constituent glycolipids, having multivalent sugar moieties and their subsequent interactions with proteins were assessed in relation to the type of linkers that connect the hydrophilic and lipophilic segments. Lipophilic segments other than alkyl chains have also been developed. Polymer based glycoliposomes and micelles form an emphasis. Further, glycoliposomes facilitate studies of carbohydrate-carbohydrate interactions. An overview of the various types of glycoliposomes and micelles used to study carbohydrate-protein and carbohydrate-carbohydrate recognition phenomena is presented.

Moore meets maxwell

Moore's Law has driven the semiconductor revolution enabling over four decades of scaling in frequency, size, complexity, and power. However, the limits of physics are preventing further scaling of speed, forcing a paradigm shift towards multicore computing and parallelization. In effect, the system is taking over the role that the single CPU was playing: high-speed signals running through chips but also packages and boards connect ever more complex systems. High-speed signals making their way through the entire system cause new challenges in the design of computing hardware. Inductance, phase shifts and velocity of light effects, material resonances, and wave behavior become not only prevalent but need to be calculated accurately and rapidly to enable short design cycle times. In essence, to continue scaling with Moore's Law requires the incorporation of Maxwell's equations in the design process. Incorporating Maxwell's equations into the design flow is only possible through the combined power that new algorithms, parallelization and high-speed computing provide. At the same time, incorporation of Maxwell-based models into circuit and system-level simulation presents a massive accuracy, passivity, and scalability challenge. In this tutorial, we navigate through the often confusing terminology and concepts behind field solvers, show how advances in field solvers enable integration into EDA flows, present novel methods for model generation and passivity assurance in large systems, and demonstrate the power of cloud computing in enabling the next generation of scalable Maxwell solvers and the next generation of Moore's Law scaling of systems. We intend to show the truly symbiotic growing relationship between Maxwell and Moore!