A reinforcement learning based algorithm for finite horizon Markov decision processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

A mathematical programming approach to optimal Markovian switching of Poisson arrival streams to queueing systems

Motivated by certain situations in manufacturing systems and communication networks, we look into the problem of maximizing the profit in a queueing system with linear reward and cost structure and having a choice of selecting the streams of Poisson arrivals according to an independent Markov chain. We view the system as a MMPP/GI/1 queue and seek to maximize the profits by optimally choosing the stationary probabilities of the modulating Markov chain. We consider two formulations of the optimization problem. The first one (which we call the PUT problem) seeks to maximize the profit per unit time whereas the second one considers the maximization of the profit per accepted customer (the PAC problem). In each of these formulations, we explore three separate problems. In the first one, the constraints come from bounding the utilization of an infinite capacity server; in the second one the constraints arise from bounding the mean queue length of the same queue; and in the third one the finite capacity of the buffer reflect as a set of constraints. In the problems bounding the utilization factor of the queue, the solutions are given by essentially linear programs, while the problems with mean queue length constraints are linear programs if the service is exponentially distributed. The problems modeling the finite capacity queue are non-convex programs for which global maxima can be found. There is a rich relationship between the solutions of the PUT and PAC problems. In particular, the PUT solutions always make the server work at a utilization factor that is no less than that of the PAC solutions.

An Actor-Critic Algorithm for Finite Horizon Markov Decision Processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

Network pricing for QoS: A 'Regulation' approach

V. S. Borkar’s work was supported in part by grant number III.5(157)/99-ET from the Department of Science and Technology, Government of India. D. Manjunath’s work was supported in part by grant number 1(1)/2004-E-Infra from the Ministry of Information Technology, Government of India.

Performance of telecommunication modules to 0.5µS-100kHz ring wave surges

This paper presents an experimental investigations performed on various electronic components used in telecommunication networks and those used in avionics for the ring wave surge voltages. IEEE Std C 62.41.1-2002 specifies a stringent requirement of waveforms to be applied for the evaluation of telecom components. To meet the necessary requirements in the absence of commercial equipment for generating the required waveforms, special efforts were made to fabricate a ring wave surge generator as per prescribed standards. The developed surge generator is capable of delivering an output of 0.5 mu s-100kHz which meets the requirements of telecom standards prescribed for evaluation of various modules used in low voltage ac power circuits used in communication networks. The results of the experimental investigations obtained on various modules used in communication networks are presented.

Gain characteristics of Er3+-doped phosphate glass fibres

An erbium-doped phosphate glass fibre has been drawn by the rod-in-tube technique in our laboratory. The gain for the Er3+-doped phosphate glass fibre with different pump powers and with different input signal wavelengths is investigated. The 2.2-cm-long fibre, pumped by a single-mode 980-nm fibre-pigtailed laser diode, can provide a net gain per unit length greater than 1.8dB/cm. The pump threshold is about 50 mW at the wavelength of 1534 nm, and below 70 mW at 1550 nm. The gain linewidth of the Er3+-doped phosphate glass fibre is greater than 34 nm and can cover the C band in optical communication networks.

Long Distance Wireless Networking for Site - Office Data Communications

Effective data communications between the project site and decision making office can be critical for the success of a construction project. It allows convenient access to centrally stored information and allows centrally located decision makers to remotely monitor the site and collect data in real-time. However, high bandwidth, flexible data communication networks, such as wired local area networks, can often be time-consuming and costly to deploy for such purposes especially when project sites (dams, highways, etc.) are located in rural, undeveloped areas where networking infrastructure is not available. In such construction sites, wireless networking could reliably link the construction site and the decision-making office. This paper presents a case study on long-distance, site – office wireless data communications. The purpose was to investigate the capability of wireless technology in exchanging construction data in a fast and efficient manner and in allowing site personnel to interact and share knowledge and data with the office staff. This study took place at the University of Michigan’s campus where performance, reliability, and cost/benefit tests were performed. The indoor and outdoor tests performed demonstrated the suitability of this technology for office-site data communications and exposed the need for more research to further improve the reliability and data handling of this technology.

The effect of human rhythm on packet delivery

In communication networks such as the Internet, the relationship between packet generation rate and time is similar to a rectangle wavefunction due to the rhythm of humans. Thus, we investigate the traffic dynamics on a network with a rectangle wavepacket generation rate. It is found that the critical delivering capacity parameter beta(c) (which separates the congested phase and the free phase) decreases significantly with the duty cycle r of the rectangle wave for package generation. And, in the congested phase, more collective generation of packets (smaller r) is helpful for decreasing the packet aggregation rate. Moreover, it is found that the congested phase can be divided into two regions, i.e., region1 and region2, where the distributions of queue lengths are nonlinear and linear, respectively. Also, the linear expression for the distribution of queue lengths in region2 is obtained analytically. Our work reveals an obvious effect of the rectangle wave on the traffic dynamics and the queue length distribution in the system, which is of essential interest and may provide insights into the designing of work-rest schedules and routing strategies.

基于CAN总线的分布式数据采集与控制系统

随着电子技术和计算机技术的不断发展，工业生产过程的控制系统正在向着智能化、数字化和网络化的方向发展。传统的集散控制方式和计算机分层控制方式已经开始让位于智能终端与网络结合的总线网络控制方式。当今，在工厂中过程控制环境下的分布式自动化系统变得越来越复杂，尤其系统内部的各设备之间需要快速交换大量的信息，以便实现对被控系统更为精确的控制和提供一些辅助的评价函数。这就意味着要不断增加带宽和提高通信速率以满足网络通信的需要。在现有的多种可利用网络设备中，CAN总线以其清晰的定义、极高的可靠性及其独特的设计，被认为是最能有效地解决这一问题的途径之一。而且市场上基于通信技术的产品中，就实时性考虑，由于CAN总线采用的非表意性的通信方式，因此其结构更为简单，实时性更好。基于此背景，我们以CAN总线作为通信媒介，将分布于各控制现场的传感器、执行器和控制器有序地连接起来，构成了一个基于CAN总线的分布式局域网络控制系统。本文首先介绍了基于CAN总线的分布式数据采集与控制系统的总体结构。然后从硬件方面描述了基于CAN总线的通信协议转换单元、数据采集单元和输出控制单元的功能、硬件配置及各单元功能的具体实现过程，给出了各单元的性能指标。软件方面，以C语言作为平台，开发了基于CAN总线的上位计算机管理与监控软件，实现了对整个网络设备的系统管理和系统控制功能。对于该总线系统，作者运用了PID控制和模糊控制算法实现了对水箱液位的控制，达到了理想的效果。基于CAN总线的控制系统很好地解决了集散控制系统难以解决的难题，模糊控制的应用能很好地把总线控制系统应用到具有非线性、大时滞和难于获得精确模型的控制系统中。

Nearest-neighbor Queries in Probabilistic Graphs

Large probabilistic graphs arise in various domains spanning from social networks to biological and communication networks. An important query in these graphs is the k nearest-neighbor query, which involves finding and reporting the k closest nodes to a specific node. This query assumes the existence of a measure of the "proximity" or the "distance" between any two nodes in the graph. To that end, we propose various novel distance functions that extend well known notions of classical graph theory, such as shortest paths and random walks. We argue that many meaningful distance functions are computationally intractable to compute exactly. Thus, in order to process nearest-neighbor queries, we resort to Monte Carlo sampling and exploit novel graph-transformation ideas and pruning opportunities. In our extensive experimental analysis, we explore the trade-offs of our approximation algorithms and demonstrate that they scale well on real-world probabilistic graphs with tens of millions of edges.

Theory of the electronic and optical properties of dilute bismide alloys

Dilute bismide alloys, containing small fractions of bismuth (Bi), have recently attracted interest due to their potential for applications in a range of semiconductor devices. Experiments have revealed that dilute bismide alloys such as GaBixAs1−x, in which a small fraction x of the atoms in the III-V semiconductor GaAs are replaced by Bi, exhibit a number of unusual and unique properties. For example, the band gap energy (E g) decreases rapidly with increasing Bi composition x, by up to 90 meV per % Bi replacing As in the alloy. This band gap reduction is accompanied by a strong increase in the spin-orbit-splitting energy (ΔSO) with increasing x, and both E g and ΔSO are characterised by strong, composition-dependent bowing. The existence of a ΔSO > E g regime in the GaBixAs1−x alloy has been demonstrated for x ≳10%, a band structure condition which is promising for the development of highly efficient, temperature stable semiconductor lasers that could lead to large energy savings in future optical communication networks. In addition to their potential for specific applications, dilute bismide alloys have also attracted interest from a fundamental perspective due to their unique properties. In this thesis we develop the theory of the electronic and optical properties of dilute bismide alloys. By adopting a multi-scale approach encompassing atomistic calculations of the electronic structure using the semi-empirical tight-binding method, as well as continuum calculations based on the k•p method, we develop a fundamental understanding of this unusual class of semiconductor alloys and identify general material properties which are promising for applications in semiconductor optoelectronic and photovoltaic devices. By performing detailed supercell calculations on both ordered and disordered alloys we explicitly demonstrate that Bi atoms act as isovalent impurities when incorporated in dilute quantities in III-V (In)GaAs(P) materials, strongly perturbing the electronic structure of the valence band. We identify and quantify the causes and consequences of the unusual electronic properties of GaBixAs1−x and related alloys, and our analysis is reinforced throughout by a series of detailed comparisons to the results of experimental measurements. Our k•p models of the band structure of GaBixAs1−x and related alloys, which we derive directly from detailed atomistic calculations, are ideally suited to the study of dilute bismide-based devices. We focus in the latter part of the thesis on calculations of the electronic and optical properties of dilute bismide quantum well lasers. In addition to developing an understanding of the effects of Bi incorporation on the operational characteristics of semiconductor lasers, we also present calculations which have been used explicitly in designing and optimising the first generation of GaBixAs1−x-based devices.

Does improved access to diagnostic imaging results reduce hospital length of stay? A retrospective study.

BACKGROUND: One year after the introduction of Information and Communication Technology (ICT) to support diagnostic imaging at our hospital, clinicians had faster and better access to radiology reports and images; direct access to Computed Tomography (CT) reports in the Electronic Medical Record (EMR) was particularly popular. The objective of this study was to determine whether improvements in radiology reporting and clinical access to diagnostic imaging information one year after the ICT introduction were associated with a reduction in the length of patients' hospital stays (LOS). METHODS: Data describing hospital stays and diagnostic imaging were collected retrospectively from the EMR during periods of equal duration before and one year after the introduction of ICT. The post-ICT period was chosen because of the documented improvement in clinical access to radiology results during that period. The data set was randomly split into an exploratory part used to establish the hypotheses, and a confirmatory part. The data was used to compare the pre-ICT and post-ICT status, but also to compare differences between groups. RESULTS: There was no general reduction in LOS one year after ICT introduction. However, there was a 25% reduction for one group - patients with CT scans. This group was heterogeneous, covering 445 different primary discharge diagnoses. Analyses of subgroups were performed to reduce the impact of this divergence. CONCLUSION: Our results did not indicate that improved access to radiology results reduced the patients' LOS. There was, however, a significant reduction in LOS for patients undergoing CT scans. Given the clinicians' interest in CT reports and the results of the subgroup analyses, it is likely that improved access to CT reports contributed to this reduction.

The use of a biased heuristic by a genetic algorithm applied to the design of multipoint connections in a local access network

This paper presents a genetic algorithm for finding a constrained minimum spanning tree. The problem is of relevance in the design of minimum cost communication networks, where there is a need to connect all the terminals at a user site to a terminal concentrator in a multipoint (tree) configuration, while ensuring that link capacity constraints are not violated. The approach used maintains a distinction between genotype and phenotype, which produces superior results to those found using a direct representation in a previous study.

A Novel Packet Marking Function for Real-Time Interactive MPEG-4 Video Applications in a Differentiated Services Network

The future convergence of voice, video and data applications on the Internet requires that next generation technology provides bandwidth and delay guarantees. Current technology trends are moving towards scalable aggregate-based systems where applications are grouped together and guarantees are provided at the aggregate level only. This solution alone is not enough for interactive video applications with sub-second delay bounds. This paper introduces a novel packet marking scheme that controls the end-to-end delay of an individual flow as it traverses a network enabled to supply aggregate- granularity Quality of Service (QoS). IPv6 Hop-by-Hop extension header fields are used to track the packet delay encountered at each network node and autonomous decisions are made on the best queuing strategy to employ. The results of network simulations are presented and it is shown that when the proposed mechanism is employed the requested delay bound is met with a 20% reduction in resource reservation and no packet loss in the network.