Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.

On Reducing Dynamic Web Page Construction Times

Many web sites incorporate dynamic web pages to deliver customized contents to their users. However, dynamic pages result in increased user response times due to their construction overheads. In this paper, we consider mechanisms for reducing these overheads by utilizing the excess capacity with which web servers are typically provisioned. Specifically, we present a caching technique that integrates fragment caching with anticipatory page pre-generation in order to deliver dynamic pages faster during normal operating situations. A feedback mechanism is used to tune the page pre-generation process to match the current system load. The experimental results from a detailed simulation study of our technique indicate that, given a fixed cache budget, page construction speedups of more than fifty percent can be consistently achieved as compared to a pure fragment caching approach.

Dynamic shear modulus as affected by displacement amplitude

The realistic estimation of the dynamic characteristics for a known set of loading conditions continues to be difficult despite many contributions in the past. The design of a machine foundation is generally made on the basis of limiting amplitude or resonant frequency. These parameters are in turn dependent on the dynamic characteristics of soil viz., the shear modulus/stiffness and damping. The work reported herein is an attempt to relate statistically the shear modulus of a soil to its resonant amplitude under a known set of static and dynamic loading conditions as well as wide ranging soil conditions. The two parameters have been statistically related with a good correlation coefficient and low standard error of estimate.

Network Reconstruction from dynamic data

Over the past decade, many powerful data mining techniques have been developed to analyze temporal and sequential data. The time is now fertile for addressing problems of larger scope under the purview of temporal data mining. The fourth SIGKDD workshop on temporal data mining focused on the question: What can we infer about the structure of a complex dynamical system from observed temporal data? The goals of the workshop were to critically evaluate the need in this area by bringing together leading researchers from industry and academia, and to identify promising technologies and methodologies for doing the same. We provide a brief summary of the workshop proceedings and ideas arising out of the discussions.

Dynamic Internal Cavities of Dendrimers as Constrained Media. A Study of Photochemical Isomerizations of Stilbene and Azobenzene Using Poly(alkyl aryl ether) Dendrimers

Dendritic rnicroenvironments defined by dynamic internal cavities of a dendrimer were probed through geometric isomerization of stilbene and azobenzene. A third-generation poly(alkyl aryl ether) dendrimer with hydrophilic exterior and hydrophobic interior was used as a reaction cavity in aqueous medium. The dynamic inner cavity sizes were varied by utilizing alkyl linkers that connect the branch junctures from ethyl to n-pentyl moiety (C(2)G(3)-C(5)G(3)). Dendrimers constituted with n-pentyl linker were found to afford higher solubilities of stilbene and azobenzene. Direct irradiation of trans-stilbene showed that C(5)G(3) and C(4)G(3) dendrimers afforded considerable phenanthrene formation, in addition to cis-stilbene, whereas C(3)G(3) and C(2)G(3) gave only cis-stilbene. An electron-transfer sensitized trans-cis isomerization, using cresyl violet perchlorate as the sensitizer, also led to similar results. Thermal isomerization of cis-azobenzene to trans-azobenzene within dendritic microenvironments revealed that the activation energy of the cis- to trans-isomer was increasing in the series C(5)G(3) < C(4)G(3) < C(3)G(3)

Dynamic Supply and Threshold Voltage Scaling for CMOS Digital Circuits Using In-Situ Power Monitor

A generalized power tracking algorithm that minimizes power consumption of digital circuits by dynamic control of supply voltage and the body bias is proposed. A direct power monitoring scheme is proposed that does not need any replica and hence can sense total power consumed by load circuit across process, voltage, and temperature corners. Design details and performance of power monitor and tracking algorithm are examined by a simulation framework developed using UMC 90-nm CMOS triple well process. The proposed algorithm with direct power monitor achieves a power savings of 42.2% for activity of 0.02 and 22.4% for activity of 0.04. Experimental results from test chip fabricated in AMS 350 nm process shows power savings of 46.3% and 65% for load circuit operating in super threshold and near sub-threshold region, respectively. Measured resolution of power monitor is around 0.25 mV and it has a power overhead of 2.2% of die power. Issues with loop convergence and design tradeoff for power monitor are also discussed in this paper.

Microelectromechanical torsional varactors with low parasitic capacitances and high dynamic range

This work focuses on the design of torsional microelectromechanical systems (MEMS) varactors to achieve highdynamic range of capacitances. MEMS varactors fabricated through the polyMUMPS process are characterized at low and high frequencies for their capacitance-voltage characteristics and electrical parasitics. The effect of parasitic capacitances on tuning ratio is studied and an equivalent circuit is developed. Two variants of torsional varactors that help to improve the dynamic range of torsional varactors despite the parasitics are proposed and characterized. A tuning ratio of 1:8, which is the highest reported in literature, has been obtained. We also demonstrate through simulations that much higher tuning ratios can be obtained with the designs proposed. The designs and experimental results presented are relevant to CMOS fabrication processes that use low resistivity substrate. (C) 2012 Society of Photo-Optical Instrumentation Engineers (SPIE). DOI: 10.1117/1.JMM.11.1.013006]

Performance of inherently compensated flat pad aerostatic bearings subject to dynamic perturbation forces

The importance of air bearing design is growing in engineering. As the trend to precision and ultra precision manufacture gains pace and the drive to higher quality and more reliable products continues, the advantages which can be gained from applying aerostatic bearings to machine tools, instrumentation and test rigs is becoming more apparent. The inlet restrictor design is significant for air bearings because it affects the static and dynamic performance of the air bearing. For instance pocketed orifice bearings give higher load capacity as compared to inherently compensated orifice type bearings, however inherently compensated orifices, also known as laminar flow restrictors are known to give highly stable air bearing systems (less prone to pneumatic hammer) as compared to pocketed orifice air bearing systems. However, they are not commonly used because of the difficulties encountered in manufacturing and assembly of the orifice designs. This paper aims to analyse the static and dynamic characteristics of inherently compensated orifice based flat pad air bearing system. Based on Reynolds equation and mass conservation equation for incompressible flow, the steady state characteristics are studied while the dynamic state characteristics are performed in a similar manner however, using the above equations for compressible flow. Steady state experiments were also performed for a single orifice air bearing and the results are compared to that obtained from theoretical studies. A technique to ease the assembly of orifices with the air bearing plate has also been discussed so as to make the manufacturing of the inherently compensated bearings more commercially viable. (c) 2012 Elsevier Inc. All rights reserved.

Nonlinear dynamic state estimation in instrumented structures with conditionally linear Gaussian substructures

Many problems of state estimation in structural dynamics permit a partitioning of system states into nonlinear and conditionally linear substructures. This enables a part of the problem to be solved exactly, using the Kalman filter, and the remainder using Monte Carlo simulations. The present study develops an algorithm that combines sequential importance sampling based particle filtering with Kalman filtering to a fairly general form of process equations and demonstrates the application of a substructuring scheme to problems of hidden state estimation in structures with local nonlinearities, response sensitivity model updating in nonlinear systems, and characterization of residual displacements in instrumented inelastic structures. The paper also theoretically demonstrates that the sampling variance associated with the substructuring scheme used does not exceed the sampling variance corresponding to the Monte Carlo filtering without substructuring. (C) 2012 Elsevier Ltd. All rights reserved.

Effective contrast recovery in rapid dynamic near-infrared diffuse optical tomography using l(1)-norm-based linear image reconstruction method

Traditional image reconstruction methods in rapid dynamic diffuse optical tomography employ l(2)-norm-based regularization, which is known to remove the high-frequency components in the reconstructed images and make them appear smooth. The contrast recovery in these type of methods is typically dependent on the iterative nature of method employed, where the nonlinear iterative technique is known to perform better in comparison to linear techniques (noniterative) with a caveat that nonlinear techniques are computationally complex. Assuming that there is a linear dependency of solution between successive frames resulted in a linear inverse problem. This new framework with the combination of l(1)-norm based regularization can provide better robustness to noise and provide better contrast recovery compared to conventional l(2)-based techniques. Moreover, it is shown that the proposed l(1)-based technique is computationally efficient compared to its counterpart (l(2)-based one). The proposed framework requires a reasonably close estimate of the actual solution for the initial frame, and any suboptimal estimate leads to erroneous reconstruction results for the subsequent frames.

Evaluation of dynamic voltage and frequency scaling for stream programs

Dynamic Voltage and Frequency Scaling (DVFS) offers a huge potential for designing trade-offs involving energy, power, temperature and performance of computing systems. In this paper, we evaluate three different DVFS schemes - our enhancement of a Petri net performance model based DVFS method for sequential programs to stream programs, a simple profile based Linear Scaling method, and an existing hardware based DVFS method for multithreaded applications - using multithreaded stream applications, in a full system Chip Multiprocessor (CMP) simulator. From our evaluation, we find that the software based methods achieve significant Energy/Throughput2(ET−2) improvements. The hardware based scheme degrades performance heavily and suffers ET−2 loss. Our results indicate that the simple profile based scheme achieves the benefits of the complex Petri net based scheme for stream programs, and present a strong case for the need for independent voltage/frequency control for different cores of CMPs, which is lacking in most of the state-of-the-art CMPs. This is in contrast to the conclusions of a recent evaluation of per-core DVFS schemes for multithreaded applications for CMPs.

Dynamic mechanism design for markets with strategic resources

The assignment of tasks to multiple resources becomes an interesting game theoretic problem, when both the task owner and the resources are strategic. In the classical, nonstrategic setting, where the states of the tasks and resources are observable by the controller, this problem is that of finding an optimal policy for a Markov decision process (MDP). When the states are held by strategic agents, the problem of an efficient task allocation extends beyond that of solving an MDP and becomes that of designing a mechanism. Motivated by this fact, we propose a general mechanism which decides on an allocation rule for the tasks and resources and a payment rule to incentivize agents' participation and truthful reports. In contrast to related dynamic strategic control problems studied in recent literature, the problem studied here has interdependent values: the benefit of an allocation to the task owner is not simply a function of the characteristics of the task itself and the allocation, but also of the state of the resources. We introduce a dynamic extension of Mezzetti's two phase mechanism for interdependent valuations. In this changed setting, the proposed dynamic mechanism is efficient, within period ex-post incentive compatible, and within period ex-post individually rational.

A Savitzky-Golay Filtering Perspective of Dynamic Feature Computation

We address the classical problem of delta feature computation, and interpret the operation involved in terms of Savitzky- Golay (SG) filtering. Features such as themel-frequency cepstral coefficients (MFCCs), obtained based on short-time spectra of the speech signal, are commonly used in speech recognition tasks. In order to incorporate the dynamics of speech, auxiliary delta and delta-delta features, which are computed as temporal derivatives of the original features, are used. Typically, the delta features are computed in a smooth fashion using local least-squares (LS) polynomial fitting on each feature vector component trajectory. In the light of the original work of Savitzky and Golay, and a recent article by Schafer in IEEE Signal Processing Magazine, we interpret the dynamic feature vector computation for arbitrary derivative orders as SG filtering with a fixed impulse response. This filtering equivalence brings in significantly lower latency with no loss in accuracy, as validated by results on a TIMIT phoneme recognition task. The SG filters involved in dynamic parameter computation can be viewed as modulation filters, proposed by Hermansky.

On the dynamic contact angle in simulation of impinging droplets with sharp interface methods

Effects of dynamic contact angle models on the flow dynamics of an impinging droplet in sharp interface simulations are presented in this article. In the considered finite element scheme, the free surface is tracked using the arbitrary Lagrangian-Eulerian approach. The contact angle is incorporated into the model by replacing the curvature with the Laplace-Beltrami operator and integration by parts. Further, the Navier-slip with friction boundary condition is used to avoid stress singularities at the contact line. Our study demonstrates that the contact angle models have almost no influence on the flow dynamics of the non-wetting droplets. In computations of the wetting and partially wetting droplets, different contact angle models induce different flow dynamics, especially during recoiling. It is shown that a large value for the slip number has to be used in computations of the wetting and partially wetting droplets in order to reduce the effects of the contact angle models. Among all models, the equilibrium model is simple and easy to implement. Further, the equilibrium model also incorporates the contact angle hysteresis. Thus, the equilibrium contact angle model is preferred in sharp interface numerical schemes.