Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market consisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL techniques have not previously been applied. We consider two representative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pricing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as number of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the partial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory replenishments.

Smart applications for energy harvested WSNs

A Wireless Sensor Network (WSN) powered using harvested energies is limited in its operation by instantaneous power. Since energy availability can be different across nodes in the network, network setup and collaboration is a non trivial task. At the same time, in the event of excess energy, exciting node collaboration possibilities exist; often not feasible with battery driven sensor networks. Operations such as sensing, computation, storage and communication are required to achieve the common goal for any sensor network. In this paper, we design and implement a smart application that uses a Decision Engine, and morphs itself into an energy matched application. The results are based on measurements using IRIS motes running on solar energy. We have done away with batteries; instead used low leakage super capacitors to store harvested energy. The Decision Engine utilizes two pieces of data to provide its recommendations. Firstly, a history based energy prediction model assists the engine with information about in-coming energy. The second input is the energy cost database for operations. The energy driven Decision Engine calculates the energy budgets and recommends the best possible set of operations. Under excess energy condition, the Decision Engine, promiscuously sniffs the neighborhood looking for all possible data from neighbors. This data includes neighbor's energy level and sensor data. Equipped with this data, nodes establish detailed data correlation and thus enhance collaboration such as filling up data gaps on behalf of nodes hibernating under low energy conditions. The results are encouraging. Node and network life time of the sensor nodes running the smart application is found to be significantly higher compared to the base application.

A smart approach to precision attitude maneuvers of spacecrafts

A nonlinear adaptive approach is presented to achieve rest-to-rest attitude maneuvers for spacecrafts in the presence of parameter uncertainties and unknown disturbances. A nonlinear controller, designed on the principle of dynamic inversion achieves the goals for the nominal model but suffers performance degradation in the presence of off-nominal parameter values and unwanted inputs. To address this issue, a model-following neuro-adaptive control design is carried out by taking the help of neural networks. Due to the structured approach followed here, the adaptation is restricted to the momentum level equations.The adaptive technique presented is computationally nonintensive and hence can be implemented in real-time. Because of these features, this new approach is named as structured model-following adaptive real-time technique (SMART). From simulation studies, this SMART approach is found to be very effective in achieving precision attitude maneuvers in the presence of parameter uncertainties and unknown disturbances.

Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.

Smart polymer mediated purification and recovery of active proteins from inclusion bodies

Obtaining correctly folded proteins from inclusion bodies of recombinant proteins expressed in bacterial hosts requires solubilization with denaturants and a refolding step. Aggregation competes with the second step. Refolding of eight different proteins was carried out by precipitation with smart polymers. These proteins have different molecular weights, different number of disulfide bridges and some of these are known to be highly prone to aggregation. A high throughput refolding screen based upon fluorescence emission maximum around 340 nm (for correctly folded proteins) was developed to identify the suitable smart polymer. The proteins could be dissociated and recovered after the refolding step. The refolding could be scaled up and high refolding yields in the range of 8 mg L-1 (for CD4D12, the first two domains of human CD4) to 58 mg L-1 (for malETrx, thioredoxin fused with signal peptide of maltose binding protein) were obtained. Dynamic light scattering (DLS) showed that polymer if chosen correctly acted as a pseuclochaperonin and bound to the proteins. It also showed that the time for maximum binding was about 50 min which coincided with the time required for incubation (with the polymer) before precipitation for maximum recovery of folded proteins. The refolded proteins were characterized by fluorescence emission spectra, circular dichroism (CD) spectroscopy, melting temperature (T-m), and surface hydrophobicity measurement by ANS (8-anilinol-naphthalene sulfonic acid) fluorescence. Biological activity assay for thioredoxin and fluorescence based assay in case of maltose binding protein (MBP) were also carried out to confirm correct refolding. (C) 2012 Elsevier B.V. All rights reserved.

Electrochemical capacitors: Technical challenges and prognosis for future markets

Electrochemical capacitors are potential devices that could help bringing about major advances in future energy storage. They are lightweight and their manufacture and disposal has no detrimental effects on the environment. A comprehensive description of fundamental science of electrochemical capacitors is presented. Similarities and differences between electrochemical capacitors and secondary batteries for electrical energy storage are highlighted and various types of electrochemical capacitors are discussed with special reference to lead-carbon hybrid ultracapacitors. Some envisaged applications of electrochemical capacitors are described along with the technical challenges and prognosis for future markets. (C) 2012 Published by Elsevier Ltd.

Smart Network Interface Selection for E-DTNs

We implement two energy models that accurately and comprehensively estimates the system energy cost and communication energy cost for using Bluetooth and Wi-Fi interfaces. The energy models running on a system is used to smartly pick the most energy optimal network interface so that data transfer between two end points is maximized.

Dynamic mechanism design for markets with strategic resources

The assignment of tasks to multiple resources becomes an interesting game theoretic problem, when both the task owner and the resources are strategic. In the classical, nonstrategic setting, where the states of the tasks and resources are observable by the controller, this problem is that of finding an optimal policy for a Markov decision process (MDP). When the states are held by strategic agents, the problem of an efficient task allocation extends beyond that of solving an MDP and becomes that of designing a mechanism. Motivated by this fact, we propose a general mechanism which decides on an allocation rule for the tasks and resources and a payment rule to incentivize agents' participation and truthful reports. In contrast to related dynamic strategic control problems studied in recent literature, the problem studied here has interdependent values: the benefit of an allocation to the task owner is not simply a function of the characteristics of the task itself and the allocation, but also of the state of the resources. We introduce a dynamic extension of Mezzetti's two phase mechanism for interdependent valuations. In this changed setting, the proposed dynamic mechanism is efficient, within period ex-post incentive compatible, and within period ex-post individually rational.

Material uncertainty effect on vibration control of smart composite plate using polynomial chaos expansion

Polynomial Chaos Expansion with Latin Hypercube sampling is used to study the effect of material uncertainty on vibration control of a smart composite plate with piezoelectric sensors/actuators. Composite material properties and piezoelectric coefficients are considered as independent and normally distributed random variables. Numerical results show substantial variation in structural dynamic response due to material uncertainty of active vibration control system. This change in response due to material uncertainty can be compensated by actively tuning the feedback control system. Numerical results also show variation in dispersion of dynamic characteristics and control parameters with respect to ply angle and stacking sequence.

Modified current space vector based powerline communication method for distributed inverters interfaced smart grids

The following paper presents a Powerline Communication (PLC) Method for grid interfaced inverters, for smart grid application. The PLC method is based on the concept of the composite vector which involves multiple components rotating at different harmonic frequencies. The pulsed information is modulated on the fundamental component of the grid current as a specific repeating sequence of a particular harmonic. The principle of communication is same as that of power flow, thus reducing the complexity. The power flow and information exchange are simultaneously accomplished by the interfacing inverters based on current programmed vector control, thus eliminating the need for dedicated hardware. Simulation results have been shown for inter-inverter communication, both under ideal and distorted conditions, using various harmonic modulating signals.