Bidding Dynamics of Rational Advertisers in Sponsored Search Auctions on the Web

In this paper, we address a key problem faced by advertisers in sponsored search auctions on the web: how much to bid, given the bids of the other advertisers, so as to maximize individual payoffs? Assuming the generalized second price auction as the auction mechanism, we formulate this problem in the framework of an infinite horizon alternative-move game of advertiser bidding behavior. For a sponsored search auction involving two advertisers, we characterize all the pure strategy and mixed strategy Nash equilibria. We also prove that the bid prices will lead to a Nash equilibrium, if the advertisers follow a myopic best response bidding strategy. Following this, we investigate the bidding behavior of the advertisers if they use Q-learning. We discover empirically an interesting trend that the Q-values converge even if both the advertisers learn simultaneously.

Incremental natural-gradient actor-critic algorithms

We present four new reinforcement learning algorithms based on actor-critic and natural-gradient ideas, and provide their convergence proofs. Actor-critic rein- forcement learning methods are online approximations to policy iteration in which the value-function parameters are estimated using temporal difference learning and the policy parameters are updated by stochastic gradient descent. Methods based on policy gradients in this way are of special interest because of their com- patibility with function approximation methods, which are needed to handle large or infinite state spaces. The use of temporal difference learning in this way is of interest because in many applications it dramatically reduces the variance of the gradient estimates. The use of the natural gradient is of interest because it can produce better conditioned parameterizations and has been shown to further re- duce variance in some cases. Our results extend prior two-timescale convergence results for actor-critic methods by Konda and Tsitsiklis by using temporal differ- ence learning in the actor and by incorporating natural gradients, and they extend prior empirical studies of natural actor-critic methods by Peters, Vijayakumar and Schaal by providing the first convergence proofs and the first fully incremental algorithms.

Simulation of RNA silencing pathway for time-dependent transgene transcription rate

The synthesis of dsRNA is analyzed using a pathway model with amplifications caused by the aberrant RNAs. The transgene influx rate is assumed time-decaying considering the fact that the number of transgenes can not be infinite. The dynamics of the transgene induced RNA silencing is investigated using a system of coupled nonautonomous ordinary nonlinear differential equations which describe the model phenomenologically. The silencing phenomena are detected after a period of transcription. Important contributions of certain parameters are discussed with several numerical examples.

Active Vibration Suppression of One-dimensional Nonlinear Structures Using Optimal Dynamic Inversion

A flexible robot arm can be modeled as an Euler-Bernoulli beam which are infinite degrees of freedom (DOF) system. Proper control is needed to track the desired motion of a robotic arm. The infinite number of DOF of beams are reduced to finite number for controller implementation, which brings in error (due to their distributed nature). Therefore, to represent reality better distributed parameter systems (DPS) should be controlled using the systems partial differential equation (PDE) directly. In this paper, we propose to use a recently developed optimal dynamic inversion technique to design a controller to suppress nonlinear vibration of a beam. The method used in this paper determines control forces directly from the PDE model of the system. The formulation has better practical significance, because it leads to a closed form solution of the controller (hence avoids computational issues).

Pricing a Shared Access Link for Fair and Efficient Operation with Variable User Data Rates

The literature on pricing implicitly assumes an "infinite data" model, in which sources can sustain any data rate indefinitely. We assume a more realistic "finite data" model, in which sources occasionally run out of data; this leads to variable user data rates. Further, we assume that users have contracts with the service provider, specifying the rates at which they can inject traffic into the network. Our objective is to study how prices can be set such that a single link can be shared efficiently and fairly among users in a dynamically changing scenario where a subset of users occasionally has little data to send. User preferences are modelled by concave increasing utility functions. Further, we introduce two additional elements: a convex increasing disutility function and a convex increasing multiplicative congestion-penally function. The disutility function takes the shortfall (contracted rate minus present rate) as its argument, and essentially encourages users to send traffic at their contracted rates, while the congestion-penalty function discourages heavy users from sending excess data when the link is congested. We obtain simple necessary and sufficient conditions on prices for fair and efficient link sharing; moreover, we show that a single price for all users achieves this. We illustrate the ideas using a simple experiment.

Resource Pricing for Fair and Efficient Link Sharing with a Finite Data Model

The literature on pricing implicitly assumes an "infinite data" model, in which sources can sustain any data rate indefinitely. We assume a more realistic "finite data" model, in which sources occasionally run out of data. Further, we assume that users have contracts with the service provider, specifying the rates at which they can inject traffic into the network. Our objective is to study how prices can be set such that a single link can be shared efficiently and fairly among users in a dynamically changing scenario where a subset of users occasionally has little data to send. We obtain simple necessary and sufficient conditions on prices such that efficient and fair link sharing is possible. We illustrate the ideas using a simple example

A Simultaneous Deterministic Perturbation Actor-Critic Algorithm with an Application to Optimal Mortgage Refinancing

We develop a simulation-based, two-timescale actor-critic algorithm for infinite horizon Markov decision processes with finite state and action spaces, with a discounted reward criterion. The algorithm is of the gradient ascent type and performs a search in the space of stationary randomized policies. The algorithm uses certain simultaneous deterministic perturbation stochastic approximation (SDPSA) gradient estimates for enhanced performance. We show an application of our algorithm on a problem of mortgage refinancing. Our algorithm obtains the optimal refinancing strategies in a computationally efficient manner

Thermal Weight Functions and Stress Intensity Factors for Bonded Dissimilar Media Using Body Analogy

In this study, an analytical method is presented for the computation of thermal weight functions in two dimensional bi-material elastic bodies containing a crack at the interface and subjected to thermal loads using body analogy method. The thermal weight functions are derived for two problems of infinite bonded dissimilar media, one with a semi-infinite crack and the other with a finite crack along the interface. The derived thermal weight functions are shown to reduce to the already known expressions of thermal weight functions available in the literature for the respective homogeneous elastic body. Using these thermal weight functions, the stress intensity factors are computed for the above interface crack problems when subjected to an instantaneous heat source.

Solute solute and solvent solute interactions in solid solutions of Cu+Sn, Au+Sn and Cu+Au+Sn alloys

The chemical potentials of tin in its α-solid solutions with Cu, Au and Cu + Au alloys have been measured using a gas-solid equilibration technique. The variation of the excess chemical potential of tin with its composition in the alloy is related to the solute-solute repulsive interaction, while the excess chemical potential at infinite dilution of the solute is a measure of solvent-solute interaction energies. It is shown that solute-solute interaction is primarily determined by the concentration of (s + p) electrons in the conduction band, although the interaction energies are smaller than those predicted by either the rigid band model or calculation based on Friedel oscillations in the potential function. Finally, the variation of the solvent-solute interaction with solvent composition in the ternary system can be accounted for in terms of a quasi-chemical treatment which takes into account the clustering of the solvent atoms around the solute.

Analysis of a linear one-dimensional active noise control system by means of block diagrams and transfer functions

In arriving at the ideal filter transfer function for an active noise control system in a duct, the effect of the auxiliary sources (generally loudspeakers) on the waves generated by the primary source has invariably been neglected in the existing literature, implying a rigid wall or infinite impedance. The present paper presents a fairly general analysis of a linear one-dimensional noise control system by means of block diagrams and transfer functions. It takes into account the passive as well as active role of a terminal primary source, wall-mounted auxiliary source, open duct radiation impedance, and the effects of mean flow and damping. It is proved that the pressure generated by a source against a load impedance can be looked upon as a sum of two pressure waves, one generated by the source against an anechoic termination and the other by reflecting the rearward wave (incident on the source) off the passive source impedance. Application of this concept is illustrated for both the types of sources. A concise closed-form expression for the ideal filter transfer function is thus derived and discussed. Finally, the dynamics of an adaptive noise control system is discussed briefly, relating its standing-wave variables and transfer functions with those of the progressive-wave model presented here.

Extending the Supramolecular Synthon Based Fragment Approach (SBFA) for Transferability of Multipole Charge Density Parameters to Monofluorobenzoic Acids and their Cocrystals with Isonicotinamide: Importance of C-H center dot center dot center dot O, C-H center dot center dot center dot F, and F center dot center dot center dot F Intermolecular Regions

An extension of the supramolecular synthon-based fragment approach (SBFA) method for transferability of multipole charge density parameters to include weak supramolecular synthons is proposed. In particular, the SBFA method is applied to C-H center dot center dot center dot O, C-H center dot center dot center dot F, and F center dot center dot center dot F containing synthons. A high resolution charge density study has been performed on 4-fluorobenzoic acid to build a synthon library for C-H center dot center dot center dot F infinite chain interactions. Libraries for C-H center dot center dot center dot O and F center dot center dot center dot F synthons were taken from earlier work. The SBFA methodology was applied successfully to 2- and 3-fluorobenzoic acids, data sets for which were collected in a routine manner at 100 K, and the modularity of the synthons was demonstrated. Cocrystals of isonicotinamide with all three fluorobenzoic acids were also studied with the SBFA method. The topological analysis of inter- and intramolecular interaction regions was performed using Bader's AIM approach. This study shows that the SBFA method is generally applicable to generate charge density maps using information from multiple intermolecular regions.

Additions to the z transform table

The ztransform method is a widely used tool for the analysis and synthesis of discrete systems. In this note a table of ztransform pairs when F(z) is an irrational function of z is given. The table is also useful for obtaining closed-form sums for some infinite series.

Topological analysis of general linear networks

A new method of network analysis, a generalization in several different senses of existing methods and applicable to all networks for which a branch-admittance (or impedance) matrix can be formed, is presented. The treatment of network determinants is very general and essentially four terminal rather than three terminal, and leads to simple expressions based on trees of a simple graph associated with the network and matrix, and involving products of low-order, usually(2 times 2)determinants of tree-branch admittances, in addition to tree-branch products as in existing methods. By comparison with existing methods, the total number of trees and of tree pairs is usually considerably reduced, and this fact, together with an easy method of tree-pair sign determination which is also presented, makes the new method simpler in general. The method can be very easily adapted, by the use of infinite parameters, to accommodate ideal transformers, operational amplifiers, and other forms of network constraint; in fact, is thought to be applicable to all linear networks.

Matrix Processors Using p-adic Arithmetic for Exact Linear Computations

A unique code (called Hensel's code) is derived for a rational number by truncating its infinite p-adic expansion. The four basic arithmetic algorithms for these codes are described and their application to rational matrix computations is demonstrated by solving a system of linear equations exactly, using the Gaussian elimination procedure.

Improvement of power-system transient stability by phase-shift insertion

Computational studies of the transient stability of a synchronous machine connected to an infinite busbar by a double-circuit transmission line are used to illustrate the effect of relative phase-shift insertion between the machine and its associated power system. This method of obtaining a change in the effective rotor-excitation angle, and thereby the power transfer, is described, together with an outline of possible methods of implementation by a phase-shifting transformer in a power system.