A Comparison of Machine Learning Techniques for Modeling River Flow Time Series: The Case of Upper Cauvery River Basin

Models of river flow time series are essential in efficient management of a river basin. It helps policy makers in developing efficient water utilization strategies to maximize the utility of scarce water resource. Time series analysis has been used extensively for modeling river flow data. The use of machine learning techniques such as support-vector regression and neural network models is gaining increasing popularity. In this paper we compare the performance of these techniques by applying it to a long-term time-series data of the inflows into the Krishnaraja Sagar reservoir (KRS) from three tributaries of the river Cauvery. In this study flow data over a period of 30 years from three different observation points established in upper Cauvery river sub-basin is analyzed to estimate their contribution to KRS. Specifically, ANN model uses a multi-layer feed forward network trained with a back-propagation algorithm and support vector regression with epsilon intensive-loss function is used. Auto-regressive moving average models are also applied to the same data. The performance of different techniques is compared using performance metrics such as root mean squared error (RMSE), correlation, normalized root mean squared error (NRMSE) and Nash-Sutcliffe Efficiency (NSE).

Applicability of Statistical Learning Algorithms for Spatial Variability of Rock Depth

Two algorithms are outlined, each of which has interesting features for modeling of spatial variability of rock depth. In this paper, reduced level of rock at Bangalore, India, is arrived from the 652 boreholes data in the area covering 220 sqa <.km. Support vector machine (SVM) and relevance vector machine (RVM) have been utilized to predict the reduced level of rock in the subsurface of Bangalore and to study the spatial variability of the rock depth. The support vector machine (SVM) that is firmly based on the theory of statistical learning theory uses regression technique by introducing epsilon-insensitive loss function has been adopted. RVM is a probabilistic model similar to the widespread SVM, but where the training takes place in a Bayesian framework. Prediction results show the ability of learning machine to build accurate models for spatial variability of rock depth with strong predictive capabilities. The paper also highlights the capability ofRVM over the SVM model.

Hormones and Cuscuta development: IAA uptake transport and metabolism in relation to growth in the absence and presence of applied cytokinin

Transport of 1-14C-IAA in successive stem segments of Cuscuta was strictly basipetal in growing and non growing regions of the vine with a flux velocity of 10-12 mm/h (intercept method). This transport showed a distinct peaked profile, increasing from a low value at 10 mm from the apex to a maximum between 50 and 90 mm before declining to a low value again around 160 mm at which elongation growth ceased. The IAA transport profile paralleled the in vivo growth rate profile, though the latter peaked ahead of transport. A better correlation was observed between the profile of growth responsiveness of the vine to exogenous IAA application and the profile of IAA transport. Growth responsiveness was determined as the differential in growth rate of stem segments in vitro in the absence and presence of growth optimal concentration of IAA (10 μm). Retention of exogenous IAA in the stem was maximal where transport decreased, and this coincided with the region of maximal conjugation of applied 1-14C-IAA to aspartic acid to form indoleacetylaspartate (IAAsp). In addition to aspartate, IAA was conjugated to a small extent to an unidentified compound. IAA destruction by decarboxylation was greatest where transport was low, particularly in the nongrowing region, where lignification occurred (i.e., beyond 180 mm). At concentrations up to 20 μM, a pulse of 1-14C-IAA chased by "cold" IAA moved as a peak (with a peak displacement velocity of 12-18 mm/h) in the "growth" region of the vine, but became diffusionlike where growth either fell off steeply or ceased. At a higher (50 μM) IAA concentration, though uptake was not saturated, transport in the growth region became diffusionlike, indicating saturation of the system. Reduced IAA flux in the region where growth responsiveness to IAA declined coincided with the region of increased IAA conjugation. However, it cannot be concluded whether increased IAA conjugation was the cause or effect of decreased IAA flux. Application of benzyladenine to the vines in vivo, a treatment that elicited haustoria formation by 72 h, resulted in the inhibition of both IAA transport and elongation growth rate in the subapical region. In vitro treatment of vine segments with BA similarly increased IAA retention and decreased IAA transport. IAA loss was suppressed, and conjugation to IAAsp was enhanced. © 1989 Springer-Verlag New York Inc.

A tutorial survey of reinforcement learning

This paper gives a compact, self-contained tutorial survey of reinforcement learning, a tool that is increasingly finding application in the development of intelligent dynamic systems. Research on reinforcement learning during the past decade has led to the development of a variety of useful algorithms. This paper surveys the literature and presents the algorithms in a cohesive framework.

Role of applied potentials on bioleaching of chalcopyrite concentrate and growth of Acidithiobacillus ferrooxidans

The effect of applied DC potentials on the bioleaching of a chalcopyrite concentrate in the presence of Acidithiobacillus ferrooxidans is discussed. Copper dissolution was the highest at an applied potential of +600mV (SCE), while all the dissolved copper got cathodically deposited at a negative potential of -600mV (SCE). Electrobioleaching at an applied potential of +600mV (SCE) was established at different pulp densities as a function of time. The effect of applied potentials and electrolytic currents on the activity and growth of bacterial cells was assessed Preadaptation of bacterial cells to the concentrate slurry and electrolytic growth conditions significantly enhanced copper dissolution. Electrochemical and biochemical mechanisms involved in electrobioleaching are illustrated with respect to oxidative dissolution and biocatalysis of anodic oxidation.

Effect of applied pressure on densification of monolithic ZrCx ceramic by reactive hot pressing

The effect of applied pressure on reactive hot pressing (RHP) of zirconium (Zr):graphite (C) in molar ratios of 1:0.5, 1:0.67, 1:0.8, and 1:1 was studied at 1200 degrees C for 60 min. The relative density achievable increased with increasing pressure and ranged from 99% at 4 MPa for ZrC0.5 to 93% for stoichiometric ZrC at 100 MPa. The diminishing influence of pressure on the final density with increasing stoichiometry is attributed to two causes: the decreasing initial volume fraction of the plastically deforming Zr metal which leads to the earlier formation of a contiguous, stress shielding carbide skeleton and the larger molar volume shrinkage during reaction which leads to pore formation in the final stages. A numerical model of the creep densification of a dynamically evolving microstructure predicts densities that are consistent with observations and confirm that the availability of a soft metal is primarily responsible for the achievement of such elevated densification during RHP. The ability to densify nonstoichiometric compositions like ZrC0.5 at pressures as low as 4 MPa offers an alternate route to fabricating dense nonstoichiometric carbides.

A hierarchical system of learning automata that can learn die globally optimal path

Systems of learning automata have been studied by various researchers to evolve useful strategies for decision making under uncertainity. Considered in this paper are a class of hierarchical systems of learning automata where the system gets responses from its environment at each level of the hierarchy. A classification of such sequential learning tasks based on the complexity of the learning problem is presented. It is shown that none of the existing algorithms can perform in the most general type of hierarchical problem. An algorithm for learning the globally optimal path in this general setting is presented, and its convergence is established. This algorithm needs information transfer from the lower levels to the higher levels. Using the methodology of estimator algorithms, this model can be generalized to accommodate other kinds of hierarchical learning tasks.

Learning Optimal Discriminant Functions through a Cooperative Game of Automata

The problem of learning correct decision rules to minimize the probability of misclassification is a long-standing problem of supervised learning in pattern recognition. The problem of learning such optimal discriminant functions is considered for the class of problems where the statistical properties of the pattern classes are completely unknown. The problem is posed as a game with common payoff played by a team of mutually cooperating learning automata. This essentially results in a probabilistic search through the space of classifiers. The approach is inherently capable of learning discriminant functions that are nonlinear in their parameters also. A learning algorithm is presented for the team and convergence is established. It is proved that the team can obtain the optimal classifier to an arbitrary approximation. Simulation results with a few examples are presented where the team learns the optimal classifier.

A Team of Continuous-Action Learning Automata for Noise-Tolerant Learning of Half-Spaces

Learning automata are adaptive decision making devices that are found useful in a variety of machine learning and pattern recognition applications. Although most learning automata methods deal with the case of finitely many actions for the automaton, there are also models of continuous-action-set learning automata (CALA). A team of such CALA can be useful in stochastic optimization problems where one has access only to noise-corrupted values of the objective function. In this paper, we present a novel formulation for noise-tolerant learning of linear classifiers using a CALA team. We consider the general case of nonuniform noise, where the probability that the class label of an example is wrong may be a function of the feature vector of the example. The objective is to learn the underlying separating hyperplane given only such noisy examples. We present an algorithm employing a team of CALA and prove, under some conditions on the class conditional densities, that the algorithm achieves noise-tolerant learning as long as the probability of wrong label for any example is less than 0.5. We also present some empirical results to illustrate the effectiveness of the algorithm.

Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market consisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL techniques have not previously been applied. We consider two representative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pricing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as number of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the partial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory replenishments.

Bioleaching of sulphides under applied potentials

The influence of applied DC potentials on the activity and growth of Thiobacillus ferrooxidans, as well as on the dissolution behaviour of some base metal sulphides is discussed with reference to bioleaching. Selective bioleaching of zinc from sphalerite could be achieved under an applied potential of −500 mV (saturated calomel electrode) from binary mineral mixtures containing the zinc mineral and chalcopyrite or pyrite. On the other hand, bioleaching of pyrite and chalcopyrite was found to be enhanced under positive potentials of +400 mV and +600 mV, respectively. Probable mechanisms in the electrobioleaching of sulphides are examined with respect to galvanic, microbiological and applied potential effects.

Machine learning modelling for predicting soil liquefaction susceptibility

This study describes two machine learning techniques applied to predict liquefaction susceptibility of soil based on the standard penetration test (SPT) data from the 1999 Chi-Chi, Taiwan earthquake. The first machine learning technique which uses Artificial Neural Network (ANN) based on multi-layer perceptions (MLP) that are trained with Levenberg-Marquardt backpropagation algorithm. The second machine learning technique uses the Support Vector machine (SVM) that is firmly based on the theory of statistical learning theory, uses classification technique. ANN and SVM have been developed to predict liquefaction susceptibility using corrected SPT (N-1)(60)] and cyclic stress ratio (CSR). Further, an attempt has been made to simplify the models, requiring only the two parameters (N-1)(60) and peck ground acceleration (a(max)/g)], for the prediction of liquefaction susceptibility. The developed ANN and SVM models have also been applied to different case histories available globally. The paper also highlights the capability of the SVM over the ANN models.

Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Reinforcement learning applications in dynamic pricing of retail markets

In this paper, we investigate the use of reinforcement learning (RL) techniques to the problem of determining dynamic prices in an electronic retail market. As representative models, we consider a single seller market and a two seller market, and formulate the dynamic pricing problem in a setting that easily generalizes to markets with more than two sellers. We first formulate the single seller dynamic pricing problem in the RL framework and solve the problem using the Q-learning algorithm through simulation. Next we model the two seller dynamic pricing problem as a Markovian game and formulate the problem in the RL framework. We solve this problem using actor-critic algorithms through simulation. We believe our approach to solving these problems is a promising way of setting dynamic prices in multi-agent environments. We illustrate the methodology with two illustrative examples of typical retail markets.