A hierarchical system of learning automata that can learn die globally optimal path

Systems of learning automata have been studied by various researchers to evolve useful strategies for decision making under uncertainity. Considered in this paper are a class of hierarchical systems of learning automata where the system gets responses from its environment at each level of the hierarchy. A classification of such sequential learning tasks based on the complexity of the learning problem is presented. It is shown that none of the existing algorithms can perform in the most general type of hierarchical problem. An algorithm for learning the globally optimal path in this general setting is presented, and its convergence is established. This algorithm needs information transfer from the lower levels to the higher levels. Using the methodology of estimator algorithms, this model can be generalized to accommodate other kinds of hierarchical learning tasks.

On the moderators of trait avoidance motivation in predicting cognitive biases and adjustment

The present research focused on motivational and personality traits measuring individual differences in the experience of negative affect, in reactivity to negative events, and in the tendency to avoid threats. In this thesis, such traits (i.e., neuroticism and dispositional avoidance motivation) are jointly referred to as trait avoidance motivation. The seven studies presented here examined the moderators of such traits in predicting risk judgments, negatively biased processing, and adjustment. Given that trait avoidance motivation encompasses reactivity to negative events and tendency to avoid threats, it can be considered surprising that this trait does not seem to be related to risk judgments and that it seems to be inconsistently related to negatively biased information processing. Previous work thus suggests that some variable(s) moderate these relations. Furthermore, recent research has suggested that despite the close connection between trait avoidance motivation and (mal)adjustment, measures of cognitive performance may moderate this connection. However, it is unclear whether this moderation is due to different response processes between individuals with different cognitive tendencies or abilities, or to the genuinely buffering effect of high cognitive ability against the negative consequences of high trait avoidance motivation. Studies 1-3 showed that there is a modest direct relation between trait avoidance motivation and risk judgments, but studies 2-3 demonstrated that state motivation moderates this relation. In particular, individuals in an avoidance state made high risk judgments regardless of their level of trait avoidance motivation. This result explained the disparity between the theoretical conceptualization of avoidance motivation and the results of previous studies suggesting that the relation between trait avoidance motivation and risk judgments is weak or nonexistent. Studies 5-6 examined threat identification tendency as a moderator for the relationship between trait avoidance motivation and negatively biased processing. However, no evidence for such moderation was found. Furthermore, in line with previous work, the results of studies 5-6 suggested that trait avoidance motivation is inconsistently related to negatively biased processing, implying that theories concerning traits and information processing may need refining. Study 7 examined cognitive ability as a moderator for the relation between trait avoidance motivation and adjustment, and demonstrated that cognitive ability moderates the relation between trait avoidance motivation and indicators of both self-reported and objectively measured adjustment. Thus, the results of Study 7 supported the buffer explanation for the moderating influence of cognitive performance. To summarize, the results showed that it is possible to find factors that consistently moderate the relations between traits and important outcomes (e.g. adjustment). Identifying such factors and studying their interplay with traits is one of the most important goals of current personality research. The present thesis contributed to this line of work in relation to trait avoidance motivation.

A Hierarchical System of Learning Automata

A learning automaton operating in a random environment updates its action probabilities on the basis of the reactions of the environment, so that asymptotically it chooses the optimal action. When the number of actions is large the automaton becomes slow because there are too many updatings to be made at each instant. A hierarchical system of such automata with assured c-optimality is suggested to overcome that problem.The learning algorithm for the hierarchical system turns out to be a simple modification of the absolutely expedient algorithm known in the literature. The parameters of the algorithm at each level in the hierarchy depend only on the parameters and the action probabilities of the previous level. It follows that to minimize the number of updatings per cycle each automaton in the hierarchy need have only two or three actions.

Learning Optimal Discriminant Functions through a Cooperative Game of Automata

The problem of learning correct decision rules to minimize the probability of misclassification is a long-standing problem of supervised learning in pattern recognition. The problem of learning such optimal discriminant functions is considered for the class of problems where the statistical properties of the pattern classes are completely unknown. The problem is posed as a game with common payoff played by a team of mutually cooperating learning automata. This essentially results in a probabilistic search through the space of classifiers. The approach is inherently capable of learning discriminant functions that are nonlinear in their parameters also. A learning algorithm is presented for the team and convergence is established. It is proved that the team can obtain the optimal classifier to an arbitrary approximation. Simulation results with a few examples are presented where the team learns the optimal classifier.

A cooperative game of a pair of learning automata

A cooperative game played in a sequential manner by a pair of learning automata is investigated in this paper. The automata operate in an unknown random environment which gives a common pay-off to the automata. Necessary and sufficient conditions on the functions in the reinforcement scheme are given for absolute monotonicity which enables the expected pay-off to be monotonically increasing in any arbitrary environment. As each participating automaton operates with no information regarding the other partner, the results of the paper are relevant to decentralized control.

Multiaction learning automata possessing ergodicity of the mean

Multiaction learning automata which update their action probabilities on the basis of the responses they get from an environment are considered in this paper. The automata update the probabilities according to whether the environment responds with a reward or a penalty. Learning automata are said to possess ergodicity of the mean if the mean action probability is the state probability (or unconditional probability) of an ergodic Markov chain. In an earlier paper [11] we considered the problem of a two-action learning automaton being ergodic in the mean (EM). The family of such automata was characterized completely by proving the necessary and sufficient conditions for automata to be EM. In this paper, we generalize the results of [11] and obtain necessary and sufficient conditions for the multiaction learning automaton to be EM. These conditions involve two families of probability updating functions. It is shown that for the automaton to be EM the two families must be linearly dependent. The vector defining the linear dependence is the only vector parameter which controls the rate of convergence of the automaton. Further, the technique for reducing the variance of the limiting distribution is discussed. Just as in the two-action case, it is shown that the set of absolutely expedient schemes and the set of schemes which possess ergodicity of the mean are mutually disjoint.

Identification of a class of non-linear systems

This paper considers the on-line identification of a non-linear system in terms of a Hammerstein model, with a zero-memory non-linear gain followed by a linear system. The linear part is represented by a Laguerre expansion of its impulse response and the non-linear part by a polynomial. The identification procedure involves determination of the coefficients of the Laguerre expansion of correlation functions and an iterative adjustment of the parameters of the non-linear gain by gradient methods. The method is applicable to situations involving a wide class of input signals. Even in the presence of additive correlated noise, satisfactory performance is achieved with the variance of the error converging to a value close to the variance of the noise. Digital computer simulation establishes the practicability of the scheme in different situations.

Construction of stability multipliers for non-linear feedback systems

This paper is concerned with the analysis of the absolute stability of a non-linear autonomous system which consists of a single non-linearity belonging to a particular class, in an otherwise linear feedback loop. It is motivated from the earlier Popovlike frequency-domain criteria using the ' multiplier ' eoncept and involves the construction of ' stability multipliers' with prescribed phase characteristics. A few computer-based methods by which this problem can be solved are indicated and it is shown that this constitutes a stop-by-step procedure for testing the stability properties of a given system.

On the positivity and stability of non-stationary systems

The positivity of operators in Hilbert spaces is an important concept finding wide application in various branches of Mathematical System Theory. A frequency- domain condition that ensures the positivity of time-varying operators in L2 with a state-space description, is derived in this paper by using certain newly developed inequalities concerning the input-state relation of such operators. As an interesting application of these results, an L2 stability criterion for time-varying feedback systems consisting of a finite-sector non-linearity is also developed.

Maltose and maltotriose transport into ale and lager brewer's yeast strains

Maltose and maltotriose are the two most abundant sugars in brewer s wort, and thus brewer s yeast s ability to utilize them efficiently is of major importance in the brewing process. The increasing tendency to utilize high and very-high-gravity worts containing increased concentrations of maltose and maltotriose renders the need for efficient transport of these sugars even more pronounced. Residual maltose and especially maltotriose are quite often present especially after high and very-high-gravity fermentations. Sugar uptake capacity has been shown to be the rate limiting factor for maltose and maltotriose utilization. The main aim of the present study was to find novel ways to improve maltose and maltotriose utilization during the main fermentation. Maltose and maltotriose uptake characteristics of several ale and lager strains were studied. Genotype determination of the genes needed for maltose and maltotriose utilization was performed. Maltose uptake inhibition studies were performed to reveal the dominant transporter types actually functioning in each of the strains. Temperature-dependence of maltose transport was studied for ale and for lager strains as well as for each of the single sugar transporter proteins Agt1p, Malx1p and Mtt1p. The AGT1 promoter regions of one ale and two lager strains were sequenced by chromosome walking and the promoter elements were searched for using computational methods. The results showed that ale and lager strains predominantly use different maltose and maltotriose transporter types for maltose and maltotriose uptake. Agt1 transporter was found to be the dominant maltose/maltotriose transporter in the ale strains whereas Malx1 and Mtt1- type transporters dominated in the lager strains. All lager strains studied were found to possess a non-functional Agt1 transporter. The ale strains were observed to be more sensitive to temperature decrease in their maltose uptake compared to the lager strains. Single transporters were observed to differ in their sensitivity to temperature decrease and their temperature-dependence was shown to decrease in the order Agt1≥Malx1>Mtt1. The different temperature-dependence between the ale and lager strains was observed to be due to the different dominant maltose/maltotriose transporters ale and lager strains possessed. The AGT1 promoter regions of ale and lager strains were found to differ markedly from the corresponding regions of laboratory strains. The ale strain was found to possess an extra MAL-activator binding site compared to the lager strains. Improved maltose and maltotriose uptake capacity was obtained with a modified lager strain where the AGT1 gene was repaired and put under the control of a strong promoter. Modified strains fermented wort faster and more completely, producing beers containing more ethanol and less residual maltose and maltotriose. Significant savings in the main fermentation time were obtained when modified strains were used. In high-gravity wort fermentations 8 20% and in very-high-gravity wort fermentations even 11 37% time savings were obtained. These are economically significant changes and would cause a marked increase in annual output from the same-size of brewhouse and fermentor facilities.

Stability of systems with power-law non-linearities

It is shown that a sufficient condition for the asymptotic stability-in-the-large of an autonomous system containing a linear part with transfer function G(jω) and a non-linearity belonging to a class of power-law non-linearities with slope restriction [0, K] in cascade in a negative feedback loop is ReZ(jω)[G(jω) + 1 K] ≥ 0 for all ω where the multiplier is given by, Z(jω) = 1 + αjω + Y(jω) - Y(-jω) with a real, y(t) = 0 for t < 0 and ∫ 0 ∞ |y(t)|dt < 1 2c2, c2 being a constant associated with the class of non-linearity. Any allowable multiplier can be converted to the above form and this form leads to lesser restrictions on the parameters in many cases. Criteria for the case of odd monotonic non-linearities and of linear gains are obtained as limiting cases of the criterion developed. A striking feature of the present result is that in the linear case it reduces to the necessary and sufficient conditions corresponding to the Nyquist criterion. An inequality of the type |R(T) - R(- T)| ≤ 2c2R(0) where R(T) is the input-output cross-correlation function of the non-linearity, is used in deriving the results.

Optimal non-linear reinforcement schemes for stochastic automata

Two optimal non-linear reinforcement schemes—the Reward-Inaction and the Penalty-Inaction—for the two-state automaton functioning in a stationary random environment are considered. Very simple conditions of symmetry of the non-linear function figuring in the reinforcement scheme are shown to be necessary and sufficient for optimality. General expressions for the variance and rate of learning are derived. These schemes are compared with the already existing optimal linear schemes in the light of average variance and average rate of learning.

Improved Conditions For L2-Stability Of Nonstationary Feedback-Systems

Concerning the L2-stability of feedback systems containing a linear time-varying operator, some of the stringent restrictions imposed on the multiplier as well as the linear part of the system, in the criteria presented earlier, are relaxed.

L2-Stability Of A Class Of Nonlinear-Systems

Sufficient conditions are given for the L2-stability of a class of feedback systems consisting of a linear operator G and a nonlinear gain function, either odd monotone or restricted by a power-law, in cascade, in a negative feedback loop. The criterion takes the form of a frequency-domain inequality, Re[1 + Z(jω)] G(jω) δ > 0 ω ε (−∞, +∞), where Z(jω) is given by, Z(jω) = β[Y1(jω) + Y2(jω)] + (1 − β)[Y3(jω) − Y3(−jω)], with 0 β 1 and the functions y1(·), y2(·) and y3(·) satisfying the time-domain inequalities, ∝−∞+∞¦y1(t) + y2(t)¦ dt 1 − ε, y1(·) = 0, t < 0, y2(·) = 0, t > 0 and ε > 0, and , c2 being a constant depending on the order of the power-law restricting the nonlinear function. The criterion is derived using Zames' passive operator theory and is shown to be more general than the existing criteria