Optimal maintenance and replacement via a semi-Markov decision model

Optimal bang-coast maintenance policies for a machine, subject to failure, are considered. The approach utilizes a semi-Markov model for the system. A simplified model for modifying the probability of machine failure with maintenance is employed. A numerical example is presented to illustrate the procedure and results.

A Markov Decision Theoretic Approach to Pilot Allocation and Receive Antenna Selection

This paper considers antenna selection (AS) at a receiver equipped with multiple antenna elements but only a single radio frequency chain for packet reception. As information about the channel state is acquired using training symbols (pilots), the receiver makes its AS decisions based on noisy channel estimates. Additional information that can be exploited for AS includes the time-correlation of the wireless channel and the results of the link-layer error checks upon receiving the data packets. In this scenario, the task of the receiver is to sequentially select (a) the pilot symbol allocation, i.e., how to distribute the available pilot symbols among the antenna elements, for channel estimation on each of the receive antennas; and (b) the antenna to be used for data packet reception. The goal is to maximize the expected throughput, based on the past history of allocation and selection decisions, and the corresponding noisy channel estimates and error check results. Since the channel state is only partially observed through the noisy pilots and the error checks, the joint problem of pilot allocation and AS is modeled as a partially observed Markov decision process (POMDP). The solution to the POMDP yields the policy that maximizes the long-term expected throughput. Using the Finite State Markov Chain (FSMC) model for the wireless channel, the performance of the POMDP solution is compared with that of other existing schemes, and it is illustrated through numerical evaluation that the POMDP solution significantly outperforms them.

Paradoxical evidence integration in rapid decision processes

Decisions about noisy stimuli require evidence integration over time. Traditionally, evidence integration and decision making are described as a one-stage process: a decision is made when evidence for the presence of a stimulus crosses a threshold. Here, we show that one-stage models cannot explain psychophysical experiments on feature fusion, where two visual stimuli are presented in rapid succession. Paradoxically, the second stimulus biases decisions more strongly than the first one, contrary to predictions of one-stage models and intuition. We present a two-stage model where sensory information is integrated and buffered before it is fed into a drift diffusion process. The model is tested in a series of psychophysical experiments and explains both accuracy and reaction time distributions. © 2012 Rüter et al.

Response decision processes and externalizing behavior problems in adolescents.

Externalizing behavior problems of 124 adolescents were assessed across Grades 7-11. In Grade 9, participants were also assessed across social-cognitive domains after imagining themselves as the object of provocations portrayed in six videotaped vignettes. Participants responded to vignette-based questions representing multiple processes of the response decision step of social information processing. Phase 1 of our investigation supported a two-factor model of the response evaluation process of response decision (response valuation and outcome expectancy). Phase 2 showed significant relations between the set of these response decision processes, as well as response selection, measured in Grade 9 and (a) externalizing behavior in Grade 9 and (b) externalizing behavior in Grades 10-11, even after controlling externalizing behavior in Grades 7-8. These findings suggest that on-line behavioral judgments about aggression play a crucial role in the maintenance and growth of aggressive response tendencies in adolescence.

Existence, recurrence and equilibrium properties of Markov branching processes with instantaneous immigration

Attention has recently focussed on stochastic population processes that can undergo total annihilation followed by immigration into state j at rate αj. The investigation of such models, called Markov branching processes with instantaneous immigration (MBPII), involves the study of existence and recurrence properties. However, results developed to date are generally opaque, and so the primary motivation of this paper is to construct conditions that are far easier to apply in practice. These turn out to be identical to the conditions for positive recurrence, which are very easy to check. We obtain, as a consequence, the surprising result that any MBPII that exists is ergodic, and so must possess an equilibrium distribution. These results are then extended to more general MBPII, and we show how to construct the associated equilibrium distributions.

Generalized Markov branching processes

A generalized Markov Brnching Process (GMBP) is a Markov branching model where the infinitesimal branching rates are modified with an interaction index. It is proved that there always exists only one GMBP. An associated differential-integral equation is derived. The extinction probalility and the mean and conditional mean extinction times are obtained. Ergodicity and stability of GMBP with resurrection are also considered. Easy checking criteria are established for ordinary and strong ergodicty. The equilibrium distribution is given in an elegant closed form. The probability meaning of our results is clear and thus explained.

Uniqueness and extinction properties of generalised Markov branching processes

This paper focuses on the basic problems regarding uniqueness and extinction properties for generalised Markov branching processes. The uniqueness criterion is firstly established and a differential–integral equation satisfied by the transition functions of such processes is derived. The extinction probability is then obtained. A closed form is presented for both the mean extinction time and the conditional mean extinction time. It turns out that these important quantities are closely related to the elementary gamma function.

Ergodicity and stability of generalised Markov branching processes with resurrection

This paper concentrates on investigating ergodicity and stability for generalised Markov branching processes with resurrection. Easy checking criteria including several clear-cut corollaries are established for ordinary and strong ergodicity of such processes. The equilibrium distribution is given in an elegant closed form for the ergodic case. The probabilistic interpretation of the results is clear and thus explained.

General Harris regularity criterion for non-linear Markov branching processes

We extend the Harris regularity condition for ordinary Markov branching process to a more general case of non-linear Markov branching process. A regularity criterion which is very easy to check is obtained. In particular, we prove that a super-linear Markov branching process is regular if and only if the per capita offspring mean is less than or equal to I while a sub-linear Markov branching process is regular if the per capita offspring mean is finite. The Harris regularity condition then becomes a special case of our criterion.

Probabilistic approach in weighted Markov branching processes

This note provides a new probabilistic approach in discussing the weighted Markov branching process (WMBP) which is a natural generalisation of the ordinary Markov branching process. Using this approach, some important characteristics regarding the hitting times of such processes can be easily obtained. In particular, the closed forms for the mean extinction time and conditional mean extinction time are presented. The explosion behaviour of the process is investigated and the mean explosion time is derived. The mean global holding time and the mean total survival time are also obtained. The close link between these newly developed processes and the well-known compound Poisson processes is investigated. It is revealed that any weighted Markov branching process (WMBP) is a random time change of a compound Poisson process.