On probability-based inference under data missing by design

Whether a statistician wants to complement a probability model for observed data with a prior distribution and carry out fully probabilistic inference, or base the inference only on the likelihood function, may be a fundamental question in theory, but in practice it may well be of less importance if the likelihood contains much more information than the prior. Maximum likelihood inference can be justified as a Gaussian approximation at the posterior mode, using flat priors. However, in situations where parametric assumptions in standard statistical models would be too rigid, more flexible model formulation, combined with fully probabilistic inference, can be achieved using hierarchical Bayesian parametrization. This work includes five articles, all of which apply probability modeling under various problems involving incomplete observation. Three of the papers apply maximum likelihood estimation and two of them hierarchical Bayesian modeling. Because maximum likelihood may be presented as a special case of Bayesian inference, but not the other way round, in the introductory part of this work we present a framework for probability-based inference using only Bayesian concepts. We also re-derive some results presented in the original articles using the toolbox equipped herein, to show that they are also justifiable under this more general framework. Here the assumption of exchangeability and de Finetti's representation theorem are applied repeatedly for justifying the use of standard parametric probability models with conditionally independent likelihood contributions. It is argued that this same reasoning can be applied also under sampling from a finite population. The main emphasis here is in probability-based inference under incomplete observation due to study design. This is illustrated using a generic two-phase cohort sampling design as an example. The alternative approaches presented for analysis of such a design are full likelihood, which utilizes all observed information, and conditional likelihood, which is restricted to a completely observed set, conditioning on the rule that generated that set. Conditional likelihood inference is also applied for a joint analysis of prevalence and incidence data, a situation subject to both left censoring and left truncation. Other topics covered are model uncertainty and causal inference using posterior predictive distributions. We formulate a non-parametric monotonic regression model for one or more covariates and a Bayesian estimation procedure, and apply the model in the context of optimal sequential treatment regimes, demonstrating that inference based on posterior predictive distributions is feasible also in this case.

Particle dynamics in a turbulent particle–gas suspension at high Stokes number. Part 1. Velocity and acceleration distributions

The effect of fluid velocity fluctuations on the dynamics of the particles in a turbulent gas–solid suspension is analysed in the low-Reynolds-number and high Stokes number limits, where the particle relaxation time is long compared with the correlation time for the fluid velocity fluctuations, and the drag force on the particles due to the fluid can be expressed by the modified Stokes law. The direct numerical simulation procedure is used for solving the Navier–Stokes equations for the fluid, the particles are modelled as hard spheres which undergo elastic collisions and a one-way coupling algorithm is used where the force exerted by the fluid on the particles is incorporated, but not the reverse force exerted by the particles on the fluid. The particle mean and root-mean-square (RMS) fluctuating velocities, as well as the probability distribution function for the particle velocity fluctuations and the distribution of acceleration of the particles in the central region of the Couette (where the velocity profile is linear and the RMS velocities are nearly constant), are examined. It is found that the distribution of particle velocities is very different from a Gaussian, especially in the spanwise and wall-normal directions. However, the distribution of the acceleration fluctuation on the particles is found to be close to a Gaussian, though the distribution is highly anisotropic and there is a correlation between the fluctuations in the flow and gradient directions. The non-Gaussian nature of the particle velocity fluctuations is found to be due to inter-particle collisions induced by the large particle velocity fluctuations in the flow direction. It is also found that the acceleration distribution on the particles is in very good agreement with the distribution that is calculated from the velocity fluctuations in the fluid, using the Stokes drag law, indicating that there is very little correlation between the fluid velocity fluctuations and the particle velocity fluctuations in the presence of one-way coupling. All of these results indicate that the effect of the turbulent fluid velocity fluctuations can be accurately represented by an anisotropic Gaussian white noise.

AN OPTIMUM SHRINKAGE ESTIMATOR BASED ON MINIMUM-PROBABILITY-OF-ERROR CRITERION AND APPLICATION TO SIGNAL DENOISING

We address the problem of designing an optimal pointwise shrinkage estimator in the transform domain, based on the minimum probability of error (MPE) criterion. We assume an additive model for the noise corrupting the clean signal. The proposed formulation is general in the sense that it can handle various noise distributions. We consider various noise distributions (Gaussian, Student's-t, and Laplacian) and compare the denoising performance of the estimator obtained with the mean-squared error (MSE)-based estimators. The MSE optimization is carried out using an unbiased estimator of the MSE, namely Stein's Unbiased Risk Estimate (SURE). Experimental results show that the MPE estimator outperforms the SURE estimator in terms of SNR of the denoised output, for low (0 -10 dB) and medium values (10 - 20 dB) of the input SNR.

Descending from infinity: Convergence of tailed distributions

We investigate the relaxation of long-tailed distributions under stochastic dynamics that do not support such tails. Linear relaxation is found to be a borderline case in which long tails are exponentially suppressed in time but not eliminated. Relaxation stronger than linear suppresses long tails immediately, but may lead to strong transient peaks in the probability distribution. We also find that a delta-function initial distribution under stronger than linear decay displays not one but two different regimes of diffusive spreading.

Reconstructing The Vertical Distribution Of The Aeolian Saltation Mass Flux Based On The Probability Distribution Of Lift-Off Velocity

The probability distribution of lift-off velocity of the saltating grains is a bridge to linking microscopic and macroscopic research of aeolian sand transport. The lift-off parameters of saltating grains (i.e., the horizontal and vertical lift-off velocities, resultant lift-off velocity, and lift-off angle) in a wind tunnel are measured by using a Phase Doppler Particle Analyzer (PDPA). The experimental results show that the probability distribution of horizontal lift-off velocity of saltating particles on a bed surface is a normal function, and that of vertical lift-off velocity is an exponential function. The probability distribution of resultant lift-off velocity of saltating grains can be expressed as a log-normal function, and that of lift-off angle complies with an exponential function. A numerical model for the vertical distribution of aeolian mass flux based on the probability distribution of lift-off velocity is established. The simulation gives a sand mass flux distribution which is consistent with the field data of Namikas (Namikas, S.L., 2003. Field measurement and numerical modelling of acolian mass flux distributions on a sandy beach, Sedimentology 50, 303-326). Therefore, these findings are helpful to further understand the probability characteristics of lift-off grains in aeolian sand transport. (c) 2007 Elsevier B.V. All rights reserved.

Stationary absolute distributions for chains of infinite order

Let {Ƶ_n}^∞_{n = -∞} be a stochastic process with state space S₁ = {0, 1, …, D – 1}. Such a process is called a chain of infinite order. The transitions of the chain are described by the functions

Q_i(i⁽⁰⁾) = Ƥ(Ƶ_n = i | Ƶ_{n - 1} = i ⁽⁰⁾₁, Ƶ_{n - 2} = i ⁽⁰⁾₂, …) (i ɛ S₁), where i⁽⁰⁾ = (i⁽⁰⁾₁, i⁽⁰⁾₂, …) ranges over infinite sequences from S₁. If i⁽ⁿ⁾ = (i⁽ⁿ⁾₁, i⁽ⁿ⁾₂, …) for n = 1, 2,…, then i⁽ⁿ⁾ → i⁽⁰⁾ means that for each k, i⁽ⁿ⁾_k = i⁽⁰⁾_k for all n sufficiently large.

Given functions Q_i(i⁽⁰⁾) such that

(i) 0 ≤ Q_i(i⁽⁰) ≤ ξ ˂ 1

(ii)D – 1/Ʃ/i = 0 Q_i(i⁽⁰⁾) Ξ 1

(iii) Q_i(i⁽ⁿ⁾) → Q_i(i⁽⁰⁾) whenever i⁽ⁿ⁾ → i⁽⁰⁾,

we prove the existence of a stationary chain of infinite order {Ƶ_n} whose transitions are given by

Ƥ (Ƶ_n = i | Ƶ_{n - 1}, Ƶ_{n - 2}, …) = Q_i(Ƶ_{n - 1}, Ƶ_{n - 2}, …)

With probability 1. The method also yields stationary chains {Ƶ_n} for which (iii) does not hold but whose transition probabilities are, in a sense, “locally Markovian.” These and similar results extend a paper by T.E. Harris [Pac. J. Math., 5 (1955), 707-724].

Included is a new proof of the existence and uniqueness of a stationary absolute distribution for an Nth order Markov chain in which all transitions are possible. This proof allows us to achieve our main results without the use of limit theorem techniques.

Parameterizing probabilities for estimating age-composition distributions for mixture models

When estimating parameters that constitute a discrete probability distribution {pj}, it is difficult to determine how constraints should be made to guarantee that the estimated parameters { pˆj} constitute a probability distribution (i.e., pˆj>0, Σ pˆj =1). For age distributions estimated from mixtures of length-at-age distributions, the EM (expectationmaximization) algorithm (Hasselblad, 1966; Hoenig and Heisey, 1987; Kimura and Chikuni, 1987), restricted least squares (Clark, 1981), and weak quasisolutions (Troynikov, 2004) have all been used. Each of these methods appears to guarantee that the estimated distribution will be a true probability distribution with all categories greater than or equal to zero and with individual probabilities that sum to one. In addition, all these methods appear to provide a theoretical basis for solutions that will be either maximum-likelihood estimates or at least convergent to a probability distribut

Achieving Csiszár's source-channel coding exponent with product distributions

We derive a random-coding upper bound on the average probability of error of joint source-channel coding that recovers Csiszár's error exponent when used with product distributions over the channel inputs. Our proof technique for the error probability analysis employs a code construction for which source messages are assigned to subsets and codewords are generated with a distribution that depends on the subset. © 2012 IEEE.

Probability-Matching Predictors for Extreme Extremes

A location- and scale-invariant predictor is constructed which exhibits good probability matching for extreme predictions outside the span of data drawn from a variety of (stationary) general distributions. It is constructed via the three-parameter {\mu, \sigma, \xi} Generalized Pareto Distribution (GPD). The predictor is designed to provide matching probability exactly for the GPD in both the extreme heavy-tailed limit and the extreme bounded-tail limit, whilst giving a good approximation to probability matching at all intermediate values of the tail parameter \xi. The predictor is valid even for small sample sizes N, even as small as N = 3. The main purpose of this paper is to present the somewhat lengthy derivations which draw heavily on the theory of hypergeometric functions, particularly the Lauricella functions. Whilst the construction is inspired by the Bayesian approach to the prediction problem, it considers the case of vague prior information about both parameters and model, and all derivations are undertaken using sampling theory.

The effects of sediment transport on grain-size distributions

Changes in statistics (mean, sorting, and skewness) describing grain-size distributions have long been used to speculate on the direction of sediment transport. We present a simple model whereby the distributions of sediment in transport are related to their source by a sediment transfer function which defines the relative probability that a grain within each particular class interval will be eroded and transported. A variety of empirically derived transfer functions exhibit negatively skewed distributions (on a phi scale). Thus, when a sediment is being eroded, the probability of any grain going into transport increases with diminishing grain size throughout more than half of its size range. This causes the sediment in transport to be finer and more negatively skewed than its source, whereas the remaining sediment (a lag) must become relatively coarser and more positively skewed. Flume experiments show that the distributions of transfer functions change from having a highly negative skewness to being nearly symmetrical (although still negatively skewed) as the energy of the transporting process increases. We call the two extremes low-energy and high-energy transfer functions , respectively. In an expanded sediment-transport model, successive deposits in the direction of transport are related by a combination of two transfer functions. If energy is decreasing and the transfer functions have low-energy distributions, successive deposits will become finer and more negatively skewed. If, however, energy is decreasing, but the initial transfer function has a high-energy distribution, successive deposits will become coarser and more positively skewed. The variance of the distributions of lags, sediment in transport, and successive deposits in the down-current direction must eventually decrease (i.e., the sediments will become better sorted). We demonstrate that it is possible for variance first to increase, but suggest that, in reality, an increasing variance in the direction of transport will seldom be observed, particularly when grain-size distributions are described in phi units. This model describing changes in sediment distributions was tested in a variety of environments where the transport direction was known. The results indicate that the model has real-world validity and can provide a method to predict the directions of sediment transport

Probability distribution of random wave-current forces

Based on the second-order solutions obtained for the three-dimensional weakly nonlinear random waves propagating over a steady uniform current in finite water depth, the joint statistical distribution of the velocity and acceleration of the fluid particle in the current direction is derived using the characteristic function expansion method. From the joint distribution and the Morison equation, the theoretical distributions of drag forces, inertia forces and total random forces caused by waves propagating over a steady uniform current are determined. The distribution of inertia forces is Gaussian as that derived using the linear wave model, whereas the distributions of drag forces and total random forces deviate slightly from those derived utilizing the linear wave model. The distributions presented can be determined by the wave number spectrum of ocean waves, current speed and the second order wave-wave and wave-current interactions. As an illustrative example, for fully developed deep ocean waves, the parameters appeared in the distributions near still water level are calculated for various wind speeds and current speeds by using Donelan-Pierson-Banner spectrum and the effects of the current and the nonlinearity of ocean waves on the distribution are studied. (c) 2006 Elsevier Ltd. All rights reserved.

Financial modelling with 2-EPT probability density functions

The class of all Exponential-Polynomial-Trigonometric (EPT) functions is classical and equal to the Euler-d’Alembert class of solutions of linear differential equations with constant coefficients. The class of non-negative EPT functions defined on [0;1) was discussed in Hanzon and Holland (2010) of which EPT probability density functions are an important subclass. EPT functions can be represented as ceAxb, where A is a square matrix, b a column vector and c a row vector where the triple (A; b; c) is the minimal realization of the EPT function. The minimal triple is only unique up to a basis transformation. Here the class of 2-EPT probability density functions on R is defined and shown to be closed under a variety of operations. The class is also generalised to include mixtures with the pointmass at zero. This class coincides with the class of probability density functions with rational characteristic functions. It is illustrated that the Variance Gamma density is a 2-EPT density under a parameter restriction. A discrete 2-EPT process is a process which has stochastically independent 2-EPT random variables as increments. It is shown that the distribution of the minimum and maximum of such a process is an EPT density mixed with a pointmass at zero. The Laplace Transform of these distributions correspond to the discrete time Wiener-Hopf factors of the discrete time 2-EPT process. A distribution of daily log-returns, observed over the period 1931-2011 from a prominent US index, is approximated with a 2-EPT density function. Without the non-negativity condition, it is illustrated how this problem is transformed into a discrete time rational approximation problem. The rational approximation software RARL2 is used to carry out this approximation. The non-negativity constraint is then imposed via a convex optimisation procedure after the unconstrained approximation. Sufficient and necessary conditions are derived to characterise infinitely divisible EPT and 2-EPT functions. Infinitely divisible 2-EPT density functions generate 2-EPT Lévy processes. An assets log returns can be modelled as a 2-EPT Lévy process. Closed form pricing formulae are then derived for European Options with specific times to maturity. Formulae for discretely monitored Lookback Options and 2-Period Bermudan Options are also provided. Certain Greeks, including Delta and Gamma, of these options are also computed analytically. MATLAB scripts are provided for calculations involving 2-EPT functions. Numerical option pricing examples illustrate the effectiveness of the 2-EPT approach to financial modelling.

An investigation on the use of SNR distributions for the optimisation of coarse-fine spectrum sensing for cognitive radio

This thesis investigates the optimisation of Coarse-Fine (CF) spectrum sensing architectures under a distribution of SNRs for Dynamic Spectrum Access (DSA). Three different detector architectures are investigated: the Coarse-Sorting Fine Detector (CSFD), the Coarse-Deciding Fine Detector (CDFD) and the Hybrid Coarse-Fine Detector (HCFD). To date, the majority of the work on coarse-fine spectrum sensing for cognitive radio has focused on a single value for the SNR. This approach overlooks the key advantage that CF sensing has to offer, namely that high powered signals can be easily detected without extra signal processing. By considering a range of SNR values, the detector can be optimised more effectively and greater performance gains realised. This work considers the optimisation of CF spectrum sensing schemes where the security and performance are treated separately. Instead of optimising system performance at a single, constant, low SNR value, the system instead is optimised for the average operating conditions. The security is still provided such that at the low SNR values the safety specifications are met. By decoupling the security and performance, the system’s average performance increases whilst maintaining the protection of licensed users from harmful interference. The different architectures considered in this thesis are investigated in theory, simulation and physical implementation to provide a complete overview of the performance of each system. This thesis provides a method for estimating SNR distributions which is quick, accurate and relatively low cost. The CSFD is modelled and the characteristic equations are found for the CDFD scheme. The HCFD is introduced and optimisation schemes for all three architectures are proposed. Finally, using the Implementing Radio In Software (IRIS) test-bed to confirm simulation results, CF spectrum sensing is shown to be significantly quicker than naive methods, whilst still meeting the required interference probability rates and not requiring substantial receiver complexity increases.

Effect of different jump distributions on the dynamics of jump processes

The paper investigates stochastic processes forced by independent and identically distributed jumps occurring according to a Poisson process. The impact of different distributions of the jump amplitudes are analyzed for processes with linear drift. Exact expressions of the probability density functions are derived when jump amplitudes are distributed as exponential, gamma, and mixture of exponential distributions for both natural and reflecting boundary conditions. The mean level-crossing properties are studied in relation to the different jump amplitudes. As an example of application of the previous theoretical derivations, the role of different rainfall-depth distributions on an existing stochastic soil water balance model is analyzed. It is shown how the shape of distribution of daily rainfall depths plays a more relevant role on the soil moisture probability distribution as the rainfall frequency decreases, as predicted by future climatic scenarios. © 2010 The American Physical Society.

Joint distributions of waves and rain

The transfer of gases between the atmosphere and ocean is affected by a number of processes, of which wave action and rainfall are two of potential significance. Efforts have been made to quantify separately their contributions; however such assessments neglect the interaction of these phenomena. Here we look at the correlation statistics of waves and rain to note which regions display a strong association between rainfall and the local sea state. The conditional probability of rain varies from ~0.5% to ~15%, with most of the equatorial belt (which contains the ITCZ) showing a greater likelihood of rain at the lowest sea states. In contrast the occurrence of rain is independent of wave height in the Southern Ocean. The 1997/98 El Niño enhances the frequency of rain in some Pacific regions, with this change showing some association with wave conditions.