An overview of Sequential Monte Carlo methods for parameter estimation in general state-space models

Nonlinear non-Gaussian state-space models arise in numerous applications in control and signal processing. Sequential Monte Carlo (SMC) methods, also known as Particle Filters, provide very good numerical approximations to the associated optimal state estimation problems. However, in many scenarios, the state-space model of interest also depends on unknown static parameters that need to be estimated from the data. In this context, standard SMC methods fail and it is necessary to rely on more sophisticated algorithms. The aim of this paper is to present a comprehensive overview of SMC methods that have been proposed to perform static parameter estimation in general state-space models. We discuss the advantages and limitations of these methods. © 2009 IFAC.

Parameter extraction and calorimetric validation for a silicon carbide jfet pspice model

This paper focuses on the PSpice model of SiC-JFET element inside a SiCED cascode device. The device model parameters are extracted from the I-V and C-V characterization curves. In order to validate the model, an inductive test rig circuit is designed and tested. The switching loss is estimated both using oscilloscope and calorimeter. These results are found to be in good agreement with the simulated results.

Reinforcement learning for parameter estimation in statistical spoken dialogue systems

Reinforcement techniques have been successfully used to maximise the expected cumulative reward of statistical dialogue systems. Typically, reinforcement learning is used to estimate the parameters of a dialogue policy which selects the system's responses based on the inferred dialogue state. However, the inference of the dialogue state itself depends on a dialogue model which describes the expected behaviour of a user when interacting with the system. Ideally the parameters of this dialogue model should be also optimised to maximise the expected cumulative reward. This article presents two novel reinforcement algorithms for learning the parameters of a dialogue model. First, the Natural Belief Critic algorithm is designed to optimise the model parameters while the policy is kept fixed. This algorithm is suitable, for example, in systems using a handcrafted policy, perhaps prescribed by other design considerations. Second, the Natural Actor and Belief Critic algorithm jointly optimises both the model and the policy parameters. The algorithms are evaluated on a statistical dialogue system modelled as a Partially Observable Markov Decision Process in a tourist information domain. The evaluation is performed with a user simulator and with real users. The experiments indicate that model parameters estimated to maximise the expected reward function provide improved performance compared to the baseline handcrafted parameters. © 2011 Elsevier Ltd. All rights reserved.

Material parameter extraction in THz-TDS using a converging beam transfer function

We demonstrate a parameter extraction algorithm based on a theoretical transfer function, which takes into account a converging THz beam. Using this, we successfully extract material parameters from data obtained for a quartz sample with a THz time domain spectrometer. © 2010 IEEE.

Direct parameter extraction in capacitively transduced micromechanical resonators using the anti-resonance

This paper presents a method for the fast and direct extraction of model parameters for capacitive MEMS resonators from their measured transmission response such as quality factor, resonant frequency, and motional resistance. We show that these parameters may be extracted without having to first de-embed the resonator motional current from the feedthrough. The series and parallel resonances from the measured electrical transmission are used to determine the MEMS resonator circuit parameters. The theoretical basis for the method is elucidated by using both the Nyquist and susceptance frequency response plots, and applicable in the limit where CF > CmQ; commonly the case when characterizing MEMS resonators at RF. The method is then applied to the measured electrical transmission for capacitively transduced MEMS resonators, and compared against parameters obtained using a Lorentzian fit to the measured response. Close agreement between the two methods is reported herein. © 2010 IEEE.

Direct parameter extraction in feedthrough-embedded capacitive MEMS resonators

This paper presents a method for fast and accurate determination of parameters relevant to the characterization of capacitive MEMS resonators like quality factor (Q), resonant frequency (fn), and equivalent circuit parameters such as the motional capacitance (Cm). In the presence of a parasitic feedthrough capacitor (CF) appearing across the input and output ports, the transmission characteristic is marked by two resonances: series (S) and parallel (P). Close approximations of these circuit parameters are obtained without having to first de-embed the resonator motional current typically buried in feedthrough by using the series and parallel resonances. While previous methods with the same objective are well known, we show that these are limited to the condition where CF ≪ CmQ. In contrast, this work focuses on moderate capacitive feedthrough levels where CF > CmQ, which are more common in MEMS resonators. The method is applied to data obtained from the measured electrical transmission of fabricated SOI MEMS resonators. Parameter values deduced via direct extraction are then compared against those obtained by a full extraction procedure where de-embedding is first performed and followed by a Lorentzian fit to the data based on the classical transfer function associated with a generic LRC series resonant circuit. © 2011 Elsevier B.V. All rights reserved.

Parameter Estimation for Hidden Markov Models with Intractable Likelihoods

Approximate Bayesian computation (ABC) is a popular technique for analysing data for complex models where the likelihood function is intractable. It involves using simulation from the model to approximate the likelihood, with this approximate likelihood then being used to construct an approximate posterior. In this paper, we consider methods that estimate the parameters by maximizing the approximate likelihood used in ABC. We give a theoretical analysis of the asymptotic properties of the resulting estimator. In particular, we derive results analogous to those of consistency and asymptotic normality for standard maximum likelihood estimation. We also discuss how sequential Monte Carlo methods provide a natural method for implementing our likelihood-based ABC procedures.

Bayesian learning of noisy Markov decision processes

This work addresses the problem of estimating the optimal value function in a Markov Decision Process from observed state-action pairs. We adopt a Bayesian approach to inference, which allows both the model to be estimated and predictions about actions to be made in a unified framework, providing a principled approach to mimicry of a controller on the basis of observed data. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from theposterior distribution over the optimal value function. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

An overview of sequential Monte Carlo methods for parameter estimation in general state-space models

Nonlinear non-Gaussian state-space models arise in numerous applications in control and signal processing. Sequential Monte Carlo (SMC) methods, also known as Particle Filters, are numerical techniques based on Importance Sampling for solving the optimal state estimation problem. The task of calibrating the state-space model is an important problem frequently faced by practitioners and the observed data may be used to estimate the parameters of the model. The aim of this paper is to present a comprehensive overview of SMC methods that have been proposed for this task accompanied with a discussion of their advantages and limitations.

Sequential Monte Carlo computation of the score and observed information matrix in state-space models with application to parameter estimation

Sequential Monte Carlo (SMC) methods are popular computational tools for Bayesian inference in non-linear non-Gaussian state-space models. For this class of models, we propose SMC algorithms to compute the score vector and observed information matrix recursively in time. We propose two different SMC implementations, one with computational complexity $\mathcal{O}(N)$ and the other with complexity $\mathcal{O}(N^{2})$ where $N$ is the number of importance sampling draws. Although cheaper, the performance of the $\mathcal{O}(N)$ method degrades quickly in time as it inherently relies on the SMC approximation of a sequence of probability distributions whose dimension is increasing linearly with time. In particular, even under strong \textit{mixing} assumptions, the variance of the estimates computed with the $\mathcal{O}(N)$ method increases at least quadratically in time. The $\mathcal{O}(N^{2})$ is a non-standard SMC implementation that does not suffer from this rapid degrade. We then show how both methods can be used to perform batch and recursive parameter estimation.