N-gram posterior probability confidence measures for statistical machine translation: An empirical study

We report an empirical study of n-gram posterior probability confidence measures for statistical machine translation (SMT). We first describe an efficient and practical algorithm for rapidly computing n-gram posterior probabilities from large translation word lattices. These probabilities are shown to be a good predictor of whether or not the n-gram is found in human reference translations, motivating their use as a confidence measure for SMT. Comprehensive n-gram precision and word coverage measurements are presented for a variety of different language pairs, domains and conditions. We analyze the effect on reference precision of using single or multiple references, and compare the precision of posteriors computed from k-best lists to those computed over the full evidence space of the lattice. We also demonstrate improved confidence by combining multiple lattices in a multi-source translation framework. © 2012 The Author(s).

Using sub-word-level information for confidence estimation with conditional random field models

The task of word-level confidence estimation (CE) for automatic speech recognition (ASR) systems stands to benefit from the combination of suitably defined input features from multiple information sources. However, the information sources of interest may not necessarily operate at the same level of granularity as the underlying ASR system. The research described here builds on previous work on confidence estimation for ASR systems using features extracted from word-level recognition lattices, by incorporating information at the sub-word level. Furthermore, the use of Conditional Random Fields (CRFs) with hidden states is investigated as a technique to combine information for word-level CE. Performance improvements are shown using the sub-word-level information in linear-chain CRFs with appropriately engineered feature functions, as well as when applying the hidden-state CRF model at the word level.

A confidence-based approach for improving keyword hypothesis scores

The task in keyword spotting (KWS) is to hypothesise times at which any of a set of key terms occurs in audio. An important aspect of such systems are the scores assigned to these hypotheses, the accuracy of which have a significant impact on performance. Estimating these scores may be formulated as a confidence estimation problem, where a measure of confidence is assigned to each key term hypothesis. In this work, a set of discriminative features is defined, and combined using a conditional random field (CRF) model for improved confidence estimation. An extension to this model to directly address the problem of score normalisation across key terms is also introduced. The implicit score normalisation which results from applying this approach to separate systems in a hybrid configuration yields further benefits. Results are presented which show notable improvements in KWS performance using the techniques presented in this work. © 2013 IEEE.

Confidence intervals for half-life deviations from purchasing power parity

Existing point estimates of half-life deviations from purchasing power parity (PPP), around 3-5 years, suggest that the speed of convergence is extremely slow. This article assesses the degree of uncertainty around these point estimates by using local-to-unity asymptotic theory to construct confidence intervals that are robust to high persistence in small samples. The empirical evidence suggests that the lower bound of the confidence interval is between four and eight quarters for most currencies, which is not inconsistent with traditional price-stickiness explanations. However, the upper bounds are infinity for all currencies, so we cannot provide conclusive evidence in favor of PPP either. © 2005 American Statistical Association.

Confidence, not consistency, characterizes flashbulb memories.

On September 12, 2001, 54 Duke students recorded their memory of first hearing about the terrorist attacks of September 11 and of a recent everyday event. They were tested again either 1, 6, or 32 weeks later. Consistency for the flashbulb and everyday memories did not differ, in both cases declining over time. However, ratings of vividness, recollection, and belief in the accuracy of memory declined only for everyday memories. Initial visceral emotion ratings correlated with later belief in accuracy, but not consistency, for flashbulb memories. Initial visceral emotion ratings predicted later posttraumatic stress disorder symptoms. Flashbulb memories are not special in their accuracy, as previously claimed, but only in their perceived accuracy.

A numerical approach to C-14 wiggle-match dating of organic deposits: best fits and confidence intervals

14C wiggle-match dating (WMD) of peat deposits uses the non-linear relationship between 14C age and calendar age to match the shape of a sequence of closely spaced peat 14C dates with the 14C calibration curve. A numerical approach to WMD enables the quantitative assessment of various possible wiggle-match solutions and of calendar year confidence intervals for sequences of 14C dates. We assess the assumptions, advantages, and limitations of the method. Several case-studies show that WMD results in more precise chronologies than when individual 14C dates are calibrated. WMD is most successful during periods with major excursions in the 14C calibration curve (e.g., in one case WMD could narrow down confidence intervals from 230 to 36 yr).

UN 2006 Climate Change Conference: a confidence building step?

An analysis of the UN 2006 COP.

Primary science teacher confidence revisited: ten years on

Background