Sequential decision making in general state space models

Many problems in control and signal processing can be formulated as sequential decision problems for general state space models. However, except for some simple models one cannot obtain analytical solutions and has to resort to approximation. In this thesis, we have investigated problems where Sequential Monte Carlo (SMC) methods can be combined with a gradient based search to provide solutions to online optimisation problems. We summarise the main contributions of the thesis as follows. Chapter 4 focuses on solving the sensor scheduling problem when cast as a controlled Hidden Markov Model. We consider the case in which the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. In sensor scheduling, our aim is to minimise the variance of the estimation error of the hidden state with respect to the action sequence. We present a novel SMC method that uses a stochastic gradient algorithm to find optimal actions. This is in contrast to existing works in the literature that only solve approximations to the original problem. In Chapter 5 we presented how an SMC can be used to solve a risk sensitive control problem. We adopt the use of the Feynman-Kac representation of a controlled Markov chain flow and exploit the properties of the logarithmic Lyapunov exponent, which lead to a policy gradient solution for the parameterised problem. The resulting SMC algorithm follows a similar structure with the Recursive Maximum Likelihood(RML) algorithm for online parameter estimation. In Chapters 6, 7 and 8, dynamic Graphical models were combined with with state space models for the purpose of online decentralised inference. We have concentrated more on the distributed parameter estimation problem using two Maximum Likelihood techniques, namely Recursive Maximum Likelihood (RML) and Expectation Maximization (EM). The resulting algorithms can be interpreted as an extension of the Belief Propagation (BP) algorithm to compute likelihood gradients. In order to design an SMC algorithm, in Chapter 8 uses a nonparametric approximations for Belief Propagation. The algorithms were successfully applied to solve the sensor localisation problem for sensor networks of small and medium size.

A differential recursive scheme for suppression of Peak to average power ratio for OFDM

A differential recursive scheme for suppression of Peak to average power ratio (PAPR) for Orthogonal frequency division multiplexing (OFDM) signal is proposed in this thesis. The pseudo-randomized modulating vector for the subcarrier series is differentially phase-encoded between successive components in frequency domain first, and recursion manipulates several samples of Inverse fast Fourier transformation (IFFT) output in time domain. Theoretical analysis and experimental result exhibit advantage of differential recursive scheme over direct output scheme in PAPR suppression. And the overall block diagram of the scheme is also given.

Evaluation of a sequential extraction for the speciation of thorium in soils from Baotou area, Inner Mongolia

Sequential extraction procedures were widely applied for speciation of radioactive elements. In this study, the sequential extraction procedure developed by Martinez-Aguirre was employed for quantification of different chemical forms of thorium in the soil. The total amount of thorium in contaminated soil was much higher by four-fold than the local background value. The soil properties affect the amount of thoriurn and distribution of fractions in contaminated soil. Results showed that the proportion of thorium in soils from Baotou was found as the residual fraction (F5 + F6) > absorbed fraction (F3), coprecipitated fraction (F4) > carbonates fraction (172) and exchangeable fraction (F1) that could be available to plants. The recovery, calculated by ratio of the sum of the six fractions to the pseudo-total content of thorium, was in the range from 96% to 110%. A comparison was carried out between the sequential extraction and the single extraction to evaluate the selectivity of the extractants. It was found that the amount of thorium of absorbed fraction (H) was higher in the single extraction than that estimated in the sequential extraction, possibly duo to transform of the labile form. While for non-residual fraction analysis, the single extraction scheme was a desirable alternative to the sequential extraction procedure.

Regularized iterative and noniterative procedures for object restoration in the presence of noise :an error analysis

An analysis is carried out, using the prolate spheroidal wave functions, of certain regularized iterative and noniterative methods previously proposed for the achievement of object restoration (or, equivalently, spectral extrapolation) from noisy image data. The ill-posedness inherent in the problem is treated by means of a regularization parameter, and the analysis shows explicitly how the deleterious effects of the noise are then contained. The error in the object estimate is also assessed, and it is shown that the optimal choice for the regularization parameter depends on the signal-to-noise ratio. Numerical examples are used to demonstrate the performance of both unregularized and regularized procedures and also to show how, in the unregularized case, artefacts can be generated from pure noise. Finally, the relative error in the estimate is calculated as a function of the degree of superresolution demanded for reconstruction problems characterized by low space–bandwidth products.

Scheduling batches with sequential job processing for two-machine flow and open shops

In this paper, we study a problem of scheduling and batching on two machines in a flow-shop and open-shop environment. Each machine processes operations in batches, and the processing time of a batch is the sum of the processing times of the operations in that batch. A setup time, which depends only on the machine, is required before a batch is processed on a machine, and all jobs in a batch remain at the machine until the entire batch is processed. The aim is to make batching and sequencing decisions, which specify a partition of the jobs into batches on each machine, and a processing order of the batches on each machine, respectively, so that the makespan is minimized. The flow-shop problem is shown to be strongly NP-hard. We demonstrate that there is an optimal solution with the same batches on the two machines; we refer to these as consistent batches. A heuristic is developed that selects the best schedule among several with one, two, or three consistent batches, and is shown to have a worst-case performance ratio of 4/3. For the open-shop, we show that the problem is NP-hard in the ordinary sense. By proving the existence of an optimal solution with one, two or three consistent batches, a close relationship is established with the problem of scheduling two or three identical parallel machines to minimize the makespan. This allows a pseudo-polynomial algorithm to be derived, and various heuristic methods to be suggested.

Potential improvements in the therapeutic ratio of prostate cancer irradiation: dose escalation of pathologically identified tumour nodules using intensity modulated radiotherapy.

The potential of intensity modulated radiotherapy (IMRT) to improve the therapeutic ratio in prostate cancer by dose escalation of intraprostatic tumour nodules (IPTNs) was investigated using a simultaneous integrated boost technique. The prostate and organs-at-risk were outlined on CT images from six prostate cancer patients. Positions of IPTNs were transferred onto the CT images from prostate maps derived from sequential large block sections of whole prostatectomy specimens. Inverse planned IMRT dose distributions were created to irradiate the prostate to 70 Gy and all the IPTNs to 90 Gy. A second plan was produced to escalate only the dominant IPTN (DIPTN) to 90 Gy, mimicking current imaging techniques. These plans were compared with homogeneous prostate irradiation to 70 Gy using dose–volume histograms, tumour control probability (TCP) and normal tissue complication probability (NTCP) for the rectum. The mean dose to IPTNs was increased from 69.8 Gy to 89.1 Gy if all the IPTNs were dose escalated (p=0.0003). This corresponded to a mean increase in TCP of 8.7–31.2% depending on the /ß ratio of prostate cancer (p

Computationally efficient sequential learning algorithms for direct link resource-allocating networks

Computionally efficient sequential learning algorithms are developed for direct-link resource-allocating networks (DRANs). These are achieved by decomposing existing recursive training algorithms on a layer by layer and neuron by neuron basis. This allows network weights to be updated in an efficient parallel manner and facilitates the implementation of minimal update extensions that yield a significant reduction in computation load per iteration compared to existing sequential learning methods employed in resource-allocation network (RAN) and minimal RAN (MRAN) approaches. The new algorithms, which also incorporate a pruning strategy to control network growth, are evaluated on three different system identification benchmark problems and shown to outperform existing methods both in terms of training error convergence and computational efficiency. (c) 2005 Elsevier B.V. All rights reserved.

Sequential versus non-sequential double ionization in strong laser fields

<p> The recollision model has been applied to separate the probability for double ionization into contributions from electron-impact ionization and electron-impact excitation for intensities at which the dielectronic interaction is important for generating double ionization. For a wavelength of 780 am, electron-impact excitation dominates just above the threshold intensity for double ionization, approximate to 1.2 x 10(14) W cm(-2), with electron-impact ionization becoming more important for higher intensities. For a wavelength of 390 nm, the ratio between electron-impact ionization and electron-impact excitation remains fairly constant for all intensities above the threshold intensity for double ionization, approximate to 6 x 10(14) W cm(-2). The results point to an explanation of the experimental results, but more detailed calculations on the behaviour of excited He+ ions are required.</p>

Prospective, Randomized Assessment of Transfer of Training (ToT) and Transfer Effectiveness Ratio (TER) of Virtual Reality Simulation Training for Laparoscopic Skill Acquisition

OBJECTIVES:: We assessed the effectiveness of ToT from VR laparoscopic simulation training in 2 studies. In a second study, we also assessed the TER. ToT is a detectable performance improvement between equivalent groups, and TER is the observed percentage performance differences between 2 matched groups carrying out the same task but with 1 group pretrained on VR simulation. Concordance between simulated and in-vivo procedure performance was also assessed. DESIGN:: Prospective, randomized, and blinded. PARTICIPANTS:: In Study 1, experienced laparoscopic surgeons (n = 195) and in Study 2 laparoscopic novices (n = 30) were randomized to either train on VR simulation before completing an equivalent real-world task or complete the real-world task only. RESULTS:: Experienced laparoscopic surgeons and novices who trained on the simulator performed significantly better than their controls, thus demonstrating ToT. Their performance showed a TER between 7% and 42% from the virtual to the real tasks. Simulation training impacted most on procedural error reduction in both studies (32- 42%). The correlation observed between the VR and real-world task performance was r > 0·96 (Study 2). CONCLUSIONS:: VR simulation training offers a powerful and effective platform for training safer skills.

Field observation of sequestration of uranium in iron-rich sediments under sequential reduction-oxidation conditions at the DOE Oak Ridge IFRC

At the U.S. DOE Oak Ridge Integrated Field Research Challenge (ORIFRC) site, the iron content of shallow subsurface materials (i.e. weathered saprolite) is relatively high (up to 5-6% as w/w), and therefore, the forms of the iron species present plays a critical role in the long-term sequestration of uranium. A long term pilot-scale study of the bioreduction and reoxidation of uranium conducted at the ORIFRC area 3 site, adjacent to the former S-3 disposal ponds (source zone), has provided us with the opportunity to study the impact of iron species on the sequestration of U(VI). The aqueous U(VI) concentrations at the site were decreased to below the EPA MCL through the intermittent injection of ethanol as the electron donor. Previous field tests indicated that both oxygen and nitrate could oxidize the bioreduced U(IV) and cause a short-term rebound of aqueous phase uranium concentration after the oxidative agents were delivered directly to the bioreduced zone.

A field test has been conducted to examine the long-term effect of exposure of bioreduced sediments to nitrate in contaminated groundwater for more than 1,380 days at the Area 3 site. Contaminated groundwater was allowed to invade the previously bioreduced zone via the natural groundwater gradient after an extended period in which reducing conditions were maintained and the bioreduced zone was protected from the influx of upgradient contaminated groundwater. The geochemical response to the invasion of contaminated groundwater was dependent on whether the monitoring location is in the middle or the fringe of the previously bioreduced zone. In general, the nitrate concentrations in the previously bioreduced area, increased gradually from near zero to ~50-300 mM within 200 days and then stabilized. The pH declined from bioreduced levels of 6.2-6.7 to below 5.0. Uranium concentrations rebounded in all monitoring wells but at different rates. At most locations U concentrations rebounded, declined and then rebounded again. Methane gas disappeared while a significant level (20,000 to 44,000 ppmv) N2O was found in the groundwater of monitoring wells after three years of reoxidization.

The U(IV) in sediments was mainly reoxidized to U(VI) species. Based on XANES analysis, the predominate uranium in all samples after re-oxidation was similar to a uranyl nitrate form. But the U content in the sediment remained as high as that determined after bioreduction activates were completed, indicating that much of the U is still sequestrated in situ. SEM observations of surged fine sediments revealed that clusters of colloidal-sized (200-500nm) U-containing precipitates appeared to have formed in situ, regardless from sample of FW106 in non-bioactivity control area or of pre-bioreduced FW101-2 and FW102-3. Additionally, SEM-EDS and microprobe analysis, showed that the U-containing precipitates (~1% U) in FW106 are notably higher in Fe, compared to the precipitates (~1-2.5% U) from FW101-2 and FW102-3. However, XRF analysis indicated that the U content was remained as high as 2180 and 1810 mg/kg with U/Fe ratio at 0.077 and 0.055 vs 0.037 g/g, respectively in pre-bioreduced FW101-2 and FW102-3, suggesting more U sequestrated by Fe in pre-bioreduced sediments.

Design methodology to trade off power, output quality and error resiliency: Application to color interpolation filtering

Power dissipation and tolerance to process variations pose conflicting design requirements. Scaling of voltage is associated with larger variations, while Vdd upscaling or transistor up-sizing for process tolerance can be detrimental for power dissipation. However, for certain signal processing systems such as those used in color image processing, we noted that effective trade-offs can be achieved between Vdd scaling, process tolerance and "output quality". In this paper we demonstrate how these tradeoffs can be effectively utilized in the development of novel low-power variation tolerant architectures for color interpolation. The proposed architecture supports a graceful degradation in the PSNR (Peak Signal to Noise Ratio) under aggressive voltage scaling as well as extreme process variations in. sub-70nm technologies. This is achieved by exploiting the fact that some computations are more important and contribute more to the PSNR improvement compared to the others. The computations are mapped to the hardware in such a way that only the less important computations are affected by Vdd-scaling and process variations. Simulation results show that even at a scaled voltage of 60% of nominal Vdd value, our design provides reasonable image PSNR with 69% power savings.

Handling Sequential Observations in Intelligent Surveillance

Demand for intelligent surveillance in public transport systems is growing due to the increased threats of terrorist attack, vandalism and litigation. The aim of intelligent surveillance is in-time reaction to information received from various monitoring devices, especially CCTV systems. However, video analytic algorithms can only provide static assertions, whilst in reality, many related events happen in sequence and hence should be modeled sequentially. Moreover, analytic algorithms are error-prone, hence how to correct the sequential analytic results based on new evidence (external information or later sensing discovery) becomes an interesting issue. In this paper, we introduce a high-level sequential observation modeling framework which can support revision and update on new evidence. This framework adapts the situation calculus to deal with uncertainty from analytic results. The output of the framework can serve as a foundation for event composition. We demonstrate the significance and usefulness of our framework with a case study of a bus surveillance project.

Heritability of refractive error and familial aggregation of myopia in an elderly American population.

PURPOSE: To determine the heritability of refractive error and the familial aggregation of myopia in an older population. METHODS: Seven hundred fifty-nine siblings (mean age, 73.4 years) in 241 families were recruited from the Salisbury Eye Evaluation (SEE) Study in eastern Maryland. Refractive error was determined by noncycloplegic subjective refraction (if presenting distance visual acuity was < or =20/40) or lensometry (if best corrected visual acuity was >20/40 with spectacles). Participants were considered plano (refractive error of zero) if uncorrected visual acuity was >20/40. Preoperative refraction from medical records was used for pseudophakic subjects. Heritability of refractive error was calculated with multivariate linear regression and was estimated as twice the residual between-sibling correlation after adjusting for age, gender, and race. Logistic regression models were used to estimate the odds ratio (OR) of myopia, given a myopic sibling relative to having a nonmyopic sibling. RESULTS: The estimated heritability of refractive error was 61% (95% confidence interval [CI]: 34%-88%) in this population. The age-, race-, and sex-adjusted ORs of myopia were 2.65 (95% CI: 1.67-4.19), 2.25 (95% CI: 1.31-3.87), 3.00 (95% CI: 1.56-5.79), and 2.98 (95% CI: 1.51-5.87) for myopia thresholds of -0.50, -1.00, -1.50, and -2.00 D, respectively. Neither race nor gender was significantly associated with an increased risk of myopia. CONCLUSIONS: Refractive error and myopia are highly heritable in this elderly population.

Myopia, spectacle wear, and risk of bicycle accidents among rural Chinese secondary school students: the Xichang Pediatric Refractive Error Study report no. 7.

OBJECTIVE: To study the effect of myopia and spectacle wear on bicycle-related injuries in rural Chinese students. Myopia is common among Chinese students but few studies have examined its effect on daily activities. METHODS: Data on visual acuity, refractive error, current spectacle wear, and history of bicycle use and accidents during the past 3 years were sought from 1891 students undergoing eye examinations in rural Guangdong province. RESULTS: Refractive and accident data were available for 1539 participants (81.3%), among whom the mean age was 14.6 years, 52.5% were girls, 26.8% wore glasses, and 12.9% had myopia of less than -4 diopters in both eyes. More than 90% relied on bicycles to get to school daily. A total of 2931 accidents were reported by 423 participants, with 68 requiring medical attention. Male sex (odds ratio, 1.55; P < .001) and spectacle wear (odds ratio, 1.38; P = .04) were associated with a higher risk of accident, but habitual visual acuity and myopia were unassociated with the crash risk, after adjusting for age, sex, time spent riding, and risky riding behaviors. CONCLUSION: These results may be consistent with data on motor vehicle accidents implicating peripheral vision (potentially compromised by spectacle wear) more strongly than central visual acuity in mediating crash risk.

Non-equilibrium modeling of the FE XVII 3C/3D Line Ratio in an Intense X-ray Free-electron Laser Excited Plasma

Recent measurements using an X-ray Free Electron Laser (XFEL) and an Electron Beam Ion Trap at the Linac Coherent Light Source facility highlighted large discrepancies between the observed and theoretical values for the Fe XVII 3C/3D line intensity ratio. This result raised the question of whether the theoretical oscillator strengths may be significantly in error, due to insufficiencies in the atomic structure calculations. We present time-dependent spectral modeling of this experiment and show that non-equilibrium effects can dramatically reduce the predicted 3C/3D line intensity ratio, compared with that obtained by simply taking the ratio of oscillator strengths. Once these non-equilibrium effects are accounted for, the measured line intensity ratio can be used to determine a revised value for the 3C/3D oscillator strength ratio, giving a range from 3.0 to 3.5. We also provide a framework to narrow this range further, if more precise information about the pulse parameters can be determined. We discuss the implications of the new results for the use of Fe XVII spectral features as astrophysical diagnostics and investigate the importance of time-dependent effects in interpreting XFEL-excited plasmas.