An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Speeded up Gaussian mixture model algorithm for background subtraction

Adaptive Gaussian Mixture Models (GMM) have been one of the most popular and successful approaches to perform foreground segmentation on multimodal background scenes. However, the good accuracy of the GMM algorithm comes at a high computational cost. An improved GMM technique was proposed by Zivkovic to reduce computational cost by minimizing the number of modes adaptively. In this paper, we propose a modification to his adaptive GMM algorithm that further reduces execution time by replacing expensive floating point computations with low cost integer operations. To maintain accuracy, we derive a heuristic that computes periodic floating point updates for the GMM weight parameter using the value of an integer counter. Experiments show speedups in the range of 1.33 - 1.44 on standard video datasets where a large fraction of pixels are multimodal.

Improving tangential resolution with a modified delay-and-sum reconstruction algorithm in photoacoustic and thermoacoustic tomography

Spatial resolution in photoacoustic and thermoacoustic tomography is ultrasound transducer (detector) bandwidth limited. For a circular scanning geometry the axial (radial) resolution is not affected by the detector aperture, but the tangential (lateral) resolution is highly dependent on the aperture size, and it is also spatially varying (depending on the location relative to the scanning center). Several approaches have been reported to counter this problem by physically attaching a negative acoustic lens in front of the nonfocused transducer or by using virtual point detectors. Here, we have implemented a modified delay-and-sum reconstruction method, which takes into account the large aperture of the detector, leading to more than fivefold improvement in the tangential resolution in photoacoustic (and thermoacoustic) tomography. Three different types of numerical phantoms were used to validate our reconstruction method. It is also shown that we were able to preserve the shape of the reconstructed objects with the modified algorithm. (C) 2014 Optical Society of America

Robust design of multiple trailing edge flaps for helicopter vibration reduction: A multi-objective bat algorithm approach

The objective of this study is to determine an optimal trailing edge flap configuration and flap location to achieve minimum hub vibration levels and flap actuation power simultaneously. An aeroelastic analysis of a soft in-plane four-bladed rotor is performed in conjunction with optimal control. A second-order polynomial response surface based on an orthogonal array (OA) with 3-level design describes both the objectives adequately. Two new orthogonal arrays called MGB2P-OA and MGB4P-OA are proposed to generate nonlinear response surfaces with all interaction terms for two and four parameters, respectively. A multi-objective bat algorithm (MOBA) approach is used to obtain the optimal design point for the mutually conflicting objectives. MOBA is a recently developed nature-inspired metaheuristic optimization algorithm that is based on the echolocation behaviour of bats. It is found that MOBA inspired Pareto optimal trailing edge flap design reduces vibration levels by 73% and flap actuation power by 27% in comparison with the baseline design.

Algorithm for Autonomous Reorganization of Mobile Wireless Camera Sensor Networks to Improve Coverage

In this paper, sensing coverage by wireless camera-embedded sensor networks (WCSNs), a class of directional sensors is studied. The proposed work facilitates the autonomous tuning of orientation parameters and displacement of camera-sensor nodes in the bounded field of interest (FoI), where the network coverage in terms of every point in the FoI is important. The proposed work is first of its kind to study the problem of maximizing coverage of randomly deployed mobile WCSNs which exploits their mobility. We propose an algorithm uncovered region exploration algorithm (UREA-CS) that can be executed in centralized and distributed modes. Further, the work is extended for two special scenarios: 1) to suit autonomous combing operations after initial random WCSN deployments and 2) to improve the network coverage with occlusions in the FoI. The extensive simulation results show that the performance of UREA-CS is consistent, robust, and versatile to achieve maximum coverage, both in centralized and distributed modes. The centralized and distributed modes are further analyzed with respect to the computational and communicational overheads.

Spectral Clustering with Jensen-type kernels and their multi-point extensions

Motivated by multi-distribution divergences, which originate in information theory, we propose a notion of `multipoint' kernels, and study their applications. We study a class of kernels based on Jensen type divergences and show that these can be extended to measure similarity among multiple points. We study tensor flattening methods and develop a multi-point (kernel) spectral clustering (MSC) method. We further emphasize on a special case of the proposed kernels, which is a multi-point extension of the linear (dot-product) kernel and show the existence of cubic time tensor flattening algorithm in this case. Finally, we illustrate the usefulness of our contributions using standard data sets and image segmentation tasks.

Computer generated hologram from point cloud using graphics processor

Computer generated holography is an extremely demanding and complex task when it comes to providing realistic reconstructions with full parallax, occlusion, and shadowing. We present an algorithm designed for data-parallel computing on modern graphics processing units to alleviate the computational burden. We apply Gaussian interpolation to create a continuous surface representation from discrete input object points. The algorithm maintains a potential occluder list for each individual hologram plane sample to keep the number of visibility tests to a minimum.We experimented with two approximations that simplify and accelerate occlusion computation. It is observed that letting several neighboring hologramplane samples share visibility information on object points leads to significantly faster computation without causing noticeable artifacts in the reconstructed images. Computing a reduced sample set via nonuniform sampling is also found to be an effective acceleration technique. © 2009 Optical Society of America.

Reference Points Based on Dynamic Optimisation: A Versatil Algorithm for Mixed Fishery Management with Bio-economic Agestructured Models

Single-species management objectives may not be consistent within mixed fisheries. They may lead species to unsafe situations, promote discarding of over-quota and/or misreporting of catches. We provide an algorithm for characterising bio-economic reference points for a mixed fishery as the steady-state solution of a dynamic optimal management problem. The optimisation problem takes into account: i) that species are fishing simultaneously in unselective fishing operations and ii)intertemporal discounting and fleet costs to relate reference points to discounted economic profits along optimal trajectories. We illustrate how the algorithm can be implemented by applying it to the European Northern Stock of Hake (Merluccius merluccius), where fleets also capture Northern megrim (Lepidorhombus whiffiagonis) and Northern anglerfish (Lophius piscatorius and Lophius budegassa). We find that optimal mixed management leads to a target reference point that is quite similar to the 2/3 of the Fmsy single-species (hake) target. Mixed management is superior to singlespecies management because it leads the fishery to higher discounted profits with higher long-term SSB for all species. We calculate that the losses due to the use of the Fmsy single-species (hake) target in this mixed fishery account for 11.4% of total discounted profits.

Single sensor boost converter-based maximum power point tracking algorithms

Two new maximum power point tracking algorithms are presented: the input voltage sensor, and duty ratio maximum power point tracking algorithm (ViSD algorithm); and the output voltage sensor, and duty ratio maximum power point tracking algorithm (VoSD algorithm). The ViSD and VoSD algorithms have the features, characteristics and advantages of the incremental conductance algorithm (INC); but, unlike the incremental conductance algorithm which requires two sensors (the voltage sensor and current sensor), the two algorithms are more desirable because they require only one sensor: the voltage sensor. Moreover, the VoSD technique is less complex; hence, it requires less computational processing. Both the ViSD and the VoSD techniques operate by maximising power at the converter output, instead of the input. The ViSD algorithm uses a voltage sensor placed at the input of a boost converter, while the VoSD algorithm uses a voltage sensor placed at the output of a boost converter. © 2011 IEEE.

Segmentation and recognition using structure from motion point clouds

We propose an algorithm for semantic segmentation based on 3D point clouds derived from ego-motion. We motivate five simple cues designed to model specific patterns of motion and 3D world structure that vary with object category. We introduce features that project the 3D cues back to the 2D image plane while modeling spatial layout and context. A randomized decision forest combines many such features to achieve a coherent 2D segmentation and recognize the object categories present. Our main contribution is to show how semantic segmentation is possible based solely on motion-derived 3D world structure. Our method works well on sparse, noisy point clouds, and unlike existing approaches, does not need appearance-based descriptors. Experiments were performed on a challenging new video database containing sequences filmed from a moving car in daylight and at dusk. The results confirm that indeed, accurate segmentation and recognition are possible using only motion and 3D world structure. Further, we show that the motion-derived information complements an existing state-of-the-art appearance-based method, improving both qualitative and quantitative performance. © 2008 Springer Berlin Heidelberg.

Application of the fixed point method for solution in time stepping finite element analysis using the inverse vector Jiles-Atherton model

An implementation of the inverse vector Jiles-Atherton model for the solution of non-linear hysteretic finite element problems is presented. The implementation applies the fixed point method with differential reluctivity values obtained from the Jiles-Atherton model. Differential reluctivities are usually computed using numerical differentiation, which is ill-posed and amplifies small perturbations causing large sudden increases or decreases of differential reluctivity values, which may cause numerical problems. A rule based algorithm for conditioning differential reluctivity values is presented. Unwanted perturbations on the computed differential reluctivity values are eliminated or reduced with the aim to guarantee convergence. Details of the algorithm are presented together with an evaluation of the algorithm by a numerical example. The algorithm is shown to guarantee convergence, although the rate of convergence depends on the choice of algorithm parameters. © 2011 IEEE.

Automated sparse 3D point cloud generation of infrastructure using its distinctive visual features

The commercial far-range (>10 m) spatial data collection methods for acquiring infrastructure’s geometric data are not completely automated because of the necessary manual pre- and/or post-processing work. The required amount of human intervention and, in some cases, the high equipment costs associated with these methods impede their adoption by the majority of infrastructure mapping activities. This paper presents an automated stereo vision-based method, as an alternative and inexpensive solution, to producing a sparse Euclidean 3D point cloud of an infrastructure scene utilizing two video streams captured by a set of two calibrated cameras. In this process SURF features are automatically detected and matched between each pair of stereo video frames. 3D coordinates of the matched feature points are then calculated via triangulation. The detected SURF features in two successive video frames are automatically matched and the RANSAC algorithm is used to discard mismatches. The quaternion motion estimation method is then used along with bundle adjustment optimization to register successive point clouds. The method was tested on a database of infrastructure stereo video streams. The validity and statistical significance of the results were evaluated by comparing the spatial distance of randomly selected feature points with their corresponding tape measurements.

The Gaussian mixture MCMC particle algorithm for dynamic cluster tracking

We present a novel filtering algorithm for tracking multiple clusters of coordinated objects. Based on a Markov chain Monte Carlo (MCMC) mechanism, the new algorithm propagates a discrete approximation of the underlying filtering density. A dynamic Gaussian mixture model is utilized for representing the time-varying clustering structure. This involves point process formulations of typical behavioral moves such as birth and death of clusters as well as merging and splitting. For handling complex, possibly large scale scenarios, the sampling efficiency of the basic MCMC scheme is enhanced via the use of a Metropolis within Gibbs particle refinement step. As the proposed methodology essentially involves random set representations, a new type of estimator, termed the probability hypothesis density surface (PHDS), is derived for computing point estimates. It is further proved that this estimator is optimal in the sense of the mean relative entropy. Finally, the algorithm's performance is assessed and demonstrated in both synthetic and realistic tracking scenarios. © 2012 Elsevier Ltd. All rights reserved.

Single-sensor maximum power point tracking algorithms

Single-sensor maximum power point tracking algorithms for photovoltaic systems are presented. The algorithms have the features, characteristics and advantages of the widely used incremental conductance (INC) algorithm. However; unlike the INC algorithm which requires two sensors (the voltage sensor and the current sensor), the single-sensor algorithms are more desirable because they require only one sensor: the voltage sensor. The algorithms operate by maximising power at the DC-DC converter output, instead of the input. © 2013 The Institution of Engineering and Technology.

Material point method for coupled hydromechanical problems

This paper describes a new formulation of the material point method (MPM) for solving coupled hydromechanical problems of fluid-saturated soil subjected to large deformation. A soil-pore fluid coupled MPM algorithm based on Biot's mixture theory is proposed for solving hydromechanical interaction problems that include changes in water table location with time. The accuracy of the proposed method is examined by comparing the results of the simulation of a one-dimensional consolidation test with the corresponding analytical solution. A sensitivity analysis of the MPM parameters used in the proposed method is carried out for examining the effect of the number of particles per mesh and mesh size on solution accuracy. For demonstrating the capability of the proposed method, a physical model experiment of a large-scale levee failure by seepage is simulated. The behavior of the levee model with time-dependent changes in water table matches well to the experimental observations. The mechanisms of seepage-induced failure are discussed by examining the pore-water pressures, as well as the effective stresses computed from the simulations © 2013 American Society of Civil Engineers.