Incorporating curvature information into on-line learning

We analyse the dynamics of a number of second order on-line learning algorithms training multi-layer neural networks, using the methods of statistical mechanics. We first consider on-line Newton's method, which is known to provide optimal asymptotic performance. We determine the asymptotic generalization error decay for a soft committee machine, which is shown to compare favourably with the result for standard gradient descent. Matrix momentum provides a practical approximation to this method by allowing an efficient inversion of the Hessian. We consider an idealized matrix momentum algorithm which requires access to the Hessian and find close correspondence with the dynamics of on-line Newton's method. In practice, the Hessian will not be known on-line and we therefore consider matrix momentum using a single example approximation to the Hessian. In this case good asymptotic performance may still be achieved, but the algorithm is now sensitive to parameter choice because of noise in the Hessian estimate. On-line Newton's method is not appropriate during the transient learning phase, since a suboptimal unstable fixed point of the gradient descent dynamics becomes stable for this algorithm. A principled alternative is to use Amari's natural gradient learning algorithm and we show how this method provides a significant reduction in learning time when compared to gradient descent, while retaining the asymptotic performance of on-line Newton's method.

The dynamics of matrix momentum

We analyse the matrix momentum algorithm, which provides an efficient approximation to on-line Newton's method, by extending a recent statistical mechanics framework to include second order algorithms. We study the efficacy of this method when the Hessian is available and also consider a practical implementation which uses a single example estimate of the Hessian. The method is shown to provide excellent asymptotic performance, although the single example implementation is sensitive to the choice of training parameters. We conjecture that matrix momentum could provide efficient matrix inversion for other second order algorithms.

Simultaneous chemical reaction and distillation of formaldehyde

The thesis is concerned with the development and testing of a mathematical model of a distillation process in which the components react chemically. The formaldehyde-methanol-water system was selected and only the reversible reactions between formaldehyde and water giving methylene glycol and between formaldehyde and methanol producing hemiformal were assumed to occur under the distillation conditions. Accordingly the system has been treated as a five component system. The vapour-liquid equilibrium calculations were performed by solving iteratively the thermodynamic relationships expressing the phase equilibria with the stoichiometric equations expressing the chemical equilibria. Using optimisation techniques, the Wilson single parameters and Henry's constants were calculated for binary systems containing formaldehyde which was assumed to be a supercritical component whilst Wilson binary parameters were calculated for the remaining binary systems. Thus the phase equilibria for the formaldehyde system could be calculated using these parameters and good accuracy was obtained when calculated values were compared with experimental values. The distillation process was modelled using the mass and energy balance equations together with the phase equilibria calculations. The plate efficiencies were obtained from a modified A.I.Ch.E. Bubble Tray method. The resulting equations were solved by an iterative plate to plate calculation based on the Newton Raphson method. Experiments were carried out in a 76mm I.D., eight sieve plate distillation column and the results were compared with the mathematical model calculations. Overall, good agreement was obtained but some discrepancies were observed in the concentration profiles and these may have been caused by the effect of limited physical property data and a limited understanding of the reactions mechanism. The model equations were solved in the form of modular computer programs. Although they were written to describe the steady state distillation with simultaneous chemical reaction of the formaldehyde system, the approach used may be of wider application.

Non-Linear oil firm force coefficients for a journal bearing operating under aligned and misaligned conditions

It is well established that hydrodynamic journal bearings are responsible for self-excited vibrations and have the effect of lowering the critical speeds of rotor systems. The forces within the oil film wedge, generated by the vibrating journal, may be represented by displacement and velocity coefficient~ thus allowing the dynamical behaviour of the rotor to be analysed both for stability purposes and for anticipating the response to unbalance. However, information describing these coefficients is sparse, misleading, and very often not applicable to industrial type bearings. Results of a combined analytical and experimental investigation into the hydrodynamic oil film coefficients operating in the laminar region are therefore presented, the analysis being applied to a 120 degree partial journal bearing having a 5.0 in diameter journal and a LID ratio of 1.0. The theoretical analysis shows that for this type of popular bearing, the eight linearized coefficients do not accurately describe the behaviour of the vibrating journal based on the theory of small perturbations, due to them being masked by the presence of nonlinearity. A method is developed using the second order terms of Taylor expansion whereby design charts are provided which predict the twentyeight force coefficients for both aligned, and for varying amounts of journal misalignment. The resulting non-linear equations of motion are solved using a modified Newton-Raphson method whereby the whirl trajectories are obtained, thus providing a physical appreciation of the bearing characteristics under dynamically loaded conditions.

Steffensen Methods for Solving Generalized Equations

2000 Mathematics Subject Classification: 65G99, 65K10, 47H04.

Volume averaging of multiphase flows with hydrate formation in subsea pipelines

In oil and gas pipeline operations, the gas, oil, and water phases simultaneously move through pipe systems. The mixture cools as it flows through subsea pipelines, and forms a hydrate formation region, where the hydrate crystals start to grow and may eventually block the pipeline. The potential of pipe blockage due to hydrate formation is one of the most significant flow-assurance problems in deep-water subsea operations. Due to the catastrophic safety and economic implications of hydrate blockage, it is important to accurately predict the simultaneous flow of gas, water, and hydrate particles in flowlines. Currently, there are few or no studies that account for the simultaneous effects of hydrate growth and heat transfer on flow characteristics within pipelines. This thesis presents new and more accurate predictive models of multiphase flows in undersea pipelines to describe the simultaneous flow of gas, water, and hydrate particles through a pipeline. A growth rate model for the hydrate phase is presented and then used in the development of a new three-phase model. The conservation equations of mass, momentum, and energy are formulated to describe the physical phenomena of momentum and heat transfer between the fluid and the wall. The governing equations are solved based on an analytical-numerical approach using a Newton-Raphson method for the nonlinear equations. An algorithm was developed in Matlab software to solve the equations from the inlet to the outlet of the pipeline. The developed models are validated against a single-phase model with mixture properties, and the results of comparative studies show close agreement. The new model predicts the volume fraction and velocity of each phase, as well as the mixture pressure and temperature profiles along the length of the pipeline. The results from the hydrate growth model reveal the growth rate and location where the initial hydrates start to form. Finally, to assess the impact of certain parameters on the flow characteristics, parametric studies have been conducted. The results show the effect of a variation in the pipe diameter, mass flow rate, inlet pressure, and inlet temperature on the flow characteristics and hydrate growth rates.

Selective Preparation of Semiconducting Single-Walled Carbon Nanotubes: From Fundamentals to Applications

Aligned single-walled carbon nanotubes (SWNTs) synthesized by the chemical vapor deposition (CVD) method have exceptional potential for next-generation nanoelectronics. However, there are considerable challenges in the preparation of semiconducting (s-) SWNTs with controlled properties (e.g., density, selectivity, and diameter) for their application in solving real-world problems. This dissertation describes research that aims to overcome the limitations by novel synthesis strategies and post-growth treatment. The application of as-prepared SWNTs as functional devices is also demonstrated. The dissertation includes the following parts: 1) decoupling the conflict between density and selectivity of s-SWNTs in CVD growth; 2) investigating the importance of diameter control for the selective synthesis of s-SWNTs; 3) synthesizing highly conductive SWNT thin film by thiophene-assisted CVD method; 4) eliminating metallic pathways in SWNT crossbars by gate-free electrical breakdown method; 5) enhancing the density of SWNT arrays by strain-release method; 6) studying the sensing mechanism of SWNT crossbar chemical sensors.

Extended nonlinear analytical models of compliant parallelogram mechanisms: third-order models

This paper proposes extended nonlinear analytical models, third-order models, of compliant parallelogram mechanisms. These models are capable of capturing the accurate effects from the very large axial force within the transverse motion range of 10% of the beam length through incorporating the terms associated with the high-order (up to third-order) axial force. Firstly, the free-body diagram method is employed to derive the nonlinear analytical model for a basic compliant parallelogram mechanism based on load-displacement relations of a single beam, geometry compatibility conditions, and load-equilibrium conditions. The procedures for the forward solutions and inverse solutions are described. Nonlinear analytical models for guided compliant multi-beam parallelogram mechanisms are then obtained. A case study of the compound compliant parallelogram mechanism, composed of two basic compliant parallelogram mechanisms in symmetry, is further implemented. This work intends to estimate the internal axial force change, the transverse force change, and the transverse stiffness change with the transverse motion using the proposed third-order model in comparison with the first-order model proposed in the prior art. In addition, FEA (finite element analysis) results validate the accuracy of the third-order model for a typical example. It is shown that in the case study the slenderness ratio affects the result discrepancy between the third-order model and the first-order model significantly, and the third-order model can illustrate a non-monotonic transverse stiffness curve if the beam is thin enough.

Low cost 3D global instability analysis and flow sensitivity based on dynamic mode decomposition and high-order numerical tools

We explore the recently developed snapshot-based dynamic mode decomposition (DMD) technique, a matrix-free Arnoldi type method, to predict 3D linear global flow instabilities. We apply the DMD technique to flows confined in an L-shaped cavity and compare the resulting modes to their counterparts issued from classic, matrix forming, linear instability analysis (i.e. BiGlobal approach) and direct numerical simulations. Results show that the DMD technique, which uses snapshots generated by a 3D non-linear incompressible discontinuous Galerkin Navier?Stokes solver, provides very similar results to classical linear instability analysis techniques. In addition, we compare DMD results issued from non-linear and linearised Navier?Stokes solvers, showing that linearisation is not necessary (i.e. base flow not required) to obtain linear modes, as long as the analysis is restricted to the exponential growth regime, that is, flow regime governed by the linearised Navier?Stokes equations, and showing the potential of this type of analysis based on snapshots to general purpose CFD codes, without need of modifications. Finally, this work shows that the DMD technique can provide three-dimensional direct and adjoint modes through snapshots provided by the linearised and adjoint linearised Navier?Stokes equations advanced in time. Subsequently, these modes are used to provide structural sensitivity maps and sensitivity to base flow modification information for 3D flows and complex geometries, at an affordable computational cost. The information provided by the sensitivity study is used to modify the L-shaped geometry and control the most unstable 3D mode.

Krylov subspace approximations for the exponential Euler method: Error estimates and the harmonic Ritz approximant

We study Krylov subspace methods for approximating the matrix-function vector product φ(tA)b where φ(z) = [exp(z) - 1]/z. This product arises in the numerical integration of large stiff systems of differential equations by the Exponential Euler Method, where A is the Jacobian matrix of the system. Recently, this method has found application in the simulation of transport phenomena in porous media within mathematical models of wood drying and groundwater flow. We develop an a posteriori upper bound on the Krylov subspace approximation error and provide a new interpretation of a previously published error estimate. This leads to an alternative Krylov approximation to φ(tA)b, the so-called Harmonic Ritz approximant, which we find does not exhibit oscillatory behaviour of the residual error.

Efficient gradient-free simplex method for estimation of optical properties in image-guided diffuse optical tomography

Typical image-guided diffuse optical tomographic image reconstruction procedures involve reduction of the number of optical parameters to be reconstructed equal to the number of distinct regions identified in the structural information provided by the traditional imaging modality. This makes the image reconstruction problem less ill-posed compared to traditional underdetermined cases. Still, the methods that are deployed in this case are same as those used for traditional diffuse optical image reconstruction, which involves a regularization term as well as computation of the Jacobian. A gradient-free Nelder-Mead simplex method is proposed here to perform the image reconstruction procedure and is shown to provide solutions that closely match ones obtained using established methods, even in highly noisy data. The proposed method also has the distinct advantage of being more efficient owing to being regularization free, involving only repeated forward calculations. (C) 2013 Society of Photo-Optical Instrumentation Engineers (SPIE)

The Graves Theorem Revisited II: Robust Convergence of the Newton Method

AMS subject classification: 65J15, 47H04, 90C30.

Ultra sensitive label free surface enhanced Raman spectroscopy method for the detection of biomolecules

We present a proof of concept for a novel nanosensor for the detection of ultra-trace amounts of bio-active molecules in complex matrices. The nanosensor is comprised of gold nanoparticles with an ultra-thin silica shell and antibody surface attachment, which allows for the immobilization and direct detection of bio-active molecules by surface enhanced Raman spectroscopy (SERS) without requiring a Raman label. The ultra-thin passive layer (~1.3 nm thickness) prevents competing molecules from binding non-selectively to the gold surface without compromising the signal enhancement. The antibodies attached on the surface of the nanoparticles selectively bind to the target molecule with high affinity. The interaction between the nanosensor and the target analyte result in conformational rearrangements of the antibody binding sites, leading to significant changes in the surface enhanced Raman spectra of the nanoparticles when compared to the spectra of the un-reacted nanoparticles. Nanosensors of this design targeting the bio-active compounds erythropoietin and caffeine were able to detect ultra-trace amounts the analyte to the lower quantification limits of 3.5×10−13 M and 1×10−9 M, respectively.

Method for planning and executing obstacle-free paths for rotating excavation machinery

This invention concerns the control of rotating excavation machinery, for instance to avoid collisions with obstacles. In a first aspect the invention is a control system for autonomous path planning in excavation machinery, comprising: A map generation subsystem to receive data from an array of disparate and complementary sensors to generate a 3-Dimensional digital terrain and obstacle map referenced to a coordinate frame related to the machine's geometry, during normal operation of the machine. An obstacle detection subsystem to find and identify obstacles in the digital terrain and obstacle map, and then to refine the map by identifying exclusion zones that are within reach of the machine during operation. A collision detection subsystem that uses knowledge of the machine's position and movements, as well as the digital terrain and obstacle map, to identify and predict possible collisions with itself or other obstacles, and then uses a forward motion planner to predict collisions in a planned path. And, a path planning subsystem that uses information from the other subsystems to vary planned paths to avoid obstacles and collisions. In other aspects the invention is excavation machinery including the control system; a method for control of excavation machinery; and firmware and software versions of the control system.