Incorporating curvature information into on-line learning

We analyse the dynamics of a number of second order on-line learning algorithms training multi-layer neural networks, using the methods of statistical mechanics. We first consider on-line Newton's method, which is known to provide optimal asymptotic performance. We determine the asymptotic generalization error decay for a soft committee machine, which is shown to compare favourably with the result for standard gradient descent. Matrix momentum provides a practical approximation to this method by allowing an efficient inversion of the Hessian. We consider an idealized matrix momentum algorithm which requires access to the Hessian and find close correspondence with the dynamics of on-line Newton's method. In practice, the Hessian will not be known on-line and we therefore consider matrix momentum using a single example approximation to the Hessian. In this case good asymptotic performance may still be achieved, but the algorithm is now sensitive to parameter choice because of noise in the Hessian estimate. On-line Newton's method is not appropriate during the transient learning phase, since a suboptimal unstable fixed point of the gradient descent dynamics becomes stable for this algorithm. A principled alternative is to use Amari's natural gradient learning algorithm and we show how this method provides a significant reduction in learning time when compared to gradient descent, while retaining the asymptotic performance of on-line Newton's method.

Using interior point algorithms for the solution of linear programs with special structural features

Linear Programming (LP) is a powerful decision making tool extensively used in various economic and engineering activities. In the early stages the success of LP was mainly due to the efficiency of the simplex method. After the appearance of Karmarkar's paper, the focus of most research was shifted to the field of interior point methods. The present work is concerned with investigating and efficiently implementing the latest techniques in this field taking sparsity into account. The performance of these implementations on different classes of LP problems is reported here. The preconditional conjugate gradient method is one of the most powerful tools for the solution of the least square problem, present in every iteration of all interior point methods. The effect of using different preconditioners on a range of problems with various condition numbers is presented. Decomposition algorithms has been one of the main fields of research in linear programming over the last few years. After reviewing the latest decomposition techniques, three promising methods were chosen the implemented. Sparsity is again a consideration and suggestions have been included to allow improvements when solving problems with these methods. Finally, experimental results on randomly generated data are reported and compared with an interior point method. The efficient implementation of the decomposition methods considered in this study requires the solution of quadratic subproblems. A review of recent work on algorithms for convex quadratic was performed. The most promising algorithms are discussed and implemented taking sparsity into account. The related performance of these algorithms on randomly generated separable and non-separable problems is also reported.

Compositional semantics of dataflow networks with query-driven communication of exact values

We develop and study the concept of dataflow process networks as used for exampleby Kahn to suit exact computation over data types related to real numbers, such as continuous functions and geometrical solids. Furthermore, we consider communicating these exact objectsamong processes using protocols of a query-answer nature as introduced in our earlier work. This enables processes to provide valid approximations with certain accuracy and focusing on certainlocality as demanded by the receiving processes through queries. We define domain-theoretical denotational semantics of our networks in two ways: (1) directly, i. e. by viewing the whole network as a composite process and applying the process semantics introduced in our earlier work; and (2) compositionally, i. e. by a fixed-point construction similarto that used by Kahn from the denotational semantics of individual processes in the network. The direct semantics closely corresponds to the operational semantics of the network (i. e. it iscorrect) but very difficult to study for concrete networks. The compositional semantics enablescompositional analysis of concrete networks, assuming it is correct. We prove that the compositional semantics is a safe approximation of the direct semantics. Wealso provide a method that can be used in many cases to establish that the two semantics fully coincide, i. e. safety is not achieved through inactivity or meaningless answers. The results are extended to cover recursively-defined infinite networks as well as nested finitenetworks. A robust prototype implementation of our model is available.

Two-phases control of DSRC Vehicle networks for road intersection coexisting safety applications

In this paper we propose a two phases control method for DSRC vehicle networks at road intersection, where multiple road safety applications may coexist. We consider two safety applications, emergency safety application with high priority and routine safety applications with low priority. The control method is designed to provide high availability and low latency for emergency safety applications while leave as much as possible bandwidth for routine applications. It is expected to be capable of adapting to changing network conditions. In the first phase of the method we use a simulation based offline approach to find out the best configurations for message rate and MAC layer parameters for given numbers of vehicles. In the second phase we use the configurations identified by simulations at roadside access point (AP) for system operation. A utilization function is proposed to balance the QoS performances provided to multiple safety applications. It is demonstrated that the proposed method can largely improve the system performance when compared to fixed control method.

Adaptive congestion control of DSRC vehicle networks for collaborative road safety applications

Congestion control is critical for the provisioning of quality of services (QoS) over dedicated short range communications (DSRC) vehicle networks for road safety applications. In this paper we propose a congestion control method for DSRC vehicle networks at road intersection, with the aims of providing high availability and low latency channels for high priority emergency safety applications while maximizing channel utilization for low priority routine safety applications. In this method a offline simulation based approach is used to find out the best possible configurations of message rate and MAC layer backoff exponent (BE) for a given number of vehicles equipped with DSRC radios. The identified best configurations are then used online by an roadside access point (AP) for system operation. Simulation results demonstrated that this adaptive method significantly outperforms the fixed control method under varying number of vehicles. The impact of estimation error on the number of vehicles in the network on system level performance is also investigated.

Fast reconstruction of harmonic functions from Cauchy data using integral equation techniques

We consider the problem of stable determination of a harmonic function from knowledge of the solution and its normal derivative on a part of the boundary of the (bounded) solution domain. The alternating method is a procedure to generate an approximation to the harmonic function from such Cauchy data and we investigate a numerical implementation of this procedure based on Fredholm integral equations and Nyström discretization schemes, which makes it possible to perform a large number of iterations (millions) with minor computational cost (seconds) and high accuracy. Moreover, the original problem is rewritten as a fixed point equation on the boundary, and various other direct regularization techniques are discussed to solve that equation. We also discuss how knowledge of the smoothness of the data can be used to further improve the accuracy. Numerical examples are presented showing that accurate approximations of both the solution and its normal derivative can be obtained with much less computational time than in previous works.

Direct femtosecond laser inscription in transparent dielectrics

Since 1996 direct femtosecond inscription in transparent dielectrics has become the subject of intensive research. This enabling technology significantly expands the technological boundaries for direct fabrication of 3D structures in a wide variety of materials. It allows modification of non-photosensitive materials, which opens the door to numerous practical applications. In this work we explored the direct femtosecond inscription of waveguides and demonstrated at least one order of magnitude enhancement in the most critical parameter - the induced contrast of the refractive index in a standard borosilicate optical glass. A record high induced refractive contrast of 2.5×10-2 is demonstrated. The waveguides fabricated possess one of the lowest losses, approaching level of Fresnel reflection losses at the glassair interface. High refractive index contrast allows the fabrication of curvilinear waveguides with low bend losses. We also demonstrated the optimisation of the inscription regimes in BK7 glass over a broad range of experimental parameters and observed a counter-intuitive increase of the induced refractive index contrast with increasing translation speed of a sample. Examples of inscription in a number of transparent dielectrics hosts using high repetition rate fs laser system (both glasses and crystals) are also presented. Sub-wavelength scale periodic inscription inside any material often demands supercritical propagation regimes, when pulse peak power is more than the critical power for selffocusing, sometimes several times higher than the critical power. For a sub-critical regime, when the pulse peak power is less than the critical power for self-focusing, we derive analytic expressions for Gaussian beam focusing in the presence of Kerr non-linearity as well as for a number of other beam shapes commonly used in experiments, including astigmatic and ring-shaped ones. In the part devoted to the fabrication of periodic structures, we report on recent development of our point-by-point method, demonstrating the shortest periodic perturbation created in the bulk of a pure fused silica sample, by using third harmonics (? =267 nm) of fundamental laser frequency (? =800 nm) and 1 kHz femtosecond laser system. To overcome the fundamental limitations of the point-by-point method we suggested and experimentally demonstrated the micro-holographic inscription method, which is based on using the combination of a diffractive optical element and standard micro-objectives. Sub-500 nm periodic structures with a much higher aspect ratio were demonstrated. From the applications point of view, we demonstrate examples of photonics devices by direct femtosecond fabrication method, including various vectorial bend-sensors fabricated in standard optical fibres, as well as a highly birefringent long-period gratings by direct modulation method. To address the intrinsic limitations of femtosecond inscription at very shallow depths we suggested the hybrid mask-less lithography method. The method is based on precision ablation of a thin metal layer deposited on the surface of the sample to create a mask. After that an ion-exchange process in the melt of Ag-containing salts allows quick and low-cost fabrication of shallow waveguides and other components of integrated optics. This approach covers the gap in direct fs inscription of shallow waveguide. Perspectives and future developments of direct femtosecond micro-fabrication are also discussed.

Femtosecond inscription of the first order Bragg gratings in pure fused silica

The fabrication of sub-micron periodic structures beyond diffraction limit is a major motivation for the present paper. We describe the fabrication of the periodic structure of 25 mm long with a pitch size of 260 nm which is less than a third of the wavelength used. This is the smallest reported period of the periodic structure inscribed by direct point-by-point method. A prototype of the add-drop filter, which utilizes such gratings, was demonstrated in one stage fabrication process of femtosecond inscription, in the bulk fused silica.

Inscription of in-fibre photonic devices by an infrared femtosecond laser

In this work, a point by point method for the inscription of fibre Bragg gratings using a tightly focused infrared femtosecond laser is implemented for the first time. Fibre Bragg gratings are wavelength-selective, retro-reflectors which have become a key component in optical communications as well as offering great potential as a sensing tool. Standard methods of fabrication are based on UV inscription in fibre with a photosensitive core. Despite the high quality of the gratings, a number of disadvantages are associated with UV inscription, in particular, the requirements of a photosensitive fibre, the low thermal stability and the need to remove the protective coating prior to inscription. By combining the great flexibility offered by the point by point method with the advantages inherent to inscription by an infrared femtosecond laser, the previous disadvantages are overcome. The method here introduced, allows a fast inscription process at a rate of ~1mm/s, gratings of lengths between 1cm and 2cm exhibiting reflections in excess of 99%. Physical dimensions of these gratings differ significantly from those inscribed by other methods, in this case the grating is confined to a fraction of the cross section of the core, leading to strong and controllable birefringence and polarisation dependent loss. Finally, an investigation of the potential for their exploitation towards novel applications is carried out, devices such as directional bend sensors inscribed in single-mode fibre, superimposed but non-overlapping gratings, and single-mode, single-polarisation fibre lasers, were designed, fabricated and characterised based on point by point femtosecond inscription.

Femtosecond inscription of the first order Bragg gratings in pure fused silica

The fabrication of sub-micron periodic structures beyond diffraction limit is a major motivation for the present paper. We describe the fabrication of the periodic structure of 25 mm long with a pitch size of 260 nm which is less than a third of the wavelength used. This is the smallest reported period of the periodic structure inscribed by direct point-by-point method. A prototype of the add-drop filter, which utilizes such gratings, was demonstrated in one stage fabrication process of femtosecond inscription, in the bulk fused silica.

An Exploration of parametric software cost estimating models for Jackson Systems Development

Most parametric software cost estimation models used today evolved in the late 70's and early 80's. At that time, the dominant software development techniques being used were the early 'structured methods'. Since then, several new systems development paradigms and methods have emerged, one being Jackson Systems Development (JSD). As current cost estimating methods do not take account of these developments, their non-universality means they cannot provide adequate estimates of effort and hence cost. In order to address these shortcomings two new estimation methods have been developed for JSD projects. One of these methods JSD-FPA, is a top-down estimating method, based on the existing MKII function point method. The other method, JSD-COCOMO, is a sizing technique which sizes a project, in terms of lines of code, from the process structure diagrams and thus provides an input to the traditional COCOMO method.The JSD-FPA method allows JSD projects in both the real-time and scientific application areas to be costed, as well as the commercial information systems applications to which FPA is usually applied. The method is based upon a three-dimensional view of a system specification as opposed to the largely data-oriented view traditionally used by FPA. The method uses counts of various attributes of a JSD specification to develop a metric which provides an indication of the size of the system to be developed. This size metric is then transformed into an estimate of effort by calculating past project productivity and utilising this figure to predict the effort and hence cost of a future project. The effort estimates produced were validated by comparing them against the effort figures for six actual projects.The JSD-COCOMO method uses counts of the levels in a process structure chart as the input to an empirically derived model which transforms them into an estimate of delivered source code instructions.

Line-by-line fiber Bragg grating made by femtosecond laser

In this letter, we report on the inscription of a fourth-order fiber Bragg grating made line-by-line in the optical fiber using a femtosecond laser. Strong Bragg resonance (~17 dB) and low insertion loss (~0.5 dB) were obtained with only 2000 periods. Measured refractive index change of these inscribed lines reaches up to 7 × 10-3. The grating was fully characterized and the low insertion loss together with low polarization-dependent loss were realized compared to gratings made by the point-by-point method. The high temperature annealing experiment shows the grating can survive up to at least 800°C.

Adaptive message rate control of infrastructured DSRC vehicle networks for coexisting road safety and non-safety applications

Intelligent transport system (ITS) has large potentials on road safety applications as well as nonsafety applications. One of the big challenges for ITS is on the reliable and cost-effective vehicle communications due to the large quantity of vehicles, high mobility, and bursty traffic from the safety and non-safety applications. In this paper, we investigate the use of dedicated short-range communications (DSRC) for coexisting safety and non-safety applications over infrastructured vehicle networks. The main objective of this work is to improve the scalability of communications for vehicles networks, ensure QoS for safety applications, and leave as much as possible bandwidth for non-safety applications. A two-level adaptive control scheme is proposed to find appropriate message rate and control channel interval for safety applications. Simulation results demonstrated that this adaptive method outperforms the fixed control method under varying number of vehicles. © 2012 Wenyang Guan et al.

QoS guarantee with adaptive transmit power and message rate control for DSRC vehicle network based road safety applications

Quality of services (QoS) support is critical for dedicated short range communications (DSRC) vehicle networks based collaborative road safety applications. In this paper we propose an adaptive power and message rate control method for DSRC vehicle networks at road intersections. The design objective is to provide high availability and low latency channels for high priority emergency safety applications while maximizing channel utilization for low priority routine safety applications. In this method an offline simulation based approach is used to find out the best possible configurations of transmit power and message rate for given numbers of vehicles in the network. The identified best configurations are then used online by roadside access points (AP) according to estimated number of vehicles. Simulation results show that this adaptive method significantly outperforms a fixed control method. © 2011 Springer-Verlag.

Adaptive power and rate control for QoS guarantee of DSRC based collaborative road safety applications

In this paper we propose an adaptive power and message rate control method for safety applications at road intersections. The design objectives are to firstly provide guaranteed QoS support to both high priority emergency safety applications and low priority routine safety applications and secondly maximize channel utilization. We use an offline simulation based approach to find out the best possible configurations of transmit power and message rate for given numbers of vehicles in the network with certain safety QoS requirements. The identified configurations are then used online by roadside access points (AP) adaptively according to estimated number of vehicles. Simulation results show that this adaptive method could provide required QoS support to safety applications and it significantly outperforms a fixed control method. © 2013 International Information Institute.