Adaptive back-propagation in on-line learning of multilayer networks

An adaptive back-propagation algorithm is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, both numerical studies and a rigorous analysis show that the adaptive back-propagation method results in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

On-line learning with adaptive back-propagation in two-layer networks

An adaptive back-propagation algorithm parameterized by an inverse temperature 1/T is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, we analyse these learning algorithms in both the symmetric and the convergence phase for finite learning rates in the case of uncorrelated teachers of similar but arbitrary length T. These analyses show that adaptive back-propagation results generally in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

Lay perceptions of type 2 diabetes in Scotland:bringing health services back in

The growing prevalence of type 2 diabetes is placing Scottish health services under considerable strain. Consequently, diabetes services are undergoing a major process of reorganisation, including the devolvement of routine diabetes care/diabetic review from secondary to primary healthcare settings. This qualitative study was devised to explore newly diagnosed type 2 diabetes patients' perceptions of their disease and the health services they receive at a time when this restructuring of services is taking place. The sample comprised 40 patients resident in Lothian, Scotland, who had diverse experiences of services, some receiving GP-based care only, others having varying contact with hospital diabetes clinics. In-depth interviews were undertaken with patients, three times at six monthly intervals over 1 year, enabling their experiences to be tracked at critical junctures during the post-diagnostic period. Disease perceptions and health service delivery were found to be mutually informing and effecting. Not only did (different types of) health service delivery influence the ways in which patients thought about and self-managed their disease, over time patients' disease perceptions also informed their expectations of, and preferences for, diabetes services. We thus argue that there is a need for a reconceptualisation within the medical social sciences to take into account the context of healthcare and the economic/policy factors that inform health service delivery when looking at patients' disease perceptions. We also discuss the logistical and ethical challenges of drawing upon patients' perspectives, preferences and views in the design and delivery of future health services. © 2004 Elsevier Ltd. All rights reserved.

Back from the future:the second chance in the past of aller + infinitive

Typological studies (Bybee et al. 1994, Dahl 2000, Bourdin 2008 among many others) have shown a tendency for spatial elements (such as movement verbs) to grammaticalise into temporal expressions. The periphrasis made of aller + infinitive has been very productive in Romance languages. If in French, Spanish or Portuguese, it refers to future, it refers to a simple past in Catalan. fr.: je vais aller au cinéma esp.: voy a ir al cine pt. Vou ira o cine cat.: vaig anar al cine Barceló & Bres (2005:168) indicate that, In French, «vers le XVIème siècle, […] la périphrase a brièvement fonctionné comme un temps du passé (alors même que l’emploi comme temps du futur émergeait)»: (1) Sur ces propos, firent leur accord et, en regardant le lieu le plus propre pour faire cette belle œuvre, elle va dire qu’elle n’en savait point de meilleure ni plus loin de tout soupçon, qu’une petite maison qui était dedans le parc, où il y avait chambre et lit tout à propos. (Marguerite de Navarre, L’Heptaméron) The future value took over so much so that Damourette & Pichon (1911-1936 :117) claimed the past value had disappeared»: Un second tour, encore plus aberrant, n’a plus, que nous sachions, aucune position en pays d’Oui. Nous voulons parler de l’expression d’un passé au moyen de l’auxiliaire aller, suivi de l’infinitif. Ce tour a eu une grande fortune dans le provençal ancien et le catalan […] En français, on n’en trouve que des traces, notamment du XIVe au XVIe siècles . Ex.:[…] Adoncques s’arrêtèrent le conte et Raimondin soubz un grand arbre ; lors va dire le conte à Raimondin: […]. Et Raimondin va lui dire : Sire, ce qu’il vous plaira. That allegedly extinct use is nonetheless alive and kicking in Contemporary French. Larreya (2005:349)notes that it is « très courant dans les récits – en particulier dans les récits oraux» and it is also found in newspaper language as shown by this except from an obituary for Loulou Gasté (Le Monde 1995): (2) Celle-ci se déroule aux Editions Micro, où il a un bureau. Séduit par la jeune interprète, il commence à lui écrire des chansons sur mesure et leur complicité va mettre cinq ans à se transformer en amour. Au début de leur rencontre, Loulou est neurasthénique parce qu’il vient de divorcer et la future Line, exclusivement préoccupée par son métier, ne songe à rien d’autre. Line et Loulou vont rattraper le temps perdu et créer ensemble un millier de petites chansons dont la plupart sont devenues immortelles. Jusqu’à ces derniers mois, il ne va pas se passer une journée sans que Loulou s’empare de sa guitare pour créer un refrain. Would we be witnessing a linguistic resurrection? The same structure seems therefore to have grammaticalised in diverging and even diametrically opposite ways in different Romance languages. In this talk, we shall try to explain how the phrase aller + infinitive is able to function both as a future and a past. We will especially concentrate on the case of contemporary French where, while the future interpretation has obtained the status of tense as futur proche or périphrastique, a past interpretation is now commonplace. Our reflection will be supported by a personal corpus of authentic examples.

A constructive learning algorithm based on back-propagation

There are been a resurgence of interest in the neural networks field in recent years, provoked in part by the discovery of the properties of multi-layer networks. This interest has in turn raised questions about the possibility of making neural network behaviour more adaptive by automating some of the processes involved. Prior to these particular questions, the process of determining the parameters and network architecture required to solve a given problem had been a time consuming activity. A number of researchers have attempted to address these issues by automating these processes, concentrating in particular on the dynamic selection of an appropriate network architecture.The work presented here specifically explores the area of automatic architecture selection; it focuses upon the design and implementation of a dynamic algorithm based on the Back-Propagation learning algorithm. The algorithm constructs a single hidden layer as the learning process proceeds using individual pattern error as the basis of unit insertion. This algorithm is applied to several problems of differing type and complexity and is found to produce near minimal architectures that are shown to have a high level of generalisation ability.

Compensation of intra-channel nonlinear fibre impairments using simplified digital back-propagation algorithm

We investigate a digital back-propagation simplification method to enable computationally-efficient digital nonlinearity compensation for a coherently-detected 112 Gb/s polarization multiplexed quadrature phase shifted keying transmission over a 1,600 km link (20x80km) with no inline compensation. Through numerical simulation, we report up to 80% reduction in required back-propagation steps to perform nonlinear compensation, in comparison to the standard back-propagation algorithm. This method takes into account the correlation between adjacent symbols at a given instant using a weighted-average approach, and optimization of the position of nonlinear compensator stage to enable practical digital back-propagation.