Dynamics of learning with restricted training sets

The dynamics of supervised learning in layered neural networks were studied in the regime where the size of the training set is proportional to the number of inputs. The evolution of macroscopic observables, including the two relevant performance measures can be predicted by using the dynamical replica theory. Three approximation schemes aimed at eliminating the need to solve a functional saddle-point equation at each time step have been derived.

Noise, regularizers, and unrealizable scenarios in online learning from restricted training sets

We study the dynamics of on-line learning in multilayer neural networks where training examples are sampled with repetition and where the number of examples scales with the number of network weights. The analysis is carried out using the dynamical replica method aimed at obtaining a closed set of coupled equations for a set of macroscopic variables from which both training and generalization errors can be calculated. We focus on scenarios whereby training examples are corrupted by additive Gaussian output noise and regularizers are introduced to improve the network performance. The dependence of the dynamics on the noise level, with and without regularizers, is examined, as well as that of the asymptotic values obtained for both training and generalization errors. We also demonstrate the ability of the method to approximate the learning dynamics in structurally unrealizable scenarios. The theoretical results show good agreement with those obtained by computer simulations.

Noise and the trading mechanism:the case of SETS

On 20 October 1997 the London Stock Exchange introduced a new trading system called SETS. This system was to replace the dealer system SEAQ, which had been in operation since 1986. Using the iterative sum of squares test introduced by Inclan and Tiao (1994), we investigate whether there was a change in the unconditional variance of opening and closing returns, at the time SETS was introduced. We show that for the FTSE-100 stocks traded on SETS, on the days following its introduction, there was a widespread increase in the volatility of both opening and closing returns. However, no synchronous volatility changes were found to be associated with the FTSE-100 index or FTSE-250 stocks. We conclude therefore that the introduction of the SETS trading mechanism caused an increase in noise at the time the system was introduced.

An illustration of variable precision rough sets model:an analysis of the findings of the UK Monopolies and Mergers Commission

This paper introduces a new technique in the investigation of limited-dependent variable models. This paper illustrates that variable precision rough set theory (VPRS), allied with the use of a modern method of classification, or discretisation of data, can out-perform the more standard approaches that are employed in economics, such as a probit model. These approaches and certain inductive decision tree methods are compared (through a Monte Carlo simulation approach) in the analysis of the decisions reached by the UK Monopolies and Mergers Committee. We show that, particularly in small samples, the VPRS model can improve on more traditional models, both in-sample, and particularly in out-of-sample prediction. A similar improvement in out-of-sample prediction over the decision tree methods is also shown.

Evaluating cost efficiency in central administrative services in UK universities

This paper describes an attempt to evaluate cost efficiency in UK university central administration. The funding councils of higher education institutions have progressively evolved elaborate systems for measuring university performance in teaching quality and research. Indeed, funding of universities is linked to their performance in research. The allocation of resources between academic and administrative activities, on the other hand, has so far not been subject to scrutiny. Yet, expenditure on administration is typically some 30% of that allocated to academic activities. This paper sets up a data envelopment analysis (DEA) framework to identify practices leading to cost-efficient central administrative services in UK universities. The problems in defining the unit of assessment and the relationship between the inputs and the outputs are clearly demonstrated. © 2005 Elsevier Ltd. All rights reserved.

Defining the Fluid framework

In this position paper we present the developing Fluid framework, which we believe offers considerable advantages in maintaining software stability in dynamic or evolving application settings. The Fluid framework facilitates the development of component software via the selection, composition and configuration of components. Fluid's composition language incorporates a high-level type system supporting object-oriented principles such as type description, type inheritance, and type instantiation. Object-oriented relationships are represented via the dynamic composition of component instances. This representation allows the software structure, as specified by type and instance descriptions, to change dynamically at runtime as existing types are modified and new types and instances are introduced. We therefore move from static software structure descriptions to more dynamic representations, while maintaining the expressiveness of object-oriented semantics. We show how the Fluid framework relates to existing, largely component based, software frameworks and conclude with suggestions for future enhancements. © 2007 IEEE.

Corpus, collocation, and lexical sets

This paper is a progress report on a research path I first outlined in my contribution to “Words in Context: A Tribute to John Sinclair on his Retirement” (Heffer and Sauntson, 2000). Therefore, I first summarize that paper here, in order to provide the relevant background. The second half of the current paper consists of some further manual analyses, exploring various parameters and procedures that might assist in the design of an automated computational process for the identification of lexical sets. The automation itself is beyond the scope of the current paper.

Sequential, Bayesian geostatistics:a principled method for large data sets

The principled statistical application of Gaussian random field models used in geostatistics has historically been limited to data sets of a small size. This limitation is imposed by the requirement to store and invert the covariance matrix of all the samples to obtain a predictive distribution at unsampled locations, or to use likelihood-based covariance estimation. Various ad hoc approaches to solve this problem have been adopted, such as selecting a neighborhood region and/or a small number of observations to use in the kriging process, but these have no sound theoretical basis and it is unclear what information is being lost. In this article, we present a Bayesian method for estimating the posterior mean and covariance structures of a Gaussian random field using a sequential estimation algorithm. By imposing sparsity in a well-defined framework, the algorithm retains a subset of “basis vectors” that best represent the “true” posterior Gaussian random field model in the relative entropy sense. This allows a principled treatment of Gaussian random field models on very large data sets. The method is particularly appropriate when the Gaussian random field model is regarded as a latent variable model, which may be nonlinearly related to the observations. We show the application of the sequential, sparse Bayesian estimation in Gaussian random field models and discuss its merits and drawbacks.

Projected sequential Gaussian processes: flexible interpolation for large data sets

Recently within the machine learning and spatial statistics communities many papers have explored the potential of reduced rank representations of the covariance matrix, often referred to as projected or fixed rank approaches. In such methods the covariance function of the posterior process is represented by a reduced rank approximation which is chosen such that there is minimal information loss. In this paper a sequential framework for inference in such projected processes is presented, where the observations are considered one at a time. We introduce a C++ library for carrying out such projected, sequential estimation which adds several novel features. In particular we have incorporated the ability to use a generic observation operator, or sensor model, to permit data fusion. We can also cope with a range of observation error characteristics, including non-Gaussian observation errors. Inference for the variogram parameters is based on maximum likelihood estimation. We illustrate the projected sequential method in application to synthetic and real data sets. We discuss the software implementation and suggest possible future extensions.

Defining an OWL ontology that could be used to integrate mental-health risk information within a decision support system

In the field of mental health risk assessment, there is no standardisation between the data used in different systems. As a first step towards the possible interchange of data between assessment tools, an ontology has been constructed for a particular one, GRiST (Galatean Risk Screening Tool). We briefly introduce GRiST and its data structures, then describe the ontology and the benefits that have already been realised from the construction process. For example, the ontology has been used to check the consistency of the various trees used in the model. We then consider potential uses in integration of data from other sources. © 2009 IEEE.

Defining response to anti-VEGF therapies in neovascular AMD

The introduction of anti-vascular endothelial growth factor (anti-VEGF) has made significant impact on the reduction of the visual loss due to neovascular age-related macular degeneration (n-AMD). There are significant inter-individual differences in response to an anti-VEGF agent, made more complex by the availability of multiple anti-VEGF agents with different molecular configurations. The response to anti-VEGF therapy have been found to be dependent on a variety of factors including patient’s age, lesion characteristics, lesion duration, baseline visual acuity (VA) and the presence of particular genotype risk alleles. Furthermore, a proportion of eyes with n-AMD show a decline in acuity or morphology, despite therapy or require very frequent re-treatment. There is currently no consensus as to how to classify optimal response, or lack of it, with these therapies. There is, in particular, confusion over terms such as ‘responder status’ after treatment for n-AMD, ‘tachyphylaxis’ and ‘recalcitrant’ n-AMD. This document aims to provide a consensus on definition/categorisation of the response of n-AMD to anti-VEGF therapies and on the time points at which response to treatment should be determined. Primary response is best determined at 1 month following the last initiation dose, while maintained treatment (secondary) response is determined any time after the 4th visit. In a particular eye, secondary responses do not mirror and cannot be predicted from that in the primary phase. Morphological and functional responses to anti-VEGF treatments, do not necessarily correlate, and may be dissociated in an individual eye. Furthermore, there is a ceiling effect that can negate the currently used functional metrics such as >5 letters improvement when the baseline VA is good (ETDRS>70 letters). It is therefore important to use a combination of both the parameters in determining the response.The following are proposed definitions: optimal (good) response is defined as when there is resolution of fluid (intraretinal fluid; IRF, subretinal fluid; SRF and retinal thickening), and/or improvement of >5 letters, subject to the ceiling effect of good starting VA. Poor response is defined as <25% reduction from the baseline in the central retinal thickness (CRT), with persistent or new IRF, SRF or minimal or change in VA (that is, change in VA of 0+4 letters). Non-response is defined as an increase in fluid (IRF, SRF and CRT), or increasing haemorrhage compared with the baseline and/or loss of >5 letters compared with the baseline or best corrected vision subsequently. Poor or non-response to anti-VEGF may be due to clinical factors including suboptimal dosing than that required by a particular patient, increased dosing intervals, treatment initiation when disease is already at an advanced or chronic stage), cellular mechanisms, lesion type, genetic variation and potential tachyphylaxis); non-clinical factors including poor access to clinics or delayed appointments may also result in poor treatment outcomes. In eyes classified as good responders, treatment should be continued with the same agent when disease activity is present or reactivation occurs following temporary dose holding. In eyes that show partial response, treatment may be continued, although re-evaluation with further imaging may be required to exclude confounding factors. Where there is persistent, unchanging accumulated fluid following three consecutive injections at monthly intervals, treatment may be withheld temporarily, but recommenced with the same or alternative anti-VEGF if the fluid subsequently increases (lesion considered active). Poor or non-response to anti-VEGF treatments requires re-evaluation of diagnosis and if necessary switch to alternative therapies including other anti-VEGF agents and/or with photodynamic therapy (PDT). Idiopathic polypoidal choroidopathy may require treatment with PDT monotherapy or combination with anti-VEGF. A committee comprised of retinal specialists with experience of managing patients with n-AMD similar to that which developed the Royal College of Ophthalmologists Guidelines to Ranibizumab was assembled. Individual aspects of the guidelines were proposed by the committee lead (WMA) based on relevant reference to published evidence base following a search of Medline and circulated to all committee members for discussion before approval or modification. Each draft was modified according to feedback from committee members until unanimous approval was obtained in the final draft. A system for categorising the range of responsiveness of n-AMD lesions to anti-VEGF therapy is proposed. The proposal is based primarily on morphological criteria but functional criteria have been included. Recommendations have been made on when to consider discontinuation of therapy either because of success or futility. These guidelines should help clinical decision-making and may prevent over and/or undertreatment with anti-VEGF therapy.

An allosteric role for receptor activity-modifying proteins in defining GPCR pharmacology

G protein-coupled receptors are allosteric proteins that control transmission of external signals to regulate cellular response. Although agonist binding promotes canonical G protein signalling transmitted through conformational changes, G protein-coupled receptors also interact with other proteins. These include other G protein-coupled receptors, other receptors and channels, regulatory proteins and receptor-modifying proteins, notably receptor activity-modifying proteins (RAMPs). RAMPs have at least 11 G protein-coupled receptor partners, including many class B G protein-coupled receptors. Prototypic is the calcitonin receptor, with altered ligand specificity when co-expressed with RAMPs. To gain molecular insight into the consequences of this protein–protein interaction, we combined molecular modelling with mutagenesis of the calcitonin receptor extracellular domain, assessed in ligand binding and functional assays. Although some calcitonin receptor residues are universally important for peptide interactions (calcitonin, amylin and calcitonin gene-related peptide) in calcitonin receptor alone or with receptor activity-modifying protein, others have RAMP-dependent effects, whereby mutations decreased amylin/calcitonin gene-related peptide potency substantially only when RAMP was present. Remarkably, the key residues were completely conserved between calcitonin receptor and AMY receptors, and between subtypes of AMY receptor that have different ligand preferences. Mutations at the interface between calcitonin receptor and RAMP affected ligand pharmacology in a RAMP-dependent manner, suggesting that RAMP may allosterically influence the calcitonin receptor conformation. Supporting this, molecular dynamics simulations suggested that the calcitonin receptor extracellular N-terminal domain is more flexible in the presence of receptor activity-modifying protein 1. Thus, RAMPs may act in an allosteric manner to generate a spectrum of unique calcitonin receptor conformational states, explaining the pharmacological preferences of calcitonin receptor-RAMP complexes. This provides novel insight into our understanding of G protein-coupled receptor-protein interaction that is likely broadly applicable for this receptor class.