Development of low entropy coding in a recurrent network.

In this paper we present an unsupervised neural network which exhibits competition between units via inhibitory feedback. The operation is such as to minimize reconstruction error, both for individual patterns, and over the entire training set. A key difference from networks which perform principal components analysis, or one of its variants, is the ability to converge to non-orthogonal weight values. We discuss the network's operation in relation to the twin goals of maximizing information transfer and minimizing code entropy, and show how the assignment of prior probabilities to network outputs can help to reduce entropy. We present results from two binary coding problems, and from experiments with image coding.

Recurrent input transformations for Hidden Markov models

This paper presents a new architecture which integrates recurrent input transformations (RIT) and continuous density HMMs. The basic HMM structure is extended to accommodate recurrent neural networks which transform the input observations before they enter the Gaussian output distributions associated with the states of the HMM. During training the parameters of both HMM and RIT are simultaneously optimized according to the Maximum Mutual Information (MMI) criterion. Results are presented for the E-set recognition task which demonstrate the ability of recurrent input transformations to exploit longer term correlations in the speech signal and to give improved discrimination.

Continuous speech recognition for the TIMIT database using neural networks

Four types of neural networks which have previously been established for speech recognition and tested on a small, seven-speaker, 100-sentence database are applied to the TIMIT database. The networks are a recurrent network phoneme recognizer, a modified Kanerva model morph recognizer, a compositional representation phoneme-to-word recognizer, and a modified Kanerva model morph-to-word recognizer. The major result is for the recurrent net, giving a phoneme recognition accuracy of 57% from the si and sx sentences. The Kanerva morph recognizer achieves 66.2% accuracy for a small subset of the sa and sx sentences. The results for the word recognizers are incomplete.

Climate resilience of schools: a case study

Social and political concerns are frequently reflected in the design of school buildings, often in turn leading to the development of technical innovations. One example is a recurrent concern about the physical health of the nation, which has at several points over the last century prompted new design approaches to natural light and ventilation. The most critical concern of the current era is the global, rather than the indoor, environment. The resultant political focus on mitigating climate change has resulted in new regulations, and in turn considerable technical changes in building design and construction. The vanguard of this movement has again been in school buildings, set the highest targets for reducing operational carbon by the previous Government. The current austerity measures have moved the focus to the refurbishment and retrofit of existing buildings, in order to bring them up to the exacting new standards. Meanwhile there is little doubt that climate change is happening already, and that the impacts will be considerable. Climate scientists have increasing confidence in their predictions for the future; if today’s buildings are to be resilient to these changes, building designers will need to understand and design for the predicted climates in order to continue to provide comfortable and healthy spaces through the lifetimes of the buildings. This paper describes the decision processes, and the planned design measures, for adapting an existing school for future climates. The project is at St Faith’s School in Cambridge, and focuses on three separate buildings: a large Victorian block built as a substantial domestic dwelling in 1885, a smaller single storey 1970s block with a new extension, and an as-yet unbuilt single storey block designed to passivhaus principles and using environmentally friendly materials. The implications of climate change have been considered for the three particular issues of comfort, construction, and water, as set out in the report on Design for Future Climate: opportunities for adaptation in the built environment (Gething, 2010). The adaptation designs aim to ensure each of the three very different buildings remains fit for purpose throughout the 21st century, continuing to provide a healthy environment for the children. A forth issue, the reduction of carbon and the mitigation of other negative environmental impacts of the construction work, is also a fundamental aim for the school and the project team. Detailed modelling of both the operational and embodied energy and carbon of the design options is therefore being carried out, in order that the whole life carbon costs of the adaptation design options may be minimised. The project has been funded by the Technology Strategy Board as part of the Design for Future Climates programme; the interdisciplinary team includes the designers working on the current school building projects and the school bursar, supported by researchers from the University of Cambridge Centre for Sustainable Development. It is hoped that lessons from the design process, as well as the solutions themselves, will be transferable to other buildings in similar climatic regions.

Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks

Cortical neurons receive balanced excitatory and inhibitory synaptic currents. Such a balance could be established and maintained in an experience-dependent manner by synaptic plasticity at inhibitory synapses. We show that this mechanism provides an explanation for the sparse firing patterns observed in response to natural stimuli and fits well with a recently observed interaction of excitatory and inhibitory receptive field plasticity. The introduction of inhibitory plasticity in suitable recurrent networks provides a homeostatic mechanism that leads to asynchronous irregular network states. Further, it can accommodate synaptic memories with activity patterns that become indiscernible from the background state but can be reactivated by external stimuli. Our results suggest an essential role of inhibitory plasticity in the formation and maintenance of functional cortical circuitry.