The learning dynamics of a universal approximator

The learning properties of a universal approximator, a normalized committee machine with adjustable biases, are studied for on-line back-propagation learning. Within a statistical mechanics framework, numerical studies show that this model has features which do not exist in previously studied two-layer network models without adjustable biases, e.g., attractive suboptimal symmetric phases even for realizable cases and noiseless data.

Finite-size effects in on-line learning of multilayer neural networks

We complement recent advances in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multi-layer networks by calculating fluctuations possessed by finite dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, increasing with the degree of symmetry of the initial conditions. In light of this, we include a term to stimulate asymmetry in the learning process, which typically also leads to a significant decrease in training time.

Globally optimal on-line learning rules

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

A variational approach to learning curves

We combine the replica approach from statistical physics with a variational approach to analyze learning curves analytically. We apply the method to Gaussian process regression. As a main result we derive approximative relations between empirical error measures, the generalization error and the posterior variance.

Efficient Bayesian inference for learning in the Ising linear perceptron and signal detection in CDMA

Efficient new Bayesian inference technique is employed for studying critical properties of the Ising linear perceptron and for signal detection in code division multiple access (CDMA). The approach is based on a recently introduced message passing technique for densely connected systems. Here we study both critical and non-critical regimes. Results obtained in the non-critical regime give rise to a highly efficient signal detection algorithm in the context of CDMA; while in the critical regime one observes a first-order transition line that ends in a continuous phase transition point. Finite size effects are also studied. © 2006 Elsevier B.V. All rights reserved.

Engaging tutors in using e-repositories for learning and teaching

Universities which set up online repositories for the management of learning and teaching resources commonly find that uptake is poor. Tutors are often reluctant to upload their materials to e-repositories, even though the same tutors are happy to upload resources to the virtual learning environment (e.g. Blackboard, Moodle, Sakai) and happy to upload their research papers to the university’s research publications repository. The paper reviews this phenomenon and suggests constructive ways in which tutors can be encouraged to engage with an e-repository. The authors have recently completed a major project “Developing Repositories at Worcester” which is part of a group of similar projects in the UK. The paper includes the feedback and the lessons learned from these projects, based on the publications and reports they have produced. They cover ways of embedding repository use into institutional working practice, and give examples of different types of repository designed to meet the needs of those using different kinds of learning and teaching resources. As well as this specific experience, the authors summarise some of the main findings from UK publications, in particular the December 2008 report of Joint Information Systems Committee: Good intentions: improving the evidence base in support of sharing learning materials and Online Innovation in Higher Education, Ron Cooke’s report to a UK government initiative on the future of Higher Education. The issues covered include the development of Web 2.0 style repositories rather than conventionally structured ones, the use of tags rather than metadata, the open resources initiative, the best use for conventional repositories, links to virtual learning environments, and the processes for the management and support of repositories within universities. In summary the paper presents an optimistic, constructive view of how to embed the use of e-repositories into the working practices of university tutors. Equally, the authors are aware of the considerable difficulties in making progress and are realistic about what can be achieved. The paper uses evidence and experience drawn from those working in this field to suggest a strategic vision in which the management of e-learning resources is productive, efficient and meets the needs of both tutors and their students.

A visual interactive modelling system as a learning tool

This paper presents a case study of the use of a visual interactive modelling system to investigate issues involved in the management of a hospital ward. Visual Interactive Modelling systems are seen to offer the learner the opportunity to explore operational management issues from a varied perspective and to provide an interactive system in which the learner receives feedback on the consequences of their actions. However to maximise the potential learning experience for a student requires the recognition that they require task structure which helps them to understand the concepts involved. These factors can be incorporated into the visual interactive model by providing an interface customised to guide the student through the experimentation. Recent developments of VIM systems in terms of their connectivity with the programming language Visual Basic facilitates this customisation.

Causality and synchronisation in complex systems with applications to neuroscience

This thesis presents an investigation, of synchronisation and causality, motivated by problems in computational neuroscience. The thesis addresses both theoretical and practical signal processing issues regarding the estimation of interdependence from a set of multivariate data generated by a complex underlying dynamical system. This topic is driven by a series of problems in neuroscience, which represents the principal background motive behind the material in this work. The underlying system is the human brain and the generative process of the data is based on modern electromagnetic neuroimaging methods . In this thesis, the underlying functional of the brain mechanisms are derived from the recent mathematical formalism of dynamical systems in complex networks. This is justified principally on the grounds of the complex hierarchical and multiscale nature of the brain and it offers new methods of analysis to model its emergent phenomena. A fundamental approach to study the neural activity is to investigate the connectivity pattern developed by the brain’s complex network. Three types of connectivity are important to study: 1) anatomical connectivity refering to the physical links forming the topology of the brain network; 2) effective connectivity concerning with the way the neural elements communicate with each other using the brain’s anatomical structure, through phenomena of synchronisation and information transfer; 3) functional connectivity, presenting an epistemic concept which alludes to the interdependence between data measured from the brain network. The main contribution of this thesis is to present, apply and discuss novel algorithms of functional connectivities, which are designed to extract different specific aspects of interaction between the underlying generators of the data. Firstly, a univariate statistic is developed to allow for indirect assessment of synchronisation in the local network from a single time series. This approach is useful in inferring the coupling as in a local cortical area as observed by a single measurement electrode. Secondly, different existing methods of phase synchronisation are considered from the perspective of experimental data analysis and inference of coupling from observed data. These methods are designed to address the estimation of medium to long range connectivity and their differences are particularly relevant in the context of volume conduction, that is known to produce spurious detections of connectivity. Finally, an asymmetric temporal metric is introduced in order to detect the direction of the coupling between different regions of the brain. The method developed in this thesis is based on a machine learning extensions of the well known concept of Granger causality. The thesis discussion is developed alongside examples of synthetic and experimental real data. The synthetic data are simulations of complex dynamical systems with the intention to mimic the behaviour of simple cortical neural assemblies. They are helpful to test the techniques developed in this thesis. The real datasets are provided to illustrate the problem of brain connectivity in the case of important neurological disorders such as Epilepsy and Parkinson’s disease. The methods of functional connectivity in this thesis are applied to intracranial EEG recordings in order to extract features, which characterize underlying spatiotemporal dynamics before during and after an epileptic seizure and predict seizure location and onset prior to conventional electrographic signs. The methodology is also applied to a MEG dataset containing healthy, Parkinson’s and dementia subjects with the scope of distinguishing patterns of pathological from physiological connectivity.

From national innovation systems to national innovation capacity:internationalisation and technology transfer perspectives

The last decade or so has witnessed the emergence of the national innovation system (NIS) phenomenon. Since then, many scholars have investigated NIS and its implementation in different countries. However, there are very few investigations into the relationship between the NIS of a country and its national innovation capacity. This paper aims to make a contribution in this area by examining the link that currently exists between these two topics. Whilst examining this relationship, we also explore internationalisation and technology transfer, being cognate areas that have been investigated during the same period. This follows our assertion that the link between NIS and national innovation capacity is the mechanism of internationalisation and technology transfer. The NIS approach was introduced in the late 1980s (see Freeman, 1987; Dosi et al., 1988) and further elaborated later (see Lundvall, 1992; Nelson, 1993; Edquist, 1997). In essence, a country?s NIS is a historically grown subsystem of the entire national economy consisting of organisations and institutions which play a major role in the innovative activity in the country. In the NIS approach, interactions within organisations as well as the interplay between organisations and institutions are of central importance. The NIS approach has been used to reveal the structure of the innovation processes and the main actors involved in them in industrialised and emerging countries. Although the national focus remains strong, it has been accompanied by studies seeking to analyse the notion of systems of innovation at an international level and at a sub-national scale (Archibugi et al., 1999). Dosi in the edition of Archibugi et al. (1999) argues that the general background of the discussion of national systems is the observation of non-random distributions across countries of: corporate capabilities; organisational forms; strategies; and ultimately revealed performances, in terms of production efficiency and inputs productivities, rates of innovation, rates of adoption/diffusion of innovation themselves, dynamics of market shares on the world markets, growth of income and employment. They also mention that there are several approaches to NIS. Nelson (1993) focuses upon the specificities of national institutions and policies supporting directly or indirectly innovation, diffusion and skills accumulation. Patel and Pavitt (1991) have stressed the links between the national patterns of technological accumulation and the competencies and innovative strategies of a few major national companies. Amable et al (1997) and Soskice (1993) and Zysman (1994) focus on the specifics of national institutions including, for example, the forms of organization, financial and labour markets, training institutions, forms of state intervention in the economy etc. However, the most common reference is by Lundvall (1992) who argues that the focus on the national level is associated with the fact that national economies vary according to their production system and their institutional framework and these differences are in turn strengthened by different historical experiences, language and culture. On the other hand, the national innovation capability consists of abilities to create and carry new technological possibilities through to economic practice. The term covers a wide range of activities from capability to invent to capability to innovate and to capability to improve existing technology beyond the original design parameters (Kim, 1997). The term innovation is often associated by many with technological change at international frontiers. However, technological capability is not the same as innovation capability. Technological capability refers to assimilation, use, adaptation, and change to existing technologies. It also enables the creation of new technologies and development of new products and processes in response to changing economic environments. It denotes operational command over knowledge (Kim, 1997). It is manifested not merely by the knowledge possessed, but, more important, by the uses to which that knowledge can be put and by the proficiency with which it is applied in the activities of investment and production and in the creation of new knowledge (Westphal et al., 1985). Therefore, the analytical framework that is used in this paper is based on the way a country derives from its NIS a national innovation capacity. There are two perspectives that are identified on this way. These are internationalisation and technology transfer. Even though NIS is not directly related to national innovation capacity, to achieve national innovation capacity from NIS, the country should have the ability for technology transfer. Technology transfer is a link between these two phenomena. On the other hand, internationalisation can be either the input or the output of the relationship between NIS and national innovation capability. If a company is investing in a country because of its national innovation capacity, this can be regarded as an input to the relationship between NIS and national innovation capacity. If this company is investigating the national innovation capacity of a country then, for its internationalisation, the national innovation capacity should be important, which in turn means this company is active in innovation and innovation is also an important success factor. The interrelationship between the investment of the company and the NIS of the country (assuming that the country is competent and competitive in technology transfer) will generate and improve that country?s national innovation capacity. This is the output of internationalisation from the relationship between NIS and national innovation capacity. When companies are evaluating whether to internationalise, they investigate certain factors in the countries in which they are considering to invest. The ability to transfer technology is dependent on ability to adopt a new technology and also on the learning derived from this technology. If countries wish to attract innovation related investment they need to show their ability to have a NIS and also the capability to transfer technology. Without the technology transfer capability, the NIS is not functioning. Therefore, companies that internationalise will investigate the factors common to NIS, technology transfer, and their business needs. Through this paper we will demonstrate this link though its mechanisms. Our research will be through extensive literature review and identifying relevant aspects of previous research carried out by the authors. It will investigate certain factors of different countries that are successful in attracting innovation related foreign direct investment. Through these, we will point out the factors that are important for the link and mechanisms of NIS and national innovation capability.

Bayesian online algorithms for learning in discrete Hidden Markov Models

We propose and analyze two different Bayesian online algorithms for learning in discrete Hidden Markov Models and compare their performance with the already known Baldi-Chauvin Algorithm. Using the Kullback-Leibler divergence as a measure of generalization we draw learning curves in simplified situations for these algorithms and compare their performances.

Visual data mining: integrating machine learning with information visualization

Today, the data available to tackle many scientific challenges is vast in quantity and diverse in nature. The exploration of heterogeneous information spaces requires suitable mining algorithms as well as effective visual interfaces. Most existing systems concentrate either on mining algorithms or on visualization techniques. Though visual methods developed in information visualization have been helpful, for improved understanding of a complex large high-dimensional dataset, there is a need for an effective projection of such a dataset onto a lower-dimension (2D or 3D) manifold. This paper introduces a flexible visual data mining framework which combines advanced projection algorithms developed in the machine learning domain and visual techniques developed in the information visualization domain. The framework follows Shneiderman’s mantra to provide an effective user interface. The advantage of such an interface is that the user is directly involved in the data mining process. We integrate principled projection methods, such as Generative Topographic Mapping (GTM) and Hierarchical GTM (HGTM), with powerful visual techniques, such as magnification factors, directional curvatures, parallel coordinates, billboarding, and user interaction facilities, to provide an integrated visual data mining framework. Results on a real life high-dimensional dataset from the chemoinformatics domain are also reported and discussed. Projection results of GTM are analytically compared with the projection results from other traditional projection methods, and it is also shown that the HGTM algorithm provides additional value for large datasets. The computational complexity of these algorithms is discussed to demonstrate their suitability for the visual data mining framework.

A comparison of integrated biomass to electricity systems

This thesis presents a comparison of integrated biomass to electricity systems on the basis of their efficiency, capital cost and electricity production cost. Four systems are evaluated: combustion to raise steam for a steam cycle; atmospheric gasification to produce fuel gas for a dual fuel diesel engine; pressurised gasification to produce fuel gas for a gas turbine combined cycle; and fast pyrolysis to produce pyrolysis liquid for a dual fuel diesel engine. The feedstock in all cases is wood in chipped form. This is the first time that all three thermochemical conversion technologies have been compared in a single, consistent evaluation.The systems have been modelled from the transportation of the wood chips through pretreatment, thermochemical conversion and electricity generation. Equipment requirements during pretreatment are comprehensively modelled and include reception, storage, drying and communication. The de-coupling of the fast pyrolysis system is examined, where the fast pyrolysis and engine stages are carried out at separate locations. Relationships are also included to allow learning effects to be studied. The modelling is achieved through the use of multiple spreadsheets where each spreadsheet models part of the system in isolation and the spreadsheets are combined to give the cost and performance of a whole system.The use of the models has shown that on current costs the combustion system remains the most cost-effective generating route, despite its low efficiency. The novel systems only produce lower cost electricity if learning effects are included, implying that some sort of subsidy will be required during the early development of the gasification and fast pyrolysis systems to make them competitive with the established combustion approach. The use of decoupling in fast pyrolysis systems is a useful way of reducing system costs if electricity is required at several sites because• a single pyrolysis site can be used to supply all the generators, offering economies of scale at the conversion step. Overall, costs are much higher than conventional electricity generating costs for fossil fuels, due mainly to the small scales used. Biomass to electricity opportunities remain restricted to niche markets where electricity prices are high or feed costs are very low. It is highly recommended that further work examines possibilities for combined beat and power which is suitable for small scale systems and could increase revenues that could reduce electricity prices.

A composite framework for the strategic alignment of information systems development

Information systems are corporate resources, therefore information systems development must be aligned with corporate strategy. This thesis proposes that effective strategic alignment of information systems requires information systems development, information systems planning and strategic management to be united. Literature in these areas is examined, breaching the academic boundaries which separate these areas, to contribute a synthesised approach to the strategic alignment of information systems development. Previous work in information systems planning has extended information systems development techniques, such as data modelling, into strategic planning activities, neglecting techniques of strategic management. Examination of strategic management in this thesis, identifies parallel trends in strategic management and information systems development; the premises of the learning school of strategic management are similar to those of soft systems approaches to information systems development. It is therefore proposed that strategic management can be supported by a soft systems approach. Strategic management tools and techniques frame individual views of a strategic situation; soft systems approaches can integrate these diverse views to explore the internal and external environments of an organisation. The information derived from strategic analysis justifies the need for an information system and provides a starting point for information systems development. This is demonstrated by a composite framework which enables each information system to be justified according to its direct contribution to corporate strategy. The proposed framework was developed through action research conducted in a number of organisations of varying types. This suggests that the framework can be widely used to support the strategic alignment of information systems development, thereby contributing to organisational success.

Investigating Turkey's national innovation and learning system

The national systems of innovation (NIS) approach focuses on the patterns and the determinants of innovation processes from the perspective of nation-states. This paper reports on continuing work on the application of an NIS model to the development of technological capability in Turkey. Initial assessment of the literature shows that there are a number of alternative conceptualisations of NIS. An attempt by the Government to identify a NIS for Turkey shows the main actors in the system but does not pay sufficient attention to the processes of interactions between agents within the system. An operational model should be capable of representing these processes and interactions and assessing the strengths and weaknesses of the NIS. For industrialising countries, it is also necessary to incorporate learning mechanisms into the model. Further, there are different levels of innovation and capability in different sectors which the national perspective may not reflect. This paper is arranged into three sections. The first briefly explains the basics of the national innovation and learning system. Although there is no single accepted definition of NIS, alternative definitions reviewed share some common characteristics. In the second section, an NIS model is applied to Turkey in order to identify the elements, which characterise the country’s NIS. This section explains knowledge flow and defines the relations between the actors within the system. The final section draws on the “from imitation to innovation” model apparently so successful in East Asia and assesses its applicability to Turkey. In assessing Turkey’s NIS, the focus is on the automotive and textile sectors.