A Theory of Networks for Appxoimation and Learning

Learning an input-output mapping from a set of examples, of the type that many neural networks have been constructed to perform, can be regarded as synthesizing an approximation of a multi-dimensional function, that is solving the problem of hypersurface reconstruction. From this point of view, this form of learning is closely related to classical approximation techniques, such as generalized splines and regularization theory. This paper considers the problems of an exact representation and, in more detail, of the approximation of linear and nolinear mappings in terms of simpler functions of fewer variables. Kolmogorov's theorem concerning the representation of functions of several variables in terms of functions of one variable turns out to be almost irrelevant in the context of networks for learning. We develop a theoretical framework for approximation based on regularization techniques that leads to a class of three-layer networks that we call Generalized Radial Basis Functions (GRBF), since they are mathematically related to the well-known Radial Basis Functions, mainly used for strict interpolation tasks. GRBF networks are not only equivalent to generalized splines, but are also closely related to pattern recognition methods such as Parzen windows and potential functions and to several neural network algorithms, such as Kanerva's associative memory, backpropagation and Kohonen's topology preserving map. They also have an interesting interpretation in terms of prototypes that are synthesized and optimally combined during the learning stage. The paper introduces several extensions and applications of the technique and discusses intriguing analogies with neurobiological data.

Extensions of a Theory of Networks for Approximation and Learning: Outliers and Negative Examples

Learning an input-output mapping from a set of examples can be regarded as synthesizing an approximation of a multi-dimensional function. From this point of view, this form of learning is closely related to regularization theory. In this note, we extend the theory by introducing ways of dealing with two aspects of learning: learning in the presence of unreliable examples and learning from positive and negative examples. The first extension corresponds to dealing with outliers among the sparse data. The second one corresponds to exploiting information about points or regions in the range of the function that are forbidden.

Stability of time-delay systems via Lyapunov functions

In this paper, a Lyapunov function candidate is introduced for multivariable systems with inner delays, without assuming a priori stability for the nondelayed subsystem. By using this Lyapunov function, a controller is deduced. Such a controller utilizes an input-output description of the original system, a circumstance that facilitates practical applications of the proposed approach.

Spatiotemporal Gesture Segmentation

Spotting patterns of interest in an input signal is a very useful task in many different fields including medicine, bioinformatics, economics, speech recognition and computer vision. Example instances of this problem include spotting an object of interest in an image (e.g., a tumor), a pattern of interest in a time-varying signal (e.g., audio analysis), or an object of interest moving in a specific way (e.g., a human's body gesture). Traditional spotting methods, which are based on Dynamic Time Warping or hidden Markov models, use some variant of dynamic programming to register the pattern and the input while accounting for temporal variation between them. At the same time, those methods often suffer from several shortcomings: they may give meaningless solutions when input observations are unreliable or ambiguous, they require a high complexity search across the whole input signal, and they may give incorrect solutions if some patterns appear as smaller parts within other patterns. In this thesis, we develop a framework that addresses these three problems, and evaluate the framework's performance in spotting and recognizing hand gestures in video. The first contribution is a spatiotemporal matching algorithm that extends the dynamic programming formulation to accommodate multiple candidate hand detections in every video frame. The algorithm finds the best alignment between the gesture model and the input, and simultaneously locates the best candidate hand detection in every frame. This allows for a gesture to be recognized even when the hand location is highly ambiguous. The second contribution is a pruning method that uses model-specific classifiers to reject dynamic programming hypotheses with a poor match between the input and model. Pruning improves the efficiency of the spatiotemporal matching algorithm, and in some cases may improve the recognition accuracy. The pruning classifiers are learned from training data, and cross-validation is used to reduce the chance of overpruning. The third contribution is a subgesture reasoning process that models the fact that some gesture models can falsely match parts of other, longer gestures. By integrating subgesture reasoning the spotting algorithm can avoid the premature detection of a subgesture when the longer gesture is actually being performed. Subgesture relations between pairs of gestures are automatically learned from training data. The performance of the approach is evaluated on two challenging video datasets: hand-signed digits gestured by users wearing short sleeved shirts, in front of a cluttered background, and American Sign Language (ASL) utterances gestured by ASL native signers. The experiments demonstrate that the proposed method is more accurate and efficient than competing approaches. The proposed approach can be generally applied to alignment or search problems with multiple input observations, that use dynamic programming to find a solution.

Fuzzy ARTMAP, Slow Learning and Probability Estimation

A nonparametric probability estimation procedure using the fuzzy ARTMAP neural network is here described. Because the procedure does not make a priori assumptions about underlying probability distributions, it yields accurate estimates on a wide variety of prediction tasks. Fuzzy ARTMAP is used to perform probability estimation in two different modes. In a 'slow-learning' mode, input-output associations change slowly, with the strength of each association computing a conditional probability estimate. In 'max-nodes' mode, a fixed number of categories are coded during an initial fast learning interval, and weights are then tuned by slow learning. Simulations illustrate system performance on tasks in which various numbers of clusters in the set of input vectors mapped to a given class.

Eliminating Redundant Training Data Using Unsupervised Clustering Techniques

Training data for supervised learning neural networks can be clustered such that the input/output pairs in each cluster are redundant. Redundant training data can adversely affect training time. In this paper we apply two clustering algorithms, ART2 -A and the Generalized Equality Classifier, to identify training data clusters and thus reduce the training data and training time. The approach is demonstrated for a high dimensional nonlinear continuous time mapping. The demonstration shows six-fold decrease in training time at little or no loss of accuracy in the handling of evaluation data.

Automatic verification of linear controller software

© 2015 IEEE.We consider the problem of verification of software implementations of linear time-invariant controllers. Commonly, different implementations use different representations of the controller's state, for example due to optimizations in a third-party code generator. To accommodate this variation, we exploit input-output controller specification captured by the controller's transfer function and show how to automatically verify correctness of C code controller implementations using a Frama-C/Why3/Z3 toolchain. Scalability of the approach is evaluated using randomly generated controller specifications of realistic size.

Computational modeling techniques for reliability of electronic components on printed circuit boards

This paper describes modeling technology and its use in providing data governing the assembly and subsequent reliability of electronic chip components on printed circuit boards (PCBs). Products, such as mobile phones, camcorders, intelligent displays, etc., are changing at a tremendous rate where newer technologies are being applied to satisfy the demands for smaller products with increased functionality. At ever decreasing dimensions, and increasing number of input/output connections, the design of these components, in terms of dimensions and materials used, is playing a key role in determining the reliability of the final assembly. Multiphysics modeling techniques are being adopted to predict a range of interacting physics-based phenomena associated with the manufacturing process. For example, heat transfer, solidification, marangoni fluid flow, void movement, and thermal-stress. The modeling techniques used are based on finite volume methods that are conservative and take advantage of being able to represent the physical domain using an unstructured mesh. These techniques are also used to provide data on thermal induced fatigue which is then mapped into product lifetime predictions.

Differential amplifier with improved gain-accuracy and linearity

A novel circuit design technique is presented which improves gain-accuracy and linearity in differential amplifiers. The technique employs negative impedance compensation and results demonstrate a significant performance improvement in precision, lowering sensitivity, and wide dynamic range. A theoretical underpinning is given together with the results of a demonstrator differential input/output amplifier with gain of 12 dB. The simulation results show that, with the novel method, both the gain-accuracy and linearity can be improved greatly. Especially, the linearity improvement in IMD can get to more than 23 dB with a required gain.

Investigating the cost Performance of UK Credit Unions using Radial and Non-Radial Efficiency Measures

This paper examines the relative efficiency of UK credit unions. Radial and non-radial measures of input cost efficiency plus associated scale efficiency measures are computed for a selection of input output specifications. Both measures highlighted that UK credit unions have considerable scope for efficiency gains. It was mooted that the documented high levels of inefficiency may be indicative of the fact that credit unions, based on clearly defined and non-overlapping common bonds, are not in competition with each other for market share. Credit unions were also highlighted as suffering from a considerable degree of scale inefficiency with the majority of scale inefficient credit unions subject to decreasing returns to scale. The latter aspect highlights that the UK Government's goal of larger credit unions must be accompanied by greater regulatory freedom if inefficiency is to be avoided. One of the advantages of computing non-radial measures is that an insight into potential over- or under-expenditure on specific inputs can be obtained through a comparison of the non-radial measure of efficiency with the associated radial measure. Two interesting findings emerged, the first that UK credit unions over-spend on dividend payments and the second that they under-spend on labour costs.

Phosphorus Retention in a Constructed Wetland System used to Treat Dairy Wastewater

The aim of this study was to develop an input/output mass balance to predict phosphorus retention in a five pond constructed wetland system (CWS) at Greenmount Farm, County Antrim, Northern Ireland. The mass balance was created using 14-months of flow data collected at inflow and outflow points on a weekly basis. Balance outputs were correlated with meteorological parameters, such as daily air temperature and hydrological flow, recorded daily onsite. The mass balance showed that phosphorus retention within the system exceeded phosphorus release, illustrating the success of constructed wetland systems to remove nutrients from agricultural effluent from a dairy farm. Pond 5 showed the greatest relative retention of 86%. Comparison of retention and mean air temperature highlighted a striking difference in trends between up-gradient and down-gradient ponds, with Ponds 1 and 2 displaying a positive quadratic relationship and ponds 3 through 5 displaying a negative quadratic relationship.

Molecular Logic Gates and Luminescent Sensors Based on Photoinduced Electron Transfer

The competition between Photoinduced electron transfer (PET) and other de-excitation pathways such as fluorescence and phosphorescence can be controlled within designed molecular structures. Depending on the particular design, the resulting optical output is thus a function of various inputs such as ion concentration and excitation light dose. Once digitized into binary code, these input-output patterns can be interpreted according to Boolean logic. The single-input logic types of YES and NOT cover simple sensors and the double- (or higher-) input logic types represent other gates such as AND and OR. The logic-based arithmetic processors such as half-adders and half-subtractors are also featured. Naturally, a principal application of the more complex gates is in multi-sensing contexts.

Parallel Programming of General-Purpose Programs Using Task-Based Programming Models

The prevalence of multicore processors is bound to drive most kinds of software development towards parallel programming. To limit the difficulty and overhead of parallel software design and maintenance, it is crucial that parallel programming models allow an easy-to-understand, concise and dense representation of parallelism. Parallel programming models such as Cilk++ and Intel TBBs attempt to offer a better, higher-level abstraction for parallel programming than threads and locking synchronization. It is not straightforward, however, to express all patterns of parallelism in these models. Pipelines are an important parallel construct, although difficult to express in Cilk and TBBs in a straightfor- ward way, not without a verbose restructuring of the code. In this paper we demonstrate that pipeline parallelism can be easily and concisely expressed in a Cilk-like language, which we extend with input, output and input/output dependency types on procedure arguments, enforced at runtime by the scheduler. We evaluate our implementation on real applications and show that our Cilk-like scheduler, extended to track and enforce these dependencies has performance comparable to Cilk++.

Discrete cosine transform generator for VLSI synthesis

A generator for the automated design of Discrete Cosine Transform (DCT) cores is presented. This can be used to rapidly create silicon circuits from a high level specification. These compare very favourably with existing designs. The DCT cores produced are scaleable in terms of point size as well as input/output and coefficient wordlengths. This provides a high degree of flexibility. An example, 8-point 1D DCT design, produced occupies less than 0.92 mm when implemented in a 0.35µ double level metal CMOS technology. This can be clocked at a rate of 100MHz.

A molecular photoionic AND gate based on fluorescent signalling

MOLECULES that perform logic operations are prerequisites for molecular information processing and computation. We and others have previously reported receptor molecules that can be considered to perform simple logic operations by coupling ionic bonding or more complex molecular-recognition processes with photonic (fluorescence) signals: in these systems, chemical binding (the 'input') results in a change in fluorescence intensity (the 'output') from the receptor. Here we describe a receptor (molecule (1) in Fig. 1) that operates as a logic device with two input channels: the fluorescence signal depends on whether the molecule binds hydrogen ions, sodium ions or both. The input/output characteristics of this molecular device correspond to those of an AND gate.