Recognition on Finite Set of Events: Bayesian Analysis of Generalization Ability and Classification Tree Pruning

The problem of recognition on finite set of events is considered. The generalization ability of classifiers for this problem is studied within the Bayesian approach. The method for non-uniform prior distribution specification on recognition tasks is suggested. It takes into account the assumed degree of intersection between classes. The results of the analysis are applied for pruning of classification trees.

Permutation Tests for Classification

We introduce and explore an approach to estimating statistical significance of classification accuracy, which is particularly useful in scientific applications of machine learning where high dimensionality of the data and the small number of training examples render most standard convergence bounds too loose to yield a meaningful guarantee of the generalization ability of the classifier. Instead, we estimate statistical significance of the observed classification accuracy, or the likelihood of observing such accuracy by chance due to spurious correlations of the high-dimensional data patterns with the class labels in the given training set. We adopt permutation testing, a non-parametric technique previously developed in classical statistics for hypothesis testing in the generative setting (i.e., comparing two probability distributions). We demonstrate the method on real examples from neuroimaging studies and DNA microarray analysis and suggest a theoretical analysis of the procedure that relates the asymptotic behavior of the test to the existing convergence bounds.

Smart-phone based electrocardiogram wavelet decomposition and neural network classification

This paper discusses ECG classification after parametrizing the ECG waveforms in the wavelet domain. The aim of the work is to develop an accurate classification algorithm that can be used to diagnose cardiac beat abnormalities detected using a mobile platform such as smart-phones. Continuous time recurrent neural network classifiers are considered for this task. Records from the European ST-T Database are decomposed in the wavelet domain using discrete wavelet transform (DWT) filter banks and the resulting DWT coefficients are filtered and used as inputs for training the neural network classifier. Advantages of the proposed methodology are the reduced memory requirement for the signals which is of relevance to mobile applications as well as an improvement in the ability of the neural network in its generalization ability due to the more parsimonious representation of the signal to its inputs.

Computerized identification of motor complications in Parkinson's disease

Objective: To investigate whether spirography-based objective measures are able to effectively characterize the severity of unwanted symptom states (Off and dyskinesia) and discriminate them from motor state of healthy elderly subjects. Background: Sixty-five patients with advanced Parkinson’s disease (PD) and 10 healthy elderly (HE) subjects performed repeated assessments of spirography, using a touch screen telemetry device in their home environments. On inclusion, the patients were either treated with levodopa-carbidopa intestinal gel or were candidates for switching to this treatment. On each test occasion, the subjects were asked trace a pre-drawn Archimedes spiral shown on the screen, using an ergonomic pen stylus. The test was repeated three times and was performed using dominant hand. A clinician used a web interface which animated the spiral drawings, allowing him to observe different kinematic features, like accelerations and spatial changes, during the drawing process and to rate different motor impairments. Initially, the motor impairments of drawing speed, irregularity and hesitation were rated on a 0 (normal) to 4 (extremely severe) scales followed by marking the momentary motor state of the patient into 2 categories that is Off and Dyskinesia. A sample of spirals drawn by HE subjects was randomly selected and used in subsequent analysis. Methods: The raw spiral data, consisting of stylus position and timestamp, were processed using time series analysis techniques like discrete wavelet transform, approximate entropy and dynamic time warping in order to extract 13 quantitative measures for representing meaningful motor impairment information. A principal component analysis (PCA) was used to reduce the dimensions of the quantitative measures into 4 principal components (PC). In order to classify the motor states into 3 categories that is Off, HE and dyskinesia, a logistic regression model was used as a classifier to map the 4 PCs to the corresponding clinically assigned motor state categories. A stratified 10-fold cross-validation (also known as rotation estimation) was applied to assess the generalization ability of the logistic regression classifier to future independent data sets. To investigate mean differences of the 4 PCs across the three categories, a one-way ANOVA test followed by Tukey multiple comparisons was used. Results: The agreements between computed and clinician ratings were very good with a weighted area under the receiver operating characteristic curve (AUC) coefficient of 0.91. The mean PC scores were different across the three motor state categories, only at different levels. The first 2 PCs were good at discriminating between the motor states whereas the PC3 was good at discriminating between HE subjects and PD patients. The mean scores of PC4 showed a trend across the three states but without significant differences. The Spearman’s rank correlations between the first 2 PCs and clinically assessed motor impairments were as follows: drawing speed (PC1, 0.34; PC2, 0.83), irregularity (PC1, 0.17; PC2, 0.17), and hesitation (PC1, 0.27; PC2, 0.77). Conclusions: These findings suggest that spirography-based objective measures are valid measures of spatial- and time-dependent deficits and can be used to distinguish drug-related motor dysfunctions between Off and dyskinesia in PD. These measures can be potentially useful during clinical evaluation of individualized drug-related complications such as over- and under-medications thus maximizing the amount of time the patients spend in the On state.

Generalization by Computation Through Memory

Usually, generalization is considered as a function of learning from a set of examples. In present work on the basis of recent neural network assembly memory model (NNAMM), a biologically plausible 'grandmother' model for vision, where each separate memory unit itself can generalize, has been proposed. For such a generalization by computation through memory, analytical formulae and numerical procedure are found to calculate exactly the perfectly learned memory unit's generalization ability. The model's memory has complex hierarchical structure, can be learned from one example by a one-step process, and may be considered as a semi-representational one. A simple binary neural network for bell-shaped tuning is described.

Experiments on three-dimensional wire-frame object recognition

Some experimental results on the recognition of three-dimensional wire-frame objects are presented. In order to overcome the limitations of a recent model, which employs radial basis functions-based neural networks, we have proposed a hybrid learning system for object recognition, featuring: an optimization strategy (simulated annealing) in order to avoid local minima of an energy functional; and an appropriate choice of centers of the units. Further, in an attempt to achieve improved generalization ability, and to reduce the time for training, we invoke the principle of self-organization which utilises an unsupervised learning algorithm.

小波分析和人工神经网络在生态学研究中的应用

墨西哥帽子小波和Morlet小波在生态格局分析中的应用 本研究采用Monte Carlo方法，探讨了对小波分析的格局进行统计显著性检验的普遍方法。为了更好利用小波分析和了解两个常用小波--墨西哥帽子小波和Morlet小波在格局分析中的优缺点，用生态学研究中常见的4个模拟格局和东灵山辽东栎林的样带数据对这两个常用小波的特性进行了分析和比较。研究结果表明：墨西哥帽子小波能较好地分析样带中的斑块和间隙以及它们的位置信息，Morlet小波能较好地分析样带中尺度及其位置信息。不同的小波通常在尺度分析和斑块和间隙分析之间有平衡，所以最好的方法是结合两种小波的优点。小波分析在处理生态数据时，受所使用小波本身特性的制约。用墨西哥帽子小波进行格局分析时，小波能量谱的等值图上不同格局所对应的峰有可能相互重叠，当所分析的多尺度格局的规模差别不大时，所分析格局规模所对应的峰可能相互融合。这些小波能量谱的等值图上不同格局所对应峰的重叠和融合可能会导致格局分析量图上多个峰的相互融合和屏蔽。所以在使用小波分析做格局研究时，也应尽可能地结合小波能量谱和量图上的信息，以得到较全面和精确的格局分析信息。本研究的结论能为小波分析的应用提供指导。 应用二维小波分析对暖温带阔叶林辽东栎更新格局的研究 本研究介绍了一种二维网格空间数据分析方法一二维小波分析。该方法不仅能分析格局的等级结构，而且也能得到所分析结构的位置信息。小波系数等值图上不同格局规模的斑块和间隙可直接和不同尺度上的生态过程和生境条件相联系。小波方差从二维小波分析导出，小波方差可将四维的小波系数降至二维的小波方差函数，并量化所分析格局规模对整个格局的贡献。本研究用三个模拟格局分析了二维小波的特性及二维墨西哥帽子小波和Halo小波的特性。因为自身的特性，Halo小波比墨西哥帽子小波能提供更高的分辨率。本研究也将二维小波分析应用于暖温带阔叶林的辽东栎更新格局研究中，分析的结果表明：辽东栎的更新发生在辽东栎成树斑块和林窗斑块重叠区域。 用交互验证和独立验证来测试神人工经网络模拟水稻分檗动态的泛化能力 人工神经网络不是基于对所模拟过程的理解，而是依赖于对所分析数据的内部结构。所以人工神经网络通常被认为是经验模型而不能外推，而且在训练数据和验证数据的范围之外肯定不能精确地预测所模拟的过程。本研究通过对交互验证和独立验证神经网络模型性能的比较，测试了神经网络模型在预测水稻分檗动态方面的泛化能力。同时，也对几种提高神经网络泛化能力的技术进行了比较。研究结果表明：在训练数据的变量范围内，神经网络在预测水稻分檗动态方面具有泛化能力。较少的训练数据样本导致了对训练数据过度吻合的和不具泛化能力的神经网络。要能使神经网络在预测水稻分檗动态方面具有泛化能力，训练数据的样本量至少9倍于神经网络的权值数目。当神经网络有多个输入变量且训练数据不足以保证神经网络的泛化能力时，建议在训练之前，采用主成分分析、对应分析及类似技术压缩输入变量的个数。在压缩输入变量的个数之后，如果训练数据的样本量还不足以保证神经网络的泛化能力，就应采用提高神经网络泛化能力的技木，如：jittering和强制训练停止等，特别是神经网络与机理模型的复合模型。因为神经网络的泛化能力问题具有普遍性，所以我们的研究结论不只是适用于水稻分檗动态的预测，也适用于其它的农业和生态神经网络模型。

Beyond dataset bias: Multi-task unaligned shared knowledge transfer

Many visual datasets are traditionally used to analyze the performance of different learning techniques. The evaluation is usually done within each dataset, therefore it is questionable if such results are a reliable indicator of true generalization ability. We propose here an algorithm to exploit the existing data resources when learning on a new multiclass problem. Our main idea is to identify an image representation that decomposes orthogonally into two subspaces: a part specific to each dataset, and a part generic to, and therefore shared between, all the considered source sets. This allows us to use the generic representation as un-biased reference knowledge for a novel classification task. By casting the method in the multi-view setting, we also make it possible to use different features for different databases. We call the algorithm MUST, Multitask Unaligned Shared knowledge Transfer. Through extensive experiments on five public datasets, we show that MUST consistently improves the cross-datasets generalization performance. © 2013 Springer-Verlag.

Paraphrastic language models and combination with neural network language models

In natural languages multiple word sequences can represent the same underlying meaning. Only modelling the observed surface word sequence can result in poor context coverage, for example, when using n-gram language models (LM). To handle this issue, paraphrastic LMs were proposed in previous research and successfully applied to a US English conversational telephone speech transcription task. In order to exploit the complementary characteristics of paraphrastic LMs and neural network LMs (NNLM), the combination between the two is investigated in this paper. To investigate paraphrastic LMs' generalization ability to other languages, experiments are conducted on a Mandarin Chinese broadcast speech transcription task. Using a paraphrastic multi-level LM modelling both word and phrase sequences, significant error rate reductions of 0.9% absolute (9% relative) and 0.5% absolute (5% relative) were obtained over the baseline n-gram and NNLM systems respectively, after a combination with word and phrase level NNLMs. © 2013 IEEE.

多机器人动态编队的强化学习算法研究

在人工智能领域中 ,强化学习理论由于其自学习性和自适应性的优点而得到了广泛关注 随着分布式人工智能中多智能体理论的不断发展 ,分布式强化学习算法逐渐成为研究的重点 首先介绍了强化学习的研究状况 ,然后以多机器人动态编队为研究模型 ,阐述应用分布式强化学习实现多机器人行为控制的方法 应用SOM神经网络对状态空间进行自主划分 ,以加快学习速度 ;应用BP神经网络实现强化学习 ,以增强系统的泛化能力 ;并且采用内、外两个强化信号兼顾机器人的个体利益及整体利益 为了明确控制任务 ,系统使用黑板通信方式进行分层控制 最后由仿真实验证明该方法的有效性

O desenvolvimento do pensamento algébrico de alunos do 4º ano de escolaridade : uma experiência de ensino

Tese de doutoramento, Educação (Didática da Matemática), Universidade de Lisboa, Instituto de Educação, 2014

On the utility of input selection and pruning for financial distress prediction models

Analyzes the use of linear and neural network models for financial distress classification, with emphasis on the issues of input variable selection and model pruning. A data-driven method for selecting input variables (financial ratios, in this case) is proposed. A case study involving 60 British firms in the period 1997-2000 is used for illustration. It is shown that the use of the Optimal Brain Damage pruning technique can considerably improve the generalization ability of a neural model. Moreover, the set of financial ratios obtained with the proposed selection procedure is shown to be an appropriate alternative to the ratios usually employed by practitioners.

Identificação não linear usando uma rede fuzzy wavelet neural network modificada

In last decades, neural networks have been established as a major tool for the identification of nonlinear systems. Among the various types of networks used in identification, one that can be highlighted is the wavelet neural network (WNN). This network combines the characteristics of wavelet multiresolution theory with learning ability and generalization of neural networks usually, providing more accurate models than those ones obtained by traditional networks. An extension of WNN networks is to combine the neuro-fuzzy ANFIS (Adaptive Network Based Fuzzy Inference System) structure with wavelets, leading to generate the Fuzzy Wavelet Neural Network - FWNN structure. This network is very similar to ANFIS networks, with the difference that traditional polynomials present in consequent of this network are replaced by WNN networks. This paper proposes the identification of nonlinear dynamical systems from a network FWNN modified. In the proposed structure, functions only wavelets are used in the consequent. Thus, it is possible to obtain a simplification of the structure, reducing the number of adjustable parameters of the network. To evaluate the performance of network FWNN with this modification, an analysis of network performance is made, verifying advantages, disadvantages and cost effectiveness when compared to other existing FWNN structures in literature. The evaluations are carried out via the identification of two simulated systems traditionally found in the literature and a real nonlinear system, consisting of a nonlinear multi section tank. Finally, the network is used to infer values of temperature and humidity inside of a neonatal incubator. The execution of such analyzes is based on various criteria, like: mean squared error, number of training epochs, number of adjustable parameters, the variation of the mean square error, among others. The results found show the generalization ability of the modified structure, despite the simplification performed

Utilização de RNA´s na construção do diagrama de vida constante de probabilidade de materiais compósitos

The static and cyclic assays are common to test materials in structures.. For cycling assays to assess the fatigue behavior of the material and thereby obtain the S-N curves and these are used to construct the diagrams of living constant. However, these diagrams, when constructed with small amounts of S-N curves underestimate or overestimate the actual behavior of the composite, there is increasing need for more testing to obtain more accurate results. Therewith, , a way of reducing costs is the statistical analysis of the fatigue behavior. The aim of this research was evaluate the probabilistic fatigue behavior of composite materials. The research was conducted in three parts. The first part consists of associating the equation of probability Weilbull equations commonly used in modeling of composite materials S-N curve, namely the exponential equation and power law and their generalizations. The second part was used the results obtained by the equation which best represents the S-N curves of probability and trained a network to the modular 5% failure. In the third part, we carried out a comparative study of the results obtained using the nonlinear model by parts (PNL) with the results of a modular network architecture (MN) in the analysis of fatigue behavior. For this we used a database of ten materials obtained from the literature to assess the ability of generalization of the modular network as well as its robustness. From the results it was found that the power law of probability generalized probabilistic behavior better represents the fatigue and composites that although the generalization ability of the MN that was not robust training with 5% failure rate, but for values mean the MN showed more accurate results than the PNL model

Artificial neural networks and clustering techniques applied in the reconfiguration of distribution systems

One objective of the feeder reconfiguration problem in distribution systems is to minimize the power losses for a specific load. For this problem, mathematical modeling is a nonlinear mixed integer problem that is generally hard to solve. This paper proposes an algorithm based on artificial neural network theory. In this context, clustering techniques to determine the best training set for a single neural network with generalization ability are also presented. The proposed methodology was employed for solving two electrical systems and presented good results. Moreover, the methodology can be employed for large-scale systems in real-time environment.