Fuzzy ARTMAP: A Neural Network Architecture for Incremental Supervised Learning of Analog Multidimensional Maps

A new neural network architecture is introduced for incremental supervised learning of recognition categories and multidimensional maps in response to arbitrary sequences of analog or binary input vectors. The architecture, called Fuzzy ARTMAP, achieves a synthesis of fuzzy logic and Adaptive Resonance Theory (ART) neural networks by exploiting a close formal similarity between the computations of fuzzy subsethood and ART category choice, resonance, and learning. Fuzzy ARTMAP also realizes a new Minimax Learning Rule that conjointly minimizes predictive error and maximizes code compression, or generalization. This is achieved by a match tracking process that increases the ART vigilance parameter by the minimum amount needed to correct a predictive error. As a result, the system automatically learns a minimal number of recognition categories, or "hidden units", to met accuracy criteria. Category proliferation is prevented by normalizing input vectors at a preprocessing stage. A normalization procedure called complement coding leads to a symmetric theory in which the MIN operator (Λ) and the MAX operator (v) of fuzzy logic play complementary roles. Complement coding uses on-cells and off-cells to represent the input pattern, and preserves individual feature amplitudes while normalizing the total on-cell/off-cell vector. Learning is stable because all adaptive weights can only decrease in time. Decreasing weights correspond to increasing sizes of category "boxes". Smaller vigilance values lead to larger category boxes. Improved prediction is achieved by training the system several times using different orderings of the input set. This voting strategy can also be used to assign probability estimates to competing predictions given small, noisy, or incomplete training sets. Four classes of simulations illustrate Fuzzy ARTMAP performance as compared to benchmark back propagation and genetic algorithm systems. These simulations include (i) finding points inside vs. outside a circle; (ii) learning to tell two spirals apart; (iii) incremental approximation of a piecewise continuous function; and (iv) a letter recognition database. The Fuzzy ARTMAP system is also compared to Salzberg's NGE system and to Simpson's FMMC system.

The lung cancer exercise training study: a randomized trial of aerobic training, resistance training, or both in postsurgical lung cancer patients: rationale and design.

BACKGROUND: The Lung Cancer Exercise Training Study (LUNGEVITY) is a randomized trial to investigate the efficacy of different types of exercise training on cardiorespiratory fitness (VO2peak), patient-reported outcomes, and the organ components that govern VO2peak in post-operative non-small cell lung cancer (NSCLC) patients. METHODS/DESIGN: Using a single-center, randomized design, 160 subjects (40 patients/study arm) with histologically confirmed stage I-IIIA NSCLC following curative-intent complete surgical resection at Duke University Medical Center (DUMC) will be potentially eligible for this trial. Following baseline assessments, eligible participants will be randomly assigned to one of four conditions: (1) aerobic training alone, (2) resistance training alone, (3) the combination of aerobic and resistance training, or (4) attention-control (progressive stretching). The ultimate goal for all exercise training groups will be 3 supervised exercise sessions per week an intensity above 70% of the individually determined VO2peak for aerobic training and an intensity between 60 and 80% of one-repetition maximum for resistance training, for 30-45 minutes/session. Progressive stretching will be matched to the exercise groups in terms of program length (i.e., 16 weeks), social interaction (participants will receive one-on-one instruction), and duration (30-45 mins/session). The primary study endpoint is VO2peak. Secondary endpoints include: patient-reported outcomes (PROs) (e.g., quality of life, fatigue, depression, etc.) and organ components of the oxygen cascade (i.e., pulmonary function, cardiac function, skeletal muscle function). All endpoints will be assessed at baseline and postintervention (16 weeks). Substudies will include genetic studies regarding individual responses to an exercise stimulus, theoretical determinants of exercise adherence, examination of the psychological mediators of the exercise - PRO relationship, and exercise-induced changes in gene expression. DISCUSSION: VO2peak is becoming increasingly recognized as an outcome of major importance in NSCLC. LUNGEVITY will identify the optimal form of exercise training for NSCLC survivors as well as provide insight into the physiological mechanisms underlying this effect. Overall, this study will contribute to the establishment of clinical exercise therapy rehabilitation guidelines for patients across the entire NSCLC continuum. TRIAL REGISTRATION: NCT00018255.

Rationale and design of the Exercise Intensity Trial (EXCITE): A randomized trial comparing the effects of moderate versus moderate to high-intensity aerobic training in women with operable breast cancer.

BACKGROUND: The Exercise Intensity Trial (EXcITe) is a randomized trial to compare the efficacy of supervised moderate-intensity aerobic training to moderate to high-intensity aerobic training, relative to attention control, on aerobic capacity, physiologic mechanisms, patient-reported outcomes, and biomarkers in women with operable breast cancer following the completion of definitive adjuvant therapy. METHODS/DESIGN: Using a single-center, randomized design, 174 postmenopausal women (58 patients/study arm) with histologically confirmed, operable breast cancer presenting to Duke University Medical Center (DUMC) will be enrolled in this trial following completion of primary therapy (including surgery, radiation therapy, and chemotherapy). After baseline assessments, eligible participants will be randomized to one of two supervised aerobic training interventions (moderate-intensity or moderate/high-intensity aerobic training) or an attention-control group (progressive stretching). The aerobic training interventions will include 150 mins.wk⁻¹ of supervised treadmill walking per week at an intensity of 60%-70% (moderate-intensity) or 60% to 100% (moderate to high-intensity) of the individually determined peak oxygen consumption (VO₂peak) between 20-45 minutes/session for 16 weeks. The progressive stretching program will be consistent with the exercise interventions in terms of program length (16 weeks), social interaction (participants will receive one-on-one instruction), and duration (20-45 mins/session). The primary study endpoint is VO₂peak, as measured by an incremental cardiopulmonary exercise test. Secondary endpoints include physiologic determinants that govern VO₂peak, patient-reported outcomes, and biomarkers associated with breast cancer recurrence/mortality. All endpoints will be assessed at baseline and after the intervention (16 weeks). DISCUSSION: EXCITE is designed to investigate the intensity of aerobic training required to induce optimal improvements in VO₂peak and other pertinent outcomes in women who have completed definitive adjuvant therapy for operable breast cancer. Overall, this trial will inform and refine exercise guidelines to optimize recovery in breast and other cancer survivors following the completion of primary cytotoxic therapy. TRIAL REGISTRATION: NCT01186367.

Graduate Attitudes towards Vocational Training

Introduction: Vocational training (VT) is a mandatory requirement for all UK dental graduates prior to entering NHS practice. The VT period provides structured, supervised experience supported by study days and interaction with peers. It is not compulsory for Irish dental graduates working in either Ireland or the UK to undertake VT but yet a proportion voluntarily do so each year.

Objectives: This study was designed to explore the choices made by Irish dental graduates. It aimed to record any benefits of VT and its impact upon future career choices.

Method: A self-completion questionnaire was developed and piloted before being circulated electronically to recent dental graduates from University College Cork. After collecting demographic information respondents were asked to indicate if they pursued vocational training on graduation, give their perception of their post-graduation experience, describe their current work profile and detail any formal postgraduate studies.

Results: 35% of respondents opted to undertake VT and 79% did so in the UK. Those who completed VT regarded it as a very positive experience with benefits including: working in a positive learning environment, help on demand and interaction with peers. Of those who chose VT, 49% have pursued some form of further formal postgraduate study as compared to 40% of those who did not. All of the respondents who completed VT indicated they would recommend it to current Irish graduates. The majority of those who took up an associate position immediately after graduation reported that this was beneficial but up to three quarters would recommend current graduates undertake VT and 45% would now chose to do so themselves.

Conclusions: Increasing numbers of Irish graduates are moving to the UK to undertake VT and they find it a beneficial experience. In addition, those who undertook VT were more likely to undertake formal postgraduate study.

A survey of perceived training differences between ophthalmology residents in Hong Kong and China.

Background: To study the differences in ophthalmology resident training between China and the Hong Kong Special Administrative Region (HKSAR).Methods: Training programs were selected from among the largest and best-known teaching hospitals. Ophthalmology residents were sent an anonymous 48-item questionnaire by mail. Work satisfaction, time allocation between training activities and volume of surgery performed were determined.Results: 50/75 residents (66.7 %) from China and 20/26 (76.9 %) from HKSAR completed the survey. Age (28.9 ± 2.5 vs. 30.2 ± 2.9 years, p = 0.15) and number of years in training (3.4 ± 1.6 vs. 2.8 ± 1.5, p = 0.19) were comparable between groups. The number of cataract procedures performed by HKSAR trainees (extra-capsular, median 80.0, quartile range: 30.0, 100.0; phacoemulsification, median: 20.0, quartile range: 0.0, 100.0) exceeded that for Chinese residents (extra-capsular: median = 0, p < 0.0001; phacoemulsification: median = 0, p < 0.0001). Chinese trainees spent more time completing medical charts (>50 % of time on charts: 62.5 % versus 5.3 %, p < 0.0001) and received less supervision (≥90 % of training supervised: 4.4 % versus 65 %, p < 0.0001). Chinese residents were more likely to feel underpaid (96.0 % vs. 31.6 %, p < 0.0001) and hoped their children would not practice medicine (69.4 % vs. 5.0 %, p = 0.0001) compared HKSAR residents.Conclusions: In this study, ophthalmology residents in China report strikingly less surgical experience and supervision, and lower satisfaction than HKSAR residents. The HKSAR model of hands-on resident training might be useful in improving the low cataract surgical rate in China.

Exercise training reduces fatty acid availability and improves the insulin sensitivity of glucose metabolism

Aims/hypothesis - It is not known whether the beneficial effects of exercise training on insulin sensitivity are due to changes in hepatic and peripheral insulin sensitivity or whether the changes in insulin sensitivity can be explained by adaptive changes in fatty acid metabolism, changes in visceral fat or changes in liver and muscle triacylglycerol content. We investigated the effects of 6 weeks of supervised exercise in sedentary men on these variables. Subjects and methods - We randomised 17 sedentary overweight male subjects (age 50 ± 2.6 years, BMI 27.6 ± 0.5 kg/m2) to a 6-week exercise programme (n = 10) or control group (n = 7). The insulin sensitivity of palmitic acid production rate (Ra), glycerol Ra, endogenous glucose Ra (EGP), glucose uptake and glucose metabolic clearance rate were measured at 0 and 6 weeks with a two-step hyperinsulinaemic–euglycaemic clamp [step 1, 0.3 (low dose); step 2, 1.5 (high dose) mU kg−1 min−1]. In the exercise group subjects were studied >72 h after the last training session. Liver and skeletal muscle triacylglycerol content was measured by magnetic resonance spectroscopy and visceral adipose tissue by cross-sectional computer tomography scanning. Results - After 6 weeks, fasting glycerol, palmitic acid Ra (p = 0.003, p = 0.042) and NEFA concentration (p = 0.005) were decreased in the exercise group with no change in the control group. The effects of low-dose insulin on EGP and of high-dose insulin on glucose uptake and metabolic clearance rate were enhanced in the exercise group but not in the control group (p = 0.026; p = 0.007 and p = 0.04). There was no change in muscle triacylglycerol and liver fat in either group. Conclusions/interpretation - Decreased availability of circulating NEFA may contribute to the observed improvement in the insulin sensitivity of EGP and glucose uptake following 6 weeks of moderate exercise.

Training deep convolutional architectures for vision

Les tâches de vision artificielle telles que la reconnaissance d’objets demeurent irrésolues à ce jour. Les algorithmes d’apprentissage tels que les Réseaux de Neurones Artificiels (RNA), représentent une approche prometteuse permettant d’apprendre des caractéristiques utiles pour ces tâches. Ce processus d’optimisation est néanmoins difficile. Les réseaux profonds à base de Machine de Boltzmann Restreintes (RBM) ont récemment été proposés afin de guider l’extraction de représentations intermédiaires, grâce à un algorithme d’apprentissage non-supervisé. Ce mémoire présente, par l’entremise de trois articles, des contributions à ce domaine de recherche. Le premier article traite de la RBM convolutionelle. L’usage de champs réceptifs locaux ainsi que le regroupement d’unités cachées en couches partageant les même paramètres, réduit considérablement le nombre de paramètres à apprendre et engendre des détecteurs de caractéristiques locaux et équivariant aux translations. Ceci mène à des modèles ayant une meilleure vraisemblance, comparativement aux RBMs entraînées sur des segments d’images. Le deuxième article est motivé par des découvertes récentes en neurosciences. Il analyse l’impact d’unités quadratiques sur des tâches de classification visuelles, ainsi que celui d’une nouvelle fonction d’activation. Nous observons que les RNAs à base d’unités quadratiques utilisant la fonction softsign, donnent de meilleures performances de généralisation. Le dernière article quand à lui, offre une vision critique des algorithmes populaires d’entraînement de RBMs. Nous montrons que l’algorithme de Divergence Contrastive (CD) et la CD Persistente ne sont pas robustes : tous deux nécessitent une surface d’énergie relativement plate afin que leur chaîne négative puisse mixer. La PCD à "poids rapides" contourne ce problème en perturbant légèrement le modèle, cependant, ceci génère des échantillons bruités. L’usage de chaînes tempérées dans la phase négative est une façon robuste d’adresser ces problèmes et mène à de meilleurs modèles génératifs.

Understanding deep architectures and the effect of unsupervised pre-training

Cette thèse porte sur une classe d'algorithmes d'apprentissage appelés architectures profondes. Il existe des résultats qui indiquent que les représentations peu profondes et locales ne sont pas suffisantes pour la modélisation des fonctions comportant plusieurs facteurs de variation. Nous sommes particulièrement intéressés par ce genre de données car nous espérons qu'un agent intelligent sera en mesure d'apprendre à les modéliser automatiquement; l'hypothèse est que les architectures profondes sont mieux adaptées pour les modéliser. Les travaux de Hinton (2006) furent une véritable percée, car l'idée d'utiliser un algorithme d'apprentissage non-supervisé, les machines de Boltzmann restreintes, pour l'initialisation des poids d'un réseau de neurones supervisé a été cruciale pour entraîner l'architecture profonde la plus populaire, soit les réseaux de neurones artificiels avec des poids totalement connectés. Cette idée a été reprise et reproduite avec succès dans plusieurs contextes et avec une variété de modèles. Dans le cadre de cette thèse, nous considérons les architectures profondes comme des biais inductifs. Ces biais sont représentés non seulement par les modèles eux-mêmes, mais aussi par les méthodes d'entraînement qui sont souvent utilisés en conjonction avec ceux-ci. Nous désirons définir les raisons pour lesquelles cette classe de fonctions généralise bien, les situations auxquelles ces fonctions pourront être appliquées, ainsi que les descriptions qualitatives de telles fonctions. L'objectif de cette thèse est d'obtenir une meilleure compréhension du succès des architectures profondes. Dans le premier article, nous testons la concordance entre nos intuitions---que les réseaux profonds sont nécessaires pour mieux apprendre avec des données comportant plusieurs facteurs de variation---et les résultats empiriques. Le second article est une étude approfondie de la question: pourquoi l'apprentissage non-supervisé aide à mieux généraliser dans un réseau profond? Nous explorons et évaluons plusieurs hypothèses tentant d'élucider le fonctionnement de ces modèles. Finalement, le troisième article cherche à définir de façon qualitative les fonctions modélisées par un réseau profond. Ces visualisations facilitent l'interprétation des représentations et invariances modélisées par une architecture profonde.

Towards Non Invasive Diagnosis of Scoliosis Using Semi-supervised Learning Approach

In this paper, a new methodology for the prediction of scoliosis curve types from non invasive acquisitions of the back surface of the trunk is proposed. One hundred and fifty-nine scoliosis patients had their back surface acquired in 3D using an optical digitizer. Each surface is then characterized by 45 local measurements of the back surface rotation. Using a semi-supervised algorithm, the classifier is trained with only 32 labeled and 58 unlabeled data. Tested on 69 new samples, the classifier succeeded in classifying correctly 87.0% of the data. After reducing the number of labeled training samples to 12, the behavior of the resulting classifier tends to be similar to the reference case where the classifier is trained only with the maximum number of available labeled data. Moreover, the addition of unlabeled data guided the classifier towards more generalizable boundaries between the classes. Those results provide a proof of feasibility for using a semi-supervised learning algorithm to train a classifier for the prediction of a scoliosis curve type, when only a few training data are labeled. This constitutes a promising clinical finding since it will allow the diagnosis and the follow-up of scoliotic deformities without exposing the patient to X-ray radiations.

Validating Co-Training Models for Web Image Classification

Co-training is a semi-supervised learning method that is designed to take advantage of the redundancy that is present when the object to be identified has multiple descriptions. Co-training is known to work well when the multiple descriptions are conditional independent given the class of the object. The presence of multiple descriptions of objects in the form of text, images, audio and video in multimedia applications appears to provide redundancy in the form that may be suitable for co-training. In this paper, we investigate the suitability of utilizing text and image data from the Web for co-training. We perform measurements to find indications of conditional independence in the texts and images obtained from the Web. Our measurements suggest that conditional independence is likely to be present in the data. Our experiments, within a relevance feedback framework to test whether a method that exploits the conditional independence outperforms methods that do not, also indicate that better performance can indeed be obtained by designing algorithms that exploit this form of the redundancy when it is present.

Multilevel Training of Binary Morphological Operators

The design of binary morphological operators that are translation-invariant and locally defined by a finite neighborhood window corresponds to the problem of designing Boolean functions. As in any supervised classification problem, morphological operators designed from a training sample also suffer from overfitting. Large neighborhood tends to lead to performance degradation of the designed operator. This work proposes a multilevel design approach to deal with the issue of designing large neighborhood-based operators. The main idea is inspired by stacked generalization (a multilevel classifier design approach) and consists of, at each training level, combining the outcomes of the previous level operators. The final operator is a multilevel operator that ultimately depends on a larger neighborhood than of the individual operators that have been combined. Experimental results show that two-level operators obtained by combining operators designed on subwindows of a large window consistently outperform the single-level operators designed on the full window. They also show that iterating two-level operators is an effective multilevel approach to obtain better results.

Efficient supervised optimum-path forest classification for large datasets

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Particle Competition and Cooperation in Networks for Semi-Supervised Learning with Concept Drift

Concept drift is a problem of increasing importance in machine learning and data mining. Data sets under analysis are no longer only static databases, but also data streams in which concepts and data distributions may not be stable over time. However, most learning algorithms produced so far are based on the assumption that data comes from a fixed distribution, so they are not suitable to handle concept drifts. Moreover, some concept drifts applications requires fast response, which means an algorithm must always be (re) trained with the latest available data. But the process of labeling data is usually expensive and/or time consuming when compared to unlabeled data acquisition, thus only a small fraction of the incoming data may be effectively labeled. Semi-supervised learning methods may help in this scenario, as they use both labeled and unlabeled data in the training process. However, most of them are also based on the assumption that the data is static. Therefore, semi-supervised learning with concept drifts is still an open challenge in machine learning. Recently, a particle competition and cooperation approach was used to realize graph-based semi-supervised learning from static data. In this paper, we extend that approach to handle data streams and concept drift. The result is a passive algorithm using a single classifier, which naturally adapts to concept changes, without any explicit drift detection mechanism. Its built-in mechanisms provide a natural way of learning from new data, gradually forgetting older knowledge as older labeled data items became less influent on the classification of newer data items. Some computer simulation are presented, showing the effectiveness of the proposed method.

Improving the accuracy of the optimum-path forest supervised classifier for large datasets

In this work, a new approach for supervised pattern recognition is presented which improves the learning algorithm of the Optimum-Path Forest classifier (OPF), centered on detection and elimination of outliers in the training set. Identification of outliers is based on a penalty computed for each sample in the training set from the corresponding number of imputable false positive and false negative classification of samples. This approach enhances the accuracy of OPF while still gaining in classification time, at the expense of a slight increase in training time. © 2010 Springer-Verlag.

Acquisition of suture skills during medical graduation by instructor-directed training: A randomized controlled study comparing senior medical students and faculty surgeons

Due to shortage of time and limited availability of faculty surgeons to teach basic surgical skills during medical graduation, the search for alternative ways of simulated training with feedback is needed. The purpose of this study was to compare the simulated teaching of suture skills to novice medical students by senior medical students and by experienced faculty surgeons. Forty-eight novice medical students were randomly assigned to three practice conditions on bench model (n = 16): self-directed suture training (control), senior medical student-directed suture skills' training, or experienced faculty surgeon-directed suture skills' training. Pre- and post-tests were applied. Global Rating Scale with blinded evaluation and self-perceived confidence based on Likert scale were used to assess all suture performances in pre- and post-training. Effect size was also calculated. The analysis made after training showed that the students who received feedback from the instructors had better performance based on the Global Rating Scale (all p < 0.0000) and felt more confident to carry out sutures (all p < 0.0000) when compared to the control. There was no significant difference (all p > 0.05) between the student-directed teaching and faculty-directed teaching groups. The magnitude of the effect (instructor-directed training suture) was considered large (>0.80) in all measurements. The acquisition of suture skills after student-directed training was similar to the training supervised by faculty surgeon, and the increase in suture performances of trainees that received instructor administered training was superior to self-directed learning. © 2013 Springer-Verlag Italia.