Discriminative training of the hidden vector state model for semantic parsing

In this paper, we discuss how discriminative training can be applied to the hidden vector state (HVS) model in different task domains. The HVS model is a discrete hidden Markov model (HMM) in which each HMM state represents the state of a push-down automaton with a finite stack size. In previous applications, maximum-likelihood estimation (MLE) is used to derive the parameters of the HVS model. However, MLE makes a number of assumptions and unfortunately some of these assumptions do not hold. Discriminative training, without making such assumptions, can improve the performance of the HVS model by discriminating the correct hypothesis from the competing hypotheses. Experiments have been conducted in two domains: the travel domain for the semantic parsing task using the DARPA Communicator data and the Air Travel Information Services (ATIS) data and the bioinformatics domain for the information extraction task using the GENIA corpus. The results demonstrate modest improvements of the performance of the HVS model using discriminative training. In the travel domain, discriminative training of the HVS model gives a relative error reduction rate of 31 percent in F-measure when compared with MLE on the DARPA Communicator data and 9 percent on the ATIS data. In the bioinformatics domain, a relative error reduction rate of 4 percent in F-measure is achieved on the GENIA corpus.

On the determination of epsilon during discriminative GMM training

Discriminative training of Gaussian Mixture Models (GMMs) for speech or speaker recognition purposes is usually based on the gradient descent method, in which the iteration step-size, ε, uses to be defined experimentally. In this letter, we derive an equation to adaptively determine ε, by showing that the second-order Newton-Raphson iterative method to find roots of equations is equivalent to the gradient descent algorithm. © 2010 IEEE.

Efeito do treino de discriminação simples sobre o repertório de pareamento ao modelo por identidade de um macaco-prego (Cebus apella)

Foi investigado como o treino de Reversões Repetidas de Discriminações Simples (RRDS) poderia facilitar a aquisição do desempenho de pareamento ao modelo por identidade (IDMTS). Utilizou-se um macaco-prego macho, com pouco mais de um ano de idade, com treino prévio de pareamento ao modelo por identidade. Foram utilizados dois conjuntos: A com quatro e B com cinco estímulos. A tarefa de IDMTS com atraso zero e com quatro escolhas com o Conjunto A foi usada como linha de base, utilizando razão três nas respostas de toque. Foi feito treino de RRDS com cada um dos pares possíveis do conjunto B. Depois das RRDS, testou-se o par em tentativas de pareamento ao modelo misturadas à linha de base. O desempenho na discriminação simples entre os pares de estímulos do conjunto B variou quanto ao número de sessões para atingir o critério de seis corretas em um máximo de 12 tentativas. No IDMTS com cada par do conjunto B, o critério foi atingido em 9 sessões na primeira etapa e 12 sessões na segunda etapa. Porém, houve erros nas tentativas de IDMTS na primeira fase da terceira etapa, quando os estímulos do conjunto B compuseram tentativas com quatro escolhas. O aumento de acertos no IDMTS nas tentativas do conjunto B ocorreu com a substituição de um dos estímulos, mostrando problemas de discriminabilidade entre o estímulo eliminado e os demais. Discute-se um possível conflito entre as topografias de controle selecionadas pelas RRDS.

Efeitos da manipulação do número de escolhas sobre o desempenho em tarefas de discriminação simples em macaco-prego (Cebus cf. apella)

Em um treino discriminativo, há diversas variáveis que podem afetar a precisão da aquisição do repertório. Uma variável pouco analisada é o número de escolhas apresentado nas tentativas discretas de treino, buscando verificar em que situações o controle discriminativo pode ser mais facilmente estabelecido. Este trabalho tem como objetivo geral descrever os efeitos da manipulação do número de escolhas sobre o desempenho em tarefas de discriminação simples em macacos-prego (Cebus cf. apella). No Experimento I, os sujeitos foram submetidos a um treino discriminativo com três tipos diferentes de tentativas (2, 4 e 9 escolhas). Um teste de controle de estímulos avaliou se o repertório aprendido podia ser mantido quando os estímulos utilizados nos três tipos de tentativa eram apresentados na forma de duas escolhas. No Experimento II, buscou-se adicionalmente avaliar se as respostas corretas nos treinos prévios de discriminação ocorriam em função de escolha por seleção do S⁺, por rejeição do S^-, ou por controle misto (seleção e rejeição). Esta avaliação foi realizada através do procedimento de máscara. Os resultados do Experimento I sugerem que a exposição a um número maior de escolhas é uma estratégia eficiente para estabelecer o responder discriminado, pois o desempenho permanece mesmo quando o número de escolhas é posteriormente reduzido para dois. Os resultados obtidos no Experimento II mostram dados diferentes para os dois sujeitos. M30 apresentou controle por rejeição e preferência pela máscara e M31 apresentou controle misto no responder. O presente estudo mostra um caminho para aprofundar a análise do controle de estímulos nos estudos específicos sobre a manipulação do número de escolhas e indica que essa variável pode ser um meio eficaz de reduzir a dificuldade de aquisição de discriminações em contexto aplicado.

Semi-supervised learning of statistical models for natural language understanding

Natural language understanding is to specify a computational model that maps sentences to their semantic mean representation. In this paper, we propose a novel framework to train the statistical models without using expensive fully annotated data. In particular, the input of our framework is a set of sentences labeled with abstract semantic annotations. These annotations encode the underlying embedded semantic structural relations without explicit word/semantic tag alignment. The proposed framework can automatically induce derivation rules that map sentences to their semantic meaning representations. The learning framework is applied on two statistical models, the conditional random fields (CRFs) and the hidden Markov support vector machines (HM-SVMs). Our experimental results on the DARPA communicator data show that both CRFs and HM-SVMs outperform the baseline approach, previously proposed hidden vector state (HVS) model which is also trained on abstract semantic annotations. In addition, the proposed framework shows superior performance than two other baseline approaches, a hybrid framework combining HVS and HM-SVMs and discriminative training of HVS, with a relative error reduction rate of about 25% and 15% being achieved in F-measure.

Efeito do tipo de treino discriminativo sobre a observação de estímulos compostos e de seus elementos

Dissertação (mestrado)—Universidade de Brasília, Instituto de Psicologia, Departamento de Processos Psicológicos Básicos, Programa de Pós-Graduação em Ciências do Comportamento, 2016.

Scene Classification Using a Hybrid Generative/Discriminative Approach

We investigate whether dimensionality reduction using a latent generative model is beneficial for the task of weakly supervised scene classification. In detail, we are given a set of labeled images of scenes (for example, coast, forest, city, river, etc.), and our objective is to classify a new image into one of these categories. Our approach consists of first discovering latent ";topics"; using probabilistic Latent Semantic Analysis (pLSA), a generative model from the statistical text literature here applied to a bag of visual words representation for each image, and subsequently, training a multiway classifier on the topic distribution vector for each image. We compare this approach to that of representing each image by a bag of visual words vector directly and training a multiway classifier on these vectors. To this end, we introduce a novel vocabulary using dense color SIFT descriptors and then investigate the classification performance under changes in the size of the visual vocabulary, the number of latent topics learned, and the type of discriminative classifier used (k-nearest neighbor or SVM). We achieve superior classification performance to recent publications that have used a bag of visual word representation, in all cases, using the authors' own data sets and testing protocols. We also investigate the gain in adding spatial information. We show applications to image retrieval with relevance feedback and to scene classification in videos

Scene Classification Using a Hybrid Generative/Discriminative Approach

We investigate whether dimensionality reduction using a latent generative model is beneficial for the task of weakly supervised scene classification. In detail, we are given a set of labeled images of scenes (for example, coast, forest, city, river, etc.), and our objective is to classify a new image into one of these categories. Our approach consists of first discovering latent ";topics"; using probabilistic Latent Semantic Analysis (pLSA), a generative model from the statistical text literature here applied to a bag of visual words representation for each image, and subsequently, training a multiway classifier on the topic distribution vector for each image. We compare this approach to that of representing each image by a bag of visual words vector directly and training a multiway classifier on these vectors. To this end, we introduce a novel vocabulary using dense color SIFT descriptors and then investigate the classification performance under changes in the size of the visual vocabulary, the number of latent topics learned, and the type of discriminative classifier used (k-nearest neighbor or SVM). We achieve superior classification performance to recent publications that have used a bag of visual word representation, in all cases, using the authors' own data sets and testing protocols. We also investigate the gain in adding spatial information. We show applications to image retrieval with relevance feedback and to scene classification in videos

Introducing the Discriminative Paraconsistent Machine (DPM)

This paper introduces a new tool for pattern recognition. Called the Discriminative Paraconsistent Machine (DPM), it is based on a supervised discriminative model training that incorporates paraconsistency criteria and allows an intelligent treatment of contradictions and uncertainties. DPMs can be applied to solve problems in many fields of science, using the tests and discussions presented here, which demonstrate their efficacy and usefulness. Major difficulties and challenges that were overcome consisted basically in establishing the proper model with which to represent the concept of paraconsistency.

Generating Discriminative Objective Proposals via Submodular Ranking

Object recognition has long been a core problem in computer vision. To improve object spatial support and speed up object localization for object recognition, generating high-quality category-independent object proposals as the input for object recognition system has drawn attention recently. Given an image, we generate a limited number of high-quality and category-independent object proposals in advance and used as inputs for many computer vision tasks. We present an efficient dictionary-based model for image classification task. We further extend the work to a discriminative dictionary learning method for tensor sparse coding. In the first part, a multi-scale greedy-based object proposal generation approach is presented. Based on the multi-scale nature of objects in images, our approach is built on top of a hierarchical segmentation. We first identify the representative and diverse exemplar clusters within each scale. Object proposals are obtained by selecting a subset from the multi-scale segment pool via maximizing a submodular objective function, which consists of a weighted coverage term, a single-scale diversity term and a multi-scale reward term. The weighted coverage term forces the selected set of object proposals to be representative and compact; the single-scale diversity term encourages choosing segments from different exemplar clusters so that they will cover as many object patterns as possible; the multi-scale reward term encourages the selected proposals to be discriminative and selected from multiple layers generated by the hierarchical image segmentation. The experimental results on the Berkeley Segmentation Dataset and PASCAL VOC2012 segmentation dataset demonstrate the accuracy and efficiency of our object proposal model. Additionally, we validate our object proposals in simultaneous segmentation and detection and outperform the state-of-art performance. To classify the object in the image, we design a discriminative, structural low-rank framework for image classification. We use a supervised learning method to construct a discriminative and reconstructive dictionary. By introducing an ideal regularization term, we perform low-rank matrix recovery for contaminated training data from all categories simultaneously without losing structural information. A discriminative low-rank representation for images with respect to the constructed dictionary is obtained. With semantic structure information and strong identification capability, this representation is good for classification tasks even using a simple linear multi-classifier.

The Effect Of Pelvic Floor Muscle Training Alone Or In Combination With Electrostimulation In The Treatment Of Sexual Dysfunction In Women With Multiple Sclerosis.

Sexual dysfunction (SD) affects up to 80% of multiple sclerosis (MS) patients and pelvic floor muscles (PFMs) play an important role in the sexual function of these patients. The objective of this paper is to evaluate the impact of a rehabilitation program to treat lower urinary tract symptoms on SD of women with MS. Thirty MS women were randomly allocated to one of three groups: pelvic floor muscle training (PFMT) with electromyographic (EMG) biofeedback and sham neuromuscular electrostimulation (NMES) (Group I), PFMT with EMG biofeedback and intravaginal NMES (Group II), and PFMT with EMG biofeedback and transcutaneous tibial nerve stimulation (TTNS) (Group III). Assessments, before and after the treatment, included: PFM function, PFM tone, flexibility of the vaginal opening and ability to relax the PFMs, and the Female Sexual Function Index (FSFI) questionnaire. After treatment, all groups showed improvements in all domains of the PERFECT scheme. PFM tone and flexibility of the vaginal opening was lower after the intervention only for Group II. All groups improved in arousal, lubrication, satisfaction and total score domains of the FSFI questionnaire. This study indicates that PFMT alone or in combination with intravaginal NMES or TTNS contributes to the improvement of SD.

Treadmill Training Increases Sirt-1 And Pgc-1 α Protein Levels And Ampk Phosphorylation In Quadriceps Of Middle-aged Rats In An Intensity-dependent Manner.

The present study investigated the effects of running at 0.8 or 1.2 km/h on inflammatory proteins (i.e., protein levels of TNF- α , IL-1 β , and NF- κ B) and metabolic proteins (i.e., protein levels of SIRT-1 and PGC-1 α , and AMPK phosphorylation) in quadriceps of rats. Male Wistar rats at 3 (young) and 18 months (middle-aged rats) of age were divided into nonexercised (NE) and exercised at 0.8 or 1.2 km/h. The rats were trained on treadmill, 50 min per day, 5 days per week, during 8 weeks. Forty-eight hours after the last training session, muscles were removed, homogenized, and analyzed using biochemical and western blot techniques. Our results showed that: (a) running at 0.8 km/h decreased the inflammatory proteins and increased the metabolic proteins compared with NE rats; (b) these responses were lower for the inflammatory proteins and higher for the metabolic proteins in young rats compared with middle-aged rats; (c) running at 1.2 km/h decreased the inflammatory proteins and increased the metabolic proteins compared with 0.8 km/h; (d) these responses were similar between young and middle-aged rats when trained at 1.2 km. In summary, the age-related increases in inflammatory proteins, and the age-related declines in metabolic proteins can be reversed and largely improved by treadmill training.

Undulatory Physical Resistance Training Program Increases Maximal Strength In Elderly Type 2 Diabetics.

To investigate the effects of a specific protocol of undulatory physical resistance training on maximal strength gains in elderly type 2 diabetics. The study included 48 subjects, aged between 60 and 85 years, of both genders. They were divided into two groups: Untrained Diabetic Elderly (n=19) with those who were not subjected to physical training and Trained Diabetic Elderly (n=29), with those who were subjected to undulatory physical resistance training. The participants were evaluated with several types of resistance training's equipment before and after training protocol, by test of one maximal repetition. The subjects were trained on undulatory resistance three times per week for a period of 16 weeks. The overload used in undulatory resistance training was equivalent to 50% of one maximal repetition and 70% of one maximal repetition, alternating weekly. Statistical analysis revealed significant differences (p<0.05) between pre-test and post-test over a period of 16 weeks. The average gains in strength were 43.20% (knee extension), 65.00% (knee flexion), 27.80% (supine sitting machine), 31.00% (rowing sitting), 43.90% (biceps pulley), and 21.10% (triceps pulley). Undulatory resistance training used with weekly different overloads was effective to provide significant gains in maximum strength in elderly type 2 diabetic individuals.

Leucine Supplementation Does Not Affect Protein Turnover And Impairs The Beneficial Effects Of Endurance Training On Glucose Homeostasis In Healthy Mice.

Endurance exercise training as well as leucine supplementation modulates glucose homeostasis and protein turnover in mammals. Here, we analyze whether leucine supplementation alters the effects of endurance exercise on these parameters in healthy mice. Mice were distributed into sedentary (C) and exercise (T) groups. The exercise group performed a 12-week swimming protocol. Half of the C and T mice, designated as the CL and TL groups, were supplemented with leucine (1.5 % dissolved in the drinking water) throughout the experiment. As well known, endurance exercise training reduced body weight and the retroperitoneal fat pad, increased soleus mass, increased VO2max, decreased muscle proteolysis, and ameliorated peripheral insulin sensitivity. Leucine supplementation had no effect on any of these parameters and worsened glucose tolerance in both CL and TL mice. In the soleus muscle of the T group, AS-160(Thr-642) (AKT substrate of 160 kDa) and AMPK(Thr-172) (AMP-Activated Protein Kinase) phosphorylation was increased by exercise in both basal and insulin-stimulated conditions, but it was reduced in TL mice with insulin stimulation compared with the T group. Akt phosphorylation was not affected by exercise but was lower in the CL group compared with the other groups. Leucine supplementation increased mTOR phosphorylation at basal conditions, whereas exercise reduced it in the presence of insulin, despite no alterations in protein synthesis. In trained groups, the total FoxO3a protein content and the mRNA for the specific isoforms E2 and E3 ligases were reduced. In conclusion, leucine supplementation did not potentiate the effects of endurance training on protein turnover, and it also reduced its positive effects on glucose homeostasis.