Designing Regularizers and Architectures for Recurrent Neural Networks

Cette thèse contribue a la recherche vers l'intelligence artificielle en utilisant des méthodes connexionnistes. Les réseaux de neurones récurrents sont un ensemble de modèles séquentiels de plus en plus populaires capable en principe d'apprendre des algorithmes arbitraires. Ces modèles effectuent un apprentissage en profondeur, un type d'apprentissage machine. Sa généralité et son succès empirique en font un sujet intéressant pour la recherche et un outil prometteur pour la création de l'intelligence artificielle plus générale. Le premier chapitre de cette thèse donne un bref aperçu des sujets de fonds: l'intelligence artificielle, l'apprentissage machine, l'apprentissage en profondeur et les réseaux de neurones récurrents. Les trois chapitres suivants couvrent ces sujets de manière de plus en plus spécifiques. Enfin, nous présentons quelques contributions apportées aux réseaux de neurones récurrents. Le chapitre \ref{arxiv1} présente nos travaux de régularisation des réseaux de neurones récurrents. La régularisation vise à améliorer la capacité de généralisation du modèle, et joue un role clé dans la performance de plusieurs applications des réseaux de neurones récurrents, en particulier en reconnaissance vocale. Notre approche donne l'état de l'art sur TIMIT, un benchmark standard pour cette tâche. Le chapitre \ref{cpgp} présente une seconde ligne de travail, toujours en cours, qui explore une nouvelle architecture pour les réseaux de neurones récurrents. Les réseaux de neurones récurrents maintiennent un état caché qui représente leurs observations antérieures. L'idée de ce travail est de coder certaines dynamiques abstraites dans l'état caché, donnant au réseau une manière naturelle d'encoder des tendances cohérentes de l'état de son environnement. Notre travail est fondé sur un modèle existant; nous décrivons ce travail et nos contributions avec notamment une expérience préliminaire.

Designing Regularizers and Architectures for Recurrent Neural Networks

Cette thèse contribue a la recherche vers l'intelligence artificielle en utilisant des méthodes connexionnistes. Les réseaux de neurones récurrents sont un ensemble de modèles séquentiels de plus en plus populaires capable en principe d'apprendre des algorithmes arbitraires. Ces modèles effectuent un apprentissage en profondeur, un type d'apprentissage machine. Sa généralité et son succès empirique en font un sujet intéressant pour la recherche et un outil prometteur pour la création de l'intelligence artificielle plus générale. Le premier chapitre de cette thèse donne un bref aperçu des sujets de fonds: l'intelligence artificielle, l'apprentissage machine, l'apprentissage en profondeur et les réseaux de neurones récurrents. Les trois chapitres suivants couvrent ces sujets de manière de plus en plus spécifiques. Enfin, nous présentons quelques contributions apportées aux réseaux de neurones récurrents. Le chapitre \ref{arxiv1} présente nos travaux de régularisation des réseaux de neurones récurrents. La régularisation vise à améliorer la capacité de généralisation du modèle, et joue un role clé dans la performance de plusieurs applications des réseaux de neurones récurrents, en particulier en reconnaissance vocale. Notre approche donne l'état de l'art sur TIMIT, un benchmark standard pour cette tâche. Le chapitre \ref{cpgp} présente une seconde ligne de travail, toujours en cours, qui explore une nouvelle architecture pour les réseaux de neurones récurrents. Les réseaux de neurones récurrents maintiennent un état caché qui représente leurs observations antérieures. L'idée de ce travail est de coder certaines dynamiques abstraites dans l'état caché, donnant au réseau une manière naturelle d'encoder des tendances cohérentes de l'état de son environnement. Notre travail est fondé sur un modèle existant; nous décrivons ce travail et nos contributions avec notamment une expérience préliminaire.

Classification outline for decisions of the National Labor Relations Board and related court decisions.

The outline, together with the Classified index of National Labor Relations Board decisions and related court decisions, comprises a legal research system for Board and court decisions.

Clinical supervision: Its influence on client-rated working alliance and client symptom reduction in the brief treatment of major depression

Supervision of psychotherapists and counselors, especially in the early years of practice, is widely accepted as being important for professional development and to ensure optimal client outcomes. Although the process of clinical supervision has been extensively studied, less is known about the impact of supervision on psychotherapy practice and client symptom outcome. This study evaluated the impact of clinical supervision on client working alliance and symptom reduction in the brief treatment of major depression. The authors randomly assigned 127 clients with a diagnosis of major depression to 127 supervised or unsupervised therapists to receive eight sessions of problems-solving treatment. Supervised therapists were randomly assigned to either alliance skill- or alliance process-focused supervision and received eight supervision sessions. Before beginning treatment, therapists received one supervision session for brief training in the working alliance supervision approach and in specific characteristics of each case. Standard measures of therapeutic alliance and symptom change were used as dependent variables. The results showed a significant effect for both supervision conditions on working alliance from the first session of therapy, symptom reduction, and treatment retention and evaluation but no effect differences between supervision conditions. It was not possible to separate the effects of supervision from the single pretreatment session and is possible that allegiance effects might have inflated results. The scientific and clinical relevance of these findings is discussed.

Learning higher-level features with convolutional restricted Boltzmann machines for sentiment analysis

In recent years, learning word vector representations has attracted much interest in Natural Language Processing. Word representations or embeddings learned using unsupervised methods help addressing the problem of traditional bag-of-word approaches which fail to capture contextual semantics. In this paper we go beyond the vector representations at the word level and propose a novel framework that learns higher-level feature representations of n-grams, phrases and sentences using a deep neural network built from stacked Convolutional Restricted Boltzmann Machines (CRBMs). These representations have been shown to map syntactically and semantically related n-grams to closeby locations in the hidden feature space. We have experimented to additionally incorporate these higher-level features into supervised classifier training for two sentiment analysis tasks: subjectivity classification and sentiment classification. Our results have demonstrated the success of our proposed framework with 4% improvement in accuracy observed for subjectivity classification and improved the results achieved for sentiment classification over models trained without our higher level features.

On stopwords, filtering and data sparsity for sentiment analysis of Twitter

Sentiment classification over Twitter is usually affected by the noisy nature (abbreviations, irregular forms) of tweets data. A popular procedure to reduce the noise of textual data is to remove stopwords by using pre-compiled stopword lists or more sophisticated methods for dynamic stopword identification. However, the effectiveness of removing stopwords in the context of Twitter sentiment classification has been debated in the last few years. In this paper we investigate whether removing stopwords helps or hampers the effectiveness of Twitter sentiment classification methods. To this end, we apply six different stopword identification methods to Twitter data from six different datasets and observe how removing stopwords affects two well-known supervised sentiment classification methods. We assess the impact of removing stopwords by observing fluctuations on the level of data sparsity, the size of the classifier's feature space and its classification performance. Our results show that using pre-compiled lists of stopwords negatively impacts the performance of Twitter sentiment classification approaches. On the other hand, the dynamic generation of stopword lists, by removing those infrequent terms appearing only once in the corpus, appears to be the optimal method to maintaining a high classification performance while reducing the data sparsity and substantially shrinking the feature space

Revisiting the Late Jurassic-Early Cretaceous of the NW South Iberian Basin: new ages and sedimentary environments

The study of the Upper Jurassic-Lower Cretaceous deposits (Higueruelas, Villar del Arzobispo and Aldea de Cortés Formations) of the South Iberian Basin (NW Valencia, Spain) reveals new stratigraphic and sedimentological data, which have significant implications on the stratigraphic framework, depositional environments and age of these units. The Higueruelas Fm was deposited in a mid-inner carbonate platform where oncolitic bars migrated by the action of storms and where oncoid production progressively decreased towards the uppermost part of the unit. The overlying Villar del Arzobispo Fm has been traditionally interpreted as an inner platform-lagoon evolving into a tidal-flat. Here it is interpreted as an inner-carbonate platform affected by storms, where oolitic shoals protected a lagoon, which had siliciclastic inputs from the continent. The Aldea de Cortés Fm has been previously interpreted as a lagoon surrounded by tidal-flats and fluvial-deltaic plains. Here it is reinterpreted as a coastal wetland where siliciclastic muddy deposits interacted with shallow fresh to marine water bodies, aeolian dunes and continental siliciclastic inputs. The contact between the Higueruelas and Villar del Arzobispo Fms, classically defined as gradual, is also interpreted here as rapid. More importantly, the contact between the Villar del Arzobispo and Aldea de Cortés Fms, previously considered as unconformable, is here interpreted as gradual. The presence of Alveosepta in the Villar del Arzobispo Fm suggests that at least part of this unit is Kimmeridgian, unlike the previously assigned Late Tithonian-Middle Berriasian age. Consequently, the underlying Higueruelas Fm, previously considered Tithonian, should not be younger than Kimmeridgian. Accordingly, sedimentation of the Aldea de Cortés Fm, previously considered Valangian-Hauterivian, probably started during the Tithonian and it may be considered part of the regressive trend of the Late Jurassic-Early Cretaceous cycle. This is consistent with the dinosaur faunas, typically Jurassic, described in the Villar del Arzobispo and Aldea de Cortés Fms.

Layered Information Modeling and a Knowledge Organization Paradigm: Inherent Classification and the Design of Interoperable Systems

In knowledge technology work, as expressed by the scope of this conference, there are a number of communities, each uncovering new methods, theories, and practices. The Library and Information Science (LIS) community is one such community. This community, through tradition and innovation, theories and practice, organizes knowledge and develops knowledge technologies formed by iterative research hewn to the values of equal access and discovery for all. The Information Modeling community is another contributor to knowledge technologies. It concerns itself with the construction of symbolic models that capture the meaning of information and organize it in ways that are computer-based, but human understandable. A recent paper that examines certain assumptions in information modeling builds a bridge between these two communities, offering a forum for a discussion on common aims from a common perspective. In a June 2000 article, Parsons and Wand separate classes from instances in information modeling in order to free instances from what they call the “tyranny” of classes. They attribute a number of problems in information modeling to inherent classification – or the disregard for the fact that instances can be conceptualized independent of any class assignment. By faceting instances from classes, Parsons and Wand strike a sonorous chord with classification theory as understood in LIS. In the practice community and in the publications of LIS, faceted classification has shifted the paradigm of knowledge organization theory in the twentieth century. Here, with the proposal of inherent classification and the resulting layered information modeling, a clear line joins both the LIS classification theory community and the information modeling community. Both communities have their eyes turned toward networked resource discovery, and with this conceptual conjunction a new paradigmatic conversation can take place. Parsons and Wand propose that the layered information model can facilitate schema integration, schema evolution, and interoperability. These three spheres in information modeling have their own connotation, but are not distant from the aims of classification research in LIS. In this new conceptual conjunction, established by Parsons and Ward, information modeling through the layered information model, can expand the horizons of classification theory beyond LIS, promoting a cross-fertilization of ideas on the interoperability of subject access tools like classification schemes, thesauri, taxonomies, and ontologies. This paper examines the common ground between the layered information model and faceted classification, establishing a vocabulary and outlining some common principles. It then turns to the issue of schema and the horizons of conventional classification and the differences between Information Modeling and Library and Information Science. Finally, a framework is proposed that deploys an interpretation of the layered information modeling approach in a knowledge technologies context. In order to design subject access systems that will integrate, evolve and interoperate in a networked environment, knowledge organization specialists must consider a semantic class independence like Parsons and Wand propose for information modeling.

Foundational, First-Order, and Second-Order Classification Theory

Both basic and applied research on the construction, implementation, maintenance, and evaluation of classification schemes is called classification theory. If we employ Ritzer’s metatheoretical method of analysis on the over one-hundred year-old body of literature, we can see categories of theory emerge. This paper looks at one particular part of knowledge organization work, namely classification theory, and asks 1) what are the contours of this intellectual space, and, 2) what have we produced in the theoretical reflection on con- structing, implementing, and evaluating classification schemes? The preliminary findings from this work are that classification theory can be separated into three kinds: foundational classification theory, first-order classification theory, and second-order classification theory, each with its own concerns and objects of study.

Variational and learning models for image and time series inverse problems

Inverse problems are at the core of many challenging applications. Variational and learning models provide estimated solutions of inverse problems as the outcome of specific reconstruction maps. In the variational approach, the result of the reconstruction map is the solution of a regularized minimization problem encoding information on the acquisition process and prior knowledge on the solution. In the learning approach, the reconstruction map is a parametric function whose parameters are identified by solving a minimization problem depending on a large set of data. In this thesis, we go beyond this apparent dichotomy between variational and learning models and we show they can be harmoniously merged in unified hybrid frameworks preserving their main advantages. We develop several highly efficient methods based on both these model-driven and data-driven strategies, for which we provide a detailed convergence analysis. The arising algorithms are applied to solve inverse problems involving images and time series. For each task, we show the proposed schemes improve the performances of many other existing methods in terms of both computational burden and quality of the solution. In the first part, we focus on gradient-based regularized variational models which are shown to be effective for segmentation purposes and thermal and medical image enhancement. We consider gradient sparsity-promoting regularized models for which we develop different strategies to estimate the regularization strength. Furthermore, we introduce a novel gradient-based Plug-and-Play convergent scheme considering a deep learning based denoiser trained on the gradient domain. In the second part, we address the tasks of natural image deblurring, image and video super resolution microscopy and positioning time series prediction, through deep learning based methods. We boost the performances of supervised, such as trained convolutional and recurrent networks, and unsupervised deep learning strategies, such as Deep Image Prior, by penalizing the losses with handcrafted regularization terms.

Estado nutricional e desempenho motor de escolares = : Nutritional state and motor performance of scholl-age children

Universidade Estadual de Campinas . Faculdade de Educação Física

The recurrence of leprosy reactional episodes could be associated with oral chronic infections and expression of serum IL-1, TNF-α, IL-6, IFN-γ and IL-10

The aim of this study was to determine whether the presence of leprosy reactional episodes could be associated with chronic oral infection. Thirty-eight leprosy patients were selected and divided into 2 groups: group I - 19 leprosy patients with oral infections, and group II - 19 leprosy patients without oral infections. Ten patients without leprosy, but presenting oral infections, were assigned to the control group. Leprosy patients were classified according to Ridley and Jopling classification and reactional episodes of the erythema nodosum type or reversal reaction were identified by clinical and histopathological features associated with serum IL-1, TNF-α, IL-6, IFN-γ and IL-10 levels. These analyses were performed immediately before and 7 days after the oral infection elimination. Patients from group I presenting oral infections reported clinical improvement of the symptoms of reactional episodes after dental treatment. Serum IL-1, TNF-α, IL-6, IFN-γ and IL-10 levels did not differ significantly before and after dental treatment as determined by the Wilcoxon test (p>0.05). Comparison of the 2 groups showed statistically significant differences in IL-1 and IL-6 at baseline and in IL-1, IL-6 and IL-10 on the occasion of both collections 7 days after therapy. Serum IL-6 and IL-10 levels in group I differed significantly at baseline compared to control (Mann-Whitney test; p<0.05). These results suggest that oral infection could be involved as a maintenance factor in the pathogenesis of leprosy reactional episodes.

Headland-bay beach planform stability of Santa Catarina State and of the Northern Coast of São Paulo State

This paper presents the results of the planform stability classification for the headland-bay beaches of the State of Santa Catarina and of the Northern Coast of São Paulo, based on the application of the Parabolic Bay-Shape Equation (PBSE) to aerial images of the beaches, using the software MEPBAY®. For this purpose, georeferenced mosaics of the QuickBird2® satellite imagery (for the State of Santa Catarina) and vertical aerial photographs (for the northern coast of São Paulo State) were used. Headland-bay beach planform stability can be classified as: (1) in static equilibrium, (2) in dynamic equilibrium, (3) unstable or (4) in a state of natural beach reshaping. Static equilibrium beaches are the most frequent along the coast of the State of Santa Catarina and the Northern Shore of São Paulo, notably along the most rugged sectors of the coast and those with experiencing lower fluvial discharge. By comparison, dynamic equilibrium beaches occur primarily on the less rugged sectors of the coast and along regions with higher fluvial discharge. Beaches in a state of natural beach reshaping have only been found in SC, associated with stabilized estuarine inlets or port breakwaters. However, it is not possible to classify any of these beaches as unstable because only one set of images was used. No clear relation was observed between a beach's planform stability and other classification factors, such as morphodynamics or orientation.

Discharge-calcium concentration relationships in streams of the Amazon and Cerrado of Brazil: soil or land use controlled

Stream discharge-concentration relationships are indicators of terrestrial ecosystem function. Throughout the Amazon and Cerrado regions of Brazil rapid changes in land use and land cover may be altering these hydrochemical relationships. The current analysis focuses on factors controlling the discharge-calcium (Ca) concentration relationship since previous research in these regions has demonstrated both positive and negative slopes in linear log(10)discharge-log(10)Ca concentration regressions. The objective of the current study was to evaluate factors controlling stream discharge-Ca concentration relationships including year, season, stream order, vegetation cover, land use, and soil classification. It was hypothesized that land use and soil class are the most critical attributes controlling discharge-Ca concentration relationships. A multilevel, linear regression approach was utilized with data from 28 streams throughout Brazil. These streams come from three distinct regions and varied broadly in watershed size (< 1 to > 10(6) ha) and discharge (10(-5.7)-10(3.2) m(3) s(-1)). Linear regressions of log(10)Ca versus log(10)discharge in 13 streams have a preponderance of negative slopes with only two streams having significant positive slopes. An ANOVA decomposition suggests the effect of discharge on Ca concentration is large but variable. Vegetation cover, which incorporates aspects of land use, explains the largest proportion of the variance in the effect of discharge on Ca followed by season and year. In contrast, stream order, land use, and soil class explain most of the variation in stream Ca concentration. In the current data set, soil class, which is related to lithology, has an important effect on Ca concentration but land use, likely through its effect on runoff concentration and hydrology, has a greater effect on discharge-concentration relationships.

Eriophyoid mites (Acari: Prostigmata: Eriophyoidea) from fruit trees in Northeastern Brazil-a new genus, three new species and a redescription

A new genus and two new species of eriophyoid mites in the family Diptilomiopidae associated with Spondias mombin L. (Anacardiaceae), namely Solivagus n. gen. alpha n. sp. and Davisella spondias n. sp., are described. In addition, a new species of Eriophyidae associated with Eugenia uniflora L. (Myrtaceae), namely Dichopelmus ibapitanga n. sp., is described and Aculus pitangae Boczek & Davis, also from E. uniflora, is redescribed including a description of the male, and its classification is discussed. All material studied was collected in the State of Pernambuco, Northeastern Brazil.