On Recurrent and Deep Neural Networks

L'apprentissage profond est un domaine de recherche en forte croissance en apprentissage automatique qui est parvenu à des résultats impressionnants dans différentes tâches allant de la classification d'images à la parole, en passant par la modélisation du langage. Les réseaux de neurones récurrents, une sous-classe d'architecture profonde, s'avèrent particulièrement prometteurs. Les réseaux récurrents peuvent capter la structure temporelle dans les données. Ils ont potentiellement la capacité d'apprendre des corrélations entre des événements éloignés dans le temps et d'emmagasiner indéfiniment des informations dans leur mémoire interne. Dans ce travail, nous tentons d'abord de comprendre pourquoi la profondeur est utile. Similairement à d'autres travaux de la littérature, nos résultats démontrent que les modèles profonds peuvent être plus efficaces pour représenter certaines familles de fonctions comparativement aux modèles peu profonds. Contrairement à ces travaux, nous effectuons notre analyse théorique sur des réseaux profonds acycliques munis de fonctions d'activation linéaires par parties, puisque ce type de modèle est actuellement l'état de l'art dans différentes tâches de classification. La deuxième partie de cette thèse porte sur le processus d'apprentissage. Nous analysons quelques techniques d'optimisation proposées récemment, telles l'optimisation Hessian free, la descente de gradient naturel et la descente des sous-espaces de Krylov. Nous proposons le cadre théorique des méthodes à région de confiance généralisées et nous montrons que plusieurs de ces algorithmes développés récemment peuvent être vus dans cette perspective. Nous argumentons que certains membres de cette famille d'approches peuvent être mieux adaptés que d'autres à l'optimisation non convexe. La dernière partie de ce document se concentre sur les réseaux de neurones récurrents. Nous étudions d'abord le concept de mémoire et tentons de répondre aux questions suivantes: Les réseaux récurrents peuvent-ils démontrer une mémoire sans limite? Ce comportement peut-il être appris? Nous montrons que cela est possible si des indices sont fournis durant l'apprentissage. Ensuite, nous explorons deux problèmes spécifiques à l'entraînement des réseaux récurrents, à savoir la dissipation et l'explosion du gradient. Notre analyse se termine par une solution au problème d'explosion du gradient qui implique de borner la norme du gradient. Nous proposons également un terme de régularisation conçu spécifiquement pour réduire le problème de dissipation du gradient. Sur un ensemble de données synthétique, nous montrons empiriquement que ces mécanismes peuvent permettre aux réseaux récurrents d'apprendre de façon autonome à mémoriser des informations pour une période de temps indéfinie. Finalement, nous explorons la notion de profondeur dans les réseaux de neurones récurrents. Comparativement aux réseaux acycliques, la définition de profondeur dans les réseaux récurrents est souvent ambiguë. Nous proposons différentes façons d'ajouter de la profondeur dans les réseaux récurrents et nous évaluons empiriquement ces propositions.

Mobilizing in borderline citizenship regimes : a comparative analysis of undocumented migrants’ collective actions

This article seeks to explain how and why groups and networks of undocumented migrants mobilizing in Berlin, Montréal, and Paris since the beginning of the 2000s construct different types of claims. The authors explore the relationship between undocumented migrants and state authorities at the local level through the concept of the citizenship regime and its specific application to undocumented migrants (which they describe as the “borderline citizenship regime”). Despite their common formal exclusion from citizenship, nonstatus migrants experience different degrees and forms of exclusion in their daily lives, in terms of access to certain rights and services, recognition, and belonging within the state (whether through formally or nonformally recognized means). As a result, they have an opportunity to create different, specific forms of leeway in the society in which they live. The concurrence of these different degrees of exclusion and different forms of leeway defines specific conditions of mobilization. The authors demonstrate how the content of their claims is influenced by these conditions of mobilization.

Women Industrial Cooperatives- An Analysis of Its Structure, Performance and Growth in Kannur District

In the present study made an attempt to analyse the structure, performance and growth of women industrial cooperatives in kannur district, Kerala. The study encompasses all women industrial cooperatives registered at the district industries center, kannur and that currently exist. The women industrial cooperatives are classified into two ie; group with network and another group without network. In Kannur there are 54 units working as women industrial cooperatives. One of the main problems the women cooperatives face is the lack of working capital followed marketing problem. The competition between cooperatives and private traders is very high. The variables examined to analyse the performance of women industrial cooperatives in Kannur showed that there exists inter unit differences in almost all the variables. The financial structure structure shows that the short term liquidity of women cooperatives in Kannur favour more the units which have political networks; but the long term financial coverage is seen to be highly geared in this group, not because of a decline is net worth but due to highly proportionate increase in financial liabilities in the form of borrowings. The encouragement given by the government through financial stake and other incentives has been the major factor in the formation and growth of women cooperatives. As a result both productivity and efficiency improves in the cooperatives. In short the present study helped to capture the impact, role and dynamics of networking in general and socio political network in particular in relation to intra and inter unit differences on the structure, growth and performance of women industrial cooperatives societies in Kannur district

Neural Network based studies on Spectroscopic Analysis and Image Processing

International School of Photonics, Cochin University of Science and Technology

Identification of spectral lines of elements using artificial neural networks

Artificial neural networks (ANNs) are relatively new computational tools that have found extensive utilization in solving many complex real-world problems. This paper describes how an ANN can be used to identify the spectral lines of elements. The spectral lines of Cadmium (Cd), Calcium (Ca), Iron (Fe), Lithium (Li), Mercury (Hg), Potassium (K) and Strontium (Sr) in the visible range are chosen for the investigation. One of the unique features of this technique is that it uses the whole spectrum in the visible range instead of individual spectral lines. The spectrum of a sample taken with a spectrometer contains both original peaks and spurious peaks. It is a tedious task to identify these peaks to determine the elements present in the sample. ANNs capability of retrieving original data from noisy spectrum is also explored in this paper. The importance of the need of sufficient data for training ANNs to get accurate results is also emphasized. Two networks are examined: one trained in all spectral lines and other with the persistent lines only. The network trained in all spectral lines is found to be superior in analyzing the spectrum even in a noisy environment.

Epoxy-modiWed, unsaturated polyester hybrid networks

Hybrid polymer networks (HPNs) based on unsaturated polyester resin (UPR) and epoxy resins were synthesized by reactive blending. The epoxy resins used were epoxidised phenolic novolac (EPN), epoxidised cresol novolac (ECN) and diglycidyl ether of bisphenol A (DGEBA). Epoxy novolacs were prepared by glycidylation of the novolacs using epichlorohydrin. The physical, mechanical, and thermal properties of the cured blends were compared with those of the control resin. Epoxy resins show good miscibility and compatibility with the UPR resin on blending and the co-cured resin showed substantial improvement in the toughness and impact resistance. Considerable enhancement of tensile strength and toughness are noticed at very low loading of EPN. Thermogravimetric analysis (TGA), dynamic mechanical analysis (DMA) and diVerential scanning calorimetry (DSC) were employed to study the thermal properties of the toughened resin. The EPN/ UPR blends showed substantial improvement in thermal stability as evident from TGA and damping data. The fracture behaviour was corroborated by scanning electron microscopy (SEM). The performance of EPN is found to be superior to other epoxy resins

Not the outside but the inside matters : functional analysis of Capricious and Tartan during Drosophila tracheal morphogenesis

The tubular structures, which transport essential gases, liquids, or cells from one site to another, are shared among various divergent organisms. These highly organized tubular networks include lung, kidney, vasculature and mammary gland in mammals as well as trachea and salivary gland in Drosophila melanogaster. Many questions regarding the tubular morphogenesis cannot be addressed sufficiently by investigating the mammalian organs because their structures are extremely complex and therefore, systematic analyses of genetic and cellular programs guiding the development is not possible. In contrast, the Drosophila tracheal development provides an excellent model system since many molecular markers and powerful tools for genetic manipulations are available. Two mechanisms were shown to be important for the outgrowth of tracheal cells: the FGF signaling pathway and the interaction between the tracheal cells and the surrounding mesodermal cells. The Drosophila FGF ligand encoded by branchless (bnl) is localized in groups of cells near tracheal metameres. The tracheal cells expressing the FGF receptor breathless (btl) respond to these sources of FGF ligand and extend towards them. However, this FGF signaling pathway is not sufficient for the formation of continuous dorsal trunk, the only muticellular tube in tracheal system. Recently, it was found out that single mesodermal cells called bridge-cells are essential for the formation of continuous dorsal trunk as they direct the outgrowth of dorsal trunk cells towards the correct targets. The results in this PhD thesis demonstrate that a cell adhesion molecule Capricious (Caps), which is specifically localized on the surface of bridge-cells, plays an essential role in guiding the outgrowing dorsal trunk cells towards their correct targets. When caps is lacking, some bridge-cells cannot stretch properly towards the adjacent posterior tracheal metameres and thus fail to interconnect the juxtaposing dorsal trunk cells. Consequently, discontinuous dorsal trunks containing interruptions at several positions are formed. On the other hand, when caps is ectopically expressed in the mesodermal cells through a twi-GAL4 driver, these mesodermal cells acquire a guidance function through ectopic caps and misguide the outgrowing dorsal trunk cells in abnormal directions. As a result, disconnected dorsal trunks are formed. These loss- and gain-of-function studies suggest that Caps presumably establishes the cell-to-cell contact between the bridge-cells and the tracheal cells and thereby mediates directly the guidance function of bridge-cells. The most similar protein known to Caps is another cell adhesion molecule called Tartan (Trn). Interestingly, trn is expressed in the mesodermal cells but not in the bridge-cells. When trn is lacking, the outgrowth of not only the dorsal trunks but also the lateral trunks are disrupted. However, in contrast to the ectopic expression of caps, the misexpression of trn does not affect tracheal development. Whereas Trn requires only its extracellular domain to mediate the matrix function, Caps requires both its extracellular and intracellular domains to function as a guidance molecule in the bridge-cells. These observations suggest that Trn functions differently from Caps during tracheal morphogenesis. Presumably, Trn mediates a matrix function of mesodermal cells, which support the tracheal cells to extend efficiently through the surrounding mesodermal tissue. In order to determine which domains dictate the functional specificity of Caps, two hybrid proteins CapsEdTrnId, which contains the Caps extracellular domain and the Trn intracellular domain, and TrnEdCapsId, which consists of the Trn extracellular domain and the Caps intracellular domain, were constructed. Gain of function and rescue experiments with these hybrid proteins suggest on one hand that the extracellular domains of Caps and Trn are functionally redundant and on the other hand that the intracellular domain dictates the functional specificity of Caps. In order to identify putative interactors of Caps, yeast two-hybrid screening was performed. An in vivo interaction assay in yeast suggests that Ras64B interacts specifically with the Caps intracellular domain. In addition, an in vitro binding assay reveals a direct interaction between an inactive form of Ras64B and the Caps intracellular domain. ras64B, which encodes a small GTPase, is expressed in the mesodermal cells concurrently as caps. Finally, a gain-of-function study with the constitutively active Ras64B suggests that Ras64B presumably functions downstream of Caps. All these results suggest consistently that the small GTPase Ras64B binds specifically to the Caps intracellular domain and may thereby mediate the guidance function of Caps.

Example Based Image Analysis and Synthesis

Image analysis and graphics synthesis can be achieved with learning techniques using directly image examples without physically-based, 3D models. In our technique: -- the mapping from novel images to a vector of "pose" and "expression" parameters can be learned from a small set of example images using a function approximation technique that we call an analysis network; -- the inverse mapping from input "pose" and "expression" parameters to output images can be synthesized from a small set of example images and used to produce new images using a similar synthesis network. The techniques described here have several applications in computer graphics, special effects, interactive multimedia and very low bandwidth teleconferencing.

A Unified Framework for Regularization Networks and Support Vector Machines

Regularization Networks and Support Vector Machines are techniques for solving certain problems of learning from examples -- in particular the regression problem of approximating a multivariate function from sparse data. We present both formulations in a unified framework, namely in the context of Vapnik's theory of statistical learning which provides a general foundation for the learning problem, combining functional analysis and statistics.

Multilevel multitrait-multimethod model: application to the measurement of egocentered social networks

Our goal in this paper is to assess reliability and validity of egocentered network data using multilevel analysis (Muthen, 1989, Hox, 1993) under the multitrait-multimethod approach. The confirmatory factor analysis model for multitrait-multimethod data (Werts & Linn, 1970; Andrews, 1984) is used for our analyses. In this study we reanalyse a part of data of another study (Kogovšek et al., 2002) done on a representative sample of the inhabitants of Ljubljana. The traits used in our article are the name interpreters. We consider egocentered network data as hierarchical; therefore a multilevel analysis is required. We use Muthen's partial maximum likelihood approach, called pseudobalanced solution (Muthen, 1989, 1990, 1994) which produces estimations close to maximum likelihood for large ego sample sizes (Hox & Mass, 2001). Several analyses will be done in order to compare this multilevel analysis to classic methods of analysis such as the ones made in Kogovšek et al. (2002), who analysed the data only at group (ego) level considering averages of all alters within the ego. We show that some of the results obtained by classic methods are biased and that multilevel analysis provides more detailed information that much enriches the interpretation of reliability and validity of hierarchical data. Within and between-ego reliabilities and validities and other related quality measures are defined, computed and interpreted

Networks of PhD Students and Academic Performance: A Comparison across Countries

In this article we compare regression models obtained to predict PhD students’ academic performance in the universities of Girona (Spain) and Slovenia. Explanatory variables are characteristics of PhD student’s research group understood as an egocentered social network, background and attitudinal characteristics of the PhD students and some characteristics of the supervisors. Academic performance was measured by the weighted number of publications. Two web questionnaires were designed, one for PhD students and one for their supervisors and other research group members. Most of the variables were easily comparable across universities due to the careful translation procedure and pre-tests. When direct comparison was not possible we created comparable indicators. We used a regression model in which the country was introduced as a dummy coded variable including all possible interaction effects. The optimal transformations of the main and interaction variables are discussed. Some differences between Slovenian and Girona universities emerge. Some variables like supervisor’s performance and motivation for autonomy prior to starting the PhD have the same positive effect on the PhD student’s performance in both countries. On the other hand, variables like too close supervision by the supervisor and having children have a negative influence in both countries. However, we find differences between countries when we observe the motivation for research prior to starting the PhD which increases performance in Slovenia but not in Girona. As regards network variables, frequency of supervisor advice increases performance in Slovenia and decreases it in Girona. The negative effect in Girona could be explained by the fact that additional contacts of the PhD student with his/her supervisor might indicate a higher workload in addition to or instead of a better advice about the dissertation. The number of external student’s advice relationships and social support mean contact intensity are not significant in Girona, but they have a negative effect in Slovenia. We might explain the negative effect of external advice relationships in Slovenia by saying that a lot of external advice may actually result from a lack of the more relevant internal advice

Availability analysis of GMPLS connections based on physical network topology

This paper presents a study of connection availability in GMPLS over optical transport networks (OTN) taking into account different network topologies. Two basic path protection schemes are considered and compared with the no protection case. The selected topologies are heterogeneous in geographic coverage, network diameter, link lengths, and average node degree. Connection availability is also computed considering the reliability data of physical components and a well-known network availability model. Results show several correspondences between suitable path protection algorithms and several network topology characteristics

TU Graz: Course: 707.000 Web Science and Web Technology: Lecture 4: Social Network Analysis

What are fundamental entities in social networks and what information is contained in social graphs? We will discuss some selected concepts in social network analysis, such as one- and two mode networks, prestige and centrality, and cliques, clans and clubs. Readings: Web tool predicts election results and stock prices, J. Palmer, New Scientist, 07 February (2008) [Protected Access] Optional: Social Network Analysis, Methods and Applications, S. Wasserman and K. Faust (1994)

Allocation rules on networks

When allocating a resource, geographical and infrastructural constraints have to be taken into account. We study the problem of distributing a resource through a network from sources endowed with the resource to citizens with claims. A link between a source and an agent depicts the possibility of a transfer from the source to the agent. Given the supplies at each source, the claims of citizens, and the network, the question is how to allocate the available resources among the citizens. We consider a simple allocation problem that is free of network constraints, where the total amount can be freely distributed. The simple allocation problem is a claims problem where the total amount of claims is greater than what is available. We focus on consistent and resource monotonic rules in claims problems that satisfy equal treatment of equals. We call these rules fairness principles and we extend fairness principles to allocation rules on networks. We require that for each pair of citizens in the network, the extension is robust with respect to the fairness principle. We call this condition pairwise robustness with respect to the fairness principle. We provide an algorithm and show that each fairness principle has a unique extension which is pairwise robust with respect to the fairness principle. We give applications of the algorithm for three fairness principles: egalitarianism, proportionality and equal sacrifice.