Curvature-driven smoothing: a learning algorithm for feed-forward networks

The performance of feed-forward neural networks in real applications can be often be improved significantly if use is made of a-priori information. For interpolation problems this prior knowledge frequently includes smoothness requirements on the network mapping, and can be imposed by the addition to the error function of suitable regularization terms. The new error function, however, now depends on the derivatives of the network mapping, and so the standard back-propagation algorithm cannot be applied. In this paper, we derive a computationally efficient learning algorithm, for a feed-forward network of arbitrary topology, which can be used to minimize the new error function. Networks having a single hidden layer, for which the learning algorithm simplifies, are treated as a special case.

Learning with regularizers in multilayer neural networks

We study the effect of regularization in an on-line gradient-descent learning scenario for a general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labelled by a two-layer teacher network with an arbitrary number of hidden units which may be corrupted by Gaussian output noise. We examine the effect of weight decay regularization on the dynamical evolution of the order parameters and generalization error in various phases of the learning process, in both noiseless and noisy scenarios.

Infer user interests via link structure regularization

Learning user interests from online social networks helps to better understand user behaviors and provides useful guidance to design user-centric applications. Apart from analyzing users' online content, it is also important to consider users' social connections in the social Web. Graph regularization methods have been widely used in various text mining tasks, which can leverage the graph structure information extracted from data. Previously, graph regularization methods operate under the cluster assumption that nearby nodes are more similar and nodes on the same structure (typically referred to as a cluster or a manifold) are likely to be similar. We argue that learning user interests from complex, sparse, and dynamic social networks should be based on the link structure assumption under which node similarities are evaluated based on the local link structures instead of explicit links between two nodes. We propose a regularization framework based on the relation bipartite graph, which can be constructed from any type of relations. Using Twitter as our case study, we evaluate our proposed framework from social networks built from retweet relations. Both quantitative and qualitative experiments show that our proposed method outperforms a few competitive baselines in learning user interests over a set of predefined topics. It also gives superior results compared to the baselines on retweet prediction and topical authority identification. © 2014 ACM.

Regularization meets GreenAI: a new framework for image reconstruction in life sciences applications

Ill-conditioned inverse problems frequently arise in life sciences, particularly in the context of image deblurring and medical image reconstruction. These problems have been addressed through iterative variational algorithms, which regularize the reconstruction by adding prior knowledge about the problem's solution. Despite the theoretical reliability of these methods, their practical utility is constrained by the time required to converge. Recently, the advent of neural networks allowed the development of reconstruction algorithms that can compute highly accurate solutions with minimal time demands. Regrettably, it is well-known that neural networks are sensitive to unexpected noise, and the quality of their reconstructions quickly deteriorates when the input is slightly perturbed. Modern efforts to address this challenge have led to the creation of massive neural network architectures, but this approach is unsustainable from both ecological and economic standpoints. The recently introduced GreenAI paradigm argues that developing sustainable neural network models is essential for practical applications. In this thesis, we aim to bridge the gap between theory and practice by introducing a novel framework that combines the reliability of model-based iterative algorithms with the speed and accuracy of end-to-end neural networks. Additionally, we demonstrate that our framework yields results comparable to state-of-the-art methods while using relatively small, sustainable models. In the first part of this thesis, we discuss the proposed framework from a theoretical perspective. We provide an extension of classical regularization theory, applicable in scenarios where neural networks are employed to solve inverse problems, and we show there exists a trade-off between accuracy and stability. Furthermore, we demonstrate the effectiveness of our methods in common life science-related scenarios. In the second part of the thesis, we initiate an exploration extending the proposed method into the probabilistic domain. We analyze some properties of deep generative models, revealing their potential applicability in addressing ill-posed inverse problems.

Contrasting Effects Of Land Use Intensity And Exotic Host Plants On The Specialization Of Interactions In Plant-herbivore Networks.

Human land use tends to decrease the diversity of native plant species and facilitate the invasion and establishment of exotic ones. Such changes in land use and plant community composition usually have negative impacts on the assemblages of native herbivorous insects. Highly specialized herbivores are expected to be especially sensitive to land use intensification and the presence of exotic plant species because they are neither capable of consuming alternative plant species of the native flora nor exotic plant species. Therefore, higher levels of land use intensity might reduce the proportion of highly specialized herbivores, which ultimately would lead to changes in the specialization of interactions in plant-herbivore networks. This study investigates the community-wide effects of land use intensity on the degree of specialization of 72 plant-herbivore networks, including effects mediated by the increase in the proportion of exotic plant species. Contrary to our expectation, the net effect of land use intensity on network specialization was positive. However, this positive effect of land use intensity was partially canceled by an opposite effect of the proportion of exotic plant species on network specialization. When we analyzed networks composed exclusively of endophagous herbivores separately from those composed exclusively of exophagous herbivores, we found that only endophages showed a consistent change in network specialization at higher land use levels. Altogether, these results indicate that land use intensity is an important ecological driver of network specialization, by way of reducing the local host range of herbivore guilds with highly specialized feeding habits. However, because the effect of land use intensity is offset by an opposite effect owing to the proportion of exotic host species, the net effect of land use in a given herbivore assemblage will likely depend on the extent of the replacement of native host species with exotic ones.

Complex networks: the key to systems biology

Though introduced recently, complex networks research has grown steadily because of its potential to represent, characterize and model a wide range of intricate natural systems and phenomena. Because of the intrinsic complexity and systemic organization of life, complex networks provide a specially promising framework for systems biology investigation. The current article is an up-to-date review of the major developments related to the application of complex networks in biology, with special attention focused on the more recent literature. The main concepts and models of complex networks are presented and illustrated in an accessible fashion. Three main types of networks are covered: transcriptional regulatory networks, protein-protein interaction networks and metabolic networks. The key role of complex networks for systems biology is extensively illustrated by several of the papers reviewed.

Neural networks and statistical analysis for classification of corneal videokeratography maps based on Zernike coefficients: a quantitative comparison

PURPOSE: The main goal of this study was to develop and compare two different techniques for classification of specific types of corneal shapes when Zernike coefficients are used as inputs. A feed-forward artificial Neural Network (NN) and discriminant analysis (DA) techniques were used. METHODS: The inputs both for the NN and DA were the first 15 standard Zernike coefficients for 80 previously classified corneal elevation data files from an Eyesys System 2000 Videokeratograph (VK), installed at the Departamento de Oftalmologia of the Escola Paulista de Medicina, São Paulo. The NN had 5 output neurons which were associated with 5 typical corneal shapes: keratoconus, with-the-rule astigmatism, against-the-rule astigmatism, "regular" or "normal" shape and post-PRK. RESULTS: The NN and DA responses were statistically analyzed in terms of precision ([true positive+true negative]/total number of cases). Mean overall results for all cases for the NN and DA techniques were, respectively, 94% and 84.8%. CONCLUSION: Although we used a relatively small database, results obtained in the present study indicate that Zernike polynomials as descriptors of corneal shape may be a reliable parameter as input data for diagnostic automation of VK maps, using either NN or DA.

Evaluation of follicular lymphoid depletion in the Bursa of Fabricius: an alternative methodology using digital image analysis and artificial neural networks

Fifty Bursa of Fabricius (BF) were examined by conventional optical microscopy and digital images were acquired and processed using Matlab® 6.5 software. The Artificial Neuronal Network (ANN) was generated using Neuroshell® Classifier software and the optical and digital data were compared. The ANN was able to make a comparable classification of digital and optical scores. The use of ANN was able to classify correctly the majority of the follicles, reaching sensibility and specificity of 89% and 96%, respectively. When the follicles were scored and grouped in a binary fashion the sensibility increased to 90% and obtained the maximum value for the specificity of 92%. These results demonstrate that the use of digital image analysis and ANN is a useful tool for the pathological classification of the BF lymphoid depletion. In addition it provides objective results that allow measuring the dimension of the error in the diagnosis and classification therefore making comparison between databases feasible.

Classification of breast masses using a committee machine of artificial neural networks

This work proposes a new approach using a committee machine of artificial neural networks to classify masses found in mammograms as benign or malignant. Three shape factors, three edge-sharpness measures, and 14 texture measures are used for the classification of 20 regions of interest (ROIs) related to malignant tumors and 37 ROIs related to benign masses. A group of multilayer perceptrons (MLPs) is employed as a committee machine of neural network classifiers. The classification results are reached by combining the responses of the individual classifiers. Experiments involving changes in the learning algorithm of the committee machine are conducted. The classification accuracy is evaluated using the area A. under the receiver operating characteristics (ROC) curve. The A, result for the committee machine is compared with the A, results obtained using MLPs and single-layer perceptrons (SLPs), as well as a linear discriminant analysis (LDA) classifier Tests are carried out using the student's t-distribution. The committee machine classifier outperforms the MLP SLP, and LDA classifiers in the following cases: with the shape measure of spiculation index, the A, values of the four methods are, in order 0.93, 0.84, 0.75, and 0.76; and with the edge-sharpness measure of acutance, the values are 0.79, 0.70, 0.69, and 0.74. Although the features with which improvement is obtained with the committee machines are not the same as those that provided the maximal value of A(z) (A(z) = 0.99 with some shape features, with or without the committee machine), they correspond to features that are not critically dependent on the accuracy of the boundaries of the masses, which is an important result. (c) 2008 SPIE and IS&T.

Fully Connected PLL Networks: How Filter Determines the Number of Nodes

Synchronization plays an important role in telecommunication systems, integrated circuits, and automation systems. Formerly, the masterslave synchronization strategy was used in the great majority of cases due to its reliability and simplicity. Recently, with the wireless networks development, and with the increase of the operation frequency of integrated circuits, the decentralized clock distribution strategies are gaining importance. Consequently, fully connected clock distribution systems with nodes composed of phase-locked loops (PLLs) appear as a convenient engineering solution. In this work, the stability of the synchronous state of these networks is studied in two relevant situations: when the node filters are first-order lag-lead low-pass or when the node filters are second-order low-pass. For first-order filters, the synchronous state of the network shows to be stable for any number of nodes. For second-order filter, there is a superior limit for the number of nodes, depending on the PLL parameters. Copyright (C) 2009 Atila Madureira Bueno et al.

A reliable measure of similarity based on dependency for short time series: an application to gene expression networks

Background: Microarray techniques have become an important tool to the investigation of genetic relationships and the assignment of different phenotypes. Since microarrays are still very expensive, most of the experiments are performed with small samples. This paper introduces a method to quantify dependency between data series composed of few sample points. The method is used to construct gene co-expression subnetworks of highly significant edges. Results: The results shown here are for an adapted subset of a Saccharomyces cerevisiae gene expression data set with low temporal resolution and poor statistics. The method reveals common transcription factors with a high confidence level and allows the construction of subnetworks with high biological relevance that reveals characteristic features of the processes driving the organism adaptations to specific environmental conditions. Conclusion: Our method allows a reliable and sophisticated analysis of microarray data even under severe constraints. The utilization of systems biology improves the biologists ability to elucidate the mechanisms underlying celular processes and to formulate new hypotheses.

The Missing Part of Seed Dispersal Networks: Structure and Robustness of Bat-Fruit Interactions

Mutualistic networks are crucial to the maintenance of ecosystem services. Unfortunately, what we know about seed dispersal networks is based only on bird-fruit interactions. Therefore, we aimed at filling part of this gap by investigating bat-fruit networks. It is known from population studies that: (i) some bat species depend more on fruits than others, and (ii) that some specialized frugivorous bats prefer particular plant genera. We tested whether those preferences affected the structure and robustness of the whole network and the functional roles of species. Nine bat-fruit datasets from the literature were analyzed and all networks showed lower complementary specialization (H(2)' = 0.3760.10, mean 6 SD) and similar nestedness (NODF = 0.5660.12) than pollination networks. All networks were modular (M=0.32 +/- 0.07), and had on average four cohesive subgroups (modules) of tightly connected bats and plants. The composition of those modules followed the genus-genus associations observed at population level (Artibeus-Ficus, Carollia-Piper, and Sturnira-Solanum), although a few of those plant genera were dispersed also by other bats. Bat-fruit networks showed high robustness to simulated cumulative removals of both bats (R = 0.55 +/- 0.10) and plants (R = 0.68 +/- 0.09). Primary frugivores interacted with a larger proportion of the plants available and also occupied more central positions; furthermore, their extinction caused larger changes in network structure. We conclude that bat-fruit networks are highly cohesive and robust mutualistic systems, in which redundancy is high within modules, although modules are complementary to each other. Dietary specialization seems to be an important structuring factor that affects the topology, the guild structure and functional roles in bat-fruit networks.

Social Networks Shape the Transmission Dynamics of Hepatitis C Virus

Hepatitis C virus (HCV) infects 170 million people worldwide, and is a major public health problem in Brazil, where over 1% of the population may be infected and where multiple viral genotypes co-circulate. Chronically infected individuals are both the source of transmission to others and are at risk for HCV-related diseases, such as liver cancer and cirrhosis. Before the adoption of anti-HCV control measures in blood banks, this virus was mainly transmitted via blood transfusion. Today, needle sharing among injecting drug users is the most common form of HCV transmission. Of particular importance is that HCV prevalence is growing in non-risk groups. Since there is no vaccine against HCV, it is important to determine the factors that control viral transmission in order to develop more efficient control measures. However, despite the health costs associated with HCV, the factors that determine the spread of virus at the epidemiological scale are often poorly understood. Here, we sequenced partial NS5b gene sequences sampled from blood samples collected from 591 patients in Sao Paulo state, Brazil. We show that different viral genotypes entered Sao Paulo at different times, grew at different rates, and are associated with different age groups and risk behaviors. In particular, subtype 1b is older and grew more slowly than subtypes 1a and 3a, and is associated with multiple age classes. In contrast, subtypes 1a and 3b are associated with younger people infected more recently, possibly with higher rates of sexual transmission. The transmission dynamics of HCV in Sao Paulo therefore vary by subtype and are determined by a combination of age, risk exposure and underlying social network. We conclude that social factors may play a key role in determining the rate and pattern of HCV spread, and should influence future intervention policies.

Characterization of subgraph relationships and distribution in complex networks

A network can be analyzed at different topological scales, ranging from single nodes to motifs, communities, up to the complete structure. We propose a novel approach which extends from single nodes to the whole network level by considering non-overlapping subgraphs (i.e. connected components) and their interrelationships and distribution through the network. Though such subgraphs can be completely general, our methodology focuses on the cases in which the nodes of these subgraphs share some special feature, such as being critical for the proper operation of the network. The methodology of subgraph characterization involves two main aspects: (i) the generation of histograms of subgraph sizes and distances between subgraphs and (ii) a merging algorithm, developed to assess the relevance of nodes outside subgraphs by progressively merging subgraphs until the whole network is covered. The latter procedure complements the histograms by taking into account the nodes lying between subgraphs, as well as the relevance of these nodes to the overall subgraph interconnectivity. Experiments were carried out using four types of network models and five instances of real-world networks, in order to illustrate how subgraph characterization can help complementing complex network-based studies.

PARTITIONED ANALYSIS FOR DIMENSIONALLY-HETEROGENEOUS HYDRAULIC NETWORKS

In this work an iterative strategy is developed to tackle the problem of coupling dimensionally-heterogeneous models in the context of fluid mechanics. The procedure proposed here makes use of a reinterpretation of the original problem as a nonlinear interface problem for which classical nonlinear solvers can be applied. Strong coupling of the partitions is achieved while dealing with different codes for each partition, each code in black-box mode. The main application for which this procedure is envisaged arises when modeling hydraulic networks in which complex and simple subsystems are treated using detailed and simplified models, correspondingly. The potentialities and the performance of the strategy are assessed through several examples involving transient flows and complex network configurations.