Unfolding kernel embeddings of graphs:enhancing class separation through manifold learning

In this paper, we investigate the use of manifold learning techniques to enhance the separation properties of standard graph kernels. The idea stems from the observation that when we perform multidimensional scaling on the distance matrices extracted from the kernels, the resulting data tends to be clustered along a curve that wraps around the embedding space, a behavior that suggests that long range distances are not estimated accurately, resulting in an increased curvature of the embedding space. Hence, we propose to use a number of manifold learning techniques to compute a low-dimensional embedding of the graphs in an attempt to unfold the embedding manifold, and increase the class separation. We perform an extensive experimental evaluation on a number of standard graph datasets using the shortest-path (Borgwardt and Kriegel, 2005), graphlet (Shervashidze et al., 2009), random walk (Kashima et al., 2003) and Weisfeiler-Lehman (Shervashidze et al., 2011) kernels. We observe the most significant improvement in the case of the graphlet kernel, which fits with the observation that neglecting the locational information of the substructures leads to a stronger curvature of the embedding manifold. On the other hand, the Weisfeiler-Lehman kernel partially mitigates the locality problem by using the node labels information, and thus does not clearly benefit from the manifold learning. Interestingly, our experiments also show that the unfolding of the space seems to reduce the performance gap between the examined kernels.

Manifold learning and the quantum Jensen-Shannon divergence kernel

The quantum Jensen-Shannon divergence kernel [1] was recently introduced in the context of unattributed graphs where it was shown to outperform several commonly used alternatives. In this paper, we study the separability properties of this kernel and we propose a way to compute a low-dimensional kernel embedding where the separation of the different classes is enhanced. The idea stems from the observation that the multidimensional scaling embeddings on this kernel show a strong horseshoe shape distribution, a pattern which is known to arise when long range distances are not estimated accurately. Here we propose to use Isomap to embed the graphs using only local distance information onto a new vectorial space with a higher class separability. The experimental evaluation shows the effectiveness of the proposed approach. © 2013 Springer-Verlag.

Support enhanced α-pinene isomerization over HPW/SBA-15

A family of mesoporous SBA-15 supported H3PW12O40 (HPW) catalysts were synthesized by wet-impregnation and compared with fumed silica analogues for the solventless isomerization of α-pinene under mild conditions. Structural and acidic properties of supported HPW materials were characterized by powder XRD, HRTEM, XPS, TGA, N2 porosimetry, DRIFTS, and ammonia and propylamine chemisorption and TPD. The high area, mesoporous SBA-15 architecture facilitates the formation of highly dispersed (isolated or low dimensional) HPW clusters and concomitant high acid site densities (up to 0.54 mmol g−1) relative to fumed silica wherein large HPW crystallites are formed even at low HPW loadings. α-Pinene exhibits a volcano dependence on HPW loading over the SBA-15 support due to competition between the number and accessibility of acid sites to the non-polar reactant, with the superior acid site accessibility for HPW/SBA-15 conferring a 10-fold rate enhancement with respect to HPW/fumed silica and pure HPW. Monocyclic limonene and terpinolene products are favoured over polycyclic camphene and β-pinene by weaker polyoxometallate analogues over SBA-15.

Study of carbon nanotube film and shape memory alloys treatment for sensing and structural vibration control

Structural vibration control is of great importance. Current active and passive vibration control strategies usually employ individual elements to fulfill this task, such as viscoelastic patches for providing damping, transducers for picking up signals and actuators for inputting actuating forces. The goal of this dissertation work is to design, manufacture, investigate and apply a new type of multifunctional composite material for structural vibration control. This new composite, which is based on multi-walled carbon nanotube (MWCNT) film, is potentially to function as free layer damping treatment and strain sensor simultaneously. That is, the new material integrates the transducer and the damping patch into one element. The multifunctional composite was prepared by sandwiching the MWCNT film between two adhesive layers. Static sensing test indicated that the MWCNT film sensor resistance changes almost linearly with the applied load. Sensor sensitivity factors were comparable to those of the foil strain gauges. Dynamic test indicated that the MWCNT film sensor can outperform the foil strain gage in high frequency ranges. Temperature test indicated the MWCNT sensor had good temperature stability over the range of 237 K-363 K. The Young’s modulus and shear modulus of the MWCNT film composite were acquired by nanoindentation test and direct shear test, respectively. A free vibration damping test indicated that the MWCNT composite sensor can also provide good damping without adding excessive weight to the base structure. A new model for sandwich structural vibration control was then proposed. In this new configuration, a cantilever beam covered with MWCNT composite on top and one layer of shape memory alloy (SMA) on the bottom was used to illustrate this concept. The MWCNT composite simultaneously serves as free layer damping and strain sensor, and the SMA acts as actuator. Simple on-off controller was designed for controlling the temperature of the SMA so as to control the SMA recovery stress as input and the system stiffness. Both free and forced vibrations were analyzed. Simulation work showed that this new configuration for sandwich structural vibration control was successful especially for low frequency system.

New Advancements of Scalable Statistical Methods for Learning Latent Structures in Big Data

Constant technology advances have caused data explosion in recent years. Accord- ingly modern statistical and machine learning methods must be adapted to deal with complex and heterogeneous data types. This phenomenon is particularly true for an- alyzing biological data. For example DNA sequence data can be viewed as categorical variables with each nucleotide taking four different categories. The gene expression data, depending on the quantitative technology, could be continuous numbers or counts. With the advancement of high-throughput technology, the abundance of such data becomes unprecedentedly rich. Therefore efficient statistical approaches are crucial in this big data era.

Previous statistical methods for big data often aim to find low dimensional struc- tures in the observed data. For example in a factor analysis model a latent Gaussian distributed multivariate vector is assumed. With this assumption a factor model produces a low rank estimation of the covariance of the observed variables. Another example is the latent Dirichlet allocation model for documents. The mixture pro- portions of topics, represented by a Dirichlet distributed variable, is assumed. This dissertation proposes several novel extensions to the previous statistical methods that are developed to address challenges in big data. Those novel methods are applied in multiple real world applications including construction of condition specific gene co-expression networks, estimating shared topics among newsgroups, analysis of pro- moter sequences, analysis of political-economics risk data and estimating population structure from genotype data.

Learning from Geometry

Subspaces and manifolds are two powerful models for high dimensional signals. Subspaces model linear correlation and are a good fit to signals generated by physical systems, such as frontal images of human faces and multiple sources impinging at an antenna array. Manifolds model sources that are not linearly correlated, but where signals are determined by a small number of parameters. Examples are images of human faces under different poses or expressions, and handwritten digits with varying styles. However, there will always be some degree of model mismatch between the subspace or manifold model and the true statistics of the source. This dissertation exploits subspace and manifold models as prior information in various signal processing and machine learning tasks.

A near-low-rank Gaussian mixture model measures proximity to a union of linear or affine subspaces. This simple model can effectively capture the signal distribution when each class is near a subspace. This dissertation studies how the pairwise geometry between these subspaces affects classification performance. When model mismatch is vanishingly small, the probability of misclassification is determined by the product of the sines of the principal angles between subspaces. When the model mismatch is more significant, the probability of misclassification is determined by the sum of the squares of the sines of the principal angles. Reliability of classification is derived in terms of the distribution of signal energy across principal vectors. Larger principal angles lead to smaller classification error, motivating a linear transform that optimizes principal angles. This linear transformation, termed TRAIT, also preserves some specific features in each class, being complementary to a recently developed Low Rank Transform (LRT). Moreover, when the model mismatch is more significant, TRAIT shows superior performance compared to LRT.

The manifold model enforces a constraint on the freedom of data variation. Learning features that are robust to data variation is very important, especially when the size of the training set is small. A learning machine with large numbers of parameters, e.g., deep neural network, can well describe a very complicated data distribution. However, it is also more likely to be sensitive to small perturbations of the data, and to suffer from suffer from degraded performance when generalizing to unseen (test) data.

From the perspective of complexity of function classes, such a learning machine has a huge capacity (complexity), which tends to overfit. The manifold model provides us with a way of regularizing the learning machine, so as to reduce the generalization error, therefore mitigate overfiting. Two different overfiting-preventing approaches are proposed, one from the perspective of data variation, the other from capacity/complexity control. In the first approach, the learning machine is encouraged to make decisions that vary smoothly for data points in local neighborhoods on the manifold. In the second approach, a graph adjacency matrix is derived for the manifold, and the learned features are encouraged to be aligned with the principal components of this adjacency matrix. Experimental results on benchmark datasets are demonstrated, showing an obvious advantage of the proposed approaches when the training set is small.

Stochastic optimization makes it possible to track a slowly varying subspace underlying streaming data. By approximating local neighborhoods using affine subspaces, a slowly varying manifold can be efficiently tracked as well, even with corrupted and noisy data. The more the local neighborhoods, the better the approximation, but the higher the computational complexity. A multiscale approximation scheme is proposed, where the local approximating subspaces are organized in a tree structure. Splitting and merging of the tree nodes then allows efficient control of the number of neighbourhoods. Deviation (of each datum) from the learned model is estimated, yielding a series of statistics for anomaly detection. This framework extends the classical {\em changepoint detection} technique, which only works for one dimensional signals. Simulations and experiments highlight the robustness and efficacy of the proposed approach in detecting an abrupt change in an otherwise slowly varying low-dimensional manifold.

Weak coupling of acoustic-like phonons and magnon dynamics in incommensurate spin ladder compound Sr14Cu24O41

Intriguing lattice dynamics has been predicted for aperiodic crystals that contain incommensurate substructures. Here we report inelastic neutron scattering measurements of phonon and magnon dispersions in Sr14Cu24O41, which contains incommensurate one-dimensional (1D) chain and two-dimensional (2D) ladder substructures. Two distinct acoustic phonon-like modes, corresponding to the sliding motion of one sublattice against the other, are observed for atomic motions polarized along the incommensurate axis. In the long wavelength limit, it is found that the sliding mode shows a remarkably small energy gap of 1.7-1.9 meV, indicating very weak interactions between the two incommensurate sublattices. The measurements also reveal a gapped and steep linear magnon dispersion of the ladder sublattice. The high group velocity of this magnon branch and weak coupling with acoustic phonons can explain the large magnon thermal conductivity in Sr14Cu24O41 crystals. In addition, the magnon specific heat is determined from the measured total specific heat and phonon density of states, and exhibits a Schottky anomaly due to gapped magnon modes of the spin chains. These findings offer new insights into the phonon and magnon dynamics and thermal transport properties of incommensurate magnetic crystals that contain low-dimensional substructures.

Bayesian Learning with Dependency Structures via Latent Factors, Mixtures, and Copulas

Bayesian methods offer a flexible and convenient probabilistic learning framework to extract interpretable knowledge from complex and structured data. Such methods can characterize dependencies among multiple levels of hidden variables and share statistical strength across heterogeneous sources. In the first part of this dissertation, we develop two dependent variational inference methods for full posterior approximation in non-conjugate Bayesian models through hierarchical mixture- and copula-based variational proposals, respectively. The proposed methods move beyond the widely used factorized approximation to the posterior and provide generic applicability to a broad class of probabilistic models with minimal model-specific derivations. In the second part of this dissertation, we design probabilistic graphical models to accommodate multimodal data, describe dynamical behaviors and account for task heterogeneity. In particular, the sparse latent factor model is able to reveal common low-dimensional structures from high-dimensional data. We demonstrate the effectiveness of the proposed statistical learning methods on both synthetic and real-world data.

Person Re-Identification using Deep Convnets with Multi-task Learning

Person re-identification involves recognizing a person across non-overlapping camera views, with different pose, illumination, and camera characteristics. We propose to tackle this problem by training a deep convolutional network to represent a person’s appearance as a low-dimensional feature vector that is invariant to common appearance variations encountered in the re-identification problem. Specifically, a Siamese-network architecture is used to train a feature extraction network using pairs of similar and dissimilar images. We show that use of a novel multi-task learning objective is crucial for regularizing the network parameters in order to prevent over-fitting due to the small size the training dataset. We complement the verification task, which is at the heart of re-identification, by training the network to jointly perform verification, identification, and to recognise attributes related to the clothing and pose of the person in each image. Additionally, we show that our proposed approach performs well even in the challenging cross-dataset scenario, which may better reflect real-world expected performance. 

Convex Optimization Algorithms and Statistical Bounds for Learning Structured Models

Thesis (Ph.D.)--University of Washington, 2016-08

Optimização dos consumos de água na irrigação de campos golfe

A presente dissertação propõe o desenvolvimento de um sistema de Irrigação de baixo custo para campos de Golfe. Este sistema é capaz de recolher a previsão meteorológica e ainda medir um conjunto de valores (temperatura, humidade, velocidade do vento) que determina quando e quanto regar. Os campos de Golfe consumem diariamente elevadas quantidades de água, sendo esta a principal crítica feita pelas organizações ambientais. Esta dissertação incorpora uma comunicação sem fios de baixo custo, que dispensa a cablagem que é necessária para haver comunicação entre os diversos equipamentos, que estão distribuídos pelo campo de Golfe. O sistema desenvolvido pretende reduzir os desperdícios dos recursos hídricos na rega, pois é um sistema inteligente que poderá ser adquirido não só por gestores de campos de Golfe, mas também por jardins residenciais e municipais. Com o objetivo de criar um sistema de baixo custo foi elaborado um algoritmo de reencaminhamento de mensagens, que permite utilizar equipamentos de comunicação sem fios de baixo custo. Todo o sistema de Irrigação é controlado e monitorizado através de uma interface, desenvolvida em Microsoft Visual Basic.

Likelihood-Based Inference for Partially Observed Multi-Type Markov Branching Processes

Thesis (Ph.D.)--University of Washington, 2016-08

Desinfeção aplicada aos cones de guta-percha

Introdução: Ao longo do tempo o Tratamento Endodôntico Não Cirúrgico tem sido das áreas da Medicina Dentária que mais tem evoluído. Todos os passos do tratamento têm sido revistos de forma a aumentar a taxa de sucesso. O controlo microbiológico é crucial para que o tratamento seja um sucesso a curto, médio e longo prazo. A assepsia deve ser mantida em todas as fases deste tratamento para que este seja um sucesso. Objetivo: Ao longo do meu percurso académico pude concluir que a fase da descontaminação dos cones, aquando a obturação (fase final do Tratamento Endodôntico Não Cirúrgico) era desvalorizada, o que me levou a efetuar uma revisão bibliográfica de modo a poder melhorar os meus conhecimentos e técnica. Material e Métodos: Para a elaboração deste trabalho foi realizada uma pesquisa bibliográfica recorrendo aos seguintes motores de busca: B-on, PubMed, Scielo e ScienceDirect, com as seguintes palavras-chave: “decontamination in endodontics”;” disinfection in endodontics”; “root canal irrigants”; “endodontics microbiology”; “Candida albicans“; “Enterococcus faecalis”; “sodium hypochlorite ”; “alcohol”; “contamination during Obturation”; “clorohexidine”; “filling materials endodontics”; “termoplastic gutta-percha”; “obturation material”; “Mineral Trioxide Aggregate”; “resilon”; “resin cement”; “resin material for root canal obturation”; “resin sealer”; “root canal”; “root canal sealing”; “root canal filling materials”; “condensation in endodontics”; “lateral condensation”; “gutta-percha”; “microlekeage”; “system B”; “fluid filtration model”;“dye penetration”. Como critério de inclusão estabeleceu-se que os artigos deveriam ser em Português, Inglês ou Espanhol e publicados entre 1995 e 2015. Dos resultados apresentados foram utilizados 110 artigos, pesquisados entre Maio de 2015 e 20 de Outubro de 2015. Foram ainda consultados livros de referência nestes mesmos locais. Conclusão: a presença de bactérias e os seus subprodutos no sistema tridimensional de canais está diretamente implicado com o insucesso do Tratamento Endodôntico. A descontaminação dos cones de guta-percha, é, portanto, um processo importante no Tratamento Endodôntico pois impede que os cones sejam colocados nos canais radiculares, estando contaminados por microorganismos que inviabilizam o tratamento efetuado. A submersão dos cones durante um minuto em clorohexidina a 2% ou hipoclorito a 5,25% está indicado e comprovado como um processo eficiente de desinfeção dos cones.

Controlling Nanostructures for in-situ TEM Characterization

Low dimensional nanostructures, such as nanotubes and 2D sheets, have unique and promising material properties both from a fundamental science and an application standpoint. Theoretical modelling and calculations predict previously unobserved phenomena that experimental scientists often struggle to reproduce because of the difficulty in controlling and characterizing the small structures under real-world constraints. The goal of this dissertation is to controlling these structures so that nanostructures can be characterized in-situ in transmission electron microscopes (TEM) allowing for direct observation of the actual physical responses of the materials to different stimuli. Of most interest to this work are the thermal and electrical properties of carbon nanotubes, boron nitride nanotubes, and graphene. The first topic of the dissertation is using surfactants for aqueous processing to fabricate, store, and deposit the nanostructures. More specifically, thorough characterization of a new surfactant, ammonium laurate (AL), is provided and shows that this new surfactant outperforms the standard surfactant for these materials, sodium dodecyl sulfate (SDS), in almost all tested metrics. New experimental set-ups have been developed by combining specialized in-situ TEM holders with innovative device fabrication. For example, electrical characterization of graphene was performed by using an STM-TEM holder and depositing graphene from aqueous solutions onto lithographically patterned, electron transparent silicon nitride membranes. These experiments produce exciting information about the interaction between graphene and metal probes and the substrate that it rests on. Then, by adding indium to the backside of the membrane and employing the electron thermal microscopy (EThM) technique, the same type of graphene samples could be characterized for thermal transport with high spatial resolution. It is found that reduced graphene oxide sheets deposited onto a silicon nitride membrane and displaying high levels of wrinkling have higher than expected electrical and thermal conduction properties. We are clearly able to visualize the ability of graphene to spread heat away from an electronic hot spot and into the substrate.