Learning from Geometry

Subspaces and manifolds are two powerful models for high dimensional signals. Subspaces model linear correlation and are a good fit to signals generated by physical systems, such as frontal images of human faces and multiple sources impinging at an antenna array. Manifolds model sources that are not linearly correlated, but where signals are determined by a small number of parameters. Examples are images of human faces under different poses or expressions, and handwritten digits with varying styles. However, there will always be some degree of model mismatch between the subspace or manifold model and the true statistics of the source. This dissertation exploits subspace and manifold models as prior information in various signal processing and machine learning tasks.

A near-low-rank Gaussian mixture model measures proximity to a union of linear or affine subspaces. This simple model can effectively capture the signal distribution when each class is near a subspace. This dissertation studies how the pairwise geometry between these subspaces affects classification performance. When model mismatch is vanishingly small, the probability of misclassification is determined by the product of the sines of the principal angles between subspaces. When the model mismatch is more significant, the probability of misclassification is determined by the sum of the squares of the sines of the principal angles. Reliability of classification is derived in terms of the distribution of signal energy across principal vectors. Larger principal angles lead to smaller classification error, motivating a linear transform that optimizes principal angles. This linear transformation, termed TRAIT, also preserves some specific features in each class, being complementary to a recently developed Low Rank Transform (LRT). Moreover, when the model mismatch is more significant, TRAIT shows superior performance compared to LRT.

The manifold model enforces a constraint on the freedom of data variation. Learning features that are robust to data variation is very important, especially when the size of the training set is small. A learning machine with large numbers of parameters, e.g., deep neural network, can well describe a very complicated data distribution. However, it is also more likely to be sensitive to small perturbations of the data, and to suffer from suffer from degraded performance when generalizing to unseen (test) data.

From the perspective of complexity of function classes, such a learning machine has a huge capacity (complexity), which tends to overfit. The manifold model provides us with a way of regularizing the learning machine, so as to reduce the generalization error, therefore mitigate overfiting. Two different overfiting-preventing approaches are proposed, one from the perspective of data variation, the other from capacity/complexity control. In the first approach, the learning machine is encouraged to make decisions that vary smoothly for data points in local neighborhoods on the manifold. In the second approach, a graph adjacency matrix is derived for the manifold, and the learned features are encouraged to be aligned with the principal components of this adjacency matrix. Experimental results on benchmark datasets are demonstrated, showing an obvious advantage of the proposed approaches when the training set is small.

Stochastic optimization makes it possible to track a slowly varying subspace underlying streaming data. By approximating local neighborhoods using affine subspaces, a slowly varying manifold can be efficiently tracked as well, even with corrupted and noisy data. The more the local neighborhoods, the better the approximation, but the higher the computational complexity. A multiscale approximation scheme is proposed, where the local approximating subspaces are organized in a tree structure. Splitting and merging of the tree nodes then allows efficient control of the number of neighbourhoods. Deviation (of each datum) from the learned model is estimated, yielding a series of statistics for anomaly detection. This framework extends the classical {\em changepoint detection} technique, which only works for one dimensional signals. Simulations and experiments highlight the robustness and efficacy of the proposed approach in detecting an abrupt change in an otherwise slowly varying low-dimensional manifold.

Design of Latent Thermal Energy Storage Systems

This dissertation documents the results of a theoretical and numerical study of time dependent storage of energy by melting a phase change material. The heating is provided along invading lines, which change from single-line invasion to tree-shaped invasion. Chapter 2 identifies the special design feature of distributing energy storage in time-dependent fashion on a territory, when the energy flows by fluid flow from a concentrated source to points (users) distributed equidistantly on the area. The challenge in this chapter is to determine the architecture of distributed energy storage. The chief conclusion is that the finite amount of storage material should be distributed proportionally with the distribution of the flow rate of heating agent arriving on the area. The total time needed by the source stream to ‘invade’ the area is cumulative (the sum of the storage times required at each storage site), and depends on the energy distribution paths and the sequence in which the users are served by the source stream. Chapter 3 shows theoretically that the melting process consists of two phases: “invasion” thermal diffusion along the invading line, which is followed by “consolidation” as heat diffuses perpendicularly to the invading line. This chapter also reports the duration of both phases and the evolution of the melt layer around the invading line during the two-dimensional and three-dimensional invasion. It also shows that the amount of melted material increases in time according to a curve shaped as an S. These theoretical predictions are validated by means of numerical simulations in chapter 4. This chapter also shows that the heat transfer rate density increases (i.e., the S curve becomes steeper) as the complexity and number of degrees of freedom of the structure are increased, in accord with the constructal law. The optimal geometric features of the tree structure are detailed in this chapter. Chapter 5 documents a numerical study of time-dependent melting where the heat transfer is convection dominated, unlike in chapter 3 and 4 where the melting is ruled by pure conduction. In accord with constructal design, the search is for effective heat-flow architectures. The volume-constrained improvement of the designs for heat flow begins with assuming the simplest structure, where a single line serves as heat source. Next, the heat source is endowed with freedom to change its shape as it grows. The objective of the numerical simulations is to discover the geometric features that lead to the fastest melting process. The results show that the heat transfer rate density increases as the complexity and number of degrees of freedom of the structure are increased. Furthermore, the angles between heat invasion lines have a minor effect on the global performance compared to other degrees of freedom: number of branching levels, stem length, and branch lengths. The effect of natural convection in the melt zone is documented.

Using virtual topology operations to generate analysis topology

Virtual topology operations have been utilized to generate an analysis topology definition suitable for downstream mesh generation. Detailed descriptions are provided for virtual topology merge and split operations for all topological entities. Current virtual topology technology is extended to allow the virtual partitioning of volume cells and the topological queries required to carry out each operation are provided. Virtual representations are robustly linked to the underlying geometric definition through an analysis topology. The analysis topology and all associated virtual and topological dependencies are automatically updated after each virtual operation, providing the link to the underlying CAD geometry. Therefore, a valid description of the analysis topology, including relative orientations, is maintained. This enables downstream operations, such as the merging or partitioning of virtual entities, and interrogations, such as determining if a specific meshing strategy can be applied to the virtual volume cells, to be performed on the analysis topology description. As the virtual representation is a non-manifold description of the sub-divided domain the interfaces between cells are recorded automatically. This enables the advantages of non-manifold modelling to be exploited within the manifold modelling environment of a major commercial CAD system, without any adaptation of the underlying CAD model. A hierarchical virtual structure is maintained where virtual entities are merged or partitioned. This has a major benefit over existing solutions as the virtual dependencies are stored in an open and accessible manner, providing the analyst with the freedom to create, modify and edit the analysis topology in any preferred sequence, whilst the original CAD geometry is not disturbed. Robust definitions of the topological and virtual dependencies enable the same virtual topology definitions to be accessed, interrogated and manipulated within multiple different CAD packages and linked to the underlying geometry.

Caracterização das comunidades de aves florestais no distrito de Bragança

Com objetivo de caracterizar a comunidade de aves florestais do Distrito de Bragança, fez-se um estudo em quatro locais de características diferentes sob ponto de vista de estrutura e composição de espécies, através de recolha de informação com base nas listas de 10 espécies de Mackinnon e pelo método captura com redes de neblina, para caracterização das biometrias e parâmetros populacionais, de modo a caracterizar a estrutura das comunidades de aves em cada local. A Combinação dos dois métodos possibilitou a captura e identificação de indivíduos de 44 espécies diferentes, pertencentes a 24 famílias distintas. Destas, as mais representativas foram determinadas através do Índice de Frequência das Listas (IFL) e foram Parus major, Erithacus rubecula, Turdus merula e Sylvia atricapilla. Os resultados referentes à riqueza específica, obtidos através das listas de MacKinnon foram analisados com ANOVA não paramétricas (Kruskal-Wallis). Dos quatro locais, a Ricafé mostrou diferenças significativas na variação de número de espécies. Em relação aos períodos de capturas, o segundo quadrimestre teve número de espécies estatisticamente diferentes dos restantes quadrimestres do ano. No Período de Invernada há menor atividade de aves em todos os locais. Pinhal e Tabuado têm pouca diversidade de espécies.

Allocating reforestation areas for sediment flow minimization: an integer programming formulation and a heuristic solution method

Policy and decision makers dealing with environmental conservation and land use planning often require identifying potential sites for contributing to minimize sediment flow reaching riverbeds. This is the case of reforestation initiatives, which can have sediment flow minimization among their objectives. This paper proposes an Integer Programming (IP) formulation and a Heuristic solution method for selecting a predefined number of locations to be reforested in order to minimize sediment load at a given outlet in a watershed. Although the core structure of both methods can be applied for different sorts of flow, the formulations are targeted to minimization of sediment delivery. The proposed approaches make use of a Single Flow Direction (SFD) raster map covering the watershed in order to construct a tree structure so that the outlet cell corresponds to the root node in the tree. The results obtained with both approaches are in agreement with expert assessments of erosion levels, slopes and distances to the riverbeds, which in turn allows concluding that this approach is suitable for minimizing sediment flow. Since the results obtained with the IP formulation are the same as the ones obtained with the Heuristic approach, an optimality proof is included in the present work. Taking into consideration that the heuristic requires much less computation time, this solution method is more suitable to be applied in large sized problems.

Synthesis of a porous sheet-like V₂O₅-CNT nanocomposite using an ice-templating 'bricks-and-mortar' assembly approach as a high-capacity, long cyclelife cathode material for lithium-ion batteries

Tailoring the nanostructures of electrode materials is an effective way to enhance their electrochemical performance for energy storage. Herein, an ice-templating "bricks-and-mortar" assembly approach is reported to make ribbon-like V2O5 nanoparticles and CNTs integrated into a two-dimensional (2D) porous sheet-like V2O5-CNT nanocomposite. The obtained sheet-like V2O5-CNT nanocomposite possesses unique structural characteristics, including a hierarchical porous structure, 2D morphology, large specific surface area and internal conducting networks, which lead to superior electrochemical performances in terms of long-term cyclability and significantly enhanced rate capability when used as a cathode material for LIBs. The sheet-like V2O5-CNT nanocomposite can charge/discharge at high rates of 5C, 10C and 20C, with discharge capacities of approximately 240 mA h g-1, 180 mA h g-1, and 160 mA h g-1, respectively. It also retains 71% of the initial discharge capacity after 300 cycles at a high rate of 5C, with only 0.097% capacity loss per cycle. The rate capability and cycling performance of the sheet-like V2O5-CNT nanocomposite are significantly better than those of commercial V2O5 and most of the reported V2O5 nanocomposite.

Supporting business intelligence usage: an integrated framework with automatic weighting

This paper presents an integrated framework that comprises an automatic weighting method for assessing data quality (DQ) of the framework so as to better support the business intelligence (BI) usage. Specifically, we utilize business process modeling (BPM) notation and information product map and frame them into a hierarchical mapping structure. Furthermore, we develop and demonstrate an automatic weight-assignment method for evaluating critical dimensions (i.e., completeness and accuracy) of DQ of the integrated framework. Through a design science paradigm, the effectiveness of the framework and the associated DQ weighting method has been rigorously validated by faculty management users of a university. The framework together with the DQ weighting method builds user confidence by enhancing the traceability of a BI product. The automatic DQ weight assignment also provides better time efficiency because the weight of each data attribute is determined automatically based on its usage on the BI dashboard.

Tree-Structured Stick Breaking Processes for Hierarchical Data

Many data are naturally modeled by an unobserved hierarchical structure. In this paper we propose a flexible nonparametric prior over unknown data hierarchies. The approach uses nested stick-breaking processes to allow for trees of unbounded width and depth, where data can live at any node and are infinitely exchangeable. One can view our model as providing infinite mixtures where the components have a dependency structure corresponding to an evolutionary diffusion down a tree. By using a stick-breaking approach, we can apply Markov chain Monte Carlo methods based on slice sampling to perform Bayesian inference and simulate from the posterior distribution on trees. We apply our method to hierarchical clustering of images and topic modeling of text data.

Mating system parameters at hierarchical levels of fruits, individuals and populations in the Brazilian insect-pollinated tropical tree, Tabebuia roseo-alba (Bignoniaceae)

For many tree species, mating system analyses have indicated potential variations in the selfing rate and paternity correlation among fruits within individuals, among individuals within populations, among populations, and from one flowering event to another. In this study, we used eight microsatellite markers to investigate mating systems at two hierarchical levels (fruits within individuals and individuals within populations) for the insect pollinated Neotropical tree Tabebuia roseo-alba. We found that T. roseo-alba has a mixed mating system with predominantly outcrossed mating. The outcrossing rates at the population level were similar across two T. roseo-alba populations; however, the rates varied considerably among individuals within populations. The correlated paternity results at different hierarchical levels showed that there is a high probability of shared paternal parentage when comparing seeds within fruits and among fruits within plants and full-sibs occur in much higher proportion within fruits than among fruits. Significant levels of fixation index were found in both populations and biparental inbreeding is believed to be the main cause of the observed inbreeding. The number of pollen donors contributing to mating was low. Furthermore, open-pollinated seeds varied according to relatedness, including half-sibs, full-sibs, self-sibs and self- half-sibs. In both populations, the effective population size within a family (seed-tree and its offspring) was lower than expected for panmictic populations. Thus, seeds for ex situ conservation genetics, progeny tests and reforestation must be collected from a large number of seed-trees to guarantee an adequate effective population in the sample.

A method for automatically eliciting node weights in a hierarchical knowledge-based structure for reasoning with uncertainty

Hierarchical knowledge structures are frequently used within clinical decision support systems as part of the model for generating intelligent advice. The nodes in the hierarchy inevitably have varying influence on the decisionmaking processes, which needs to be reflected by parameters. If the model has been elicited from human experts, it is not feasible to ask them to estimate the parameters because there will be so many in even moderately-sized structures. This paper describes how the parameters could be obtained from data instead, using only a small number of cases. The original method [1] is applied to a particular web-based clinical decision support system called GRiST, which uses its hierarchical knowledge to quantify the risks associated with mental-health problems. The knowledge was elicited from multidisciplinary mental-health practitioners but the tree has several thousand nodes, all requiring an estimation of their relative influence on the assessment process. The method described in the paper shows how they can be obtained from about 200 cases instead. It greatly reduces the experts’ elicitation tasks and has the potential for being generalised to similar knowledge-engineering domains where relative weightings of node siblings are part of the parameter space.

Clustering with random indexing K-tree and XML structure

This paper describes the approach taken to the clustering task at INEX 2009 by a group at the Queensland University of Technology. The Random Indexing (RI) K-tree has been used with a representation that is based on the semantic markup available in the INEX 2009 Wikipedia collection. The RI K-tree is a scalable approach to clustering large document collections. This approach has produced quality clustering when evaluated using two different methodologies.

Fabrication of macro-mesoporous zirconia-alumina materials with a 1D hierarchical structure

A series of one dimensional (1D) zirconia/alumina nanocomposites were prepared by the deposition of zirconium species onto the 3D framework of boehmite nanofibres formed by dispersing boehmite nanofibres into butanol solution. The materials were calcined at 773K and characterized by X-ray diffraction (XRD), scanning electron microscopy (SEM), transmission electron microscope (TEM), N2 adsorption/desorption, infrared emission spectroscopy (IES). The results demonstrated that when the molar percentage X=100*Zr/(Al+Zr) was > 30 %, extremely long ZrO2/Al2O3 composite nanorods with evenly distributed ZrO2 nanocrystals on the surface were formed. The stacking of such nanorods gave rise to a new kind of macroporous material without the use of any organic space filler\template or other specific technologies. The mechanism for the formation of long ZrO2/Al2O3 composite nanorods was proposed in this work.

Simplified computation and generalization of the refined process structure tree

A business process is often modeled using some kind of a directed flow graph, which we call a workflow graph. The Refined Process Structure Tree (RPST) is a technique for workflow graph parsing, i.e., for discovering the structure of a workflow graph, which has various applications. In this paper, we provide two improvements to the RPST. First, we propose an alternative way to compute the RPST that is simpler than the one developed originally. In particular, the computation reduces to constructing the tree of the triconnected components of a workflow graph in the special case when every node has at most one incoming or at most one outgoing edge. Such graphs occur frequently in applications. Secondly, we extend the applicability of the RPST. Originally, the RPST was applicable only to graphs with a single source and single sink such that the completed version of the graph is biconnected. We lift both restrictions. Therefore, the RPST is then applicable to arbitrary directed graphs such that every node is on a path from some source to some sink. This includes graphs with multiple sources and/or sinks and disconnected graphs.

BPMN Miner: Automated Discovery of BPMN Process Models with Hierarchical Structure

Existing techniques for automated discovery of process models from event logs gen- erally produce flat process models. Thus, they fail to exploit the notion of subprocess as well as error handling and repetition constructs provided by contemporary process modeling notations, such as the Business Process Model and Notation (BPMN). This paper presents a technique for automated discovery of hierarchical BPMN models con- taining interrupting and non-interrupting boundary events and activity markers. The technique employs functional and inclusion dependency discovery techniques in order to elicit a process-subprocess hierarchy from the event log. Given this hierarchy and the projected logs associated to each node in the hierarchy, parent process and subprocess models are then discovered using existing techniques for flat process model discovery. Finally, the resulting models and logs are heuristically analyzed in order to identify boundary events and markers. By employing approximate dependency discovery tech- niques, it is possible to filter out noise in the event log arising for example from data entry errors or missing events. A validation with one synthetic and two real-life logs shows that process models derived by the proposed technique are more accurate and less complex than those derived with flat process discovery techniques. Meanwhile, a validation on a family of synthetically generated logs shows that the technique is resilient to varying levels of noise.

Landscape structure influences tree density patterns in fragmented woodlands in semi-arid eastern Australia.

Landscape and local-scale influences are important drivers of plant community structure. However, their relative contribution and the degree to which they interact remain unclear. We quantified the extent to which landscape structure, within-patch habitat and their confounding effects determine post-clearing tree densities and composition in agricultural landscapes in eastern subtropical Australia. Landscape structure (incorporating habitat fragmentation and loss) and within-patch (site) features were quantified for 60 remnant patches of Eucalyptus populnea (Myrtaceae) woodland. Tree density and species for three ecological maturity classes (regeneration, early maturity, late maturity) and local site features were assessed in one 100 × 10 m plot per patch. All but one landscape characteristic was determined within a 1.3-km radius of plots; Euclidean nearest neighbour distance was measured inside a 5-km radius. Variation in tree density and composition for each maturity class was partitioned into independent landscape, independent site and joint effects of landscape and site features using redundancy analysis. Independent site effects explained more variation in regeneration density and composition than pure landscape effects; significant predictors were the proportion of early and late maturity trees at a site, rainfall and the associated interaction. Conversely, landscape structure explained greater variation in early and late maturity tree density and composition than site predictors. Area of remnant native vegetation within a landscape and patch characteristics (area, shape, edge contrast) were significant predictors of early maturity tree density. However, 31% of the explained variation in early mature tree differences represented confounding influences of landscape and local variables. We suggest that within-patch characteristics are important in influencing semi-arid woodland tree regeneration. However, independent and confounding effects of landscape structure resulting from previous vegetation clearing may have exerted a greater historical influence on older cohorts and should be accounted for when examining woodland dynamics across a broader range of environments.