A dynamic factor model for mid-term forecasting of wind power generation

The main objective of this paper is the development and application of multivariate time series models for forecasting aggregated wind power production in a country or region. Nowadays, in Spain, Denmark or Germany there is an increasing penetration of this kind of renewable energy, somehow to reduce energy dependence on the exterior, but always linked with the increaseand uncertainty affecting the prices of fossil fuels. The disposal of accurate predictions of wind power generation is a crucial task both for the System Operator as well as for all the agents of the Market. However, the vast majority of works rarely onsider forecasting horizons longer than 48 hours, although they are of interest for the system planning and operation. In this paper we use Dynamic Factor Analysis, adapting and modifying it conveniently, to reach our aim: the computation of accurate forecasts for the aggregated wind power production in a country for a forecasting horizon as long as possible, particularly up to 60 days (2 months). We illustrate this methodology and the results obtained for real data in the leading country in wind power production: Denmark

Energy-entropy-momentum time integration methods for coupled smooth dissipative problems

Esta tesis aborda la formulación, análisis e implementación de métodos numéricos de integración temporal para la solución de sistemas disipativos suaves de dimensión finita o infinita de manera que su estructura continua sea conservada. Se entiende por dichos sistemas aquellos que involucran acoplamiento termo-mecánico y/o efectos disipativos internos modelados por variables internas que siguen leyes continuas, de modo que su evolución es considerada suave. La dinámica de estos sistemas está gobernada por las leyes de la termodinámica y simetrías, las cuales constituyen la estructura que se pretende conservar de forma discreta. Para ello, los sistemas disipativos se describen geométricamente mediante estructuras metriplécticas que identifican claramente las partes reversible e irreversible de la evolución del sistema. Así, usando una de estas estructuras conocida por las siglas (en inglés) de GENERIC, la estructura disipativa de los sistemas es identificada del mismo modo que lo es la Hamiltoniana para sistemas conservativos. Con esto, métodos (EEM) con precisión de segundo orden que conservan la energía, producen entropía y conservan los impulsos lineal y angular son formulados mediante el uso del operador derivada discreta introducido para asegurar la conservación de la Hamiltoniana y las simetrías de sistemas conservativos. Siguiendo estas directrices, se formulan dos tipos de métodos EEM basados en el uso de la temperatura o de la entropía como variable de estado termodinámica, lo que presenta importantes implicaciones que se discuten a lo largo de esta tesis. Entre las cuales cabe destacar que las condiciones de contorno de Dirichlet son naturalmente impuestas con la formulación basada en la temperatura. Por último, se validan dichos métodos y se comprueban sus mejores prestaciones en términos de la estabilidad y robustez en comparación con métodos estándar. This dissertation is concerned with the formulation, analysis and implementation of structure-preserving time integration methods for the solution of the initial(-boundary) value problems describing the dynamics of smooth dissipative systems, either finite- or infinite-dimensional ones. Such systems are understood as those involving thermo-mechanical coupling and/or internal dissipative effects modeled by internal state variables considered to be smooth in the sense that their evolutions follow continuos laws. The dynamics of such systems are ruled by the laws of thermodynamics and symmetries which constitutes the structure meant to be preserved in the numerical setting. For that, dissipative systems are geometrically described by metriplectic structures which clearly identify the reversible and irreversible parts of their dynamical evolution. In particular, the framework known by the acronym GENERIC is used to reveal the systems' dissipative structure in the same way as the Hamiltonian is for conserving systems. Given that, energy-preserving, entropy-producing and momentum-preserving (EEM) second-order accurate methods are formulated using the discrete derivative operator that enabled the formulation of Energy-Momentum methods ensuring the preservation of the Hamiltonian and symmetries for conservative systems. Following these guidelines, two kind of EEM methods are formulated in terms of entropy and temperature as a thermodynamical state variable, involving important implications discussed throughout the dissertation. Remarkably, the formulation in temperature becomes central to accommodate Dirichlet boundary conditions. EEM methods are finally validated and proved to exhibit enhanced numerical stability and robustness properties compared to standard ones.

Dinâmica molecular de sistemas iônicos poliatômicos: modelos polarizável e não-polarizável

Simulações de sais de carbonato fundidos pelo método de Dinâmica Molecular (MD) foram efetuadas com o modelo polarizável de cargas flutuantes (FC). O modelo de cargas flutuantes implementa os efeitos de polarização pelo método de Lagrangiano estendido, onde as variáveis extras são as próprias cargas parciais do íon poliatômico. O modelo FC foi parametrizado por meio de cálculos ab inito, aplicado ao ânion carbonato. Cálculos de Química Quântica ab initio foram utilizados para corroborar o modelo proposto para o ânion carbonato. Os sistemas investigados consistem em misturas de carbonatos alcalinos fundidos, Li2CO3/K2CO3, os quais são utilizados como eletrólitos em células a combustível. As simulações MD foram utilizadas para verificar o efeito da polarização dos ânions sobre a estrutura e dinâmica do líquido. Estudamos o efeito da inclusão de polarização sobre a condutividade do eletrólito.

Multivariate monitoring of a biological wastewater treatment process: a case study at Melbourne Water's Western Treatment Plant

Biological wastewater treatment is a complex, multivariate process, in which a number of physical and biological processes occur simultaneously. In this study, principal component analysis (PCA) and parallel factor analysis (PARAFAC) were used to profile and characterise Lagoon 115E, a multistage biological lagoon treatment system at Melbourne Water's Western Treatment Plant (WTP) in Melbourne, Australia. In this study, the objective was to increase our understanding of the multivariate processes taking place in the lagoon. The data used in the study span a 7-year period during which samples were collected as often as weekly from the ponds of Lagoon 115E and subjected to analysis. The resulting database, involving 19 chemical and physical variables, was studied using the multivariate data analysis methods PCA and PARAFAC. With these methods, alterations in the state of the wastewater due to intrinsic and extrinsic factors could be discerned. The methods were effective in illustrating and visually representing the complex purification stages and cyclic changes occurring along the lagoon system. The two methods proved complementary, with each having its own beneficial features. (C) 2003 Elsevier B.V. All rights reserved.

Analysis of lipoproteins using 2D diffusion-edited NMR spectroscopy and multi-way chemometrics

This study represents the first application of multi-way calibration by N-PLS and multi-way curve resolution by PARAFAC to 2D diffusion-edited H-1 NMR spectra. The aim of the analysis was to evaluate the potential for quantification of lipoprotein main- and subtractions in human plasma samples. Multi-way N-PLS calibrations relating the methyl and methylene peaks of lipoprotein lipids to concentrations of the four main lipoprotein fractions as well as 11 subfractions were developed with high correlations (R = 0.75-0.98). Furthermore, a PARAFAC model with four chemically meaningful components was calculated from the 2D diffusion-edited spectra of the methylene peak of lipids. Although the four extracted PARAFAC components represent molecules of sizes that correspond to the four main fractions of lipoproteins, the corresponding concentrations of the four PARAFAC components proved not to be correlated to the reference concentrations of these four fractions in the plasma samples as determined by ultracentrifugation. These results indicate that NMR provides complementary information on the classification of lipoprotein fractions compared to ultracentrifugation. (C) 2004 Elsevier B.V. All rights reserved.

MBORAYU, O ESPÍRITO QUE NOS UNE: UM CONCEITO DA ESPIRITUALIDADE GUARANI

Neste texto trato de um conceito da espiritualidade do povo Guarani que é o Mborayu , a força-espírito: que integra e desintegra os elementos que compõem o Ñande Reko (a maneira de ser Guarani); que aglutina ou dispersa os elementos e os corpos (rete kwere) que compõem o indivíduo; que catalisa ou dilui Ñamandu (a natureza de todos os mundos). O Mborayu também abarca o Aywu (a palavra) que nomina e organiza uma compreensão do Ñeem (termo-idéia), do espírito das coisas, que não é a realidade (ete), porque a última realidade (opa wa erã) pertence a Ñemi Guaxu (o grande mistério). Nesse tom, percorro neste estudo, todo um universo da mítica e da espiritualidade Guarani, porque o Mborayu reúne todos os elementos que compõem Ñamandu, em uma totalidade inacabada e, por sua força de atração a vida (ikowe) é gerada, encontrando, no entanto, sua última expressão, na morte (mano), na desintegração, seu derradeiro sentido.(AU)

Enhancing visibility in EPCIS governing agri-food supply chains via linked pedigrees

Data integration for the purposes of tracking, tracing and transparency are important challenges in the agri-food supply chain. The Electronic Product Code Information Services (EPCIS) is an event-oriented GS1 standard that aims to enable tracking and tracing of products through the sharing of event-based datasets that encapsulate the Electronic Product Code (EPC). In this paper, the authors propose a framework that utilises events and EPCs in the generation of "linked pedigrees" - linked datasets that enable the sharing of traceability information about products as they move along the supply chain. The authors exploit two ontology based information models, EEM and CBVVocab within a distributed and decentralised framework that consumes real time EPCIS events as linked data to generate the linked pedigrees. The authors exemplify the usage of linked pedigrees within the fresh fruit and vegetables supply chain in the agri-food sector.

Detecting EPCIS exceptions in linked traceability streams across supply chain business processes

The EPCIS specification provides an event oriented mechanism to record product movement information across stakeholders in supply chain business processes. Besides enabling the sharing of event-based traceability datasets, track and trace implementations must also be equipped with the capabilities to validate integrity constraints and detect runtime exceptions without compromising the time-to-deliver schedule of the shipping and receiving parties. In this paper we present a methodology for detecting exceptions arising during the processing of EPCIS event datasets. We propose an extension to the EEM ontology for modelling EPCIS exceptions and show how runtime exceptions can be detected and reported. We exemplify and evaluate our approach on an abstraction of pharmaceutical supply chains.

Modelling and linking transformations in EPCIS governing supply chain business processes

Supply chains comprise of complex processes spanning across multiple trading partners. The various operations involved generate large number of events that need to be integrated in order to enable internal and external traceability. Further, provenance of artifacts and agents involved in the supply chain operations is now a key traceability requirement. In this paper we propose a Semantic web/Linked data powered framework for the event based representation and analysis of supply chain activities governed by the EPCIS specification. We specifically show how a new EPCIS event type called "Transformation Event" can be semantically annotated using EEM - The EPCIS Event Model to generate linked data, that can be exploited for internal event based traceability in supply chains involving transformation of products. For integrating provenance with traceability, we propose a mapping from EEM to PROV-O. We exemplify our approach on an abstraction of the production processes that are part of the wine supply chain.

Tíz év az Európai Unióban - az új tagországok agrárteljesítményei = A decade in the EU: the agricultural performances of the new member-states

Több mint tíz év telt el az Európai Unió 2004. évi kibővülése óta. A tízéves évforduló jó lehetőséget kínált a mérlegkészítésre, annak vizsgálatára, hogy a legfrissebb elérhető adatok tükrében milyen fejlődési pályát tudhatnak maguk mögött az új tagországok mezőgazdasági szektorai. Írásunk célja a tíz kelet- európai EU-tagállam agrárteljesítményének értékelése, illetve ez alapján a csatlakozás nyerteseinek, illetve veszteseinek azonosítása. A rendelkezésre álló adatokat a többdimenziós faktoranalízis módszerével feldolgozva arra az eredményre jutottunk, hogy Lengyelország, Észtország és Litvánia hármasa tekinthető az agrárcsatlakozás abszolút nyertesének, míg a többi új tagállam nem volt képes teljes mértékben kihasználni a csatlakozás adta lehetőségeket. Az eredményekből az is látható, hogy a magas hozzáadott értékű termékekre való szakosodás jó stratégiának bizonyult, mert gyorsabb fejlődést biztosított, mint a mezőgazdasági alaptermékekre való koncentrálás. ____ The period of over ten years since the 2004 round of EU accessions provides a good opportunity to assess and take stock of agricultural developments in the new member- States, in light of the latest available data. The paper sets out to assess their agricultural performances and to identify the winners and losers by accession in this regard. Ranking individual country performances using Parallel Factor Analysis (PARAFAC) suggests that Poland, Estonia and Lithuania can be considered as winners, whereas the other countries failed to use the potentials of membership to the full. The results also suggest that focusing on agri-food products with a high added value proved a good development strategy for the sector. Countries that concentrated on producing agri-food raw materials lagged behind.

Large scale data processing using MapReduce

As massive data sets become increasingly available, people are facing the problem of how to effectively process and understand these data. Traditional sequential computing models are giving way to parallel and distributed computing models, such as MapReduce, both due to the large size of the data sets and their high dimensionality. This dissertation, as in the same direction of other researches that are based on MapReduce, tries to develop effective techniques and applications using MapReduce that can help people solve large-scale problems. Three different problems are tackled in the dissertation. The first one deals with processing terabytes of raster data in a spatial data management system. Aerial imagery files are broken into tiles to enable data parallel computation. The second and third problems deal with dimension reduction techniques that can be used to handle data sets of high dimensionality. Three variants of the nonnegative matrix factorization technique are scaled up to factorize matrices of dimensions in the order of millions in MapReduce based on different matrix multiplication implementations. Two algorithms, which compute CANDECOMP/PARAFAC and Tucker tensor decompositions respectively, are parallelized in MapReduce based on carefully partitioning the data and arranging the computation to maximize data locality and parallelism.

Bayesian Inference in Large-scale Problems

Many modern applications fall into the category of "large-scale" statistical problems, in which both the number of observations n and the number of features or parameters p may be large. Many existing methods focus on point estimation, despite the continued relevance of uncertainty quantification in the sciences, where the number of parameters to estimate often exceeds the sample size, despite huge increases in the value of n typically seen in many fields. Thus, the tendency in some areas of industry to dispense with traditional statistical analysis on the basis that "n=all" is of little relevance outside of certain narrow applications. The main result of the Big Data revolution in most fields has instead been to make computation much harder without reducing the importance of uncertainty quantification. Bayesian methods excel at uncertainty quantification, but often scale poorly relative to alternatives. This conflict between the statistical advantages of Bayesian procedures and their substantial computational disadvantages is perhaps the greatest challenge facing modern Bayesian statistics, and is the primary motivation for the work presented here.

Two general strategies for scaling Bayesian inference are considered. The first is the development of methods that lend themselves to faster computation, and the second is design and characterization of computational algorithms that scale better in n or p. In the first instance, the focus is on joint inference outside of the standard problem of multivariate continuous data that has been a major focus of previous theoretical work in this area. In the second area, we pursue strategies for improving the speed of Markov chain Monte Carlo algorithms, and characterizing their performance in large-scale settings. Throughout, the focus is on rigorous theoretical evaluation combined with empirical demonstrations of performance and concordance with the theory.

One topic we consider is modeling the joint distribution of multivariate categorical data, often summarized in a contingency table. Contingency table analysis routinely relies on log-linear models, with latent structure analysis providing a common alternative. Latent structure models lead to a reduced rank tensor factorization of the probability mass function for multivariate categorical data, while log-linear models achieve dimensionality reduction through sparsity. Little is known about the relationship between these notions of dimensionality reduction in the two paradigms. In Chapter 2, we derive several results relating the support of a log-linear model to nonnegative ranks of the associated probability tensor. Motivated by these findings, we propose a new collapsed Tucker class of tensor decompositions, which bridge existing PARAFAC and Tucker decompositions, providing a more flexible framework for parsimoniously characterizing multivariate categorical data. Taking a Bayesian approach to inference, we illustrate empirical advantages of the new decompositions.

Latent class models for the joint distribution of multivariate categorical, such as the PARAFAC decomposition, data play an important role in the analysis of population structure. In this context, the number of latent classes is interpreted as the number of genetically distinct subpopulations of an organism, an important factor in the analysis of evolutionary processes and conservation status. Existing methods focus on point estimates of the number of subpopulations, and lack robust uncertainty quantification. Moreover, whether the number of latent classes in these models is even an identified parameter is an open question. In Chapter 3, we show that when the model is properly specified, the correct number of subpopulations can be recovered almost surely. We then propose an alternative method for estimating the number of latent subpopulations that provides good quantification of uncertainty, and provide a simple procedure for verifying that the proposed method is consistent for the number of subpopulations. The performance of the model in estimating the number of subpopulations and other common population structure inference problems is assessed in simulations and a real data application.

In contingency table analysis, sparse data is frequently encountered for even modest numbers of variables, resulting in non-existence of maximum likelihood estimates. A common solution is to obtain regularized estimates of the parameters of a log-linear model. Bayesian methods provide a coherent approach to regularization, but are often computationally intensive. Conjugate priors ease computational demands, but the conjugate Diaconis--Ylvisaker priors for the parameters of log-linear models do not give rise to closed form credible regions, complicating posterior inference. In Chapter 4 we derive the optimal Gaussian approximation to the posterior for log-linear models with Diaconis--Ylvisaker priors, and provide convergence rate and finite-sample bounds for the Kullback-Leibler divergence between the exact posterior and the optimal Gaussian approximation. We demonstrate empirically in simulations and a real data application that the approximation is highly accurate, even in relatively small samples. The proposed approximation provides a computationally scalable and principled approach to regularized estimation and approximate Bayesian inference for log-linear models.

Another challenging and somewhat non-standard joint modeling problem is inference on tail dependence in stochastic processes. In applications where extreme dependence is of interest, data are almost always time-indexed. Existing methods for inference and modeling in this setting often cluster extreme events or choose window sizes with the goal of preserving temporal information. In Chapter 5, we propose an alternative paradigm for inference on tail dependence in stochastic processes with arbitrary temporal dependence structure in the extremes, based on the idea that the information on strength of tail dependence and the temporal structure in this dependence are both encoded in waiting times between exceedances of high thresholds. We construct a class of time-indexed stochastic processes with tail dependence obtained by endowing the support points in de Haan's spectral representation of max-stable processes with velocities and lifetimes. We extend Smith's model to these max-stable velocity processes and obtain the distribution of waiting times between extreme events at multiple locations. Motivated by this result, a new definition of tail dependence is proposed that is a function of the distribution of waiting times between threshold exceedances, and an inferential framework is constructed for estimating the strength of extremal dependence and quantifying uncertainty in this paradigm. The method is applied to climatological, financial, and electrophysiology data.

The remainder of this thesis focuses on posterior computation by Markov chain Monte Carlo. The Markov Chain Monte Carlo method is the dominant paradigm for posterior computation in Bayesian analysis. It has long been common to control computation time by making approximations to the Markov transition kernel. Comparatively little attention has been paid to convergence and estimation error in these approximating Markov Chains. In Chapter 6, we propose a framework for assessing when to use approximations in MCMC algorithms, and how much error in the transition kernel should be tolerated to obtain optimal estimation performance with respect to a specified loss function and computational budget. The results require only ergodicity of the exact kernel and control of the kernel approximation accuracy. The theoretical framework is applied to approximations based on random subsets of data, low-rank approximations of Gaussian processes, and a novel approximating Markov chain for discrete mixture models.

Data augmentation Gibbs samplers are arguably the most popular class of algorithm for approximately sampling from the posterior distribution for the parameters of generalized linear models. The truncated Normal and Polya-Gamma data augmentation samplers are standard examples for probit and logit links, respectively. Motivated by an important problem in quantitative advertising, in Chapter 7 we consider the application of these algorithms to modeling rare events. We show that when the sample size is large but the observed number of successes is small, these data augmentation samplers mix very slowly, with a spectral gap that converges to zero at a rate at least proportional to the reciprocal of the square root of the sample size up to a log factor. In simulation studies, moderate sample sizes result in high autocorrelations and small effective sample sizes. Similar empirical results are observed for related data augmentation samplers for multinomial logit and probit models. When applied to a real quantitative advertising dataset, the data augmentation samplers mix very poorly. Conversely, Hamiltonian Monte Carlo and a type of independence chain Metropolis algorithm show good mixing on the same dataset.