Prediction and improved estimation in linear models /

Includes index.

Exponentiated gradient algorithms for conditional random fields and max-margin Markov Networks

Log-linear and maximum-margin models are two commonly-used methods in supervised machine learning, and are frequently used in structured prediction problems. Efficient learning of parameters in these models is therefore an important problem, and becomes a key factor when learning from very large data sets. This paper describes exponentiated gradient (EG) algorithms for training such models, where EG updates are applied to the convex dual of either the log-linear or max-margin objective function; the dual in both the log-linear and max-margin cases corresponds to minimizing a convex function with simplex constraints. We study both batch and online variants of the algorithm, and provide rates of convergence for both cases. In the max-margin case, O(1/ε) EG updates are required to reach a given accuracy ε in the dual; in contrast, for log-linear models only O(log(1/ε)) updates are required. For both the max-margin and log-linear cases, our bounds suggest that the online EG algorithm requires a factor of n less computation to reach a desired accuracy than the batch EG algorithm, where n is the number of training examples. Our experiments confirm that the online algorithms are much faster than the batch algorithms in practice. We describe how the EG updates factor in a convenient way for structured prediction problems, allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing. We perform extensive evaluation of the algorithms, comparing them to L-BFGS and stochastic gradient descent for log-linear models, and to SVM-Struct for max-margin models. The algorithms are applied to a multi-class problem as well as to a more complex large-scale parsing task. In all these settings, the EG algorithms presented here outperform the other methods.

Casamento e estratificação social : um estudo sobre seletividade marital por escolaridade e origem social no Brasil

Buscou-se, nesse estudo, quantificar e avaliar a homogamia, a heterogamia e as barreiras de cruzamento ao matrimônio via escolaridade (anos de estudo) e origem social (categorias ocupacionais dos pais). As tendências temporais desses padrões também foram examinadas. Analisou-se, ainda, a associação entre escolaridade dos maridos, escolaridade das esposas (status realizado), origem social dos maridos e origem social das esposas (status atribuído). Esse trabalho teve o intuito também de discutir o viés de seletividade marital segundo os diferenciais sociais (anos de estudo e origem social). Para isso, foram analisados parâmetros que mostram como se configuram os padrões de nupcialidade (idade média ao casar e celibato definitivo), bem como foram examinados os determinantes da união sob a perspectiva de três níveis de fatores condicionantes (nível das características individuais, nível do status atribuído e nível do status realizado). Verificou-se que as mulheres com alta escolaridade, no Brasil, permanecem num período maior na condição de solteiras (alta idade média ao casar e alto celibato definitivo). Os homens com alta escolaridade também apresentaram uma alta idade média ao casar, entretanto, o casamento demonstrou ser praticamente universal para esse segmento. Os resultados também mostraram que o aumento de um ano na idade dos indivíduos elevam a chance de união em aproximadamente 5%. Ter uma baixa escolaridade também aumenta a chance dos indivíduos se casarem. A variável origem social apresentou um comportamento dúbio ao ser incorporada no modelo com a variável anos de estudo. Constatou-se que há uma alta proporção de uniões homogâmicas por escolaridade. Para efetuar uma análise adequada das tendências temporais na seletividade marital foi proposto modelos log-lineares em que a dimensão do tempo foi incorporada. O ajustamento dos modelos indicou que a interpretação mais plausível para as tendências temporais na seletividade marital por escolaridade é a da estabilidade dos parâmetros indicativos das propensões homogâmicas. Em relação a análise da seletividade marital e origem social os resultados mostraram que a maior proporção de homogamia pôde ser verificada entre os casais que tinham como origem social a categoria de pequenos proprietários rurais. A conclusão mais plausível ao se analisar os modelos que consideraram as tendências temporais é que a variação temporal dos parâmetros indicativos da seletividade marital por origem social é a característica mais forte dos dados analisados. Ao analisar as chances relativas oriundas desse modelo observou-se que as barreiras de origem social de curta distância (entre segmentos de origem social próximos) são as mais fáceis de serem transpostas. Ao passo que as barreiras mais difíceis de serem ultrapassadas estão concentradas nos dois extremos. Verificou-se, ainda, que as associações entre as interações escolaridade do marido e escolaridade da esposa e origem social do marido e origem social da esposa não são independentes. Assim, pode-se presumir que a origem social (status atribuído) continua influenciando a escolha conjugal mesmo quando se leva em consideração o status realizado (escolaridade dos cônjuges)

Compositional analysis of bivariate discrete probabilities

A joint distribution of two discrete random variables with finite support can be displayed as a two way table of probabilities adding to one. Assume that this table has n rows and m columns and all probabilities are non-null. This kind of table can be seen as an element in the simplex of n · m parts. In this context, the marginals are identified as compositional amalgams, conditionals (rows or columns) as subcompositions. Also, simplicial perturbation appears as Bayes theorem. However, the Euclidean elements of the Aitchison geometry of the simplex can also be translated into the table of probabilities: subspaces, orthogonal projections, distances. Two important questions are addressed: a) given a table of probabilities, which is the nearest independent table to the initial one? b) which is the largest orthogonal projection of a row onto a column? or, equivalently, which is the information in a row explained by a column, thus explaining the interaction? To answer these questions three orthogonal decompositions are presented: (1) by columns and a row-wise geometric marginal, (2) by rows and a columnwise geometric marginal, (3) by independent two-way tables and fully dependent tables representing row-column interaction. An important result is that the nearest independent table is the product of the two (row and column)-wise geometric marginal tables. A corollary is that, in an independent table, the geometric marginals conform with the traditional (arithmetic) marginals. These decompositions can be compared with standard log-linear models. Key words: balance, compositional data, simplex, Aitchison geometry, composition, orthonormal basis, arithmetic and geometric marginals, amalgam, dependence measure, contingency table

Missing data mechanisms and their implications on the analysis of categorical data

We review some issues related to the implications of different missing data mechanisms on statistical inference for contingency tables and consider simulation studies to compare the results obtained under such models to those where the units with missing data are disregarded. We confirm that although, in general, analyses under the correct missing at random and missing completely at random models are more efficient even for small sample sizes, there are exceptions where they may not improve the results obtained by ignoring the partially classified data. We show that under the missing not at random (MNAR) model, estimates on the boundary of the parameter space as well as lack of identifiability of the parameters of saturated models may be associated with undesirable asymptotic properties of maximum likelihood estimators and likelihood ratio tests; even in standard cases the bias of the estimators may be low only for very large samples. We also show that the probability of a boundary solution obtained under the correct MNAR model may be large even for large samples and that, consequently, we may not always conclude that a MNAR model is misspecified because the estimate is on the boundary of the parameter space.

Global pharmacogenomics: Impact of population diversity on the distribution of polymorphisms in the CYP2C cluster among Brazilians

The impact of biogeographical ancestry, self-reported 'race/color' and geographical origin on the frequency distribution of 10 CYP2C functional polymorphisms (CYP2C8*2, *3, *4, CYP2C9*2, *3, *5, *11, CYP2C19*2, *3 and *17) and their haplotypes was assessed in a representative cohort of the Brazilian population (n = 1034). TaqMan assays were used for allele discrimination at each CYP2C locus investigated. Individual proportions of European, African and Amerindian biogeographical ancestry were estimated using a panel of insertion-deletion polymorphisms. Multinomial log-linear models were applied to infer the statistical association between the CYP2C alleles and haplotypes (response variables), and biogeographical ancestry, self-reported Color and geographical origin (explanatory variables). The results showed that CYP2C19*3, CYP2C9*5 and CYP2C9*11 were rare alleles (<1%), the frequency of other variants ranged from 3.4% (CYP2C8*4) to 17.3% (CYP2C19*17). Two distinct haplotype blocks were identified: block 1 consists of three single nucleotide polymorphisms (SNPs) (CYP2C19*17, CYP2C19*2 and CYP2C9*2) and block 2 of six SNPs (CYP2C9*11, CYP2C9*3, CYP2C9*5, CYP2C8*2, CYP2C8*4 and CYP2C8*3). Diplotype analysis generated 41 haplotypes, of which eight had frequencies greater than 1% and together accounted for 96.4% of the overall genetic diversity. The distribution of CYP2C8 and CYP2C9 (but not CYP2C19) alleles, and of CYP2C haplotypes was significantly associated with self-reported Color and with the individual proportions of European and African genetic ancestry, irrespective of Color self-identification. The individual odds of having alleles CYP2C8*2, CYP2C8*3, CYP2C9*2 and CYP2C9*3, and haplotypes including these alleles, varied continuously as the proportion of European ancestry increased. Collectively, these data strongly suggest that the intrinsic heterogeneity of the Brazilian population must be acknowledged in the design and interpretation of pharmacogenomic studies of the CYP2C cluster in order to avoid spurious conclusions based on improper matching of study cohorts. This conclusion extends to other polymorphic pharmacogenes among Brazilians, and most likely to other admixed populations of the Americas. The Pharmacogenomics Journal (2012) 12, 267-276; doi: 10.1038/tpj.2010.89; published online 21 December 2010

Robust statistical modeling using the Birnbaum-Saunders-t distribution applied to insurance

In this paper, we carry out robust modeling and influence diagnostics in Birnbaum-Saunders (BS) regression models. Specifically, we present some aspects related to BS and log-BS distributions and their generalizations from the Student-t distribution, and develop BS-t regression models, including maximum likelihood estimation based on the EM algorithm and diagnostic tools. In addition, we apply the obtained results to real data from insurance, which shows the uses of the proposed model. Copyright (c) 2011 John Wiley & Sons, Ltd.

Differenze nella prestazione tecnico-tattica tra diverse categorie di atleti nel tennistavolo di alto livello

Nell’ambito della ricerca scientifica nel campo dello sport, la Performance Analysis si sta ritagliando un crescente spazio di interesse. Per Performance Analysis si intende l’analisi della prestazione agonistica sia dal punto di vista biomeccanico che dal punto di vista dell’analisi notazionale. In questa tesi è stata analizzata la prestazione agonistica nel tennistavolo attraverso lo strumento dell’analisi notazionale, partendo dallo studio degli indicatori di prestazione più importanti dal punto di vista tecnico-tattico e dalla loro selezione attraverso uno studio sull’attendibilità nella raccolta dati. L’attenzione è stata posta quindi su un aspetto tecnico originale, il collegamento spostamenti e colpi, ricordando che una buona tecnica di spostamento permette di muoversi rapidamente nella direzione della pallina per effettuare il colpo migliore. Infine, l’obbiettivo principale della tesi è stato quello di confrontare le tre categorie di atleti selezionate: alto livello mondiale maschile (M), alto livello junior europeo (J) ed alto livello mondiale femminile (F). La maggior parte delle azioni cominciano con un servizio corto al centro del tavolo, proseguono con una risposta in push (M) o in flik di rovescio (J). Il colpo che segue è principalmente il top spin di dritto dopo un passo pivot o un top di rovescio senza spostamento. Gli alteti M e J contrattaccano maggiormente con top c. top di dritto e le atlete F prediligono colpi meno spregiudicati, bloccando di rovescio e proseguendo con drive di rovescio. Attraverso lo studio della prestazione di atleti di categorie e generi diversi è possibile migliorare le scelte strategiche prima e durante gli incontri. Le analisi statistiche multivariate (modelli log-lineari) hanno permesso di validare con metodo scientifico sia le procedure già utilizzate in letteratura che quelle innovative messe a punto per la prima volta in occasione di questo studio.

DECOMPOSITION OF REGRESSION ESTIMATORS TO EXPLORE THE INFLUENCE OF "UNMEASURED" TIME-VARYING CONFOUNDERS

In environmental epidemiology, exposure X and health outcome Y vary in space and time. We present a method to diagnose the possible influence of unmeasured confounders U on the estimated effect of X on Y and to propose several approaches to robust estimation. The idea is to use space and time as proxy measures for the unmeasured factors U. We start with the time series case where X and Y are continuous variables at equally-spaced times and assume a linear model. We define matching estimator b(u)s that correspond to pairs of observations with specific lag u. Controlling for a smooth function of time, St, using a kernel estimator is roughly equivalent to estimating the association with a linear combination of the b(u)s with weights that involve two components: the assumptions about the smoothness of St and the normalized variogram of the X process. When an unmeasured confounder U exists, but the model otherwise correctly controls for measured confounders, the excess variation in b(u)s is evidence of confounding by U. We use the plot of b(u)s versus lag u, lagged-estimator-plot (LEP), to diagnose the influence of U on the effect of X on Y. We use appropriate linear combination of b(u)s or extrapolate to b(0) to obtain novel estimators that are more robust to the influence of smooth U. The methods are extended to time series log-linear models and to spatial analyses. The LEP plot gives us a direct view of the magnitude of the estimators for each lag u and provides evidence when models did not adequately describe the data.

COVARIATE-ADJUSTED NONPARAMETRIC ANALYSIS OF MAGNETIC RESONANCE IMAGES USING MARKOV CHAIN MONTE CARLO

Permutation tests are useful for drawing inferences from imaging data because of their flexibility and ability to capture features of the brain that are difficult to capture parametrically. However, most implementations of permutation tests ignore important confounding covariates. To employ covariate control in a nonparametric setting we have developed a Markov chain Monte Carlo (MCMC) algorithm for conditional permutation testing using propensity scores. We present the first use of this methodology for imaging data. Our MCMC algorithm is an extension of algorithms developed to approximate exact conditional probabilities in contingency tables, logit, and log-linear models. An application of our non-parametric method to remove potential bias due to the observed covariates is presented.

Juvenile–adult tree associations in a continental Mediterranean ecosystem: no evidence for sustained and general facilitation at increased aridity

Question: How do tree species identity, microhabitat and water availability affect inter- and intra-specific interactions between juvenile and adult woody plants? Location: Continental Mediterranean forests in Alto Tajo Natural Park, Guadalajara, Spain. Methods: A total of 2066 juveniles and adults of four co-occurring tree species were mapped in 17 plots. The frequency of juveniles at different microhabitats and water availability levels was analysed using log-linear models. We used nearest-neighbour contingency table analysis of spatial segregation and J-functions to describe the spatial patterns. Results: We found a complex spatial pattern that varied according to species identity and microhabitat. Recruitment was more frequent in gaps for Quercus ilex, while the other three species recruited preferentially under shrubs or trees depending on the water availability level. Juveniles were not spatially associated to conspecific adults, experiencing segregation from them inmany cases. Spatial associations, both positive and negative, were more common at higher water availability levels. Conclusions: Our results do not agree with expectations from the stressgradient hypothesis, suggesting that positive interactions do not increase in importance with increasing aridity in the study ecosystem. Regeneration patterns are species-specific and depend on microhabitat characteristics and dispersal strategies. In general, juveniles do not look for conspecific adult protection. This work contributes to the understanding of species co-existence, proving the importance of considering a multispecies approach at several plots to overcome limitations of simple pair-wise comparisons in a limited number of sites.

Frequency and Intensity of drought events over Ebro River basin.

Lately, several researchers have pointed out that climate change is expected to increase temperatures and lower rainfall in Mediterranean regions, simultaneously increasing the intensity of extreme rainfall events. These changes could have consequences regarding rainfall regime, erosion, sediment transport and water quality, soil management, and new designs in diversion ditches. Climate change is expected to result in increasingly unpredictable and variable rainfall, in amount and timing, changing seasonal patterns and increasing the frequency of extreme weather events. Consequently, the evolution of frequency and intensity of drought periods is of most important as in agro-ecosystems many processes will be affected by them. Realising the complex and important consequences of an increasing frequency of extreme droughts at the Ebro River basin, our aim is to study the evolution of drought events at this site statistically, with emphasis on the occurrence and intensity of them. For this purpose, fourteen meteorological stations were selected based on the length of the rainfall series and the climatic classification to obtain a representative untreated dataset from the river basin. Daily rainfall series from 1957 to 2002 were obtained from each meteorological station and no-rain period frequency as the consecutive numbers of days were extracted. Based on this data, we study changes in the probability distribution in several sub-periods. Moreover we used the Standardized Precipitation Index (SPI) for identification of drought events in a year scale and then we use this index to fit log-linear models to the contingency tables between the SPI index and the sub-periods, this adjusted is carried out with the help of ANOVA inference.

Bayesian Inference in Large-scale Problems

Many modern applications fall into the category of "large-scale" statistical problems, in which both the number of observations n and the number of features or parameters p may be large. Many existing methods focus on point estimation, despite the continued relevance of uncertainty quantification in the sciences, where the number of parameters to estimate often exceeds the sample size, despite huge increases in the value of n typically seen in many fields. Thus, the tendency in some areas of industry to dispense with traditional statistical analysis on the basis that "n=all" is of little relevance outside of certain narrow applications. The main result of the Big Data revolution in most fields has instead been to make computation much harder without reducing the importance of uncertainty quantification. Bayesian methods excel at uncertainty quantification, but often scale poorly relative to alternatives. This conflict between the statistical advantages of Bayesian procedures and their substantial computational disadvantages is perhaps the greatest challenge facing modern Bayesian statistics, and is the primary motivation for the work presented here.

Two general strategies for scaling Bayesian inference are considered. The first is the development of methods that lend themselves to faster computation, and the second is design and characterization of computational algorithms that scale better in n or p. In the first instance, the focus is on joint inference outside of the standard problem of multivariate continuous data that has been a major focus of previous theoretical work in this area. In the second area, we pursue strategies for improving the speed of Markov chain Monte Carlo algorithms, and characterizing their performance in large-scale settings. Throughout, the focus is on rigorous theoretical evaluation combined with empirical demonstrations of performance and concordance with the theory.

One topic we consider is modeling the joint distribution of multivariate categorical data, often summarized in a contingency table. Contingency table analysis routinely relies on log-linear models, with latent structure analysis providing a common alternative. Latent structure models lead to a reduced rank tensor factorization of the probability mass function for multivariate categorical data, while log-linear models achieve dimensionality reduction through sparsity. Little is known about the relationship between these notions of dimensionality reduction in the two paradigms. In Chapter 2, we derive several results relating the support of a log-linear model to nonnegative ranks of the associated probability tensor. Motivated by these findings, we propose a new collapsed Tucker class of tensor decompositions, which bridge existing PARAFAC and Tucker decompositions, providing a more flexible framework for parsimoniously characterizing multivariate categorical data. Taking a Bayesian approach to inference, we illustrate empirical advantages of the new decompositions.

Latent class models for the joint distribution of multivariate categorical, such as the PARAFAC decomposition, data play an important role in the analysis of population structure. In this context, the number of latent classes is interpreted as the number of genetically distinct subpopulations of an organism, an important factor in the analysis of evolutionary processes and conservation status. Existing methods focus on point estimates of the number of subpopulations, and lack robust uncertainty quantification. Moreover, whether the number of latent classes in these models is even an identified parameter is an open question. In Chapter 3, we show that when the model is properly specified, the correct number of subpopulations can be recovered almost surely. We then propose an alternative method for estimating the number of latent subpopulations that provides good quantification of uncertainty, and provide a simple procedure for verifying that the proposed method is consistent for the number of subpopulations. The performance of the model in estimating the number of subpopulations and other common population structure inference problems is assessed in simulations and a real data application.

In contingency table analysis, sparse data is frequently encountered for even modest numbers of variables, resulting in non-existence of maximum likelihood estimates. A common solution is to obtain regularized estimates of the parameters of a log-linear model. Bayesian methods provide a coherent approach to regularization, but are often computationally intensive. Conjugate priors ease computational demands, but the conjugate Diaconis--Ylvisaker priors for the parameters of log-linear models do not give rise to closed form credible regions, complicating posterior inference. In Chapter 4 we derive the optimal Gaussian approximation to the posterior for log-linear models with Diaconis--Ylvisaker priors, and provide convergence rate and finite-sample bounds for the Kullback-Leibler divergence between the exact posterior and the optimal Gaussian approximation. We demonstrate empirically in simulations and a real data application that the approximation is highly accurate, even in relatively small samples. The proposed approximation provides a computationally scalable and principled approach to regularized estimation and approximate Bayesian inference for log-linear models.

Another challenging and somewhat non-standard joint modeling problem is inference on tail dependence in stochastic processes. In applications where extreme dependence is of interest, data are almost always time-indexed. Existing methods for inference and modeling in this setting often cluster extreme events or choose window sizes with the goal of preserving temporal information. In Chapter 5, we propose an alternative paradigm for inference on tail dependence in stochastic processes with arbitrary temporal dependence structure in the extremes, based on the idea that the information on strength of tail dependence and the temporal structure in this dependence are both encoded in waiting times between exceedances of high thresholds. We construct a class of time-indexed stochastic processes with tail dependence obtained by endowing the support points in de Haan's spectral representation of max-stable processes with velocities and lifetimes. We extend Smith's model to these max-stable velocity processes and obtain the distribution of waiting times between extreme events at multiple locations. Motivated by this result, a new definition of tail dependence is proposed that is a function of the distribution of waiting times between threshold exceedances, and an inferential framework is constructed for estimating the strength of extremal dependence and quantifying uncertainty in this paradigm. The method is applied to climatological, financial, and electrophysiology data.

The remainder of this thesis focuses on posterior computation by Markov chain Monte Carlo. The Markov Chain Monte Carlo method is the dominant paradigm for posterior computation in Bayesian analysis. It has long been common to control computation time by making approximations to the Markov transition kernel. Comparatively little attention has been paid to convergence and estimation error in these approximating Markov Chains. In Chapter 6, we propose a framework for assessing when to use approximations in MCMC algorithms, and how much error in the transition kernel should be tolerated to obtain optimal estimation performance with respect to a specified loss function and computational budget. The results require only ergodicity of the exact kernel and control of the kernel approximation accuracy. The theoretical framework is applied to approximations based on random subsets of data, low-rank approximations of Gaussian processes, and a novel approximating Markov chain for discrete mixture models.

Data augmentation Gibbs samplers are arguably the most popular class of algorithm for approximately sampling from the posterior distribution for the parameters of generalized linear models. The truncated Normal and Polya-Gamma data augmentation samplers are standard examples for probit and logit links, respectively. Motivated by an important problem in quantitative advertising, in Chapter 7 we consider the application of these algorithms to modeling rare events. We show that when the sample size is large but the observed number of successes is small, these data augmentation samplers mix very slowly, with a spectral gap that converges to zero at a rate at least proportional to the reciprocal of the square root of the sample size up to a log factor. In simulation studies, moderate sample sizes result in high autocorrelations and small effective sample sizes. Similar empirical results are observed for related data augmentation samplers for multinomial logit and probit models. When applied to a real quantitative advertising dataset, the data augmentation samplers mix very poorly. Conversely, Hamiltonian Monte Carlo and a type of independence chain Metropolis algorithm show good mixing on the same dataset.

Quantification of air quality in space and time and its effects on health

The long-term adverse effects on health associated with air pollution exposure can be estimated using either cohort or spatio-temporal ecological designs. In a cohort study, the health status of a cohort of people are assessed periodically over a number of years, and then related to estimated ambient pollution concentrations in the cities in which they live. However, such cohort studies are expensive and time consuming to implement, due to the long-term follow up required for the cohort. Therefore, spatio-temporal ecological studies are also being used to estimate the long-term health effects of air pollution as they are easy to implement due to the routine availability of the required data. Spatio-temporal ecological studies estimate the health impact of air pollution by utilising geographical and temporal contrasts in air pollution and disease risk across $n$ contiguous small-areas, such as census tracts or electoral wards, for multiple time periods. The disease data are counts of the numbers of disease cases occurring in each areal unit and time period, and thus Poisson log-linear models are typically used for the analysis. The linear predictor includes pollutant concentrations and known confounders such as socio-economic deprivation. However, as the disease data typically contain residual spatial or spatio-temporal autocorrelation after the covariate effects have been accounted for, these known covariates are augmented by a set of random effects. One key problem in these studies is estimating spatially representative pollution concentrations in each areal which are typically estimated by applying Kriging to data from a sparse monitoring network, or by computing averages over modelled concentrations (grid level) from an atmospheric dispersion model. The aim of this thesis is to investigate the health effects of long-term exposure to Nitrogen Dioxide (NO2) and Particular matter (PM10) in mainland Scotland, UK. In order to have an initial impression about the air pollution health effects in mainland Scotland, chapter 3 presents a standard epidemiological study using a benchmark method. The remaining main chapters (4, 5, 6) cover the main methodological focus in this thesis which has been threefold: (i) how to better estimate pollution by developing a multivariate spatio-temporal fusion model that relates monitored and modelled pollution data over space, time and pollutant; (ii) how to simultaneously estimate the joint effects of multiple pollutants; and (iii) how to allow for the uncertainty in the estimated pollution concentrations when estimating their health effects. Specifically, chapters 4 and 5 are developed to achieve (i), while chapter 6 focuses on (ii) and (iii). In chapter 4, I propose an integrated model for estimating the long-term health effects of NO2, that fuses modelled and measured pollution data to provide improved predictions of areal level pollution concentrations and hence health effects. The air pollution fusion model proposed is a Bayesian space-time linear regression model for relating the measured concentrations to the modelled concentrations for a single pollutant, whilst allowing for additional covariate information such as site type (e.g. roadside, rural, etc) and temperature. However, it is known that some pollutants might be correlated because they may be generated by common processes or be driven by similar factors such as meteorology. The correlation between pollutants can help to predict one pollutant by borrowing strength from the others. Therefore, in chapter 5, I propose a multi-pollutant model which is a multivariate spatio-temporal fusion model that extends the single pollutant model in chapter 4, which relates monitored and modelled pollution data over space, time and pollutant to predict pollution across mainland Scotland. Considering that we are exposed to multiple pollutants simultaneously because the air we breathe contains a complex mixture of particle and gas phase pollutants, the health effects of exposure to multiple pollutants have been investigated in chapter 6. Therefore, this is a natural extension to the single pollutant health effects in chapter 4. Given NO2 and PM10 are highly correlated (multicollinearity issue) in my data, I first propose a temporally-varying linear model to regress one pollutant (e.g. NO2) against another (e.g. PM10) and then use the residuals in the disease model as well as PM10, thus investigating the health effects of exposure to both pollutants simultaneously. Another issue considered in chapter 6 is to allow for the uncertainty in the estimated pollution concentrations when estimating their health effects. There are in total four approaches being developed to adjust the exposure uncertainty. Finally, chapter 7 summarises the work contained within this thesis and discusses the implications for future research.