Electricity price forecasting accounting for renewable energies: optimal combined forecasts

Electricity price forecasting is an interesting problem for all the agents involved in electricity market operation. For instance, every profit maximisation strategy is based on the computation of accurate one-day-ahead forecasts, which is why electricity price forecasting has been a growing field of research in recent years. In addition, the increasing concern about environmental issues has led to a high penetration of renewable energies, particularly wind. In some European countries such as Spain, Germany and Denmark, renewable energy is having a deep impact on the local power markets. In this paper, we propose an optimal model from the perspective of forecasting accuracy, and it consists of a combination of several univariate and multivariate time series methods that account for the amount of energy produced with clean energies, particularly wind and hydro, which are the most relevant renewable energy sources in the Iberian Market. This market is used to illustrate the proposed methodology, as it is one of those markets in which wind power production is more relevant in terms of its percentage of the total demand, but of course our method can be applied to any other liberalised power market. As far as our contribution is concerned, first, the methodology proposed by García-Martos et al(2007 and 2012) is generalised twofold: we allow the incorporation of wind power production and hydro reservoirs, and we do not impose the restriction of using the same model for 24h. A computational experiment and a Design of Experiments (DOE) are performed for this purpose. Then, for those hours in which there are two or more models without statistically significant differences in terms of their forecasting accuracy, a combination of forecasts is proposed by weighting the best models(according to the DOE) and minimising the Mean Absolute Percentage Error (MAPE). The MAPE is the most popular accuracy metric for comparing electricity price forecasting models. We construct the combi nation of forecasts by solving several nonlinear optimisation problems that allow computation of the optimal weights for building the combination of forecasts. The results are obtained by a large computational experiment that entails calculating out-of-sample forecasts for every hour in every day in the period from January 2007 to Decem ber 2009. In addition, to reinforce the value of our methodology, we compare our results with those that appear in recent published works in the field. This comparison shows the superiority of our methodology in terms of forecasting accuracy.

Aplicación de técnicas de inteligencia artificial a la predicción de contaminantes atmosféricos

Abstract Air pollution is a big threat and a phenomenon that has a specific impact on human health, in addition, changes that occur in the chemical composition of the atmosphere can change the weather and cause acid rain or ozone destruction. Those are phenomena of global importance. The World Health Organization (WHO) considerates air pollution as one of the most important global priorities. Salamanca, Gto., Mexico has been ranked as one of the most polluted cities in this country. The industry of the area led to a major economic development and rapid population growth in the second half of the twentieth century. The impact in the air quality is important and significant efforts have been made to measure the concentrations of pollutants. The main pollution sources are locally based plants in the chemical and power generation sectors. The registered concerning pollutants are Sulphur Dioxide (SO2) and particles on the order of ∼10 micrometers or less (PM10). The prediction in the concentration of those pollutants can be a powerful tool in order to take preventive measures such as the reduction of emissions and alerting the affected population. In this PhD thesis we propose a model to predict concentrations of pollutants SO2 and PM10 for each monitoring booth in the Atmospheric Monitoring Network Salamanca (REDMAS - for its spanish acronym). The proposed models consider the use of meteorological variables as factors influencing the concentration of pollutants. The information used along this work is the current real data from REDMAS. In the proposed model, Artificial Neural Networks (ANN) combined with clustering algorithms are used. The type of ANN used is the Multilayer Perceptron with a hidden layer, using separate structures for the prediction of each pollutant. The meteorological variables used for prediction were: Wind Direction (WD), wind speed (WS), Temperature (T) and relative humidity (RH). Clustering algorithms, K-means and Fuzzy C-means, are used to find relationships between air pollutants and weather variables under consideration, which are added as input of the RNA. Those relationships provide information to the ANN in order to obtain the prediction of the pollutants. The results of the model proposed in this work are compared with the results of a multivariate linear regression and multilayer perceptron neural network. The evaluation of the prediction is calculated with the mean absolute error, the root mean square error, the correlation coefficient and the index of agreement. The results show the importance of meteorological variables in the prediction of the concentration of the pollutants SO2 and PM10 in the city of Salamanca, Gto., Mexico. The results show that the proposed model perform better than multivariate linear regression and multilayer perceptron neural network. The models implemented for each monitoring booth have the ability to make predictions of air quality that can be used in a system of real-time forecasting and human health impact analysis. Among the main results of the development of this thesis we can cite: A model based on artificial neural network combined with clustering algorithms for prediction with a hour ahead of the concentration of each pollutant (SO2 and PM10) is proposed. A different model was designed for each pollutant and for each of the three monitoring booths of the REDMAS. A model to predict the average of pollutant concentration in the next 24 hours of pollutants SO2 and PM10 is proposed, based on artificial neural network combined with clustering algorithms. Model was designed for each booth of the REDMAS and each pollutant separately. Resumen La contaminación atmosférica es una amenaza aguda, constituye un fenómeno que tiene particular incidencia sobre la salud del hombre. Los cambios que se producen en la composición química de la atmósfera pueden cambiar el clima, producir lluvia ácida o destruir el ozono, fenómenos todos ellos de una gran importancia global. La Organización Mundial de la Salud (OMS) considera la contaminación atmosférica como una de las más importantes prioridades mundiales. Salamanca, Gto., México; ha sido catalogada como una de las ciudades más contaminadas en este país. La industria de la zona propició un importante desarrollo económico y un crecimiento acelerado de la población en la segunda mitad del siglo XX. Las afectaciones en el aire son graves y se han hecho importantes esfuerzos por medir las concentraciones de los contaminantes. Las principales fuentes de contaminación son fuentes fijas como industrias químicas y de generación eléctrica. Los contaminantes que se han registrado como preocupantes son el Bióxido de Azufre (SO2) y las Partículas Menores a 10 micrómetros (PM10). La predicción de las concentraciones de estos contaminantes puede ser una potente herramienta que permita tomar medidas preventivas como reducción de emisiones a la atmósfera y alertar a la población afectada. En la presente tesis doctoral se propone un modelo de predicción de concentraci ón de los contaminantes más críticos SO2 y PM10 para cada caseta de monitorización de la Red de Monitorización Atmosférica de Salamanca (REDMAS). Los modelos propuestos plantean el uso de las variables meteorol ógicas como factores que influyen en la concentración de los contaminantes. La información utilizada durante el desarrollo de este trabajo corresponde a datos reales obtenidos de la REDMAS. En el Modelo Propuesto (MP) se aplican Redes Neuronales Artificiales (RNA) combinadas con algoritmos de agrupamiento. La RNA utilizada es el Perceptrón Multicapa con una capa oculta, utilizando estructuras independientes para la predicción de cada contaminante. Las variables meteorológicas disponibles para realizar la predicción fueron: Dirección de Viento (DV), Velocidad de Viento (VV), Temperatura (T) y Humedad Relativa (HR). Los algoritmos de agrupamiento K-means y Fuzzy C-means son utilizados para encontrar relaciones existentes entre los contaminantes atmosféricos en estudio y las variables meteorológicas. Dichas relaciones aportan información a las RNA para obtener la predicción de los contaminantes, la cual es agregada como entrada de las RNA. Los resultados del modelo propuesto en este trabajo son comparados con los resultados de una Regresión Lineal Multivariable (RLM) y un Perceptrón Multicapa (MLP). La evaluación de la predicción se realiza con el Error Medio Absoluto, la Raíz del Error Cuadrático Medio, el coeficiente de correlación y el índice de acuerdo. Los resultados obtenidos muestran la importancia de las variables meteorológicas en la predicción de la concentración de los contaminantes SO2 y PM10 en la ciudad de Salamanca, Gto., México. Los resultados muestran que el MP predice mejor la concentración de los contaminantes SO2 y PM10 que los modelos RLM y MLP. Los modelos implementados para cada caseta de monitorizaci ón tienen la capacidad para realizar predicciones de calidad del aire, estos modelos pueden ser implementados en un sistema que permita realizar la predicción en tiempo real y analizar el impacto en la salud de la población. Entre los principales resultados obtenidos del desarrollo de esta tesis podemos citar: Se propone un modelo basado en una red neuronal artificial combinado con algoritmos de agrupamiento para la predicción con una hora de anticipaci ón de la concentración de cada contaminante (SO2 y PM10). Se diseñó un modelo diferente para cada contaminante y para cada una de las tres casetas de monitorización de la REDMAS. Se propone un modelo de predicción del promedio de la concentración de las próximas 24 horas de los contaminantes SO2 y PM10, basado en una red neuronal artificial combinado con algoritmos de agrupamiento. Se diseñó un modelo para cada caseta de monitorización de la REDMAS y para cada contaminante por separado.

Prediction of PM10 concentrations using Fuzzy c-Means and ANN

Salamanca has been considered among the most polluted cities in Mexico. The vehicular park, the industry and the emissions produced by agriculture, as well as orography and climatic characteristics have propitiated the increment in pollutant concentration of Particulate Matter less than 10 μg/m3 in diameter (PM10). In this work, a Multilayer Perceptron Neural Network has been used to make the prediction of an hour ahead of pollutant concentration. A database used to train the Neural Network corresponds to historical time series of meteorological variables (wind speed, wind direction, temperature and relative humidity) and air pollutant concentrations of PM10. Before the prediction, Fuzzy c-Means clustering algorithm have been implemented in order to find relationship among pollutant and meteorological variables. These relationship help us to get additional information that will be used for predicting. Our experiments with the proposed system show the importance of this set of meteorological variables on the prediction of PM10 pollutant concentrations and the neural network efficiency. The performance estimation is determined using the Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). The results shown that the information obtained in the clustering step allows a prediction of an hour ahead, with data from past 2 hours

ANN and Fuzzy c-Means applied to environmental pollution prediction

Salamanca, situated in center of Mexico is among the cities which suffer most from the air pollution in Mexico. The vehicular park and the industry, as well as orography and climatic characteristics have propitiated the increment in pollutant concentration of Sulphur Dioxide (SO2). In this work, a Multilayer Perceptron Neural Network has been used to make the prediction of an hour ahead of pollutant concentration. A database used to train the Neural Network corresponds to historical time series of meteorological variables and air pollutant concentrations of SO2. Before the prediction, Fuzzy c-Means and K-means clustering algorithms have been implemented in order to find relationship among pollutant and meteorological variables. Our experiments with the proposed system show the importance of this set of meteorological variables on the prediction of SO2 pollutant concentrations and the neural network efficiency. The performance estimation is determined using the Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). The results showed that the information obtained in the clustering step allows a prediction of an hour ahead, with data from past 2 hours.

Evaluation and improvement of empirical models of global solar irradiation: Case study northern Spain

This paper presents a new methodology to build parametric models to estimate global solar irradiation adjusted to specific on-site characteristics based on the evaluation of variable im- portance. Thus, those variables higly correlated to solar irradiation on a site are implemented in the model and therefore, different models might be proposed under different climates. This methodology is applied in a study case in La Rioja region (northern Spain). A new model is proposed and evaluated on stability and accuracy against a review of twenty-two already exist- ing parametric models based on temperatures and rainfall in seventeen meteorological stations in La Rioja. The methodology of model evaluation is based on bootstrapping, which leads to achieve a high level of confidence in model calibration and validation from short time series (in this case five years, from 2007 to 2011). The model proposed improves the estimates of the other twenty-two models with average mean absolute error (MAE) of 2.195 MJ/m2 day and average confidence interval width (95% C.I., n=100) of 0.261 MJ/m2 day. 41.65% of the daily residuals in the case of SIAR and 20.12% in that of SOS Rioja fall within the uncertainty tolerance of the pyranometers of the two networks (10% and 5%, respectively). Relative differences between measured and estimated irradiation on an annual cumulative basis are below 4.82%. Thus, the proposed model might be useful to estimate annual sums of global solar irradiation, reaching insignificant differences between measurements from pyranometers.

Downscaling of global solar irradiation in R

A methodology for downscaling solar irradiation from satellite-derived databases is described using R software. Different packages such as raster, parallel, solaR, gstat, sp and rasterVis are considered in this study for improving solar resource estimation in areas with complex topography, in which downscaling is a very useful tool for reducing inherent deviations in satellite-derived irradiation databases, which lack of high global spatial resolution. A topographical analysis of horizon blocking and sky-view is developed with a digital elevation model to determine what fraction of hourly solar irradiation reaches the Earth's surface. Eventually, kriging with external drift is applied for a better estimation of solar irradiation throughout the region analyzed. This methodology has been implemented as an example within the region of La Rioja in northern Spain, and the mean absolute error found is a striking 25.5% lower than with the original database.

Towards downscaling of aerosol gridded dataset for improving solar resource assessment, an application to Spain

Solar radiation estimates with clear sky models require estimations of aerosol data. The low spatial resolution of current aerosol datasets, with their remarkable drift from measured data, poses a problem in solar resource estimation. This paper proposes a new downscaling methodology by combining support vector machines for regression (SVR) and kriging with external drift, with data from the MACC reanalysis datasets and temperature and rainfall measurements from 213 meteorological stations in continental Spain. The SVR technique was proven efficient in aerosol variable modeling. The Linke turbidity factor (TL) and the aerosol optical depth at 550 nm (AOD 550) estimated with SVR generated significantly lower errors in AERONET positions than MACC reanalysis estimates. The TL was estimated with relative mean absolute error (rMAE) of 10.2% (compared with AERONET), against the MACC rMAE of 18.5%. A similar behavior was seen with AOD 550, estimated with rMAE of 8.6% (compared with AERONET), against the MACC rMAE of 65.6%. Kriging using MACC data as an external drift was found useful in generating high resolution maps (0.05° × 0.05°) of both aerosol variables. We created high resolution maps of aerosol variables in continental Spain for the year 2008. The proposed methodology was proven to be a valuable tool to create high resolution maps of aerosol variables (TL and AOD 550). This methodology shows meaningful improvements when compared with estimated available databases and therefore, leads to more accurate solar resource estimations. This methodology could also be applied to the prediction of other atmospheric variables, whose datasets are of low resolution.

An adaptive population importance sampler

Monte Carlo (MC) methods are widely used in signal processing, machine learning and communications for statistical inference and stochastic optimization. A well-known class of MC methods is composed of importance sampling and its adaptive extensions (e.g., population Monte Carlo). In this work, we introduce an adaptive importance sampler using a population of proposal densities. The novel algorithm provides a global estimation of the variables of interest iteratively, using all the samples generated. The cloud of proposals is adapted by learning from a subset of previously generated samples, in such a way that local features of the target density can be better taken into account compared to single global adaptation procedures. Numerical results show the advantages of the proposed sampling scheme in terms of mean absolute error and robustness to initialization.

Orthogonal MCMC algorithms

Monte Carlo (MC) methods are widely used in signal processing, machine learning and stochastic optimization. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In this work, we introduce a novel parallel interacting MCMC scheme, where the parallel chains share information using another MCMC technique working on the entire population of current states. These parallel ?vertical? chains are led by random-walk proposals, whereas the ?horizontal? MCMC uses a independent proposal, which can be easily adapted by making use of all the generated samples. Numerical results show the advantages of the proposed sampling scheme in terms of mean absolute error, as well as robustness w.r.t. to initial values and parameter choice.

Age regression from soft aligned face images using low computational resources

The initial step in most facial age estimation systems consists of accurately aligning a model to the output of a face detector (e.g. an Active Appearance Model). This fitting process is very expensive in terms of computational resources and prone to get stuck in local minima. This makes it impractical for analysing faces in resource limited computing devices. In this paper we build a face age regressor that is able to work directly on faces cropped using a state-of-the-art face detector. Our procedure uses K nearest neighbours (K-NN) regression with a metric based on a properly tuned Fisher Linear Discriminant Analysis (LDA) projection matrix. On FG-NET we achieve a state-of-the-art Mean Absolute Error (MAE) of 5.72 years with manually aligned faces. Using face images cropped by a face detector we get a MAE of 6.87 years in the same database. Moreover, most of the algorithms presented in the literature have been evaluated on single database experiments and therefore, they report optimistically biased results. In our cross-database experiments we get a MAE of roughly 12 years, which would be the expected performance in a real world application.

Desarrollo de un método estadístico para la generación de escenarios climáticos de alta resolución a partir de simulaciones de baja resolución de Modelos de Circulación General: aplicación a España

Esta tesis doctoral presenta el desarrollo, verificación y aplicación de un método original de regionalización estadística para generar escenarios locales de clima futuro de temperatura y precipitación diarias, que combina dos pasos. El primer paso es un método de análogos: los "n" días cuya configuración atmosférica de baja resolución es más parecida a la del día problema, se seleccionan de un banco de datos de referencia del pasado. En el segundo paso, se realiza un análisis de regresión múltiple sobre los "n" días más análogos para la temperatura, mientras que para la precipitación se utiliza la distribución de probabilidad de esos "n" días análogos para obtener la estima de precipitación. La verificación de este método se ha llevado a cabo para la España peninsular y las Islas Baleares. Los resultados muestran unas buenas prestaciones para temperatura (BIAS cerca de 0.1ºC y media de errores absolutos alrededor de 1.9ºC); y unas prestaciones aceptables para la precipitación (BIAS razonablemente bajo con una media de -18%; error medio absoluto menor que para una simulación de referencia (la persistencia); y una distribución de probabilidad simulada similar a la observada según dos test no-paramétricos de similitud). Para mostrar la aplicabilidad de la metodología desarrollada, se ha aplicado en detalle en un caso de estudio. El método se aplicó a cuatro modelos climáticos bajo diferentes escenarios futuros de emisiones de gases de efecto invernadero, para la región de Aragón, produciendo así proyecciones futuras de precipitación y temperaturas máximas y mínimas diarias. La fiabilidad de la técnica de regionalización fue evaluada de nuevo para el caso de estudio mediante un proceso de verificación. Para determinar la capacidad de los modelos climáticos para simular el clima real, sus simulaciones del pasado (la denominada salida 20C3M) se regionalizaron y luego se compararon con el clima observado (los resultados son bastante robustos para la temperatura y menos concluyentes para la precipitación). Las proyecciones futuras a escala local presentan un aumento significativo durante todo el siglo XXI de las temperaturas máximas y mínimas para todos los futuros escenarios de emisiones considerados. Las simulaciones de precipitación presentan mayores incertidumbres. Además, la aplicabilidad práctica del método se demostró también mediante su utilización para producir escenarios climáticos futuros para otros casos de estudio en los distintos sectores y regiones del mundo. Se ha prestado especial atención a una aplicación en Centroamérica, una región que ya está sufriendo importantes impactos del cambio climático y que tiene un clima muy diferente. ABSTRACT This doctoral thesis presents the development, verification and application of an original downscaling method for daily temperature and precipitation, which combines two statistical approaches. The first step is an analogue approach: the “n” days most similar to the day to be downscaled are selected. In the second step, a multiple regression analysis using the “n” most analogous days is performed for temperature, whereas for precipitation the probability distribution of the “n” analogous days is used to obtain the amount of precipitation. Verification of this method has been carried out for the Spanish Iberian Peninsula and the Balearic Islands. Results show good performance for temperature (BIAS close to 0.1ºC and Mean Absolute Errors around 1.9ºC); and an acceptable skill for precipitation (reasonably low BIAS with a mean of - 18%, Mean Absolute Error lower than for a reference simulation, i.e. persistence, and a well-simulated probability distribution according to two non-parametric tests of similarity). To show the applicability of the method, a study case has been analyzed. The method was applied to four climate models under different future emission scenarios for the region of Aragón, thus producing future projections of daily precipitation and maximum and minimum temperatures. The reliability of the downscaling technique was re-assessed for the study case by a verification process. To determine the ability of the climate models to simulate the real climate, their simulations of the past (the 20C3M output) were downscaled and then compared with the observed climate – the results are quite robust for temperature and less conclusive for the precipitation. The downscaled future projections exhibit a significant increase during the entire 21st century of the maximum and minimum temperatures for all the considered future emission scenarios. Precipitation simulations exhibit greater uncertainties. Furthermore, the practical applicability of the method was demonstrated also by using it to produce future climate scenarios for some other study cases in different sectors and regions of the world. Special attention was paid to an application of the method in Central America, a region that is already suffering from significant climate change impacts and that has a very different climate from others where the method was previously applied.

Comparative assessment of global irradiation from a satellite estimate model (CM SAF) and on-ground measurements (SIAR): a Spanish case study

An analysis and comparison of daily and yearly solar irradiation from the satellite CM SAF database and a set of 301 stations from the Spanish SIAR network is performed using data of 2010 and 2011. This analysis is completed with the comparison of the estimations of effective irradiation incident on three different tilted planes (fixed, two axis tracking, north-south hori- zontal axis) using irradiation from these two data sources. Finally, a new map of yearly values of irradiation both on the horizontal plane and on inclined planes is produced mixing both sources with geostatistical techniques (kriging with external drift, KED) The Mean Absolute Difference (MAD) between CM SAF and SIAR is approximately 4% for the irradiation on the horizontal plane and is comprised between 5% and 6% for the irradiation incident on the inclined planes. The MAD between KED and SIAR, and KED and CM SAF is approximately 3% for the irradiation on the horizontal plane and is comprised between 3% and 4% for the irradiation incident on the inclined planes. The methods have been implemented using free software, available as supplementary ma- terial, and the data sources are freely available without restrictions.

Comparative assessment of global irradiation from a satellite estimate model (CM SAF) and on-ground measurements (SIAR): a Spanish case study

An analysis and comparison of daily and yearly solar irradiation from the satellite CM SAF database and a set of 301 stations from the Spanish SIAR network is performed using data of 2010 and 2011. This analysis is completed with the comparison of the estimations of effective irradiation incident on three different tilted planes (fixed, two axis tracking, north-south hori- zontal axis) using irradiation from these two data sources. Finally, a new map of yearly values of irradiation both on the horizontal plane and on inclined planes is produced mixing both sources with geostatistical techniques (kriging with external drift, KED) The Mean Absolute Difference (MAD) between CM SAF and SIAR is approximately 4% for the irradiation on the horizontal plane and is comprised between 5% and 6% for the irradiation incident on the inclined planes. The MAD between KED and SIAR, and KED and CM SAF is approximately 3% for the irradiation on the horizontal plane and is comprised between 3% and 4% for the irradiation incident on the inclined planes. The methods have been implemented using free software, available as supplementary ma- terial, and the data sources are freely available without restrictions.

Video Prioritization for Unequal Error Protection

We analyze the effect of packet losses in video sequences and propose a lightweight Unequal Error Protection strategy which, by choosing which packet is discarded, reduces strongly the Mean Square Error of the received sequence

GIS analysis for the study of rural depopulation: the Kingdom of Valencia (Tomas Lopez 1789)

One of the more aspects that have shaped the landscape is the human impact. The human impact has the clearest indicator of the density of settlements in a particular geographic region. In this paper we study all settlements shown on the map of the Kingdom of Valencia, Spain Geographic Atlas (AGE) of Tomas Lopez (1788), and their correspondence with the current ones. To meet this goal we have developed a specific methodology, the systematic study of all existing settlements in historical cartography. This will determine which have disappeared and which have been renamed. The material used has been the historical cartography of Tomas Lopez, part of the AGE (1789), the Kingdom of Valencia (1789), sheets numbers (78, 79, 80 and 81); Current mapping of the provinces of Alicante, Valencia, Castellon, Teruel, Tattagona and Cuenca; As main software ArcGis V.9.3. The steps followed in the methodology are as follows: 1. Check the scale of the maps. Analyze the possible use of a spherical earth model. 2. Geo-reference of maps with latitude and longitude framework. Move the historical longitude origin to the origin longitude of modern cartography. 3 Digitize of all population settlements or cities. 4 Identify historic settlements or cities corresponding with current ones. 5. If the maps have the same orientation and scale, replace the coordinate transformation of historical settlements with a new one, by a translation in latitude and longitude equal to the calculated mean value of all ancient map points corresponding to the new. 6. Calculation of absolute accuracy of the two maps, i.e. the linear distance between the points of both maps. 7 draw in the GIS, the settlements without correspondence, in the current coordinates, and with a circle of mean error of the sheet, in order to locate their current location. If there are actual settlements exist within this circle, they are candidates to be the searched settlements. We analyzed more than 2000 settlements represented in the Atlas of Tomas Lopez of the Kingdom of Valencia (1789), of which almost 14.5% have no correspondence with the existing settlements. The rural landscape evolution of the Valencia, oldest kingdom of Valencia, one can say that can be severely affected by the anthropization suffered in the period from 1789 to the present, since 70% of existing settlements actually have appeared after Tomas Lopez¿s cartography, dated on 1789