Analysis and clustering of residential customers energy behavioral demand using smart meter data

Clustering methods are increasingly being applied to residential smart meter data, providing a number of important opportunities for distribution network operators (DNOs) to manage and plan the low voltage networks. Clustering has a number of potential advantages for DNOs including, identifying suitable candidates for demand response and improving energy profile modelling. However, due to the high stochasticity and irregularity of household level demand, detailed analytics are required to define appropriate attributes to cluster. In this paper we present in-depth analysis of customer smart meter data to better understand peak demand and major sources of variability in their behaviour. We find four key time periods in which the data should be analysed and use this to form relevant attributes for our clustering. We present a finite mixture model based clustering where we discover 10 distinct behaviour groups describing customers based on their demand and their variability. Finally, using an existing bootstrapping technique we show that the clustering is reliable. To the authors knowledge this is the first time in the power systems literature that the sample robustness of the clustering has been tested.

An Ensemble Smoother with Error Estimates

A smoother introduced earlier by van Leeuwen and Evensen is applied to a problem in which real obser vations are used in an area with strongly nonlinear dynamics. The derivation is new , but it resembles an earlier derivation by van Leeuwen and Evensen. Again a Bayesian view is taken in which the prior probability density of the model and the probability density of the obser vations are combined to for m a posterior density . The mean and the covariance of this density give the variance-minimizing model evolution and its errors. The assumption is made that the prior probability density is a Gaussian, leading to a linear update equation. Critical evaluation shows when the assumption is justified. This also sheds light on why Kalman filters, in which the same ap- proximation is made, work for nonlinear models. By reference to the derivation, the impact of model and obser vational biases on the equations is discussed, and it is shown that Bayes’ s for mulation can still be used. A practical advantage of the ensemble smoother is that no adjoint equations have to be integrated and that error estimates are easily obtained. The present application shows that for process studies a smoother will give superior results compared to a filter , not only owing to the smooth transitions at obser vation points, but also because the origin of features can be followed back in time. Also its preference over a strong-constraint method is highlighted. Further more, it is argued that the proposed smoother is more efficient than gradient descent methods or than the representer method when error estimates are taken into account

Evaluation of a polynya flux model by means of thermal infrared satellite estimates

We test the ability of a two-dimensional flux model to simulate polynya events with narrow open-water zones by comparing model results to ice-thickness and ice-production estimates derived from thermal infrared Moderate Resolution Imaging Spectroradiometer (MODIS) observations in conjunction with an atmospheric dataset. Given a polynya boundary and an atmospheric dataset, the model correctly reproduces the shape of an 11 day long event, using only a few simple conservation laws. Ice production is slightly overestimated by the model, owing to an underestimated ice thickness. We achieved best model results with the consolidation thickness parameterization developed by Biggs and others (2000). Observed regional discrepancies between model and satellite estimates might be a consequence of the missing representation of the dynamic of the thin-ice thickening (e.g. rafting). We conclude that this simplified polynya model is a valuable tool for studying polynya dynamics and estimating associated fluxes of single polynya events.

Independent uncertainty estimates for coefficient based sea surface temperature retrieval from the Along-Trck Scanning Radiometer instruments

We establish a methodology for calculating uncertainties in sea surface temperature estimates from coefficient based satellite retrievals. The uncertainty estimates are derived independently of in-situ data. This enables validation of both the retrieved SSTs and their uncertainty estimate using in-situ data records. The total uncertainty budget is comprised of a number of components, arising from uncorrelated (eg. noise), locally systematic (eg. atmospheric), large scale systematic and sampling effects (for gridded products). The importance of distinguishing these components arises in propagating uncertainty across spatio-temporal scales. We apply the method to SST data retrieved from the Advanced Along Track Scanning Radiometer (AATSR) and validate the results for two different SST retrieval algorithms, both at a per pixel level and for gridded data. We find good agreement between our estimated uncertainties and validation data. This approach to calculating uncertainties in SST retrievals has a wider application to data from other instruments and retrieval of other geophysical variables.

Uncertainties in tidally adjusted estimates of sea level rise flooding (bathtub model) for the Greater London

Sea-level rise (SLR) from global warming may have severe consequences for coastal cities, particularly when combined with predicted increases in the strength of tidal surges. Predicting the regional impact of SLR flooding is strongly dependent on the modelling approach and accuracy of topographic data. Here, the areas under risk of sea water flooding for London boroughs were quantified based on the projected SLR scenarios reported in Intergovernmental Panel on Climate Change (IPCC) fifth assessment report (AR5) and UK climatic projections 2009 (UKCP09) using a tidally-adjusted bathtub modelling approach. Medium- to very high-resolution digital elevation models (DEMs) are used to evaluate inundation extents as well as uncertainties. Depending on the SLR scenario and DEMs used, it is estimated that 3%–8% of the area of Greater London could be inundated by 2100. The boroughs with the largest areas at risk of flooding are Newham, Southwark, and Greenwich. The differences in inundation areas estimated from a digital terrain model and a digital surface model are much greater than the root mean square error differences observed between the two data types, which may be attributed to processing levels. Flood models from SRTM data underestimate the inundation extent, so their results may not be reliable for constructing flood risk maps. This analysis provides a broad-scale estimate of the potential consequences of SLR and uncertainties in the DEM-based bathtub type flood inundation modelling for London boroughs.

Simultaneous activities in the household and residential electricity demand in Spain

Recent research and policy studies on the low-carbon future highlight the importance of flexible electricity demand. This might be problematic particularly for residential electricity demand, which is related to simultaneous consumers’ practices in the household. This paper analyses issues of simultaneity in residential electricity demand in Spain. It makes use of the 2011 Spanish Time Use Survey data with comparisons from the previous Spanish Time Use Survey and the Harmonised European Time Use Surveys. Findings show that media activities are associated the highest levels of continuity and simultaneity, particularly in the early and late parts of the evening during weekdays.

Apple pollination: demand depends on variety and supply depends on pollinator identity

Insect pollination underpins apple production but the extent to which different pollinator guilds supply this service, particularly across different apple varieties, is unknown. Such information is essential if appropriate orchard management practices are to be targeted and proportional to the potential benefits pollinator species may provide. Here we use a novel combination of pollinator effectiveness assays (floral visit effectiveness), orchard field surveys (flower visitation rate) and pollinator dependence manipulations (pollinator exclusion experiments) to quantify the supply of pollination services provided by four different pollinator guilds to the production of four commercial varieties of apple. We show that not all pollinators are equally effective at pollinating apples, with hoverflies being less effective than solitary bees and bumblebees, and the relative abundance of different pollinator guilds visiting apple flowers of different varieties varies significantly. Based on this, the taxa specific economic benefits to UK apple production have been established. The contribution of insect pollinators to the economic output in all varieties was estimated to be £92.1M across the UK, with contributions varying widely across taxa: solitary bees (£51.4M), honeybees (£21.4M), bumblebees (£18.6M) and hoverflies (£0.7M). This research highlights the differences in the economic benefits of four insect pollinator guilds to four major apple varieties in the UK. This information is essential to underpin appropriate investment in pollination services management and provides a model that can be used in other entomolophilous crops to improve our understanding of crop pollination ecology.

Use of a predictive model for food insecurity estimates in Brazil

In 2004 the National Household Survey (Pesquisa Nacional par Amostras de Domicilios - PNAD) estimated the prevalence of food and nutrition insecurity in Brazil. However, PNAD data cannot be disaggregated at the municipal level. The objective of this study was to build a statistical model to predict severe food insecurity for Brazilian municipalities based on the PNAD dataset. Exclusion criteria were: incomplete food security data (19.30%); informants younger than 18 years old (0.07%); collective households (0.05%); households headed by indigenous persons (0.19%). The modeling was carried out in three stages, beginning with the selection of variables related to food insecurity using univariate logistic regression. The variables chosen to construct the municipal estimates were selected from those included in PNAD as well as the 2000 Census. Multivariate logistic regression was then initiated, removing the non-significant variables with odds ratios adjusted by multiple logistic regression. The Wald Test was applied to check the significance of the coefficients in the logistic equation. The final model included the variables: per capita income; years of schooling; race and gender of the household head; urban or rural residence; access to public water supply; presence of children; total number of household inhabitants and state of residence. The adequacy of the model was tested using the Hosmer-Lemeshow test (p=0.561) and ROC curve (area=0.823). Tests indicated that the model has strong predictive power and can be used to determine household food insecurity in Brazilian municipalities, suggesting that similar predictive models may be useful tools in other Latin American countries.

Extended time weather forecasts contributes to agricultural productivity estimates

Weather conditions in critical periods of the vegetative crop development influence crop productivity, thus being a basic parameter for crop forecast. Reliable extended period weather forecasts may contribute to improve the estimation of agricultural productivity. The production of soybean plays an important role in the Brazilian economy, because this country is ranked among the largest producers of soybeans in the world. This culture can be significantly affected by water conditions, depending on the intensity of water deficit. This work explores the role of extended period weather forecasts for estimating soybean productivity in the southern part of Brazil, Passo Fundo, and Londrina (State of Rio Grande do Sul and Parana, respectively) in the 2005/2006 harvest. The goal was to investigate the possible contribution of precipitation forecasts as a substitute for the use of climatological data on crop forecasts. The results suggest that the use of meteorological forecasts generate more reliable productivity estimates during the growth period than those generated only through climatological information.

Downward longwave radiation estimates for clear and all-sky conditions in the Sertozinho region of So Paulo, Brazil

The accurate estimate of the surface longwave fluxes contribution is important for the calculation of the surface radiation budget, which in turn controls all the components of the surface energy budget, such as evaporation and the sensible heat fluxes. This study evaluates the performance of the various downward longwave radiation parameterizations for clear and all-sky days applied to the Sertozinho region in So Paulo, Brazil. Equations have been adjusted to the observations of longwave radiation. The adjusted equations were evaluated for every hour throughout the day and the results showed good fits for most of the day, except near dawn and sunset, followed by nighttime. The seasonal variation was studied by comparing the dry period against the rainy period in the dataset. The least square linear regressions resulted in coefficients equal to the coefficients found for the complete period, both in the dry period and in the rainy period. It is expected that the best fit equation to the observed data for this site be used to produce estimates in other regions of the State of So Paulo, where such information is not available.

Satellite Rainfall Estimates Over South America - Possible Applicability to the Water Management of Large Watersheds1

This work analyzes high-resolution precipitation data from satellite-derived rainfall estimates over South America, especially over the Amazon Basin. The goal is to examine whether satellite-derived precipitation estimates can be used in hydrology and in the management of larger watersheds of South America. High spatial-temporal resolution precipitation estimates obtained with the CMORPH method serve this purpose while providing an additional hydrometeorological perspective on the convective regime over South America and its predictability. CMORPH rainfall estimates at 8-km spatial resolution for 2003 and 2004 were compared with available rain gauge measurements at daily, monthly, and yearly accumulation time scales. The results show the correlation between satellite-derived and gauge-measured precipitation increases with accumulation period from daily to monthly, especially during the rainy season. Time-longitude diagrams of CMORPH hourly rainfall show the genesis, strength, longevity, and phase speed of convective systems. Hourly rainfall analyses indicate that convection over the Amazon region is often more organized than previously thought, thus inferring that basin scale predictions of rainfall for hydrological and water management purposes have the potential to become more skillful. Flow estimates based on CMORPH and the rain gauge network are compared to long-term observed average flow. The results suggest this satellite-based rainfall estimation technique has considerable utility. Other statistics for monthly accumulations also suggest CMORPH can be an important source of rainfall information at smaller spatial scales where in situ observations are lacking.

Effects of detectability on estimates of geographic range size in bignonieae

Extinction risk has not been evaluated for 96% of all described plant species. Given that the Global Strategy for Plant Conservation proposes preliminary conservation assessments of all described plant species by 2010, herbarium specimens (i.e., primary occurrence data) are increasingly being used to infer threat components from estimates of geographic range size. Nevertheless, estimates of range size based on herbarium data may be inaccurate due to collection bias associated with interspecific variation in detectability. We used data on 377 species of Bignonieae to test the hypothesis that there is a positive relationship between detectability and estimates of geographic range size derived from herbarium specimens. This relationship is expected if the proportion of the true geographic range size of a species that is documented by herbarium specimens is given by the product of the true geographic range size and the detectability of the species, assuming no relationship between true geographic range size and detectability. We developed 4 measures of detectability that can be estimated from herbarium data and examined the relationship between detectability and 2 types of estimates of geographic range size: area of occupancy and extent of occurrence. Our results from regressing estimates of extent of occurrence and area of occupancy on detectability across genera provided no support for this hypothesis. The same was true for regressions of estimated extent of occurrence on detectability across species within genera. Nevertheless, regressions of estimated area of occupancy on detectability across species within genera provided partial support for our hypothesis. We considered 3 possible explanations for this mixed outcome: violation of the assumption of no relationship between true geographic range size and detectability; the relationships between estimated geographic range size and detectability may be an artifact of a negative relationship between estimated area of occupancy and the sampling variance of detectability; detectability may have had 2 opposite effects on estimated species range sizes: one determines the proportion of the true range of a species documented by herbarium specimens and the other determines the distribution of true range size for the species actually observed with herbarium data. Our findings should help improve understanding of the potential biases incurred with the use of herbarium data.

Do individuals in better condition survive for longer? Field survival estimates according to male alternative reproductive tactics and sex

There is a gap in terms of the supposed survival differences recorded in the field according to individual condition. This is partly due to our inability to assess survival in the wild. Here we applied modern statistical techniques to field-gathered data in two damselfly species whose males practice alternative reproductive tactics (ARTs) and whose indicators of condition in both sexes are known. In Paraphlebia zoe, there are two ART: a larger black-winged (BW) male which defends mating territories and a smaller hyaline-winged (HW) male that usually acts as a satellite. In this species, condition in both morphs is correlated with body size. In Calopteryx haemorrhoidalis, males follow tactics according to their condition with males in better condition practicing a territorial ART. In addition, in this species, condition correlates positively with wing pigmentation in both sexes. Our prediction for both species was that males practicing the territorial tactic will survive less longer than males using a nonterritorial tactic, and larger or more pigmented animals will survive for longer. In P. zoe, BW males survived less than females but did not differ from HW males, and not necessarily larger individuals survived for longer. In fact, size affected survival but only when group identity was analysed, showing a positive relationship in females and a slightly negative relationship in both male morphs. For C. haemorrhoidalis, survival was larger for more pigmented males and females, but size was not a good survival predictor. Our results partially confirm assumptions based on the maintenance of ARTs. Our results also indicate that female pigmentation, correlates with a fitness component - survival - as proposed by recent sexual selection ideas applied to females.

On the cutting stock problem under stochastic demand

This paper addresses the one-dimensional cutting stock problem when demand is a random variable. The problem is formulated as a two-stage stochastic nonlinear program with recourse. The first stage decision variables are the number of objects to be cut according to a cutting pattern. The second stage decision variables are the number of holding or backordering items due to the decisions made in the first stage. The problem`s objective is to minimize the total expected cost incurred in both stages, due to waste and holding or backordering penalties. A Simplex-based method with column generation is proposed for solving a linear relaxation of the resulting optimization problem. The proposed method is evaluated by using two well-known measures of uncertainty effects in stochastic programming: the value of stochastic solution-VSS-and the expected value of perfect information-EVPI. The optimal two-stage solution is shown to be more effective than the alternative wait-and-see and expected value approaches, even under small variations in the parameters of the problem.

Analysis and risk estimates to workers of Brazilian granitic industries and sandblasters exposed to respirable crystalline silica and natural radionuclides

Occupational exposure to respirable crystalline silica and to radiation emitted by natural radionuclides present both in rocks and sands was studied in the Brazilian extractive process and granite product manufacture. Respirable airborne dust samples were collected in working environments, where workers perform different tasks with distinct commercial granites types, and also in places where sandblasters work with sands from different origins. The free crystalline silica contents were determined using X-ray diffraction of the respirable particulate fraction of each sample. Dust samples from granite cutting and sandblasting ambient had the natural radionuclides concentrations measured by gamma spectrometry. Dust concentrations in the workplaces were quite variable, reaching values up to 10 times higher than the respirable particle mass threshold limit value (TLV) set by the American Conference for Governmental Industrial Hygienists of 3 mg m(-3). Also the free crystalline silica concentrations were high. reaching values up to 48 times the TLV of 0.025 mg m(-3). Additionally, our results suggest that the risk of radiation-induced cancer in the granite or marble industries is negligible. However, the combined exposure to dust, gamma radiation, and radon daughter products could result in the enhancement of lung cancer risks associated to sandblasting activities. (C) 2010 Elsevier Ltd. All rights reserved.