Integrating structured and unstructured data in a business-intelligence-system

The recent liberalization of the German energy market has forced the energy industry to develop and install new information systems to support agents on the energy trading floors in their analytical tasks. Besides classical approaches of building a data warehouse giving insight into the time series to understand market and pricing mechanisms, it is crucial to provide a variety of external data from the web. Weather information as well as political news or market rumors are relevant to give the appropriate interpretation to the variables of a volatile energy market. Starting from a multidimensional data model and a collection of buy and sell transactions a data warehouse is built that gives analytical support to the agents. Following the idea of web farming we harvest the web, match the external information sources after a filtering and evaluation process to the data warehouse objects, and present this qualified information on a user interface where market values are correlated with those external sources over the time axis.

Replication Data Management,needs and solutions: an initial evaluation of conceptual approaches for integrating heterogeneous replication study data

Replication Data Management (RDM) aims at enabling the use of data collections from several iterations of an experiment. However, there are several major challenges to RDM from integrating data models and data from empirical study infrastructures that were not designed to cooperate, e.g., data model variation of local data sources. [Objective] In this paper we analyze RDM needs and evaluate conceptual RDM approaches to support replication researchers. [Method] We adapted the ATAM evaluation process to (a) analyze RDM use cases and needs of empirical replication study research groups and (b) compare three conceptual approaches to address these RDM needs: central data repositories with a fixed data model, heterogeneous local repositories, and an empirical ecosystem. [Results] While the central and local approaches have major issues that are hard to resolve in practice, the empirical ecosystem allows bridging current gaps in RDM from heterogeneous data sources. [Conclusions] The empirical ecosystem approach should be explored in diverse empirical environments.

Retrieval of grassland biophysical parameters using multitemporal optical and radar satellite data

The amount and quality of available biomass is a key factor for the sustainable livestock industry and agricultural management related decision making. Globally 31.5% of land cover is grassland while 80% of Ireland’s agricultural land is grassland. In Ireland, grasslands are intensively managed and provide the cheapest feed source for animals. This dissertation presents a detailed state of the art review of satellite remote sensing of grasslands, and the potential application of optical (Moderate–resolution Imaging Spectroradiometer (MODIS)) and radar (TerraSAR-X) time series imagery to estimate the grassland biomass at two study sites (Moorepark and Grange) in the Republic of Ireland using both statistical and state of the art machine learning algorithms. High quality weather data available from the on-site weather station was also used to calculate the Growing Degree Days (GDD) for Grange to determine the impact of ancillary data on biomass estimation. In situ and satellite data covering 12 years for the Moorepark and 6 years for the Grange study sites were used to predict grassland biomass using multiple linear regression, Neuro Fuzzy Inference Systems (ANFIS) models. The results demonstrate that a dense (8-day composite) MODIS image time series, along with high quality in situ data, can be used to retrieve grassland biomass with high performance (R2 = 0:86; p < 0:05, RMSE = 11.07 for Moorepark). The model for Grange was modified to evaluate the synergistic use of vegetation indices derived from remote sensing time series and accumulated GDD information. As GDD is strongly linked to the plant development, or phonological stage, an improvement in biomass estimation would be expected. It was observed that using the ANFIS model the biomass estimation accuracy increased from R2 = 0:76 (p < 0:05) to R2 = 0:81 (p < 0:05) and the root mean square error was reduced by 2.72%. The work on the application of optical remote sensing was further developed using a TerraSAR-X Staring Spotlight mode time series over the Moorepark study site to explore the extent to which very high resolution Synthetic Aperture Radar (SAR) data of interferometrically coherent paddocks can be exploited to retrieve grassland biophysical parameters. After filtering out the non-coherent plots it is demonstrated that interferometric coherence can be used to retrieve grassland biophysical parameters (i. e., height, biomass), and that it is possible to detect changes due to the grass growth, and grazing and mowing events, when the temporal baseline is short (11 days). However, it not possible to automatically uniquely identify the cause of these changes based only on the SAR backscatter and coherence, due to the ambiguity caused by tall grass laid down due to the wind. Overall, the work presented in this dissertation has demonstrated the potential of dense remote sensing and weather data time series to predict grassland biomass using machine-learning algorithms, where high quality ground data were used for training. At present a major limitation for national scale biomass retrieval is the lack of spatial and temporal ground samples, which can be partially resolved by minor modifications in the existing PastureBaseIreland database by adding the location and extent ofeach grassland paddock in the database. As far as remote sensing data requirements are concerned, MODIS is useful for large scale evaluation but due to its coarse resolution it is not possible to detect the variations within the fields and between the fields at the farm scale. However, this issue will be resolved in terms of spatial resolution by the Sentinel-2 mission, and when both satellites (Sentinel-2A and Sentinel-2B) are operational the revisit time will reduce to 5 days, which together with Landsat-8, should enable sufficient cloud-free data for operational biomass estimation at a national scale. The Synthetic Aperture Radar Interferometry (InSAR) approach is feasible if there are enough coherent interferometric pairs available, however this is difficult to achieve due to the temporal decorrelation of the signal. For repeat-pass InSAR over a vegetated area even an 11 days temporal baseline is too large. In order to achieve better coherence a very high resolution is required at the cost of spatial coverage, which limits its scope for use in an operational context at a national scale. Future InSAR missions with pair acquisition in Tandem mode will minimize the temporal decorrelation over vegetation areas for more focused studies. The proposed approach complements the current paradigm of Big Data in Earth Observation, and illustrates the feasibility of integrating data from multiple sources. In future, this framework can be used to build an operational decision support system for retrieval of grassland biophysical parameters based on data from long term planned optical missions (e. g., Landsat, Sentinel) that will ensure the continuity of data acquisition. Similarly, Spanish X-band PAZ and TerraSAR-X2 missions will ensure the continuity of TerraSAR-X and COSMO-SkyMed.

Predicting effects of monotony on driver’s vigilance

Monotony has been identified as a contributing factor to road crashes. Drivers’ ability to react to unpredictable events deteriorates when exposed to highly predictable and uneventful driving tasks, such as driving on Australian rural roads, many of which are monotonous by nature. Highway design in particular attempts to reduce the driver’s task to a merely lane-keeping one. Such a task provides little stimulation and is monotonous, thus affecting the driver’s attention which is no longer directed towards the road. Inattention contributes to crashes, especially for professional drivers. Monotony has been studied mainly from the endogenous perspective (for instance through sleep deprivation) without taking into account the influence of the task itself (repetitiveness) or the surrounding environment. The aim and novelty of this thesis is to develop a methodology (mathematical framework) able to predict driver lapses of vigilance under monotonous environments in real time, using endogenous and exogenous data collected from the driver, the vehicle and the environment. Existing approaches have tended to neglect the specificity of task monotony, leaving the question of the existence of a “monotonous state” unanswered. Furthermore the issue of detecting vigilance decrement before it occurs (predictions) has not been investigated in the literature, let alone in real time. A multidisciplinary approach is necessary to explain how vigilance evolves in monotonous conditions. Such an approach needs to draw on psychology, physiology, road safety, computer science and mathematics. The systemic approach proposed in this study is unique with its predictive dimension and allows us to define, in real time, the impacts of monotony on the driver’s ability to drive. Such methodology is based on mathematical models integrating data available in vehicles to the vigilance state of the driver during a monotonous driving task in various environments. The model integrates different data measuring driver’s endogenous and exogenous factors (related to the driver, the vehicle and the surrounding environment). Electroencephalography (EEG) is used to measure driver vigilance since it has been shown to be the most reliable and real time methodology to assess vigilance level. There are a variety of mathematical models suitable to provide a framework for predictions however, to find the most accurate model, a collection of mathematical models were trained in this thesis and the most reliable was found. The methodology developed in this research is first applied to a theoretically sound measure of sustained attention called Sustained Attention Response to Task (SART) as adapted by Michael (2010), Michael and Meuter (2006, 2007). This experiment induced impairments due to monotony during a vigilance task. Analyses performed in this thesis confirm and extend findings from Michael (2010) that monotony leads to an important vigilance impairment independent of fatigue. This thesis is also the first to show that monotony changes the dynamics of vigilance evolution and tends to create a “monotonous state” characterised by reduced vigilance. Personality traits such as being a low sensation seeker can mitigate this vigilance decrement. It is also evident that lapses in vigilance can be predicted accurately with Bayesian modelling and Neural Networks. This framework was then applied to the driving task by designing a simulated monotonous driving task. The design of such task requires multidisciplinary knowledge and involved psychologist Rebecca Michael. Monotony was varied through both the road design and the road environment variables. This experiment demonstrated that road monotony can lead to driving impairment. Particularly monotonous road scenery was shown to have the most impact compared to monotonous road design. Next, this study identified a variety of surrogate measures that are correlated with vigilance levels obtained from the EEG. Such vigilance states can be predicted with these surrogate measures. This means that vigilance decrement can be detected in a car without the use of an EEG device. Amongst the different mathematical models tested in this thesis, only Neural Networks predicted the vigilance levels accurately. The results of both these experiments provide valuable information about the methodology to predict vigilance decrement. Such an issue is quite complex and requires modelling that can adapt to highly inter-individual differences. Only Neural Networks proved accurate in both studies, suggesting that these models are the most likely to be accurate when used on real roads or for further research on vigilance modelling. This research provides a better understanding of the driving task under monotonous conditions. Results demonstrate that mathematical modelling can be used to determine the driver’s vigilance state when driving using surrogate measures identified during this study. This research has opened up avenues for future research and could result in the development of an in-vehicle device predicting driver vigilance decrement. Such a device could contribute to a reduction in crashes and therefore improve road safety.

Disseminação de dados estatísticos georeferenciados através de WEB services e software livre.

Diferentes organizações públicas e privadas coletam e disponibilizam uma massa de dados sobre a realidade sócio-econômica das diferentes nações. Há hoje, da parte do governo brasileiro, um interesse manifesto de divulgar uma gama diferenciada de informações para os mais diversos perfis de usuários. Persiste, contudo, uma série de limitações para uma divulgação mais massiva e democrática, entre elas, a heterogeneidade das fontes de dados, sua dispersão e formato de apresentação pouco amigável. Devido à complexidade inerente à informação geográfica envolvida, que produz incompatibilidade em vários níveis, o intercâmbio de dados em sistemas de informação geográfica não é problema trivial. Para aplicações desenvolvidas para a Web, uma solução são os Web Services que permitem que novas aplicações possam interagir com aquelas que já existem e que sistemas desenvolvidos em plataformas diferentes sejam compatíveis. Neste sentido, o objetivo do trabalho é mostrar as possibilidades de construção de portais usando software livre, a tecnologia dos Web Services e os padrões do Open Geospatial Consortium (OGC) para a disseminação de dados espaciais. Visando avaliar e testar as tecnologias selecionadas e comprovar sua efetividade foi desenvolvido um exemplo de portal de dados sócio-econômicos, compreendendo informações de um servidor local e de servidores remotos. As contribuições do trabalho são a disponibilização de mapas dinâmicos, a geração de mapas através da composição de mapas disponibilizados em servidores remotos e local e o uso do padrão OGC WMC. Analisando o protótipo de portal construído, verifica-se, contudo, que a localização e requisição de Web Services não são tarefas fáceis para um usuário típico da Internet. Nesta direção, os trabalhos futuros no domínio dos portais de informação geográfica poderiam adotar a tecnologia Representational State Transfer (REST).

Content Based Integration of Construction Site Images in AEC/FM Model Based Systems

The Architecture, Engineering, Construction and Facilities Management (AEC/FM) industry is rapidly becoming a multidisciplinary, multinational and multi-billion dollar economy, involving large numbers of actors working concurrently at different locations and using heterogeneous software and hardware technologies. Since the beginning of the last decade, a great deal of effort has been spent within the field of construction IT in order to integrate data and information from most computer tools used to carry out engineering projects. For this purpose, a number of integration models have been developed, like web-centric systems and construction project modeling, a useful approach in representing construction projects and integrating data from various civil engineering applications. In the modern, distributed and dynamic construction environment it is important to retrieve and exchange information from different sources and in different data formats in order to improve the processes supported by these systems. Previous research demonstrated that a major hurdle in AEC/FM data integration in such systems is caused by its variety of data types and that a significant part of the data is stored in semi-structured or unstructured formats. Therefore, new integrative approaches are needed to handle non-structured data types like images and text files. This research is focused on the integration of construction site images. These images are a significant part of the construction documentation with thousands stored in site photographs logs of large scale projects. However, locating and identifying such data needed for the important decision making processes is a very hard and time-consuming task, while so far, there are no automated methods for associating them with other related objects. Therefore, automated methods for the integration of construction images are important for construction information management. During this research, processes for retrieval, classification, and integration of construction images in AEC/FM model based systems have been explored. Specifically, a combination of techniques from the areas of image and video processing, computer vision, information retrieval, statistics and content-based image and video retrieval have been deployed in order to develop a methodology for the retrieval of related construction site image data from components of a project model. This method has been tested on available construction site images from a variety of sources like past and current building construction and transportation projects and is able to automatically classify, store, integrate and retrieve image data files in inter-organizational systems so as to allow their usage in project management related tasks.

Supporting interoperability frameworks in the UK public sector

This paper describes the design and implementation of a component based software application, which alleviates the problem of software interoperability in the UK public sector. We analyze the current interoperability frameworks across the United Kingdom (UK) and European Union (EU) and propose a software solution that enhances such interoperability initiatives. Our example scenario is placed within a UK local authority, which shares data stored within the Police databases, for making efficient and more accurate operational decisions. The prototype, implemented as a J2EE application and built upon existing databases, proves our concept that it is possible to achieve data and application interoperability without integrating data sources and without using XML formats for data sharing.

Genome-wide meta-analysis for serum calcium identifies significantly associated SNPs near the calcium-sensing receptor (CASR) gene.

Calcium has a pivotal role in biological functions, and serum calcium levels have been associated with numerous disorders of bone and mineral metabolism, as well as with cardiovascular mortality. Here we report results from a genome-wide association study of serum calcium, integrating data from four independent cohorts including a total of 12,865 individuals of European and Indian Asian descent. Our meta-analysis shows that serum calcium is associated with SNPs in or near the calcium-sensing receptor (CASR) gene on 3q13. The top hit with a p-value of 6.3 x 10(-37) is rs1801725, a missense variant, explaining 1.26% of the variance in serum calcium. This SNP had the strongest association in individuals of European descent, while for individuals of Indian Asian descent the top hit was rs17251221 (p = 1.1 x 10(-21)), a SNP in strong linkage disequilibrium with rs1801725. The strongest locus in CASR was shown to replicate in an independent Icelandic cohort of 4,126 individuals (p = 1.02 x 10(-4)). This genome-wide meta-analysis shows that common CASR variants modulate serum calcium levels in the adult general population, which confirms previous results in some candidate gene studies of the CASR locus. This study highlights the key role of CASR in calcium regulation.

Revising the Program Portfolio: Elevating Academic Program Performance and Strategic Alignment

At many institutions, program review is an underproductive exercise. Review of existing programs is often a check-the-box formality, with inconsistent criteria and little connection to institutional priorities or funding considerations. Decisions about where to concentrate resources across the portfolio can be highly politicized. This report profiles how academic planning exemplars use program review as a strategic tool, integrating data on academic quality, student demand, and resource utilization to improve the economics of challenged programs and prioritize programs for investment and expansion.

Trajetórias tecnológicas e sistemas agrários: qualificação territorial utilizando geoprocessamento em Paragominas e Santarém/PA

A Amazônia tem sido observada principalmente por meio do fenômeno do desmatamento, utilizando recursos tradicionais do sensoriamento remoto como a quantificação de área desflorestada e seu posterior incremento anual, que parece se constituir em uma metodologia eficaz. Ratificando este raciocínio, constatei num levantamento de 16.591 multas aplicadas pelo IBAMA/PA no período de 2000 até 2008, que mais de 85,0% das autuações estavam relacionadas apenas à componente flora;e na jurisdição da Gerência do IBAMA em Santarém, oeste do Pará, em 2008, quase 60% das multas se deu por conta de desflorestamento, identificados via sensoriamento remoto. Ressalta-se que as análises de imagens de satélites por si só não definem elementos da superfície terrestre, contribuindo pouco para o entendimento e posterior intervenção da realidade. Nesse contexto, foram investigados e vetorizados 479 estabelecimentos rurais nas regiões de Paragominas e Santarém, estado do Pará, que possuem históricos de uso e ocupação distintos, qualificando-os segundo suas trajetórias tecnológicas prevalentes, na perspectiva apresentada por Costa, concretizando um importante passo para correção das distorções no desenvolvimento econômico, agregando informação ao dado de sensoriamento remoto. Aplicaram-se recursos geotecnológicos de métricas de paisagem, construíram-se banco de dados celular integrado com estatísticas e algoritmos de otimização probabilística, associando dados de classificação não supervisionada isodata (validadas com kappa= 0,87, classificação considerada“excelente”) com os tipos de produção coletados em campo, gerando uma classificação final "integrada"(kappa= 0,78, classificação “muito boa”). Na região de Paragominas, foram qualificadas 3 tipos de trajetórias tecnológicas, a camponesa T8 (domínio de culturas temporárias), a camponesa T3 e patronal T4 (especializadas em pecuária de corte). Em Santarém, revelaram-se 2 trajetórias, a camponesa T2 (forte presença de culturas permanentes, temporárias e sistemas agro florestais) e a patronal T7 (mutação da T4, aumentando a participação das culturas temporárias). A metodologia aplicada logrou êxito, espacializando as propriedades rurais segundo seus tipos de trajetórias tecnológicas e gerando classes de uso mais representativas, como cultura temporária e pastagem, mas que na classificação via sensoriamento remoto isodata é englobada pela classe “agropecuária”, possibilitando uma visão mais realista das atividades de produção desenvolvidas na área investigada, concretizando a geração de informações espaciais integrando dados de diferentes fontes e o aumento do poder de leitura do pixel.

Caracterização da sensibilidade ambiental da Baixada Santista no estado de São Paulo

This study aimed to perform the analysis and characterization of environmental sensitivity to oil from Baixada Santista in the State of São Paulo. The work was done by integrating data of physical environmet, socio-economic activities and the biological / ecological provided by the Research Group on Environmental Sensitivity to Oil Spill in the Institute of Geosciences and Exact Sciences of UNESP “Julio de Mesquita Filho” wich works in the conjunction with the Program of Human Resources Training in Geosciences and Environmental Sciences Applied to the Oil and Gas (PRH-05) of the National Petroleum Agency (ANP). Were also performed descriptive statistical analysis, based on dispersion and trend parameters, which allowed to answer questions related to environmental sensitivity index (ISL) prevalence, the predominance of the ISL by environment and the predominance of the environments of the area, thus providing an overview of Baixada Santista’s main towns about sensitivity to oil. Analyses performed in this study may also help mitigate the environmental and socioeconomic impacts and contribute to contingency plans development for Baixada Santista

Mapeamento de dados para sistema de informação de Pós-graduação

Pós-graduação em Ciência da Informação - FFC

What drives the seasonality of photosynthesis across the Amazon basin? A cross-site analysis of eddy flux tower measurements from the Brasil flux network

We investigated the seasonal patterns of Amazonian forest photosynthetic activity, and the effects thereon of variations in climate and land-use, by integrating data from a network of ground-based eddy flux towers in Brazil established as part of the ‘Large-Scale Biosphere Atmosphere Experiment in Amazonia’ project. We found that degree of water limitation, as indicated by the seasonality of the ratio of sensible to latent heat flux (Bowen ratio) predicts seasonal patterns of photosynthesis. In equatorial Amazonian forests (5◦ N–5◦ S), water limitation is absent, and photosynthetic fluxes (or gross ecosystem productivity, GEP) exhibit high or increasing levels of photosynthetic activity as the dry season progresses, likely a consequence of allocation to growth of new leaves. In contrast, forests along the southern flank of the Amazon, pastures converted from forest, and mixed forest-grass savanna, exhibit dry-season declines in GEP, consistent with increasing degrees of water limitation. Although previous work showed tropical ecosystem evapotranspiration (ET) is driven by incoming radiation, GEP observations reported here surprisingly show no or negative relationships with photosynthetically active radiation (PAR). Instead, GEP fluxes largely followed the phenology of canopy photosynthetic capacity (Pc), with only deviations from this primary pattern driven by variations in PAR. Estimates of leaf flush at three

Aprendizaje, simulación y desarrollo sobre las tecnologías de voz y datos

El objetivo del Proyecto Fin de Carrera (PFC) es el de conocer, simular y crear una red VoIP sobre una red de datos en un entorno docente, más concretamente, en la asignatura Redes y Servicios de telecomunicación en Grado en Ingeniería de Telecomunicaciones en la Universidad Politécnica de Madrid (UPM). Una vez se adquieran los conocimientos necesarios, se propondrán una serie de prácticas para que los alumnos se vayan familiarizando con el software y hardware utilizados, de manera que, se irá subiendo el grado de dificultad hasta que puedan realizar una auténtica red VoIP por sí mismos. A parte de la realización de las prácticas, los alumnos deberán pasar una prueba de los conocimientos adquiridos al final de cada práctica mediante preguntas tipo test. Los sistemas elegidos para la implantación de una red VoIP en los módulos de laboratorio son: 3CX System Phone y Asteisk-Trixbox. Los cuales, son capaces de trabajar mediante gestores gráficos para simplificar el nivel de dificultad de la configuración. 3CX es una PBX que trabaja sobre Windows y se basa exclusivamente en el protocolo SIP. Esto facilita el manejo para usuarios que solo han usado Windows sin quitar funcionalidades que tienen otras centralitas en otros sistemas operativos. La versión demo activa todas las opciones para poder familiarizarse con este sistema. Por otro lado, Asterisk trabaja en todas las plataformas, aunque se ha seleccionado trabajar sobre Linux. Esta selección se ha realizado porque el resto de plataformas limitan la configuración de la IP PBX, esta es de código abierto y permite realizar todo tipo de configuraciones. Además, es un software gratuito, esto es una ventaja a la hora de configurar novedades o resolver problemas, ya que hay muchos especialistas que dan soporte y ayudan de forma gratuita. La voz sobre Internet es habitualmente conocida como VoIP (Voice Over IP), debido a que IP (Internet Protocol) es el protocolo de red de Internet. Como tecnología, la VoIP no es solo un paso más en el crecimiento de las comunicaciones por voz, sino que supone integrar las comunicaciones de datos y las de voz en una misma red, y en concreto, en la red con mayor cobertura mundial: Internet. La mayor importancia y motivación de este Proyecto Fin de Carrera es que el alumno sea capaz de llegar a un entorno laboral y pueda tener unos conocimientos capaces de afrontar esta tecnología que esta tan a la orden del día. La importancia que estas redes tienen y tendrán en un futuro muy próximo en el mundo de la informática y las comunicaciones. Cabe decir, que se observa que estas disciplinas tecnológicas evolucionan a pasos agigantados y se requieren conocimientos más sólidos. ABSTRACT. The objective of my final project during my studies in university was, to simulate and create a VoIP network over a data network in a teaching environment, more specifically on the subject of telecommunications networks and services in Telecommunication Engineering Degree in Polytechnic University of Madrid (UPM). Once acquiring the necessary knowledge a number of practices were proposed to the students to become familiar with the software and hardware used, so that it would rise to the level of difficulty that they could make a real VoIP network for themselves. Parts of the experimental practices were that students must pass a test of knowledge acquired at the end of each practice by choice questions. The systems chosen for the implementation of a VoIP network in the laboratory modules are: 3CX Phone System and Asteisk - Trixbox. Which were able to work with graphics operators to simplify the difficulty level of the configuration. 3CX is a PBX that works on Windows and is based solely on the SIP protocol. This facilitates handling for users who have only used Windows without removing functionality with other exchanges in other operating systems. Active demo version all options to get to grips with this system. Moreover, Asterisk works on all platforms, but has been selected to work on Linux. This selection was made because other platforms limit the IP PBX configuration, as this is open source and allows all kinds of configurations. Also, Linux is a free software and an advantage when configuring new or solve problems, as there are many specialists that support and help for free. Voice over Internet is commonly known as VoIP (Voice Over IP), because IP (Internet Protocol) is the Internet protocol network. As technology, VoIP is not just another step in the growth of voice communications, but communications of integrating data and voice on a single network, and in particular, in the network with the largest global coverage: Internet. The increased importance and motivation of this Thesis is that the student is able to reach a working environment and may have some knowledge to deal with these technologies that is so much the order of the day. The importances of these networks have and will be of essences in the very near future in the world of computing and communications. It must be said it is observed that these technological disciplines evolve by leaps and bounds stronger knowledge required.