Dynamic data retrieval on the World Wide Web

Methods for accessing data on the Web have been the focus of active research over the past few years. In this thesis we propose a method for representing Web sites as data sources. We designed a Data Extractor data retrieval solution that allows us to define queries to Web sites and process resulting data sets. Data Extractor is being integrated into the MSemODB heterogeneous database management system. With its help database queries can be distributed over both local and Web data sources within MSemODB framework. ^ Data Extractor treats Web sites as data sources, controlling query execution and data retrieval. It works as an intermediary between the applications and the sites. Data Extractor utilizes a twofold “custom wrapper” approach for information retrieval. Wrappers for the majority of sites are easily built using a powerful and expressive scripting language, while complex cases are processed using Java-based wrappers that utilize specially designed library of data retrieval, parsing and Web access routines. In addition to wrapper development we thoroughly investigate issues associated with Web site selection, analysis and processing. ^ Data Extractor is designed to act as a data retrieval server, as well as an embedded data retrieval solution. We also use it to create mobile agents that are shipped over the Internet to the client's computer to perform data retrieval on behalf of the user. This approach allows Data Extractor to distribute and scale well. ^ This study confirms feasibility of building custom wrappers for Web sites. This approach provides accuracy of data retrieval, and power and flexibility in handling of complex cases. ^

Dynamic data retrieval on the world wide web

Methods for accessing data on the Web have been the focus of active research over the past few years. In this thesis we propose a method for representing Web sites as data sources. We designed a Data Extractor data retrieval solution that allows us to define queries to Web sites and process resulting data sets. Data Extractor is being integrated into the MSemODB heterogeneous database management system. With its help database queries can be distributed over both local and Web data sources within MSemODB framework. Data Extractor treats Web sites as data sources, controlling query execution and data retrieval. It works as an intermediary between the applications and the sites. Data Extractor utilizes a two-fold "custom wrapper" approach for information retrieval. Wrappers for the majority of sites are easily built using a powerful and expressive scripting language, while complex cases are processed using Java-based wrappers that utilize specially designed library of data retrieval, parsing and Web access routines. In addition to wrapper development we thoroughly investigate issues associated with Web site selection, analysis and processing. Data Extractor is designed to act as a data retrieval server, as well as an embedded data retrieval solution. We also use it to create mobile agents that are shipped over the Internet to the client's computer to perform data retrieval on behalf of the user. This approach allows Data Extractor to distribute and scale well. This study confirms feasibility of building custom wrappers for Web sites. This approach provides accuracy of data retrieval, and power and flexibility in handling of complex cases.

Structural damage estimation using dynamic data and multi-objective optimization

Peer reviewed

Dynamic-Data Driven Real-Time Identification for Electric Power Systems

Power system engineers face a double challenge: to operate electric power systems within narrow stability and security margins, and to maintain high reliability. There is an acute need to better understand the dynamic nature of power systems in order to be prepared for critical situations as they arise. Innovative measurement tools, such as phasor measurement units, can capture not only the slow variation of the voltages and currents but also the underlying oscillations in a power system. Such dynamic data accessibility provides us a strong motivation and a useful tool to explore dynamic-data driven applications in power systems. To fulfill this goal, this dissertation focuses on the following three areas: Developing accurate dynamic load models and updating variable parameters based on the measurement data, applying advanced nonlinear filtering concepts and technologies to real-time identification of power system models, and addressing computational issues by implementing the balanced truncation method. By obtaining more realistic system models, together with timely updated parameters and stochastic influence consideration, we can have an accurate portrait of the ongoing phenomena in an electrical power system. Hence we can further improve state estimation, stability analysis and real-time operation.

Spettroscopia di desorbimento termico: progettazione e costruzione di un apparato e primi esperimenti.

L’elaborato descrive la progettazione di un prototipo per misure spettroscopiche di desorbimento termico e i primi esperimenti effettuati con esso. Vengono descritti in dettaglio gli strumenti peculiari di tale apparato, come lo spettrometro di massa quadrupolo e la pompa a diffusione e le parti costruite ad hoc per tale dispositivo, ovvero la struttura del portacampione e del sostegno al forno utilizzato per il riscaldamento delle sostanze analizza- te. Particolare importanza `e posta nella descrizione della parte software del prototipo, che utilizza la tecnologia del DDE (Dynamic Data Exchange) per comunicare i dati tra due programmi diversi operanti su una medesima piattaforma; viene quindi illustrato il funzionamento del software comunicante direttamente con lo spettrometro e del programma LabView creato per il monitoraggio e il salvataggio dei dati raccolti da tale apparato. L’ultima parte dell’elaborato riguarda i primi esperimenti di spettroscopia di desorbimento termico effettuati, comprendendo sia quelli preliminari per testare la qualità del prototipo sia quelli da cui `e possibile ottenere una curva di desorbimento termico per i vari gas analizzati in camera, come ad esempio idrogeno.

A low-cost, easy-to-use, real-time power system simulator

We present the design rationale and basic workings of a low-cost, easy-to-use power system simulator developed to support investigations into human interface design for a hydropower plant. The power system simulator is based on three important components: models of power system components, a data repository, and human interface elements. Dynamic Data Exchange (DDE) allows simulator components to communicate with each other within the simulator. To construct the modules of the simulator we have combined the advantages of commercial software such as Matlab/Simulink, ActiveX Control, Visual Basic and Excel and integrated them in the simulator. An important advantage of our approach is that further components of the simulator now can be developed independently. An initial assessment of the simulator indicates it is fit for intended purpose.

Exchange of Product-related Data between a Pulp and Paper Machinery Supplier and Its Partners

Kiihtyvä kilpailu yritysten välillä on tuonut yritykset vaikeidenhaasteiden eteen. Tuotteet pitäisi saada markkinoille nopeammin, uusien tuotteiden pitäisi olla parempia kuin vanhojen ja etenkin parempia kuin kilpailijoiden vastaavat tuotteet. Lisäksi tuotteiden suunnittelu-, valmistus- ja muut kustannukset eivät saisi olla suuria. Näiden haasteiden toteuttamisessa yritetään usein käyttää apuna tuotetietoja, niiden hallintaa ja vaihtamista. Andritzin, kuten muidenkin yritysten, on otettava nämä asiat huomioon pärjätäkseen kilpailussa. Tämä työ on tehty Andritzille, joka on maailman johtavia paperin ja sellun valmistukseen tarkoitettujen laitteiden valmistajia ja huoltopalveluiden tarjoajia. Andritz on ottamassa käyttöön ERP-järjestelmän kaikissa toimipisteissään. Sitä halutaan hyödyntää mahdollisimman tehokkaasti, joten myös tuotetiedot halutaan järjestelmään koko elinkaaren ajalta. Osan tuotetiedoista luo Andritzin kumppanit ja alihankkijat, joten myös tietojen vaihto partnereiden välillä halutaan hoitaasiten, että tiedot saadaan suoraan ERP-järjestelmään. Tämän työn tavoitteena onkin löytää ratkaisu, jonka avulla Andritzin ja sen kumppaneiden välinen tietojenvaihto voidaan hoitaa. Tämä diplomityö esittelee tuotetietojen, niiden hallinnan ja vaihtamisen tarkoituksen ja tärkeyden. Työssä esitellään erilaisia ratkaisuvaihtoehtoja tiedonvaihtojärjestelmän toteuttamiseksi. Osa niistä perustuu yleisiin ja toimialakohtaisiin standardeihin. Myös kaksi kaupallista tuotetta esitellään. Tarkasteltavana onseuraavat standardit: PaperIXI, papiNet, X-OSCO, PSK-standardit sekä RosettaNet. Lisäksi työssä tarkastellaan ERP-järjestelmän toimittajan, SAP:in ratkaisuja tietojenvaihtoon. Näistä vaihtoehdoista parhaimpia tarkastellaan vielä yksityiskohtaisemmin ja lopuksi eri ratkaisuja vertaillaan keskenään, jotta löydettäisiin Andritzin tarpeisiin paras vaihtoehto.

Optimizing the Dynamic Distribution of Data-stream for High Speed Communications

The performances of high-speed network communications frequently rest with the distribution of data-stream. In this paper, a dynamic data-stream balancing architecture based on link information is introduced and discussed firstly. Then the algorithms for simultaneously acquiring the passing nodes and links of a path between any two source-destination nodes rapidly, as well as a dynamic data-stream distribution planning are proposed. Some related topics such as data fragment disposal, fair service, etc. are further studied and discussed. Besides, the performance and efficiency of proposed algorithms, especially for fair service and convergence, are evaluated through a demonstration with regard to the rate of bandwidth utilization. Hoping the discussion presented here can be helpful to application developers in selecting an effective strategy for planning the distribution of data-stream.

A 802.11p prototype implementation

This paper presents an IEEE 802.11p full-stack prototype implementation to data exchange among vehicles and between vehicles and the roadway infrastructures. The prototype architecture is based on FPGAs for Intermediate Frequency (IF) and base band purposes, using 802.11a based transceivers for RF interfaces. Power amplifiers were also addressed, by using commercial and in-house solutions. This implementation aims to provide technical solutions for Intelligent Transportation Systems (ITS) field, namely for tolling and traffic management related services, in order to promote safety, mobility and driving comfort through the dynamic and real-time cooperation among vehicles and/or between vehicles and infrastructures. The performance of the proposed scheme is tested under realistic urban and suburban driving conditions. Preliminary results are promising, since they comply with most of the 802.11p standard requirements.

Consistency and Context Management in a Multi Agent Belief Revision Testbed

Multi-agent architectures are well suited for complex inherently distributed problem solving domains. From the many challenging aspects that arise within this framework, a crucial one emerges: how to incorporate dynamic and conflicting agent beliefs? While the belief revision activity in a single agent scenario is concentrated on incorporating new information while preserving consistency, in a multi-agent system it also has to deal with possible conflicts between the agents perspectives. To provide an adequate framework, each agent, built as a combination of an assumption based belief revision system and a cooperation layer, was enriched with additional features: a distributed search control mechanism allowing dynamic context management, and a set of different distributed consistency methodologies. As a result, a Distributed Belief Revision Testbed (DiBeRT) was developed. This paper is a preliminary report presenting some of DiBeRT contributions: a concise representation of external beliefs; a simple and innovative methodology to achieve distributed context management; and a reduced inter-agent data exchange format.

Sharing Open Source License and Copyright Data with SPDX

Organizations across the globe are creating and distributing products that include open source software. To ensure compliance with the open source licenses, each company needs to evaluate exactly what open source licenses and copyrights are included - resulting in duplicated effort and redundancy. This talk will provide an overview of a new Software Package Data Exchange (SPDX) specification. This specification will provide a common format to share information about the open source licenses and copyrights that are included in any software package, with the goal of saving time and improving data accuracy. This talk will review the progress of the initiative; discuss the benefits to organizations using open source and share information on how you can contribute.

Tiedonvaihto - ja tietojärjestelmäyhteentoimivuus kahden sotilaallisen koalition välillä

Tämä diplomityö käsittelee kahden sotilaallisen koalition yhteentoimivuuden haasteita ja toteutusta kaupallisintyökaluin. Työ pohjautuu kahteen todelliseen, palvelusuuntautunutta arkkitehtuurisuunnittelua (SOA) hyödyntäneeseen integraatioprojektiin jotka on toteutettu Suomen IBM:n Palveluyksikössä vuosina 2006 - 2007. Työn tavoitteena on ollut tutkia sotilaallisten koalitioiden järjestelmä - ja tiedonvaihtoyhteentoimivuuden menetelmiä, näkökulmia ja teknistä toteutusta kaupallisin ohjelmistotuottein ja yhteistä tietomallia käyttäen. Lisäksi esitetään puolustustoimialan erityispiirteet tietojärjestelmätoimittajien ohjelmistokehitys - prosesseihin liittyen. Tätä varten tutkittiin koalitioiden käyttöön tarkoitettuja olemassa olevia ohjelmistoarkkitehtuureja ja yhteentoimivuusmalleja sekä sovitettiin niitä SOA - arkkitehtuuriajatteluun. Työn teoreettisena pohjana käytettiin organisatorisen ja teknisen yhteentoimivuuden kuvaavaa Layers of Coalition Interoperability (LCI) - mallia, minkä jälkeen mallin teknistä osiota käytettiin pohjana SOA - palveluihin perustuvan esimerkkijärjestelmän kehittämiseen kahden kuvitteellisen koalition tiedonvaihtoa varten. Työn keskeisinä tuloksina on syntynyt suunnitelma koalitioiden taustajärjestelmien yhdistämisestä dynaamisten SOA- palveluiden avulla yhteiseen JC3IEDM - tietomalliin. Tämä tietomalli vuorostaan antaa mahdollisuuden järjestelmän laajentamiseen esimerkiksi avustusjärjestöjen, poliisivoimien ja terveydenhuollon organisaatioiden tarpeisiin.

Macroeconomic News and Exchange Rates: Evidence of Unstable Effect

Although the link between macroeconomic news announcements and exchange rates is well documented in recent literature, this connection may be unstable. By using a broad set of macroeconomic news announcements and high frequency forex data for the Euro/Dollar, Pound/Dollar and Yen/Dollar from Nov 1, 2004 to Mar 31, 2014, we obtain two major findings with regards to this instability. First, many macroeconomic news announcements exhibit unstable effects with certain patterns in foreign exchange rates. These news effects may change in magnitude and even in their sign over time, over business cycles and crises within distinctive contexts. This finding is robust because the results are obtained by applying a Two-Regime Smooth Transition Regression Model, a Breakpoints Regression Model, and an Efficient Test of Parameter Instability which are all consistent with each other. Second, when we explore the source of this instability, we find that global risks and the reaction by central bank monetary policy to these risks to be possible factors causing this instability.

Ocean data dissemination: new challenges for data integration

In the decade since OceanObs `99, great advances have been made in the field of ocean data dissemination. The use of Internet technologies has transformed the landscape: users can now find, evaluate and access data rapidly and securely using only a web browser. This paper describes the current state of the art in dissemination methods for ocean data, focussing particularly on ocean observations from in situ and remote sensing platforms. We discuss current efforts being made to improve the consistency of delivered data and to increase the potential for automated integration of diverse datasets. An important recent development is the adoption of open standards from the Geographic Information Systems community; we discuss the current impact of these new technologies and their future potential. We conclude that new approaches will indeed be necessary to exchange data more effectively and forge links between communities, but these approaches must be evaluated critically through practical tests, and existing ocean data exchange technologies must be used to their best advantage. Investment in key technology components, cross-community pilot projects and the enhancement of end-user software tools will be required in order to assess and demonstrate the value of any new technology.

Semantic integration of gene expression analysis tools and data sources using software connectors

Abstract Background The study and analysis of gene expression measurements is the primary focus of functional genomics. Once expression data is available, biologists are faced with the task of extracting (new) knowledge associated to the underlying biological phenomenon. Most often, in order to perform this task, biologists execute a number of analysis activities on the available gene expression dataset rather than a single analysis activity. The integration of heteregeneous tools and data sources to create an integrated analysis environment represents a challenging and error-prone task. Semantic integration enables the assignment of unambiguous meanings to data shared among different applications in an integrated environment, allowing the exchange of data in a semantically consistent and meaningful way. This work aims at developing an ontology-based methodology for the semantic integration of gene expression analysis tools and data sources. The proposed methodology relies on software connectors to support not only the access to heterogeneous data sources but also the definition of transformation rules on exchanged data. Results We have studied the different challenges involved in the integration of computer systems and the role software connectors play in this task. We have also studied a number of gene expression technologies, analysis tools and related ontologies in order to devise basic integration scenarios and propose a reference ontology for the gene expression domain. Then, we have defined a number of activities and associated guidelines to prescribe how the development of connectors should be carried out. Finally, we have applied the proposed methodology in the construction of three different integration scenarios involving the use of different tools for the analysis of different types of gene expression data. Conclusions The proposed methodology facilitates the development of connectors capable of semantically integrating different gene expression analysis tools and data sources. The methodology can be used in the development of connectors supporting both simple and nontrivial processing requirements, thus assuring accurate data exchange and information interpretation from exchanged data.