Pragmatic development of service based real-time change data capture

This thesis makes a contribution to the Change Data Capture (CDC) field by providing an empirical evaluation on the performance of CDC architectures in the context of realtime data warehousing. CDC is a mechanism for providing data warehouse architectures with fresh data from Online Transaction Processing (OLTP) databases. There are two types of CDC architectures, pull architectures and push architectures. There is exiguous data on the performance of CDC architectures in a real-time environment. Performance data is required to determine the real-time viability of the two architectures. We propose that push CDC architectures are optimal for real-time CDC. However, push CDC architectures are seldom implemented because they are highly intrusive towards existing systems and arduous to maintain. As part of our contribution, we pragmatically develop a service based push CDC solution, which addresses the issues of intrusiveness and maintainability. Our solution uses Data Access Services (DAS) to decouple CDC logic from the applications. A requirement for the DAS is to place minimal overhead on a transaction in an OLTP environment. We synthesize DAS literature and pragmatically develop DAS that eciently execute transactions in an OLTP environment. Essentially we develop effeicient RESTful DAS, which expose Transactions As A Resource (TAAR). We evaluate the TAAR solution and three pull CDC mechanisms in a real-time environment, using the industry recognised TPC-C benchmark. The optimal CDC mechanism in a real-time environment, will capture change data with minimal latency and will have a negligible affect on the database's transactional throughput. Capture latency is the time it takes a CDC mechanism to capture a data change that has been applied to an OLTP database. A standard definition for capture latency and how to measure it does not exist in the field. We create this definition and extend the TPC-C benchmark to make the capture latency measurement. The results from our evaluation show that pull CDC is capable of real-time CDC at low levels of user concurrency. However, as the level of user concurrency scales upwards, pull CDC has a significant impact on the database's transaction rate, which affirms the theory that pull CDC architectures are not viable in a real-time architecture. TAAR CDC on the other hand is capable of real-time CDC, and places a minimal overhead on the transaction rate, although this performance is at the expense of CPU resources.

True real-time change data capture with web service database encapsulation

This research is investigating the claim that Change Data Capture (CDC) technologies capture data changes in real-time. Based on theory, our hypothesis states that real-time CDC is not achievable with traditional approaches (log scanning, triggers and timestamps). Traditional approaches to CDC require a resource to be polled, which prevents true real-time CDC. We propose an approach to CDC that encapsulates the data source with a set of web services. These web services will propagate the changes to the targets and eliminate the need for polling. Additionally we propose a framework for CDC technologies that allow changes to flow from source to target. This paper discusses current CDC technologies and presents the theory about why they are unable to deliver changes in real-time. Following, we discuss our web service approach to CDC and accompanying framework, explaining how they can produce real-time CDC. The paper concludes with a discussion on the research required to investigate the real-time capabilities of CDC technologies. © 2010 IEEE.

WordPress as a framework for automated data capture, filtering and structuring processes

The Exhibitium Project , awarded by the BBVA Foundation, is a data-driven project developed by an international consortium of research groups . One of its main objectives is to build a prototype that will serve as a base to produce a platform for the recording and exploitation of data about art-exhibitions available on the Internet . Therefore, our proposal aims to expose the methods, procedures and decision-making processes that have governed the technological implementation of this prototype, especially with regard to the reuse of WordPress (WP) as development framework.

Analysis of full-waveform LiDAR data for classification of an orange orchard scene

Full-waveform laser scanning data acquired with a Riegl LMS-Q560 instrument were used to classify an orange orchard into orange trees, grass and ground using waveform parameters alone. Gaussian decomposition was performed on this data capture from the National Airborne Field Experiment in November 2006 using a custom peak-detection procedure and a trust-region-reflective algorithm for fitting Gauss functions. Calibration was carried out using waveforms returned from a road surface, and the backscattering coefficient c was derived for every waveform peak. The processed data were then analysed according to the number of returns detected within each waveform and classified into three classes based on pulse width and c. For single-peak waveforms the scatterplot of c versus pulse width was used to distinguish between ground, grass and orange trees. In the case of multiple returns, the relationship between first (or first plus middle) and last return c values was used to separate ground from other targets. Refinement of this classification, and further sub-classification into grass and orange trees was performed using the c versus pulse width scatterplots of last returns. In all cases the separation was carried out using a decision tree with empirical relationships between the waveform parameters. Ground points were successfully separated from orange tree points. The most difficult class to separate and verify was grass, but those points in general corresponded well with the grass areas identified in the aerial photography. The overall accuracy reached 91%, using photography and relative elevation as ground truth. The overall accuracy for two classes, orange tree and combined class of grass and ground, yielded 95%. Finally, the backscattering coefficient c of single-peak waveforms was also used to derive reflectance values of the three classes. The reflectance of the orange tree class (0.31) and ground class (0.60) are consistent with published values at the wavelength of the Riegl scanner (1550 nm). The grass class reflectance (0.46) falls in between the other two classes as might be expected, as this class has a mixture of the contributions of both vegetation and ground reflectance properties.

Integrating a mobile accessible electronic system into dockside monitoring: How can small-scale fisheries data collection programs transition from paper-based to digital data collection?

Thesis (Master's)--University of Washington, 2016-06

Myoelectric Signal Monitoring System

The Electromyography (EMG) is an important tool for gait analyzes and disorders diagnoses. Traditional methods involve equipment that can disturb the analyses, being gradually substituted by different approaches, like wearable and wireless systems. The cable replacement for autonomous systems demands for technologies capable of meeting the power constraints. This work presents the development of an EMG and kinematic data capture wireless module, designed taking into account power consumption issues. This module captures and converts the analog myoeletric signal to digital, synchronously with the capture of kinetic information. Both data are time multiplexed and sent to a PC via Bluetooth link. The work carried out comprised the development of the hardware, the firmware and a graphical interface running in an external PC. The hardware was developed using the PIC18F14K22, a low power family of microcontrollers. The link was established via Bluetooth, a protocol designed for low power communication. An application was also developed to recover and trace the signal to a Graphic User Interface (GUI), coordinating the message exchange with the firmware. Results were obtained which allowed validating the conceived system in static and with the subject performing short movements. Although it was not possible to perform the tests within more dynamic movements, it is shown that it is possible to capture, transmit and display the captured data as expected. Some suggestions to improve the system performance also were made.

Public Participation geographic information systems in developing countries: lessons from Uganda on the impacts of interactive screens PPGIS and community perceptions of space

Dissertation submitted in partial fulfillment of the requirements for the Degree of Master of Science in Geospatial Technologies.

Indicadores de qualidade na emergência médica pré-hospitalar : painel operacional para o INEM, I.P.

RESUMO - A Emergência Médica Pré-Hospitalar é uma parte importante na cadeia de cuidados de saúde, quer pela gravidade das situações a que atende, quer por funcionar como gatekeeper. O estudo é dedicado a instrumentos de medição e em simultâneo a medições de qualidade como processo de melhoria contínua da qualidade na EMPH. Existem projetos a nível internacional para implementação de painéis de indicadores, alguns atribuem significativa relevância aos processos assistenciais. O European Emergency Data Project (EDD) foi selecionado para servir como painel de indicadores a explorar neste trabalho. Porque a nível científico está bem desenhado, as definições e variáveis estão bem explicitadas e interessa ao INEM, I.P. utilizar um painel que lhe permita um benchmarking. Criou-se um dashboard para os indicadores selecionados e com ele analisaram-se os dados para aferir a qualidade do SEM. Não foi possível, pela sua inexistência, coletar dados para dois dos indicadores propostos. Foram apresentadas propostas de melhoria no sistema de captação de dados do INEM, I.P. para que a instituição possa monitorizar periodicamente os indicadores selecionados. A existência de dados e a sua fiabilidade para utilizar na medição por indicadores é algo que se explorou neste trabalho de campo, e as fraquezas encontradas foram matéria para oportunidades, recomendações, e caminho para discussão e o fortalecimento da instituição.

A relational algebra approach to ETL modeling

The MAP-i Doctoral Programme in Informatics, of the Universities of Minho, Aveiro and Porto

Influence of the hip joint modeling approaches on the kinematics of human gait

The influence of the hip joint formulation on the kinematic response of the model of human gait is investigated throughout this work. To accomplish this goal, the fundamental issues of the modeling process of a planar hip joint under the framework of multibody systems are revisited. In particular, the formulations for the ideal, dry, and lubricated revolute joints are described and utilized for the interaction of femur head inside acetabulum or the hip bone. In this process, the main kinematic and dynamic aspects of hip joints are analyzed. In a simple manner, the forces that are generated during human gait, for both dry and lubricated hip joint models, are computed in terms of the system’s state variables and subsequently introduced into the dynamics equations of motion of the multibody system as external generalized forces. Moreover, a human multibody model is considered, which incorporates the different approaches for the hip articulation, namely ideal joint, dry, and lubricated models. Finally, several computational simulations based on different approaches are performed, and the main results presented and compared to identify differences among the methodologies and procedures adopted in this work. The input conditions to the models correspond to the experimental data capture from an adult male during normal gait. In general, the obtained results in terms of positions do not differ significantly when the different hip joint models are considered. In sharp contrast, the velocity and acceleration plotted vary significantly. The effect of the hip joint modeling approach is clearly measurable and visible in terms of peaks and oscillations of the velocities and accelerations. In general, with the dry hip model, intra-joint force peaks can be observed, which can be associated with the multiple impacts between the femur head and the cup. In turn, when the lubricant is present, the system’s response tends to be smoother due to the damping effects of the synovial fluid.

EORTC Radiation Oncology Group quality assurance platform: establishment of a digital central review facility.

OBJECTIVE: Quality assurance (QA) in clinical trials is essential to ensure treatment is safely and effectively delivered. As QA requirements have increased in complexity in parallel with evolution of radiation therapy (RT) delivery, a need to facilitate digital data exchange emerged. Our objective is to present the platform developed for the integration and standardization of QART activities across all EORTC trials involving RT. METHODS: The following essential requirements were identified: secure and easy access without on-site software installation; integration within the existing EORTC clinical remote data capture system; and the ability to both customize the platform to specific studies and adapt to future needs. After retrospective testing within several clinical trials, the platform was introduced in phases to participating sites and QART study reviewers. RESULTS: The resulting QA platform, integrating RT analysis software installed at EORTC Headquarters, permits timely, secure, and fully digital central DICOM-RT based data review. Participating sites submit data through a standard secure upload webpage. Supplemental information is submitted in parallel through web-based forms. An internal quality check by the QART office verifies data consistency, formatting, and anonymization. QART reviewers have remote access through a terminal server. Reviewers evaluate submissions for protocol compliance through an online evaluation matrix. Comments are collected by the coordinating centre and institutions are informed of the results. CONCLUSIONS: This web-based central review platform facilitates rapid, extensive, and prospective QART review. This reduces the risk that trial outcomes are compromised through inadequate radiotherapy and facilitates correlation of results with clinical outcomes.

Time to Move On from Congregated Settings: A Strategy for Community Inclusion

19.6.2011 I am pleased to present the Report of the Working Group on Congregated Settings, which is the outcome and culmination of a very significant piece of data capture, research and analysis. The Report was initiated by the Primary, Community and Community Care Directorate in 2007 to develop a national plan and associated change programme for moving people from congregated settings to the community in line with Government policy. Click here to download the document

International ADHD in substance use disorders prevalence study.

Dr Van Hout has been invited by the ICASA network and IASP research team [Drs Geurt van de Glind; Trimbos Institute, The Netherlands; Dr Pieter-Jan Carpentier, ICASA; Josep Antoni Ramos-Quiroga, University of Barcelona, Spain, Professor Dr Frances Levin, University of Columbia, New York, USA and Professor Dr. Wim van den Brink, University of Amsterdam, The Netherlands] to undertake the research protocol for Ireland as part of this European study of the prevalence of ADHD in adult patients referred for treatment of addiction problems. The research team at Waterford Institute of Technology, School of Health Sciences will undertake this national study as part of the International Collaboration on ADHD and Substance Abuse [ICASA] â?~International ADHD in Substance Use Disorders Prevalence Studyâ?T [IASP study]. The International Collaboration on ADHD and Substance Abuse [ICASA] will provide Dr Van Hout and her team with full support from ICASA of the measurement instruments available and a central database at the University of Amsterdam, and will undergo training for procedures for data capture from Dr van de Glind, Trimbos Institute, The Netherlands. Eight European countries (Norway, Sweden, the Netherlands, Belgium, France, Spain, Switzerland and Hungary) USA and Australia have already participated in the first phase of the IASP study, which will close in September 2011. Over 2500 Substance Use Disorder [SUD] patients were sampled with approximately 38% scoring positive on the ADHD screener (ASRS). Of these 2500 patients over 1000 patients were evaluated on ADHD, Depression, Bipolar Disorder, Anti-Social Personality and Borderline Personality Disorder. A preliminary estimate of the prevalence of ADHD in SUD treatment seeking patients was recorded at 20 %. The second phase of study [IASP 2011] will commence in September 2011 for countries including Ireland, South Africa, Egypt and Brazil. Dr Van Hout has also been invited to partake in a systematic review paper on the risk factors for development of SUD in children/adolescents with ADHD in collaboration with the ICASA foundation.This resource was contributed by The National Documentation Centre on Drug Use.

A Comparison of Mobile Scanning to a Total Station Survey at the I-35 and IA 92 Interchange in Warren County, Iowa, August 15, 2012

The purpose of this project was to investigate the potential for collecting and using data from mobile terrestrial laser scanning (MTLS) technology that would reduce the need for traditional survey methods for the development of highway improvement projects at the Iowa Department of Transportation (Iowa DOT). The primary interest in investigating mobile scanning technology is to minimize the exposure of field surveyors to dangerous high volume traffic situations. Issues investigated were cost, timeframe, accuracy, contracting specifications, data capture extents, data extraction capabilities and data storage issues associated with mobile scanning. The project area selected for evaluation was the I-35/IA 92 interchange in Warren County, Iowa. This project covers approximately one mile of I-35, one mile of IA 92, 4 interchange ramps, and bridges within these limits. Delivered LAS and image files for this project totaled almost 31GB. There is nearly a 6-fold increase in the size of the scan data after post-processing. Camera data, when enabled, produced approximately 900MB of imagery data per mile using a 2- camera, 5 megapixel system. A comparison was done between 1823 points on the pavement that were surveyed by Iowa DOT staff using a total station and the same points generated through the MTLS process. The data acquired through the MTLS and data processing met the Iowa DOT specifications for engineering survey. A list of benefits and challenges is included in the detailed report. With the success of this project, it is anticipate[d] that additional projects will be scanned for the Iowa DOT for use in the development of highway improvement projects.