Big Data Processing in the Cloud: a Hydra/Sufia Experience

Presentation at Open Repositories 2014, Helsinki, Finland, June 9-13, 2014

Spatial and temporal variation in fish populations and assemblages in coastal waters of the northern Baltic Proper

The distribution and traits of fish are of interest both ecologically and socio-economically. In this thesis, phenotypic and structural variation in fish populations and assemblages was studied on multiple spatial and temporal scales in shallow coastal areas in the archipelago of the northern Baltic Proper. In Lumparn basin in Åland Islands, the fish assemblage displayed significant seasonal variation in depth zone distribution. The results indicate that investigating both spatial and temporal variation in small scale is crucial for understanding patterns in fish distribution and community structure in large scale. The local population of Eurasian perch Perca fluviatilis L displayed habitat-specific morphological and dietary variation. Perch in the pelagic zone were on average deeper in their body shape than the littoral ones and fed on fish and benthic invertebrates. The results differ from previous studies conducted in freshwater habitats, where the pelagic perch typically are streamlined in body shape and zooplanktivorous. Stable isotopes of carbon and nitrogen differed between perch with different stomach contents, suggesting differentiation of individual diet preferences. In the study areas Lumparn and Ivarskärsfjärden in Åland Islands and Galtfjärden in Swedish east coast, the development in fish assemblages during the 2000’s indicated a general shift towards higher abundances of small-bodied lower-order consumers, especially cyprinids. For European pikeperch Sander lucioperca L., recent declines in adult fish abundances and high mortalities (Z = 1.06–1.16) were observed, which suggests unsustainably high fishing pressure on pikeperch. Based on the results it can be hypothesized that fishing has reduced the abundances of large predatory fish, which together with bottom-up forcing by eutrophication has allowed the lower-order consumer species to increase in abundances. This thesis contributes to the scientific understanding of aquatic ecosystems with new descriptions on morphological and dietary adaptations in perch in brackish water, and on the seasonal variation in small-scale spatial fish distribution. The results also demonstrate anthropogenic effects on coastal fish communities and underline the urgency of further reducing nutrient inputs and regulating fisheries in the Baltic Sea region.

Online data processing consent under EU law: a theoretical framework and empirical evidence from the UK

This article analyses the results of an empirical study on the 200 most popular UK-based websites in various sectors of e-commerce services. The study provides empirical evidence on unlawful processing of personal data. It comprises a survey on the methods used to seek and obtain consent to process personal data for direct marketing and advertisement, and a test on the frequency of unsolicited commercial emails (UCE) received by customers as a consequence of their registration and submission of personal information to a website. Part One of the article presents a conceptual and normative account of data protection, with a discussion of the ethical values on which EU data protection law is grounded and an outline of the elements that must be in place to seek and obtain valid consent to process personal data. Part Two discusses the outcomes of the empirical study, which unveils a significant departure between EU legal theory and practice in data protection. Although a wide majority of the websites in the sample (69%) has in place a system to ask separate consent for engaging in marketing activities, it is only 16.2% of them that obtain a consent which is valid under the standards set by EU law. The test with UCE shows that only one out of three websites (30.5%) respects the will of the data subject not to receive commercial communications. It also shows that, when submitting personal data in online transactions, there is a high probability (50%) of incurring in a website that will ignore the refusal of consent and will send UCE. The article concludes that there is severe lack of compliance of UK online service providers with essential requirements of data protection law. In this respect, it suggests that there is inappropriate standard of implementation, information and supervision by the UK authorities, especially in light of the clarifications provided at EU level.

Error, reproducibility and sensitivity: a pipeline for data processing of Agilent oligonucleotide expression arrays

Background: Expression microarrays are increasingly used to obtain large scale transcriptomic information on a wide range of biological samples. Nevertheless, there is still much debate on the best ways to process data, to design experiments and analyse the output. Furthermore, many of the more sophisticated mathematical approaches to data analysis in the literature remain inaccessible to much of the biological research community. In this study we examine ways of extracting and analysing a large data set obtained using the Agilent long oligonucleotide transcriptomics platform, applied to a set of human macrophage and dendritic cell samples. Results: We describe and validate a series of data extraction, transformation and normalisation steps which are implemented via a new R function. Analysis of replicate normalised reference data demonstrate that intrarray variability is small (only around 2 of the mean log signal), while interarray variability from replicate array measurements has a standard deviation (SD) of around 0.5 log(2) units (6 of mean). The common practise of working with ratios of Cy5/Cy3 signal offers little further improvement in terms of reducing error. Comparison to expression data obtained using Arabidopsis samples demonstrates that the large number of genes in each sample showing a low level of transcription reflect the real complexity of the cellular transcriptome. Multidimensional scaling is used to show that the processed data identifies an underlying structure which reflect some of the key biological variables which define the data set. This structure is robust, allowing reliable comparison of samples collected over a number of years and collected by a variety of operators. Conclusions: This study outlines a robust and easily implemented pipeline for extracting, transforming normalising and visualising transcriptomic array data from Agilent expression platform. The analysis is used to obtain quantitative estimates of the SD arising from experimental (non biological) intra- and interarray variability, and for a lower threshold for determining whether an individual gene is expressed. The study provides a reliable basis for further more extensive studies of the systems biology of eukaryotic cells.

Data Processing Student Working

A student from the Data Processing program at the New York Trade School is shown working. Black and white photograph with some edge damage due to writing in black along the top.

Felice Gigante, Data Processing Customer Engineer

Felice Gigante a graduate from the New York Trade School Electronics program works on a machine in his job as Data Processing Customer Engineer for the International Business Machines Corp. Original caption reads, "Felice Gigante - Electronices, International Business Machines Corp." Black and white photograph with caption glued to reverse.

Road network and GPS tracking with data processing and quality assessment

GPS technology has been embedded into portable, low-cost electronic devices nowadays to track the movements of mobile objects. This implication has greatly impacted the transportation field by creating a novel and rich source of traffic data on the road network. Although the promise offered by GPS devices to overcome problems like underreporting, respondent fatigue, inaccuracies and other human errors in data collection is significant; the technology is still relatively new that it raises many issues for potential users. These issues tend to revolve around the following areas: reliability, data processing and the related application. This thesis aims to study the GPS tracking form the methodological, technical and practical aspects. It first evaluates the reliability of GPS based traffic data based on data from an experiment containing three different traffic modes (car, bike and bus) traveling along the road network. It then outline the general procedure for processing GPS tracking data and discuss related issues that are uncovered by using real-world GPS tracking data of 316 cars. Thirdly, it investigates the influence of road network density in finding optimal location for enhancing travel efficiency and decreasing travel cost. The results show that the geographical positioning is reliable. Velocity is slightly underestimated, whereas altitude measurements are unreliable.Post processing techniques with auxiliary information is found necessary and important when solving the inaccuracy of GPS data. The densities of the road network influence the finding of optimal locations. The influence will stabilize at a certain level and do not deteriorate when the node density is higher.

Fish ladders select fish traits on migration - still a growing problem for natural fish populations

We investigated the potential selective effect of fish ladders on physiological and morphological profiles of the curimbata, Prochilodus lineatus, during reproductive migration in Brazil. We registered sex, body weight and length, plasma glucose, hepatosomatic and gonadosomatic indices (HSI and GSI, respectively), hematocrit, leucocrit, blood cell and nucleus areas, and the diameter of white and red muscle fibers in fish sampled at the bottom (downstream) and at the top (upstream) of a fish ladder at a hydroelectric dam. Males and females at the top of the ladder showed higher size (weight and length), white muscle fiber diameters, plasma glucose levels and lower hematocrit when compared with those at the bottom. These size and muscle traits assist fish to overcome the ladder barrier and bypass the dam, an effort that might be reflected in the glucose levels. Females also showed higher GSI at the top of the fish ladder, a trait possibly facilitating their reproduction upstream. These results indicate that a dam system favors fish with specific morphological-physiological profile. This may have a strong influence upon upstream fish populations over generations and implies the presence of artificial selective pressure.

CMS data processing workflows during an extended cosmic ray run

The CMS Collaboration conducted a month-long data taking exercise, the Cosmic Run At Four Tesla, during October-November 2008, with the goal of commissioning the experiment for extended operation. With all installed detector systems participating, CMS recorded 270 million cosmic ray events with the solenoid at a magnetic field strength of 3.8 T. This paper describes the data flow from the detector through the various online and offline computing systems, as well as the workflows used for recording the data, for aligning and calibrating the detector, and for analysis of the data. © 2010 IOP Publishing Ltd and SISSA.

The impact of adding SBAS data on GPS data processing in southeast of Brazil: Preliminary result

Nowadays, L1 SBAS signals can be used in a combined GPS+SBAS data processing. However, such situation restricts the studies over short baselines. Besides of increasing the satellite availability, SBAS satellites orbit configuration is different from that of GPS. In order to analyze how these characteristics can impact GPS positioning in the southeast area of Brazil, experiments involving GPS-only and combined GPS+SBAS data were performed. Solutions using single point and relative positioning were computed to show the impact over satellite geometry, positioning accuracy and short baseline ambiguity resolution. Results showed that the inclusion of SBAS satellites can improve the accuracy of positioning. Nevertheless, the bad quality of the data broadcasted by these satellites limits their usage. © Springer-Verlag Berlin Heidelberg 2012.

Latin America and Caribbean experiences in electronic statistical data processing

Includes bibliography

Preliminary draft of the sections of the CELADE 1980-83 proposal to UNFPA on subprogramme 6: documentation and data processing (INFOPAL, DOCPAL) (for discussion only; not to be quoted)

El Sub-Programa de Documentacion en Poblacion y Procesamiento de Datos del CELADE se compone de tres elementos que corresponden a los primeros objetivos inmediatos del mismo: el Sistema de Documentacion en Poblacion para America Latina (DOCPAL), el Banco de Datos y la Unidad de Procesamiento de Informacion. El presente documento describe los objetivos, antecedentes y justificacion de cada uno de estos tres nucleos, asi como las actividades contempladas para el periodo 1980-1983 y que fundamentan el presupuesto contenido en la solicitud de fondos al UNFPA