Investigation of petabyte-scale data transfer performances with PhEDEx for the CMS experiment

PhEDEx, the CMS transfer management system, during the first LHC Run has moved about 150 PB and currently it is moving about 2.5 PB of data per week over the Worldwide LHC Computing Grid (WLGC). It was designed to complete each transfer required by users at the expense of the waiting time necessary for its completion. For this reason, after several years of operations, data regarding transfer latencies has been collected and stored into log files containing useful analyzable informations. Then, starting from the analysis of several typical CMS transfer workflows, a categorization of such latencies has been made with a focus on the different factors that contribute to the transfer completion time. The analysis presented in this thesis will provide the necessary information for equipping PhEDEx in the future with a set of new tools in order to proactively identify and fix any latency issues. PhEDEx, il sistema di gestione dei trasferimenti di CMS, durante il primo Run di LHC ha trasferito all’incirca 150 PB ed attualmente trasferisce circa 2.5 PB di dati alla settimana attraverso la Worldwide LHC Computing Grid (WLCG). Questo sistema è stato progettato per completare ogni trasferimento richiesto dall’utente a spese del tempo necessario per il suo completamento. Dopo svariati anni di operazioni con tale strumento, sono stati raccolti dati relativi alle latenze di trasferimento ed immagazzinati in log files contenenti informazioni utili per l’analisi. A questo punto, partendo dall’analisi di una ampia mole di trasferimenti in CMS, è stata effettuata una suddivisione di queste latenze ponendo particolare attenzione nei confronti dei fattori che contribuiscono al tempo di completamento del trasferimento. L’analisi presentata in questa tesi permetterà di equipaggiare PhEDEx con un insieme di utili strumenti in modo tale da identificare proattivamente queste latenze e adottare le opportune tattiche per minimizzare l’impatto sugli utenti finali.

Tecnologie per la fruizione e rappresentazione di open data

La presente tesi è uno studio sugli strumenti e le tecnologie che caratterizzano l'utilizzo degli open data, in particolare, nello sviluppo di applicazioni web moderne che fanno uso di questo tipo di dati.

Big data, nosql e machine learning: un'applicazione di prediction e recommendation basata sulle api di Amazon

In questa tesi sono stati introdotti e studiati i Big Data, dando particolare importanza al mondo NoSQL, approfondendo MongoDB, e al mondo del Machine Learning, approfondendo PredictionIO. Successivamente è stata sviluppata un'applicazione attraverso l'utilizzo di tecnologie web, nodejs, node-webkit e le tecnologie approfondite prima. L'applicazione utilizza l'interpolazione polinomiale per predirre il prezzo di un bene salvato nello storico presente su MongoDB. Attraverso PredictionIO, essa analizza il comportamento degli altri utenti consigliando dei prodotti per l'acquisto. Infine è stata effetuata un'analisi dei risultati dell'errore prodotto dall'interpolazione.

BIRD-A: una piattaforma per la creazione di interfacce Web basate su ontologie per l'authoring di dati RDF

Lo scopo del progetto Bird-A è di mettere a disposizione uno strumento basato su ontologie per progettare un'interfaccia web collaborativa di creazione, visualizzazione, modifica e cancellazione di dati RDF e di fornirne una prima implementazione funzionante. La visione che sta muovendo la comunità del web semantico negli ultimi anni è quella di creare un Web basato su dati strutturati tra loro collegati, più che su documenti. Questo modello di architettura prende il nome di Linked Data ed è basata sulla possibilità di considerare cose, concetti, persone come risorse identificabili tramite URI e di poter fornire informazioni e descrivere collegamenti tra queste risorse attraverso l'uso di formati standard come RDF. Ciò che ha però frenato la diffusione di questi dati strutturati ed interconnessi sono stati gli alti requisiti di competenze tecniche necessarie sia alla loro creazione che alla loro fruizione. Il progetto Bird-A si prefigge di semplificare la creazione e la fruizione di dati RDF, favorendone la condivisione e la diffusione anche fra persone non dotate di conoscenze tecniche specifiche.

Mining tissue microarray data to uncover combinations of biomarker expression patterns that improve intermediate staging and grading of clear cell renal cell cancer

PURPOSE: Tumor stage and nuclear grade are the most important prognostic parameters of clear cell renal cell carcinoma (ccRCC). The progression risk of ccRCC remains difficult to predict particularly for tumors with organ-confined stage and intermediate differentiation grade. Elucidating molecular pathways deregulated in ccRCC may point to novel prognostic parameters that facilitate planning of therapeutic approaches. EXPERIMENTAL DESIGN: Using tissue microarrays, expression patterns of 15 different proteins were evaluated in over 800 ccRCC patients to analyze pathways reported to be physiologically controlled by the tumor suppressors von Hippel-Lindau protein and phosphatase and tensin homologue (PTEN). Tumor staging and grading were improved by performing variable selection using Cox regression and a recursive bootstrap elimination scheme. RESULTS: Patients with pT2 and pT3 tumors that were p27 and CAIX positive had a better outcome than those with all remaining marker combinations. A prolonged survival among patients with intermediate grade (grade 2) correlated with both nuclear p27 and cytoplasmic PTEN expression, as well as with inactive, nonphosphorylated ribosomal protein S6. By applying graphical log-linear modeling for over 700 ccRCC for which the molecular parameters were available, only a weak conditional dependence existed between the expression of p27, PTEN, CAIX, and p-S6, suggesting that the dysregulation of several independent pathways are crucial for tumor progression. CONCLUSIONS: The use of recursive bootstrap elimination, as well as graphical log-linear modeling for comprehensive tissue microarray (TMA) data analysis allows the unraveling of complex molecular contexts and may improve predictive evaluations for patients with advanced renal cancer.

Heterogeneity in vaccination coverage explains the size and occurrence of measles epidemics in German surveillance data

The objective of this study was to characterize empirically the association between vaccination coverage and the size and occurrence of measles epidemics in Germany. In order to achieve this we analysed data routinely collected by the Robert Koch Institute, which comprise the weekly number of reported measles cases at all ages as well as estimates of vaccination coverage at the average age of entry into the school system. Coverage levels within each federal state of Germany are incorporated into a multivariate time-series model for infectious disease counts, which captures occasional outbreaks by means of an autoregressive component. The observed incidence pattern of measles for all ages is best described by using the log proportion of unvaccinated school starters in the autoregressive component of the model.

A Morphological Box for Handling Temporal Data in B2C Systems

User interfaces are key properties of Business-to-Consumer (B2C) systems, and Web-based reservation systems are an important class of B2C systems. In this paper we show that these systems use a surprisingly broad spectrum of different approaches to handling temporal data in their Web inter faces. Based on these observations and on a literature analysis we develop a Morphological Box to present the main options for handling temporal data and give examples. The results indicate that the present state of developing and maintaining B2C systems has not been much influenced by modern Web Engi neering concepts and that there is considerable potential for improvement.

Femur Specific Polyaffine Model to Regularize the Log-Domain Demons Registration

Osteoarticular allograft transplantation is a popular treatment method in wide surgical resections with large defects. For this reason hospitals are building bone data banks. Performing the optimal allograft selection on bone banks is crucial to the surgical outcome and patient recovery. However, current approaches are very time consuming hindering an efficient selection. We present an automatic method based on registration of femur bones to overcome this limitation. We introduce a new regularization term for the log-domain demons algorithm. This term replaces the standard Gaussian smoothing with a femur specific polyaffine model. The polyaffine femur model is constructed with two affine (femoral head and condyles) and one rigid (shaft) transformation. Our main contribution in this paper is to show that the demons algorithm can be improved in specific cases with an appropriate model. We are not trying to find the most optimal polyaffine model of the femur, but the simplest model with a minimal number of parameters. There is no need to optimize for different number of regions, boundaries and choice of weights, since this fine tuning will be done automatically by a final demons relaxation step with Gaussian smoothing. The newly developed synthesis approach provides a clear anatomically motivated modeling contribution through the specific three component transformation model, and clearly shows a performance improvement (in terms of anatomical meaningful correspondences) on 146 CT images of femurs compared to a standard multiresolution demons. In addition, this simple model improves the robustness of the demons while preserving its accuracy. The ground truth are manual measurements performed by medical experts.

Geometry-Aware Multiscale Image Registration via OBBTree-Based Polyaffine Log-Demons

Non-linear image registration is an important tool in many areas of image analysis. For instance, in morphometric studies of a population of brains, free-form deformations between images are analyzed to describe the structural anatomical variability. Such a simple deformation model is justified by the absence of an easy expressible prior about the shape changes. Applying the same algorithms used in brain imaging to orthopedic images might not be optimal due to the difference in the underlying prior on the inter-subject deformations. In particular, using an un-informed deformation prior often leads to local minima far from the expected solution. To improve robustness and promote anatomically meaningful deformations, we propose a locally affine and geometry-aware registration algorithm that automatically adapts to the data. We build upon the log-domain demons algorithm and introduce a new type of OBBTree-based regularization in the registration with a natural multiscale structure. The regularization model is composed of a hierarchy of locally affine transformations via their logarithms. Experiments on mandibles show improved accuracy and robustness when used to initialize the demons, and even similar performance by direct comparison to the demons, with a significantly lower degree of freedom. This closes the gap between polyaffine and non-rigid registration and opens new ways to statistically analyze the registration results.

Attempting to improve standardized test results using Study Islands' web-based mastery program

State standardized testing has always been a tool to measure a school’s performance and to help evaluate school curriculum. However, with the school of choice legislation in 1992, the MEAP test became a measuring stick to grade schools by and a major tool in attracting school of choice students. Now, declining enrollment and a state budget struggling to stay out of the red have made school of choice students more important than ever before. MEAP scores have become the deciding factor in some cases. For the past five years, the Hancock Middle School staff has been working hard to improve their students’ MEAP scores in accordance with President Bush's “No Child Left Behind” legislation. In 2005, the school was awarded a grant that enabled staff to work for two years on writing and working towards school goals that were based on the improvement of MEAP scores in writing and math. As part of this effort, the school purchased an internet-based program geared at giving students practice on state content standards. This study examined the results of efforts by Hancock Middle School to help improve student scores in mathematics on the MEAP test through the use of an online program called “Study Island.” In the past, the program was used to remediate students, and as a review with an incentive at the end of the year for students completing a certain number of objectives. It had also been used as a review before upcoming MEAP testing in the fall. All of these methods may have helped a few students perform at an increased level on their standardized test, but the question remained of whether a sustained use of the program in a classroom setting would increase an understanding of concepts and performance on the MEAP for the masses. This study addressed this question. Student MEAP scores and Study Island data from experimental and comparison groups of students were compared to understand how a sustained use of Study Island in the classroom would impact student test scores on the MEAP. In addition, these data were analyzed to determine whether Study Island results provide a good indicator of students’ MEAP performance. The results of the study suggest that there were limited benefits related to sustained use of Study Island and gave some indications about the effectiveness of the mathematics curriculum at Hancock Middle School. These results and implications for instruction are discussed.

Application of well log tomography to the Dundee and Rogers City Limestones, Michigan Basin, USA

The Michigan Basin is located in the upper Midwest region of the United States and is centered geographically over the Lower Peninsula of Michigan. It is filled primarily with Paleozoic carbonates and clastics, overlying Precambrian basement rocks and covered by Pleistocene glacial drift. In Michigan, more than 46,000 wells have been drilled in the basin, many producing significant quantities of oil and gas since the 1920s in addition to providing a wealth of data for subsurface visualization. Well log tomography, formerly log-curve amplitude slicing, is a visualization method recently developed at Michigan Technological University to correlate subsurface data by utilizing the high vertical resolution of well log curves. The well log tomography method was first successfully applied to the Middle Devonian Traverse Group within the Michigan Basin using gamma ray log curves. The purpose of this study is to prepare a digital data set for the Middle Devonian Dundee and Rogers City Limestones, apply the well log tomography method to this data and from this application, interpret paleogeographic trends in the natural radioactivity. Both the Dundee and Rogers City intervals directly underlie the Traverse Group and combined are the most prolific reservoir within the Michigan Basin. Differences between this study and the Traverse Group include increased well control and “slicing” of a more uniform lithology. Gamma ray log curves for the Dundee and Rogers City Limestones were obtained from 295 vertical wells distributed over the Lower Peninsula of Michigan, converted to Log ASCII Standard files, and input into the well log tomography program. The “slicing” contour results indicate that during the formation of the Dundee and Rogers City intervals, carbonates and evaporites with low natural radioactive signatures on gamma ray logs were deposited. This contrasts the higher gamma ray amplitudes from siliciclastic deltas that cyclically entered the basin during Traverse Group deposition. Additionally, a subtle north-south, low natural radioactive trend in the center of the basin may correlate with previously published Dundee facies tracts. Prominent trends associated with the distribution of limestone and dolomite are not observed because the regional range of gamma ray values for both carbonates are equivalent in the Michigan Basin and additional log curves are needed to separate these lithologies.

Sonic log prediction in carbonates

This work is conducted to study the complications associated with the sonic log prediction in carbonate logs and to investigate the possible solutions to accurately predict the sonic logs in Traverse Limestone. Well logs from fifty different wells were analyzed to define the mineralogy of the Traverse Limestone by using conventional 4-mineral and 3-mineral identification approaches. We modified the conventional 3-mineral identification approach (that completely neglects the gamma ray response) to correct the shale effects on the basis of gamma ray log before employing the 3-mineral identification. This modification helped to get the meaningful insight of the data when a plot was made between DGA (dry grain density) and UMA (Photoelectric Volumetric Cross-section) with the characteristic ternary diagram of the quartz, calcite and dolomite. The results were then compared with the 4-mineral identification approach. Contour maps of the average mineral fractions present in the Traverse Limestone were prepared to see the basin wide mineralogy of Traverse Limestone. In the second part, sonic response of Traverse Limestone was predicted in fifty randomly distributed wells. We used the modified time average equation that accounts for the shale effects on the basis of gamma ray log, and used it to predict the sonic behavior from density porosity and average porosity. To account for the secondary porosity of dolomite, we subtracted the dolomitic fraction of clean porosity from the total porosity. The pseudo-sonic logs were then compared with the measured sonic logs on the root mean square (RMS) basis. Addition of dolomite correction in modified time average equation improved the results of sonic prediction from neutron porosity and average porosity. The results demonstrated that sonic logs could be predicted in carbonate rocks with a root mean square error of about 4μsec/ft. We also attempted the use of individual mineral components for sonic log prediction but the ambiguities in mineral fractions and in the sonic properties of the minerals limited the accuracy of the results.