A Framework for Building Hypercubes Using MapReduce

The European Space Agency's Gaia mission will create the largest and most precise three dimensional chart of our galaxy (the Milky Way), by providing unprecedented position, parallax, proper motion, and radial velocity measurements for about one billion stars. The resulting catalogue will be made available to the scientific community and will be analyzed in many different ways, including the production of a variety of statistics. The latter will often entail the generation of multidimensional histograms and hypercubes as part of the precomputed statistics for each data release, or for scientific analysis involving either the final data products or the raw data coming from the satellite instruments. In this paper we present and analyze a generic framework that allows the hypercube generation to be easily done within a MapReduce infrastructure, providing all the advantages of the new Big Data analysis paradigmbut without dealing with any specific interface to the lower level distributed system implementation (Hadoop). Furthermore, we show how executing the framework for different data storage model configurations (i.e. row or column oriented) and compression techniques can considerably improve the response time of this type of workload for the currently available simulated data of the mission. In addition, we put forward the advantages and shortcomings of the deployment of the framework on a public cloud provider, benchmark against other popular solutions available (that are not always the best for such ad-hoc applications), and describe some user experiences with the framework, which was employed for a number of dedicated astronomical data analysis techniques workshops.

A Framework for Building Hypercubes Using MapReduce

The European Space Agency's Gaia mission will create the largest and most precise three dimensional chart of our galaxy (the Milky Way), by providing unprecedented position, parallax, proper motion, and radial velocity measurements for about one billion stars. The resulting catalogue will be made available to the scientific community and will be analyzed in many different ways, including the production of a variety of statistics. The latter will often entail the generation of multidimensional histograms and hypercubes as part of the precomputed statistics for each data release, or for scientific analysis involving either the final data products or the raw data coming from the satellite instruments. In this paper we present and analyze a generic framework that allows the hypercube generation to be easily done within a MapReduce infrastructure, providing all the advantages of the new Big Data analysis paradigmbut without dealing with any specific interface to the lower level distributed system implementation (Hadoop). Furthermore, we show how executing the framework for different data storage model configurations (i.e. row or column oriented) and compression techniques can considerably improve the response time of this type of workload for the currently available simulated data of the mission. In addition, we put forward the advantages and shortcomings of the deployment of the framework on a public cloud provider, benchmark against other popular solutions available (that are not always the best for such ad-hoc applications), and describe some user experiences with the framework, which was employed for a number of dedicated astronomical data analysis techniques workshops.

Federalismo e forme di governo. L´inopportunità delle differenziazioni di modello

La legge costituzionale 1/1999 per le Regioni ordinarie (e la successiva 2/2001 per le Speciali) ha rappresentato un punto di svolta fondamentale del regionalismo italiano. Essa ha stabilito il principio dell’elezione popolare diretta del Presidente della Regione, a cui si collega un premio di maggioranza nel Consiglio regionale secondo il cosiddetto modello neo-parlamentare. Qualsiasi interruzione del rapporto fiduciario per dimissioni del Presidente o approvazione di una mozione di sfiducia porterebbe a nuove elezioni, cosa che rappresenta un serissimo deterrente alle crisi. La riforma prevedeva anche la possibilità per le Regioni di derogare col proprio Statuto a tali scelte tornando all’elezione consiliare e a sostituzioni della maggioranza. Nonostante alcuni tentativi di sfuggire alla regola del governo di legislatura utilizzando tale deroga in modo esplicito o surrettizio, seguendo vecchi retaggi assemblearisti, l’elezione diretta si è imposta ovunque, garantendo a tutte le Regioni analoghi e inediti standards di governabilità.