A Framework for Building Hypercubes Using MapReduce

The European Space Agency's Gaia mission will create the largest and most precise three dimensional chart of our galaxy (the Milky Way), by providing unprecedented position, parallax, proper motion, and radial velocity measurements for about one billion stars. The resulting catalogue will be made available to the scientific community and will be analyzed in many different ways, including the production of a variety of statistics. The latter will often entail the generation of multidimensional histograms and hypercubes as part of the precomputed statistics for each data release, or for scientific analysis involving either the final data products or the raw data coming from the satellite instruments. In this paper we present and analyze a generic framework that allows the hypercube generation to be easily done within a MapReduce infrastructure, providing all the advantages of the new Big Data analysis paradigmbut without dealing with any specific interface to the lower level distributed system implementation (Hadoop). Furthermore, we show how executing the framework for different data storage model configurations (i.e. row or column oriented) and compression techniques can considerably improve the response time of this type of workload for the currently available simulated data of the mission. In addition, we put forward the advantages and shortcomings of the deployment of the framework on a public cloud provider, benchmark against other popular solutions available (that are not always the best for such ad-hoc applications), and describe some user experiences with the framework, which was employed for a number of dedicated astronomical data analysis techniques workshops.

A Framework for Building Hypercubes Using MapReduce

The European Space Agency's Gaia mission will create the largest and most precise three dimensional chart of our galaxy (the Milky Way), by providing unprecedented position, parallax, proper motion, and radial velocity measurements for about one billion stars. The resulting catalogue will be made available to the scientific community and will be analyzed in many different ways, including the production of a variety of statistics. The latter will often entail the generation of multidimensional histograms and hypercubes as part of the precomputed statistics for each data release, or for scientific analysis involving either the final data products or the raw data coming from the satellite instruments. In this paper we present and analyze a generic framework that allows the hypercube generation to be easily done within a MapReduce infrastructure, providing all the advantages of the new Big Data analysis paradigmbut without dealing with any specific interface to the lower level distributed system implementation (Hadoop). Furthermore, we show how executing the framework for different data storage model configurations (i.e. row or column oriented) and compression techniques can considerably improve the response time of this type of workload for the currently available simulated data of the mission. In addition, we put forward the advantages and shortcomings of the deployment of the framework on a public cloud provider, benchmark against other popular solutions available (that are not always the best for such ad-hoc applications), and describe some user experiences with the framework, which was employed for a number of dedicated astronomical data analysis techniques workshops.

Prestacions en benefici de la comunitat i realització de tasques socioeducatives en justícia juvenil

Els joves als quals se’ls ha imposat una mesura de realització de tasques socioeducatives (RTS) o de prestacions en benefici de la comunitat (PBC) presenten un alt fracàs escolar, majoritàriament amb baix rendiment i abandonament dels estudis en la meitat dels casos. Un de cada tres joves prové d’una família amb algun tipus de problemàtica específica o una economia insuficient. Els resultats indiquen, però, que els joves que realitzen tasques socioeducatives solen acumular més problemàtiques que els de PBC, ja sigui a nivell personal, familiar o social. Quant al perfil penal i criminològic, el 63% dels nois i noies d’aquestes dues mesures té antecedents. La taxa de reincidència de RTS és del 31,1% i la de PBC del 25,2%. Alguns dels factors que els joves reincidents mostren en més proporció són tenir un grup d’iguals dissocial, estar en contacte amb els serveis socials o tenir antecedents. Els reincidents de RTS també presenten més proporció de consum de tòxics, problemes de salut mental, fracàs escolar i una ocupació del temps desestructurada i amb conductes de risc. Comparant aquests resultats amb estudis respecte altres programes i mesures de Justícia juvenil podem concloure que la taxa de reincidència global a Catalunya és del 28,9% i l’específica de Medi obert és del 27,8%.

A user-friendly web portal for T-Coffee on supercomputers

Background: Parallel T-Coffee (PTC) was the first parallel implementation of the T-Coffee multiple sequence alignment tool. It is based on MPI and RMA mechanisms. Its purpose is to reduce the execution time of the large-scale sequence alignments. It can be run on distributed memory clusters allowing users to align data sets consisting of hundreds of proteins within a reasonable time. However, most of the potential users of this tool are not familiar with the use of grids or supercomputers. Results: In this paper we show how PTC can be easily deployed and controlled on a super computer architecture using a web portal developed using Rapid. Rapid is a tool for efficiently generating standardized portlets for a wide range of applications and the approach described here is generic enough to be applied to other applications, or to deploy PTC on different HPC environments. Conclusions: The PTC portal allows users to upload a large number of sequences to be aligned by the parallel version of TC that cannot be aligned by a single machine due to memory and execution time constraints. The web portal provides a user-friendly solution.

Prestaciones en beneficio de la comunidad y realización de tareas socioeducativas en justicia juvenil

Los jóvenes a los que se les ha impuesto una medida de realización de tareas socioeducativas (RTS) o de prestaciones en beneficio de la comunidad (PBC) presentan un alto fracaso escolar, mayoritariamente con bajo rendimiento y abandono de los estudios en la mitad de los casos. Uno de cada tres jóvenes proviene de una familia con algún tipo de problemática específica o una economía insuficiente. Los resultados indican, sin embargo, que los jóvenes que realizan tareas socioeducativas suelen acumular más problemáticas que a los que se les ha impuesto una PBC, ya sea a nivel personal, familiar o social. En cuanto el perfil penal y criminológico, el 63% de los chicos y chicas de estas dos medidas tiene antecedentes. La tasa de reincidencia de la RTS es del 31,1% y la de PBC del 25,2%. Algunos de los factores que los jóvenes reincidentes muestran en mayor proporción son tener un grupo de iguales disocial, estar en contacto con los servicios sociales o tener antecedentes. Los reincidentes de RTS también presentan mayor proporción de consumo de tóxicos, problemas de salud mental, fracaso escolar y una ocupación del tiempo desestructurada y con conductas de riesgo. Comparando estos resultados con estudios respecto a otros programas y medidas de Justicia juvenil podemos concluir que la tasa de reincidencia global en Cataluña es del 28,9% y la específica de Medio abierto es del 27,8%.

Statistical mechanics of multi-edge networks

Statistical properties of binary complex networks are well understood and recently many attempts have been made to extend this knowledge to weighted ones. There are, however, subtle yet important considerations to be made regarding the nature of the weights used in this generalization. Weights can be either continuous or discrete magnitudes, and in the latter case, they can additionally have undistinguishable or distinguishable nature. This fact has not been addressed in the literature insofar and has deep implications on the network statistics. In this work we face this problem introducing multiedge networks as graphs where multiple (distinguishable) connections between nodes are considered. We develop a statistical mechanics framework where it is possible to get information about the most relevant observables given a large spectrum of linear and nonlinear constraints including those depending both on the number of multiedges per link and their binary projection. The latter case is particularly interesting as we show that binary projections can be understood from multiedge processes. The implications of these results are important as many real-agent-based problems mapped onto graphs require this treatment for a proper characterization of their collective behavior.

Molecular Physics of Elementary Processes relevant to Hypersonics: atom-molecule, molecule-molecule and atom-surface processes.

In the present chapter some prototype gas and gas-surface processes occurring within the hypersonic flow layer surrounding spacecrafts at planetary entry are discussed. The discussion is based on microscopic dynamical calculations of the detailed cross sections and rate coefficients performed using classical mechanics treatments for atoms, molecules and surfaces. Such treatment allows the evaluation of the efficiency of thermal processes (both at equilibrium and nonequilibrium distributions) based on state-to-state and state specific calculations properly averaged over the population of the initial states. The dependence of the efficiency of the considered processes on the initial partitioning of energy among the various degrees of freedom is discussed.

Trophic versus geographic structure in stable isotope signatures of pelagic seabirds breeding in the northeast Atlantic

Feeding ecology and geographic location are 2 major factors influencing animal stable isotope signatures, but their relative contributions are poorly understood, which limits the usefulness of stable isotope analysis in the study of animal ecology. To improve our knowledge of the main sources of isotopic variability at sea, we determined δ15N and δ13C signatures in the first primary feather of adult birds from 11 Procellariiform species (n = 609) across 16 northeast Atlantic localities, from Cape Verde (20°N) to Iceland (60°N). Post-breeding areas (where the studied feather is thought to be grown) were determined using light-level geolocation for 6 of the 11 species. Isotopic variability was geographically unstructured within the mid-northeast Atlantic (Macaronesian archipelagos), but trophically structured according to species and regardless of the breeding location, presumably as a result of trophic segregation among species. Indeed, the interspecific isotopic overlap resulting from combining δ15N and δ13C signatures of seabirds was low, which suggests that most species exploited exclusive trophic resources consistently across their geographic range. Species breeding in north temperate regions (Iceland, Scotland and Northern Ireland) showed enriched δ15N compared to the same or similar species breeding in tropical and subtropical regions, suggesting some differences in baseline levels between these regions. The present study illustrates a noticeable trophic segregation of northeast Atlantic Procellariiformes. Our results show that the isotopic approach has limited applicability for the study of animal movements in the northeast Atlantic at a regional scale, but is potentially useful for the study of long-distance migrations between large marine systems

Pose-based SLAM with probabilistic scan matching algorithm using a mechanical scanned imaging sonar

This paper proposes a pose-based algorithm to solve the full SLAM problem for an autonomous underwater vehicle (AUV), navigating in an unknown and possibly unstructured environment. The technique incorporate probabilistic scan matching with range scans gathered from a mechanical scanning imaging sonar (MSIS) and the robot dead-reckoning displacements estimated from a Doppler velocity log (DVL) and a motion reference unit (MRU). The proposed method utilizes two extended Kalman filters (EKF). The first, estimates the local path travelled by the robot while grabbing the scan as well as its uncertainty and provides position estimates for correcting the distortions that the vehicle motion produces in the acoustic images. The second is an augment state EKF that estimates and keeps the registered scans poses. The raw data from the sensors are processed and fused in-line. No priory structural information or initial pose are considered. The algorithm has been tested on an AUV guided along a 600 m path within a marina environment, showing the viability of the proposed approach

Genetic diversity and species richness patterns in Baetidae (Ephemeroptera) in the Montseny Mountain range (North-East Iberian Peninsula)

This study aimed to describe patterns of diversity of Baetidae (Ephemeroptera) at the ommunity and population levels within the Montseny Mountain range (North-East Iberian Peninsula). We studied both the distribution of 4 species of baetids in 20 sites among three catchments along the altitudinal gradient (350-1700 masl); and the genetic diversity of the mtDNA cytochrome c oxidase subunit I (cox1) gene of the two common species Baetis alpinus and Baetis rhodani. We found a gradual replacement of the dominant species along the altitudinal gradient. Baetis alpinus inhabited sites at high-altitudes, and this species was replaced by B. rhodani when the altitude decreased. Baetis melanonyx and Alainites muticus attained low abundance at all river sections, and no clear altitudinal trend appeared. Our hypothesis at the population level was that genetic structuring is associated with geographic distance and limited by drainage boundaries among the three studied catchments because of the short-time dispersion of adults. Unexpectedly, analyses of molecular variance (AMOVA) and isolation-bydistance (IBD) showed genetic diversity was unstructured by distance for both species, which may be explained by the relatively short spatial scale studied and small topographic barriers among the three catchments. The Generalized Mixed Yule-Coalescent (GMYC) model showed that B. rhodani had two differentiated genetic lineages that co-occurred in all sites. Overall, diversity of baetids was structured at the community level along the altitudinal gradient, whereas it was unstructured at the population level within the Montseny Mountain range.

Genetic diversity and species richness patterns in Baetidae (Ephemeroptera) in the Montseny Mountain range (North-East Iberian Peninsula)

This study aimed to describe patterns of diversity of Baetidae (Ephemeroptera) at the ommunity and population levels within the Montseny Mountain range (North-East Iberian Peninsula). We studied both the distribution of 4 species of baetids in 20 sites among three catchments along the altitudinal gradient (350-1700 masl); and the genetic diversity of the mtDNA cytochrome c oxidase subunit I (cox1) gene of the two common species Baetis alpinus and Baetis rhodani. We found a gradual replacement of the dominant species along the altitudinal gradient. Baetis alpinus inhabited sites at high-altitudes, and this species was replaced by B. rhodani when the altitude decreased. Baetis melanonyx and Alainites muticus attained low abundance at all river sections, and no clear altitudinal trend appeared. Our hypothesis at the population level was that genetic structuring is associated with geographic distance and limited by drainage boundaries among the three studied catchments because of the short-time dispersion of adults. Unexpectedly, analyses of molecular variance (AMOVA) and isolation-bydistance (IBD) showed genetic diversity was unstructured by distance for both species, which may be explained by the relatively short spatial scale studied and small topographic barriers among the three catchments. The Generalized Mixed Yule-Coalescent (GMYC) model showed that B. rhodani had two differentiated genetic lineages that co-occurred in all sites. Overall, diversity of baetids was structured at the community level along the altitudinal gradient, whereas it was unstructured at the population level within the Montseny Mountain range.

Leaving the Dark ages with AMIGA

We present an Analytic Model of Intergalactic-medium and GAlaxy (AMIGA) evolution since the dark ages. AMIGA is in the spirit of the popular semi-analytic models of galaxy formation, although it does not use halo merger trees but interpolates halo properties in grids that are progressively built. This strategy is less memory-demanding and allows one to start modeling at sufficiently high redshifts and low halo masses to have trivial boundary conditions. The number of free parameters is minimized by making a causal connection between physical processes usually treated as independent of each other, which leads to more reliable predictions. However, the strongest points of AMIGA are the following: (1) the inclusion of molecular cooling and metal-poor, population III (Pop III) stars with the most dramatic feedback and (2) accurate follow up of the temperature and volume filling factor of neutral, singly ionized, and doubly ionized regions, taking into account the distinct halo mass functions in those environments. We find the following general results. Massive Pop III stars determine the intergalactic medium metallicity and temperature, and the growth of spheroids and disks is self-regulated by that of massive black holes (MBHs) developed from the remnants of those stars. However, the properties of normal galaxies and active galactic nuclei appear to be quite insensitive to Pop III star properties due to the much higher yield of ordinary stars compared to Pop III stars and the dramatic growth of MBHs when normal galaxies begin to develop, which cause the memory loss of the initial conditions.