A Framework for Building Hypercubes Using MapReduce

The European Space Agency's Gaia mission will create the largest and most precise three dimensional chart of our galaxy (the Milky Way), by providing unprecedented position, parallax, proper motion, and radial velocity measurements for about one billion stars. The resulting catalogue will be made available to the scientific community and will be analyzed in many different ways, including the production of a variety of statistics. The latter will often entail the generation of multidimensional histograms and hypercubes as part of the precomputed statistics for each data release, or for scientific analysis involving either the final data products or the raw data coming from the satellite instruments. In this paper we present and analyze a generic framework that allows the hypercube generation to be easily done within a MapReduce infrastructure, providing all the advantages of the new Big Data analysis paradigmbut without dealing with any specific interface to the lower level distributed system implementation (Hadoop). Furthermore, we show how executing the framework for different data storage model configurations (i.e. row or column oriented) and compression techniques can considerably improve the response time of this type of workload for the currently available simulated data of the mission. In addition, we put forward the advantages and shortcomings of the deployment of the framework on a public cloud provider, benchmark against other popular solutions available (that are not always the best for such ad-hoc applications), and describe some user experiences with the framework, which was employed for a number of dedicated astronomical data analysis techniques workshops.

A Framework for Building Hypercubes Using MapReduce

The European Space Agency's Gaia mission will create the largest and most precise three dimensional chart of our galaxy (the Milky Way), by providing unprecedented position, parallax, proper motion, and radial velocity measurements for about one billion stars. The resulting catalogue will be made available to the scientific community and will be analyzed in many different ways, including the production of a variety of statistics. The latter will often entail the generation of multidimensional histograms and hypercubes as part of the precomputed statistics for each data release, or for scientific analysis involving either the final data products or the raw data coming from the satellite instruments. In this paper we present and analyze a generic framework that allows the hypercube generation to be easily done within a MapReduce infrastructure, providing all the advantages of the new Big Data analysis paradigmbut without dealing with any specific interface to the lower level distributed system implementation (Hadoop). Furthermore, we show how executing the framework for different data storage model configurations (i.e. row or column oriented) and compression techniques can considerably improve the response time of this type of workload for the currently available simulated data of the mission. In addition, we put forward the advantages and shortcomings of the deployment of the framework on a public cloud provider, benchmark against other popular solutions available (that are not always the best for such ad-hoc applications), and describe some user experiences with the framework, which was employed for a number of dedicated astronomical data analysis techniques workshops.

Gaia photometry for white dwarfs

Context. White dwarfs can be used to study the structure and evolution of the Galaxy by analysing their luminosity function and initial mass function. Among them, the very cool white dwarfs provide the information for the early ages of each population. Because white dwarfs are intrinsically faint only the nearby (~ 20 pc) sample is reasonably complete. The Gaia space mission will drastically increase the sample of known white dwarfs through its 5-6 years survey of the whole sky up to magnitude V = 20-25. Aims. We provide a characterisation of Gaia photometry for white dwarfs to better prepare for the analysis of the scientific output of the mission. Transformations between some of the most common photometric systems and Gaia passbands are derived. We also give estimates of the number of white dwarfs of the different galactic populations that will be observed. Methods. Using synthetic spectral energy distributions and the most recent Gaia transmission curves, we computed colours of three different types of white dwarfs (pure hydrogen, pure helium, and mixed composition with H/He = 0.1). With these colours we derived transformations to other common photometric systems (Johnson-Cousins, Sloan Digital Sky Survey, and 2MASS). We also present numbers of white dwarfs predicted to be observed by Gaia. Results. We provide relationships and colourcolour diagrams among different photometric systems to allow the prediction and/or study of the Gaia white dwarf colours. We also include estimates of the number of sources expected in every galactic population and with a maximum parallax error. Gaia will increase the sample of known white dwarfs tenfold to about 200 000. Gaia will be able to observe thousands of very cool white dwarfs for the first time, which will greatly improve our understanding of these stars and early phases of star formation in our Galaxy.

The Besançon Galaxy Model renewed. I. Constraints on the Galactic thin disc evolution from Tycho data

Context. The understanding of Galaxy evolution can be facilitated by the use of population synthesis models, which allow to test hypotheses on the star formation history, star evolution, as well as chemical and dynamical evolution of the Galaxy. Aims. The new version of the Besanc¸on Galaxy Model (hereafter BGM) aims to provide a more flexible and powerful tool to investigate the Initial Mass Function (IMF) and Star Formation Rate (SFR) of the Galactic disc. Methods. We present a new strategy for the generation of thin disc stars which assumes the IMF, SFR and evolutionary tracks as free parameters. We have updated most of the ingredients for the star count production and, for the first time, binary stars are generated in a consistent way. We keep in this new scheme the local dynamical self-consistency as in Bienayme et al (1987). We then compare simulations from the new model with Tycho-2 data and the local luminosity function, as a first test to verify and constrain the new ingredients. The effects of changing thirteen different ingredients of the model are systematically studied. Results. For the first time, a full sky comparison is performed between BGM and data. This strategy allows to constrain the IMF slope at high masses which is found to be close to 3.0, excluding a shallower slope such as Salpeter"s one. The SFR is found decreasing whatever IMF is assumed. The model is compatible with a local dark matter density of 0.011 M pc−3 implying that there is no compelling evidence for significant amount of dark matter in the disc. While the model is fitted to Tycho2 data, a magnitude limited sample with V<11, we check that it is still consistent with fainter stars. Conclusions. The new model constitutes a new basis for further comparisons with large scale surveys and is being prepared to become a powerful tool for the analysis of the Gaia mission data.

A Be-type star with a black-hole companion

Stellar-mass black holes have all been discovered through X-ray emission, which arises from the accretion of gas from their binary companions (this gas is either stripped from low-mass stars or supplied as winds from massive ones). Binary evolution models also predict the existence of black holes accreting from the equatorial envelope of rapidly spinning Be-type stars (stars of the Be type are hot blue irregular variables showing characteristic spectral emission lines of hydrogen). Of the ~80 Be X-ray binaries known in the Galaxy, however, only pulsating neutron stars have been found as companions. A black hole was formally allowed as a solution for the companion to the Be star MWC 656 (also known as HD 215227), although that was based on a single radial velocity curve of the Be star, a mistaken spectral classification and rough estimates of the inclination angle. Here we report observations of an accretion disk line mirroring the orbit of the Be star. This, together with an improved radial velocity curve of the Be star through fitting sharp Fe II profiles from the equatorial disk, and a refined Be classification (to that of a B1.5-B2 III star), reveals a black hole of 3.8 to 6.9 solar masses orbiting MWC 656, the candidate counterpart of the gamma-ray source AGL J2241+4454. The black hole is X-ray quiescent and fed by a radiatively inefficient accretion flow giving a luminosity less than 1.6 x 10-7 times the Eddington luminosity. This implies that Be binaries with black-hole companions are difficult to detect by conventional X-ray surveys.

The star forming region Monoceros R2 as a gamma-ray source

After the release of the gamma-ray source catalog produced by the Fermi satellite during its first two years of operation, a significant fraction of sources still remain unassociated at lower energies. In addition to well-known high-energy emitters (pulsars, blazars, supernova remnants, etc.), theoretical expectations predict new classes of gamma-ray sources. In particular, gamma-ray emission could be associated with some of the early phases of stellar evolution, but this interesting possibility is still poorly understood. Aims: The aim of this paper is to assess the possibility of the Fermi gamma-ray source 2FGL J0607.5-0618c being associated with the massive star forming region Monoceros R2. Methods: A multi-wavelength analysis of the Monoceros R2 region is carried out using archival data at radio, infrared, X-ray, and gamma-ray wavelengths. The resulting observational properties are used to estimate the physical parameters needed to test the different physical scenarios. Results: We confirm the 2FGL J0607.5-0618c detection with improved confidence over the Fermi two-year catalog. We find that a combined effect of the multiple young stellar objects in Monoceros R2 is a viable picture for the nature of the source.

Very-High-Energy gamma rays from a Distant Quasar: How Transparent Is the Universe?

The atmospheric Cherenkov gamma-ray telescope MAGIC, designed for a low-energy threshold, has detected very-high-energy gamma rays from a giant flare of the distant Quasi-Stellar Radio Source (in short: radio quasar) 3C 279, at a distance of more than 5 billion light-years (a redshift of 0.536). No quasar has been observed previously in very-high-energy gamma radiation, and this is also the most distant object detected emitting gamma rays above 50 gigaelectron volts. Because high-energy gamma rays may be stopped by interacting with the diffuse background light in the universe, the observations by MAGIC imply a low amount for such light, consistent with that known from galaxy counts.

Confirmation of persistent radio jets in the microquasar LS 5039

We present here new observations conducted with the EVN and MERLIN of the persistent microquasar LS 5039 discovered by Paredes et al. (2000) with the VLBA. The new observations confirm the presence of an asymmetric two-sided jet reaching up to 1000 AU on the longest jet arm. The results suggest a bending of the jets with increasing distance from the core and/or precession. The origin and location of the high-energy gamma-ray emission associated with the system is discussed and an estimate of the magnetic field at the base of the jet given. Our results suggest a well collimated radio jet. We also comment on new observing strategies to be used with satellites and forthcoming detectors, since this persistent source appears to be a rather good laboratory to explore the accretion/ejection processes taking place near compact objects.

The first SEPServer event catalogue 68-MeV solar proton events observed at 1 AU in 1996-2010

SEPServer is a three-year collaborative project funded by the seventh framework programme (FP7-SPACE) of the European Union. The objective of the project is to provide access to state-of-the-art observations and analysis tools for the scientific community on solar energetic particle (SEP) events and related electromagnetic (EM) emissions. The project will eventually lead to better understanding of the particle acceleration and transport processes at the Sun and in the inner heliosphere. These processes lead to SEP events that form one of the key elements of space weather. In this paper we present the first results from the systematic analysis work performed on the following datasets: SOHO/ERNE, SOHO/EPHIN, ACE/EPAM, Wind/WAVES and GOES X-rays. A catalogue of SEP events at 1 AU, with complete coverage over solar cycle 23, based on high-energy (~68-MeV) protons from SOHO/ERNE and electron recordings of the events by SOHO/EPHIN and ACE/EPAM are presented. A total of 115 energetic particle events have been identified and analysed using velocity dispersion analysis (VDA) for protons and time-shifting analysis (TSA) for electrons and protons in order to infer the SEP release times at the Sun. EM observations during the times of the SEP event onset have been gathered and compared to the release time estimates of particles. Data from those events that occurred during the European day-time, i.e., those that also have observations from ground-based observatories included in SEPServer, are listed and a preliminary analysis of their associations is presented. We find that VDA results for protons can be a useful tool for the analysis of proton release times, but if the derived proton path length is out of a range of 1 AU < s[3 AU, the result of the analysis may be compromised, as indicated by the anti-correlation of the derived path length and release time delay from the asso ciated X-ray flare. The average path length derived from VDA is about 1.9 times the nominal length of the spiral magnetic field line. This implies that the path length of first-arriving MeV to deka-MeV protons is affected by interplanetary scattering. TSA of near-relativistic electrons results in a release time that shows significant scatter with respect to the EM emissions but with a trend of being delayed more with increasing distance between the flare and the nominal footpoint of the Earth-connected field line.

Spectral energy distribution of the gamma-ray microquasar LS 5039

The microquasar LS 5039 has recently been detected as a source of very high energy (VHE) $\gamma$-rays. This detection, that confirms the previously proposed association of LS 5039 with the EGRET source 3EG~J1824$-$1514, makes of LS 5039 a special system with observational data covering nearly all the electromagnetic spectrum. In order to reproduce the observed spectrum of LS 5039, from radio to VHE $\gamma$-rays, we have applied a cold matter dominated jet model that takes into account accretion variability, the jet magnetic field, particle acceleration, adiabatic and radiative losses, microscopic energy conservation in the jet, and pair creation and absorption due to the external photon fields, as well as the emission from the first generation of secondaries. The radiative processes taken into account are synchrotron, relativistic Bremsstrahlung and inverse Compton (IC). The model is based on a scenario that has been characterized with recent observational results, concerning the orbital parameters, the orbital variability at X-rays and the nature of the compact object. The computed spectral energy distribution (SED) shows a good agreement with the available observational data.

Gaia photometry for white dwarfs

Context. White dwarfs can be used to study the structure and evolution of the Galaxy by analysing their luminosity function and initial mass function. Among them, the very cool white dwarfs provide the information for the early ages of each population. Because white dwarfs are intrinsically faint only the nearby (~ 20 pc) sample is reasonably complete. The Gaia space mission will drastically increase the sample of known white dwarfs through its 5-6 years survey of the whole sky up to magnitude V = 20-25. Aims. We provide a characterisation of Gaia photometry for white dwarfs to better prepare for the analysis of the scientific output of the mission. Transformations between some of the most common photometric systems and Gaia passbands are derived. We also give estimates of the number of white dwarfs of the different galactic populations that will be observed. Methods. Using synthetic spectral energy distributions and the most recent Gaia transmission curves, we computed colours of three different types of white dwarfs (pure hydrogen, pure helium, and mixed composition with H/He = 0.1). With these colours we derived transformations to other common photometric systems (Johnson-Cousins, Sloan Digital Sky Survey, and 2MASS). We also present numbers of white dwarfs predicted to be observed by Gaia. Results. We provide relationships and colour-colour diagrams among different photometric systems to allow the prediction and/or study of the Gaia white dwarf colours. We also include estimates of the number of sources expected in every galactic population and with a maximum parallax error. Gaia will increase the sample of known white dwarfs tenfold to about 200 000. Gaia will be able to observe thousands of very cool white dwarfs for the first time, which will greatly improve our understanding of these stars and early phases of star formation in our Galaxy.

All-sky search algorithms for monochromatic signals in bar gravitational wave detector data

In this paper we design and develop several filtering strategies for the analysis of data generated by a resonant bar gravitational wave (GW) antenna, with the goal of assessing the presence (or absence) therein of long-duration monochromatic GW signals, as well as the eventual amplitude and frequency of the signals, within the sensitivity band of the detector. Such signals are most likely generated in the fast rotation of slightly asymmetric spinning stars. We develop practical procedures, together with a study of their statistical properties, which will provide us with useful information on the performance of each technique. The selection of candidate events will then be established according to threshold-crossing probabilities, based on the Neyman-Pearson criterion. In particular, it will be shown that our approach, based on phase estimation, presents a better signal-to-noise ratio than does pure spectral analysis, the most common approach.

The 2010 May Flaring Episode of Cygnus X-3 in Radio, X-rays, and γ-rays

In 2009, Cygnus X-3 (Cyg X-3) became the first microquasar to be detected in the GeV γ-ray regime, via the satellites Fermi and AGILE. The addition of this new band to the observational toolbox holds promise for building a more detailed understanding of the relativistic jets of this and other systems. We present a rich data set of radio, hard and soft X-ray, and γ-ray observations of Cyg X-3 made during a flaring episode in 2010 May. We detect a ~3 day softening and recovery of the X-ray emission, followed almost immediately by a ~1 Jy radio flare at 15 GHz, followed by a 4.3σ γ-ray flare (E > 100 MeV) ~1.5 days later. The radio sampling is sparse, but we use archival data to argue that it is unlikely the γ-ray flare was followed by any significant unobserved radio flares. In this case, the sequencing of the observed events is difficult to explain in a model in which the γ-ray emission is due to inverse Compton scattering of the companion star's radiation field. Our observations suggest that other mechanisms may also be responsible for γ-ray emission from Cyg X-3.

Long-term X-ray variability of the microquasar system LS5039/RX J1826.2-1450

We report on the results of the spectral and timing analysis of a BeppoSAX observation of the microquasar system LS 5039/RX J1826.2-1450. The source was found in a low-flux state with Fx(1-10 keV)= 4.7 x 10^{-12} erg cm^{-2} s^{-1}, which represents almost one order of magnitude lower than a previous RXTE observation 2.5 years before. The 0.1--10 keV spectrum is described by an absorbed power-law continuum with photon-number spectral index Gamma=1.8+-0.2 and hydrogen column density of NH=1.0^{+0.4}_{-0.3} x 10^{22} cm^{-2}. According to the orbital parameters of the system the BeppoSAX observation covers the time of an X-ray eclipse should one occur. However, the 1.6-10 keV light curve does not show evidence for such an event, which allows us to give an upper limit to the inclination of the system. The low X-ray flux detected during this observation is interpreted as a decrease in the mass accretion rate onto the compact object due to a decrease in the mass-loss rate from the primary.

On the multiwavelength spectrum of the microquasar 1E1740.7-2942

The microquasar 1E 1740.7-2942 is a source located in the direction of the Galactic Center. It has been detected at X-rays, soft gamma-rays, and in the radio band, showing an extended radio component in the form of a double-sided jet. Although no optical counterpart has been found so far for 1E 1740.7-2942, its X-ray activity strongly points to a galactic nature. Aims.We aim to improve our understanding of the hard X-ray and gamma-ray production in the system, exploring whether the jet can emit significantly at high energies under the light of the present knowledge. Methods.We have modeled the source emission, from radio to gamma-rays, with a cold-matter dominated jet model. INTEGRAL data combined with radio and RXTE data, as well as EGRET and HESS upper-limits, are used to compare the computed and the observed spectra. Results.From our modeling, we find out that jet emission cannot explain the high fluxes observed at hard X-rays without violating at the same time the constraints from the radio data, favoring the corona origin of the hard X-rays. Also, 1E 1740.7-2942 might be detected by GLAST or AGILE at GeV energies, and by HESS and HESS-II beyond 100 GeV, with the spectral shape likely affected by photon-photon absorption in the disk and corona photon fields.