NanoStreams: Advancing the Hardware and Software Stack for Real-Time Analytics on Fast Data Streams

NanoStreams is a consortium project funded by the European Commission under its FP7 programme and is a major effort to address the challenges of processing vast amounts of data in real-time, with a markedly lower carbon footprint than the state of the art. The project addresses both the energy challenge and the high-performance required by emerging applications in real-time streaming data analytics. NanoStreams achieves this goal by designing and building disruptive micro-server solutions incorporating real-silicon prototype micro-servers based on System-on-Chip and reconfigurable hardware technologies.

Power Efficient Dynamic-Range Utilisation for DSP On FPGA

A power and resource efficient ‘dynamic-range utilisation’ technique to increase operational capacity of DSP IP cores by exploiting redundancy in the data epresentation of sampled analogue input data, is presented. By cleverly partitioning dynamic-range into separable processing threads, several data streams are computed concurrently on the same hardware. Unlike existing techniques which act solely to reduce power consumption due to sign extension, here the dynamic range is exploited to increase operational capacity while still achieving reduced power consumption. This extends an existing system-level, power efficient framework for the design of low power DSP IP cores, which when applied to the design of an FFT IP core in a digital receiver system gives an architecture requiring 50% fewer multipliers, 12% fewer slices and 51%-56% less power.

Efficient Marginal Likelihood Computation for Gaussian Process Regression

In a Bayesian learning setting, the posterior distribution of a predictive model arises from a trade-off between its prior distribution and the conditional likelihood of observed data. Such distribution functions usually rely on additional hyperparameters which need to be tuned in order to achieve optimum predictive performance; this operation can be efficiently performed in an Empirical Bayes fashion by maximizing the posterior marginal likelihood of the observed data. Since the score function of this optimization problem is in general characterized by the presence of local optima, it is necessary to resort to global optimization strategies, which require a large number of function evaluations. Given that the evaluation is usually computationally intensive and badly scaled with respect to the dataset size, the maximum number of observations that can be treated simultaneously is quite limited. In this paper, we consider the case of hyperparameter tuning in Gaussian process regression. A straightforward implementation of the posterior log-likelihood for this model requires O(N^3) operations for every iteration of the optimization procedure, where N is the number of examples in the input dataset. We derive a novel set of identities that allow, after an initial overhead of O(N^3), the evaluation of the score function, as well as the Jacobian and Hessian matrices, in O(N) operations. We prove how the proposed identities, that follow from the eigendecomposition of the kernel matrix, yield a reduction of several orders of magnitude in the computation time for the hyperparameter optimization problem. Notably, the proposed solution provides computational advantages even with respect to state of the art approximations that rely on sparse kernel matrices.

Orthogonal Vector Approach for Synthesis of Multi-beam Directional Modulation Transmitters

An orthogonal vector approach is proposed for the synthesis of multi-beam directional modulation (DM) transmitters. These systems have the capability of concurrently projecting independent data streams into different specified spatial directions while simultaneously distorting signal constellations in all other directions. Simulated bit error rate (BER) spatial distributions are presented for various multi-beam system configurations in order to illustrate representative examples of physical layer security performance enhancement that can be achieved.

Fourier Rotman Lens Enabled Directional Modulation Transmitter

A 10 GHz Fourier Rotman lens enabled dynamic directional modulation (DM) transmitter is experimentally evaluated. Bit error rate (BER) performance is obtained via real-time data transmission. It is shown that Fourier Rotman DM functionality enhances system security performance in terms of narrower decodable low BER region and higher BER values associated with BER sidelobes especially under high signal to noise ratio (SNR) scenarios. This enhancement is achieved by controlled corruption of constellation diagrams in IQ space by orthogonal injection of interference. Furthermore, the paper gives the first report of a functional dual-beam DM transmitter, which has the capability of simultaneously projecting two independent data streams into two different spatial directions while simultaneously scrambling the information signals along all other directions.

Blind Characterisation of Sensors with Second-Order Dynamic Response

Compensation for the dynamic response of a temperature sensor usually involves the estimation of its input on the basis of the measured output and model parameters. In the case of temperature measurement, the sensor dynamic response is strongly dependent on the measurement environment and fluid velocity. Estimation of time-varying sensor model parameters therefore requires continuous textit{in situ} identification. This can be achieved by employing two sensors with different dynamic properties, and exploiting structural redundancy to deduce the sensor models from the resulting data streams. Most existing approaches to this problem assume first-order sensor dynamics. In practice, however second-order models are more reflective of the dynamics of real temperature sensors, particularly when they are encased in a protective sheath. As such, this paper presents a novel difference equation approach to solving the blind identification problem for sensors with second-order models. The approach is based on estimating an auxiliary ARX model whose parameters are related to the desired sensor model parameters through a set of coupled non-linear algebraic equations. The ARX model can be estimated using conventional system identification techniques and the non-linear equations can be solved analytically to yield estimates of the sensor models. Simulation results are presented to demonstrate the efficiency of the proposed approach under various input and parameter conditions.

Machine learning for transient discovery in Pan-STARRS1 difference imaging

Efficient identification and follow-up of astronomical transients is hindered by the need for humans to manually select promising candidates from data streams that contain many false positives. These artefacts arise in the difference images that are produced by most major ground-based time-domain surveys with large format CCD cameras. This dependence on humans to reject bogus detections is unsustainable for next generation all-sky surveys and significant effort is now being invested to solve the problem computationally. In this paper, we explore a simple machine learning approach to real-bogus classification by constructing a training set from the image data of similar to 32 000 real astrophysical transients and bogus detections from the Pan-STARRS1 Medium Deep Survey. We derive our feature representation from the pixel intensity values of a 20 x 20 pixel stamp around the centre of the candidates. This differs from previous work in that it works directly on the pixels rather than catalogued domain knowledge for feature design or selection. Three machine learning algorithms are trained (artificial neural networks, support vector machines and random forests) and their performances are tested on a held-out subset of 25 per cent of the training data. We find the best results from the random forest classifier and demonstrate that by accepting a false positive rate of 1 per cent, the classifier initially suggests a missed detection rate of around 10 per cent. However, we also find that a combination of bright star variability, nuclear transients and uncertainty in human labelling means that our best estimate of the missed detection rate is approximately 6 per cent.

Software defined FFT architecture for IEEE 802.11ac

The upcoming IEEE 802.11ac standard boosts the throughput of previous IEEE 802.11n by adding wider 80 MHz and 160 MHz channels with up to 8 antennas (versus 40 MHz channel and 4 antennas in 802.11n). This necessitates new 1-8 stream 256/512-point Fast Fourier Transform (FFT) / inverse FFT (IFFT) processing with 80/160 MSample/s throughput. Although there are abundant related work, they all fail to meet the requirements of IEEE 802.11ac FFT/IFFT on point size, throughput and multiple data streams at the same time. This paper proposes the first software defined FFT/IFFT architecture as a solution. By making use of a customised soft stream processor on FPGA, we show how a software defined FFT architecture can meet all the requirements of IEEE 802.11ac with low cost and high resource efficiency. When compared with dedicated Xilinx FFT core, our implementation exhibits only one third of the resources also up to three times of resource efficiency.

Leveraging Stratification in Twitter Sampling

With Tweet volumes reaching 500 million a day, sampling is inevitable for any application using Twitter data. Realizing this, data providers such as Twitter, Gnip and Boardreader license sampled data streams priced in accordance with the sample size. Big Data applications working with sampled data would be interested in working with a large enough sample that is representative of the universal dataset. Previous work focusing on the representativeness issue has considered ensuring the global occurrence rates of key terms, be reliably estimated from the sample. Present technology allows sample size estimation in accordance with probabilistic bounds on occurrence rates for the case of uniform random sampling. In this paper, we consider the problem of further improving sample size estimates by leveraging stratification in Twitter data. We analyze our estimates through an extensive study using simulations and real-world data, establishing the superiority of our method over uniform random sampling. Our work provides the technical know-how for data providers to expand their portfolio to include stratified sampled datasets, whereas applications are benefited by being able to monitor more topics/events at the same data and computing cost.

The Conditional Phase-type distribution: Graphical Models for Continuous Non-Normal Data:The Conditional Phase-type (C-Ph) distribution:

Conditional Gaussian (CG) distributions allow the inclusion of both discrete and continuous variables in a model assuming that the continuous variable is normally distributed. However, the CG distributions have proved to be unsuitable for survival data which tends to be highly skewed. A new method of analysis is required to take into account continuous variables which are not normally distributed. The aim of this paper is to introduce the more appropriate conditional phase-type (C-Ph) distribution for representing a continuous non-normal variable while also incorporating the causal information in the form of a Bayesian network.