Streaming Elements for FPGA Signal and Image Processing Accelerators

Field programmable gate array devices boast abundant resources with which custom accelerator components for signal, image and data processing may be realised; however, realising high performance, low cost accelerators currently demands manual register transfer level design. Software-programmable ’soft’ processors have been proposed as a way to reduce this design burden but they are unable to support performance and cost comparable to custom circuits. This paper proposes a new soft processing approach for FPGA which promises to overcome this barrier. A high performance, fine-grained streaming processor, known as a Streaming Accelerator Element, is proposed which realises accelerators as large scale custom multicore networks. By adopting a streaming execution approach with advanced program control and memory addressing capabilities, typical program inefficiencies can be almost completely eliminated to enable performance and cost which are unprecedented amongst software-programmable solutions. When used to realise accelerators for fast fourier transform, motion estimation, matrix multiplication and sobel edge detection it is shown how the proposed architecture enables real-time performance and with performance and cost comparable with hand-crafted custom circuit accelerators and up to two orders of magnitude beyond existing soft processors.

A Steganalysis System Utilizing Temporal Pixel Correlation Of HEVC Video

High Efficiency Video Coding (HEVC) is the most recent video codec coming after currently most popular H.264/MPEG4 codecs and has promising compression capabilities. It is conjectured that it will be a substitute for current video compression standards. However, to the best knowledge of the authors, none of the current video steganalysis methods designed or tested with HEVC video. In this paper, pixel domain steganography applied on HEVC video is targeted for the first time. Also, its the first paper that employs accordion unfolding transformation, which merges temporal and spatial correlation, in pixel domain video steganalysis. With help of the transformation, temporal correlation is incorporated into the system. Its demonstrated for three different feature sets that integrating temporal dependency substantially increased the detection accuracy.

FPGA based Streaming Image Processing Processor (SIPPro) architecture to accelerate data intensive applications

Current data-intensive image processing applications push traditional embedded architectures to their limits. FPGA based hardware acceleration is a potential solution but the programmability gap and time consuming HDL design flow is significant. The proposed research approach to develop “FPGA based programmable hardware acceleration platform” that uses, large number of Streaming Image processing Processors (SIPPro) potentially addresses these issues. SIPPro is pipelined in-order soft-core processor architecture with specific optimisations for image processing applications. Each SIPPro core uses 1 DSP48, 2 Block RAMs and 370 slice-registers, making the processor as compact as possible whilst maintaining flexibility and programmability. It is area efficient, scalable and high performance softcore architecture capable of delivering 530 MIPS per core using Xilinx Zynq SoC (ZC7Z020-3). To evaluate the feasibility of the proposed architecture, a Traffic Sign Recognition (TSR) algorithm has been prototyped on a Zedboard with the color and morphology operations accelerated using multiple SIPPros. Simulation and experimental results demonstrate that the processing platform is able to achieve a speedup of 15 and 33 times for color filtering and morphology operations respectively, with a significant reduced design effort and time.

Neurology assessment by objective structured video examination (OSVE)

Summary
Background
The ability to carry out a neurological examination and make an appropriate differential diagnosis is one of the mainstays of our final Bachelor of Medicine (MB) exam; however, with the introduction of objective structured clinical examinations (OSCEs) it has become impossible to arrange for adequate numbers of suitable real patients to participate in the exam.

Context
It is vital that newly qualified doctors can perform a basic neurological examination, interpret the physical signs and formulate a differential diagnosis.

It is vital that newly qualified doctors can perform a basic neurological examination

Innovation
Since 2010 we have introduced an objective structured video examination (OSVE) of a neurological examination of a real patient as part of our final MB OSCE exam. The students view clips of parts of the examination process. They answer questions on the signs that are demonstrated and formulate a differential diagnosis.

Implications
This type of station is logistically a lot easier to organise than a large number of real patients at different examination sites. The featured patients have clearly demonstrated signs and, as every student sees the same patient, are perfectly standardised. It is highly acceptable to examiners and performed well as an assessment tool. There are, however, certain drawbacks in that we are not examining the student's examination technique or their interaction with the patient. Also, certain signs, in particular the assessment of muscle tone and power, are more difficult for a student to estimate in this situation

Recurrent Convolutional Network for Video-based Person Re-Identification

In this paper we propose a novel recurrent neural networkarchitecture for video-based person re-identification.Given the video sequence of a person, features are extracted from each frame using a convolutional neural network that incorporates a recurrent final layer, which allows information to flow between time-steps. The features from all time steps are then combined using temporal pooling to give an overall appearance feature for the complete sequence. The convolutional network, recurrent layer, and temporal pooling layer, are jointly trained to act as a feature extractor for video-based re-identification using a Siamese network architecture.Our approach makes use of colour and optical flow information in order to capture appearance and motion information which is useful for video re-identification. Experiments are conduced on the iLIDS-VID and PRID-2011 datasets to show that this approach outperforms existing methods of video-based re-identification.

https://github.com/niallmcl/Recurrent-Convolutional-Video-ReID
Project Source Code

Spatio-temporal Rich Model Based Video Steganalysis on Cross Sections of Motion Vector Planes

A rich model based motion vector steganalysis benefiting from both temporal and spatial correlations of motion vectors is proposed in this work. The proposed steganalysis method has a substantially superior detection accuracy than the previous methods, even the targeted ones. The improvement in detection accuracy lies in several novel approaches introduced in this work. Firstly, it is shown that there is a strong correlation, not only spatially but also temporally, among neighbouring motion vectors for longer distances. Therefore, temporal motion vector dependency along side the spatial dependency is utilized for rigorous motion vector steganalysis. Secondly, unlike the filters previously used, which were heuristically designed against a specific motion vector steganography, a diverse set of many filters which can capture aberrations introduced by various motion vector steganography methods is used. The variety and also the number of the filter kernels are substantially more than that of used in previous ones. Besides that, filters up to fifth order are employed whereas the previous methods use at most second order filters. As a result of these, the proposed system captures various decorrelations in a wide spatio-temporal range and provides a better cover model. The proposed method is tested against the most prominent motion vector steganalysis and steganography methods. To the best knowledge of the authors, the experiments section has the most comprehensive tests in motion vector steganalysis field including five stego and seven steganalysis methods. Test results show that the proposed method yields around 20% detection accuracy increase in low payloads and 5% in higher payloads.

The player and video game interplay in the gameplay experience construct

Among the many discussions and studies related to video games, one of the most recurrent, widely debated and important relates to the experience of playing video games. The gameplay experience – as appropriated in this study – is the result of the interplay between two essential elements: a video game and a player. Existing studies have explored the resulting experience of video game playing from the perspective of the video game or the player, but none appear to equally balance both of these elements. The study presented here contributes to the ongoing debate with a gameplay experience model. The proposed model, which looks to equally balance the video game and the player elements, considers the gameplay experience to be both an interactive experience (related to the process of playing the video game) and an emotional experience (related to the outcome of playing the video game). The mutual influence of these two experiences during video game play ultimately defines the gameplay experience. To this gameplay experience contributes several dimensions, related to both the video game and player: the video game includes a mechanics, interface and narrative dimension; the player includes a motivations, expectations and background dimension. Also, the gameplay experience is initially defined by a gameplay situation, conditioned by an ambient in which gameplay takes place and a platform on which the video game is played. In order to initially validate the proposed model and attempt to show a relationship among the multiple model dimensions, a multi-case study was carried out using two different video games and player samples. In one study, results show significant correlations between multiple model dimensions, and evidence that video game related changes influence player motivations as well as player visual behavior. In specific player related analysis, results show that while players may be different in terms of background and expectations regarding the game, their motivation to play are not necessarily different, even if their performance in the game is weak. While further validation is necessary, this model not only contributes to the gameplay experience debate, but also demonstrates in a given context how player and video game dimensions evolve during video game play.

An Empirical Evaluation of the Reversal Theory State Measure: Tracking Changes in the Experience of Video Commercials of Three Leading Running Brands

Abstract of paper delivered at the 17th International Reversal Theory Conference, Day 3, session 4, 01.07.15

An Empirical Evaluation of the Reversal Theory State Measure Using Three Running Brand Video Commercials

This paper reports on the first known empirical use of the Reversal Theory State Measure (RTSM) since its publication by Desselles et al. (2014). The RTSM was employed to track responses to three purposely-selected video commercials in a between-subjects design. Results of the study provide empirical support for the central conceptual premise of reversal theory, the experience of metamotivational reversals and the ability of the RTSM to capture them. The RTSM was also found to be psychometrically sound after adjustments were made to two of its three component subscales. Detailed account and rationale is provided for the analytical process of assessing the psychometric robustness of the RTSM, with a number of techniques and interpretations relating to component structure and reliability discussed. Agreeability and critique of the two available versions of the RTSM – the bundled and the branched – is also examined. Researchers are encouraged to assist development of the RTSM through further use, taking into account the analysis and recommendations presented.

Radio over fiber broadband access networks architectures based on wavelength division multiplexing techniques

The recent remarkable growth in bandwidth of both wired optical and wireless access networks supports a burst of new high bandwidth Internet applications such as: peer-topeer file sharing, cloud storage, on-line gaming, video streaming, etc. Within this scenario, the convergence of fixed and wireless access networks offers significant opportunities for network operators to satisfy user demands, and simultaneously reduce the cost of implementing and running separated wireless and wired networks. The integration of wired and wireless network can be accomplished within several scenarios and at several levels. In this thesis we will focus on converged radio over fiber architectures, particularly on two application scenarios: converged optical 60 GHz wireless networks and wireless overlay backhauling over bidirectional colorless wavelength division multiplexing passive optical networks (WDM-PONs). In the first application scenario, optical 60 GHz signal generation using external modulation of an optical carrier by means of lithium niobate (LiNbO3) Mach- Zehnder modulators (MZM) is considered. The performance of different optical modulation techniques, robust against fiber dispersion is assessed and dispersion mitigation strategies are identified. The study is extended to 60 GHz carriers digitally modulated with data and to systems employing subcarrier multiplexed (SCM) mm-wave channels. In the second application scenario, the performance of WDM-PONs employing reflective semiconductor optical amplifiers (RSOAs), transmitting an overlay orthogonal frequency-division multiplexing (OFDM) wireless signal is assessed analytically and experimentally, with the relevant system impairments being identified. It is demonstrated that the intermodulation due to the beating of the baseband signal and wireless signal at the receiver can seriously impair the wireless channel. Performance degradation of the wireless channel caused by the RSOA gain modulation owing to the downstream baseband data is also assessed, and system design guidelines are provided.

Evaluation of the H.264 scalable video coding in error prone IP networks

The Joint Video Team, composed by the ISO/IEC Moving Picture Experts Group (MPEG) and the ITU-T Video Coding Experts Group (VCEG), has standardized a scalable extension of the H.264/AVC video coding standard called Scalable Video Coding (SVC). H.264/SVC provides scalable video streams which are composed by a base layer and one or more enhancement layers. Enhancement layers may improve the temporal, the spatial or the signal-to-noise ratio resolutions of the content represented by the lower layers. One of the applications, of this standard is related to video transmission in both wired and wireless communication systems, and it is therefore important to analyze in which way packet losses contribute to the degradation of quality, and which mechanisms could be used to improve that quality. This paper provides an analysis and evaluation of H.264/SVC in error prone environments, quantifying the degradation caused by packet losses in the decoded video. It also proposes and analyzes the consequences of QoS-based discarding of packets through different marking solutions.

BitTorrent based transmission of real-time scalable video over P2P networks

The number of software applications available on the Internet for distributing video streams in real time over P2P networks has grown quickly in the last two years. Typical this kind of distribution is made by television channel broadcasters which try to make their content globally available, using viewer's resources to support a large scale distribution of video without incurring in incremental costs. However, the lack of adaptation in video quality, combined with the lack of a standard protocol for this kind of multimedia distribution has driven content providers to basically ignore it as a solution for video delivery over the Internet. While the scalable extension of the H. 264 encoding (H.264/SVC) can be used to support terminal and network heterogeneity, it is not clear how it can be integrated in a P2P overlay to form a large scale and real time distribution. In this paper, we start by defining a solution that combines the most popular P2P file-sharing protocol, the BitTorrent, with the H. 264/SVC encoding for a real-time video content delivery. Using this solution we then evaluate the effect of several parameters in the quality received by peers.

Comparing segmentation strategies for efficient video passage retrieval

We compare the effect of different text segmentation strategies on speech based passage retrieval of video. Passage retrieval has mainly been studied to improve document retrieval and to enable question answering. In these domains best results were obtained using passages defined by the paragraph structure of the source documents or by using arbitrary overlapping passages. For the retrieval of relevant passages in a video, using speech transcripts, no author defined segmentation is available. We compare retrieval results from 4 different types of segments based on the speech channel of the video: fixed length segments, a sliding window, semantically coherent segments and prosodic segments. We evaluated the methods on the corpus of the MediaEval 2011 Rich Speech Retrieval task. Our main conclusion is that the retrieval results highly depend on the right choice for the segment length. However, results using the segmentation into semantically coherent parts depend much less on the segment length. Especially, the quality of fixed length and sliding window segmentation drops fast when the segment length increases, while quality of the semantically coherent segments is much more stable. Thus, if coherent segments are defined, longer segments can be used and consequently less segments have to be considered at retrieval time.

Designing personalized video-based crossmedia informal learning environments beyond iTV

Tese de doutoramento, Informática (Engenharia Informática), Universidade de Lisboa, Faculdade de Ciências, 2014