Crowdsourcing based subjective quality assessment of adaptive video streaming

In order to cater for user's quality of experience (QoE) requirements, HTTP adaptive streaming (HAS) based solutions of video services have become popular recently. User QoE feedback can be instrumental in improving the capabilities of such services. Perceptual quality experiments that involve humans are considered to be the most valid method of the assessment of QoE. Besides lab-based subjective experiments, crowdsourcing based subjective assessment of video quality is gaining popularity as an alternative method. This paper presents insights into a study that investigates perceptual preferences of various adaptive video streaming scenarios through crowdsourcing based subjective quality assessment.

Subjective quality study of adaptive streaming of monoscopic and stereoscopic video

Nowadays, HTTP adaptive streaming (HAS) has become a reliable distribution technology offering significant advantages in terms of both user perceived Quality of Experience (QoE) and resource utilization for content and network service providers. By trading-off the video quality, HAS is able to adapt to the available bandwidth and display requirements so that it can deliver the video content to a variety of devices over the Internet. However, until now there is not enough knowledge of how the adaptation techniques affect the end user's visual experience. Therefore, this paper presents a comparative analysis of different bitrate adaptation strategies in adaptive streaming of monoscopic and stereoscopic video. This has been done through a subjective experiment of testing the end-user response to the video quality variations, considering the visual comfort issue. The experimental outcomes have made a good insight into the factors that can influence on the QoE of different adaptation strategies.

Evidence of the presence of bias in subjective metrics: analysis within a family of experiments

Context: Measurement is crucial and important to empirical software engineering. Although reliability and validity are two important properties warranting consideration in measurement processes, they may be influenced by random or systematic error (bias) depending on which metric is used. Aim: Check whether, the simple subjective metrics used in empirical software engineering studies are prone to bias. Method: Comparison of the reliability of a family of empirical studies on requirements elicitation that explore the same phenomenon using different design types and objective and subjective metrics. Results: The objectively measured variables (experience and knowledge) tend to achieve more reliable results, whereas subjective metrics using Likert scales (expertise and familiarity) tend to be influenced by systematic error or bias. Conclusions: Studies that predominantly use variables measured subjectively, like opinion polls or expert opinion acquisition.

Subjective QoE Analysis of HTTP Adaptive Streaming Applications

Actualmente la optimization de la calidad de experiencia (Quality of Experience- QoE) de HTTP Adaptive Streaming (HAS) de video recibe una atención creciente. Este incremento de interés proviene fundamentalmente de las carencias de las soluciones actuales HAS, que, al no ser QoE-driven, no incluyen la percepción de la calidad de los usuarios finales como una parte integral de la lógica de adaptación. Por lo tanto, la obtención de información de referencia fiable en QoE en HAS presenta retos importantes, ya que las metodologías de evaluación subjetiva de la calidad de vídeo propuestas en las normas actuales no son adecuadas para tratar con la variación temporal de la calidad que es consustancial de HAS. Esta tesis investiga la influencia de la adaptación dinámica en la calidad de la transmisión de vídeo considerando métodos de evaluación subjetiva. Tras un estudio exhaustivo del estado del arte en la evaluación subjetiva de QoE en HAS, se han resaltado los retos asociados y las líneas de investigación abiertas. Como resultado, se han seleccionado dos líneas principales de investigación: el análisis del impacto en la QoE de los parámetros de las técnicas de adaptación y la investigación de las metodologías de prueba subjetiva adecuada para evaluación de QoE en HAS. Se han llevado a cabo un conjunto de experimentos de laboratorio para investigar las cuestiones planteadas mediante la utilización de diferentes metodologáas para pruebas subjetivas. El análisis estadístico muestra que no son robustas todas las suposiciones y reivindicaciones de las referencias analizadas, en particular en lo que respecta al impacto en la QoE de la frecuencia de las variaciones de calidad, de las adaptaciones suaves o abruptas y de las oscilaciones de calidad. Por otra parte, nuestros resultados confirman la influencia de otros parámetros, como la longitud de los segmentos de vídeo y la amplitud de las oscilaciones de calidad. Los resultados también muestran que tomar en consideración las características objetivas de los contenidos puede ser beneficioso para la mejora de la QoE en HAS. Además, todos los resultados han sido validados mediante extensos análisis experimentales que han incluido estudio tanto en otros laboratorios como en crowdsourcing Por último, sobre los aspectos metodológicos de las pruebas subjetivas de QoE, se ha realizado la comparación entre los resultados experimentales obtenidos a partir de un método estandarizado basado en estímulos cortos (ACR) y un método semi continuo (desarrollado para la evaluación de secuencias prolongadas de vídeo). A pesar de algunas diferencias, el resultado de los análisis estadísticos no muestra ningún efecto significativo de la metodología de prueba. Asimismo, aunque se percibe la influencia de la presencia de audio en la evaluación de degradaciones del vídeo, no se han encontrado efectos estadísticamente significativos de dicha presencia. A partir de la ausencia de influencia del método de prueba y de la presencia de audio, se ha realizado un análisis adicional sobre el impacto de realizar comparaciones estadísticas múltiples en niveles estadísticos de importancia que aumentan la probabilidad de los errores de tipo-I (falsos positivos). Nuestros resultados muestran que, para obtener un efectos sólido en el análisis estadístico de los resultados subjetivos, es necesario aumentar el número de sujetos de las pruebas claramente por encima de los tamaños de muestras propuestos por las normas y recomendaciones actuales. ABSTRACT Optimizing the Quality of Experience (QoE) of HTTP adaptive video streaming (HAS) is receiving increasing attention nowadays. The growth of interest is mainly caused by the fact that current HAS solutions are not QoE-driven, i.e. end-user quality perception is not integral part of the adaptation logic. However, obtaining the necessary reliable ground truths on HAS QoE faces substantial challenges, since the subjective video quality assessment methodologies as proposed by current standards are not well-suited for dealing with the time-varying quality properties that are characteristic for HAS. This thesis investigates the influence of dynamic quality adaptation on the QoE of streaming video by means of subjective evaluation approaches. Based on a comprehensive survey of related work on subjective HAS QoE assessment, the related challenges and open research questions are highlighted and discussed. As a result, two main research directions are selected for further investigation: analysis of the QoE impact of different technical adaptation parameters, and investigation of testing methodologies suitable for HAS QoE evaluation. In order to investigate related research issues and questions, a set of laboratory experiments have been conducted using different subjective testing methodologies. Our statistical analysis demonstrates that not all assumptions and claims reported in the literature are robust, particularly as regards the QoE impact of switching frequency, smooth vs. abrupt switching, and quality oscillation. On the other hand, our results confirm the influence of some other parameters such as chunk length and switching amplitude on perceived quality. We also show that taking the objective characteristics of the content into account can be beneficial to improve the adaptation viewing experience. In addition, all aforementioned findings are validated by means of an extensive cross-experimental analysis that involves external laboratory and crowdsourcing studies. Finally, to address the methodological aspects of subjective QoE testing, a comparison between the experimental results obtained from a (short stimuli-based) ACR standardized method and a semi-continuous method (developed for assessment of long video sequences) has been performed. In spite of observation of some differences, the result of statistical analysis does not show any significant effect of testing methodology. Similarly, although the influence of audio presence on evaluation of video-related degradations is perceived, no statistically significant effect of audio presence could be found. Motivating by this finding (no effect of testing method and audio presence), a subsequent analysis has been performed investigating the impact of performing multiple statistical comparisons on statistical levels of significance which increase the likelihood of Type-I errors (false positives). Our results show that in order to obtain a strong effect from the statistical analysis of the subjective results, it is necessary to increase the number of test subjects well beyond the sample sizes proposed by current quality assessment standards and recommendations.

Formación y determinación del valor del suelo: confrontación y discrepancias entre mecanismos de mercado y regulaciones administrativas

La determinación del valor del suelo implicado en los procesos de urbanización se mueve en una compleja dialéctica en la que algunos fenómenos de naturaleza morfológica como el modelado del espacio social, o el que se deduce de las determinaciones del planeamiento (aprovechamiento especialmente) se enfrentan a instrumentos de regulación que se rigen por normas de carácter administrativo (urbanísticas y expropiatorias, fiscales, financieras-hipotecarias etc.). En ese marco en el que las discrepancias son frecuentes se pretende analizar las posibles actuaciones encaminadas a influir sobre la formación de ese valor en el sentido de limitar las expectativas de aprovechamiento tradicionalmente vinculadas al sistema urbanístico establecido, destacando al respecto la implantación de criterios de sostenibilidad y participación ciudadana, así como que la utilización de normas administrativas de valoración con la finalidad de reducir tales expectativas y tratar de influir sobre el sistema urbanístico en su conjunto, no se considera suficiente para conseguir el logro de tales objetivos. Añadiendo, respecto del conjunto de normas administrativas de determinación del valor del suelo, que si bien las reglas urbanísticas y expropiatorias establecen actualmente reglas objetivas de valoración, a través de la normativa de valoración destinada a entidades financieras, o de carácter fiscal, se introducen criterios subjetivos que tratan de aproximarse al mercado, incluso en el caso del valor catastral, estructuralmente reglado. Esta separación de criterios ha llevado a un enfrentamiento al que ha venido a dar respuesta la reciente sentencia del Tribunal Supremo de 30 de mayo de 2014, que impone los criterios urbanísticos de consideración de situaciones básicas de suelo, frente a los fiscalmente regulados en relación con la normativa catastral en cuanto a la consideración de suelo urbanizable sin desarrollar, que pasa a ser considerado como en situación rural, siendo por tanto aplicables sobre el mismo las reglas de valoración que la normativa catastral establece para esta categoría de suelo (rústico) que no considera, incluso con la posterior modificación, de junio de 2015, incorporada para adaptarse a la mencionada sentencia, expectativas de aprovechamiento. Pero como se ha señalado, la exclusiva utilización de la normativa urbanística de valoración no se considera suficiente para evitar los efectos que sobre el valor de suelo produce la asignación de tales aprovechamientos, por lo que acuerdo con la actual legislación de suelo, deben asimismo potenciarse actuaciones encaminadas a un desarrollo sostenible así como a la participación ciudadana, la transparencia y el control, con la finalidad de lograr un mayor rigor en la utilización del suelo que ha de traducirse en su valoración. ABSTRACT Determining the land value involved in the urbanisation process implies a complex dialectic in which certain morphological natural phenomena like the modelling of social space or that deduced from planning determinations (especially land use) face regulatory instruments governed by administrative norms (urban planning and expropriator, tax, financial-mortgage etc.). In this context where discrepancies are common, the purpose is to analyse the possible actions geared to influence the formation of that value in the sense of limiting land use expectations traditionally linked to the established urban planning system, highlighting the implementation of sustainability criteria and citizen participation, as well as to consider that the administrative standards of evaluation for reducing said expectations, and trying to influence the urban system as a whole, are not considered sufficient for achieving these objectives. We might add, with respect to the set of administrative norms for determining the land value, that while urban and expropriations norms currently provide objective rules of valuation, subjective criteria are introduced trying to approach the market, including the structurally regulated cadastral value through valuation rules aimed at financial or fiscal institutions. This separation of criteria has led to a confrontation to which the recent ruling of the Supreme Court of May 30, 2014 responds. The aforementioned judgement imposes the urban core criteria for basic land use situations against those fiscally regulated in relation to cadastral regulations regarding the consideration of undeveloped building land, which is considered to be in a rural situation, making it subject to the same rules that cadastral valuation norms established for this category of land use (rustic) which do not consider, even with the subsequent amendment of June 2015, introduced to fit that judgment, use expectations. But as noted, the exclusive use of valuation planning regulations is not considered sufficient to avoid the effects on the land value produced by the allocation of said uses, so that according to the current land use legislation, actions aimed at sustainable development as well as citizen participation, transparency and control, shall be enhanced in order to achieve greater rigor in the use of land which should be translated in their valuation.

Quality of experience of adaptive video streaming: investigation in service parameters and subjective quality assessment methodology

The usage of HTTP adaptive streaming (HAS) has become widely spread in multimedia services. Because it allows the service providers to improve the network resource utilization and user׳s Quality of Experience (QoE). Using this technology, the video playback interruption is reduced since the network and server status in addition to capability of user device, all are taken into account by HAS client to adapt the quality to the current condition. Adaptation can be done using different strategies. In order to provide optimal QoE, the perceptual impact of adaptation strategies from point of view of the user should be studied. However, the time-varying video quality due to the adaptation which usually takes place in a long interval introduces a new type of impairment making the subjective evaluation of adaptive streaming system challenging. The contribution of this paper is two-fold: first, it investigates the testing methodology to evaluate HAS QoE by comparing the subjective experimental outcomes obtained from ACR standardized method and a semi-continuous method developed to evaluate the long sequences. In addition, influence of using audiovisual stimuli to evaluate the video-related impairment is inquired. Second, impact of some of the adaptation technical factors including the quality switching amplitude and chunk size in combination with high range of commercial content type is investigated. The results of this study provide a good insight toward achieving appropriate testing method to evaluate HAS QoE, in addition to designing switching strategies with optimal visual quality.

Subjective comparison of consumer television technologies for 3D visualization

This paper presents a comparison among different consumer 3D display technologies by means of a subjective assessment test. Therefore, four 55-in displays have been considered: one autostereoscopic display, one stereoscopic with polarized passive glasses, and two with active shutter glasses. In addition, a high-quality 3D video database has been used to show diverse material with both views in high definition. To carry out the test, standard recommendations have been followed considering also some modifications looking for a test environment more similar to real home viewing conditions, with the objective of obtaining more representative conclusions. Moreover, several perceptual factors have been considered to study the performance of the displays, such as picture quality, depth perception, and visual discomfort. The obtained results show interesting issues, like the performance improvement of active shutter glasses technology, the high performance of the polarized glasses technology in terms of quality and comfort, and the need of improvement of the autostereoscopic displays to complement the visual comfort to reach a global high-quality visual experience.