3D Conversion using Vanishing Points and Image Warping

We describe a user assisted technique for 3D stereo conversion from 2D images. Our approach exploits the geometric structure of perspective images including vanishing points. We allow a user to indicate lines, planes, and vanishing points in the input image, and directly employ these as constraints in an image warping framework to produce a stereo pair. By sidestepping explicit construction of a depth map, our approach is applicable to more general scenes and avoids potential artifacts of depth-image-based rendering. Our method is most suitable for scenes with large scale structures such as buildings.

A smart multi-hop hierarchical routing protocol for efficient video communication over wireless multimedia sensor networks

For smart applications, nodes in wireless multimedia sensor networks (MWSNs) have to take decisions based on sensed scalar physical measurements. A routing protocol must provide the multimedia delivery with quality level support and be energy-efficient for large-scale networks. With this goal in mind, this paper proposes a smart Multi-hop hierarchical routing protocol for Efficient VIdeo communication (MEVI). MEVI combines an opportunistic scheme to create clusters, a cross-layer solution to select routes based on network conditions, and a smart solution to trigger multimedia transmission according to sensed data. Simulations were conducted to show the benefits of MEVI compared with the well-known Low-Energy Adaptive Clustering Hierarchy (LEACH) protocol. This paper includes an analysis of the signaling overhead, energy-efficiency, and video quality.

QoE-aware Multiple Path Video Transmission for Wireless Multimedia Sensor Networks

Wireless Multimedia Sensor Networks (WMSNs) promise a wide scope of emerging potential applications in both civilian and military areas, which require visual and audio information to enhance the level of collected information. The transmission of multimedia content requires a minimal video quality level from the user’s perspective. However, links in WMSN communi- cations are typically unreliable, as they often experience fluctuations in quality and weak connectivity, and thus, the routing protocol must evaluate the routes by using end-to-end link quality information to increase the packet delivery ratio. Moreover, the use multiple paths together with key video metrics can enhance the video quality level. In this paper, we propose a video-aware multiple path hierarchical routing protocol for efficient multimedia transmission over WMSN, called video-aware MMtransmission. This protocol finds node-disjoint multiple paths, and implements an end-to-end link quality estimation with minimal over- head to score the paths. Thus, our protocol assures multimedia transmission with Quality of Experience (QoE) and energy-efficiency support. The simula- tion results show the benefits of video-aware MMtransmission for disseminating video content by means of energy-efficiency and QoE analysis.

An OMNeT++ Framework to Evaluate Video Transmission in Mobile Wireless Multimedia Sensor Networks

The development and evaluation of new algorithms and protocols for Wireless Multimedia Sensor Networks (WMSNs) are usually supported by means of a discrete event network simulator, where OMNeT++ is one of the most important ones. However, experiments involving multimedia transmission, video flows with different characteristics, genres, group of pictures lengths, and coding techniques must be evaluated based also on Quality of Experience (QoE) metrics to reflect the user's perception. Such experiments require the evaluation of video-related information, i.e., frame type, received/lost, delay, jitter, decoding errors, as well as inter and intra-frame dependency of received/distorted videos. However, existing OMNeT++ frameworks for WMSNs do not support video transmissions with QoE-awareness, neither a large set of mobility traces to enable evaluations under different multimedia/mobile situations. In this paper, we propose a Mobile MultiMedia Wireless Sensor Network OMNeT++ framework (M3WSN) to support transmission, control and evaluation of real video sequences in mobile WMSNs.

A real-time video quality estimator for emerging wireless multimedia systems

Wireless Mesh Networks (WMNs) are increasingly deployed to enable thousands of users to share, create, and access live video streaming with different characteristics and content, such as video surveillance and football matches. In this context, there is a need for new mechanisms for assessing the quality level of videos because operators are seeking to control their delivery process and optimize their network resources, while increasing the user’s satisfaction. However, the development of in-service and non-intrusive Quality of Experience assessment schemes for real-time Internet videos with different complexity and motion levels, Group of Picture lengths, and characteristics, remains a significant challenge. To address this issue, this article proposes a non-intrusive parametric real-time video quality estimator, called MultiQoE that correlates wireless networks’ impairments, videos’ characteristics, and users’ perception into a predicted Mean Opinion Score. An instance of MultiQoE was implemented in WMNs and performance evaluation results demonstrate the efficiency and accuracy of MultiQoE in predicting the user’s perception of live video streaming services when compared to subjective, objective, and well-known parametric solutions.

Adaptive Beaconless Opportunistic Routing for Multimedia Distribution

User experience on watching live videos must be satisfactory even under the inuence of different network conditions and topology changes, such as happening in Flying Ad-Hoc Networks (FANETs). Routing services for video dissemination over FANETs must be able to adapt routing decisions at runtime to meet Quality of Experience (QoE) requirements. In this paper, we introduce an adaptive beaconless opportunistic routing protocol for video dissemination over FANETs with QoE support, by taking into account multiple types of context information, such as link quality, residual energy, buffer state, as well as geographic information and node mobility in a 3D space. The proposed protocol takes into account Bayesian networks to define weight vectors and Analytic Hierarchy Process (AHP) to adjust the degree of importance for the context information based on instantaneous values. It also includes a position prediction to monitor the distance between two nodes in order to detect possible route failure.

Reproducibility and validity of video screen measurements of gait in children with spastic cerebral palsy

PURPOSE: To determine the reproducibility and validity of video screen measurement (VSM) of sagittal plane joint angles during gait. METHODS: 17 children with spastic cerebral palsy walked on a 10m walkway. Videos were recorded and 3d-instrumented gait analysis was performed. Two investigators measured six sagittal joint/segment angles (shank, ankle, knee, hip, pelvis, and trunk) using a custom-made software package. The intra- and interrater reproducibility were expressed by the intraclass correlation coefficient (ICC), standard error of measurements (SEM) and smallest detectable difference (SDD). The agreement between VSM and 3d joint angles was illustrated by Bland-Altman plots and limits of agreement (LoA). RESULTS: Regarding the intrarater reproducibility of VSM, the ICC ranged from 0.99 (shank) to 0.58 (trunk), the SEM from 0.81 degrees (shank) to 5.97 degrees (trunk) and the SDD from 1.80 degrees (shank) to 16.55 degrees (trunk). Regarding the interrater reproducibility, the ICC ranged from 0.99 (shank) to 0.48 (trunk), the SEM from 0.70 degrees (shank) to 6.78 degrees (trunk) and the SDD from 1.95 degrees (shank) to 18.8 degrees (trunk). The LoA between VSM and 3d data ranged from 0.4+/-13.4 degrees (knee extension stance) to 12.0+/-14.6 degrees (ankle dorsiflexion swing). CONCLUSION: When performed by the same observer, VSM mostly allows the detection of relevant changes after an intervention. However, VSM angles differ from 3d-IGA and do not reflect the real sagittal joint position, probably due to the additional movements in the other planes.

Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy

PURPOSE: The aim of this study is to implement augmented reality in real-time image-guided interstitial brachytherapy to allow an intuitive real-time intraoperative orientation. METHODS AND MATERIALS: The developed system consists of a common video projector, two high-resolution charge coupled device cameras, and an off-the-shelf notebook. The projector was used as a scanning device by projecting coded-light patterns to register the patient and superimpose the operating field with planning data and additional information in arbitrary colors. Subsequent movements of the nonfixed patient were detected by means of stereoscopically tracking passive markers attached to the patient. RESULTS: In a first clinical study, we evaluated the whole process chain from image acquisition to data projection and determined overall accuracy with 10 patients undergoing implantation. The described method enabled the surgeon to visualize planning data on top of any preoperatively segmented and triangulated surface (skin) with direct line of sight during the operation. Furthermore, the tracking system allowed dynamic adjustment of the data to the patient's current position and therefore eliminated the need for rigid fixation. Because of soft-part displacement, we obtained an average deviation of 1.1 mm by moving the patient, whereas changing the projector's position resulted in an average deviation of 0.9 mm. Mean deviation of all needles of an implant was 1.4 mm (range, 0.3-2.7 mm). CONCLUSIONS: The developed low-cost augmented-reality system proved to be accurate and feasible in interstitial brachytherapy. The system meets clinical demands and enables intuitive real-time intraoperative orientation and monitoring of needle implantation.

Internet video telephony allows speech reading by deaf individuals and improves speech perception by cochlear implant users

OBJECTIVE To analyze speech reading through Internet video calls by profoundly hearing-impaired individuals and cochlear implant (CI) users. METHODS Speech reading skills of 14 deaf adults and 21 CI users were assessed using the Hochmair Schulz Moser (HSM) sentence test. We presented video simulations using different video resolutions (1280 × 720, 640 × 480, 320 × 240, 160 × 120 px), frame rates (30, 20, 10, 7, 5 frames per second (fps)), speech velocities (three different speakers), webcameras (Logitech Pro9000, C600 and C500) and image/sound delays (0-500 ms). All video simulations were presented with and without sound and in two screen sizes. Additionally, scores for live Skype™ video connection and live face-to-face communication were assessed. RESULTS Higher frame rate (>7 fps), higher camera resolution (>640 × 480 px) and shorter picture/sound delay (<100 ms) were associated with increased speech perception scores. Scores were strongly dependent on the speaker but were not influenced by physical properties of the camera optics or the full screen mode. There is a significant median gain of +8.5%pts (p = 0.009) in speech perception for all 21 CI-users if visual cues are additionally shown. CI users with poor open set speech perception scores (n = 11) showed the greatest benefit under combined audio-visual presentation (median speech perception +11.8%pts, p = 0.032). CONCLUSION Webcameras have the potential to improve telecommunication of hearing-impaired individuals.

Assessment of a robust opportunistic routing for video transmission in dynamic topologies

Mobile multimedia ad hoc services run on dynamic topologies due to node mobility or failures and wireless channel impairments. A robust routing service must adapt to topology changes with the aim of recovering or maintaining the video quality level and reducing the impact of the user's experience. In those scenarios, beacon-less Opportunistic Routing (OR) increases the robustness by supporting routing decisions in a completely distributed manner based on protocol-specific characteristics. However, the existing beacon-less OR approaches do not efficiently combine multiple metrics for forwarding selection, which cause higher packet loss rate, and consequently reduce the video quality level. In this paper, we assess the robustness and reliability of our recently developed OR protocol under node failures, called cross-layer Link quality and Geographical-aware OR protocol (LinGO). Simulation results show that LinGO achieves multimedia dissemination with QoE support and robustness in scenarios with dynamic topologies.