Modelling dependency in multimedia streams

Abstract

Expressing and analysing data dependency in multimedia streams is promising, since content-aware policies at a transport level would benefit from such services. In this paper we present a format-independent dependency model aimed at specifying, validating and reasoning about structural dependency in multimedia streams. Based on this model, we developed a universal dependency description language and a dependency validation service to serve as an infrastructure for content-aware transport layers. Driven by application knowledge, this special form of a cross-layer design enables lower layers to reason about the impact of data loss and drops during transmission while being unaware of the real data format.We outline, how this infrastructure can be used to build content-aware error protection policies and explain how applications need to specify dependency and prepare media streams in order to gain benefits from those policies. While costs and benefits of a dependency model are only quantifiable in conjunction with special policies, we report on the general worst-case costs of our model here.

What Are You Trying to Say? Format-Independent Semantic-Aware Streaming and Delivery

“…As will be seen in Section 4.1, a considerable number of algorithms have been developed for optimising the Rate-Distortion (R-D) performance of multimedia delivery (Chakareski et al, 2004a;Chou, 2006;Cranley & Murphy, 2006;Eichhorn, 2006). These algorithms vary in their guarantees of tractability, complexity, and the range of metadata required as inputs to the process.…”

“…Consequently, Chakareski proposes a low-complexity approximation of the lagrangian optimisation problem, by ignoring interdependencies between Data Units and instead assuming that distortion from packet loss on subsequent packets is additive. Eichhorn (2006) suggests the opposite approximation: Chakareski ignores actual dependencies; Eichhorn ignores actual distortions, and asserts that dependency alone may be sufficient. Finally, Cranley & Murphy (2006) trade temporal resolution against spatial resolution and use subjective testing to arrive at a so-called Optimum Adaptation Trajectory.…”

Semantic-aware delivery of multimedia

“…Examples include the approaches by Chou [4], Chakareski [14], Eichhorn [15] and Cranley and Murphy [11]. Typically, these algorithms base rate-distortion optimization on error-probability cost functions, where errors are characterised in such a way as to encompass bit error rate, packet loss, and delay (such that the packet is too late to be useful).…”

Video-aware opportunistic network coding over wireless networks

“…(4) describes the probability that a packet, which is not lost, 2 This is an approximation as the actual distortion that may also depend on the delivery status of prior and subsequent NALs. The distortion model can be extended to capture these loss correlations [29]- [31]. Furthermore, we assume that distortions caused by loss of multiple packets are additive, which is reasonable for sparse losses.…”

“…Most distortion occurs in the first few frames after a loss and breaks after the next I frame; the error depends on the video content of subsequent frames and on the coding decisions. Another approach is to assign distortion values based solely on the GOP structure, ignoring the video content and coding decisions, or to use a model for dependencies [31].…”