Non-equilibrium information envelopes and the capacity-delay-error-tradeoff of source coding

Abstract

Abstract-This paper develops an envelope-based approach to establish a link between information and queueing theory. Unlike classical, equilibrium information theory, information envelopes focus on the dynamics of sources and coders, using functions of time that bound the number of bits generated. In the limit the information envelopes converge to the average behavior and recover the entropy of a source, respectively, the average codeword length of a coder. In contrast, on short time scales and for sources with memory it is shown that large deviations from known equilibrium results occur with nonnegligible probability. These can cause significant network delays. Compared to well-known traffic models from queueing theory, information envelopes consider the functioning of information sources and coders, avoiding a priori assumptions, such as exponential traffic, or empirical, trace-based traffic models. Using results from the stochastic network calculus, the envelopes yield a characterization of the operating points of source coders by the triplet of capacity, delay, and error. In the limit, assuming an optimal coder the required capacity approaches the entropy with arbitrarily small probability of error if infinitely large delays are permitted. We derive a corresponding characterization of channels and prove that the model has the desirable property of additivity, that allows analyzing coders and channels separately.

Capacity–Delay–Error Boundaries: A Composable Model of Sources and Systems

“…We also consider Markov sources, which we parameterize using empirical data from video compression. Further results for a variety of other coders can also be found in the technical report [27].…”

“…The operator ⊗ is referred to as the minplus convolution 1 . In this work, S(τ, t) is non-negative, nonincreasing in τ , non-decreasing in t, and S(t, t) = 0 for all t ≥ 0.…”

“…Statistical arrival envelopes [10], [13] are used to specify an upper bound of the arrival process, that may be exceeded at most with a defined probability ε A ∈ [0, 1]. The arrival 1 More commonly known is the min-plus convolution of univariate functions…”

“…Lübben is supported by a DFG grant (NetMeter) and N. Becker is supported by an ERC Starting Grant (UnIQue). Parts of this work appeared at IEEE WoWMoM 2012 [1] and at the IEEE Sarnoff Symposium 2012 [2]. networks, as it potentiates significant resource savings due to statistical multiplexing [4].…”

Service characterizations for multi-Hop multiaccess wireless networks

“…We view the channel capacity as a time varying function of the underlying instantaneous signal-to-noise ratio (SNR). Our work is related to [4] and [14], which also explore links between information theory and communication networks using network calculus. We note that a joint analysis of different multiaccess methods using network calculus does not exist.…”