Forward-Aware Information Bottleneck-Based Vector Quantization: Multiterminal Extensions for Parallel and Successive Retrieval

Abstract

Consider the following setup: Through a joint design, multiple observations of a remote data source shall be locally compressed before getting transmitted via several error-prone, rate-limited forward links to a (distant) processing unit. For addressing this specific instance of multiterminal Joint Source-Channel Coding problem, in this article, the foundational principle of the Information Bottleneck method is fully extended to obtain purely statistical design approaches, enjoying the Mutual Information as their fidelity criterion. Specifically, the forms of stationary points for two types of distributed compression schemes are characterized here. Subsequently, those acquired solutions are utilized as the centerpiece of the proposed generic, iterative algorithm, termed the Multiterminal Forward-Aware Vector Information Bottleneck (M-FAVIB), for addressing the design optimizations. Leveraging an unfolding trick, it will be proven that both distributed compression schemes fall into the category of Successive Upper-Bound Minimization, ensuring their convergence to a stationary point. Eventually, the effectiveness of the proposed compression schemes will be substantiated as well by means of numerical investigations over some typical transmission scenarios.

Semantic Information Recovery in Wireless Networks

“…Basically, we implement joint source-channel coding of messages conveying the semantic RV, but not differentiating between Levels A and B. We formulate the semantic communication design either as an Information Maximization or as an Information Bottleneck (IB) optimization problem [ 29 , 30 , 31 ]. Although the approach pursued here again leads to an IB problem as in [ 26 ], our article introduces a new classification and perspective of semantic communication and different ML-based solution approaches.…”

“…Therefore, we can formulate an optimization problem where we like to maximize the relevant information subject to the constraint to limit the compression rate to a maximum information rate : Problem ( 15 ) is an important variation of the InfoMax principle and called the Information Bottleneck (IB) problem [ 10 , 29 , 40 , 41 ]. The IB method introduced by Tishby et al [ 29 ] has been the subject of intensive research for years and has proven to be a suitable mathematical/information-theoretical framework for solving numerous problems—as well as in wireless communications [ 30 , 31 , 42 , 43 ]. Note that we aim for an encoder that compresses into a compact representation for discrete RVs by clustering and for continuous RVs by dimensionality reduction.…”

“…Basically, we implement joint source-channel coding of messages conveying the semantic RV, but not differentiating between Levels A and B. We formulate the semantic communication design either as an Information Maximization or as an Information Bottleneck (IB) optimization problem [29][30][31].…”

Information Bottleneck Signal Processing and Learning to Maximize Relevant Information for Communication Receivers

“…The applications studied in the communications context lead from the design of channel output quantizers [ 9 , 10 , 11 ] over the decoding of low-density parity-check (LDPC) codes [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 ] and polar codes [ 32 , 33 , 34 ] to entire receiver chains that include channel estimation and detection [ 35 , 36 , 37 , 38 ]. Moreover, the information bottleneck method has been applied in joint source–channel coding, forwarding and relaying applications [ 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 ] and in distributed sensor networks [ 49 , 50 , 51 , 52 , 53 ] successfully. Related works with a focus on inference with the distributiveness of data among multiple nodes and network learning aspects include [ 54 , 55 , 56 ].…”