Minimum Impurity Partitions

Burshtein, David; Pietra, V. Della; Kanevsky, Dimitri; Nádas, Arthur

doi:10.1214/aos/1176348789

Cited by 48 publications

(46 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Proof: The quantization problem considered here is an example of impurity-minimization partitions from machine learning [15]. Defining the backward channelP j|i = Pr(X = j|Y = i) for the given DMC P i|j and a fixed input distribution p j , the sorting condition (14) can also be expressed as…”

Section: B Necessary Condition For Optimalitymentioning

confidence: 99%

“…Theorem 1 in [15] states that, if the objective function is concave inT j|k = Pr(X = j|Z = k), then the optimal K partitions consist of K convex subsets of {P 1|i | i = 1, . .…”

Section: B Necessary Condition For Optimalitymentioning

confidence: 99%

“…While restricted to binaryinput DMCs, one can extend the claim of Lemma 3 to memoryless channels with any finite input alphabet and (possibly continuous) output alphabet using [15,Theorem1].…”

Section: Remarkmentioning

confidence: 99%

See 2 more Smart Citations

Channel quantizers that maximize random coding exponents for binary-input memoryless channels

Yagi

Kurkoski

2012

2012 IEEE International Conference on Communications (ICC)

View full text Add to dashboard Cite

The problem of finding the optimum output quantizer for a given discrete memoryless channel is investigated, where the quantizer output has fewer values than the channel output. While mutual information has received attention as an objective function for optimization, the focus of this paper is use of the random coding exponent, which was originally derived by Gallager, as criteria. Two problems are addressed, where one problem is a partial problem of the other. The main result is a quantizer design algorithm, and a proof that it finds the optimum quantizer in the partial problem. The quantizer design algorithm is based on a dynamic programming approach, and is an extension of a mutual-information maximization method. For the binary-input case, it is shown that the optimum quantizer can be found with complexity that is polynomial in the number of channel outputs.

show abstract

Section: B Necessary Condition For Optimalitymentioning

confidence: 99%

“…Theorem 1 in [15] states that, if the objective function is concave inT j|k = Pr(X = j|Z = k), then the optimal K partitions consist of K convex subsets of {P 1|i | i = 1, . .…”

Section: B Necessary Condition For Optimalitymentioning

confidence: 99%

See 1 more Smart Citation

Channel quantizers that maximize random coding exponents for binary-input memoryless channels

Yagi

Kurkoski

2012

2012 IEEE International Conference on Communications (ICC)

View full text Add to dashboard Cite

show abstract

“…Compared with [12,Eqn. (2)], we can see that C(Q) considered in this paper is a specific case of that considered in [12]. C(Q) given by (2) includes many practical quantizers' cost functions as subcases.…”

Section: Preliminariesmentioning

confidence: 99%

Dynamic Programming for Quantization of q-ary Input Discrete Memoryless Channels

Cai

Song

et al. 2019

2019 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

In this paper, under a general cost function, we present a dynamic programming (DP) method to obtain an optimal sequential deterministic quantizer (SDQ) for q-ary input discrete memoryless channel (DMC). The DP method has complexity O(q(N − M ) 2 M ), where N and M are the alphabet sizes of the DMC output and quantizer output, respectively. Then, starting from the quadrangle inequality (QI), two techniques are applied to reduce the DP method's complexity. One technique makes use of the SMAWK algorithm and achieves complexity O(q(N − M )M ). The other technique is much easier to be implemented and achieves complexity O(q(N 2 − M 2 )). We further derive a sufficient condition under which the optimal SDQ is optimal among all quantizers and the two techniques are also applicable. This generalizes the results in the literature for binary-input DMC. Next, we show that the cost function of α-mutual information (α-MI)-maximizing quantizer belongs to the general cost function we adopt earlier. We further prove that under a weaker condition than the sufficient condition we derived, the aforementioned two techniques are applicable to the design of α-MI-maximizing quantizer. Finally, we propose a new algorithm called iterative DP (IDP). Theoretical analysis and simulation results demonstrate that IDP can improve the quantizer design over the state-of-the-art methods in the literature.Index Terms-α-mutual information (α-MI), discrete memoryless channel (DMC), dynamic programming (DP), quadrangle inequality (QI), sequential deterministic quantizer (SDQ). P Z|Y (z|y) = 1 z = z , 0 z = z , or equivalently, we say y's quantization result Q(y) is a deterministic element in Z. In this paper, a general cost function given by (2) is considered, which is widely adopted in the literature [12]-[14]. Under such a condition, we show that there always exists at least one DQ that is optimal among all quantizers. Due to this reason as well as that DQ is more practical than non-deterministic quantizer, we focus only on DQs in this paper. For any DQ Q : Y → Z, denote Q −1 (z) ⊂ Y as the preimage of z ∈ Z.For binary-input DMC, dynamic programming (DP) [15, Section 15.3] was applied by Kurkoski and Yagi [16] to design quantizers that maximize the mutual information (MI) between X and Z, i.e., I(X; Z). The complexity (refer to the computational complexity throughout this paper unless the storage complexity is specified) of this DP method was reduced [17], [18] by applying the SMAWK algorithm [19]. However, for the general q-ary input DMC with q > 2, design of the optimal quantizers that maximize I(X; Z) is an NP-hard problem [14], [20]. Up till now, only the necessary condition [12], rather than the sufficient condition, has been established for the optimal quantizer; meanwhile, there only exist some suboptimal design methods in practice [5], [14], [21]-[23]. Moreover, to the best of our knowledge, so far no work has arXiv:1901.01659v3 [cs.IT] 28 Oct 2019

show abstract

“…This quantization problem is an example of impurity partitions from machine learning, where convex subsets are known to be optimal [11]. While restricted to binary-input DMCs, this approach finds the optimal quantizer for otherwise arbitrary channels.…”

Section: Previous Work and Contributionmentioning

confidence: 99%

Finding the capacity of a quantized binary-input DMC

Kurkoski

Yagi

2012

2012 IEEE International Symposium on Information Theory Proceedings

View full text Add to dashboard Cite

Consider a binary-input, M -output discrete memoryless channel (DMC) where the outputs are quantized to K levels, with K < M. The subject of this paper is the maximization of mutual information between the input and quantizer output, over both the input distribution and channel quantizer. This can be regarded as finding the capacity of a quantized DMC. An algorithm is given, which either finds the optimal input distribution and corresponding quantizer, or declares a failure.

show abstract

Minimum Impurity Partitions

Cited by 48 publications

References 5 publications

Channel quantizers that maximize random coding exponents for binary-input memoryless channels

Channel quantizers that maximize random coding exponents for binary-input memoryless channels

Dynamic Programming for Quantization of q-ary Input Discrete Memoryless Channels

Finding the capacity of a quantized binary-input DMC

Contact Info

Product

Resources

About