The Compandor-An Aid Against Static in Radio Telephony*

Mathes, R. C.; Wright, S. B.

doi:10.1002/j.1538-7305.1934.tb00668.x

Cited by 23 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 1 shows an overview of LCQ. Our method is based on a companding (compressing and expanding) technique [21] that is widely used in telecommunication and signal processing to reduce the bit-width of input signals by nonlinearly limiting their dynamic range. We assume that companding is effective for quantization of DNNs in two aspects.…”

Section: Learnable Compandingmentioning

confidence: 99%

Learnable Companding Quantization for Accurate Low-bit Neural Networks

Yamamoto

2021

Preprint

View full text Add to dashboard Cite

Quantizing deep neural networks is an effective method for reducing memory consumption and improving inference speed, and is thus useful for implementation in resourceconstrained devices. However, it is still hard for extremely low-bit models to achieve accuracy comparable with that of full-precision models. To address this issue, we propose learnable companding quantization (LCQ) as a novel nonuniform quantization method for 2-, 3-, and 4-bit models. LCQ jointly optimizes model weights and learnable companding functions that can flexibly and non-uniformly control the quantization levels of weights and activations. We also present a new weight normalization technique that allows more stable training for quantization. Experimental results show that LCQ outperforms conventional state-ofthe-art methods and narrows the gap between quantized and full-precision models for image classification and object detection tasks. Notably, the 2-bit ResNet-50 model on ImageNet achieves top-1 accuracy of 75.1% and reduces the gap to 1.7%, allowing LCQ to further exploit the potential of non-uniform quantization.

show abstract

Section: Learnable Compandingmentioning

confidence: 99%

Learnable Companding Quantization for Accurate Low-bit Neural Networks

Yamamoto

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…It has been hard to answer the latter objection; few real communication systems designed since 1948 show decisive influence by information theory. Moreover, earlier inventions, such as the Morse code, the vocoder (11), the compandor (12), and pulse code modulation (13), demonstrate that engineers can achieve by "common sense" results close in spirit to information theory. One answer (14) to the objection is that information theory provides insights which guide the engineer without solving his problem in detail.…”

mentioning

confidence: 87%

Information Theory after 18 Years

Gilbert¹

1966

Science

View full text Add to dashboard Cite

There are fashions in science. Who in 1900 foresaw the neglect which modern mathematicians accord quaternions and elliptic functions? By the year 2000 information theory may exist only in a few unread definitive treatises, preserved by college librarians as forlorn monuments to misspent lives. To avert this end, information theory must supply something more tangible than vague insights. I hope to show that the gap between theory and practice is closing. CodesThe identifying feature of information theory, the feature which distinguishes it from cybernetics or from other branches of statistical communication theory, is its use of a particular way of "measuring" information. Although earlier authors (1) had suggested various measures of information, C. E. Shannon is the real father of information theory. In a paper which appeared in 1948 (2), he not only defined a reasonable measure for information but also applied it to prove remarkable theorems which provide criteria for evaluating and comparing different communication systems. These theorems came at an advanced stage of the communications art; a radio engineer in 1948 could use amplitude modulation, frequency modulation, phase modulation, single side-band modulation, or pulse code modulation, or he could invent something new without much trouble. The need for good ways of evaluating communication systems was evident, and Shannon's paper received immediate attention.Not all the attention came from communication -engineers, however. Mathematicians found his paper a gold mine of statistical and combinatorial problems, partly because Shannon, writing for engineering readers, had not proved his theorems in the greatest possible generality or with the meticulous rigor that some mathematicians require (3), but mainly because he raised difficult mathematical questions which even today are unanswered. Physicists (4) were interested in a new interpretation of entropy as information. Psychologists found that the new information measure gave a convenient quantitative estimate of the difficulty of certain experimental tasks. Other applications have been to linguistics (5), music (6), cryptography (7), and gambling (8). 320 sites (or, depending on how you count, seven, since South Barrington was listed along with an alternate site in the Chicago area.) These were, Ann Arbor, Michigan; the Brookhaven National Laboratory, Upton, Long Island, N.Y.; Denver, Colorado; Madison, Wisconsin; the Sierra foothills, near Sacramento, California; and South Barrington, or Weston, near Chicago.The proximity of the finalists to major northern universities or research centers inspired senators Sparkman, of Alabama, and Russell, of Georgia, to scriptural sarcasm, with Sparkman declaring, "For unto every one that hath shall be given, and he shall have abundance ... ," and Russell concluding, "But from him that hath not shall be taken away even that which he hath." But though they were joined by Symington, of Missouri, and Mansfield, of Montana, in lamenting the geographic distribution of the fina...

show abstract

“…Companding can be used in a signal processor to make sure that the internal signals: 1) remain above noise levels as much as possible and/or 2) remain below overload levels imposed by power supply voltages and element nonlinearities. These goals are similar to those achieved by companding in transmission systems [20]. In integrated signal processors companding may relax the internal noise requirements, thus making possible smaller capacitances and saving chip area and power dissipation [10], [14]- [16].…”

mentioning

confidence: 92%