Distributive Thermometer: A New Unary Encoding for Weightless Neural Networks

Abstract

The binary encoding of real valued inputs is a crucial part of Weightless Neural Networks. The Linear Thermometer and its variations are the most prominent methods to determine binary encoding for input data but, as they make assumptions about the input distribution, the resulting encoding is sub-optimal and possibly wasteful when the assumption is incorrect. We propose a new thermometer approach that doesn't require such assumptions. Our results show that it achieves similar or better accuracy when compared to a thermometer that correctly assumes the distribution, and accuracy gains up to 26.3% when other thermometer representations assume an unsound distribution.

Efficient Knowledge Aggregation Methods for Weightless Neural Networks

“…achieve good results, the distributive thermometer was used. The distributive thermometer [11] is a variant of the thermometer encoder, where the data is split into percentiles of same probability and then encoded. The tuple size (n) was defined based on the work of Santiago et al [3], as well as the values for the encoder and the resolution, that were then tuned experimentally.…”

Improving the DRASiW performance by exploiting its own "Mental Images"

“…In the last few years, WNNs have been applied as classifiers in various application domains such as NLP [1,2], medicine [3,4], real time video analysis [5] and every time new extensions and/or algorithms [6,7] have been proposed to improve computational time and classification power. This is in line with today's demand for classifier models that are easy and fast during both training and prediction phases, and produce increasingly accurate results.…”