Stochastic computing (SC) computes with probabilities using randomised bit-streams and standard logic circuits. Its major advantages include ultra-low area and power, coupled with high error tolerance. However, due to its randomness features, SC's accuracy is often low and hard to control, thus severely limiting its practical applications. Random fluctuation errors (RFEs) in SC data are a major factor affecting accuracy, and are usually addressed by increasing the bit-stream length N. However, increasing N can result in excessive computation time and energy consumption, counteracting the main advantages of SC. In this work, the authors first observe that many SC designs heavily rely on constant inputs, which contribute significantly to RFEs. They then investigate the role of constant inputs in SC and propose a systematic algorithm constant elimination algorithm for suppression of errors to eliminate the constant-induced RFEs by introducing memory elements into the target circuits. The resulting optimal modulo-counting (OMC) circuits remove all constant inputs and, at the same time, minimise RFEs. Analytical and experimental results are presented demonstrating other aspects of OMC circuits, including their initialisation and autocorrelation properties, as well as their optimality in terms of minimising RFEs.