ARMDN: Associative and Recurrent Mixture Density Networks for eRetail Demand Forecasting

Abstract

Accurate demand forecasts can help on-line retail organizations better plan their supply-chain processes. The challenge, however, is the large number of associative factors that result in large, non-stationary shifts in demand, which traditional time series and regression approaches fail to model. In this paper, we propose a Neural Network architecture called AR-MDN, that simultaneously models associative factors, time-series trends and the variance in the demand. We first identify several causal features and use a combination of feature embeddings, MLP and LSTM to represent them. We then model the output density as a learned mixture of Gaussian distributions. The AR-MDN can be trained end-to-end without the need for additional supervision. We experiment on a dataset of an year's worth of data over tens-of-thousands of products from Flipkart. The proposed architecture yields a significant improvement in forecasting accuracy when compared with existing alternatives.

Deep Factors for Forecasting

“…Changing the emission probability to Gaussian Mixtures results in AR-MDN (Mukherjee et al, 2018). Sequenceto-Sequence models for forecasting (Wen et al, 2017) are another family of models that are a part of our framework.…”

“…Neural network-based models have been recently shown to excel in extracting complex patterns from large datasets of related time series by exploiting similar smoothness and temporal dynamics, and common responses to exogenous input, i.e. dependencies of type (1) and (3) (Flunkert et al, 2017;Wen et al, 2017;Mukherjee et al, 2018;Gasthaus et al, 2019). These models struggle in producing calibrated uncertainty estimates.…”

Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting

“…Despite the substantial progress, existing algorithms still fail to predict long sequence time series with satisfying accuracy. Typical state-of-the-art approaches (Seeger et al 2017;Seeger, Salinas, and Flunkert 2016), especially deep-learning methods Qin et al 2017;Flunkert, Salinas, and Gasthaus 2017;Mukherjee et al 2018;Wen et al 2017), remain as a sequence to sequence prediction paradigm with step-by-step process, which have the following limitations: (i) Even though they may achieve accurate prediction for one step forward, they often suffer from accumulated error from the dynamic decoding, resulting in the large errors for LSTF problem (Liu et al 2019;Qin et al 2017). The prediction accuracy decays along with the increase of the predicted sequence length.…”

The Effectiveness of Discretization in Forecasting: An Empirical Study on Neural Time Series Models

“…Many deep learning architectures that have seen success in other domains (e.g. computer vision or natural language processing) have been adapted to and evaluated in the forecasting setting, ranging from simple feed forward models, convolutional neural networks (CNNs), in particular using 1-dimensional dilated causal convolutions [3,4,32,45], recurrent neural networks (RNNs) [31,38,40], and attention-based models [26,27,42].…”

“…To that end, the various aforementioned deep learning architectures have been combined with techniques for modeling probabilistic outputs. These techniques range from parametric distributions and parametric mixtures [31,38], over quantile regression-based techniques like quantile grids [45], to parametric quantile functions models [14], semi-parametric probability integral transform / copula based techniques [36,44], and approaches based on discretization/bucketing [32].…”