Sequential Recommendation with Bidirectional Chronological Augmentation of Transformer

Abstract

Sequential recommendation can capture user chronological preferences from their historical behaviors, yet the learning of short sequences is still an open challenge. Recently, data augmentation with pseudo-prior items generated by transformers has drawn considerable attention in improving recommendation in short sequences and addressing the cold-start problem. These methods typically generate pseudo-prior items sequentially in reverse chronological order (i.e., from the future to the past) to obtain longer sequences for subsequent learning. However, the performance can still degrade for very short sequences than for longer ones. In fact, reverse sequential augmentation does not explicitly take into account the forward direction, and so the underlying temporal correlations may not be fully preserved in terms of conditional probabilities. In this paper, we propose a Bidirectional Chronological Augmentation of Transformer (BiCAT) that uses a forward learning constraint in the reverse generative process to capture contextual information more effectively. The forward constraint serves as a bridge between reverse data augmentation and forward recommendation. It can also be used as pretraining to facilitate subsequent learning. Extensive experiments on two public datasets with detailed comparisons to multiple baseline models demonstrate the effectiveness of our method, especially for very short sequences (3 or fewer items). Source code is available at https://github.com/juyongjiang/BiCAT.

Is Contrastive Learning Necessary? A Study of Data Augmentation vs Contrastive Learning in Sequential Recommendation

“…Recent works regarding the aforementioned approaches focus on time-aware approaches, which better retain time coherence between the augmented sequences and the original ones, and further enhance the model's performance [6,32]. Except for the aforementioned basic approaches, some data augmentation approaches choose to create highly plausible sequences by synthesizing and injecting/prepending fake samples into the original sequence [12,15,28], or modeling counterfactual data distribution [47,58]. Apart from being applied in the sequential recommendation, data augmentation techniques are also applied in collaborative filtering to alleviate the data sparsity problem [43] or bypass negative sampling [21] during training.…”

Uniform Sequence Better: Time Interval Aware Data Augmentation for Sequential Recommendation

“…Most existing works attempt to learn user preference as a transition pattern from the sequential data, i.e., how the previous item can be properly transited to the next item. The ordering of interacted items matters a lot in this task (Jiang et al 2021). Typical networks such as Transformer are applied to model sequential data and produce recommendations (Kang and McAuley 2018;Sun et al 2019;Li, Wang, and McAuley 2020;Ma et al 2020;Jiang et al 2021).…”

“…The ordering of interacted items matters a lot in this task (Jiang et al 2021). Typical networks such as Transformer are applied to model sequential data and produce recommendations (Kang and McAuley 2018;Sun et al 2019;Li, Wang, and McAuley 2020;Ma et al 2020;Jiang et al 2021).…”

AdaMCT: Adaptive Mixture of CNN-Transformer for Sequential Recommendation

“…Recently, Transformer has drawn considerable interest in modeling sequential data [9,22,24,25,46]. Self-attention with global feature interaction outperforms RNNs [32,57] in many tasks to capture personalized patterns effectively [8,50], since it allows feature interaction between each item in the sequence as an even stronger global inductive bias than the RNNs.…”