“…To better capture the semantic representations, they replace conventional word embedding models with a transformer model: DistilBERT (Sanh et al., 2019). DistilBERT is adapted from the original Bidirectional Encoder Representations from Transformers (BERT) model (Devlin et al., 2018), which is widely viewed as a milestone in natural language processing and has clear advantages over previous methods in terms of handling long sequences, such as newspaper articles (Fu et al., 2022). They also use Bayesian mixture modules to decipher a latent semantic space, investigate clusters, and interpret heterogeneous meanings.…”