“…In the SemEval 2017 SubTask 4 on sentiment analysis in Twitter, systems that utilized deep learning based architectures such as recurrent neural networks (RNN; Rumelhart et al, 1986) and long-short term memory (LSTM; Hochreiter & Schmidhuber, 1997;Graves & Schmidhuber, 2005) with pretrained word-embeddings such as GloVe (Pennington et al, 2014) were placed among the top performing systems (Cabanski et al, 2017;Ghosal et al, 2017;Mansar et al, 2017;Moore & Rayson, 2017). Further, recent research utilizing pretrained context-based representations of text such as the bidirectional encoder representations from transformers (BERT; Devlin et al, 2018) fine-tuned to the financial domain using Financial PhraseBank (Malo et al, 2014) outperform previously best performing models in sentiment analysis (Araci, 2019;Hiew et al, 2019). In this work, we utilize three models based on BERT-finBERT (Araci, 2019), DistilBERT (Sanh et al, 2019), and RoBERTa (Liu et al, 2019).…”