“…Pretrained transformer protein MLMs contain structural information [Rao et al, 2019, Rives et al, 2021, Chowdhury et al, 2021], encode evolutionary trajectories [Hie et al, 2022a, 2021], are zero-shot predictors of mutation fitness effects [Meier et al, 2021], improve out-of-domain generalization on protein engineering datasets [Dallago et al, 2021], and suggest improved sequences for engineering [Hie et al, 2022b]. Protein MLMs are now incorporated into the latest machine-learning methods for detecting signal peptides [Teufel et al, 2021] and predicting intracellular localization[Thumuluri et al, 2022].…”