“…Large scale transfer learning by pre-training on JFT , Dosovitskiy et al, 2020, Ryoo et al, 2021, Mustafa et al, 2021, Tay et al, 2021a, Puigcerver et al, 2020, Ngiam et al, 2018 or ImageNet21K [Dosovitskiy et al, 2020, Mustafa et al, 2021, Puigcerver et al, 2020 has been done extensively. Mensink et al [2021] considers a two-step transfer chain, where the model is pre-trained on ImageNet, fine-tuned on the source task and then transferred to the target task.…”