Teacher-Critical Training Strategies for Image Captioning

Abstract

Existing image captioning models are usually trained by cross-entropy (XE) loss and reinforcement learning (RL), which set ground-truth words as hard targets and force the captioning model to learn from them. However, the widely adopted training strategies suffer from misalignment in XE training and inappropriate reward assignment in RL training. To tackle these problems, we introduce a teacher model that serves as a bridge between the ground-truth caption and the caption model by generating some easier-to-learn word proposals as soft targets. The teacher model is constructed by incorporating the ground-truth image attributes into the baseline caption model. To effectively learn from the teacher model, we propose Teacher-Critical Training Strategies (TCTS) for both XE and RL training to facilitate better learning processes for the caption model. Experimental evaluations of several widely adopted caption models on the benchmark MSCOCO dataset show the proposed TCTS comprehensively enhances most evaluation metrics, especially the Bleu and Rouge-L scores, in both training stages. TCTS is able to achieve to-date the best published single model Bleu-4 and Rouge-L performances of 40.2% and 59.4% on the MSCOCO Karpathy test split. Our codes and pre-trained models will be open-sourced.

CaMEL: Mean Teacher Learning for Image Captioning

“…This was later combined with a finetuning phase based on the application of the REINFORCE algorithm, to allow using as optimization objectives captioning metrics directly [9], [23], overcoming the issue of their nondifferentiability and boosting the final performance. As a strategy to improve both training phases, in [51] it is proposed to exploit a teacher model trained on image attributes to generate additional supervision signals for the captioning model. These are in the form of soft-labels, which the captioning model has to align with in the cross-entropy phase, and reweighting of the caption words to guide the fine-tuning phase.…”

“…GCN-LSTM [34], SGAE [35], and MT [36]) or self-attention (i.e. AoANet [7], X-LAN [12], DPA [8], and TCTS [51]), and captioning architectures entirely based on the Transformer network such as ORT [11], M 2 Transformer [13], X-Transformer [12], CPTR [37], DLCT [14], and RSTNet [44].…”

Efficient Modeling of Future Context for Image Captioning

“…This was later combined with a fine-tuning phase based on the application of the REINFORCE method, to allow use as optimization objectives captioning metrics directly [36,46], boosting the final performance. As a strategy to improve both training phases, in [25] it is proposed to exploit a teacher model trained on image attributes to generate additional supervision signals for the captioning model. These are in the form of soft labels, which the captioning model has to align within the cross-entropy phase, and re-weighting of the caption words to guide the fine-tuning phase.…”