Continual Adaptation for Efficient Machine Communication

Abstract

To communicate with new partners in new contexts, humans rapidly form new linguistic conventions. Recent neural language models are able to comprehend and produce the existing conventions present in their training data, but are not able to flexibly and interactively adapt those conventions on the fly as humans do. We introduce an interactive repeated reference task as a benchmark for models of adaptation in communication and propose a regularized continual learning framework that allows an artificial agent initialized with a generic language model to more accurately and efficiently communicate with a partner over time. We evaluate this framework through simulations on COCO and in real-time reference game experiments with human partners.

Is Information Density Uniform in Task-Oriented Dialogues?

“…The study presented in this paper provides new empirical evidence on language production in dialogue which we believe can directly inform the development of natural language generation models. Our findings suggest that models that take relevant contextual units into account (Takmaz et al, 2020;Hawkins et al, 2020) are better suited for reproducing human patterns of information transmission, and confirm that the use of training objectives that enforce a uniform organisation of information density (Meister et al, 2020;Wei et al, 2021) is a promising avenue for training language models.…”

From partners to populations: A hierarchical Bayesian account of coordination and convention

“…For instance, hierarchical architectures that appropriately incorporate compositionality or incrementality into the speaker's production model may be able to reinforce component parts of longer utterances in the shared history (e.g. Hawkins, Kwon, Sadigh, & Goodman, 2020). Still, such an approach would have more in common with our proposal than to the model-free heuristics in the existing literature.…”

Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior

“…We vary different design decisions, and experiment for seven interaction rounds. 9 We experiment with four system variants: (a) FULL: our full approach described in Section 5; (b) POS-ONLY: use only examples with positive labels y = +1; (c) TC-ONLY: ignore the feedback questions, instead if the user completes the task according to our task success measure we add positive examples with both the system plan and user execution, otherwise we add a negative example using the system plan; (d) NO-ENSEMBLE: train and deploy a single model each round, starting from an initial model randomly sampled from these we use for FULL; and (e) FINE-TUNING: train model parameters θ r+1 on D r for N epochs, starting from θ r , avoiding overfitting with rehearsal (Rebuffi et al, 2017;Hawkins et al, 2020a). In rehearsal, in each batch, half the examples are sampled randomly from the previous datasets D 0 ,.…”