BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage

Abstract

We present BlenderBot 3, a 175B parameter dialogue model capable of open-domain conversation with access to the internet and a longterm memory, and having been trained on a large number of user defined tasks. We release both the model weights and code, and have also deployed the model on a public web page to interact with organic users. This technical report describes how the model was built (architecture, model and training scheme), and details of its deployment, including safety mechanisms. Human evaluations show its superiority to existing open-domain dialogue agents, including its predecessors Komeili et al., 2022). Finally, we detail our plan for continual learning using the data collected from deployment, which will also be publicly released. The goal of this research program is thus to enable the community to study ever-improving responsible agents that learn through interaction. * * We use the phrase continual learning in the sense of learning that continues over time using data from the model's interactions, but training itself will actually be performed in successive large batches; the model is not updated online.† Equal contribution.

A Case Study in Engineering a Conversational Programming Assistant's Persona

“…Recently several language models, such as Blenderbot [18] Lamda [20], and ChatGPT [13] have been introduced that are specifically tuned for dialog applications, but achieving conversational interaction can be achieved via prompt engineering with general purpose large language models as well. Valvoda et al found that fine-tuning a large language model for dialog resulted in duller and more repetitive output, while generating dynamic prompts resulted in more novel and diverse responses [21].…”

Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback

“…• OPT-175B (Zhang et al, 2022) and BB3-175B (Shuster et al, 2022b): we compare the 175B language model OPT (either 0-shot or few-shot 2 ) with BlenderBot 3, which is fine-tuned with conversational datasets including modular supervision, and internetaugmentation, from our task. This setting examines if our experiments and results are applicable to very large language models.…”

“…Very large models benefit from feedback from smaller models OPT-175B, either in zero-shot or few-shot variants is only pre-trained on dialogue data, and not fine-tuned on our task, and performs reasonably -but not better than smaller models that are fine-tuned. BlenderBot 3 (Shuster et al, 2022b) is trained with the modular supervision feedback data collected from the smaller (3B parameter) models, in addition to fine-tuning on other standard dialogue datasets. This model provides the best human evaluation metrics of all the systems we test, with a good response rate of 64.8% and a rating of 4.08.…”

“…Five examples are provided with input+response for each module in turn, mimicking the SeeKeR setup, seeShuster et al (2022b) for more details.…”

Comparing Computational Architectures for Automated Journalism

“…Our goal here was to learn a direct mapping from a intent-attribute-value input text to a human-readable Portuguese text. We tested four text-to-text transformer-based models: Bart [Lewis et al 2019], T5 [Xue et al 2020], Blenderbot [Shuster et al 2022] and GPT2 [Radford et al 2019]. While GPT2 utilizes a decoder only module, the first three models use a encoder-decoder scheme [Cho et al 2014].…”