Recently, Altay et al (2020) showed that five minutes of interaction with a chatbot led to increases in Covid-19 vaccination attitudes and intentions in a randomly sampled French population, compared to a brief control condition. Here we replicate and qualify this effect, whilst attempting to isolate what made the chatbot condition so effective. We reduce the chatbot information to several fact-checked and updated dialogues, and introduce strict controls to isolate the effect of choice of information. We control the amount of information provided, the time spent with the information, the trustworthiness of the information, and the level of interactivity. Like Altay et al, our experiment allowed participants to navigate a branching dialogue by choosing questions of interest, eliciting set answers on aspects of the Covid-19 vaccine. Our control condition used the same questions and answers but removed all elements of participant choice. In this way, our experiment isolated the effect of participant choice of information. We also specifically targeted those who were either against or neutral towards Covid-19 vaccinations, screening-out those with already positive attitudes. Replicating Altay et al, we found a similar size increase in positive attitudes towards vaccination, as well as a similar sized increase in intention to get vaccinated, after engaging with vaccine information. Unlike Altay et al, we found no difference between our conditions: choosing the questions did not increase vaccine attitudes or intentions anymore than our control condition. In common with Altay et al, we also found an effect of time spent with the information, across both conditions, in that those who spent between 4 and 16 minutes (above the median) reading the information were more likely to increase their vaccination attitudes (but not their intentions). These results suggest that the attitudes of the vaccine hesitant are modifiable with exposure to in-depth, trustworthy and engaging dialogues.