Trial finds chatbots have mixed effectiveness in increasing COVID-19 vaccine confidence in Asia

Chatbots had mixed success in improving COVID-19 vaccine confidence and acceptance among the most hesitant individuals in Asia, according to a new randomized control trial.

In a cohort of over 2,000 guardians of children and seniors who were unvaccinated or had delayed their vaccinations in Thailand, Hong Kong and Singapore during the first half of 2022, using a vaccine-related chatbot for a week had no overall beneficial impact on improving vaccine confidence. However, a few Thai parents reported a protective effect against a decline in confidence in vaccine effectiveness.

Further, user feedback suggested that there is considerable public interest in the technology, with large majorities of participants in all three countries reporting that they intended to use the chatbots again.

The findings are published in the journal npj Digital Medicine and involve research conducted by the London School of Hygiene & Tropical Medicine (LSHTM) in partnership with the Laboratory of Data Discovery for Health (D24H). The project was supported by the Vaccine Confidence Fund (VCF).

Leesa Lin, Assistant Professor at LSHTM and Co-Director, as well as Digital and Behavioural Innovations Lead, of the Vaccine Confidence Project, who spearheaded the study, said, "To our understanding, this marks a pioneering application of a randomized control trial in the comparative evaluation of conversational AI services, namely chatbots, across a wide array of countries, platforms, and demographic categories specifically focusing on vaccine acceptance."

"The gathered insights emphasize the crucial need for broadening our comprehension of the manifold aspects of these technologies. This extends from the nature of information they provide to the user experience and user literacy, and the influence these factors exert on the potential results attainable for diverse groups."

Heidi Larson, professor of anthropology, risk and decision science at LSHTM and the Founder and Director of the Vaccine Confidence Project, remarked, "Currently, conversational AI has gained substantial traction, particularly following the launch of ChatGPT late last year. However, the evidence concerning the use of chatbots in the healthcare sector remains rather inconsistent."

"Given the rapid advancement of this technology, it is crucial to bridge this knowledge gap urgently. Doing so would enable us to discern if and how this technology can foster positive health outcomes, enhance public confidence, and mitigate any potential inadvertent negative impacts."