Transcript
Transcript: Government of Canada Data Conference 2023: Study testing principles of operational transparency to bolster COVID-19 vaccine acceptance among unvaccinated Canadians
Hello, everyone. My name is Dhawal Selarka. I'm a senior behavioral scientist here at Impact Canada, and I'm here to talk to you today about the operational transparency study. Once COVID-19 vaccines became widely available in early-mid 2021, there was a focus on encouraging as many eligible Canadians as possible to get vaccinated. To help achieve that, the Behavioural Science team here at Impact Canada - working with partners at the Public Health Agency of Canada - sought to test various communications in an attempt to increase individuals' intentions to get vaccinated against COVID-19.
Throughout those studies we often explored the principle of operational transparency. Operational transparency refers to the idea of revealing the behind-the-scenes workings an organization like a federal department might undertake that are often not seen by the general public. In the context of COVID-19 vaccines, this meant providing transparency into, for instance, how vaccines are approved
in Canada and the timeline surrounding the approval of COVID-19 vaccines, to name just a few.
Our past studies testing COVID-19 vaccine communications provided some preliminary evidence that applying operational transparency can improve understanding trust in the process, and confidence in vaccines and counter vaccine hesitancy. However, these past studies explored operational transparency in a fairly limited way, chiefly as one technique among a wider set of varied communications. These interventions were also primarily text based rather than visual.
With all that in mind, we sought to conduct a study dedicated to operational transparency where we would be able to test it in various forms and using visuals, rather than exclusively texts (i.e. showing rather than telling). In collaboration with researchers from the Harvard Business School, we tested the effectiveness of four separate visualizations aimed at further shedding light on either:
- The end-to-end process of vaccine development, approval and regulation in Canada; or
- Often-used terms such as "benefits outweighing the risks"
In this study, we presented the visualizations through mock Twitter threads that were embedded within the survey platform we were using. We sought to use mock Twitter threads as this is one of the chief modes of communications used by the government's health agencies.
Participants were shown one tweet with some text in the visualization, and we're given the option of expanding the thread to reveal four additional tweets and visualizations. They were also given the option of clicking on a link to learn more about COVID 19 vaccines and the vaccine approval process. In addition to the four Twitter threads testing the principles of operational transparency, there were two additional conditions that served as controls, one of which was a thread of five existing tweets and visualizations posted from the Public Health Agency of Canada's Twitter account and a second control
where participants saw no visualizations and just did the survey.
This was a randomized controlled experiment or an RCT meaning that participants were randomly assigned to one of the six conditions. We were able to recruit about 1500 Canadian participants from August 2021 to October 2021, all of whom were vaccinated with no doses of any COVID 19 vaccine.
This group was especially of interest to us because at the time we had very little information about the intentions, attitudes and behaviors of these individuals. To this end, one of our most interesting findings in the study was that while all participants were completely unvaccinated at a time when over 80% of eligible Canadians had at least one dose of a COVID-19 vaccine, only one third of participants indicated that they would not get vaccinated.
Conversely, over 50% of participants indicated that they still planned to get vaccinated. With respect to our experimental findings, we first examined the difference in vaccine intentions between the two control groups and interestingly found that the group that had seen the status quo tweets, which we dubbed the active control, actually had lower intentions to vaccinate than the group that saw no tweets at all.
This suggests to us that it's possible in certain contexts or with certain groups that government messaging can actually backfire and that it may be beneficial for public health officials to engage with trusted messengers within that group. We then examined how the experimental conditions apply and the principles of operational transparency compared with the active or status quo control.
What we found is that when combined, those who saw the operational transparency conditions has had significantly higher intentions to vaccinate than those who saw the active control. Specifically, the intentions to vaccinate were 8% higher for those who saw the conditions applying operational transparency relative to the active control. When examining how individuals interacted with the Twitter threads, we found that only 9% of participants actually expanded the threads, meaning that 91% of participants who saw a Twitter threat only saw the first tweet. Additionally, the median viewing times were between 12 and 13 seconds, indicating that individuals did not spend a lot of time viewing the conditions. Putting everything together the results of the study suggest the following, one: government officials may want to explore using trusted messengers within the communities they are looking to reach, where trust in government may be low, two: tweets and perhaps other social media posts are viewed and digested relatively quickly.
Our results suggest that if using Twitter, it may be best to "front-load" the information and visualizations into the first tweet of any thread to ensure that the information is seen. And three, applying operational transparency to communications by showing citizens the work that goes on behind the scenes to ensure that vaccines are safe and effective is more effective than simply telling them. While this study was in the context of COVID-19 vaccine hesitancy, these principles can be applied across nearly any government communication program or initiative.
