AI revolutionizes scientific outreach: podcasts about research so realistic it fools authors

The first study using artificial intelligence (AI) technology to generate podcasts about research published in scientific articles has shown that the results were so good that half of the authors of the articles thought the podcasters were human, according to data from the University of Leuven in Belgium.

This research was published in the European Journal of Cardiovascular Nursing (EJCN) by researchers led by Professor Philip Moons of the University of Leuven. The researchers used Google NotebookLM, a custom AI research assistant created by Google Labs, to create podcasts explaining research recently published in the EJCN.

Professor Moons, who also presented the findings at the Association of Cardiovascular Nursing and Allied Professions (ACNAP) conference in Sophia Antipolis, France, said: “In September 2024, Google launched a new feature in NotebookLM that allows users to create AI-generated podcasts. This got me thinking about how researchers and publishers might use it.”

"When I conducted my first test on one of my own articles, I was amazed by its high quality and how natural it sounded. At that moment, I realized that such a system could have the potential to be used by scientific journals for communication," says the professor, who also adds that "of course, quality and accuracy needed to be assessed. Therefore, we designed this study to evaluate its potential."

Professor Moons, Professor of Health Sciences and Nursing at the Catholic University of Louvain and Editor-in-Chief of the EJCN, and his colleagues selected ten different types of articles and contacted the authors to ask if they would participate in the study. Participants were not informed that the podcast about their study would be generated using AI.

The researchers sent the AI-generated podcasts to the authors to evaluate their engagement, reliability, and AI detection. They then conducted a questionnaire (generated with ChatGPT) and a 30-minute interview via Microsoft Teams.

Thus, the authors reported that the podcasts captured the key points of their articles in very simple and easy-to-understand terms, were well-structured, had a good balance of length and depth, and that the presenters were professional; some authors even assumed the speakers had nursing or medical training. The presenters' conversational interaction was a valuable asset.

Furthermore, most authors stated that the podcasts were reliable sources of information. However, some commented on the American accent and style, including some exaggeration of research findings with the use of words like "incredible," "revolutionary," and "totally." They noted that there were some inaccuracies and misrepresentations, sometimes missing context, incorrect use of medical terminology, and mispronunciation of medical terms. The podcasts would need to be carefully reviewed for accuracy before publication.

All authors stated that patients and the general public would be the most appropriate target audience for podcasts, primarily due to their tone and ability to explain articles simply. However, some commented that podcasts could also be useful for healthcare professionals to stay up-to-date with the latest research, access it more easily, and increase the visibility of original research articles.

Some authors have suggested that podcasts could be tailored to specific audiences based on age, interests, or ethnicity as the technology evolves and improves. Currently, it's not possible to change the voices or language of AI hosts, but these features are likely to be enabled in the future.

Professor Moons comments: “The overall accuracy of the podcasts was surprising. Since we are just starting out with these types of AI-generated podcasts, the quality will improve over time, likely in the coming months. Another important aspect is that these podcasts appear to be more suitable for non-technical audiences, for example, the general public or patients.”

If AI could generate podcasts, according to the authors, it would be a real game-changer. Podcasts could be created with very little effort, simply by uploading the article and perhaps with a little help. This could be a sustainable model for spreading the message to those who don't usually read scientific journals.

He suggests that this technology will allow publishers, journals, and researchers to communicate science to the general public. It won't make human podcasters redundant. There will always be a market for human-created podcasts, probably because AI can't address every topic accurately or adequately. I even imagine there will be hybrid podcasts, where human podcasters and AI come together for different sections of an episode.

Now, the researchers plan to explore further possibilities of these podcasts for scientific communication, including input from patients and other members of the public.

They also want to investigate whether it would be possible to use AI-generated podcasts for scientific conference sessions. "For example, creating a podcast about the content of conference sessions, as a summary for those who didn't attend and would like a review," concludes Professor Moons.