4.4: Case Study- Language Models and the Spread of Fake News

Research conducted by Georgetown University, OpenAI, and Stanford Internet Observatory (SIO) highlights the dangers of large language models (LLMs) and the potential for them to manipulate public viewpoints.

LLMs are trained on vast amounts of textual data and can generate meaningful text similar to humans. They are commonly used for tasks such as content creation, language translation, and text summarisation, due to their ability to generate good-quality text at scale.

The concern is that LLMs can be used to produce fake news and impersonate real individuals or organisations. The researchers used the ABC model of disinformation to examine how LLMs can be misused. The model breaks down the various aspects that contribute to the escalation of false information. ‘A’ refers to ‘Actor,’ which can be a group of individuals who create and broadcast disinformation. ‘B’ stands for ‘Behaviour,’ which refers to the strategies used to spread propaganda. Lastly, ‘C’ stands for ‘Content’ which is untrue information.

The research found that LLMs can be used to promote a hoax agenda and negatively influence people. Since LLMs can generate large amounts of text quickly, they can overflow the internet with false information, making it difficult for people to differentiate between what is true and what is wrong. Even the scale of the campaigns can be amplified with minimal costs, making manipulation harder to detect.

The researchers recommend that careful attention should be paid to the type and source of news to avoid misuse, and users and developers should ethically use the model. While LLMs are not inherently malicious, they have the potential to be wrongly used for manipulation and disinformation.