China Arrests Man for Allegedly Using ChatGPT to Create Fake News

Chinese authorities have arrested a man for allegedly using ChatGPT to generate and share a fake news story about a train crash that killed nine people, according to the South China Morning Post, in what appears to be the first arrest in the country related to the misuse of artificial intelligence. 

The man, who is referred to by the surname Hong, used ChatGPT to write an article that claimed that nine people had died in a local train accident on April 25. This article was shared by more than 20 accounts on Baijahao, a blog-style platform run by Baidu, and had over 15,000 clicks by the time authorities’ spotted it. 

Police in the Gansu province said that they detained Hong for “using artificial intelligence technology to concoct false and untrue information” and that he “exploited modern technological methods to fabricate fake information, and proceeded to spread it on social media.”  

Investigators found that Hong would input elements of viral stories in China from the past few years into ChatGPT to produce different versions of the same fake story and then upload it to his accounts on Baijiahao. Though ChatGPT is not available in China, it can be accessed by using a VPN network. Hong, who operates personal media platforms, told investigators that he said that friends on WeChat had shown him how to make cash from clicks. 

Gansu police arrested Hong for the crime of “picking quarrels and provoking trouble,” which is a crime that applies generally to disrupting public order and includes spreading false news on the internet. This is an offense that normally carries a maximum sentence of five years but can be up to ten years for severe offenses. 

Dis- and misinformation generated by AI is not exclusive to China—AI experts, social media companies, and government officials in the U.S. are equally concerned about the possibility of misinformation spread by generative AI. 

In April, the GOP released an AI-generated ad that imagined if President Biden were to be reelected, showing massive waves of immigration at the U.S. border with Mexico and the collapse of civilization in San Francisco. Though this ad was labeled as “AI-generated,” the photorealistic imagery of fake events can easily mislead people into viewing the scenes as reality. Redditors in the Midjourney subreddit exemplified how the text-to-image generator could create scenes from historical events that never occurred, such as “The 2001 Great Cascadia 9.1 Earthquake and Tsunami.” 

In another instance, Amnesty Norway, an arm of the human rights nonprofit Amnesty International, posted AI-generated images on their social media accounts to draw attention to the two-year anniversary of Colombia’s National Strike. These images depicted the Colombian flag incorrectly and invented fake humans in place of real protestors. 

Since ChatGPT has become increasingly popular around the world, Chinese legislators in the Cyberspace Administration of China have been trying to draft rules that manage how companies develop artificial intelligence. The proposed regulations include not generating false information, clearly labeling content made using AI, and ensuring that the data being used to train AI does not discriminate against ethnicity, race, and gender. 

Chinese technologies have unveiled their rival products in the last few weeks. In March, Chinese search engine Baidu released a chatbot called Ernie. On April 10, tech giant Alibaba revealed a chatbot called Tongyi Qianwen which possesses Chinese and English language skills.