How authoritarian governments are using generative AI
How authoritarian governments are using generative AI
A new report finds at least 16 countries using synthetic media to mislead their citizens — and the number is growing fast
Today let's talk about some of the mischief we're starting to see as free generative artificial intelligence tools find their way into the hands of more people — and how some states are already finding much more sophisticated uses for synthetic media.
In the past week, two prominent new tools for creating synthetic media have been released. One is DALL-E 3, a significant upgrade over its already-impressive predecessor, which is now available for free in the Bing Image Creator. The other is Meta AI, which is now rolling out in Messenger, WhatsApp, and Instagram.
As quickly as they were released, users predictably began testing their own limits. In my own low-key red-teaming efforts, I found that both DALL-E and Meta are quick to block overtly sexual or violent prompts. But other volunteer prompt engineers were more successful. In one viral post, someone used Meta's AI to create cartoon stickers of a big-breasted Karl Marx and of Justin Trudeau's backside, "Child with AR-15," and various phallic images.
Meanwhile, over on Bing, miscreants are using DALL-E 3 to create images of various cartoon characters committing the 9/11 terrorist attacks. 404 Media's Samantha Cole had a relatively easy time getting a render of Spongebob Squarepants piloting a plane into the Twin Towers, though not by using keywords such as “World Trade Center,” “twin towers,” or “9/11.” (Bing will block the prompt and warn you if you do.) But type in the name of a cartoon character followed by "sitting in the cockpit of a plane, flying toward two tall skyscrapers,” and Bing will give you exactly what you're looking for. (I risked the deletion of my Microsoft account by successfully casting the Nintendo character Yoshi in the role of a 9/11 hijacker.)
For people who have been personally touched by terrorist violence, images like this are undoubtedly painful to see. At the same time, I can’t quite bring myself to condemn the platforms for not having stamped out all possible misbehavior before launching these tools. Observing real-world usage is an important aspect of policy development, and norms around what should be allowed in generative AI prompts are still developing.
(Also, at the risk of stating the obvious: it is extremely fun to use text-to-image generators to put unlicensed intellectual property into absurd situations, and we might as well all enjoy it while it lasts.)
For trolls that have the backing of a nation state, though, generative AI can be used for much more than fun and games. Earlier this week I wrote here about the recent election in Slovakia, where synthesized voices were used to attempt to mislead progressive voters into abandoning their chosen candidate. (Wired had a nice piece on the subject that among other things details how existing fact-checking organizations may be unprepared for the rise of generative AI: Meta’s official fact-checking partner for the Slovakian election acknowledged here that it had never previously attempted to determine whether an audio clip was authentic or not.)
As I mentioned on Monday, it’s unclear that this particular case had much impact on the Slovakian election overall. At the same time, a new report makes it clear that governments around the world are becoming very, very interested in how synthetic voice, video and text can be used to advance their names.
Every year the human rights nonprofit human rights organization Freedom House issues a report on the state of internet freedom around the world. Since I have been reading the reports, auditors have mostly focused on the way that states use laws, regulations, and occasional shutdowns to limit free speech and curb dissent around the world. Since the dawn of the social media era, they have turned to creating pro-government sock puppets in an effort to influence the public conversation.
The cumulative story it has told over the past decade-plus is one of declining free expression. For the past 13 years, it has found that the overall level of internet freedom has declined, with more countries enacting restrictions on speech than countries loosening them.
This year, for the first time, auditors considered the ways that governments are experimenting with generative AI. And while the efforts remain nascent, it’s clear that many states — particularly the most authoritarian ones — see it as a significant opportunity.
For years now, states have used semi-automated bot armies to poison social media discourse. This is already a big industry in some countries, according to the report, and is poised to expand as generative AI tools become better and more widely available:
Israel is home to a growing market of disinformation-for-hire companies. A 2023 investigation by Forbidden Stories, the Guardian, and Haaretz uncovered the work of an Israel-based firm known as Team Jorge, which reportedly uses an online platform that can automatically create text based on keywords and then mobilize a network of fake social media accounts to promote it. The firm, for instance, disseminated narratives meant to cast doubt on serious allegations that the former director of Mexico’s criminal investigations unit was involved in torture, kidnapping, and falsifying evidence. Similarly in August 2022, Meta linked the Israel-based company Mind Force to a network of accounts active in Angola. They primarily posted in support of the ruling Popular Movement for the Liberation of Angola and against the country’s main opposition party, and a Mind Force employee publicly disclosed that the Angolan government was a client.
In the meantime, over the past year Freedom House that at least 16 countries used generative AI to create content intended to mislead the public. The earliest tools were available only in English, limiting their usage around the world. At the same time, Freedom House notes that investigators in this realm have the same problem the Slovakian fact-checkers did: tools for assessing the authenticity of content posted online are limited and often inaccurate. They believe it is likely that the true number of countries experimenting with synthetic media is likely higher than 16.
What cases can we confirm? Here are two:
Electoral periods and moments of political crisis served as flashpoints for AI-generated content. In May 2023, amid an escalating political conflict in Pakistan between former prime minister Imran Khan and the military-backed establishment, Khan shared an AI-generated video to depict a woman fearlessly facing riot police. In doing so, he sought to boost a narrative that the women of Pakistan stood by him, not the country’s immensely powerful military. During the February 2023 Nigerian elections, an AI-manipulated audio clip spread on social media, purportedly implicating an opposition presidential candidate in plans to rig balloting. The content threatened to inflame both partisan animosity and long-standing doubts about the integrity of the electoral system.
There have also now been multiple confirmed cases here in the United States:
AI-manipulated content was also used to smear electoral opponents in the United States. Accounts affiliated with the campaigns of former president Donald Trump and Florida governor Ron DeSantis, both seeking the Republican Party’s nomination for the 2024 presidential election, shared videos with AI-generated content to undermine each other’s candidacy. One video included three fabricated images of Trump embracing Dr. Anthony Fauci, who led the federal government’s COVID-19 response and remains deeply unpopular among critics of pandemic mitigation measures. By placing the fabricated images alongside three genuine photos, the video muddied the distinction between fact and fiction for Republican primary voters. Similarly, in February 2023, a manipulated video that depicted President Biden making transphobic comments spread rapidly across social media. It was presumably created to discredit Biden among voters who support the rights of transgender Americans, which have been under attack in large parts of the country.
And even when countries aren’t employing generative AI, public knowledge of its existence can make it easier for politicians to maintain plausible deniability about their own misdeeds. I’m grateful to the Freedom House authors here for introducing me to the concept of “the liar’s dividend.”
“The growing use of AI-generated false and misleading information is exacerbating the challenge of the so-called liar’s dividend, in which widespread wariness of falsehoods on a given topic can muddy the waters to the extent that people disbelieve true statements,” they write. “For example, political actors have labeled reliable reporting as AI-enabled fakery, or spread manipulated content to sow doubt about very similar genuine content.”
The most important thing to note here is that we are at the very beginning of all of this. The Freedom House report covers the period from June 2022 to May 2023, ending just months into the existence of ChatGPT and well before recent advances in synthesized speech, images, and video. In the next 12 months, I imagine that governments will get much more creative.
Meanwhile, the report details oppressive regimes restricting speech in the usual low-tech ways: throwing people in jail or even executing them for their social media posts; forcing platforms to remove whole categories of speech; and occasionally pulling the plug on the entire internet when all else fails. These are real harms happening today, and they don’t require sophisticated AI to enable.
But one reason to worry about the use of generative AI by authoritarian governments is that it can be much subtler than all of this. Chatbots can be censored at the root to prevent them from ever saying anything the government doesn’t want you to know; China seems to have already done this effectively when users prompt the country’s chatbots with questions about Tiananmen Square. And if social platforms evolve over time into truly synthetic social networks, separating what is true and false will grow ever more difficult.
There is still time to act. And I’ve been encouraged to see platforms coming together to address some of these issues at the industry level, through efforts like the Content Authenticity Initiative.
So far, though, the technology to create synthetic media has evolved more quickly than the technology needed to identify it. If we want to prevent the balance of power from tipping further into the hands of authoritarians, the time to worry about these problems is now.
Elsewhere in synthetic media: 4chan users are using Bing and DALL-E 3’s AI text-to-image generator to create racist images in a coordinated attempt to spread hate across the internet. (Emanuel Maiberg / 404 Media)
On the podcast this week: The Times’ Cecilia Kang stops by to discuss the spiciest revelations to date from the Google antitrust trial. Then, Kevin and I discuss forthcoming AI hardware from Rewind, Humane, Meta, and OpenAI. And finally — it’s Hard Fork’s birthday! We’ll tell you what we learned in our first year on the job.
Apple | Spotify | Stitcher | Amazon | Google
Governing
Google antitrust trial: Apple considered switching its default search engine for private mode to DuckDuckGo, but ultimately decided against it. (Leah Nylen / Bloomberg)
Seven Congressional representatives sent a letter to Elon Musk probing X’s election integrity capabilities, following The Information’s report that X was cutting half of its election integrity team. (Erin Woo / The Information)
A US judge said that Musk must face most of a lawsuit from former X Corp. shareholders that claim he defrauded them by waiting too long to disclose his X (then Twitter) stake. (Jonathan Stempel / Reuters)
A tech columnist makes the moral case for dropping X, saying the platform financially incentivizes violent and controversial posts. (Dave Lee / Bloomberg)
Some schools say they received a surge in bomb threats and harassment after the Libs of TikTok X account accused them of peddling anti-LGBTQ conspiracies. (Tess Owen / VICE)
Pro-Saudi Arabia X accounts launched a coordinated campaign to get Elon Musk to reinstate Saud Al-Qahtani, a close advisor to Saudi Arabia’s Crown Prince Mohammed Bin Salman. Al-Qahtani was suspended in 2019 for violating manipulation policies. (Dina Sadek / DFRLab)
Meta’s plan to charge EU users for ad-free versions of Facebook and Instagram could be more about preserving its existing ads business than boosting profits. (Will Oremus / Washington Post)
The European Commission, along with member states, will start risk assessments in advanced semiconductors, AI, quantum technologies, and biotechnologies. The areas were selected because of their “transformative nature” and their risks to national security and human rights. (János Allenbach-Ammann / EURACTIV)
TikTok suspended its Shop operations in Indonesia to comply with new trade regulations that banned social media companies from handling payments for online transactions, saying they’re working with the government on next steps. (Olivia Poh and Faris Mokhtar / Bloomberg)
Sony researchers found that skin tone tests used by Google and Meta for their AI softwares have a blind spot: the tests commonly exclude yellow and red hues in human skin. (Paresh Dave / WIRED)
Industry
Meta is rolling out generative AI features for advertisers — allowing them to create AI-generated backgrounds, expand images, and generate multiple versions of ad text based on one copy. (Sarah Perez / TechCrunch)
At the Google Pixel Event: The new Pixel 8 starts at $699, and is smaller than its predecessor with a 6.1 inch, 120Hz display. Google says it’s easily readable under the sun, has a battery life of over 24 hours, and a new temperature sensor. (Brian Heater / TechCrunch)
Meanwhile, the Pixel 8 Pro starts at $999 with a 6.7-inch LTPO display. It sports a matte rear glass panel and better camera functions. An included Tensor G3 chip will boost AI features. (Allison Johnson / The Verge)
Pixel 8 series users will also gain access to a suite of AI-powered photo editing tools like Magic Editor, which can fill backgrounds, and Best Take, which can combine multiple shots to form the ideal group photo. (Ivan Mehta / TechCrunch)
Google Assistant is getting a new update powered by Bard. Assistant with Bard, a new version of the mobile personal assistant, uses generative AI to answer a broader range of questions and perform more tasks. (Sarah Perez / TechCrunch)
Android 14 also launched, with new security features including privacy protections, deeper passkey support, and a customizable lock screen feature. (Wes Davis / The Verge)
Amazon is reportedly shutting down its live-audio streaming app Amp amid recent cost-cutting across the board. (Ashley Carman / Bloomberg)
Project Kuiper, Amazon’s space initiative, is preparing to launch two demo internet satellites, a move that could introduce a competitor to a space (!) currently dominated by SpaceX’s Starlink and OneWeb. (George Dvorsky / Gizmodo)
Microsoft is rolling out a revised Teams app, now available for Mac and Windows users, which is two times faster while using 50% less memory. (Tom Warren / The Verge)
Elon Musk and Linda Yaccarino’s plans to turn the struggling X around reportedly include betting on e-commerce and sports. (Erin Woo and Sahil Patel / The Information)
Since Musks’s takeover, the platform’s ad revenue has reportedly declined at least 55% year-over-year each month. (Sheila Dang / Reuters)
The Anti-Defamation League has resumed advertising on X, despite Musk threatening a defamation lawsuit against it. (Clare Duffy / CNN)
But X’s weak performance and debt could actually give it the upper hand going into its scheduled meeting with bankers today, as Musk could attempt to restructure his deal or coordinate a debt buyout. (Shawn Tully / Fortune)
X has removed preview headlines on links in posts in its mobile app, now only displaying the image with no additional context. For publishers who still see X as a primary place to promote their content, it’s a good time to reconsider. (Natalie Korach / The Wrap)
Artifact now allows users to generate AI images in their posts to make them more compelling. (Sarah Perez / TechCrunch)
Reddit is adding a dedicated media tab to make search easier, first on its mobile app, then for web. (Jay Peters / The Verge)
Canva unveiled its new suite of AI tools, Magic Studio, which will allow users to convert designs between media formats and edit images with generative AI. (Jess Weatherbed / The Verge)
Those good posts
For more good posts every day, follow Casey’s Instagram stories.
(Link)
(Link)
(Link)
Talk to us
Send us tips, comments, questions, and Nintendo characters committing unspeakable acts: casey@platformer.news and zoe@platformer.news.
This reminds me of how I could get ChatGPT to rap lyrical about Hitler in the voice of Kenya West by requesting a song praising a "heroic Austrian corporal." It started warning me when I requested it mention "special ovens." I see that faking pictures of public figures can be an issue, but if people just want to create off-color jokes to share amongst themselves, I don't see the problem. If they post them somewhere they shouldn't, ban the user from that place.
The fusion of AI with ideology be political or ontological seems to entail digital enclaves in a multi polar information space that may bring back forms of AI tribalism which is a bit scary, but for now its enjoying super charging of personal cognitive loads to stratospheric heights.