LESSON 7
From Voice Cloning to Shrimp Jesus
Developed by Carl T. Bergstrom and Jevin D. West
On May 22, 2023, a shocking image of a bombing at the Pentagon spread across social and traditional media.
In a matter of minutes, the US stock exchanges dropped half a trillion dollars.
But this bombing never actually happened.
The image was computer-generated.
The text-generating LLMs that we focus on in this course are just one kind of generative AI. Other systems create cartoons, photorealistic images, technical diagrams, audio, and video. Spies, scammers, propagandists, spammers, and other unsavory characters have found a wide range of uses for generative AI systems.
In November of 2023, attorney Gary Schildhorn testified to the U.S. Senate about how his son called him in urgent need of $9,000 for bail—but it wasn’t actually his son, it was a scammer who has cloned his son’s voice using new generative AI technology.
Of course, scams are nothing new. From catfishing and pigeon drops to SIn a world of fake videos, the problem is not only that people believe false things; they also doubt true ones. A presidential candidate is caught saying something incriminating? “It’s a deepfake!” This so-called “liars dividend” provides cover for nearly any recorded act.
A few seconds of voice recording provide enough information for an AI to create audio of that person saying whatever you want them to. Add a few still photos, and an AI can create so-called deepfake video of the speaker as well. One video was so convincing it resulted in the $25 million heist from a finance worker. Another may have altered the results of the 2024 Slovakian election.
In a world of fake videos, the problem is not only that people believe false things; they also doubt true ones. A presidential candidate is caught saying something incriminating? “It’s a deepfake!” This so-called “liars dividend” provides cover for nearly any recorded act.
It's not just about deception, however.
Images and video capture attention — and attention is currency in the ad-driven internet economy.
So-called engagement hackers use generative AI to create and post vast numbers of bizarre images and videos to social media accounts. One famous example is Shrimp Jesus an image of the Christian savior rising through the water and thronged with shrimp.
As part of a massive trial-and-error effort to capture internet users’ attention without any concern for truth, meaning, or effective communication, these bizarre images are bullshit in the precise sense of our definition.
So-called AI-Slop is everywhere, designed to catch your eye for even a moment.
So-called AI-Slop is everywhere, designed to catch your eye for even a moment.
To see examples, click the right arrow >
AI image of John Cena with a sandcastle in his own likeness
AI image of John Cena with a sandcastle in his own likeness
AI image of a child on a strange motorbike with legs that looks half Transformer, half Mad Max.
AI image of a child on a strange motorbike with legs that looks half Transformer, half Mad Max.
Sludge video, often wholly or partially AI-generated, is taking over TikTok. This is the next step in the evolution of engagement hacking.
AI slop and sludge video both illustrate what happens when the cost of producing content drops low enough.
People produce and post all kinds of random material, simply to see what works in the competition for a few seconds of attention.
PRINCIPLE
By faking images, voice, and even video, other generative AI technologies can fool even savvy targets. Be vigilant for scams that use generative AI, and triangulate across sources when you encounter a suspicious photo, audio recording, or video.
DISCUSSION
What can we do — and what should we do — to protect society from these kinds of scams?
VIDEO
Built with Shorthand