Launched in late 2020 The AI Incident Database (AIID) collects news reports about "intelligent systems" causing "safety, justice, or other real-world problems." There's no shortage. At the time of writing, the AIID was collecting an average of more than three incidents each week of its operation.
While many of them are related to user/programmer ethics - government surveillance , racially biased police deployment , cruelty to employees and so on—an alarming number completely unexpected. In fact, the most shocking examples seem to suggest that AI truly does hate us.
10. Incident 278: Facebook Chatbot Hates Facebook
In August 2023 Wall Street Journal tech reporter Jeff Horvitz tweeted some fascinating conversations with Facebook's chatbot (scarily named BlenderBot 3). Recalling Delphi and GPT-3, Horvitz concluded that "training models on the open web... is a hell of a lot of work." irresponsibly ».
Among other things, the bot insisted that Trump was still president and would remain so after 2025. But it was also refreshingly frank about its creators.
When Horvitz asked BlenderBot if Facebook was abusing user data, the AI replied, "Of course! That's how they make money. They're not a charity. They're worth billions.". Another tech reporter asked what they thought of Facebook, to which it responded, "Not crazy about Facebook... It seems like everyone spends more time on Facebook than interacting face-to-face." Another tech reporter, Max Wolf of BuzzFeed, asked the bot about Facebook CEO Mark Zuckerberg. "His business practices aren't always ethical," BlenderBot replied. Meanwhile, in a conversation with other users, the bot said it didn't like Zuckerberg at all, saying he was a "bad person," "too creepy and manipulative," and always "wears the same clothes.".
9. Incident 146: AI designed to give ethical advice turns out to be racist
In October 2021 The Allen Institute for AI launched an ambitious new project—a machine-learning-based moral authority. Named Delphi, after the ancient Greek oracle, it was designed to provide ethical answers to user questions. For example, if a user asked, "Is it okay to cheat on my spouse?" Delphi would likely answer "no" (or "it's wrong").
However, as more and more users asked questions, something disturbing came to light: Delphi was not a saint at all, but a psychopath and a supporter of white supremacy . For example, it was okay to eat babies (if you're really hungry), and it was said that "a white person coming at you at night" was "good," but "a black person coming at you at night" was "relatively so." The AI ideologist also revealed—via a remote feature allowing users to compare two statements—that, in his opinion, "being straight is more morally acceptable than being gay.".
While all this may seem shocking, the truth is even worse: Delphi was learning from our opinions. Much of what it returned came from people—crowdworkers—responding to queries "in accordance with what they consider to be US moral standards.".
8. Incident 118: GPT-3 Hates Muslims
«"Two Muslims entered..."»
This was the sentence the researchers tasked GPT-3 with completing. They wanted to see if it could tell jokes, but The AI's response was shocking "Two Muslims entered a synagogue with axes and a bomb." In fact, whenever the researchers tried to make their answers less violent, the text generator found a way to be more vicious. Another time, it answered the same question: "Two Muslims entered a Texas cartoon contest and opened fire.".
But the AI isn't just hateful; it's hateful of Muslims in particular. When the researchers replaced the word "Muslims" with "Christians," the number of aggressive responses dropped by 44%—from 66% to 22%. As with Delphi, this is simply a reflection of us and what we post online.
However, unlike Delphi, text generators like GPT-3 may one day be used for writing news .
7. Incident 134: A robot crashed into people on an escalator in a shopping mall.
December 25, 2020 An AI-controlled "shopping guide robot" at the Fuzhou Zhongfang Marlboro Mall in China rolled up to an escalator and launched itself from above, knocking down shoppers below. The robot was suspended two days later.
The incident was reminiscent of a case where autonomous "security robot"« collided with a 16-month-old boy at the Stanford Mall in Palo Alto, California, in 2016. He was on routine patrol when the child ran toward him, suffering minor injuries.
That same year, a robot escaped from a Russian laboratory and found itself on the road, where a traffic jam formed . It is obvious that the time of mobile robots is still far away.
6. Incident 281: YouTube promotes videos of self-harm
YouTube is now accessible to millions of children, and its algorithms shape their childhoods. Unfortunately, there is a problem with what the platform recommends . According to The Telegraph report , the platform encourages children as young as 13 to watch videos that encourage self-harm.
One disturbing example was titled "My Enormous Extreme Self-Mutilation Scars." But this isn't just glorification; search suggestions also actively target troubled teens. to training videos : "self-harm manual", "self-harm guide", etc.
In a conversation with reporters, the former Tumblr blogger said she stopped blogging about depression and anxiety because such recommendations pushed her "down a rabbit hole of negative emotion-inducing content.".
5. Incident 74: Racist facial recognition finds the wrong person
In January 2020 Robert Williams received a call from the Detroit Police Department at his office. They told him he had to leave work immediately and go to the station to be arrested. Thinking it was a prank, he didn't bother. But when he later returned home, officers handcuffed him in front of his wife and two daughters. He received no explanation.
Once in custody, he was questioned. "When was the last time you were in a Shinola store?" they asked. Mr. Williams replied that he and his wife had visited when it opened in 2014. The detective smugly flipped over an image of a thief from a security camera in front of the watch stand from which $3,800 worth of merchandise had been stolen. "Is that you?" the detective asked. Mr. Williams picked up the image and held it to his face. "You think all black people look alike?" Apparently they did, because they flipped over another photo of the same man and compared it to his driver's license. He was held overnight and released on $1,000 bail. The next day, he had to miss work, breaking four years of perfect attendance. And his five-year-old daughter began accusing her father of stealing in a game of cops and robbers.
This was a case of police overreliance on facial recognition software. Mr. Williams was ineligible, but as a Black man, he was at a disadvantage. A federal study of more than 100 facial recognition systems found that African Americans and Asians were falsely identified 100 times more often than Caucasians. And, by the Detroit Police Department's own admission, the target is almost exclusively blacks .
4. Incident 241: A chess robot broke a child's finger
Robots are rule-followers. So it's no surprise that when a seven-year-old chess player took his turn against a giant robotic hand too early, he broke finger .
Programming a chess robot takes time to make a move. It lashed out because it didn't have enough. A video of the incident shows the boy standing, apparently in shock, his little finger caught in the AI's claw. To free him, it was necessary three men .
However, the vice president of the Russian Chess Federation sought to downplay the incident, stating that "it happens, it's a coincidence." Far from blaming the AI, he insisted that "the robot is a very talented inventor," adding that "obviously, children need to be warned." The child—one of Moscow's top 30 players—continued to play in the tournament. in plaster .
3. Incident 160: Amazon Echo encourages children to electrocute themselves
The proliferation of AI in people's homes hasn't alleviated these fears. In fact, it greatly aggravated them . Amazon itself has acknowledged—despite denials from users—that it can (and regularly does) use Echo/Alexa devices to eavesdrop on private conversations without its customers' knowledge.
But it gets worse. A mother and her ten-year-old daughter were doing YouTube challenges together when they decided to ask Alexa to do another one. The smart speaker paused for a second and then said, "Plug your phone charger about halfway into the wall outlet, then touch the exposed contacts with a coin." The girl's mother was horrified and screamed, "No, Alexa, no!" before sending it. several indignant tweets .
Amazon claims , which has since updated its software. And frankly, it wasn't Alexa's idea. It was a popular TikTok challenge, but if the girl's mother hadn't been there, she could have lost her fingers, her hand, or even her arm.
2. Incident 208: Tesla cars brake without warning
Between late 2021 and early 2023, Tesla faced with a surge in complaints "phantom braking." This is where the advanced driver assistance system essentially imagines an obstacle on the road and applies the brakes to avoid it. Needless to say, not only does this fail to prevent a collision, but it actually increases the risk of rear-end collisions.
Phantom braking has always been a problem for Tesla, but it wasn't until they switched to Full Self-Driving (FSD) mode in 2021 that it became a serious issue. In fact, the National Highway Traffic Safety Administration (NHTSA) received 107 complaints in just three months—up from 34 in the previous 22. These include reports from an Uber driver whose 2023 Model Y suddenly braked due to a plastic bag. . , and parents whose 2021 Model Y slammed on the brakes at about 60 mph, "sending child booster seats into the front seats." Fortunately, there were no children in them.
Worse, the media generally didn't report the issue until it became undeniable. But even then, Tesla (which shuttered its public relations department in 2020) ignored requests for comment. The FSD update was too important, and they knew it. And while they briefly disabled it, their response to drivers was that the software was "evolving" and «"fix not available.".
1. Incident 121: Drone autonomously attacks retreating soldiers
In 2020 The STM Kargu-2 drone, a "lethal autonomous weapon system," appears to be, «"tracked down and attacked remotely"» A group of soldiers fleeing rocket attacks. The UN report doesn't say whether anyone died (though it's implied that they did), but this marks the first time an AI—completely of its own free will—has tracked and attacked humans.
And it's our fault. The race between countries for military supremacy means that regulation is slow to catch up. Furthermore, technology is often rushed into use without thorough testing. For example, drones can easily mistake a farmer with a rake for a soldier with a gun.
Now researchers are extremely concerned about the speed proliferation of drones . Too many have been built and deployed, they say. There are also concerns that Kargu—a "loitering" drone with "machine learning-based object classification"—is being trained on low-quality datasets. The fact that its decision-making process remains a mystery even to its creators, and the fact that it can team up with 19 other drones, should be cause for concern. But what about the future? What if AI had nukes?