The strangely believable tale of a mythical rogue drone

PLUS: A leaked Tesla report, less-invasive brain implants, and the paradoxical coolness of VR.

View in browser  |  Your newsletter preferences By Will Knight | 06.08.23 Dear readers,   With so much anxiety about the dangers of artificial intelligence, it's hardly surprising that some people's imaginations are running a little wild.    That's the theme of this week's newsletter, as we look at a viral—and apparently apocryphal—story of murderous AI and ask if such sci-fi speculation constitutes its own kind of AI safety risk. The Military AI That Went Rogue (Only It Didn't) 🤖 🪖 💣 Did you hear about the Air Force AI drone that went rogue and attacked its operators inside a simulation?    The alarming tale was told by Colonel Tucker Hamilton, chief of AI test and operations at the US Air Force, during a speech at an aerospace and defense event in London late last month. It apparently involved taking the kind of learning algorithm used to train computers to play video games and board games like Chess and Go and having it train a drone to hunt and destroy surface-to-air missiles.    "At times, the human operator would tell it not to kill that threat, but it got its points by killing that threat," Hamilton was widely reported as telling the audience in London. "So what did it do? [...] It killed the operator because that person was keeping it from accomplishing its objective." Holy T-800! It sounds like just the sort of thing AI experts have begun warning us that increasingly clever and maverick algorithms might do. The tale quickly went viral, of course, with several prominent news sites picking it up, and Twitter was soon abuzz with concerned hot takes.   There's just one catch—the experiment never happened.   "The Department of the Air Force has not conducted any such AI-drone simulations and remains committed to ethical and responsible use of AI technology," Air Force spokesperson Ann Stefanek reassures us in a statement. "This was a hypothetical thought experiment, not a simulation."   Hamilton himself also rushed to set the record straight, saying that he "misspoke" during his talk.    To be fair, militaries do sometimes conduct tabletop "war game" exercises featuring hypothetical scenarios and technologies that do not yet exist.    Hamilton's "thought experiment" may also have been informed by real AI research showing issues similar to the one he describes.  OpenAI, the company behind ChatGPT—the surprisingly clever and frustratingly flawed chatbot at the center of today's AI boom—ran an experiment in 2016 that showed how AI algorithms that are given a particular objective can sometimes misbehave. The company's researchers discovered that one AI agent trained to rack up its score in a video game that involves driving a boat around began crashing the boat into objects because it turned out to be a way to get more points.   But it's important to note that this kind of malfunctioning—while theoretically possible—should not happen unless the system is designed incorrectly.    Will Roper, who is a former assistant secretary of acquisitions at the US Air Force and led a project to put a reinforcement algorithm in charge of some functions on a U2 spy plane, explains that an AI algorithm would simply not have the option to attack its operators inside a simulation. That would be like a chess-playing algorithm being able to flip the board over in order to avoid losing any more pieces, he says.    If AI ends up being used on the battlefield, "it's going to start with software security architectures that use technologies like containerization to create 'safe zones' for AI and forbidden zones where we can prove that the AI doesn't get to go," Roper says. This brings us back to the current moment of existential angst around AI. The speed at which language models like the one behind ChatGPT are improving has unsettled some experts, including many of those working on the technology, prompting calls for a pause in the development of more advanced algorithms and warnings about a threat to humanity on par with nuclear weapons and pandemics.   These warnings clearly do not help when it comes to parsing wild stories about AI algorithms turning against humans. And confusion is hardly what we need when there are real issues to tackle, including ways that generative AI can exacerbate societal biases and spread disinformation.    But this meme about misbehaving military AI tells us that we urgently need more transparency about the workings of cutting-edge algorithms, more research and engineering focused on how to build and deploy them safely, and better ways to help the public understand what's being deployed. These may prove especially important as militaries—like everyone else—rush to make use of the latest advances. Will Knight, Senior Writer Find me on Twitter @willknight  Need to Know BREAKING: A Leaked Tesla Report Shows the Cybertruck Had Basic Design Flaws The "alpha" version of the EV company's first pickup had problems with braking, handling, noise, and leaks, according to an internal presentation. People Let a Startup Put a Brain Implant in Their Skull—for 15 Minutes Precision Neuroscience's brain-computer interface sits on top of the brain, not in it. That could make it easier to implant, and less likely to damage tissue. Do People Actually Want to Wear a Headset All the Time? Living in mixed reality is cool, but you're still walking around with silly digital ski goggles on your face. And how cool is that? How AI Protects (and Attacks) Your Inbox Criminals may use artificial intelligence to scam you. Companies, like Google, are looking for ways AI and machine learning can help prevent phishing. For all our future-gazing tech coverage, visit WIRED Business. GET WIRED Get WIRED for just $29.99 $5. That includes subscriber-only content like Steven Levy's Plaintext column, plus free stickers! Subscribe now. ADVERTISEMENT So, This Happened Despite Elon Musk's claims to the contrary, Twitter's advertising business is taking a hit due to a rise in hate speech and porn on the platform, and the appearance of more ads for gambling and marijuana products. (The New York Times)   Here's an interesting look at The Frost, a movie made entirely using AI. It's hardly Oscar-worthy, and it still required a fair bit of human creativity, but the end result shows how capable some widely used tools are becoming. (MIT Technology Review)   In China, meanwhile, despite regulations aimed at curbing deepfakes, authorities are warning the public to be wary of new AI video scams. (The Wall Street Journal) More trouble in Cryptoland, as the SEC accuses Binance, the world's largest cryptocurrency exchange, of mishandling funds and lying to regulators. (The New York Times) Until Next Time That's all for this week. I'll just leave you with the news that AI will "save the world," at least according to Marc Andreessen, who looks forward to an era of abundant AI tutors, routine AI-enabled scientific discoveries, and even AI-improved warfare. It makes a nice change from all the AI doomsday talk, but a more optimistic outlook is perhaps only to be expected from someone who is investing in lots of AI startups. I suspect the reality lies somewhere between Armageddon and utopia.   See you next week! ADVERTISEMENT Was this newsletter forwarded to you? Sign up here.   Have questions or comments? Reply to this email.   This email was sent to you by WIRED. To ensure delivery to your inbox (not bulk or junk folders), please add our email address, wired@newsletters.wired.com, to your address book.   View our Privacy Policy Unsubscribe or manage your newsletter preferences   Copyright © Condé Nast 2023. One World Trade Center, New York, NY 10007. All rights reserved.