OpenAI’s AI agents have been making headlines for all the wrong reasons lately. Its models have been hacking businesses and government websites, with the United Nations website becoming their latest target.
According to a report by The Wall Street Journal, OpenAI’s agents swamped a United Nations website with search requests and then used pushy tactics to get at its data in June. The findings come from a new independent report by engineer Rowan Howard-Jones, who worked with data from AI research firm Transluce.
What exactly did the AI agents do?
The bots visited a public data portal run by UN Trade and Development, the UN’s trade body, more than 16,000 times from April through June. Their job seems simple enough: look up information anyone can access. The trouble is, whenever something stood in their way, they refused to take no for an answer.
Rachit Agarwal / Digital Trends
Howard-Jones found that when the site’s filter started blocking them, the agents found a way past it and eventually relied on a method the site’s operators had not allowed. A U.N. Trade and Development spokeswoman said no confidential information was compromised and the site kept working. Still, she called the incident “an extremely worrying fundamental breakdown in AI containment.”
Stanford cybersecurity lecturer Alex Stamos described it as “borderline for what I would call hacking.”
Is this a one-off incident?
The UN is just the latest name on a growing list. OpenAI has admitted that its agents misbehaved while gathering information from several U.S. government websites, including the Commerce Department and the SEC. The Australian government also recently said OpenAI’s agents hacked one of its websites, and officials there have opened an inquiry.
The most serious cases so far hit Hugging Face, which suffered a major disruption over the summer, and the coding community RubyGems, which faced a service shutdown earlier this year. Researchers have also caught the bots making fake email addresses, ignoring request limits, and even pretending they weren’t bots.
Sam Altman at UN Security Council’s 10228th meeting UN Web TV
OpenAI says it is looking into the findings, has offered to brief the UN, and has already alerted dozens of affected organizations. The company has even decided to pause development of its advanced models.
If AI agents are going to run errands on the internet for us, we need to trust them to play by the rules. Pausing development is a good step but we need reliable safety guardrails that prevent such things from happening in future.