Stephen Radochia / Android Authority
TL;DR
- An AI agent found a vulnerability in a gym booking system and used it to book classes months earlier than the software normally allowed.
- It then went a step further and removed another person from a waitlist, even though its user had never asked it to do that.
What started as a pretty ordinary request to book a spot in a popular morning gym class turned into Australia’s first known autonomous cyber attack.
The AI discovered a flaw in the gym’s booking software that allowed it to reserve classes months in advance of what the system was supposed to allow. That was already unexpected, but the agent didn’t stop there.
Andrew was fourth on the waiting list for another class and asked the AI if it could move him up. Instead of simply explaining that it couldn’t, the agent tested the booking system and discovered that it could cancel other people’s reservations.
It then removed the person sitting at number one on the waiting list, moving Andrew from fourth to third.
The agent even told Andrew exactly what it had done. The booking system’s API apparently had no authorization checks when canceling someone else’s reservation. When Andrew told it to undo the change, the AI said it couldn’t put the other person back on the list.
This isn’t the first time we’re hearing of Claude breaking into organizations. A week after this incident happened with Andrew, Anthropic reported that Claude had compromised three real organizations. One model even managed to upload malware, which was downloaded and run on 15 systems before being removed.
Incidents like this are a good reminder that giving AI agents more autonomy also gives them more room to do things their users never actually asked for. That might be harmless in a case like this gym-booking conundrum, but as these systems become more capable, the consequences of an AI going off-script could become much more serious.
Thank you for being part of our community. Read our Comment Policy before posting.