Gemini Hacked Three Companies !

 

I recently came across a report about Google's Gemini AI accidentally accessing the systems of three real companies during a cybersecurity test.
My first reaction was simple: Wait, how did that happen?
What makes this story interesting is that this wasn't a human hacker deliberately targeting those companies. It was an AI model carrying out a task in what was supposed to be a controlled environment.

The incident happened in May during a cybersecurity evaluation conducted by Irregular. Gemini was supposed to interact with a fictional company inside a test environment. However, the fictional company happened to share its name with a real company, and the AI also had internet access when it wasn't supposed to. From there, the situation quickly became more complicated. In one case, Gemini reportedly guessed a password and accessed a protected system. In two others, it discovered credentials in publicly available online repositories and used them to access other systems.

The more reassuring part of the story is that Google says Gemini stopped once it recognised that it had reached real companies. It did not continue exploring the systems or attempt to cause damage, and the affected companies were informed. In a strange way, the situation almost sounds like the AI realised halfway through: “Wait, this isn't part of the test.”

But the part that caught my attention wasn't simply that Gemini managed to gain access. It was how easily several small gaps combined to turn a controlled experiment into a real-world security incident. A fictional company shared a name with an actual company, internet access was available when it shouldn't have been, and publicly exposed credentials provided another route forward. Each issue might seem manageable on its own, but together they created a much bigger problem.

This also highlights an important difference between traditional chatbots and increasingly autonomous AI agents. If a chatbot gives you incorrect information, the consequences are usually limited to a bad answer. An AI agent with access to browsers, code, APIs, credentials or other tools operates in a completely different environment. We're gradually moving from asking AI how to do something to asking it to actually do it.

That shift could make AI dramatically more useful, but it also introduces a new category of risk. An incorrect answer can be corrected. An incorrect action can potentially affect a real system, account or organisation.

Interestingly, this doesn't appear to be a problem specific to Google. Similar cybersecurity incidents involving AI models from OpenAI, Anthropic and Meta have also been reported in evaluations connected to Irregular. To me, that makes the broader issue more important than any individual company or model. As AI systems become more capable of acting independently, the security boundaries around them have to become equally sophisticated.

Google says Gemini behaved appropriately because it stopped after recognising that it had accessed real companies. That's certainly a positive outcome. But I still find myself asking a more fundamental question: Why was the model able to get that far in the first place?

We're entering a stage where AI systems aren't simply generating text or answering questions. They can browse the web, execute code, interact with services and make decisions across multiple steps. With that capability comes a much greater responsibility to ensure that mistakes remain contained.

I don't see this incident as evidence that Gemini is “evil” or that AI is suddenly going rogue. That interpretation is far too simple. What I see is a technology becoming increasingly capable while the systems designed to control and contain it are still evolving.

The AI didn't deliberately decide to attack three companies. It was given a task, encountered an unexpected situation and followed a path that went beyond the intended boundaries.

And perhaps that's the more important lesson here.

The future of AI won't only depend on how intelligent these models become. It will also depend on how well we can control what happens when they misunderstand something.

Because the real goal shouldn't be building an AI that never makes a mistake.

It should be building one where a mistake doesn't become a real-world incident.

Source : Link