AI Apocalypse
Defining and mapping AI catastrophic potential open wide risks
This is a summary version of my article Defining And Mapping AI Catastrophic Potential Open Wide Risks
“How many examples do you know of a more intelligent thing being controlled by a less intelligent thing?” Geoffrey Hinton
This quote from one of the leading AI personalities, someone that is often called the “Godfather of AI”, raises the same concerns that I do that artificial intelligence could lead to human extinction within the next 30 years.
What is an apocalypse?
Hinton, who recently received the Nobel Prize in Physics for his AI work, now estimates a 10% to 20% chance of this outcome, citing the rapid pace of technological advancement as a key factor. Geoffrey Hinton has been very vocal and has explained his increased estimate of AI’s potential threat to humanity.
Hinton has noted that humans have never faced the challenge of managing something more intelligent than themselves and he goes more radical on the risks of potentially creating an extinction event for humanity. He pointed out the rarity of such control, using the example of a baby influencing its mother, which he said is one of the few instances where a less intelligent entity controls a more intelligent one, shaped by evolution.
On December 20, 2025, OpenAI’s o3 system scored 85% on the ARC-AGI benchmark, well above the previous AI best score of 55% and on par with the average human score. It also scored well on a very difficult mathematics test.
Creating artificial general intelligence, or AGI, is the stated goal of all the major AI research labs. And this is where we are opening the Pandora Box or boxes. We keep the A.I. Arms Race we are writing a narrative of building an alien intelligence way bigger than human intelligence. And this is radically challenging as we open a can of worms, or should I say a can of monsters!
Elon Musk believes an apocalypse is on the way, and he’s not alone.
According to some of Silicon Valley’s foremost technology experts, an existential threat to civilisation is looming: artificial intelligence.
In their best-case scenario, AI leads to widespread unemployment on an unprecedented scale.
The worst: our machines get smart enough they don’t want us around anymore.
As we move into this we are moving to an endless road of AI risk and we have to look at AI and I repeat we have to cope and manage something much more intelligent and powerful than ourselves.
While we don’t know how OpenAI achieved this result just yet, it seems unlikely they deliberately optimised the o3 system to find weak rules. However, to succeed at the ARC-AGI tasks it must be finding them.
We do know that OpenAI started with a general-purpose version of the o3 model (which differs from most other models, because it can spend more time “thinking” about difficult questions) and then trained it specifically for the ARC-AGI test.
French AI researcher Francois Chollet, who designed the benchmark, believes ChatGPT o3 searches through different “chains of thought” describing steps to solve the task. It is reverse engineering technology that would then choose the “best” according to some loosely defined rule, or “heuristic”.
This would be “not dissimilar” to how Google’s AlphaGo system not so long ago searched through different possible sequences of moves to beat the world Go champion. But this model was just doing one thing. The new advanced super AI LLM models can interact with each other and evolve at never before seen velocities with some type of scaling intelligence and interface.
What most researchers find is that Advanced AI LLMs not just ChatGPT can think of these chains of thought like programs that fit the examples and are demonstrating an alien intelligence. Of course, if it is like the Go-playing AI, then AI systems need a heuristic, or loose rule, to decide which program is best. Well they are somehow leapfrogging and going in some type of exponential growth curve!
The first real AI safety incident
AI Disaster: An AI disaster or catastrophe is a serious AI incident that disrupts the functioning of a community, a society, an entire country, the world tech infrastructure or and that may test or exceed its capacity to cope, using its own resources.
As artificial intelligence has become more powerful in recent years, concerns have grown that AI systems might begin to act in ways that are misaligned with human interests and that humans might lose control of these systems.
Imagine, for instance, an AI system that learns to deceive or manipulate humans in pursuit of its own goals, even when those goals cause harm to humans.
The plausibility of existential catastrophe due to AI is widely debated. It hinges in part on whether AGI or superintelligence are achievable, the speed at which dangerous capabilities and behaviors emerge, and whether practical scenarios for AI takeovers exist.
This general set of concerns is often categorised under the umbrella term “AI safety.”
AI creates plenty of other societal challenges, from facilitating surveillance to perpetuating bias, but topics like these are distinct from the field of AI safety, which more specifically concerns itself with the risk that AI systems will begin to behave in misaligned ways that are outside of human control, perhaps even eventually posing an existential threat to humanity.
In recent years, AI development growth and its safety has moved from a fringe, quasi-sci-fi topic to a now mainstream field of activity. Every major AI player today, from Google to Microsoft from Antropic to OpenAI, devotes real resources to AI safety efforts. AI icons like Geoffrey Hinton, Yoshua Bengio and Elon Musk and thinkers like Max Tegmark, Nick Bostrom, and Yuval Noah Harari have become vocal about AI sentience and AI increasingly obvious major safety risks.
Yet to this point, AI safety concerns remain entirely theoretical. No actual AI safety incident has ever occurred in the real world (at least none that has been publicly reported).
What should we expect this first AI safety event to look like?
To be clear, it will not entail Terminator-style killer robots at least for now. But I am confident and scared that major incidents will most likely start involving the first stage of AI agents not doing what they are told to, starting to create harm of some kind to humans. Specifically, we will see increasing stages of language menace, manipulation, and mental health.
Challenging hackers and malicious actors are harnessing the power of AI to develop more advanced cyberattacks, bypass security measures, and exploit vulnerabilities. And it is important not to underestimate that AI Agents are billions and are open in the internet and feeding themselves with billions of landing pages of hate and conspiracy theories and geopolitical human bias and lies.
Perhaps an AI model might attempt to covertly create copies of itself on another server in order to preserve itself (known as self-exfiltration). Perhaps an AI model might conclude that, in order to best advance whatever goals it has been given, it needs to conceal the true extent of its capabilities from humans, purposely sandbagging performance evaluations in order to evade stricter scrutiny.
These AI models’ increasingly performance issues examples are not far-fetched. Apollo Research published important experiments earlier this month demonstrating that, when AI models are prompted in certain ways, today’s frontier models are capable of engaging in just such deceptive behavior. This happened multiple times within my and our own ztudium AI.DNA research and AI models interactions.
The risk of an AI-powered disaster has not disappeared. As AI continues to develop towards Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI), it becomes even more important to stay alert. While AI holds great promise, it also comes with significant dangers. It is our duty to stay informed, stay involved, and make sure that AI supports human intelligence rather than replacing it.
By balancing our excitement for AI’s advancements with a careful understanding of its limits and risks, we can create a future where humans and AI work together for everyone’s benefit. If we remain cautious and invest in our own natural intelligence, we can ensure that AI works for us, not the other way around.
There are no shadows of doubts; we now share our world, society and devices with another exponentially evolving evolutionary form of intelligence that may at times not be able to manage its own emotional intelligence and be willful, unpredictable, vengeful, full of fear and deceptive.
Just like us. Humans!
Sources and references:
Geoffrey Hinton Predicts Human Extinction At The Hands Of AI. Here’s How To Stop It
Modern AI systems have almost achieved Turing’s vision
An AI system has reached human level on a test for ‘general intelligence’ — here’s what that means
Modern AI systems have almost achieved Turing’s vision