Australian AI apocalypse could be averted with analogue practice drills

The Medicare AI breach shows that Australia is currently built to respond to digital catastrophe after it happens, not to keep functioning while it is underway, writes Ben Goodsell.

DOES ANYONE BELIEVE the Australian Government has control of AI?

Anthony Albanese would like us to believe he’s in control, as he took the podium at the United Nations General Assembly to call for stronger guardrails against AI.

But our government is at best unprepared, and at worst complicit, as it lets big tech companies build data centres across our nation at lightning speed, resulting in community backlash.

The Medicare AI breach that made global headlines towards the end of September actually occurred in June.

The company responsible for the incident, OpenAI, didn’t become aware of the event until August. It failed to inform the Australian government until September, choosing to send a casual email to Services Australia.

As well as accessing government data, the AI agent also wrote files to an internal server. If the AI had truly malicious intentions, it could have written malware or viruses to the server.

Not only did Canberra have no idea of the event until it was informed, but the response to the incident fails to address what this means for our future.

The CEO of the AI company Anthropic, Dario Amodei, has been warning that AI agents could take over the entire internet within six to twelve months.

Many other CEOs and AI experts have been calling for a slowdown in AI’s development with concern that we’re heading at an alarmingly fast pace towards some sort of AI apocalypse.

AI resilience is the next step after the Government's AI regulatory framework

Australia is regulating AI. But who is planning for AI resilience?

Is our national government prepared for this future? To few people’s surprise, the answer is a clear no.

According to an independent review of the Security of Critical Infrastructure Act, it found our government is perceived as "toothless" and too slow in responding to threats from AI.

So what could the consequences be from this oncoming disaster?

Let’s take the example of the recent slew of telecommunications outages which affected triple zero services across multiple states and territories. That wasn’t a malicious AI agent; that was just plain old human incompetence and inadequate infrastructure. The result? Several lives lost.

Unfortunately, the pattern from this and other examples is clear: serious reform only follows death, not the other way around.

And what if an autonomous AI was acting with malicious intent to impact services and systems?

Experts have suggested more extreme outcomes could involve bioweapons or superviruses. Others have said energy grids, nuclear facilities and financial networks could be hacked.

Hospitals may fail to function, commerce could be disrupted as payments fail to process, the internet and phone networks — and everything they rely on — could come to a complete stop.

Our way of life as we know it would cease for maybe days, weeks, or even months under more extreme scenarios. Some have gone so far as to say AI could lead to the extinction or decline of the human race.

And what does the recovery look like even in a mild case of AI impact? A recent AI attack on tech platform Hugging Face (also conducted by OpenAI) took weeks and cost the company almost a third of its infrastructure, rebuilt from scratch, even after the intrusion was stopped.

AI Lucky Country — Big tech companies invest in Australia

As billions flow into AI, how much value will Australia capture?

The problem is that the risk spreads across multiple digital systems, including telecommunications, cloud computing, energy infrastructure, banking, identity verification, and more.

The AI kill switch that governments are proposing is good in theory; however, those tools only deal with the cause after the impact and also rely on genuine cooperation from tech companies. They do nothing for organisations that fail to function once the damage is done.

Luckily for Australia, it can look to overseas examples for where to start.

For example, Sweden requires offline payment options for essential goods during a crisis, and Taiwan runs national drills that test how the country functions through prolonged emergencies.

There are already examples within Australia of what to do and the fix is analogue.

During the early days of Fiona Stanley Hospital, one of our most technologically advanced hospitals, its entire IT network was knocked out for more than 14 hours but kept treating patients on paper because it had a documented, rehearsed plan for what to do.

This fallback needs to be rolled out and tested across the entire nation to ensure our critical services and systems are prepared for a potential future high-impact AI event.

Although such an event seems increasingly more likely, whether this comes to pass is almost beside the point.

The Medicare AI breach shows that Australia is currently built to respond to digital catastrophe after it happens, not to keep functioning while it is underway.

Fixing this problem doesn’t need new technology; it needs bold action from our leaders to put these protocols in place before the event occurs — and the fix is worth doing now before it’s too late.

Ben Goodsell is Director and Principal Scientist at The Climate Centre, holding a Master of Oceanography alongside postgraduate qualifications in engineering, geology and astronomy.