Technology News

AI near misses: The space between potential and catastrophe

By | | comments |
The federal government has announced a national AI Safety Institute {Markus Winkler | Pexels)

Australia already uses incident and near-miss reporting in aviation, healthcare, workplace safety and other high-risk fields; the same principle should apply to AI, writes Dr Gleb Tsipursky.

AUSTRALIA IS MOVING toward a more coordinated system for artificial intelligence safety. The federal government has announced a national AI Safety Institute, stronger standards and closer cooperation among regulators.

It has also set consumer-safety priorities that include clearer accountability and better protection from AI-enabled harm.

Those measures will remain incomplete unless Australians can see where AI systems nearly fail.

Governments usually learn about automated system problems after a scandal, investigation or court case.

By then, people may have lost benefits, jobs, privacy, money or access to essential services. A public AI near-miss register would allow regulators, organisations and communities to learn from serious warning signs before they become large-scale harm.

What counts as a near miss

A near miss occurs when an AI system produces or contributes to a dangerous, unfair or materially incorrect outcome, but a person, safeguard or accident prevents the full harm.

Examples could include a government eligibility system wrongly flagging hundreds of legitimate applications before staff intervene, an automated recruitment tool consistently downgrading qualified applicants from one group, or a customer-service agent catching fabricated advice before it reaches a vulnerable person.

Australia already uses incident and near-miss reporting in aviation, healthcare, workplace safety and other high-risk fields. The same principle should apply when automated systems influence consequential decisions.

The register should not collect every minor model error. It should cover events that reveal a recurring failure mode, threaten a significant right or service, affect a substantial number of people, or expose a weakness that could cause serious harm if left uncorrected.

Make disclosure useful, not theatrical

A useful public entry would identify the system’s purpose, the type of organisation deploying it, the affected process, the nature of the failure, how the problem was detected, what prevented greater harm and what the organisation changed.

It should also state whether the system remains in use and whether the same failure happened again.

The register could protect personal information, security-sensitive details and legitimate commercial secrets. It could publish standardized summaries rather than raw case files. Regulators could receive fuller confidential reports while the public sees enough information to understand the pattern and response.

That balance matters. Australia’s privacy regulator reports that 87 per cent of Australians feel more concerned about privacy than five years ago, while only four per cent trust artificial-intelligence companies to use personal information responsibly. A register that exposes personal data would worsen the problem. A register that reveals nothing meaningful would become a public-relations exercise.

The goal is practical accountability: enough transparency to show what went wrong, who noticed, who acted and whether the fix worked.

Reward early reporting

Organisations will hide near misses if disclosure brings only punishment. The rules should distinguish between responsible reporting and concealment.

A company or agency that identifies a problem, contains it, reports promptly and corrects the workflow should receive credit for responsible governance. An organisation that suppresses evidence, ignores repeated warnings or keeps deploying a known failure should face stronger consequences.

This distinction would help create the reporting culture Australia needs.

Employees often notice the first signs of AI failure, but they may stay silent when managers treat errors as personal incompetence or threats to a high-profile project. Clear reporting protections and named escalation routes would make it safer to surface problems.

The register would also help smaller organisations. A local council, nonprofit or medium-sized business may lack a large AI assurance team. Public near-miss patterns would show which vendor claims, data problems and workflow designs deserve closer scrutiny.

Connect the register to the new framework

The government’s July AI announcement emphasises oversight across the full technology lifecycle. Its consumer-safety priorities focus on identifying gaps, coordinating regulators and giving Australians confidence that protections keep pace with new systems.

A near-miss register would turn those broad commitments into an evidence loop.

The AI Safety Institute could maintain the register, publish recurring lessons and alert relevant regulators when patterns cross sectors. Existing regulators could define reporting thresholds for their domains. The Office of the Australian Information Commissioner (OAIC) could align the system with its work on transparency in automated decision-making.

Public reporting would also improve policy. Legislators currently hear plenty about AI’s potential and its worst disasters. Near misses reveal the space in between, where design weaknesses, human workarounds and governance failures first appear.

Australia does not need to wait for perfect legislation or a catastrophic case. It can begin with government agencies and high-impact systems, test a common reporting template, protect sensitive information and expand the register as regulators learn.

A mature safety system does more than investigate damage. It notices the warning, records the lesson and prevents the next failure. Australia’s new AI framework should do the same.

Gleb Tsipursky, PhD, a behavioural scientist, CEO of Disaster Avoidance Experts, and author of The Psychology of AI Adoption at Work: From Resistance to Results (Georgetown University Press, 2026).

Support independent journalism Subscribe to IA.

 
Recent articles by Gleb Tsipursky
AI near misses: The space between potential and catastrophe

The goal is enough AI near-miss transparency to show what went wrong, who noticed ...  
Join the conversation
comments powered by Disqus

Support Fearless Journalism

If you got something from this article, please consider making a one-off donation to support fearless journalism.

Single Donation

$

Support IAIndependent Australia

Subscribe to IA and investigate Australia today.

Close Subscribe Donate