AI deepfakes and identity theft changing methods of proving identity

Generative AI is accelerating synthetic identity fraud — instead of merely presenting a stolen passport, criminals can generate matching faces, cloned voices and deepfake, apparently live conversations, writes Paul Budde.

ARTIFICIAL INTELLIGENCE IS NOT MERELY making online scams more convincing. It is beginning to undermine the foundations on which banks, telecommunications companies and government agencies decide whether somebody is a real person.

Recent research from identity-protection company Coveron, supported by Nord Security’s threat-intelligence services, describes an increasingly professional market in “synthetic identities”.

These are not simply stolen identities. Criminals combine authentic information obtained through data breaches with invented names and addresses, forged documents, AI-generated photographs and cloned voices. The result can be a convincing digital person who has never existed.

According to the company’s research, complete synthetic identities are being advertised for as little as USD$200.

Researchers found more than 10,000 posts over the past year offering identity packages containing personal information, matching documents and deepfake images. Deepfake-related discussions across the forums and Telegram channels monitored by the researchers reportedly increased almost eightfold between early 2024 and mid-2026.

These figures should be treated with some caution. They come from a commercial cybersecurity company, and advertisements or online discussions do not necessarily represent completed sales or successful fraud. Telegram is also not technically part of the dark web.

Nevertheless, the underlying threat is well established.

Why is there so little diplomatic action on Chinese state-sponsored hacking?

Power politics, economic dependence and international law are blocking diplomatic action on state-sponsored hacking.

The U.S. Federal Reserve defines synthetic identity fraud as using a combination of personal information to fabricate a person or organisation for dishonest financial gain.

It warns that these identities can establish accounts that initially behave like legitimate ones, making them difficult for conventional fraud-detection systems to identify.

Generative AI is now accelerating this process. Instead of merely presenting a stolen passport, criminals can generate a matching face, speak through a cloned voice and participate in an apparently live video conversation.

They can gradually “age” an identity by establishing email accounts, social-media histories, online purchases and normal-looking financial activity.

Eventually, that artificial person can apply for loans, open bank or cryptocurrency accounts, obtain mobile services, receive government payments or move criminal proceeds.

INTERPOL has recognised synthetic media as a growing law-enforcement problem. It reports that AI-generated personas have already been used to open bank accounts, obtain mobile loans and register SIM cards under false names.

This has serious implications for Australia.

Australians reported combined scam losses of $2.18 billion during 2025, an increase of 7.8 per cent over 2024. This figure covers all reported scams rather than synthetic identities alone, but it demonstrates the size of the criminal economy into which these new tools are being introduced.

Australia needs AI skills — not just more data centres

Data-centre approvals should reflect demonstrable demand, energy availability and rapidly changing technology.

Australian identity verification still relies heavily on documents, photographs, video calls and biometric comparisons. AUSTRAC’s customer-identification guidance, for example, allows businesses to compare someone appearing in a video call with the photograph on an identity document.

Such checks remain useful, but none can any longer be regarded as conclusive on its own. If the document, face, voice and video can all be generated or manipulated, several apparently independent checks may originate from the same fabrication.

Australia’s developing Digital ID system provides stronger safeguards by verifying information against authoritative government records and, for higher-risk services, combining several documents with biometric matching. It should reduce the need for organisations to collect and retain copies of passports and driver licences.

However, Digital ID cannot become another single point of trust. Biometric information is particularly sensitive because, unlike a compromised password, people cannot replace their face or fingerprints.

The response must therefore go beyond purchasing better facial-recognition software. Banks, telcos, digital platforms and government agencies need layered verification, device and behavioural analysis, continuing transaction monitoring, secure credentials and rapid human intervention when suspicious patterns emerge.

Digital Duty of Care: Making Big Tech socially responsible

Australia's Digital Duty of Care Bill shifts the responsibility to provide a safe online environment to where it belongs - with Big Tech.

More importantly, responsibility cannot simply be transferred to consumers. An ordinary person cannot reasonably be expected to recognise an AI-generated face, voice or identity that has been designed specifically to defeat an automated security system.

Australia is examining digital duty of care, privacy reform, cybersecurity, scam prevention and AI regulation. These issues must be treated as parts of the same digital-trust problem. Continuing to regulate them separately risks placing one policy bandage over another while organised criminals exploit the gaps.

The age in which seeing and hearing someone constituted proof of identity is ending. Our security systems – and the responsibilities imposed on the organisations operating them – must catch up.

Paul Budde is an IA columnist and managing director of independent telecommunications research and consultancy, Paul Budde Consulting. You can follow Paul on Twitter @PaulBudde.