Key points
- Generative AI has cut the time needed to draft a convincing phishing email from around 16 hours to about 5 minutes, according to IBM’s X-Force research team.
- Roughly 82.6 percent of phishing emails now contain some form of AI generated content, and those messages get clicked about four times more often than older, human written lures.
- Voice cloning tools need as little as 3 seconds of sample audio, and researchers say cloned voices have crossed the point where a listener can no longer reliably tell them apart from the real thing.
- Deepfake enabled voice phishing surged more than 1,600 percent in a single quarter in the US, and deepfake video scam activity is up roughly 700 percent year over year.
- Analysts project deepfake enabled fraud could cause $40 billion in global losses by 2027.
The old advice stopped working
For two decades, phishing training told people to watch for the tells. Clumsy grammar. A generic greeting. A sender address that looked almost right but not quite. That advice assumed a human was typing the message, usually working from a template, usually not writing in their first language. Large language models removed that assumption entirely. An attacker can feed a model a target’s name, employer, and a handful of details pulled from LinkedIn or a leaked credential dump, and get back an email that reads like it came from someone who genuinely knows the company’s internal shorthand.
IBM’s X-Force research puts a number on how much that shift matters. What used to take an attacker about 16 hours to draft now takes roughly 5 minutes. That is not a small efficiency gain, it is a different economics of crime. One operator can now run dozens of personalized campaigns a day, and the messages no longer carry the awkward phrasing that used to give them away.
Three ways AI supercharges the attack
Text that reads like it actually knows you
Modern phishing kits plug a language model into scraped data from social media, breached credential dumps, and company websites. The result is an email that references a real project, a real coworker’s name, or a recent transaction, aimed at one person instead of blasted to ten thousand inboxes. Security researchers describe this as the collapse of the old spray and pray model in favor of narrow, high value targeting that used to require a skilled human operator and hours of manual reconnaissance. Our own guide to locking down AI agents before deployment covers a related problem, since the same automation that writes a better phishing email can also be pointed at an internal AI agent with excessive permissions.
Voices that sound exactly right
Voice cloning tools trained on a few seconds of audio, pulled from a voicemail greeting, a conference talk, or a social video, can now produce speech that mimics tone, pacing, and accent closely enough to fool people who know the real person well. In one widely reported case, criminals cloned the voice of Italy’s defense minister and used it to call business leaders with a fabricated kidnapping story built to extract a ransom payment. In another, the ShinyHunters group used a vishing call, a phone call built around social engineering rather than malware, to trick an employee at Charter Communications into handing over account access that ultimately exposed close to 4.9 million customer records.
Video calls with nobody real in them
Real time deepfake video has moved from research demo to working criminal tool. Attackers now join live video calls wearing a synthetic face and a cloned voice, responding naturally to whatever is asked, which is exactly what made the $25 million wire fraud case at the top of this piece so effective. The employee was not fooled by a static image or a pre recorded clip, they were on a live call with what appeared to be several familiar executives, all fabricated at once.
Notable incidents worth knowing
| Incident | Method | Outcome |
|---|---|---|
| Multinational firm, Hong Kong finance office | Real time deepfake video call impersonating the CFO and other executives | Fifteen transfers totaling roughly $25.6 million sent to fraudulent accounts |
| Italian Defense Minister voice clone | Cloned voice used in phone calls to business leaders | Fabricated kidnapping ransom scheme targeting high profile contacts |
| Charter Communications breach | Vishing call used to obtain employee account access | Roughly 4.9 million customer records exposed |
“The challenge in 2026 is that attackers use generative AI to produce flawless, personalized messages at scale, eliminating the typos and awkward phrasing that once served as red flags.” Industry threat research summary, 2026
Why filters and old training are losing ground
Email security tools built to catch phishing largely rely on pattern matching, known bad domains, suspicious formatting, telltale phrasing repeated across mass campaigns. AI generated messages are built precisely to avoid those patterns, often because they were refined by testing against the very detection tools defenders use. Hoxhunt’s research team reported a fourteen fold jump in AI generated phishing attempts that slipped past email filters and landed directly in inboxes, which suggests the arms race currently favors the attackers.
Human training runs into a similar wall. Awareness programs spent years teaching people to spot bad grammar and mismatched logos. Those tells are fading fast. What has not faded is the underlying psychological playbook, urgency, authority, fear of consequences, which is exactly why the finance employee on that deepfake video call approved the transfers. The technology changed. The manipulation tactic did not.
What actually helps
Security teams converging on 2026 threat data point to a common set of countermeasures, and none of them rely on an employee spotting a typo.
- Phishing resistant multifactor authentication. Hardware security keys and passkeys are far harder to defeat than SMS codes or simple push approvals, and CISA now recommends them as a priority control rather than a nice to have.
- Independent verification for money and access requests. If a call, email, or video meeting asks for a wire transfer, a password reset, or a permissions change, confirm it through a separate channel that was established before the request came in, not by replying to the message or calling a number it provided.
- Challenge protocols for voice and video. Some organizations now use a pre agreed codeword or an unscripted, unexpected question during sensitive calls, since live deepfakes still struggle with responses they were not built to anticipate.
- AI aware filtering at the perimeter. Modern email and network security tools increasingly use machine learning themselves to flag statistically unusual requests and behavioral anomalies rather than relying only on known bad signatures.
- Training aimed at the manipulation rather than the spelling. Effective programs now center on the pressure tactics, urgency, secrecy, invoking authority, since those remain constant even as the method of generating the message keeps changing.
Honest limitations in the data
Most of the figures in circulation about AI phishing come from security vendors selling detection or training products, which is worth keeping in mind. Definitions of what counts as AI generated also vary between reports, some count any message that used a generative tool anywhere in its production, others count only fully autonomous campaigns. That makes exact numbers hard to compare across firms even when the overall direction, more AI involvement, faster production, higher click rates, shows up consistently across every source cited here. Treat the individual percentages as directional signals from the organizations that measured them rather than settled, independently audited facts.
It is also worth being honest that not every reported incident includes full technical verification available to outside researchers. The $25 million Hong Kong case and the Charter Communications breach were widely reported by multiple outlets and by the companies involved, but individual figures such as exact loss totals sometimes shift slightly between the earliest reports and later official statements.
Where this goes next
None of this is likely to plateau soon. The tools that clone a voice from three seconds of audio or generate a live synthetic face on a call cost very little to run, and they keep improving on the same cycle as the rest of generative AI. That means the volume problem gets worse before it gets better, more attackers gain access to convincing tools even if the tools themselves stop advancing.
The defensive side is not standing still either. Detection systems built specifically to spot synthetic audio and video in real time are an active area of research, and some of the same behavioral and anomaly detection techniques used elsewhere in AI security, the kind covered in our piece on reinforcement learning based defenses against poisoning attacks, are being adapted to flag synthetic media rather than poisoned training data. Whether detection keeps pace with generation is the open question, and right now the honest answer is that it has not, at least not yet.
What is unlikely to change is the underlying trick. Every incident described here still comes down to convincing one person, under time pressure, that someone with authority needs something right now. AI changed the packaging. It did not invent a new kind of manipulation, and the organizations that treat this as a verification problem rather than a spelling problem tend to fare better against it.
For individuals, the same logic applies at a smaller scale. A call claiming to be a family member in trouble, a voicemail that sounds exactly like a relative asking for money, these are becoming common enough that security researchers now recommend families agree on a private verification phrase the way companies do for wire transfers. It sounds excessive until the moment it is not.
The organizations quoted throughout this piece, from IBM to CrowdStrike to CISA, largely agree on the shape of the fix even where they disagree on the exact numbers. Slow down high stakes requests long enough to verify them through a channel the attacker does not control. That single habit, more than any filter or training module, is what currently separates the people who get fooled from the people who do not.
Frequently asked questions
How can I tell if a phishing email was written by AI?
Often you cannot tell from the writing alone anymore. Well crafted AI phishing avoids the grammar mistakes and generic phrasing that used to be reliable indicators. Focus on what the message is asking for, urgent payments, credential resets, or access changes, and verify those requests independently rather than judging the email by how polished it reads.
Can voice cloning really work from just a few seconds of audio?
Yes. Multiple research groups describe cloning tools producing usable results from as little as 3 seconds of sample audio, often pulled from voicemail greetings, public talks, or social media clips.
Are deepfake video calls common, or a rare edge case?
They are still less common than text based phishing, but growing fast, with some trackers reporting deepfake video scam activity up roughly 700 percent year over year. The $25 million Hong Kong case shows the technique is viable against well resourced corporate targets, not only individuals.
Does multifactor authentication still help against AI phishing?
It helps, but not all forms are equal. SMS codes and simple push approvals can be defeated through social engineering or push bombing, where attackers flood a user with approval requests until one gets accepted. Phishing resistant options such as hardware security keys or passkeys hold up much better.
What is the single most effective defense right now?
Independent, out of band verification for any request involving money, credentials, or access changes. No matter how convincing the email, call, or video looks, confirming the request through a separate channel the attacker does not control remains the defense researchers cite most consistently.
Is this only a problem for large companies?
No. The largest reported losses tend to involve corporate finance departments, but AI phishing kits are cheap and widely available, and individuals get targeted through cloned voices of family members, fake customer support calls, and SMS phishing in volumes that rival what businesses see.
Go deeper on the source reporting
Read the underlying threat research this piece draws from, and see how the same AI security thinking applies to defending your own systems.