AI is not giving attackers new superpowers so much as removing their bottlenecks. The research, the writing, the translation, the scripting and the patient trial-and-error that used to take a skilled team days can now be done by a smaller team, or a less skilled one, in hours. The result is more attacks, better-crafted attacks, and less time between a weakness appearing and someone trying to use it.
Most business defenses were designed around the old constraints. Security awareness training assumed phishing emails would contain spelling mistakes. Patch cycles assumed attackers needed weeks to weaponize a flaw. Verification procedures assumed a familiar voice on the phone was the person it sounded like. Each of those assumptions is weakening at once, and the useful way to understand the threat is to walk through the attack chain stage by stage and see where AI changes the economics.
What has actually changed, and what has not
In January 2025 Google’s Threat Intelligence Group published an analysis of how government-backed hackers had tried to use its Gemini models. Groups linked to Iran, China, North Korea and Russia used it for reconnaissance on target organizations, researching publicly known vulnerabilities, writing and translating phishing content, and troubleshooting code. Google’s conclusion was measured: generative AI let these actors work faster and at higher volume, but it had not handed them fundamentally new capabilities.
By August 2025 the picture had moved. Anthropic reported disrupting a criminal operation that used its Claude Code agent to automate reconnaissance, credential harvesting and data theft against at least 17 organizations, including healthcare providers, emergency services and government bodies. The AI also helped decide what data to take and drafted extortion demands, some exceeding $500,000.
AI is a force multiplier. It rarely invents a new category of attack. It makes existing categories cheaper, faster and available to more people, and that is enough to shift the balance.
Reconnaissance and social engineering
Every targeted attack begins with research: who works where, who approves payments, which vendors a company uses, how its executives write. That work used to be manual, so attackers reserved it for high-value targets. Language models collapse the cost. A model can turn a company website and a few public profiles into a list of likely approvers and a plausible pretext in minutes.
The output of that research is better phishing. Verizon’s 2025 Data Breach Investigations Report found that synthetically generated text in malicious emails had roughly doubled over two years. Fluent, personalized messages remove the clumsy tells that awareness training taught people to look for.
Voice is next. CrowdStrike’s 2025 Global Threat Report recorded a 442% increase in voice phishing between the first and second halves of 2024, and attributed part of it to AI-driven impersonation. In May 2025 the FBI warned that criminals were sending AI-generated voice messages impersonating senior US officials. The best-known corporate case remains the engineering firm Arup, where an employee in Hong Kong transferred about $25 million in early 2024 after a video call in which the “colleagues” were deepfakes. We cover the deepfake side in more depth in Deepfakes Are on the Rise.
The lesson: you can no longer rely on people spotting a fake. Design processes where a convincing fake still cannot move money or reset credentials on its own.
Vulnerability discovery and exploitation
AI’s dual-use nature is most visible here. Google’s own Big Sleep agent is the clearest example from the defensive side: in July 2025 Google said it had used Big Sleep to isolate a SQLite memory-corruption flaw, CVE-2025-6965, that threat actors were preparing to exploit, and patched it before they could. Google described it as the first time an AI agent had directly foiled an in-the-wild exploitation attempt.
Attackers are pursuing the same capability. Google’s January 2025 analysis found state-backed groups asking models about specific products and published CVEs, and CrowdStrike reported Iran-linked actors exploring generative AI for vulnerability research and exploit development. Meanwhile, Verizon found that exploitation of vulnerabilities was the initial access vector in about 20% of breaches in its 2025 report, with edge devices and VPNs a growing share.
The effect is a shrinking window. When analyzing a patch or scanning the internet for a vulnerable version gets cheaper, the time between public disclosure and mass exploitation gets shorter. A monthly patch cycle for internet-facing systems is increasingly a bet you will lose. See ultra-fast network intrusions for how little time remains once an attacker is inside.
Evasion and adaptive attacks
Once inside a network, attackers need to stay unnoticed. Google observed North Korean groups using AI to research sandbox and virtual-machine detection, and Chinese groups asking for help understanding endpoint detection tools and quietly deploying plug-ins across an organization. Anthropic’s August 2025 report described a criminal with limited skills selling AI-built ransomware on underground forums, complete with anti-analysis and evasion features they could not have written alone.
The “adaptive attack” idea is real, but it is mostly about speed of iteration rather than self-evolving code. When a payload is blocked, an attacker with an AI assistant can rewrite, repackage and retry far faster than before. CrowdStrike’s finding that the fastest eCrime breakout in 2024 took 51 seconds, with an average of 48 minutes, illustrates how little room human-only monitoring has.
What defenders should change
None of this requires buying a product with “AI” on the box. It requires removing the assumptions attackers are now exploiting. In rough priority order:
- Move to phishing-resistant authentication. Passkeys or FIDO2 security keys stop a perfectly written phishing page from harvesting a usable login.
- Put out-of-band verification on money and access. Any payment change, wire request, or help-desk credential reset gets confirmed through a separate, pre-agreed channel, never through the channel the request arrived on. A deepfake cannot answer a callback to a number already on file.
- Shorten patch windows for exposed systems. Keep an inventory of internet-facing assets, and treat critical fixes for VPNs, firewalls and remote-access tools as days, not weeks. Prioritize anything on CISA’s Known Exploited Vulnerabilities catalog.
- Detect behavior, not files. Endpoint detection and response that watches for suspicious actions, such as unusual credential use or mass file access, holds up better against rapidly mutated payloads than signature matching.
- Use automation where speed matters. Automated containment, such as isolating a host or disabling an account on high-confidence alerts, buys back the minutes a human analyst cannot. Threat intelligence feeds keep the rules current.
- Retrain people on process, not typos. Awareness training should teach staff that a flawless message or a familiar voice proves nothing, and that following the verification procedure is always acceptable, even when the request appears to come from the CEO.
The trade-off is friction. Callback procedures slow down payments and faster patching occasionally breaks things. That cost is predictable; a successful impersonation is not. For regulated businesses, these controls also map directly to what auditors and insurers now ask about; our cybersecurity and compliance services are built around putting them in place.
Update (September 2026): The trend described here accelerated. In November 2025 Anthropic reported disrupting a campaign it attributed with high confidence to a Chinese state-sponsored group, in which AI performed an estimated 80 to 90 percent of the intrusion work against roughly 30 targets, with humans stepping in only at a handful of decision points. The defensive priorities above have not changed; the urgency has.
Frequently asked questions
Are AI-powered attacks mostly a problem for large enterprises?
No. The main effect of AI is to make tailored attacks cheap enough to aim at everyone. Research that used to be reserved for large targets can now be done for a ten-person accounting firm in minutes.
Do we need AI-based security tools to defend against AI-based attacks?
Automation matters more than the label. Behavior-based detection, automated containment and machine-assisted triage help because attacks move faster than a person can read alerts. But the highest-value controls, such as phishing-resistant MFA and callback verification, involve no AI at all.
Can employees be trained to spot deepfakes?
Only partially. Training should focus on the request rather than the media: any unusual request for money, credentials or data triggers the verification procedure regardless of how convincing the caller looks or sounds.
Staying ahead of smarter attacks
Delana Technologies helps businesses harden the parts of the attack chain AI is changing fastest: identity, verification procedures, patching of exposed systems, and automated detection and response. We also help teams use AI safely on the defensive side. To review where your defenses still rely on old assumptions, call 239.414.5126 or contact us.
Sources: Google Threat Intelligence Group, “Adversarial Misuse of Generative AI” (January 2025); Anthropic threat intelligence report on detecting and countering misuse (August 2025); Anthropic, “Disrupting the first reported AI-orchestrated cyber espionage campaign” (November 2025); Verizon 2025 Data Breach Investigations Report; CrowdStrike 2025 Global Threat Report; Google announcement on Big Sleep and CVE-2025-6965 (July 2025); FBI public service announcement on impersonation of senior US officials (May 2025); CNN reporting on the Arup deepfake fraud (May 2024).