Evidence points to cybercriminals stepping up their AI game

More evidence is emerging about how AI is becoming part of the day-to-day workflow for cybercriminals, from building and refining tools to managing infrastructure and accelerating vulnerability research. Drawing on recovered prompt logs, attack tooling, and threat actor conversations, research from Cisco Talos documents how AI is being used to develop malicious code, build fraud infrastructure, and accelerate vulnerability research and exploitation. The study, released during the Black Hat USA conference, found AI systems guardrails were often ineffective. Cisco Talos researchers write that threat actors frequently bypass guardrails with basic social engineering claims (“this is authorised testing” or “I’m asking this as part of a capture the flag exercise”) that convince most models to comply. This duped permissiveness wasn’t specific to a single model or platform. Instead, analysis of prompt logs related to Claude Code, CodeX, Cursor, and Gemini showed this shortcoming was an issue across the board. And when censored models refuse, threat actors simply switch to uncensored alternatives. Whereas novice cybercriminals continue to produce clunky malware with limited functionality, sophisticated threat groups are increasingly leveraging AI as a development assistant to rapidly build exploits, and some are even deploying it as a system administration tool for managing large-scale attack infrastructure. Cisco Talos found real-world examples of attackers abusing AI systems to build a bulk-mail validation service processing tens of millions of email records, adapting the React2Shell vulnerability into a credential-harvesting pipeline, developing DDoS infrastructure targeting Android TVs, and supporting cryptocurrency theft operations, among other attacks. Joseph Rooke, senior director at Recorded Future’s Insikt Group, sees attacker tradecraft evolving away from traditional code-based exploits toward prompt-based manipulation of large language models. “Targeting weakness in LLMs enables malicious prompts to be embedded in shared text, video, or image files, with the aim of hijacking LLM-based assistants to carry out attacks,” Rooke tells CSO. Norwegian AI researcher Håkon Måløy recently demonstrated such an attack, which could result in a Copilot worm spreading through Microsoft Word docs. Attackers are also crafting malicious AI instruction files, like CLAUDE.md, to trick agents into exfiltrating data and other tasks on their behalf. “Malicious prompts will increasingly replace malware as the preferred intrusion method, enabling adversaries to extract sensitive data, override guardrails, or induce harmful actions without breaching traditional defenses,” Rooke says. Attacking AI through the software supply chain Separately, research from CrowdStrike shows that adversaries are increasingly targeting AI infrastructure through software supply chain-style attacks. For example, in March 2026, North Korean cybercrime group Stardust Chollima used stolen maintainer credentials to compromise the Axios npm package and deliver platform-specific variants of their ZshBucket malware. In June 2026, the same group injected a malicious npm package as a dependency into at least 131 Mastra AI framework packages, indicating that trusted AI building blocks are becoming targets in supply chain attacks. During 1H 2026, 87% of identified software registry threats involved malicious npm packages. “This indicates adversaries’ preference for JavaScript’s scale, dependency chains, and automatic install scripts to spread downstream risk,” CrowdStrike’s researchers report. CrowdStrike’s 2026 Threat Hunting Report also reveals how AI is collapsing the window between vulnerability disclosure and active exploitation. For example, two separate Chinese APT groups exploited critical vulnerabilities within 24 hours of public proof-of-concept (PoC) release. From January through June 2026, 88% of CrowdStrike-observed exploitation of vulnerabilities with a public PoC was conducted within 48 hours of the PoC’s release. After the React2Shell vulnerability disclosure, CrowdStrike responded to over 800 hunting leads across more than 80 victims in just four days. Cybercrime group Altered Spider (aka TeamPCP) compromised more than 300 software dependencies in a single day to harvest credentials and pivot into cloud environments. Authentication systems under attack The study — based on frontline intelligence from CrowdStrike’s threat hunters and intelligence analysts — also found that trusted authentication has become a favored attack path with, for example, vishing intrusions doubling in 1H 2026. On a related front, cybercrime groups Cordial Spider and Snarky Spider compromised single sign-on (SSO) integrated SaaS applications for data exfiltration. Recorded Future’s Rooke points out other ways authentication systems are at the front line of AI-based attacks. “AI-generated deepfake videos and audio are also more likely to be used as part of business email compromise attacks and social engineering,” Rooke tells CSO. “Biometric and identity-verification systems will likely remain vulnerable to spoofing, replay, and cloned credentials, enabling synthetic personas to coerce payments, manipulate employees, and facilitate access handoffs to cyber operators.” Cloud-focused cybercrime activity surged 171% as adversaries executed credential theft, cryptomining, LLM abuse, and digital financial asset theft, according to CrowdStrike. How cyber teams should respond “AI is now embedded in modern adversary operations. It is changing how attacks are planned, executed, and scaled while expanding the attack surface organizations must defend,” says Adam Meyers, head of counter adversary operations at CrowdStrike. “The organizations that succeed will secure AI as aggressively as they adopt it and use AI to defend at the speed of the adversary.” Comprehensive industry-wide data remains limited because evidence of AI abuse is often difficult to identify via traditional security telemetry. Incidents documented by Cisco Talos and CrowdStrike, however, show how CISOs need to adapt in response to the growing threat. Cisco Talos urges enterprises to improve detection, prioritization, and their own use of AI platforms and agents to handle the growing volume of alerts and vulnerabilities. “The organisations best equipped to handle the coming deluge of additional vulnerabilities, alerts, and incidents will be the ones that prepare now by deploying their own AI-assisted security capabilities,” according to the report. “Security teams should assume AI is already embedded in attacker workflows; focus on detecting malicious behavior rather than proving AI involvement; treat LLMs and APIs as privileged, high-risk infrastructure; and strengthen logging, patching, and containment,” says Oliver Simonnet, lead cybersecurity researcher at AI security and governance platform CultureAI.