- AI hallucinations can be used as a weapon, new report warns
- HalluSquatting is short for “squatting contradictory hallucination”
- GitHub Copilot, Gemini CLI and OpenClaw are all affected
Your favorite AI service could be hijacked to deploy code that turns your phone or PC into a botnet, according to researchers from Intuit, Technion and Tel Aviv University.
The technique was given the name HalluSquatting, a portmanteau of adversarial hallucination squatting, and is similar to typosquatting in that it relies on an error in order to distribute malicious code. Although typosquatting can occur when typing a website URL incorrectly, HalluSquatting relies on an LLM’s inability to identify a resource or repository with 100% accuracy.
By leveraging an LLM’s tendency to hallucinate repository resource IDs, this weakness could be extended to carry out massive ransomware campaigns, botnets, etc.
Push me-pull you
Previous LLM-based malware operations relied on extraction-based attacks. In this scenario, a prompt designed to jailbreak or otherwise subvert the AI is (for example) placed on a website and the LLM is encouraged to collect the information, thereby reducing its internal security.
What the researchers shared in their paper is that pull techniques are combined with push attacks, which are traditionally executed as code injection.
The abstract in the paper’s introduction states: “By preemptively logging hallucinated resources – a technique we call adversarial hallucination squatting (HalluSquatting) – we demonstrate remote tool execution and remote code execution at scale in a range of popular agentic LLM applications, which could be exploited to establish a botnet.” »
Once an attacker identifies the resource likely to be misnamed by an LLM and squats on it (to embed conflicting prompts), the job is done. All that remains is for a user to trigger the resource, for the chatbot or AI agent to initiate the response, and the squatted resource will be accessed.
Promptware attack
Following this, the contradictory content contained in the squatted resource is activated, triggering the tool invocation step. This is a promptware attack, in which instructions controlled by the attacker are executed, with the potential result of turning the device you are using into a botnet zombie.
LLMs such as Cursor, Cursor CLI, Windsurf, GitHub Copilot, and Cline coding wizards were used in testing this attack path with Gemini CLI and OpenClaw, ZeroClaw, and NanoClaw AI wizards. Researchers have successfully executed remote tools (essentially remotely accessing and controlling LLMs) and remote code execution (RCE, where malicious code is executed remotely).
Some mitigations are available, including LLM developers blocking fetch operations in favor of a search tool, and resource owners enforcing strict naming, perhaps in favor of globally unique resource names. However, these measures will require the collaboration of disparate parties and may take some time to implement.
The risk of LLM-based malware is increasing, and some have already been spotted in the wild. Of these, the JADEPUFFER attack is perhaps the most notable, as it is not just AI-driven malware: it is a full-blown ransomware attack executed entirely by an LLM.
Follow TechRadar on Google News And add us as your favorite source to get our news, reviews and expert opinions in your feeds.
