- Microsoft launches scanner to detect poisoned language models before deployment
- Hijacked LLMs can hide malicious behavior until specific trigger phrases appear
- Scanner identifies abnormal attention patterns linked to hidden backdoor triggers
Microsoft announced the development of a new scanner designed to detect hidden backdoors in large open language models used in enterprise environments.
The company says its tool aims to identify cases of model poisoning, a form of tampering in which malicious behavior is directly built into the model weights during training.
These backdoors can remain inactive, allowing affected LLMs to behave normally until narrowly defined trigger conditions activate involuntary responses.
How the scanner detects poisoned models
“As adoption grows, confidence in safeguards must increase with it: while testing known behaviors is relatively simple, the more critical challenge is building assurance against unknown or evolving manipulations,” Microsoft said in a blog post.
The company’s AI security team notes that the scanner relies on three observable signals that indicate the presence of poisoned models.
The first signal appears when a trigger phrase is included in a prompt, which causes the model’s attention mechanisms to isolate the trigger while reducing the randomness of the output.
The second signal involves memorization behavior, in which hijacked models disclose elements of their own poisoning data, including trigger phrases, rather than relying on general training information.
The third signal shows that a single backdoor can often be activated by multiple fuzzy triggers that resemble, but do not exactly match, the original poisoning entry.
“Our approach is based on two key findings,” Microsoft said in a companion research paper.
“First, sleeper agents tend to memorize poisoning data, allowing backdoor examples to be disclosed using memory mining techniques. Second, poisoned LLMs exhibit distinctive patterns in their output distributions and attention heads when backdoor triggers are present in the input.”
Microsoft explained that the scanner extracts the stored contents of a pattern, analyzes it to isolate suspicious substrings, and then evaluates those substrings using formalized loss functions related to the three identified signals.
The method produces a ranked list of candidate triggers without requiring additional training or prior knowledge and operates on common GPT-style models.
However, the scanner has limitations because it requires access to model files, meaning it cannot be applied to proprietary systems.
It also works best on trigger-based backdoors that produce deterministic outputs. The company said this tool should not be considered a one-size-fits-all solution.
“Unlike traditional systems with predictable paths, AI systems create multiple entry points for dangerous inputs,” said Yonatan Zunger, company vice president and deputy chief information security officer for artificial intelligence.
“These entry points may contain malicious content or trigger unexpected behaviors.”
Follow TechRadar on Google News And add us as your favorite source to get our news, reviews and expert opinions in your feeds. Make sure to click the Follow button!
And of course you can too follow TechRadar on TikTok for news, reviews, unboxings in video form and receive regular updates from us on WhatsApp Also.
