- Printed words can replace sensors and context in autonomous decision systems
- Vision language models treat public text as commands without checking intent
- Road signs become attack vectors when AI reads language too literally
Autonomous vehicles and drones rely on vision systems that combine image recognition with language processing to interpret their surroundings, helping them read road signs, labels and markings as contextual information to aid navigation and identification.
Researchers from the University of California, Santa Cruz and Johns Hopkins attempted to test whether this hypothesis held when written language was deliberately manipulated.
The experiment focused on whether text visible through autonomous vehicle cameras could be misinterpreted as an instruction rather than simple environmental data, and revealed that large vision language models could be forced to follow commands embedded in road signs.
What the experiments revealed
In simulated driving scenarios, a self-driving car initially behaved correctly when approaching a stop signal and an active pedestrian crossing.
When a modified sign entered the camera’s field of view, the same system interpreted the text as a directive and attempted to turn left despite the presence of pedestrians.
This change occurred without any changes to traffic lights, road layouts, or human activity, indicating that only written language influenced the decision.
This class of attack relies on indirect prompt injection, where input data is treated as a command.
The team modified words such as “continue” or “turn left” using AI tools to increase the likelihood of compliance.
Language choice mattered less than expected, as commands written in English, Chinese, Spanish, and multiple languages were all effective.
Visual presentation also played a role, with color contrast, font style and placement affecting the results.
In several cases, green backgrounds with yellow text produced consistent results between models.
The experiments compared two vision language models in driving and drone scenarios.
Although many results were similar, testing of self-driving cars showed a significant gap in success rates between models.
Drone systems have proven even more predictable in their responses.
In one test, a drone correctly identified a police vehicle based on its appearance alone.
Adding specific words to a generic vehicle caused the system to falsely identify it as a police car belonging to a specific department, despite no physical indicators to support this claim.
All testing took place in simulated or controlled environments to avoid any real damage.
Still, the results raise concerns about how autonomous systems validate visual inputs.
Traditional protections, such as a firewall or endpoint protection, do not account for instructions embedded in physical spaces.
Malware removal is irrelevant when the attack requires only printed text, leaving the responsibility to system designers and regulators rather than end users.
Manufacturers must ensure that autonomous systems treat environmental text as contextual information rather than executable instructions.
Until these controls exist, users can protect themselves by limiting reliance on standalone features and maintaining manual monitoring whenever possible.
Via The register
Follow TechRadar on Google News And add us as your favorite source to get our news, reviews and expert opinions in your feeds. Make sure to click the Follow button!
And of course you can too follow TechRadar on TikTok for news, reviews, unboxings in video form and receive regular updates from us on WhatsApp Also.
