- Genai can hallucinate the names of open source packages, warn experts
- He did not always hallucinate a different name
- Cybercriminals can use names to record malware
Security researchers have warned against a new method by which generative AI (GENAI) can be abused in cybercrime, called “slopsquat”.
It starts with the fact that different Genai tools, such as Chat-Gpt, Copilot and others, hallucinous. In the context of AI, “hallucination” is when AI simply invents things. It can constitute a quote that a person has never said, an event that has never occurred, or – in the development of software – an open source software that has never been created.
Now, according to Sarah Gooding of Socket, many software developers count strongly on Genai during code writing. The tool could write the lines itself, or it could suggest to the different download packages to download and include in the product.
Unhappiness
The report adds that AI did not always hallucing a different name or a different package – some things are repeated.
“During the resettlement of the same triggering of hallucinations, 43% of hallucinated packages were repeated each time, while 39% never reappeared,” he said.
“Overall, 58% of hallucinated packages have been repeated more than once in ten races, indicating that a majority of hallucinations are not only a random noise, but reproducible artifacts in the way the models react to certain invites.”
This is purely theoretical at this stage, but apparently, cybercriminals could map the different packages that AI is amazing and record them on open source platforms.
Therefore, when a developer obtains a suggestion and visit GitHub, Pypi or similar – he will find the package and will install it happily, without knowing that it is malicious.
Fortunately, there are no confirmed cases of slopsquat in nature at the time of the press, but it is prudent to say that this is only a matter of time. Since hallucinated names can be traced, we can assume that safety researchers will eventually discover them.
The best way to protect yourself from these attacks is to be careful when accepting anyone, living or otherwise.
