A new experiment from cybersecurity company Surfshark suggests that even people who consider themselves savvy online users have trouble distinguishing AI bots from real humans on social media.
Of the 710 participants in the study of master’s students at Malmö University, only 53% correctly identified more robots than humans. This means that almost half (47%) failed completely.
Recent industry estimates suggest that bot-driven amplification now accounts for about 23% of political speech on X during election seasons.
Previous research from Surfshark found that major platforms remove more than 6.3 billion fake accounts each year, or about 47 times the number of babies born globally each year.
Even the best VPN can’t help you better recognize an AI-written comment, and that’s exactly the gap this experiment attempts to highlight.
The “Bot or Not” simulation puts you in the shoes of a content moderator and asks a simple question: can you really still trust your own instincts when scrolling through content?
Inside Surfshark’s “Bot or Not” experiment
The game “Bot or Not” is a timed interactive simulation designed by students of the Interaction Design master’s degree at Malmö University for the UNFOLD exhibition during Milan Design Week.
Players are dropped into a simulated social media comments section and given 120 seconds to spot 10 bot-written comments across four discussion topics.
Two of these subjects were deliberately “cold”, that is to say not very emotionally charged: data centers and the eternal debate of pineapple over pizza. The other two were “hot” and politically charged: immigration and women’s rights. It was in the contrast between the four that the data appeared most revealing.
When participants discussed data centers, they identified 71% of bots with an accuracy rate of 76%, the strongest result in the study. The pineapple on pizza was almost as good, with a detection rate of 64% and accuracy of 69%.
However, as soon as the simulation moved into emotional territory, performance collapsed.
Regarding immigration, detection fell to 54% and accuracy to 63%. On women’s rights, the detection rate fell to just 49%, with accuracy dropping to 61%, meaning users were missing more robots and falsely accusing more real humans of being machines.
Who has the most difficulty and how to take the test
The study also highlights a clear “generational cliff” around the age of 40. Players under the age of 20 were the strongest bot hunters in the dataset, finding almost 65% of bots with over 71% accuracy. Performance remained stable during the 20s and 30s, then dropped sharply for the 41-50 range, where detection fell to 42% and accuracy to 59%. Users over 50 fare barely better.
According to Luís Costa, head of research at Surfshark, the takeaway isn’t really about reading skills or media literacy in the traditional sense. The biggest blind spot revealed by the experiment was emotion: when a debate gets heated, it hijacks the mental “radar” that people rely on to flag suspicious content.
To combat automated deception, he argues, what users really need is a cool head and a better awareness of their own vulnerabilities, not more precise textual analysis.
The “Bot or Not” game is now publicly available at botornot.one, and anyone can play it in their browser to see their score compared to the original 710 participants.
The broadest point of the study is harder to eliminate than the score of any individual game. Robots are being produced by the billions, the technology that powers them is becoming more and more integrated, and our own emotional reactions are the lever for which they are increasingly designed.
A few minutes with “Bot or Not” is a quick way to find out how often this lever is already working on you.
Follow TechRadar on Google News And add us as your favorite source to get our news, reviews and expert opinions in your feeds. Make sure to click the Follow button!
