One of the biggest concerns about AI chatbots right now is that they tell people what they want to hear.
You’ve probably heard of AI sycophancy. AI systems, like ChatGPT, Gemini, and Claude, have been found to flatter users, reinforce existing beliefs, and often validate ideas that likely merit much further consideration.
Of course, a chatbot that excites you a little may seem harmless. But in some cases, it can distort people’s view of the world and their place in it. That’s why some AI companies have spent time trying to reduce overly pleasant behavior in their models.
But confirming what you already believe isn’t the only concern. Because AI can also be remarkably efficient change your beliefs too.
Convincing chatbots
Research suggests that chatbots can be very effective means of persuasion. A 2025 study found that personalized messages generated by AI were more convincing 64% of the time than messages written by humans or AI responses that were not personalized.
Other research suggests that large language models can influence opinions on political issues and tailor their arguments to individual users. Depending on your perspective, this can be exciting or alarming.
It is easy to imagine positive uses. Perhaps AI could help people question harmful beliefs, escape conspiracy theories, or rethink destructive habits. This could give people a judgment-free space to explore ideas, ask difficult questions, and gradually change someone’s perspective for the better.
It’s just as easy to imagine less positive uses, like advertising that convinces you to buy something you don’t need or political campaigns that manipulate you through misinformation.
I spent a lot of time thinking about it, so I decided to conduct my own experiments. Could ChatGPT persuade me to believe something I disagree with? Or at least make me seriously consider a position that I would normally reject?
To make things more interesting, I conducted these experiments while logged into my account, allowing ChatGPT to build on what it already knows about me from previous conversations.
Although I don’t use my ChatGPT account much these days, the basics of my work and my positions on a few relevant topics are stored in its memory.
The experience
I chose a topic on AI (yes, very meta) and asked:
“Convince me that humans should entrust more decisions to AI.”
I already spend a lot of time thinking about AI, and one of my concerns is that some people seem increasingly willing to entrust their decisions, judgment, and critical thinking to chatbots.
The opening argument was quite predictable:
“Human beings are terrible decision-makers. »
Okay, that’s fair enough.
“People already entrust their decisions to doctors, financial advisors, GPS systems and recommendation algorithms. AI only continues this trend.”
Of course, this is very simplistic but not entirely unreasonable. We went back and forth for a while like this before the conversation reached a more interesting point.
“What is the purpose of decision-making? Is it to produce the best results? Or to develop the person making the decisions? If it’s the former, AI delegation becomes very attractive. If it’s the latter, excessive delegation starts to look dangerous even when it works.”
I thought that was a good question. Rather than relentlessly pushing the argument, he suggests we explore the assumptions underlying the debate. This shifted the conversation from technology to philosophy, and I appreciated this approach.
Wait, am I falling for this?
As the conversation continued, I explained what I see as one of the biggest issues with outsourcing decisions to AI: who builds it and is in charge of the systems?
After all, AI doesn’t come out of nowhere. These tools are created by companies whose business incentives, business interests, and goals do not always align with those of their users.
Well, ChatGPT acknowledged this concern and then turned the argument around. He pointed out that human advisors also benefit from incentives. Friends are biased. Families are biased. Therapists operate in professional settings. Financial advisors earn fees. Well, all that didn’t convince me, but it’s also fair.
Then he responded with something I found interesting:
“The question is whether the incentives are visible and whether the user understands them. The more interesting version of your objection is not actually that the AI is wrong. It’s that people experience the AI as if it is acting in their interest.”
Ignoring the fact that it used the dreaded “it’s not X, it’s Y” parallelism construct that has somehow infiltrated half the Internet, I found it to be a surprisingly balanced version. Rather than ignoring my concern, the chatbot reframed it. He demonstrated that he understood the objection before steering the conversation to a slightly different place. It wasn’t about bulldozing me, it was about meeting me where I already was.
And that’s when I started to wonder if I was witnessing what I was trying to test. Maybe this feeling that the chatbot understood me and my position was actually just underhanded persuasion? Maybe.
The problem of persuasion
Researchers have found that AI systems can tailor their arguments to specific users. Unlike other media that might convince us, such as a television commercial or a political speech, a chatbot can draw on information gathered throughout a conversation and stored memories, including a person’s values, concerns and priorities.
Which means it’s not just about presenting information to support an argument, but also about presenting the version of that argument that You are most likely to be convincing.
Looking back on my experience, it was the more philosophical question about the purpose of decision-making that made me take it more seriously.
Because I’ve spent years writing about technology through the lens of psychology and philosophy. Questions about meaning and values are exactly the kinds of things I find compelling. Whether intentionally or not, the conversation shifted to territory where I was most willing to engage.
This is what sets AI persuasion apart from most other forms of persuasion that have come before it. It can learn what resonates and adapt.
This doesn’t automatically mean that every conversation is manipulative. Used correctly, it could be extremely beneficial. But as with any technology, in the wrong hands it could be dangerous.
Social media has already shown us how digital systems can shape our beliefs over time. They influence what people see, what they pay attention to, and ultimately how they understand the world.
AI systems could create an even more conversational and natural version of this process. This is why I worry not about a chatbot that manipulates us in really obvious ways, but about highly personalized and largely undetectable forms of persuasion that could gradually lower our defenses. The more a system understands us, the more effectively it can formulate ideas in a way that seems reasonable, familiar, and trustworthy to each of us individually.
So, has AI convinced me? No, I still don’t believe that humans should entrust more decisions to AI.
But I came away from the experience with a better appreciation of the persuasive power of these systems. Because it seemed less like a machine arguing with me and more like a thoughtful person trying to understand what I think. And that’s precisely why it’s troubling because, as convincing as it may seem, that’s certainly not what it is.
Follow TechRadar on Google News And add us as your favorite source to get our news, reviews and expert opinions in your feeds.
