- The experienced developers have in fact spent 19% more time on tasks when using AI, report complaints
- Only 44% of the code generated by AI was accepted by experienced developers
- Developers always believe that work is easier when using AI
Artificial intelligence may not be as beneficial for experienced developers as for new beginners and those who have not yet developed good skills, said new research.
A new study by model assessment and threat research (METR) suggested not only that developers were less optimistic after the study, but real results suggest that artificial intelligence ended up costing them time.
The study revealed that, with AI, the developers spent less time coding and looking for, and more time, pending, pending and more important, the exit of AI. It is estimated that 9% of the time continued to examine and clean the code generated by AI, with suggestions of AI generally on the right ways but missing retail.
AI is not really the same these developers at any time
Observing 16 experienced developers through 246 real tasks on mature open source projects they already knew, researchers have analyzed how developers interacted with popular Cursor Pro and Claude 3.5 / 3.7 tools.
Before the study, the 16 experienced developers in question expected to reduce the 24% task time during the combination of their expertise with artificial intelligence. After the study, they reduced their expectations to only 20%, but a post-study analysis reveals that AI has in fact increased the time to perform tasks by 19%.
Less than 44% of the suggestions have been accepted, with a lack of contextual knowledge and large complex standards highlighted as contributory factors to the slowdown in developers. The study also noted that experienced developers were already experiencing great familiarity with code bases, leaving little room for AI to add significant value.
However, despite the slowdown, many developers continued to use AI tools because work felt less effort, which makes work more pleasant even if it was not faster.
“AI’s capacities in nature can be lower than results on commonly used benchmarks may suggest,” concludes the research document.
