Smarter AI Starts by Knowing Its Limits
A new study on AI coding agents shows a clear trend: the longer a task takes, the more likely AI is to fail. Even the most advanced models struggle to complete extended tasks reliably. Humans, however, often succeed where AI falters, especially when the problem calls for persistence, reevaluation, and creative problem-solving.
"Humans do not follow this failure pattern," says Peter Vickers, Data Scientist at Northeastern’ University's AI Solutions Hub. "We are resilient. We rethink, restart, and adapt. LLMs tend to settle on one approach and stay there."
Visual comparisons in the study show this difference clearly. AI agents perform well on short tasks but see steep drops in success as tasks grow in duration. Human performance stays more stable, suggesting a different kind of reasoning at work.
"There is nothing wrong with learning from data," says Omer Alis, Associate Director of AI Solutions Hub. "But AI models only see the end result, not the decision-making that produced it. That reasoning stays hidden, like most of an iceberg below the surface."
At the AI Solutions Hub, we work with organizations that want to use AI effectively, not blindly. We design systems that play to AI’s strengths: speed, scale, automation, while preserving the value of human insight where it matters most.
Let’s explore what that could look like for your team.
