Will Agentic AI Replace Classic ML? Wrong Question.
Agentic AI won't out-predict a tuned gradient booster on large, clean data. Its value is in choosing and building the method, which sometimes means picking the booster.
Lately, when I need to model or predict some behaviour or a pattern buried in a messy dataset, I no longer reach first for a random forest, an SVM, or a neural network. I hand the problem to an agent.
So is agentic AI replacing classic machine learning? That question assumes the agent and the random forest compete for the same job.
The honest answer first
On raw predictive accuracy, no. Classic, data-driven ML still wins, especially on large, clean tabular datasets. The research is candid about this. When researchers had an LLM build interpretable decision trees from world knowledge alone, those trees beat their data-driven counterparts on barely a quarter of datasets. This is now a properly surveyed field.
Why the gap is real
An agent’s advantage comes from prior knowledge. It knows that income predicts loan default before it sees a single row. With fifty data points, that prior supplies information the data can’t. But on a large, clean dataset the signal is already in the data, and a well-tuned booster extracts it close to optimally.
Better agentic structure (smarter search, more careful refinement) keeps closing the gaps where structure was the bottleneck, and time will close more of them. A gap set by the data stays open, because no amount of agentic cleverness conjures signal the data doesn’t contain.
The agent is the modeller
The agent’s job is to choose and build the method, and increasingly to do it programmatically, refining the artifact as it goes. It can train a classic model when that’s the right tool, or wrap an intrinsically interpretable one when transparency matters. And when nothing off the shelf fits, it writes something bespoke.
I built Urika, my multi-agent analysis platform, to do that job. You give it a dataset and a question, and it writes a fresh method for that problem. It might train and tune a classic model and hand you a leaderboard of what worked, or its planning agent might decide a capability is missing and build a new tool on the fly.
In much of my behavioural-modelling work, the best model is a heuristic decision tree the agent authors directly, and can re-shape programmatically for each persona or individual I’m trying to model. It’s transparent, it works with very little data, and I can bend it to the problem instead of bending the problem to fit an algorithm.
The tree can cost some accuracy against a tuned booster, and that cost shrinks as the dataset gets smaller and the agent’s prior matters more. For high-stakes work where data is thin and each decision has to be explained, like the persona models above, I usually prefer the tree because I can read and audit it.
So, will it replace classic ML?
No. Classic ML becomes one of the methods the agent chooses from. On a large, clean tabular dataset, a well-built agent should pick a tuned booster like XGBoost.
Michael Richardson Professor, School of Psychological Sciences Faculty of Medicine, Health and Human Sciences Macquarie University
AI Disclosure: I wrote this article with help from AI tools, including Claude. The ideas and experiences described here are my own. The AI helped with research and with drafting and editing the text. I use AI openly in my research and teaching, and I encourage others to do the same.