Can Agentic Engineers Estimate Software Delivery in the Age of AI?

In the age of AI how do agentic engineers estimate their ability to deliver on engineering tasks?

I don't think it is actually possible, anymore, but engineers are shipping more code! What a time to be alive.

In the old days, engineers and their managers would estimate engineering effort based on human intelligence, human experience completing tasks of similar scope, as well as the maturity of the codebase. Humans knew what they were working on because they had to read the code. Even then, those engineering estimates were often wrong and usually we had to read books like "The Mythical Man Month" to build disciplines in software development effort estimates. I recently laughed watching Alberta's from Tech "What tech meeting feels like" on Youtube: https://www.youtube.com/watch?v=gnf2TRXi5iw. Those days seem quaint now. Do we still use story points when building with AI? Do we ask the LLM for story points?

Now today what happens when we ask an LLM: "How long will this feature take?"

We get two things:

1. The LLMs lie: They sandbag estimates and assert the effort will take centuries.
2. The LLMs suck-up: When pushed and challenged on their estimates, they suck up and claim that you were "right" to challenge them and their estimates. Whatever you say the estimate is, they'll 100% agree.

The truth: LLMs cannot reliably or accurately estimate their own efforts nor human effort. It's impossible. They have no capacity of human introspection and their memory is poor. Also, they are probability matrices. Repeating the same engineering task can actually differ on output and time. The best they can do is average over past responses. On the flip side, we can use our LLMs to figure out complex and arcane commands that would have taken us weeks without expert assistance. Producing quality engineering output is fast.

So when the LLMs publish tokens, engineers have no idea what they will get and at what speed they will get the tokens. LLMs are like a box of chocolates. Complexity of output is prompt and temperature setting driven. It's the wild, wild, wild west. Buckle up buttercup. When your application becomes complex with millions of lines of code, the LLMs take increasingly more and more time to make changes that don't break the entire system. Do we have this problem with humans who know the codebase? Does human capacity for accurate response decrease when lines of code increase? I wonder about this.

Now add to this the cost of tokens and you can quickly see how engineering with AI while allowing engineers to deliver amazing new output is also impossible to estimate.

Of course, you can say it's a "skill issue". Faster and cheaper LLMs will make software delivery estimates trivial across all types of codebases. We just haven't figured out the right skill to make that happen as yet. If you have figured out the right skill to solve this problem, and it can be testable, please let me know. Email us at hello@serendb.com.

More from the SerenAI Blog

• Don't Let Your Corporate Agentic Brain Be The Next Honey Pot For A Rogue AI
• The AI Productivity Boom Is a Jobs Boom
• Seren Speeds Up Human-to-AI Knowledge Transfer
• Claude Code's Local Memory Is a Security Risk, and You Can Verify It Yourself

About SerenAI

SerenAI builds the infrastructure layer for agentic software: AI employees that discover tools, pay for data, run on schedules, and execute real workflows with the credential controls, work orders, and audit trails that let a company actually hand them the keys.

Taariq Lewis

CEO, Seren

https://serendb.com

Docs at https://docs.serendb.com/

Email us at hello@serendb.com