You've probably heard the hype: AI is going to transform your business! Cut costs by 480%! 10x your revenue! Your competitors are already doing it! And somewhere in all that excitement, someone approved a budget and a project kicked off without anyone ever stopping to ask: Wait, how will we actually know if this is cost-effective? Now try to imagine an even scarier scenario: Your business doesn’t experiment with AI at all.
That's what this post is about. By the time you finish reading, you'll know how to think through:
How to identify the right inputs before you build anything;
The crucial difference between automating existing work and creating something new;
How to build a real Total Cost of Ownership (TCO) model, not a *back-of-napkin guess;
Why great AI plus terrible UX equals a failed project;
One surprising thing about AI math that most people get wrong.
Note: no napkins were harmed during the writing process of this article
We know math is hard, and most people don’t like it, but trust us on this: math is as afraid of you as you are afraid of it. But you still need it, so bear with us.
Before you prompt a single line of code or sign a single contract, you need to answer three deceptively simple questions:
What problem are you actually solving?
Who is doing that work today, and how long does it take?
What does it cost you (in time, money, and resources) currently?
Think of it like planning a road trip. You wouldn't just hop in the car and start driving (unless something has gone really wrong). You'd figure out where you're starting, where you're going, how long it will take, and how much gas you'll need. Your AI project is the same deal.
If it is an especially treacherous trip, up winding roads and mountain passes, through inclement weather, and you’re in a hurry to-boot, sometimes it’s nice to have a Chauffeur who has been that way before. Like Metal Toad. Just sayin’.
Volume: How many times does this task happen per day/week/month?
Time: How many minutes does a human spend on it each time?
Cost: What's the fully-loaded hourly cost of the person doing it?
Revenue: If you can reduce Churn or drive New Revenue, what would 1% look like?
Error rate: How often does something go wrong, and what's the cost of that?
Opportunity cost: What could your team be doing instead of this?
Quick gut-check: if you can't fill in those fields with real numbers, you're not ready to build yet. And that's okay, getting the inputs right is half the work.
There are two fundamentally different ways AI can create value for your business (we’re simplifying here):
Example: Your sales team of 10 spends 10 hours a month manually updating their sales forecasts & CRM hygiene. AI Agents automates those tasks in minutes. You just got 100 hours back. Indeed, these were hours your reps previously spent tearing their hair out, especially Billy, your top performer.
The ROI math here is relatively straightforward: (Hours saved × Hourly cost) - Cost of AI = Your Return. See? Nothing to be scared about. You couldn’t help but notice that Billy’s hair is starting to grow back.
Example: You build an AI-powered recommendation engine that upsells customers at checkout, something your sales team was doing manually.
The ROI math here is trickier because you're estimating revenue that hasn't happened yet. You'll need conversion assumptions, average deal sizes, and a healthy dose of humility about your projections.
Many projects are actually both; and that's fine! Just make sure you're accounting for both in your model, and being honest about which numbers are hard data versus educated guesses.
Pro tip: Start with automation. The math is cleaner, the gains are tangible, your teams will love you, and you build both credibility and muscle to go after the bigger opportunities later.
TCO stands for Total Cost of Ownership. It sounds intimidating, but it's just a fancy way of saying: add up everything it actually costs to build and run this thing + when do I see returns.
Most people only budget for the show-ier things they can see. They forget about:
Integration with your existing systems
API’s with external systems
Security and compliance
Driving user adoption
Ongoing maintenance (more on this in a minute)
Baking in engineering time to fix it if something breaks
A solid TCO workbook has three columns: Year 1, Year 2, and Year 3. That's because the ROI curve for AI projects usually looks like a hockey stick, not a straight line. #GoOilers
Here's a simplified example to download
Here's a saying we love at Metal Toad:
"Good code is like milk. It goes bad."
The world changes. Your business logic evolves and AI models get updated. Security updates require patching. That beautiful thing you built six months ago needs tending. And if it’s old milk it needs throwing out.
Budget for maintenance from day one. A good rule of thumb: plan to spend 15-20% of your initial build cost per year on ongoing upkeep. That includes:
Model retraining or tuning as your data changes
Monitoring for accuracy drift (when the app slowly starts getting worse and nobody notices)
Keeping up with security patches and updates (non-negotiable)
Bug fixes and edge cases you didn't anticipate
The User eXperience (UX): Problem No One Talks About
You could build the most sophisticated AI system in the history of your industry. Flawless accuracy. Blazing speed. Jaw-dropping capabilities. Much Wow.
And if it's confusing to use, nobody will use it.
User adoption is where AI projects go to die quietly. People fall back to their old spreadsheets and habits; most go back to their preferred AI tools (disparate across your organization), and the project gets labeled a failure. Not because the AI failed but because nobody made it easy to use. Do not go gentle into that good night.
Great UX for AI tools means:
The output is displayed where the user already works (not in a separate tab they have to remember to open)
There's a clear way to give feedback when the AI gets it wrong (also more on that later)
The first experience makes it obvious what the tool does and why it helps
Training is built in, not bolted on as an afterthought
If you don't have a plan for how people will actually start using your AI tool on day one, your ROI numbers belong in the Fiction section at Barnes and Noble.
Okay, here's something most AI vendors won't tell you upfront.
Large Language Models (LLMs), a.k.a. ChatGPT, Claude, and most of the AI tools you've heard of, are incredibly good at understanding language, reasoning through problems, and generating text.
They are terrible at math.
“LLMs are probabilistic, not deterministic. That means they don't calculate: they predict. And sometimes they predict wrong answers with complete confidence.”
Elia, founder of TrueMath
This is sometimes called "hallucination" when an AI just... makes something up. It's not lying, exactly. It's more akin to a person spouting something demonstrably bonkers, confidently, and believing they are right, like your uncle Barry talking about politics at Thanksgiving. It’s time to stop doing that, Barry (start reading books again or pick up a new hobby like birding). In any case, being right is not just about being confident, and unfortunately when people (or AI) say things confidently, it can be easy to miss the facts.
For your ROI calculations, your financial models, your pricing logic: never let the LLM do the actual arithmetic. Instead:
Use the LLM to understand and organize the data
Pass the actual calculations to a deterministic system (like a real math engine, a database, or even a spreadsheet)
Let the LLM explain the results in plain language
Think of it this way: LLMs are brilliant writers and researchers. They're not accountants. Build your system accordingly, and you'll avoid a whole category of expensive surprises.