Machine Learning
Toadbook Hackathon Project
Discover how Metal Toad's team utilized AWS services and ML to create a simple social network platform at their latest hackathon. Explore the...
Discover the hidden maintenance costs of Generative AI models and how to manage frequent updates effectively to maximize productivity and longevity.
Generative AI has the potential to transform business by allowing the development of tools to increase productivity. But there is a hidden cost of maintenance when developing and supporting these tools.
Lets take a quick step back for people who don’t know software development. When building software there is a general life cycle that is followed. This is referred to as SDLC. The exact phases vary slightly but in general it’s like the below:
It’s represented as a circle because software needs to be continually supported both to keep it secure. But also to maintain its value.
In most cases this is simple. Software comes out with version numbers like 4.2.1
4 represents the major version. Updating from 4-5 will often have breaking changes that will need to be evaluated and fixed.
2 represents minor patches. These are generally safe to update because they are backwards compatible and can contain new features.
1 represents the patch version. These are generally what security falls into.
So if developing code with Python and then go from 3.12 to 3.13. You are generally safe. More importantly if you make any changes that do happen, it will produce the same result as before.
The same is not true for Generative AI Models. Though some owners like to use similar version numbers each new model is a new application. For example: the same input for sonnet 3.5 and sonnet 3.7 can create different outputs. Each has its own strengths and weaknesses.
So why is this a hidden danger? Well software is generally amortized over 3-5 years depending on the type and function. Vendors like Ubuntu help accommodate this by having long term release (LTR) versions that they guarantee to support for a longer window.
On the GenAI side it's not there yet. For example Anthropic Claude 3.5 Sonnet was released June 2024. Support ended August 2025, and it will be retired October 2025. And AWS Titan models had a similar short life time. These life times of 16ish months is far short of the 36-60 month goal of software.
Not only do you need to update much more frequently than most software. These are major updates that could require refactoring all of your prompts to get similar results.
This shouldn’t be a deal breaker. Even with this change you can still save money using these Generative AI. It's just more important to design it in a modular manner and prepare for the support that comes with it.