If you are like me, and have played around with GenAI, you may have been delighted when you were chatting with the AI and it remembered what you had just written. Unlike most other types of technology, with generative AI each line entered doesn't live in isolation. This is a function of tokens.
What are tokens
Tokens are a core concept in artificial intelligence. In the world of generative AI, like the models used for chat or content creation, tokens have a different meaning. For the sake of simplicity, you can think of tokens as the number of words a GenAI model has the ability to process at once, before it starts losing track of things. A token could be as small as a single letter or punctuation mark, or as large as recurring phrase. The more tokens a model can handle, the longer and more complex the conversations or outputs it can generate without getting confused.
think of tokens as the number of words a GenAI model has the ability to process at once, before it starts losing track of things
Once a system runs out of tokens, the output you see from a model can deteriorate quickly:

Though the term "token" has different meanings in these contexts, the unifying idea is that tokens represent the basic units a system uses to function. Whether it’s enabling digital transactions or powering AI conversations, tokens are the building blocks that make these technologies work.
GenAI models & their token count
The number of tokens each model has varies pretty wildly. Older models of ChatGPT might have 4,000 tokens while the latest releases from Google are up to 2 million.
That said, more tokens don't necessarily mean better; you have other dimension you can also consider: cost, speed, and more.
There's a lot to keep track of in the world of GenAI, so I've created this living Google spreadsheet which catalogs all the models I'm aware of along with the following information:
- Model name
- Token count
- Inputs (Text, Image, Video, audio)
- Outputs (Text, Image, Video, audio)
- Supported language #
- Can be run in Bedrock (yes/no)
- Allows fine tuning (yes/no)
- Whether it is deprecated or not
