You ask AI one little question.
So why does someone suddenly say,
we're spending too many tokens?
That sounds as though the AI escaped into an arcade.
It did not.
When an AI model works with text,
it processes small pieces called tokens.
A token might be a whole word,
part of a word, or punctuation.
Spaces can be included in those pieces too.
The exact split depends on the model.
So counting words and counting tokens are not quite the
same thing.
Input tokens,
what goes in.
Output tokens,
what comes back.
But the input can include more than the question you
just typed.
Instructions,
conversation history,
documents,
information fetched from other systems.
Your little question may have brought luggage.
All of that gives the model material to work with,
which brings us to the work table.
The model’s context window is the amount of material it
can work with at once,
including room for its answer.
Imagine asking someone to find the holiday policy.
You could hand them the relevant document,
or you could empty the entire filing cabinet onto their
desk and say,
it's probably in there.
You have certainly given them more information.
You have not necessarily given them more help.
The same principle applies here.
Relevant context helps.
Unnecessary context can bury the thing that matters.
And that work table has edges.
A context limit tells you how much fits at once.
A rate limit tells you how many requests,
or how many tokens,
a service allows over a period of time.
The size of the table,
the pace of the work,
different limits.
Tokens also matter because some AI services charge
according to how many they process.
What goes in and what comes back can have different prices.
But a token count is not an intelligence score.
And information sitting on the work table is not a
promise that the system will remember it next week.
So before you paste 80 pages into a prompt,
ask,
what am I asking it to do?
Which information does it need?
And how much answer would actually help?
Sometimes it needs all 80 pages.
Sometimes it needs paragraph three.
More is not the goal.
Enough of the right information is.
Understand first,
automate second.
This is DoNotEnergize.
Next time,
what does autonomous actually mean?
Spoiler,
it does not mean haunted.