Tokens & limits
Understand AI usage and limits¶
AI tools measure text in tokens. Knowing what they count helps you understand long chats, interrupted tasks, and the usage of an agent that reads files and uses tools.
What is a token?¶
A token is a chunk of text. It can be a whole word, part of a word, punctuation, or a combination that includes spaces. The count depends on the model's tokenizer and the language. See OpenAI's explanation of tokens.
A longer document usually contains more tokens, but words and tokens are different units. You don't need to count them by hand for ordinary chat.
What affects usage?¶
-
The material
Instructions, conversation history, selected file content, and tool results can all become input.
-
The work
An agent may make several model calls while researching, editing, and checking a result.
-
The answer
A longer answer adds output tokens. Some models also use internal reasoning tokens.
-
The product
API billing, chat-plan allowances, and agent usage limits can follow different rules.
Input, output, and context¶
| Term | What it means |
|---|---|
| Input tokens | The instructions and material supplied to the model for a request |
| Output tokens | Content generated by the model; some providers include internal reasoning in this count |
| Context window | The maximum amount of tokenized content a model can handle in one request |
For OpenAI reasoning models, internal reasoning uses context space and is billed as output in the API. It may not appear in the visible answer. See the reasoning documentation.
A context window is a working limit, not permanent memory. An app may select, retrieve, or summarize information before sending it to the model. It may not send every file or the entire chat history each time. OpenAI's conversation-state guide explains the limits for API requests.
How to use context well¶
- Name the relevant sources and say what the agent should use them for.
- Keep the task, constraints, and required output easy to find.
- Ask for enough detail to review the result.
- For a large job, save decisions and progress in a file the agent can read again.
- Preserve original sources when making summaries. A summary can omit a detail you later need.
Don't shorten a brief by removing facts the task depends on. The aim is to keep useful context available and reduce irrelevant material.
A clearer brief¶
Instead of “read everything and do what you think is best,” give the agent a bounded result:
Use these three notes to draft a beginner guide.
Preserve the approved facts and links. Aim for 700 words.
Save a new draft; do not overwrite or publish the originals.
Check that each factual claim is supported by the notes.
This makes the work easier to assess. It doesn't promise a particular token saving.
When bigger context helps¶
A larger context window can be useful for comparing documents, checking contradictions, or editing a project that depends on several files. It doesn't guarantee that every detail will be used correctly.
If the agent loses track of an instruction, check what information it actually received. Missing source access, conflicting rules, and context limits are different problems.
Check the right limit¶
A usage cap is different from a context-window limit or a temporary rate limit. When a task stops, read the message and check your tool's usage screen before changing the prompt or buying anything.
For agent work, review total usage across the run. A short final answer can follow many tool calls and model requests.