AI Token Limits Explained: Context Windows, Output Caps, and Word Counts

An AI token limit tells you how much tokenized information a model can accept or generate in one request. The two limits people often confuse are the context window, which constrains the information available during generation, and the maximum output, which caps how much the model can return. A model can advertise a million-token context and still have a much smaller output cap.

For rough English planning, 1 token is about 0.75 words, so 1,000 tokens is approximately 750 words. This is only an estimate. The real count changes with the provider's tokenizer, language, code, numbers, punctuation, and formatting.

Quick Token-to-Word Conversion

Tokens	Approximate English words	Practical reference
8,000	6,000	Long article or short report
32,000	24,000	Short book or detailed code context
64,000	48,000	Long manuscript
128,000	96,000	Multiple documents or a small repository
200,000	150,000	Large document collection
1,000,000	750,000	Large corpus; retrieval quality still varies

These figures use the 0.75-word English estimate. They are not safe values for enforcing an API limit. Count the actual request with the provider's tokenizer or token-counting endpoint.

What Is a Token?

A token is a unit produced by a tokenizer. It may be a whole word, part of a word, punctuation, whitespace, code syntax, or a sequence of characters. Models operate on token IDs rather than directly on the words you see.

The same text can produce different counts across model families because tokenizers differ. English prose often lands near the 0.75-word rule of thumb. Source code, compact JSON, uncommon names, and some non-English languages can use more tokens per visible word.

If you need to understand why a word is split a certain way, read how AI tokenization works. For capacity planning, remember only this: characters, words, and tokens are not interchangeable units.

Context Window vs Maximum Output

The context window is the model's working budget for a request. Depending on the provider and interface, that budget can include:

• system and developer instructions;
• the current user message and attached files;
• earlier messages retained in the conversation;
• tool definitions, tool calls, and tool results;
• retrieved passages from search or a knowledge base;
• reasoning or thinking tokens; and
• the response being generated.

The maximum output is a separate ceiling on generated tokens. Suppose a model supports a 1M context window but a 65,536-token output limit. It can inspect a very large input, but it cannot return a million-token answer.

Provider behavior near the limit is not universal. An API may reject an oversized request, stop generation with a context-related reason, compact older context, or expose a smaller usable input area because space is reserved for instructions and output. Read the documentation for the exact model and endpoint rather than assuming every chat product behaves like its underlying API.

Current Model Token Limits

The table below uses provider documentation retrieved on August 12, 2026. Model aliases, preview status, and limits can change, so follow the source link before designing a production workload.

Model	Input or context limit	Maximum output	Status and caveat
OpenAI GPT-5.6 Sol	1,050,000	128,000	Current frontier API model; long prompts can have different pricing rules.
Anthropic Claude Opus 5	1,000,000	128,000	Current Claude API model; thinking tokens also consume budget.
Anthropic Claude Sonnet 5	1,000,000	128,000	Faster current Claude model with the same documented context class.
Google Gemini 3.6 Flash	1,048,576 input	65,536	Stable Gemini API model as of this update.
Google Gemini 3.1 Pro Preview	1,048,576 input	65,536	Preview endpoint; preview models can change or deprecate faster.
Meta Llama 4 Scout	10,000,000 supported context	Check your serving stack	Meta says it was pre-trained and post-trained at 256K and uses length generalization for longer contexts.

That Meta caveat matters. A supported length is not the same thing as the length used for every training stage, nor does it promise identical accuracy at every position. With open-weight models, the serving provider, quantization, hardware, and inference configuration may impose a smaller operational limit.

Why Long Context Can Still Miss Information

A larger window makes more information available, but it does not guarantee perfect retrieval, ranking, or reasoning. Long inputs create several failure modes:

• the decisive detail is buried among irrelevant material;
• similar passages compete for attention;
• instructions conflict across documents;
• tables or scanned PDFs parse poorly;
• the answer requires a relationship the model does not infer; or
• older conversation context contains stale decisions.

Avoid a universal rule such as “use only 60% of the window.” The safe operating point depends on the model, task, document structure, and acceptable error rate. Build an evaluation set that places answer-bearing evidence at the beginning, middle, and end of representative inputs. Measure citation accuracy and task success as context grows.

Why Token Limits Matter in Practice

Document analysis

A large context may let you submit an entire contract set, research corpus, or manual. You still need to verify that all files parsed correctly and require page-level citations. For repetitive question answering over many documents, retrieval is usually more economical than resending the corpus.

Long conversations

Every retained turn competes with the current task. Chat systems may summarize or drop older messages. Production assistants should persist important state as structured facts, decisions, and artifacts rather than relying on an indefinitely growing transcript.

Agents and tools

Agent loops consume context through plans, code, screenshots, schemas, tool outputs, and reasoning. A verbose tool result can crowd out the task itself. Return only fields the next step needs, paginate large results, and store bulky artifacts outside the conversation.

Cost and latency

API billing is usually token-based, and long inputs take time to transmit and process. Output tokens are often priced differently from input tokens. Calculate cost from the exact model's current price sheet; there is no useful universal “cost per million tokens” across providers.

How to Work Within Token Limits

1. Count before sending

Use the provider's token-counting method for the exact model. Google documents a models endpoint that returns input and output limits, Anthropic exposes token counting and model capability fields, and OpenAI publishes model specifications. Reject or transform oversized payloads before they reach the generation call.

2. Reserve output space

Do not fill the available context with input and then request a long answer. Reserve enough space for tool calls, reasoning, and the maximum useful response. Put a smaller explicit output cap on tasks that should return a table, classification, or short JSON object.

3. Retrieve only relevant evidence

Retrieval-augmented generation indexes a larger collection and selects likely relevant passages for each query. It works well when a question needs a small part of a large knowledge base. Preserve source IDs and metadata so the answer can cite its evidence.

4. Chunk by structure

Split on document boundaries, headings, clauses, or semantic units rather than arbitrary character counts. Keep enough neighboring context to interpret a passage, but do not duplicate large overlaps across every chunk. Evaluate several chunk sizes with your actual questions.

5. Summarize state, not evidence

Conversation compaction is useful for settled decisions, completed steps, and user preferences. Do not replace critical source evidence with an unsupported summary. Keep authoritative documents retrievable and distinguish source text from generated state.

6. Cache repeated context

Provider prompt caching can reduce the cost or latency of repeated prefixes. It does not increase the model's context window, but it can make repeated policies, schemas, and reference material more efficient.

7. Use a smaller model when the task is small

A huge context window is not automatically a better or cheaper model. Route short extraction, classification, and formatting tasks to a model that meets the quality requirement with lower latency and cost.

Large Context or RAG?

Use large context when the model genuinely needs to compare many distant parts of the input in one pass: reconciling clauses across related agreements, reviewing repository-wide architecture, or tracing a narrative across a long transcript.

Use retrieval when each question needs only a few passages from a much larger collection: support documentation, product catalogs, policies, or an internal knowledge base.

Use both when retrieval can narrow the corpus and the model then needs to reason over a substantial evidence set. This hybrid is often easier to evaluate because you can inspect both retrieval recall and answer accuracy.

A Practical Capacity Checklist

Before choosing a model, answer these questions:

1. What is the 95th-percentile token count of the real input?
2. How many tokens must remain for the useful answer and tool calls?
3. Does the task need global comparison or targeted retrieval?
4. Can every conclusion cite a source location?
5. What happens when the request exceeds the limit?
6. How will conversation state be compacted or persisted?
7. What accuracy, latency, and cost thresholds must the workflow meet?
8. Which automated tests cover evidence near different context positions?

That checklist is more useful than selecting the model with the largest number in a comparison table.

Frequently Asked Questions

Related Guides

• What Is a Large Language Model (LLM): Explained Simply
• What Is AI Inference vs Training: Key Differences
• What Is a Chatbot vs an AI Assistant vs an AI Agent