token
The unit an LLM processes — a sub-word chunk produced by the model's tokenizer, billed individually.
Definition
A token is the unit an LLM ingests and emits — a sub-word fragment produced by the model's tokenizer (BPE for GPT, sentencepiece-ish for Claude). English text averages roughly four characters per token. All LLM pricing and rate limits are denominated in tokens. Counting tokens before sending long prompts is essential for budgeting.
When to use
See also
- context window — The maximum number of tokens an LLM can consider at once — input prompt plus output combined.
- LLM — Large Language Model — a transformer-based model trained on internet-scale text to generate and reason.
Mentioned in
- Best LLM for AI Agent Workflows: 2026 Cost/Latency Comparison
- Claude Code "advisor tool is unavailable" with Fable 5: workaround
- Claude Code pricing explained: plans, token costs, and limits (2026)
- Fix n8n MCP AI Agent OAuth2 Silent Tool Failure
- How to Deploy an AI Agent to Production on AWS Lambda (Serverless)
- How to Deploy an AI Agent to Production with Express and Docker
- How to fix the Cursor Connection Error when the agent stops working
- Slack user_huddle_changed event: when it fires and how to use it in n8n