Glossary · Technical concept
Token (LLM)
The atomic unit an LLM operates on. Tokenisation converts text into integer ids the model can process; one token is typically ~3-4 characters of English text. Token counts drive both cost (most APIs price per token) and capability bounds (context-window limits are token-denominated).