# Token

> A token is the smallest unit a language model works with: a word, part of a word or a single character. All probabilities of an LLM refer to tokens.

Source: https://www.codaai.ai/en/knowledge/geo-glossary/token/
Published: 2026-09-03
May be quoted with attribution and a link to the source.

---

A token is the smallest unit a [language model](https://www.codaai.ai/en/knowledge/geo-glossary/llm/) computes with. Depending on the model, a token is a whole word, part of a word or a single character. The German word "Sichtbarkeit" (visibility), for example, is split into two or three tokens, and a full stop is a token of its own. Everything a model does — reading, weighting, generating — happens at this level.

## How do tokens work?

Before a model processes a text, a tokenizer splits it into tokens and assigns a number to each one. The model then estimates, for each position, which token follows with which probability. For further processing as meaning, tokens or whole sections are translated into vectors (see [embedding](https://www.codaai.ai/en/knowledge/geo-glossary/embedding/)).

Tokens are also the currency of the context window. Each model can only keep a certain number of tokens "in view" at once. The user's question, the system's instructions and all excerpts from web pages the system found in a web search must fit into this window.

## Why do tokens matter for AI visibility?

Because context is scarce and expensive, AI systems do not give a page unlimited room. They extract passages ([grounding snippets](https://www.codaai.ai/en/knowledge/geo-glossary/grounding-snippets/)) instead of taking over whole pages. A text that delivers its core statement early and in short, self-contained sentences has better chances within this budget than a text that gets to the point on page three.

## What does this mean for your website?

Keep an eye on the ratio of visible text to source code. On pages where readable text makes up only a few percent of the HTML, agents and crawlers mainly process markup. Lean, semantic HTML and content that is available without JavaScript ensure that the tokens a system spends on your page actually carry your content.