Skip to main content

How language models work

Model knowledge

Model knowledge is everything a language model saw during training, no more and no less. Answers from model knowledge are produced without a web search.

Also: Parametric knowledge · Training knowledge

Anja Miebach

Founder & Owner, CodaAI

Updated: 2 min read

Model knowledge is the knowledge a language model learned from texts during training and that is stored in its parameters. A model knows only what appeared in its training data. It does not know anything that happened afterwards, unless it fetches that knowledge via a web search.

How does model knowledge come about?

The training data comes from a few large sources: Common Crawl as the largest web snapshot, Wikipedia for structured knowledge, books and literature for long, coherent text, scientific papers for specialist depth, Reddit and forums for conversational tone, GitHub for code, news archives for events up to the knowledge cut-off, and licensed data depending on the provider.

From these texts the model does not learn facts in the sense of a database, but probabilities: which terms appear together, which brand is associated with which service, which answer typically follows which question. What is described often and consistently becomes reliably retrievable. What appears rarely or inconsistently stays vague.

Why does model knowledge matter for AI visibility?

Answers from model knowledge are produced without a web search, and thus without any chance to influence anything at the moment of the question. When the model is confident, it answers from memory, and then only what was on the web about a brand up to the time of training counts. That is why measures on third-party sites (brand mentions, trade media, Wikipedia) act on model knowledge in the long term, while your own website works mainly through grounding.

What does this mean for your website?

Model knowledge only changes with the next training run. In the short term you reach the AI via web search, in the long term via a consistent brand image across the whole web. Check both separately: ask the model with web search switched off what it knows about your company, and compare the result with the answer including web search. The difference shows which path you need to work on first.

From the term to the number

How does your company score on this factor?

The Digital Visibility Audit measures, on real buyer questions, whether ChatGPT and Google AI Overviews mention your company — and what is holding it back if they do not.

Where should we send your audit?

Thank you. The CodaAI team is preparing your personal Digital Visibility Audit and will send it to you within 24 hours.