Language model definition
A language model is an AI model trained to predict the next word, or part of a word, in a piece of text. Large language models (LLMs) such as GPT, Claude and Gemini use that ability to answer questions, write, translate, summarize and generate code.
Language models are much older than chatbots. Phone keyboards have suggested your next word for years, and speech recognition used language models long before that to pick the likeliest sentence out of ambiguous audio. What changed in the late 2010s was scale. Models with billions of parameters, trained on a large share of the public web, turned out to be useful for far more than autocomplete.
How does a language model work?
Text is split into tokens, which can be whole words or fragments of them. The model, almost always a transformer neural network today, looks at the tokens so far and assigns a probability to every possible next token. One gets picked and appended, and the process repeats until the answer is complete.
Training happens in stages. Pretraining teaches the model to predict text by showing it an enormous amount of it. Fine-tuning, often with human feedback, then teaches it to follow instructions and hold a conversation instead of just continuing whatever it was given.
Language model vs. LLM
Every LLM is a language model, but plenty of language models are small. In everyday use the terms have merged, and "language model" almost always means an LLM. Small language models still have a place: some run on a laptop or a phone, giving up some ability in exchange for privacy and speed.
Language models in web projects
A website talks to a language model through an API. The server sends a prompt, the model sends text back, and the site uses it to draft alt text inside the CMS or to answer questions in a support widget.
The model only knows its training data and whatever is in the prompt. Prices, opening hours and the name of the product you launched last month all have to be passed in with each request, usually by searching your own content first and including the relevant parts. Skip that step and the model fills the gap with something plausible.
Related terms
- Generative AIGenerative AI is artificial intelligence that creates new content, such as text, images, audio or code, from a prompt. It learns patterns from large amounts of training data and produces output that resembles that data, instead of only sorting or scoring what already exists.
- AI agentAn AI agent is software that uses a language model to work toward a goal on its own, step by step. It decides what to do next, calls tools such as search, APIs or a browser, checks the result and keeps going until the task is done or it needs a human.
- APIAn API (application programming interface) is a defined way for one piece of software to request data or actions from another. On the web it usually means sending an HTTP request to a URL and getting a structured response back, typically JSON.