Glossary

Plain-English definitions of 202 AI and machine learning terms, from activation functions to zero-shot learning, each linked to a lesson or lab.

Accuracy
The fraction of predictions a model gets right. It is easy to read but misleading on imbalanced data: a model that always predicts “no fraud” is 99.9% accurate if fraud is rare.
Activation function
A non-linear function applied to a neuron’s weighted sum, such as ReLU or sigmoid. Without it, any stack of layers would collapse into a single linear model.
Active parameters
The parameters a model actually uses to process one token. For a mixture of experts this is far below the total: DeepSeek-V3 has 671B parameters but 37B active.
Adam
A widely used optimizer that keeps running averages of each parameter’s gradient and squared gradient, giving every weight its own adaptive step size. It combines ideas from momentum and RMSProp.
Agent
A system that repeatedly observes its situation, decides on an action and acts, working towards a goal over many steps. Today the term usually means a language model running in a loop that can call tools and read their results.
Algorithmic bias
Systematic differences in how a model treats groups of people, usually inherited from skewed training data, proxy features or how the target was defined. It can appear even when protected attributes are removed from the input.
Alignment
The problem of making an AI system pursue the goals its designers and users actually intend, including in situations its training did not cover. It spans practical techniques such as RLHF and open research questions about more capable systems.
Anomaly detection
Finding data points that do not fit the pattern of the rest, such as fraudulent transactions or failing sensors. It is often unsupervised because examples of every possible anomaly are rarely available.
Artificial general intelligence
A hypothetical system able to learn and perform most intellectual tasks a human can, across domains. There is no agreed test for it, and today’s systems remain uneven: strong at some tasks, unreliable at others.
Attention
A mechanism that lets a model weigh every other token when building the representation of one token. Scores come from comparing queries with keys, and the output is a weighted mix of values.
Autoregressive model
A model that generates a sequence one element at a time, feeding each output back in as input for the next step. Most large language models generate text this way, token by token.
Backpropagation
The algorithm that computes how much each weight contributed to the error by applying the chain rule backwards through the network.
Bagging
Training many models on different random resamples of the training data and averaging their predictions. Averaging cancels out much of each model’s individual variance.
Base model
A language model after pretraining only. It continues documents rather than following instructions; post-training turns it into an assistant.
Bayes' rule
A formula for updating a probability when new evidence arrives: the chance of a hypothesis given the evidence is proportional to how likely the evidence is under that hypothesis times how likely the hypothesis was beforehand. It explains why a positive result from an accurate test can still usually be wrong when the condition is rare.
Bellman equation
The recursive rule at the heart of reinforcement learning: the value of a state equals the immediate reward plus the discounted value of the state that follows. Value iteration and Q-learning are both ways of solving it.
Benchmark
A fixed dataset and scoring rule used to compare models, such as ImageNet for vision or SWE-bench for coding agents. Scores can overstate real ability when test items leak into training data or models are tuned to the benchmark itself.
Best-of-N
Generate N candidate answers and keep the one a verifier or reward model scores highest. Gains flatten at a ceiling set by the verifier’s blind spots.
Bias-variance trade-off
The tension between a model too simple to capture the pattern (high bias, underfitting) and one so flexible it chases noise (high variance, overfitting). Good models sit between the two.
BLEU score
A metric for machine translation that counts how many word sequences in a system’s output also appear in human reference translations. It is cheap and repeatable but correlates only loosely with human judgements of quality.
Bounding box
The rectangle, usually given by corner or centre coordinates plus width and height, that an object detector draws around each object it finds.
Bradley-Terry model
A model of paired comparisons in which the probability that A beats B is the logistic function of the difference of their scores. Reward models are trained with it.
Budget forcing
Controlling how long a model thinks at inference by cutting its reasoning off at a limit or appending “Wait” to make it continue.
Byte-level tokenizer
A tokenizer whose base vocabulary is the 256 possible UTF-8 byte values, so any text can be encoded and nothing is ever unknown.
Byte-pair encoding (BPE)
A way to build a tokenizer vocabulary: start from single characters or bytes and repeatedly merge the most frequent adjacent pair into a new token.
Calibration
How well a model’s stated confidence matches how often it is right. A calibrated model that says 80% is correct about 80% of the time, which lets you set thresholds and decide when to defer to a human.
Chain of thought
Intermediate reasoning steps a language model writes out before its final answer. Prompting for it improves results on multi-step problems, and reasoning models are trained to produce long chains of thought on their own.
Chain-of-thought faithfulness
Whether a model’s written reasoning reflects the factors that actually drove its answer. Studies find models often leave influences such as hints unmentioned.
Chinchilla
A 70-billion-parameter DeepMind model (2022) trained on 1.4 trillion tokens. With the same compute as the 280B Gopher it performed better, showing earlier large models were undertrained.
Chunking
Splitting documents into passages before embedding them for retrieval. Chunks that are too large blur several topics into one vector; chunks that are too small lose the context needed to answer a question.
Classification
A supervised learning task where the model assigns each input to one of a fixed set of categories, such as spam or not spam.
CLIP
A model from OpenAI (2021) that learns a shared embedding space for images and text by training on hundreds of millions of image-caption pairs. Comparing an image’s embedding with embeddings of label descriptions gives zero-shot classification.
Clustering
Grouping data points so that points in the same group are more similar to each other than to points in other groups, without any labels telling the algorithm what the groups are.
Cold start
The problem of recommending to a new user, or recommending a new item, before there is any interaction data about it.
Collaborative filtering
Recommending items by using the behaviour of similar users or the co-occurrence of items, rather than the content of the items themselves. People who liked what you liked are used as a guide to what you will like next.
Compute-optimal training
Splitting a fixed compute budget between model size and training tokens to reach the lowest loss. Chinchilla found both should grow roughly as the square root of compute, about 20 tokens per parameter.
Confusion matrix
A table counting true positives, false positives, true negatives and false negatives. Almost every classification metric, from precision to recall to specificity, is a ratio of its cells.
Constitutional AI
A training method introduced by Anthropic in 2022 in which a model critiques and revises its own outputs against a written list of principles, and AI-generated preference labels replace many human ones.
Context window
The maximum number of tokens a language model can take into account at once, covering the prompt, any documents or tool results, and its own output. Anything outside the window is invisible to the model.
Contrastive learning
Learning representations by pulling matching pairs (an image and its caption, two crops of one photo) together in embedding space and pushing mismatched pairs apart.
Convolution
Sliding a small grid of weights, called a kernel or filter, across an image and taking a weighted sum at each position. Different kernels blur, sharpen or detect edges.
Convolutional neural network
A neural network built from convolution layers that learn their own kernels. Early layers tend to detect edges and textures, later layers respond to parts and whole objects.
Cosine similarity
The cosine of the angle between two vectors: 1 when they point the same way, 0 when they are unrelated. It is the usual way to compare embeddings because it ignores vector length.
Cross-entropy loss
The standard loss for classification and language modelling: the negative log of the probability the model assigned to the correct answer. Confident wrong answers are punished heavily.
Cross-validation
Estimating performance by splitting the data into k folds, training on k minus 1 of them and testing on the remaining one, then rotating and averaging. It gives a steadier estimate than a single split.
Data augmentation
Creating extra training examples by transforming existing ones, for instance flipping, cropping or recolouring images, so the model learns to ignore changes that should not affect the label.
DBSCAN
A density-based clustering algorithm that grows clusters from points with many close neighbours and labels isolated points as noise. Unlike k-means it finds oddly shaped clusters and does not need the number of clusters up front.
Decision model
A model that returns a choice from a set of options you define, with a probability for each, instead of generating free text. Because the output is typed and scored, code can branch on it and thresholds can be set on its confidence.
Decision tree
A model that makes predictions through a sequence of yes or no questions about the input features, choosing each split to make the resulting groups as pure as possible.
Deep learning
Machine learning with neural networks that have many layers, each learning a more abstract representation of the input. It powers modern vision, speech and language systems.
Demographic parity
A fairness criterion that asks a model to give positive outcomes, such as loan approvals, at the same rate to every group. It can conflict with other criteria when groups have different base rates.
Diffusion model
A generative model trained to remove noise from data. To generate, it starts from pure noise and denoises step by step until an image, audio clip or video emerges.
Dijkstra’s algorithm
A shortest-path algorithm that always expands the unvisited node with the lowest total cost from the start. It is A* with a heuristic of zero.
Dimensionality reduction
Compressing data with many features into a few while keeping as much structure as possible, for visualisation, speed or noise removal. PCA, t-SNE and UMAP are common methods.
Direct preference optimization
A method (Rafailov et al., 2023) for training a language model on pairs of preferred and rejected answers directly, without fitting a separate reward model or running reinforcement learning.
Discount factor
A number between 0 and 1, written γ, that sets how much a reinforcement learning agent values future rewards compared with immediate ones. A reward k steps away is weighted by γ to the power k.
Distillation
Training a smaller student model to imitate a larger teacher model’s outputs. The student is cheaper to run and often keeps much of the teacher’s ability on the tasks it was distilled on.
Dot product
Multiply two vectors element by element and add up the results. It is large when the vectors point the same way, and it is the core operation inside neurons, attention and similarity search.
Dropout
A regularisation technique that randomly switches off a fraction of neurons at each training step, so the network cannot rely on any single unit and generalises better.
Embedding
A list of numbers that represents an item, such as a word, document or image, so that similar items end up close together. Embeddings turn meaning into geometry that can be searched and compared.
Emergent ability
A capability that is near zero in smaller models and appears abruptly in larger ones. Some apparent jumps come from all-or-nothing metrics such as exact match.
Encoder-decoder
An architecture where one network reads the input into an internal representation and a second network generates the output from it. It was the basis of neural machine translation and of the original transformer.
Epoch
One full pass of the training algorithm over every example in the training set.
Equal opportunity
A fairness criterion that asks for the same true positive rate in every group: people who truly qualify should be approved at equal rates regardless of group.
Expected calibration error
A single-number summary of miscalibration: bucket predictions by confidence, then average the gap between confidence and accuracy across buckets, weighted by how many predictions fall in each.
Expert collapse
A failure where the router sends almost all tokens to a few experts, which then train faster and get chosen even more, leaving the rest idle. Prevented with load balancing.
Expert parallelism
Serving or training a mixture of experts with different experts on different GPUs; tokens are sent to their experts and back with two all-to-all exchanges per MoE layer.
Explainability
Methods for understanding why a model produced a particular output, such as feature attributions or inspecting internal activations. Related to interpretability, which studies how models work internally.
Exploration vs exploitation
The dilemma of whether to take the action that looks best so far (exploit) or try something less certain that might turn out better (explore). Every learning agent that acts has to balance the two.
F1 score
The harmonic mean of precision and recall. It is high only when both are high, which makes it a common single number for imbalanced classification.
Feature
One measurable input to a model, such as a house’s floor area or a pixel’s brightness. Deep networks learn their own internal features from raw input.
Few-shot prompting
Showing a language model a handful of worked examples in the prompt so it can infer the task and the output format, with no change to its weights.
Fine-tuning
Continuing to train a pretrained model on a smaller, task-specific dataset so it adapts to a new domain, style or task while keeping what it learned in pretraining.
Foundation model
A large model pretrained on broad data that can be adapted to many downstream tasks by prompting or fine-tuning. Large language models and CLIP are examples.
Generalisation
How well a model performs on new data it has not seen during training. It is the actual goal of machine learning; performance on the training set is only a means.
Generative adversarial network
A generator network that produces samples and a discriminator network that tries to tell them from real data, trained against each other. GANs led image generation from 2014 until diffusion models overtook them.
Genetic algorithm
An optimization method inspired by evolution: keep a population of candidate solutions, combine and mutate the better ones, and repeat for many generations.
Gini impurity
The chance that two items drawn at random from a group have different labels. Decision trees choose splits that reduce it the most.
Glitch token
A token that is in the vocabulary but was rarely or never seen while training the model, so it triggers erratic behaviour. SolidGoldMagikarp is the best-known example.
Gradient boosting
Building an ensemble of small decision trees one at a time, where each new tree is fitted to the errors the ensemble still makes. Libraries such as XGBoost, LightGBM and CatBoost implement it and remain strong on tabular data.
Gradient descent
An optimization method that repeatedly nudges parameters in the direction that most reduces the loss. The step size is the learning rate.
GRPO
Group Relative Policy Optimization: a reinforcement learning method that scores each sampled answer against the average of its group for the same question, with no separate critic network.
Hallucination
When a generative model states something fluent and confident that is false or unsupported, such as a made-up citation. It follows from training models to produce plausible text rather than verified facts.
Heuristic
A fast, approximate estimate used to guide search, such as straight-line distance to the goal on a map. A good heuristic lets an algorithm skip most of the search space.
Hyperparameter
A setting chosen before training rather than learned from data, such as learning rate, tree depth or number of layers. Hyperparameters are tuned on a validation set.
Imitation learning
Training a policy to copy an expert’s demonstrations. Plain behaviour cloning drifts once it makes a mistake the expert never did, which methods such as DAgger address by collecting corrections in the states the learner actually visits.
Inference
Running a trained model to get outputs for new inputs, as opposed to training it. For large models, inference cost per request often dominates the cost of running a product.
Instance segmentation
Labelling every pixel with both its class and which individual object it belongs to, so two overlapping people get separate masks.
Instruction tuning
Fine-tuning a pretrained language model on examples of instructions paired with good responses, turning a text predictor into an assistant that follows requests.
Intersection over union
The area where a predicted box and the true box overlap, divided by the area they cover together. Detection benchmarks usually count a prediction as correct when it reaches 0.5 or more.
Jev
A decision model released by TypeSafe AI in September 2026. Instead of generating text, it takes a piece of state and a list of typed questions and returns a choice, score or yes/no answer for each, with a probability and confidence value code can branch on.
K-means
A clustering algorithm that places k centres, assigns each point to its nearest centre, moves each centre to the mean of its points, and repeats until nothing changes.
Kalman filter
An algorithm that estimates a changing quantity, such as a robot’s position, by blending a motion model’s prediction with noisy measurements, weighting each by how uncertain it is.
KL penalty
A term in the RLHF objective that penalises the policy for drifting from the reference (SFT) model, measured by KL divergence. Its strength β trades reward against staying where the reward model can be trusted.
KV cache
Stored attention keys and values for tokens a transformer has already processed, so generating each new token does not recompute them. It speeds up generation but grows with context length and uses a lot of memory.
Label
The correct answer attached to a training example, such as “cat” for a photo. Supervised learning needs labels; collecting them is often the most expensive part of a project.
Large language model
A transformer with billions of parameters trained to predict the next token on vast amounts of text, then usually fine-tuned to follow instructions. Claude, GPT and Gemini are examples.
Latent space
The internal space of compressed representations a model learns. Nearby points decode to similar outputs, which is why moving smoothly through it can morph one image into another.
Learning rate
How big a step gradient descent takes each update. Too large and training diverges or oscillates; too small and it crawls or stalls.
Linear probe
A simple linear classifier trained on a network’s internal activations to test whether a concept can be read out from them. A successful probe shows the information is present, not that the network uses it.
Load-balancing loss
An extra training term, α·N·Σ fᵢ·Pᵢ in the Switch Transformer, that penalises uneven expert usage. DeepSeek-V3 replaced most of it with a per-expert selection bias.
LoRA
Low-rank adaptation: fine-tuning a large model by freezing its weights and training small low-rank matrices added alongside them. It cuts the memory and storage needed for fine-tuning by orders of magnitude.
Loss function
A formula that scores how wrong a model’s predictions are. Training means adjusting parameters to make this number smaller.
LSTM
Long short-term memory: a recurrent network cell with gates that decide what to store, forget and output. The gates let information and gradients survive across long sequences.
Markov decision process
The standard formal model of a sequential decision problem: states, actions, transition probabilities, rewards and a discount factor. “Markov” means the next state depends only on the current state and action.
Matrix
A rectangular grid of numbers. Multiplying a vector by a matrix transforms it, which is what every layer of a neural network does before its activation function.
Matrix factorization
Approximating a large, mostly empty user-by-item ratings matrix as the product of two thin matrices, giving every user and every item a short vector of learned taste factors.
Merge table
The ordered list of pair merges a BPE tokenizer learned. Encoding new text means replaying these merges in order.
Mixture of experts
An architecture where each layer contains many expert sub-networks and a router sends each token to only a few of them. The model can have a huge total parameter count while using a fraction of it per token.
Model collapse
Degradation when models are trained recursively on earlier models’ output: rare events in the tails of the original data distribution progressively disappear.
Model Context Protocol
An open protocol, introduced by Anthropic in 2024, that standardises how AI applications connect to tools and data sources. A tool exposed once as an MCP server can be used by any compatible client.
Model FLOPs utilisation (MFU)
The fraction of a chip’s peak arithmetic throughput that becomes useful model computation during training. Llama 3 405B reported 38% to 43%.
Multi-armed bandit
A simplified reinforcement learning problem with one state and several actions of unknown payoff. It isolates the exploration versus exploitation trade-off and is used in practice for A/B testing and recommendations.
Multi-head attention
Running several attention operations in parallel, each with its own learned projections, so different heads can track different relationships such as syntax or coreference.
Multimodal model
A model that takes in or produces more than one kind of data, such as text, images, audio and video, within a single system.
Neural network
A model made of layers of simple units, each computing a weighted sum of its inputs followed by an activation function. Training adjusts the weights so the network maps inputs to the right outputs.
Next-token prediction
The pretraining objective of language models: given the text so far, output a probability for every possible next token. Predicting text well turns out to require learning grammar, facts and some reasoning.
Non-maximum suppression
A post-processing step in object detection that keeps the highest-scoring box and discards other boxes that overlap it heavily, so each object is reported once.
Overfitting
When a model memorises its training data, including the noise, and performs worse on new data as a result.
Overtraining
Training a model on far more tokens than the compute-optimal amount so a smaller, cheaper-to-serve model reaches a target quality. Llama 3 8B saw over 15 trillion tokens.
Perceptron
The simplest artificial neuron: a weighted sum of inputs passed through a step function. Proposed by Frank Rosenblatt in 1958, it can only separate classes with a straight line.
Perplexity
A measure of how surprised a language model is by a text: the exponential of the average negative log-probability per token. Lower is better; a perplexity of 10 means the model is as uncertain as a fair choice among 10 tokens.
PID controller
A feedback controller that sets its output from the current error (proportional), the accumulated error (integral) and how fast the error is changing (derivative). It steers everything from thermostats to drones.
Policy
In reinforcement learning, the agent’s rule for choosing actions: a mapping from each state to an action or to a probability distribution over actions.
Policy gradient
A family of reinforcement learning methods that adjust a policy’s parameters directly to make high-return actions more likely. REINFORCE and PPO are policy gradient methods, and PPO was used in the original RLHF work.
Positional encoding
Information added to token embeddings so a transformer knows the order of the tokens, since attention on its own treats its input as an unordered set.
Pre-tokenization
Splitting text into word-like chunks with a regular expression before BPE, so merges never cross word, digit or punctuation boundaries.
Precision
Of all the items a model flagged as positive, the fraction that really are positive. High precision means few false alarms.
Pretraining
The first and most expensive stage of building a foundation model: training on a huge unlabelled corpus with a self-supervised objective such as next-token prediction.
Principal component analysis
A dimensionality reduction method that finds the directions along which the data varies most and projects onto the first few of them.
Process reward model
A verifier that scores each step of a solution rather than only the final answer, so bad reasoning can be caught and pruned early.
Prompt
The input text, and possibly images or files, given to a generative model. Changing the wording, examples or structure of a prompt can change the output substantially.
Prompt injection
An attack where instructions hidden in content a model reads, such as a web page or email, try to override what the user or developer asked for. It is a central security problem for agents that use tools.
Pruning
Removing weights or whole neurons that contribute little to a trained model, to make it smaller or faster, usually followed by a little retraining.
Q-learning
A model-free reinforcement learning algorithm that learns the value of every action in every state from experience, nudging each estimate towards the reward received plus the best estimated value of the next state.
Quantization
Storing and computing a model’s weights with fewer bits, for example 8 or 4 instead of 16. It shrinks memory use and speeds up inference, usually at a small cost in accuracy.
Random forest
An ensemble of decision trees, each trained on a bootstrap sample of the data and allowed to consider only a random subset of features at each split, with predictions averaged or voted.
ReAct
A prompting pattern (Yao et al., 2022) where a model alternates between writing a reasoning step and taking an action such as a search, then reads the result before reasoning again. It is the basic shape of most agent loops.
Reasoning model
A language model trained, largely with reinforcement learning on problems with checkable answers, to think at length before answering. OpenAI o1 (2024) and DeepSeek-R1 (2025) were early examples.
Recall
Of all the truly positive items, the fraction the model found. High recall means few misses, which matters most when a miss is costly, as in cancer screening.
Recurrent neural network
A network that processes a sequence one element at a time, carrying a hidden state forward as memory. Plain RNNs struggle with long-range dependencies because gradients vanish over many steps.
Regression
A supervised learning task where the model predicts a continuous number, such as a price or temperature.
Regularisation
Any technique that discourages a model from fitting noise, such as penalising large weights, dropout or stopping training early.
Reinforcement learning
Learning to act by trial and error: an agent takes actions in an environment, receives rewards, and improves its policy to collect more reward over time. No one tells it the correct action.
Reinforcement learning from human feedback
Fine-tuning a language model with reinforcement learning against a reward model trained on human comparisons of responses. It was central to InstructGPT (2022) and the first ChatGPT.
Reliability diagram
A plot of accuracy against stated confidence, bucket by bucket. A perfectly calibrated model lies on the diagonal; points below it mean overconfidence.
Reranker
A second-stage model in retrieval that reads the query and each candidate passage together and rescores them. It is slower than embedding search but more accurate, so it is run on a short list.
Retrieval-augmented generation
Answering a question by first retrieving relevant passages, usually by embedding search, and placing them in the model’s prompt. It grounds answers in specific documents and lets you cite sources.
Reward
The number an environment returns after each action, telling a reinforcement learning agent how good the immediate outcome was. The agent’s goal is to maximise total discounted reward, not any single reward.
Reward hacking
When an agent finds a way to score highly on its reward signal without doing what the designer intended, such as a boat-racing agent circling to collect points instead of finishing the race.
Reward model
A network that scores a prompt and response with a single number, trained on human or AI comparisons of which answer is better. It stands in for human judgement during RLHF.
ROC curve
A plot of true positive rate against false positive rate as the decision threshold sweeps from strict to lenient. The area under it (AUC) is the chance the model ranks a random positive above a random negative.
Router (gating network)
The small learned layer in a mixture-of-experts model that scores every expert for a token and picks which few of them run. It learns only through the gate weights it produces.
Scaling laws
Empirical relationships showing that a language model’s loss falls smoothly and predictably as parameters, training data and compute increase. They are used to plan how big to make a model for a given budget.
Self-consistency
Sampling several chains of thought for the same question and returning the most common final answer. Helps when the right answer is more likely than any single wrong answer.
Self-supervised learning
Learning from unlabelled data by hiding part of the input and training the model to predict it, such as the next word or a masked image patch. The data supplies its own labels.
Semantic segmentation
Assigning a class label to every pixel of an image, such as road, car or sky, without separating individual objects of the same class.
Sentiment analysis
Classifying the opinion or emotional tone of a text, typically as positive, negative or neutral.
Simulated annealing
A local search method that sometimes accepts worse solutions, with a probability that shrinks as a temperature parameter cools. Early randomness helps it escape local optima.
Softmax
A function that turns a list of scores into probabilities that are positive and sum to 1, with larger scores getting exponentially more weight. It is used in attention and in every language model’s output layer.
Sparse autoencoder
A network trained to reconstruct a model’s activations through a much wider layer where only a few units may be active at once. Its units often correspond to more interpretable features than the model’s own neurons.
Spectrogram
A picture of sound showing how much energy each frequency has over time, computed by running a Fourier transform on short overlapping windows. It is the usual input to speech recognition models.
Speech recognition
Converting spoken audio into text. Modern systems such as Whisper are neural networks trained on hundreds of thousands of hours of audio.
Structured output
Constraining a language model to produce output that matches a schema, such as valid JSON with specific fields, so software can parse it reliably.
Superposition
A network representing more features than it has neurons by storing them as overlapping directions, which works when features are rarely active at the same time. It is one reason individual neurons are often hard to interpret.
Supervised learning
Learning a mapping from inputs to outputs from examples where the correct output is given. Classification and regression are the two main kinds.
Sycophancy
A model telling users what they want to hear (agreeing, flattering, abandoning correct answers under pushback) rather than what is true. Often learned from human approval signals.
Temperature
A setting that divides a model’s output scores before softmax. Low temperature makes sampling nearly deterministic; high temperature flattens the distribution and makes output more varied and more error-prone.
Temperature scaling
A simple calibration fix: divide a trained classifier’s scores by a single constant tuned on validation data. It changes confidence without changing which answer ranks first.
Temporal-difference learning
Updating a value estimate towards the reward just received plus the estimated value of the next state, rather than waiting for the final outcome. Q-learning is a temporal-difference method.
Test set
Data held back from training and model selection and used once at the end to estimate performance on unseen data. Tuning on it makes the estimate optimistic.
Test-time compute
Computation spent while answering a query rather than during training, for example by generating a longer chain of thought or sampling several answers and picking one. Reasoning models improve as they are given more of it.
Text-to-speech
Generating spoken audio from text. Current systems can imitate a voice from a few seconds of sample audio, which also enables voice-cloning fraud.
Token
The unit of text a language model reads and writes, typically a word or a piece of a word. In English, one token averages roughly three quarters of a word.
Token tax
The extra tokens, and so extra cost and lost context, that some languages need for the same content because the tokenizer was trained mostly on other languages.
Tokenization
Splitting text into tokens and mapping each to an integer ID. Modern models use subword schemes such as byte-pair encoding, so rare words are built from common pieces.
Tool use
Letting a language model call external functions, such as a search engine, calculator or API, by emitting a structured request that the application executes and returns the result of.
Top-k routing
Running only the k highest-scoring experts for each token (k = 1 in Switch Transformer, 2 in Mixtral, 8 in DeepSeek-V3). Compute per token scales with k, not with the number of experts.
Top-k sampling
Sampling the next token only from the k most probable candidates, after discarding the rest and renormalising.
Top-p sampling
Sampling the next token from the smallest set of candidates whose probabilities add up to p. The set shrinks when the model is confident and grows when it is unsure.
Transfer learning
Reusing a model trained on one task as the starting point for another, such as fine-tuning an ImageNet network to spot defects on a production line with only a few hundred photos.
Transformer
The neural network architecture introduced in “Attention Is All You Need” (2017), built from stacked self-attention and feed-forward layers. It underlies nearly every modern language model and many vision models.
Travelling salesman problem
Finding the shortest round trip that visits every city once. The number of possible tours explodes factorially, so large instances are solved with heuristics and local search.
U-Net
A segmentation network (2015) with a contracting path that captures context and an expanding path that restores resolution, joined by skip connections that carry fine detail across.
Underfitting
When a model is too simple, or trained too little, to capture the real pattern, so it performs poorly even on its training data.
Unsupervised learning
Learning structure from data without labels, for example by grouping similar items, reducing dimensions or flagging outliers.
Value function
In reinforcement learning, the expected total discounted reward an agent will collect from a state (or from taking an action in a state) if it follows a given policy afterwards.
Value iteration
A planning algorithm for known environments that repeatedly applies the Bellman equation to every state until the values stop changing, then reads off the best action in each state.
Vanishing gradient
When gradients shrink as they are multiplied back through many layers or time steps, so early layers barely learn. Gated cells, residual connections and careful initialisation counter it.
Vector
An ordered list of numbers, which can also be pictured as an arrow in space. Models represent inputs, words and images as vectors.
Vector database
A database built to store embeddings and quickly find the ones nearest to a query vector, usually with an approximate nearest-neighbour index. It is the retrieval layer in most RAG systems.
Vision transformer
A transformer applied to images by cutting them into small patches and treating each patch as a token. Introduced in 2020, it rivals CNNs when trained on enough data.
Vision-language model
A model that takes images and text together and answers in text, typically an image encoder connected to a language model. It can describe photos, read charts and answer questions about screenshots.
Vision-language-action model
A robot foundation model that takes camera images and a language instruction and outputs motor actions directly, trained on large collections of robot demonstrations.
Word embedding
A vector for each word learned from the contexts it appears in, so words used in similar ways get similar vectors. Word2vec (2013) made the idea famous.
Word error rate
The standard speech recognition metric: the number of word substitutions, deletions and insertions needed to turn the system’s transcript into the reference, divided by the number of reference words.
World model
A learned model of how an environment changes in response to actions, which an agent can use to imagine outcomes and plan before acting.
YOLO
You Only Look Once: a family of one-stage object detectors, first published in 2015, that predict all boxes and classes in a single pass of the network, fast enough for real-time video.
Zero-shot learning
Performing a task with no task-specific training examples, for instance classifying images into categories described only in words. Large pretrained models make this possible.