Embedding Explorer

About

What you are looking at

Every document in the corpus becomes a list of numbers: a vector. Documents about similar things should end up pointing in similar directions, so “find related text” turns into “find the nearest vectors”. That is the idea behind semantic search, recommendation and the retrieval step of retrieval-augmented generation.

The map is a 2D picture of a space with tens or hundreds of dimensions. The results list is computed in the full space, which is why a neighbour can look far away on the map and still rank first.

What is real here, and what is not

Every vector on this page is computed in your browser from the text you see, with classical methods: TF-IDF weighting and latent semantic analysis (a truncated singular value decomposition). Nothing is precomputed or simulated.

Production systems use neural embedding models trained on hundreds of millions of text pairs. They are far better at paraphrase and at words they have never seen in your corpus, but the geometry is the same: unit vectors, cosine similarity, nearest neighbours. The Embeddings and Retrieval lesson covers the difference in depth.

How it works

1. TF-IDF: one dimension per word

Text is lowercased, split into words, common function words (the, of, and) are dropped, and a light stemmer folds “puppies” into “puppy”. Each remaining word gets its own axis. A document’s value on that axis is

where tf is how often the word appears in the document, N the number of documents and df how many contain the word. Rare words weigh more. Each vector is then scaled to length 1.

2. LSA: compress with the SVD

Stack the TF-IDF vectors into a matrix X (documents × words) and factor it as X = UΣV^T. Keeping only the k largest singular values gives the best rank-k approximation of X in the least-squares sense. Words that tend to occur in the same documents load on the same singular directions, so “physician” and “doctor” can end up close even though they are different axes in TF-IDF.

A new query is “folded in” by projecting its TF-IDF vector onto the same k directions. Here the SVD is computed exactly, by an eigen-decomposition of the small document-by-document matrix XX^T.

3. Cosine similarity and search

Similarity is the cosine of the angle between two vectors: their dot product divided by the product of their lengths. Because every vector is unit length, it is just the dot product. The search is exact: the query is compared with every document and the top k are kept.

4. Projection to 2D

PCA finds the two directions of greatest variance and drops a perpendicular shadow onto them, so a new query can be placed exactly. t-SNE instead moves points around until each one’s nearest neighbours in 2D match its nearest neighbours in the full space. It runs 750 gradient steps here, with a four-times “early exaggeration” for the first 100 that lets clusters form. It has no formula for new points, so the query is drawn at the weighted mean of its neighbours.

Neural embeddings

Classical embeddings count words. Neural embedding models read them. The table compares what you are running here with what a production retrieval system typically uses in 2026.

Modern models are usually transformers trained with a contrastive objective: pull a question and the passage that answers it together, push unrelated passages apart. Try a query whose words are not in the corpus (for example “footballer injury” on the everyday corpus) and you will see the classical methods give up entirely. A neural model would still place it near the football and health documents.

Measuring retrieval

Precision@k and recall@k

For a query with R relevant documents, suppose h of the top k results are relevant. Precision@k = h / k asks how clean the result list is. Recall@k = h / R asks how much of what exists you found. With 16 documents per topic and k = 5, a perfect system scores precision 1.0 but recall only 5 / 15 = 0.33.

Mean reciprocal rank

MRR looks only at the first relevant result. If it is at rank 1 the query scores 1, at rank 2 it scores 0.5, at rank 4 it scores 0.25. Averaging over queries rewards systems that put something useful at the top, which is what matters when an LLM reads only the first few passages.

The caveat

Topic labels are a coarse stand-in for relevance. A cooking sentence about “bread” is only loosely relevant to one about “steak”. Real evaluations use human or LLM-judged relevance for each query, as in the BEIR and MTEB benchmarks.

Experiments

1. Synonyms. On the everyday corpus search “physician” with TF-IDF, then with LSA. TF-IDF finds the single sentence containing the word and scores everything else 0. LSA also returns sentences about doctors and surgeons that never use it.
2. The k trade-off. Drag k from 2 to the maximum and watch the quality curve. At k = 2 the topics collapse onto each other. Near the maximum LSA becomes TF-IDF again. The best value sits in between.
3. Two meanings. Switch to “One word, two meanings” and search “bank”. Results mix rivers and money because a single word carries no context. Now search “river bank fishing”.
4. Projection illusions. Switch to t-SNE, re-run it a few times and change perplexity. The clusters move and change size, while the result list does not change at all. The picture is a view; the list is the truth.
5. Out of vocabulary. Search “healthy eating”. Neither word appears in the corpus, so every method returns nothing. This is the gap neural models, trained on far more text and reading subword pieces, were built to close.
6. Your own text. Pick “Paste your own” and add a few labelled lines from two topics. How many documents do you need before LSA starts to help?