What is Embeddings?

How it works

An embedding model takes an input, such as a sentence, a paragraph or an image, and outputs a fixed-length vector, often a few hundred to a few thousand numbers. The model is trained so that inputs with similar meaning produce vectors that point in similar directions. "How do I reset my password?" and "I forgot my login" end up close together even though they share almost no words.

Closeness is measured with a simple calculation, usually cosine similarity or dot product. Word2vec (2013) made the idea famous for single words; modern embedding models are transformers that embed whole passages, and some are multimodal, placing text and images in the same space.

Inside every LLM there are embeddings too: each token is turned into a vector at the first layer. But when people say "embeddings" in an application, they usually mean the output of a dedicated embedding model used for search.

Why it matters

• Semantic search. Embeddings let you find relevant content by meaning, which keyword search misses.
• They power RAG. In retrieval-augmented generation, documents are embedded and stored in a vector database; the user's question is embedded and the nearest chunks are fetched for the model to read.
• Other uses. Classification, clustering, de-duplication and recommendations all work well on embeddings.
• Choice matters. Different embedding models give different quality, languages and vector sizes, and vectors from one model cannot be compared with vectors from another. Changing models means re-embedding everything.

Example

A support team embeds 5,000 help articles once. When a customer types "app keeps logging me out", the system embeds that query, finds the five closest articles, including one titled "Session expires too quickly", and passes them to a chat model to write the reply.