AI Embeddings

AI embeddings are numerical vector representations of data that capture useful semantic relationships. Text, images, products, users, documents, source code, and other objects can be converted into embeddings so that software can compare them mathematically rather than relying only on exact keywords or identifiers.

Embeddings are a fundamental building block behind semantic search, recommendation systems, clustering, duplicate detection, retrieval-augmented generation, and many modern AI applications. For software engineers, the key idea is that similar meaning can be represented as nearby points in a multidimensional vector space.

Table of Contents

• What Is an Embedding?
• How Embeddings Represent Meaning
  • Embedding Dimensions
  • Semantic Similarity
• How Embeddings Are Created
• Measuring Vector Similarity
  • Cosine Similarity
  • Dot Product and Euclidean Distance
• Semantic Search with Embeddings
• Embeddings in RAG
• Chunking and Embedding Quality
• Storing and Searching Embeddings
  • Exact and Approximate Search
  • Metadata Filtering
• Production Considerations
• Conclusion
  • More Articles to Read

What Is an Embedding?

An embedding is an array of numbers representing some input in a mathematical space. An embedding model converts the original input into this vector.

For example, the text:

PostgreSQL replication is delayed.

might be converted into a vector conceptually similar to:

[0.18, -0.42, 0.07, 0.91, -0.13, ...]

Real embeddings commonly contain hundreds or thousands of dimensions. Individual values usually do not have simple meanings such as database, replication, or delay. Meaning is distributed across the vector.

The vector itself becomes useful when compared with other vectors generated by the same compatible embedding model.

Consider these sentences:

PostgreSQL replication is delayed.

The database replica is falling behind the primary.

The sentences share relatively few exact words, but they describe similar situations. A useful embedding model should produce vectors that are relatively close in its embedding space.

Compare them with:

The customer requested a refund for the annual subscription.

That sentence represents a different concept and should normally appear farther away.

This property allows applications to search by semantic similarity rather than exact text matching.

How Embeddings Represent Meaning

An embedding model learns representations during training. Inputs with related characteristics tend to acquire representations that allow downstream mathematical comparisons to identify useful relationships.

This does not mean an embedding is a perfect representation of meaning. It is a compressed representation optimized according to the model's training objective and data.

Two pieces of text can therefore be close for one task but still differ in ways important to the application. Embedding similarity should be treated as a useful signal, not as proof that two inputs are equivalent.

Embedding Dimensions

An embedding can be represented as a vector:

v = [v1, v2, v3, ..., vd]

where d is the embedding dimension.

A three-dimensional vector can be visualized geometrically, but production embeddings often have hundreds or thousands of dimensions. The same mathematical ideas still apply even though the space cannot be directly visualized.

Higher dimensionality can provide more representational capacity, but it also increases storage, memory bandwidth, index size, and similarity-search cost.

Suppose an embedding contains 1,536 32-bit floating-point values. Ignoring index and metadata overhead, one vector requires approximately:

1,536 × 4 bytes = 6,144 bytes ≈ 6 KB

Ten million such vectors would require roughly 60 GB for raw vector values alone.

This is why embedding dimension becomes an infrastructure concern at scale rather than merely a model characteristic.

Semantic Similarity

The central property of embeddings is that geometric relationships can correspond to useful semantic relationships.

Suppose an application embeds three support requests:

A: "My payment was charged twice."
B: "I see a duplicate charge on my card."
C: "The API returns 503 errors."

Vectors A and B should generally be closer to each other than either is to C.

An application can exploit this property without understanding what individual vector dimensions mean. It only needs a consistent embedding model and an appropriate similarity function.

Embeddings can represent more than sentences. Depending on the model, they can represent paragraphs, documents, images, audio, source code, products, or other objects.

Multimodal embedding models can even place different data types into compatible spaces, allowing tasks such as finding images using natural-language descriptions.

How Embeddings Are Created

An embedding model is usually a neural network trained so that its internal representations become useful for comparing inputs.

At application time, creating an embedding is typically straightforward:

def embed_document(text: str) -> list[float]:
    response = embedding_model.embed(text)

    return response.vector

The difficult work happened during model training. The application receives the resulting representation through a local model or external inference service.

Text embedding typically begins with tokenization. The model processes token representations through neural-network layers and produces a fixed-size vector representing the supplied text.

The resulting embedding differs from the contextual token representations used internally by a language model. Application-facing embedding models are designed to produce vectors suitable for operations such as retrieval, clustering, or classification.

Tokenization and model context limits still matter. Very large documents may exceed the embedding model's accepted input length or produce overly broad representations. Documents are therefore commonly split into smaller chunks before embedding.

AI Tokens and Context Windows explains tokenization and context limits in more detail.

Measuring Vector Similarity

Once data has been converted into vectors, the application needs a mathematical function to determine which vectors are most similar to a query vector.

Common choices include cosine similarity, dot product, and Euclidean distance. The correct metric depends on how the embedding model was trained and how vectors are normalized.

Cosine Similarity

Cosine similarity measures the angle between two vectors rather than their absolute magnitude. Vectors pointing in similar directions receive higher similarity scores.

It can be calculated as:

import math


def cosine_similarity(
    left: list[float],
    right: list[float],
) -> float:
    dot_product = sum(a * b for a, b in zip(left, right))

    left_norm = math.sqrt(sum(a * a for a in left))
    right_norm = math.sqrt(sum(b * b for b in right))

    if left_norm == 0 or right_norm == 0:
        raise ValueError("Cannot compare a zero vector")

    return dot_product / (left_norm * right_norm)

If two non-zero vectors point in the same direction, their cosine similarity approaches 1. Lower values indicate less similar directions.

Cosine similarity is widely used for semantic search because it focuses on vector orientation rather than magnitude.

Dot Product and Euclidean Distance

The dot product multiplies corresponding dimensions and sums them:

def dot_product(
    left: list[float],
    right: list[float],
) -> float:
    return sum(a * b for a, b in zip(left, right))

For normalized vectors, dot product and cosine similarity can produce equivalent rankings because every vector has the same magnitude.

Euclidean distance instead measures straight-line distance between vectors:

import math


def euclidean_distance(
    left: list[float],
    right: list[float],
) -> float:
    return math.sqrt(
        sum((a - b) ** 2 for a, b in zip(left, right))
    )

Unlike similarity scores where larger may mean closer, smaller Euclidean distance means vectors are nearer.

Similarity metrics should not be chosen arbitrarily. Use the metric recommended for the embedding model and ensure that the database index uses compatible semantics.

Semantic Search with Embeddings

Traditional text search often relies on terms appearing in both the query and documents. This works extremely well for exact names, identifiers, error codes, and distinctive keywords.

Semantic search solves a different problem: finding content with similar meaning even when vocabulary differs.

Suppose a documentation system contains:

Replica lag occurs when a standby database cannot replay WAL records as quickly as the primary generates them.

A user searches:

Why is my PostgreSQL secondary falling behind?

Exact keyword matching may struggle because secondary and falling behind differ from standby and replica lag. Embeddings can place the query and document near each other based on their semantic relationship.

A basic semantic-search pipeline works as follows:

Query → Embedding → Vector Search → Similar Documents

The query is converted into an embedding using the same embedding space used for indexed documents. The vector index then finds nearby document vectors.

A simplified implementation might look like:

def semantic_search(query: str, limit: int = 5):
    query_vector = embedding_model.embed(query).vector

    return vector_store.search(
        vector=query_vector,
        limit=limit,
    )

This approach can retrieve conceptually related content without requiring exact vocabulary overlap.

However, semantic search is not automatically superior to lexical search. A query containing an exact error code such as ORA-01555, a SKU, UUID, or function name may be better served by keyword matching.

Production search systems therefore often combine lexical and vector retrieval. Semantic similarity and exact matching solve complementary problems.

Embeddings in RAG

Embeddings are commonly used as the retrieval layer in Retrieval-Augmented Generation (RAG).

Documents are divided into chunks, each chunk is embedded, and the vectors are stored in a searchable index. When a question arrives, the application embeds the question and retrieves nearby chunks.

The retrieved text, not the vectors themselves, is normally placed into the language model's context.

Question → Query Embedding → Vector Search → Relevant Text → LLM

This distinction is important. The embedding model identifies potentially relevant information; the language model interprets that information and generates the answer.

Suppose an organization stores thousands of operational documents. A user asks:

What happens to a shipment after three failed delivery attempts?

The query embedding may retrieve chunks from the delivery-attempt policy even if the exact wording differs. Those chunks can then be provided to the LLM as evidence.

Retrieval quality becomes an upper bound on answer quality. If the relevant policy never reaches the LLM, prompt engineering cannot reliably recover the missing information.

RAG (Retrieval-Augmented Generation) covers retrieval, ranking, grounding, generation, and evaluation as a complete architecture.

Chunking and Embedding Quality

Embedding an entire large document into one vector can lose useful local distinctions. A 50-page technical manual may discuss authentication, databases, networking, deployment, and troubleshooting. One vector must compress all of those topics into a single representation.

Instead, retrieval systems commonly divide documents into chunks.

Chunk size introduces a trade-off.

Splitting every document after an arbitrary number of characters is simple but can cut through paragraphs, code blocks, tables, or sections.

Structure-aware chunking can preserve headings, paragraphs, functions, or other meaningful boundaries. A source-code search system, for example, may obtain better results by embedding complete functions or classes rather than arbitrary 1,000-character slices.

Overlap is sometimes added between neighboring chunks to preserve information crossing boundaries. Excessive overlap, however, increases vector count, storage, indexing work, and duplicate search results.

Chunking should therefore be evaluated using actual retrieval tasks. There is no universal chunk size that works best for every dataset.

Storing and Searching Embeddings

Once an application contains more than a small number of embeddings, comparing a query vector against every stored vector can become expensive.

A vector database or vector-capable search engine stores embeddings and provides indexes designed for nearest-neighbor search.

A stored record usually contains more than the vector:

{
  "id": "doc-1842-chunk-7",
  "document_id": "doc-1842",
  "text": "Replica lag occurs when...",
  "embedding": [0.18, -0.42, 0.07],
  "tenant_id": "tenant-91",
  "language": "en",
  "updated_at": "2026-08-20"
}

The vector supports semantic search, while metadata supports filtering, authorization, freshness, and reconstruction of the original content.

Vector Databases for AI covers vector indexing and database architecture in depth.

Exact and Approximate Search

For N vectors, exact search can calculate similarity between the query and every vector. This produces exact nearest neighbors but becomes expensive as the dataset grows.

Large systems often use Approximate Nearest Neighbor (ANN) indexes. These data structures reduce the number of vectors that must be examined directly.

The trade-off is recall: the index may occasionally miss a true nearest neighbor in exchange for substantially lower latency and higher throughput.

The correct index configuration depends on dataset size, vector dimension, latency requirements, memory budget, and acceptable retrieval recall.

Metadata Filtering

Semantic similarity alone is rarely sufficient for production retrieval.

Suppose a multi-tenant application searches customer documents. A vector from another customer's document might be extremely similar to the query, but it must never be returned.

Retrieval should enforce metadata constraints such as:

results = vector_store.search(
    vector=query_vector,
    filters={
        "tenant_id": authenticated_tenant_id,
        "language": "en",
        "status": "published",
    },
    limit=10,
)

Filters can also restrict results by document type, product, date, region, access level, or other application properties.

Vector similarity is a ranking signal, not an authorization mechanism. Access control must be enforced by deterministic application or database rules.

Filtering strategy also affects index performance. Some vector engines apply filters before vector search, others during traversal, and others after candidate retrieval. Highly selective filters can behave differently depending on the index architecture.

Production Considerations

Embedding systems introduce their own data lifecycle. Changing the embedding model is not equivalent to changing a stateless API dependency because stored vectors were produced by a specific model and representation space.

Vectors created by unrelated embedding models generally should not be compared directly. If an application switches models, existing content may need to be re-embedded.

A production record should therefore track an embedding version or model identifier:

{
  "chunk_id": "doc-1842-chunk-7",
  "embedding_model": "embedding-model-v3",
  "embedding_version": 3
}

A migration can build a new index in parallel, re-embed content gradually, evaluate retrieval quality, switch traffic, and retire the previous index after validation. Replacing vectors in place without a migration strategy can produce inconsistent retrieval while old and new representations coexist.

Updates to source documents also need propagation. If a policy changes but its stored embedding remains based on the old text, semantic search can continue retrieving outdated content.

Useful production metrics include:

• Embedding latency. Track p50, p95, and p99 time required to generate vectors.
• Embedding throughput. Measure documents or tokens embedded per second during ingestion.
• Vector-search latency. Track retrieval performance independently from LLM generation.
• Retrieval recall. Measure whether expected relevant documents appear among retrieved candidates.
• Index size. Monitor vector count, memory, and storage growth.
• Stale embedding count. Detect source content that changed without successful re-embedding.
• Embedding failures. Detect malformed content, model errors, and ingestion backlog.
• Cost per embedded document. Track model and infrastructure cost as the corpus grows.

Embedding generation can often be asynchronous. When millions of documents need indexing, batching improves throughput and reduces unnecessary pressure on the embedding service.

Interactive query embeddings have different requirements. They sit directly on the search request path, so embedding latency contributes to end-to-end search latency. Caching repeated queries can help when query repetition is high.

Privacy also matters. Embeddings are numerical representations, but they should not automatically be considered anonymous or harmless. Sensitive source data should remain subject to appropriate access controls, retention policies, encryption, and data-handling requirements throughout the embedding pipeline.

Finally, retrieval should be evaluated end to end. A vector index can achieve low latency while returning poor documents, and an LLM cannot reliably compensate for missing evidence. The useful metric is not how quickly vectors are compared but whether the retrieval system consistently delivers the information required by the downstream task.

Conclusion

AI embeddings convert text and other data into numerical vectors whose geometric relationships can represent useful semantic similarity. This makes it possible to search, rank, cluster, recommend, and retrieve information without depending entirely on exact keyword matches.

Production embedding systems require more than calling an embedding model. Similarity metrics, chunking, vector dimensions, indexing, metadata filters, model versioning, re-embedding, access control, and retrieval evaluation all affect system quality and cost.

An embedding is not an answer and similarity is not truth. Embeddings provide a powerful retrieval and ranking signal; application logic must still enforce authorization, freshness, business constraints, and the meaning of the retrieved results.

More Articles to Read

• Large Language Models (LLMs)
• AI Tokens and Context Windows
• Vector Databases for AI
• RAG (Retrieval-Augmented Generation)