Overview
Level 2: Build · Module 5: Embeddings and Vector Search · 20 min read
Why this matters
When querying a vector database like pgvector, Pinecone, or Qdrant, you are required to specify an index distance metric (e.g., <=> for cosine distance, <#> for negative dot product, or <-> for L2 Euclidean distance). Choosing the wrong metric leads to slow index performance, incorrect search results, or incompatible query rankings. Furthermore, understanding the mathematical relationship between normalized unit vectors, dot products, and cosine similarity allows you to optimize query execution by orders of magnitude.
Learning objectives
- Derive and compare the mathematical formulas for Cosine Similarity, Dot Product, and Euclidean Distance.
- Understand vector normalization ($||A|| = 1$) and why it makes Dot Product identical to Cosine Similarity.
- Compare hardware execution speeds across SIMD/AVX and GPU tensor cores.
- Select the optimal distance metric for pgvector and cloud vector databases.
- Answer interview questions on distance metric tradeoffs and normalization tricks.
The Three Foundational Metrics Compared
graph TD
subgraph Metrics["Vector Distance Metrics"]
Cos["Cosine Similarity<br>cos(θ) = (A · B) / (||A|| * ||B||)<br>Measures angle, ignores magnitude"]
Dot["Dot Product (Inner Product)<br>A · B = Σ (Ai * Bi)<br>Fastest compute; scales with length"]
Euc["Euclidean Distance (L2)<br>d = √Σ (Ai - Bi)²<br>Straight-line spatial distance"]
end| Metric | Mathematical Formula | Range | Ignores Magnitude? | Hardware Compute Cost |
|---|---|---|---|---|
| Cosine Similarity | $\frac{A \cdot B}{|A| |B|}$ | $[-1.0, +1.0]$ | Yes (pure angle) | Medium (requires square roots for norms) |
| Dot Product | $\sum_{i=1}^{n} A_i B_i$ | $(-\infty, +\infty)$ | No (longer vectors score higher) | Fastest (pure multiply-accumulate FMA) |
| Euclidean (L2) | $\sqrt{\sum_{i=1}^{n} (A_i - B_i)^2}$ | $[0, +\infty)$ | No (shorter distance = closer) | Medium-High (subtractions, squares, root) |
The Unit Normalization Breakthrough
Why do production systems almost universally normalize their vectors?
A vector $A$ is unit-normalized (or has $L_2$ norm of 1) when its magnitude equals 1: $$|A| = \sqrt{\sum A_i^2} = 1.0$$
When both vectors $A$ and $B$ are normalized to unit length ($|A| = 1$ and $|B| = 1$): $$\text{Cosine}(A, B) = \frac{A \cdot B}{1.0 \times 1.0} = A \cdot B$$
[!TIP] The Key Optimization: If you normalize embeddings upon generation (which OpenAI
text-embedding-3-*and Cohere do by default), Cosine Similarity is mathematically identical to the Dot Product!Because the Dot Product requires only multiply-add operations without expensive square roots or divisions, database query engines can execute SIMD/AVX hardware-accelerated dot products up to 3x to 5x faster than non-normalized cosine computations.
Concrete TypeScript Implementation
Here is how each metric is calculated in TypeScript:
// lib/ai/vector-math.ts
/**
* Calculates dot product of two equal-length vectors.
* A · B = sum(Ai * Bi)
*/
export function dotProduct(a: number[], b: number[]): number {
if (a.length !== b.length) throw new Error("Vector dimension mismatch");
let sum = 0;
for (let i = 0; i < a.length; i++) {
sum += a[i] * b[i];
}
return sum;
}
/**
* Calculates the Euclidean (L2) magnitude of a vector.
* ||A|| = sqrt(sum(Ai^2))
*/
export function magnitude(v: number[]): number {
let sum = 0;
for (let i = 0; i < v.length; i++) {
sum += v[i] * v[i];
}
return Math.sqrt(sum);
}
/**
* Calculates Cosine Similarity between two vectors.
* Range: -1.0 (opposite) to +1.0 (identical)
*/
export function cosineSimilarity(a: number[], b: number[]): number {
const dot = dotProduct(a, b);
const magA = magnitude(a);
const magB = magnitude(b);
if (magA === 0 || magB === 0) return 0;
return dot / (magA * magB);
}
/**
* Calculates Euclidean Distance (L2).
* Range: 0 (identical) to infinity
*/
export function euclideanDistance(a: number[], b: number[]): number {
if (a.length !== b.length) throw new Error("Vector dimension mismatch");
let sum = 0;
for (let i = 0; i < a.length; i++) {
const diff = a[i] - b[i];
sum += diff * diff;
}
return Math.sqrt(sum);
}Postgres & pgvector Syntax Reference
In PostgreSQL with pgvector, query operators differ by distance metric:
-- 1. Cosine Distance Operator (<=>)
-- Returns 1 - cosine_similarity (0 = identical, 2 = opposite)
SELECT id, content, 1 - (embedding <=> $1) AS similarity
FROM document_chunks
ORDER BY embedding <=> $1
LIMIT 5;
-- 2. Negative Dot Product Operator (<#>)
-- Best for normalized vectors (fastest query execution)
SELECT id, content, (embedding <#> $1) * -1 AS similarity
FROM document_chunks
ORDER BY embedding <#> $1
LIMIT 5;
-- 3. Euclidean Distance Operator (<->)
-- Returns L2 straight-line distance (0 = identical)
SELECT id, content, embedding <-> $1 AS distance
FROM document_chunks
ORDER BY embedding <-> $1 ASC
LIMIT 5;Interview Angle: How to Answer in System Design
- Question: "Why do we prefer Cosine Similarity over Euclidean distance when ranking text documents of different lengths?"
- Target Answer:
"Cosine similarity evaluates the angle between vectors while completely discounting magnitude. In text processing, a 50-word summary and a 500-word comprehensive guide on the exact same subject may share identical semantic directions, but the longer document's vector might possess greater raw magnitude if unnormalized.
Euclidean distance measures the absolute spatial distance between points, so it would incorrectly penalize the longer text simply due to document volume.
Furthermore, by pre-normalizing all vectors to unit length ($|V|=1$), Cosine Similarity collapses directly into the Dot Product ($A \cdot B$), allowing the vector database to compute similarities using SIMD matrix multiplication without computing costly square roots."
Quick Check
- Under what condition is Cosine Similarity mathematically identical to Dot Product?
- Which distance metric is sensitive to differences in vector magnitude?
- What is the operator for cosine distance in PostgreSQL
pgvector?