Know Your Embedding Model

Every threshold in Pelorus — query matching, chunk fallback, cluster proximity — is expressed in the units of one specific embedding model. This page explains how modern embedding models actually behave, so you can tune those settings deliberately, know what to expect from the default model, and know exactly what to re-check if you integrate a different one.

What a similarity score really is

Pelorus compares vectors with cosine similarity, and the scores it exposes are the raw values: mathematically a range of -1 to 1, where 1 is an identical direction, 0 is orthogonal, and -1 is opposite. That is the geometry. It is not how scores behave in practice.

Modern embedding models do not spread text across the whole vector space. Trained embeddings share a large common component — a direction present in nearly every vector — so every pairwise comparison starts from a positive floor before content differences enter at all. This property (often called anisotropy) has practical consequences:

The takeaway: absolute score values only mean something relative to the score distribution of your own model on your own data. Rank order is reliable at coarse separations; raw magnitudes are model-specific, and a very small gap between two scores is not a ranking signal at all (see below).

What the score means — and what it doesn't

A score is the embedding model's geometric opinion about two pieces of text. It is not a calibrated probability that the match is correct, and it is not a measure of answer quality. Pelorus reads it accordingly, and the way it does is worth knowing before you build anything on the numbers Pelorus returns.

• Scores are served at four decimal places. That is a wire contract, not a precision claim. The same question can be answered by the engine or by the Pelorus client library, and the two runtimes must emit byte-identical output; full-precision floats disagree in their trailing digits, and those digits are float32 and serializer noise rather than information. Vector stores commonly serialize fifteen or sixteen digits of a float32 value — everything past roughly the seventh is an artifact of how the number was written down.
• Scores are compared against lines, not against each other. Every serving decision is a threshold test — the 0.60 chunk-supplement floor, cluster proximity, the 0.98 identity threshold — or a banded tie-break, where differences inside the noise band are treated as ties and broken deterministically. Nothing in serving promotes one result over another because it scored a fraction higher.
• Small differences are not signal. An edit that changes what a question means moves the score in the second or third decimal; float noise lives around the seventh. A gap below 0.001 is under the embedding model's discrimination floor, so ranking on it ranks accidents, not relevance.

Embeddings read topic, not polarity

Embedding models are trained to place text that appears in similar contexts close together. The direction of a vector encodes what a text is about, not what stance it takes. Antonyms embed close together because they share nearly identical contexts, and negation barely moves the score: "customers who churned" and "customers who did not churn" score as near-duplicates.

This shapes which query pairs a similarity score can and cannot tell apart:

This is why Pelorus matches queries to queries and returns Extracts — data scope — rather than caching final answers. At the data layer, "same topic" is usually the right grouping criterion, and topic is exactly what embeddings measure well. The distinctions a similarity score cannot surface are handled by the layers above: cluster curation, trained cluster classifiers, and the consuming LLM reading the real query against the returned Extract.

Why opposites don't score −1

A natural objection: if the model really captured negation, shouldn't "orders that shipped" and "orders that did not ship" point in opposite directions and score near -1? They don't — and anisotropy is only the smaller half of the reason. The deeper point is that opposite meaning is not opposite direction.

Pointing in the opposite direction means disagreeing in every semantic feature at once. A statement and its negation agree on almost everything — topic, entities, domain, register, sentence shape — and differ in exactly one feature: polarity. A faithful geometric encoding of that is what models actually produce: the two texts sit at nearly the same point, displaced by a small offset along a low-magnitude polarity direction. The true antipode of "orders that shipped" would be a text maximally unrelated to it in every respect at once, which barely exists as coherent language.

Notably, the polarity information is usually present in the vectors — probing research shows a trained linear classifier can separate negated from non-negated sentences. What cannot see it is cosine similarity. Cosine reduces two vectors to a single number: a sum of per-dimension agreement, weighted by the vectors' own magnitudes. The shared topical mass dominates that sum, and the small polarity disagreement is outvoted. In toy numbers: shared content contributes +0.97, a genuinely opposed polarity component contributes −0.02, and cosine reports 0.95. The opposition is in there — it nudged the score down — but one scalar cannot distinguish "0.95 because the wording differs" from "0.95 because the meaning is inverted." The signal survives; the attribution is destroyed.

A trained probe recovers the distinction because it gets to weight dimensions — amplify the polarity direction, ignore the topical mass. Cosine applies no weights, so it reports whatever the largest components say, and those encode topic. The same property is what cross-encoder rerankers exploit: by letting two texts attend to each other token-by-token instead of comparing two pre-compressed vectors, they can react to a "not" directly.

The full catalog of look-alike traps

The four examples above are the most common cases of a broader pattern: question pairs whose answers differ but whose vectors barely do. Pelorus maintains an internal taxonomy of these failure modes, each one measured against the default model. Knowing the whole catalog matters for curation: each trap has a different cause, and each is handled in a different place.

Symmetric and asymmetric matching

Retrieval-tuned embedding models distinguish two situations, and many encode the distinction directly in how text is embedded:

• Asymmetric (query → document): a short question must match a long passage that answers it. Models trained for this often apply different treatments to the two sides — prefixes like query: / passage:, or an input-type parameter — effectively projecting queries and documents slightly differently.
• Symmetric (query → query): both sides are the same kind of text, as in paraphrase or semantic-similarity tasks. The correct usage is to embed both sides identically.

Pelorus performs both comparisons with a single embedding of the incoming query. Chunk retrieval is asymmetric: chunks are embedded with document treatment at indexing time. Query matching is symmetric: stored queries and incoming queries receive the same query-side treatment, so query-to-query comparison stays consistent. This is also why query-to-query similarity is usually a cleaner signal than query-to-chunk similarity — it compares like with like.

The default model: bge-small-en-v1.5

The no-key demo and default local configuration use BAAI/bge-small-en-v1.5 via fastembed — a small, fast, well-benchmarked model that runs locally with no provider API key. Useful facts from the model's own guidance:

• Relative order matters more than absolute values. The vendor's explicit guidance is to choose similarity thresholds from the score distribution on your own data.
• The top of the range is compressed. BGE-family scores run high: unrelated text can still score above 0.6, and the vendor suggests similarity-filter thresholds like 0.8, 0.85, or even 0.9. Expect the meaningful action to happen roughly between 0.6 and 1.0.
• Query instruction is optional for v1.5. The v1.5 release was tuned so retrieval works well without the query instruction; the instruction mainly helps short-query-to-long-passage retrieval.

This is the context behind the default retrieval settings: a high query threshold in a compressed score range, and two lower chunk thresholds for the asymmetric comparison — a wider floor for a plain chunk-fallback hit, and a narrower one for a chunk riding along with a matched Extract.

retrieval_config:
  query_extract_threshold: 0.98    # symmetric query-to-query, compressed range
  chunks_threshold: 0.45           # asymmetric query-to-chunk fallback
  chunk_supplement_threshold: 0.60 # min similarity for a chunk to ride along an extract hit

With score bands this narrow, close calls — such as whether a strong chunk should surface alongside a matched Extract — can hinge on differences of a few hundredths. Treat small margins as meaningful and tune them against observed results, not intuition about what "0.98" should mean.

Tuning from your own distribution

The reliable way to set thresholds is empirical: look at where your traffic actually lands. The read-only analysis surface exposes everything needed to do this in a notebook — query_vectors(), chunk_vectors(), similarities(), and project_query() — and the Query Map lets you drop an ad hoc query onto the layout to see where it lands relative to queries and clusters.

• Histogram the scores. Run a representative query set with thresholds relaxed and plot query-to-query and query-to-chunk scores separately. Identify the floor (unrelated), the paraphrase band, and the near-duplicate band for each.
• Set thresholds between bands, not above them. The query threshold should admit the paraphrase band — that is where the valuable "same need, different words" matches live.
• Watch the misses. Queries that fall back to chunks and queue new Extract demand just below a cluster boundary are a signal the boundary or the Extract needs attention, not just the threshold.
• Re-check after change. Re-ingesting the corpus with different chunking, upgrading the model, or shifting traffic all move the distribution. Thresholds are tied to it.

Distance measures and vector normalization

A similarity score is produced by three choices, not one: the embedding model, the distance measure the vector store compares with, and whether the stored vectors are L2-normalized (scaled to unit length). Most vector stores let every collection or index choose its own measure, and the choice varies from store to store — so the same corpus, embedded by the same model, can produce incomparable scores in two different indexes. Every threshold on this page assumes one specific configuration.

(Binary/sparse measures — Hamming, Jaccard — apply to quantized or keyword vectors, not the dense embeddings Pelorus serves on, and are out of scope here.)

Why normalization is the hinge. When every vector is unit length, all information lives in direction: cosine and dot product become the same number, and Euclidean distance becomes a monotone transform of it — one calibrated space, three equivalent readouts. Unnormalized vectors break the equivalence: magnitude leaks into dot-product scores (biasing toward longer or more frequent text, and amplifying the hub-vector effect above), and a threshold that worked on one corpus silently means something else on the next.

How Pelorus validates this. The RAG Connector contract makes these properties queriable instead of assumed: a connector reports its embedding space's measure and normalization, and can enumerate its backend's collections/indexes — each with its own measure, dimension, normalization, and chunking, read from the live backend rather than cached configuration (the built-in connector does exactly this against its configured collection). A connector may also honestly report a property as unknown: unreported values are treated as "assumed, unvalidated" rather than silently trusted, and any property that is asserted and contradicts the calibrated configuration (a non-cosine measure, unnormalized vectors) is flagged before serving thresholds are applied to it.

What this means for Extracts. Everything measured on this page — the score bands, the trap catalog, the identity and supplement thresholds — was measured in cosine units over normalized vectors. Changing the distance measure or normalization is the same class of event as changing the embedding model: the score distribution, the geometry profile, and every threshold must be re-derived in the new units before Extract matching can be trusted.

Integrating another embedding model

The embedding model is a pluggable choice, pinned per Dataset Group so stored vectors and query embeddings share one space. Swapping it is supported by the architecture — the vector store is a rebuildable projection of the system of record — but nothing tuned against the old model survives the change. Before adopting a different model, establish its technical details: