Research-backed calculator guide
RAG Re-Embedding Trigger Policy: Set Thresholds Without Reprocessing Everything
A practical RAG re-embedding trigger policy for deciding when content changes, model changes, quality drift, or scheduled checks justify refresh work, with cost math and rollback controls.
Quick answer
A RAG re-embedding trigger policy should not be one timer that rebuilds every vector. Treat embeddings as versioned derived data and trigger the smallest safe action for the event: update only changed text chunks, skip metadata-only changes, queue a broader refresh when retrieval quality falls, and rebuild the full namespace when the embedding model or preprocessing contract changes without documented compatibility. Set thresholds against business freshness, retrieval quality, and the cost of stale answers—not against an arbitrary percentage alone.
For most systems, the policy has four lanes:
- Content trigger: re-embed a chunk when its embedded text or its chunking inputs change. A metadata-only change can usually update metadata without changing the vector.
- Model or pipeline trigger: create a new index or namespace and re-embed the affected corpus when the model, dimensions, task format, normalization, or chunking contract changes.
- Quality trigger: sample known queries and re-embed documents linked to falling precision, recall, nDCG, or confidence, then investigate before expanding the scope.
- Reconciliation trigger: run a periodic audit for missed events, stale revisions, deletes, and queue failures. This is a safety net, not permission to reprocess everything by default.
The practical rule is: if the searchable text did not change, do not pay for a new vector merely because a record changed; if the vector space contract changed, do not mix old and new vectors in one search path. Provider APIs expose model-specific embedding outputs and usage fields, so store the model identifier, input representation, source revision, and token count with each build [1].
What should cause a re-embedding?
A useful policy starts by separating changes that affect meaning from changes that affect only filtering or presentation. An embedding represents an input under a particular model and task format. Google’s embedding documentation distinguishes retrieval queries from retrieval documents and says the task should be used consistently; its examples also show that task-specific formatting is part of the retrieval contract [2]. That means a change to a title prefix, task instruction, chunking rule, or normalization step can be as important as a model-name change.
1. Embedded-text changes
This is the ordinary incremental path. If a source revision changes the words that are sent to the embedding model, hash the canonical embedded text and enqueue the affected chunk. Use a stable document ID plus a chunk ID and source revision. The worker should upsert the new vector idempotently, then retire chunks that no longer exist after a split or merge.
Do not use a raw “document updated” timestamp as the sole trigger. A permissions flag, owner field, or display label may change while the embedded text remains identical. If that field is used in a vector-database filter, update the metadata or filter index instead. Pinecone’s documentation distinguishes metadata updates from vector updates and says that replacing whole documents or records uses upsert [3]. Its API reference states that upserting an existing vector ID overwrites the previous value [4]. The implementation details differ across databases, but the policy distinction is portable: change the vector only when the vector input changed.
A diff threshold can reduce churn, but it should be a guardrail rather than a truth machine. For example, a one-word change in a legal prohibition may be more important than a large formatting rewrite. Use field-aware rules first: legal terms, product limits, prices, safety statements, and access instructions can be “always re-embed” fields. Only then consider a normalized text-diff ratio for low-risk narrative fields.
2. Model, dimension, or preprocessing changes
A model migration is not an ordinary document update. The OpenAI API returns the model used and model-dependent embedding vectors in its response, while Google documents model-specific output dimensions and task settings [1] [2]. Unless a provider explicitly documents compatibility, assume that changing the model, task format, vector dimension, distance or normalization assumptions, or chunking contract creates a new retrieval namespace.
Build the target index separately. Backfill from authoritative source text, not from old vectors. During the migration, dual-write new revisions if the old index still serves traffic. Run shadow retrieval with a query vector generated by the target model against the target index. Compare application-level relevance and known-answer coverage rather than treating raw similarity scores from different spaces as directly comparable.
This is the trigger most likely to justify a full-corpus job, but “full” still needs a scope. If archived records are out of scope for current search, document that exclusion and keep their migration status visible. If only one tenant or collection changes its embedding contract, migrate that namespace rather than every tenant. Retain the old compatible model-index pair until rollback is no longer required.
3. Quality or drift signals
A scheduled quality test can expose a missed content event or a model/data drift problem. Maintain a small, versioned evaluation set containing representative queries, expected documents, and critical failure cases. Alert on a sustained change in metrics, such as precision@k, recall@k, nDCG@k, answer-supported-by-source rate, or the share of low-confidence retrievals.
Avoid setting a trigger on one bad answer. A single miss can be labeling noise, a query rewrite issue, or a source problem. A stronger rule is “two consecutive evaluation windows below the floor,” with a minimum sample count and a slice by corpus class. If only product manuals degrade, refresh or investigate that class. If every slice degrades after a provider change, open a model-compatibility incident instead of launching a blind incremental job.
Quality-triggered re-embedding is also a useful escape hatch for changes that look small in text but matter semantically. Keep a reason code such as content_change, critical_field_change, quality_reembed, model_migration, or reconciliation_repair. Without reason codes, it is difficult to tell whether a lower bill reflects a better filter or an unhealthy backlog.
4. Reconciliation and deletion triggers
Event-driven ingestion is fast but can lose messages, fail after a partial write, or process revisions out of order. A daily or weekly reconciliation job should compare source revisions with vector lineage, find missing chunks, verify that deleted records are absent, and check that the latest revision won the race.
Reconciliation should not blindly re-embed every row. It can first compare source hashes, revision numbers, model IDs, and expected chunk manifests. Only mismatches enter the repair queue. Keep deletes as durable tombstones or another replay-safe record so an old retry cannot resurrect content that a newer source revision removed.
A policy template you can tune
Write the policy in terms an on-call engineer can execute:
| Trigger | Detection rule | Action | Freshness target | Rollback or evidence |
|---|---|---|---|---|
| Embedded text changed | Canonical embedded-text hash or source revision differs | Re-embed affected chunks; upsert by stable ID; retire removed chunks | For critical policy text, for example, 15 minutes; choose your own SLO | Record old and new revision, job ID, and write result |
| Metadata-only change | Embedded-text hash unchanged; filter/display fields changed | Update metadata or source record without generating a vector | Same as the application’s filter SLO | Verify filter result and vector hash stayed unchanged |
| Critical field changed | A protected field such as price, dosage, prohibition, or entitlement changed | Re-embed immediately, even if the overall diff is small | Near-real-time or an explicitly approved window | Require field-level audit and a retrieval smoke test |
| Model or pipeline contract changed | Model ID, task format, dimensions, chunker, normalization, or distance contract changes | Build a separate target index, backfill, shadow-read, then cut over the pair | Migration deadline based on corpus and traffic risk | Keep the old pair serving-ready until acceptance |
| Quality degradation | Evaluation slice misses its floor in two windows with adequate sample size | Investigate source lineage; re-embed affected slice or start migration review | Restore the quality SLO, not merely queue depth | Preserve evaluation results before and after repair |
| Reconciliation mismatch | Source revision, hash, delete, or vector lineage does not match | Repair only the mismatched records; escalate repeated failures | Finish before the maximum tolerated stale age | Produce a mismatch count and final zero-or-explained remainder |
The numbers in the freshness column are examples of policy shape, not universal defaults. Choose a target per document class. A frequently changing pricing page and a static historical report should not share the same refresh interval just because they live in one index.
Worked cost example
Suppose a knowledge base contains 12,000,000 billable input tokens. An ordinary source update changes 8% of the canonical embedded text. You are testing a provider rate of $0.10 per 1,000,000 input tokens, an illustrative estimate rather than a vendor quote. Add a 20% retry and failure allowance for this planning case. Exclude vector-database writes, queue compute, storage, evaluation, and engineering labor so the comparison stays focused on embedding generation.
| Step | Assumption or input | Calculation | Output |
|---|---|---|---|
| 1 | Full corpus | 12,000,000 tokens | 12.00 million tokens |
| 2 | Changed share | 12,000,000 × 8% | 960,000 tokens |
| 3 | Retry allowance | 960,000 × 120% | 1,152,000 billable-estimate tokens |
| 4 | Incremental refresh estimate | 1.152 × $0.10 per million | $0.1152 |
| 5 | Full refresh estimate | 12 × $0.10 per million | $1.20 |
| 6 | Avoided embedding charge | $1.20 − $0.1152 | $1.0848 |
The incremental path is therefore about 90.4% lower for this embedding-meter assumption, calculated as $1.0848 divided by $1.20. That is not a promise of total system savings. If the changed chunks cause many index writes, require expensive OCR or parsing, or force a quality investigation, total work can be higher. Conversely, if your provider bills a different unit or your self-hosted workload is dominated by GPU idle time, token price is not the right primary driver.
Use the RAG, embedding and vector database cost calculator to replace the illustrative rate with your own input volume, change share, retry rate, and storage assumptions. For self-hosted embedding jobs, compare the queue’s actual GPU utilization with the Self-hosted GPU inference cost calculator. For a month-end control, reconcile estimated embedding events to provider usage with the AI API invoice reconciliation calculator.
Thresholds: how to avoid a false economy
A threshold is safe only when it has an exception path. Start with four questions for each corpus class:
- What is the maximum stale-answer age the business accepts?
- Which fields are semantically critical even when their character diff is tiny?
- What retrieval-quality floor should stop an optimization and force investigation?
- What is the largest queue or migration that can be rolled back within the service window?
Then use a two-stage filter. First, classify the event: embedded text, metadata, critical field, model contract, quality signal, delete, or reconciliation mismatch. Second, apply a numeric threshold only inside the class where it makes sense. For narrative text, a normalized diff ratio may be a useful prioritization signal. For a regulated policy, a protected-field change should bypass that ratio.
A lazy re-embedding RAG design can be appropriate during a model migration or for a large low-traffic corpus. On access, enqueue the document for the new index, while a background worker drains the long tail. Make the serving path explicit about which index is authoritative for a query. Lazy migration is not permission to mix query and document vectors from incompatible models, and it can leave rarely accessed documents stale indefinitely unless a maximum-age or coverage target is enforced.
Keep the policy observable. At minimum, log source ID, source revision, canonical text hash, embedding model ID, task format, dimensions, index version, trigger reason, enqueue time, completion time, attempt count, token usage, and final status. OpenAI’s embedding response includes model and token-usage fields [1], which makes those useful accounting dimensions when that API is used. Google’s task-specific retrieval guidance reinforces why the input format belongs in lineage, not only the provider model name [2].
FAQ
Should every document update trigger a new embedding?
No. Compare the canonical text sent to the embedding model, not only the database row timestamp. If only metadata, ownership, or a filter field changed, update that metadata path and leave the vector alone. If the embedded text or a protected semantic field changed, enqueue the affected chunk. The policy should document exceptions for fields where a small textual change has a large business consequence.
When should RAG embeddings be refreshed on a schedule?
Use a schedule for reconciliation and evaluation, not as the default replacement for event-driven updates. A daily or weekly audit can find missed events, stale revisions, deletes, and quality drift. The interval should follow your maximum tolerated stale age and operational risk. Stable archives may need only a weekly audit; fast-changing prices or policies may need near-real-time events plus a shorter reconciliation window.
Can I change the embedding model without reprocessing everything?
Only if the provider documents compatibility or you have validated an alignment mechanism for your exact use case. The safe default is a new index or namespace, a source-text backfill, compatible query/index pairs during shadow testing, and a controlled cutover. Do not put new-model document vectors and old-model document vectors into one candidate set merely because their dimensions happen to match.
Is a text-diff percentage a good re-embedding trigger rule?
It is useful for prioritizing low-risk narrative changes, but it is not sufficient by itself. A tiny edit to a prohibition, price, eligibility rule, or dosage can change the answer. Combine the diff ratio with field-aware exceptions, source revision ordering, and a retrieval-quality feedback loop. If the quality monitor detects a miss tied to a skipped document, allow a quality_reembed trigger to override the original skip.
What should a rollback plan retain?
Retain the previous query model, the previous index or namespace, source revisions, and enough dual-write or replay information to bring the old pair current. A configuration flag alone is not a rollback if the old index missed updates during the migration. Test rollback with a small traffic slice and record which retrieval-quality and freshness checks must pass before deleting the old pair.
Run your scenario
Turn this framework into an editable estimate.
Use dated, source-linked assumptions and see cost per request, successful task and month.
Primary sources checked
Pricing and product behavior can change. These sources support the time-sensitive claims in this guide; verify current provider terms before committing spend.