Expand description
Hot-swappable embedding backend handle.
The runtime shares ONE embedding backend between the vector matcher
(queries), the ingest service (writes), and diagnostics. Persisted
llm.embedding.* settings are authoritative after first boot (ADR-010), so
a REST mutation must take effect on the next embed call without a restart —
which the previously fixed Arc<dyn EmbeddingBackend> cannot do.
Coherence design:
- Backend and metadata live together in ONE
BackendSlot; a swap publishes the slot with a single atomic store. There is no window in which a reader can see one backend’s metadata with another backend’s vectors. - Jobs PIN a slot (
pin()) and use that one snapshot for the whole job — fingerprint check, chunking embeds, and document embeds. A concurrent swap therefore cannot interleave inside a job: the job finishes in the space it started in, and the next job starts in the new space. - Durable knowledge-base chunks are protected a second time by the KB
space guard (
KbConfig::ensure_embedding_space): after a swap, previously indexed KBs refuse the new space explicitly instead of mixing vectors. The in-memory skill index is derived data and is re-indexed by the vector matcher on space change (seematching/vector.rs).
The trait’s borrowed-return metadata (backend_name, model_id,
endpoint) is served from String fields owned by the slot, so no
per-swap leak and no guard-escaping borrows are needed.
Structs§
- Backend
Slot - One coherent backend + metadata snapshot. Metadata is captured eagerly so every borrow the trait returns is tied to the slot itself.
- Swappable
Embedding Backend - Shared handle publishing the current
BackendSlot. Consumers pin a slot per job; they must not assume the handle itself is anEmbeddingBackend.