Skip to main content

Module swappable

Module swappable 

Source
Expand description

Hot-swappable embedding backend handle.

The runtime shares ONE embedding backend between the vector matcher (queries), the ingest service (writes), and diagnostics. Persisted llm.embedding.* settings are authoritative after first boot (ADR-010), so a REST mutation must take effect on the next embed call without a restart — which the previously fixed Arc<dyn EmbeddingBackend> cannot do.

Coherence design:

  • Backend and metadata live together in ONE BackendSlot; a swap publishes the slot with a single atomic store. There is no window in which a reader can see one backend’s metadata with another backend’s vectors.
  • Jobs PIN a slot (pin()) and use that one snapshot for the whole job — fingerprint check, chunking embeds, and document embeds. A concurrent swap therefore cannot interleave inside a job: the job finishes in the space it started in, and the next job starts in the new space.
  • Durable knowledge-base chunks are protected a second time by the KB space guard (KbConfig::ensure_embedding_space): after a swap, previously indexed KBs refuse the new space explicitly instead of mixing vectors. The in-memory skill index is derived data and is re-indexed by the vector matcher on space change (see matching/vector.rs).

The trait’s borrowed-return metadata (backend_name, model_id, endpoint) is served from String fields owned by the slot, so no per-swap leak and no guard-escaping borrows are needed.

Structs§

BackendSlot
One coherent backend + metadata snapshot. Metadata is captured eagerly so every borrow the trait returns is tied to the slot itself.
SwappableEmbeddingBackend
Shared handle publishing the current BackendSlot. Consumers pin a slot per job; they must not assume the handle itself is an EmbeddingBackend.