Struct IngestionWorkerPool 

pub struct IngestionWorkerPool { /* private fields */ }

High-level wrapper around the prometheus_parking_lot WorkerPool for document ingestion.

Implementations

impl IngestionWorkerPool

pub fn new( worker_count: usize, max_queue_depth: usize, ingest_service: Arc<IngestService>, persistence: Arc<dyn PersistenceLayer>, config: Arc<AppConfig>, ) -> Result<Self, PoolError>

Create a new ingestion worker pool.

Arguments

• worker_count - Number of worker threads (0 = auto-detect based on CPU)
• max_queue_depth - Maximum pending jobs before backpressure
• ingest_service - Shared ingest service
• persistence - Persistence layer for status updates
• config - Application config (used for file-processing provider selection and, optionally, per-task deadline from server.shutdown_timeout_secs)

pub fn with_task_deadline_ms(self, deadline_ms: Option<u128>) -> Self

Set the per-task deadline in milliseconds from now.

When set, each submitted task receives an absolute deadline. Tasks still running when the deadline passes are abandoned after the cancellation check between stages.

pub async fn submit( &self, document: KnowledgeDocument, file_content: Vec<u8>, ) -> Result<String, PoolError>

Submit a document for ingestion.

Returns a job key that can be used to retrieve the result.

pub async fn retrieve( &self, key: &MailboxKey, timeout: Duration, ) -> Result<IngestionResult, PoolError>

Retrieve the result of an ingestion job.

Blocks until the job completes or the timeout expires.

pub fn shutdown(&self)

Shut down the worker pool gracefully.

Signals the cancellation token so in-flight executors can abort between processing stages, then calls the fixed WorkerPool::shutdown(&self) which honours a join timeout and detaches wedged workers.

This method takes &self so it is callable through an Arc.

Trait Implementations

impl Debug for IngestionWorkerPool

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.