Skip to main content

LlmDriver

universal_agent_runtime::llm

Trait LlmDriver 

Source 
pub trait LlmDriver: Send + Sync {
    // Required method
    fn stream<'life0, 'async_trait>(
        &'life0 self,
        req: LlmRequest,
    ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<NormalizedEvent>> + Send>>>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;

    // Provided method
    fn with_bound_model(&self, _model: &str) -> Result<Arc<dyn LlmDriver>> { ... }
}
Expand description

Trait for LLM streaming drivers.

Implementations of this trait provide streaming access to LLM responses, emitting NormalizedEvents as the model generates output.

Required Methods§

Source

fn stream<'life0, 'async_trait>( &'life0 self, req: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<NormalizedEvent>> + Send>>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Stream a response from the LLM.

§Errors

Returns an error if the request fails or the connection is interrupted.

Provided Methods§

Source

fn with_bound_model(&self, _model: &str) -> Result<Arc<dyn LlmDriver>>

Select another model on this exact provider/client binding. The trusted host checks the child’s policy before calling; this grants no authority. Implementations must not resolve new credentials or change endpoints.

§Errors

Host drivers without model-rebinding support refuse it. The host can still share the original driver for its original model unchanged.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§