Skip to main content

LlmDriver

Trait LlmDriver 

Source
pub trait LlmDriver: Send + Sync {
    // Required method
    fn stream<'life0, 'async_trait>(
        &'life0 self,
        req: LlmRequest,
    ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<NormalizedEvent>> + Send>>>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;

    // Provided methods
    fn with_bound_model(&self, _model: &str) -> Result<Arc<dyn LlmDriver>> { ... }
    fn supports_admitted_inline_images(&self) -> bool { ... }
    fn prepared_final_wire_budget(
        &self,
        request: &LlmRequest,
    ) -> Result<PreparedFinalWireBudget> { ... }
}
Expand description

Trait for LLM streaming drivers.

Implementations of this trait provide streaming access to LLM responses, emitting NormalizedEvents as the model generates output.

Required Methods§

Source

fn stream<'life0, 'async_trait>( &'life0 self, req: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<NormalizedEvent>> + Send>>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Stream a response from the LLM.

§Errors

Returns an error if the request fails or the connection is interrupted.

Provided Methods§

Source

fn with_bound_model(&self, _model: &str) -> Result<Arc<dyn LlmDriver>>

Select another model on this exact provider/client binding. The trusted host checks the child’s policy before calling; this grants no authority. Implementations must not resolve new credentials or change endpoints.

§Errors

Host drivers without model-rebinding support refuse it. The host can still share the original driver for its original model unchanged.

Source

fn supports_admitted_inline_images(&self) -> bool

Whether this captured driver binding uses the ordinary generic chat adapter that preserves admitted inline-image parts on its wire format.

This is a construction fact, not a provider or model-name capability declaration. Native adapters retain their own established paths.

Source

fn prepared_final_wire_budget( &self, request: &LlmRequest, ) -> Result<PreparedFinalWireBudget>

Return a final-wire input bound for a request carrying a proven budget contract. Drivers without captured final-wire preparation must refuse; a generic request serializer is not equivalent evidence.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§