Skip to main content

RESPONSE_RESERVE_TOKENS

Constant RESPONSE_RESERVE_TOKENS 

Source
pub const RESPONSE_RESERVE_TOKENS: usize = 1_000;
Expand description

Output tokens withheld from the model context for the response. This is the reserve reduce_history subtracts when it derives the input allowance.