Struct OllamaGateway 

pub struct OllamaGateway { /* private fields */ }

Gateway for Ollama local LLM service

This gateway provides access to local LLM models through Ollama, supporting text generation, structured output, tool calling, and embeddings.

Implementations

impl OllamaGateway

pub fn new() -> Self

Create a new Ollama gateway with default configuration

pub fn with_config(config: OllamaConfig) -> Self

Create a new Ollama gateway with custom configuration

pub fn with_host(host: impl Into<String>) -> Self

Create gateway with custom host

pub async fn pull_model(&self, model: &str) -> Result<()>

Pull a model from Ollama library

Trait Implementations

impl Default for OllamaGateway

fn default() -> Self

Returns the “default value” for a type.

impl LlmGateway for OllamaGateway

fn complete<'life0, 'life1, 'life2, 'life3, 'life4, 'async_trait>( &'life0 self, model: &'life1 str, messages: &'life2 [LlmMessage], tools: Option<&'life3 [Box<dyn LlmTool>]>, config: &'life4 CompletionConfig, ) -> Pin<Box<dyn Future<Output = Result<LlmGatewayResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait, 'life4: 'async_trait,

Complete an LLM request with text response

fn complete_json<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, model: &'life1 str, messages: &'life2 [LlmMessage], schema: Value, config: &'life3 CompletionConfig, ) -> Pin<Box<dyn Future<Output = Result<Value>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Complete an LLM request with structured JSON response

fn get_available_models<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<String>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Get list of available models

fn calculate_embeddings<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, text: &'life1 str, model: Option<&'life2 str>, ) -> Pin<Box<dyn Future<Output = Result<Vec<f32>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Calculate embeddings for text

fn complete_stream<'a>( &'a self, model: &'a str, messages: &'a [LlmMessage], tools: Option<&'a [Box<dyn LlmTool>]>, config: &'a CompletionConfig, ) -> Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send + 'a>>

Stream LLM responses chunk by chunk