Skip to main content
Omniference is a flexible, multi-protocol inference engine written in Rust. It gives you one unified interface to AI model providers such as Ollama and OpenAI.

Features

  • Multi-provider support: Ollama, OpenAI and an extensible adapter architecture
  • Streaming: real-time streaming responses from models
  • OpenAI-compatible API: a drop-in replacement for OpenAI’s API
  • Multiple interfaces: HTTP server, Discord bot, CLI and library usage
  • Embeddable: integrates into existing Axum applications
  • Type-safe: strong typing throughout, built on Tokio

Architecture

  • Core layer: router, adapters and types (pure inference logic)
  • Service layer: provider management and model resolution
  • Interface layer: HTTP APIs, Discord bot, CLI
  • Application layer: full server or embeddable components

Quickstart

Add Omniference to your project.

Source code

Omniference on GitHub (MIT license).