HTTP API

Two endpoints do the real work; everything else is convenience. If your client speaks OpenAI or Anthropic, it already speaks OdyssAI-X.

OdyssAI-X exposes a public inference surface (/v1/*, always open) and an admin surface (/admin/*, open on a trusted LAN, token-gated when you choose). A single-Mac OpenAI-compatible server fronted by an http-proxy pool is reached through the same /v1/* surface.

The two endpoints that matter

POST /v1/chat/completions     OpenAI dialect
POST /v1/messages             Anthropic dialect (+ /v1/messages/count_tokens)

Every engine in the stack speaks both. Clients use OpenAI by default and Anthropic for Claude-style flows (Claude Code, the Anthropic SDK). The schemas are the standard ones — point an existing SDK at the base URL and it works:

export OPENAI_BASE_URL="http://<server>:8000/v1"
export OPENAI_API_KEY="dummy"          # no key required on a LAN

Listing models

GET /v1/models                     currently servable models
GET /v1/models?include_unloaded=true   + on-disk inventory (x_odyssai.ready=false)

By default only servable models are listed: loaded local pools, enrolled http-proxy clusters, and published cloud aliases. Each entry carries an x_odyssai block with the real backend, the concrete model behind an alias, and per-model capabilities. The CoeOS model id is published by the CoeOS box, not by the engine — see the CoeOS box.

Three extensions

On top of the standard schemas, OdyssAI-X reads three optional fields:

Field	Type	Effect
`session_id`	string	Scopes the KV prefix cache to a conversation. Reuse the same id across turns and the shared prefix is prefilled once — big TTFT win. Clients attach one per chat.
`enable_thinking`	bool	Toggles the reasoning block on reasoner models (default `true` on those that ship it).
`reasoning_effort`	`minimal` / `low` / `medium` / `high`	Budget for the reasoning block. `minimal` roughly halves completion tokens on always-think models.

The usage block in OpenAI responses adds prompt_tokens_details.cached_tokens — the prefix-cache hit count, for clients to surface.

Aliases and routing

The model field accepts several kinds of id, all resolved by the orchestrator:

`model` value	Routes to
a cluster name (e.g. `default`)	the loaded pool on that cluster
an `http-proxy` cluster id (`<cluster>` or `<cluster>:<model>`; its `kind` is `telemak`, a legacy kind name)	the enrolled single-Mac OpenAI-compatible server, over HTTP proxy
a cloud alias (e.g. `or:claude-haiku`)	the configured cloud provider

The caller doesn’t need to know the topology — it picks a published id, the orchestrator hides the rest. CoeOS is not one of these ids: it is published by the CoeOS box, which sits in front of the engine.

Admin endpoints

/admin/* controls the cluster. A few you’ll actually use:

GET  /admin/clusters                       # registered clusters + status
POST /admin/<cluster>/load                 # load a model  {"model": "...", "sharding"?: "pipeline"}
POST /admin/<cluster>/unload               # free the pool
GET  /admin/settings    PUT /admin/settings    # server-wide defaults
POST /admin/sync/rsync                     # push a model from one node to others
GET  /health                               # {"status":"idle|busy", "version": "..."}

/admin/* is open by default (trusted-LAN, single operator). Set ODYSSAI_X_ADMIN_TOKEN to require Authorization: Bearer <token> if you expose the engine beyond your LAN; /v1/* stays public regardless.

Cloud passthrough

Cloud providers are first-class. Add one in the dashboard (Settings → Cloud providers → paste an OpenRouter / Anthropic / OpenAI key) and aliases like or:claude-haiku appear in /v1/models instantly, callable through the same /v1/chat/completions. Nothing else to install.