mlx-optiq
OptiQ Code · Cloud & remote models

Cloud & remote models

OptiQ Code is built for local models, and that stays the default. But the client is a plain OpenAI-compatible consumer, so it drives any endpoint that speaks the Chat Completions API. When you would rather not run a model locally, point it at a cloud provider like OpenRouter, or at your own remote server, and bring your own key.

Point it at an endpoint

Three settings do it: the base URL, your API key, and the model name. Set them as environment variables, or with the matching flags.

EnvWhat it is
OPTIQ_BASE_URLThe provider's OpenAI-compatible base URL, e.g. https://openrouter.ai/api/v1
OPTIQ_API_KEYYour key for that provider, sent as an Authorization: Bearer header
OPTIQ_MODELThe model id to use (also --model). Required for a cloud endpoint; optional against a local optiq serve, which reports what it is serving
OPTIQ_CONTEXT_WINDOWThe model's context window in tokens, so compaction has a budget (see below)

Example: OpenRouter

Run OptiQ Code against Qwen3.5-397B-A17B hosted on OpenRouter. The model runs on their hardware; OptiQ Code drives it exactly as it drives a local one.

terminalbash
# point at OpenRouter with your key, and name the model
$ export OPTIQ_BASE_URL=https://openrouter.ai/api/v1
$ export OPTIQ_API_KEY=sk-or-v1-...
$ export OPTIQ_MODEL=qwen/qwen3.5-397b-a17b
$ export OPTIQ_CONTEXT_WINDOW=131072

# drive it in a repo (interactive)
$ cd my-project && optiq code

# or headless toward a goal
$ optiq code -p "Fix the failing test in parser.py"

Because OPTIQ_BASE_URL is set, OptiQ Code attaches to that endpoint and never spawns a local server. Name the model with OPTIQ_MODEL (or --model): a cloud endpoint lists many models, so there is nothing to auto-discover, and an explicit name is required.

Which model a session records

The two settings are not two names for one thing. OPTIQ_BASE_URL says where to send requests; OPTIQ_MODEL says which model to ask for, and it is also what OptiQ Code loads when it has to start a server for you. That last part is why it exists: with nothing serving, optiq code runs optiq serve --model <your model> itself.

Against a local optiq serve you can leave it unset. The server says what it is serving and OptiQ Code uses that.

When both are set and they disagree, the server wins if it says so unambiguously — a single-model list, or the :think / :no-think labels optiq serve exposes for the model it has loaded. OptiQ Code prints the override:

  [server] http://127.0.0.1:8096/v1 serves mlx-community/Qwen3.5-0.8B-OptiQ-4bit; ignoring configured model google/gemini-3.6-flash

A cloud endpoint listing hundreds of models cannot know which one you want, so there your explicit name still decides.

This matters beyond the console line. The model id is recorded in the session and travels in the header of any trace you export, so a stale value in ~/.optiq/config.yaml would otherwise label a run with a model that never saw it.

Compaction needs a window

On a local optiq serve, OptiQ Code reads the context window from the server and compacts old tool output before it overflows. A cloud endpoint has no such probe, so set OPTIQ_CONTEXT_WINDOW to the model's window in tokens. Without it, compaction stays off and a long task can run past the model's limit.

Claude through Anthropic's API

Pointing OptiQ Code directly at Anthropic's OpenAI-compatible endpoint (https://api.anthropic.com/v1) is not supported. Two things go wrong:

  • Claude 5 models fail on the first request. On an endpoint other than optiq serve, OptiQ Code always sends a sampling temperature, 0.6 unless you set OPTIQ_TEMPERATURE. Anthropic rejects that field on these models with an HTTP 400.
  • Nothing is cached. This endpoint does not do prompt caching, so on a model that accepts the request, every turn bills the whole conversation again. On a long task that cost adds up quickly.