When the server uses deployment profiles, supply x-werk-deployment: <instance-id>
on control requests. The ID selects a concrete adapter; replica aliases are not
control targets. Adapters expose runtime.placement for the resolved plan, without
claiming live expert movement. The authenticated diagnostic extension
GET /werk/v1/deployments returns inventory/plans/slots as plain JSON; it does not
replace existing protocol envelopes. See profiles.
Back to the documentation index
Werk Protocol 1.0 is the versioned runtime-control API under /werk/v1. It is
separate from the OpenAI-compatible and Werk-native inference routes under
/v1; adding it does not change those established contracts.
The architecture, backend matrix and persistence guarantees are documented in Runtime persistence and memory architecture.
The default server is:
http://127.0.0.1:11434
When werk serve has API keys configured, every protocol route accepts either:
Authorization: Bearer sk-werk-example
or:
X-API-Key: sk-werk-example
The authenticated key is converted to an opaque principal for runtime-state
isolation. It is not sent to the transport-neutral service or used as a
filesystem name. When the server is deliberately started without configured
keys, all requests share the local principal.
Werk does not terminate TLS. The bundled client accepts only an explicit
http://HOST:PORT URL, so it is intended for loopback or a trusted private
hop. Do not send a key over an untrusted network.
Clients should advertise the JSON representation and the greatest Werk Protocol version they can consume:
Accept: application/json
X-Werk-Protocol-Version: 1.0
For compatibility with initial 1.0 clients, either request header may be
omitted. If Accept is present, it must allow application/json,
application/* or */* with a non-zero quality value. Every supplied
X-Werk-Protocol-Version value must be able to consume the server’s 1.0
response. Otherwise the server returns HTTP 406 with the typed
incompatible_protocol error.
Every success response has this envelope:
{
"protocol": {"major": 1, "minor": 0},
"request_id": "req_opaque-value",
"data": {}
}
Every protocol error has this envelope:
{
"protocol": {"major": 1, "minor": 0},
"request_id": "req_opaque-value",
"error": {
"code": "invalid_request",
"message": "human-readable explanation",
"retryable": false
}
}
X-Werk-Request-Id repeats the response request ID. Every protocol response,
including an error, declares X-Werk-Protocol-Version: 1.0,
Content-Type: application/json, Cache-Control: no-store, Vary: accept
and X-Content-Type-Options: nosniff.
Compatibility requires the same major version. The bundled 1.x client accepts a producer minor version no newer than the client’s minor version and rejects an incompatible envelope before returning its payload. It also verifies a response version header when present and rejects disagreement with the JSON envelope. A missing response version header remains accepted for compatibility with an earlier 1.0 server; the envelope itself is always required.
Discovery is the required first step. A client must use the reported status,
not infer support from active_backend.
Returns the service, active adapter and negotiated limits:
{
"service": "werk1112",
"service_version": "1.10.2",
"protocol": {"major": 1, "minor": 0},
"active_backend": "llama-server-cpu",
"limits": {
"max_page_size": 100,
"max_state_ids_per_operation": 100,
"max_expert_ids_per_operation": 256,
"max_request_bytes": 1048576,
"max_handoff_bytes": 4096,
"max_ttl_seconds": 2592000
}
}
The body is the data member of the common success envelope. Examples on this
page show data values alone where that keeps them readable.
Returns an array of:
| Field | Type | Meaning |
|---|---|---|
id |
string | Stable capability identifier such as runtime.pd.prefill. |
status |
enum | supported, unsupported, unavailable, experimental, externally_managed, or metadata_only. |
detail |
string | Current backend-specific reason or scope. |
operations |
string array | Operation names scoped by this entry. The status remains authoritative; a listed operation can still be presently unavailable. |
The capability semantics and production matrix
are normative for client behavior. experimental requires an effective
request-level opt-in. A client can send it explicitly; for Prefill only, a
server started with --persistence can supply it when the member is absent.
That server option does not opt other protocol operations in.
api.chat.omlx_options reports whether the chat endpoint accepts the
werk.omlx.thinking, werk.omlx.expert_cache_mb and
werk.omlx.ngram_cache_mb request fields. Its operations are thinking,
expert_cache_mb and ngram_cache_mb. The latter accepts a MiB integer or "auto"; omission inherits server settings. A supported status means
the API understands and validates these fields; model and runtime
compatibility are checked when the chat request is made. It does not imply
expert residency, named state or Prefill support. Clients must require this
capability before sending explicit options, because older servers may ignore
unknown chat fields. See the chat API contract.
runtime.model_residency is deliberately independent of every
runtime.state.* capability. Its automatic_reuse operation reports that an
exact model or pipeline can remain loaded between ordinary requests. It does
not produce a state_id or handoff and does not promise prompt/KV reuse,
durability, or cross-restart recovery. A local Werk-owned process or worker can
report supported; a remote service can report externally_managed; a
one-shot runner can report unsupported; and an adapter with a temporarily
missing runtime or process prerequisite can report unavailable. The current
backend-specific detail remains authoritative.
For vLLM, runtime.state.prefix_cache is externally_managed with the sole
operation automatic_reuse only when at least one active Werk-started process
is local, no active process is remote, and every active process explicitly
enables APC without an effective disable flag. No process is unavailable;
all-disabled is unsupported; remote, mixed or ambiguous evidence is
metadata_only. None of those statuses makes the named-state, prune,
persistence, handoff or decode-from-state endpoints operational.
An active local vLLM process separately reports model residency as
supported; an active remote endpoint reports it as externally_managed.
werk serve --persistence may supply native APC arguments only when Werk
starts vLLM locally and the operator has not explicitly selected an APC flag.
It sends no launch setting to remote vLLM.
| Method | Path | Request | data response |
|---|---|---|---|
| GET | /werk/v1/info |
none | Runtime info and limits |
| GET | /werk/v1/capabilities |
none | Capability array |
| GET | /werk/v1/memory |
none | Memory telemetry and accounting |
| GET | /werk/v1/observability |
none | Server-wide chat, worker and memory observations |
| POST | /werk/v1/chat/count_tokens |
OpenAI chat request | Exact native input token count |
| GET | /werk/v1/chat/reasoning |
model, max_output_tokens query |
Model/runtime tiers, default and effective limits |
| GET | /werk/v1/chat/limits |
model query |
Read-only model/server context, output and concurrency limits |
| GET | /werk/v1/states |
query filters | One page of runtime states |
| POST | /werk/v1/states/{id}/actions |
State action | Updated/projected state |
| POST | /werk/v1/states/prune |
Explicit selector | Match/removal summary |
| GET | /werk/v1/experts |
query filters | One page of expert metadata |
| POST | /werk/v1/experts/actions |
Explicit expert IDs/action | Expert action summary |
| POST | /werk/v1/prefill |
Input and persistence policy | Opaque handoff and state metadata |
| POST | /werk/v1/decode |
Opaque handoff and decode options | Completion and optional next handoff |
All listed POST routes have a fixed 1 MiB HTTP body limit. Individual DTOs have tighter semantic limits where described below. Unknown fields are rejected on the mutation, prefill and decode request DTOs.
This is an optional control-plane extension. Existing OpenAI-compatible and Anthropic-compatible generation routes keep their request/response formats and do not require a Werk handshake, version header or prior counting call. Clients may ignore this capability; generation still performs its own context admission.
api.chat.token_count with status supported and operation count_tokens
advertises POST /werk/v1/chat/count_tokens. This additive route accepts the
same chat request as /v1/chat/completions, including tool definitions,
assistant tool calls, their results and supported template/API options. It
counts the selected backend’s actual chat template without trimming history or
generating output. stream is ignored; the response is always a protocol JSON
envelope:
{"protocol":{"major":1,"minor":0},"request_id":"req_example",
"data":{"model":"selected-model-or-alias","input_tokens":3012}}
The count excludes the reserved output budget. Clients must add their output reserve and margin before context admission. Cache counts only for the exact model, messages, tools and options. The server validates context again during generation; a count does not reserve capacity or promise a reusable state. The selected model may be prepared to access its tokenizer.
An unsupported native counter returns the typed unsupported error, never
an estimate labeled as an exact count. Temporary native failures return
unavailable with retryable: true; clients may retry this read-only operation
with a bounded backoff. Authentication, versioning and the 1 MiB request bound
are the same as other protocol routes.
api.chat.admission with operation admission advertises
POST /werk/v1/chat/admission. It accepts the same complete chat request and
returns a planning budget without generation or history trimming:
{"protocol":{"major":1,"minor":0},"request_id":"req_example",
"data":{"model":"selected-model-or-alias","input_tokens":14000,
"max_output_tokens":4096,"context_tokens":32768,"token_count_method":"estimate"}}
The method is native when the backend counted the complete request, or
estimate when its native counter is unavailable (including unsupported image
counting). The fallback is exactly the generation admission estimator, not a
client’s independently calibrated image estimate. context_tokens is null if
the server has no established limit. Clients add the returned output budget and
their margin to the input budget, compact/reduce before dispatch, and keep
estimates separate from exact-count caches. The native-only count_tokens
contract is unchanged. Authentication, request bounds, deployment routing and
template/options validation match the counting route. This planning response
does not reserve inference capacity; generation validates the final request again.
For existing local llama.cpp workers, /werk/v1/observability also reports
context_per_request_tokens: the minimum n_ctx across all validated native
slots. context_capacity_tokens remains the sum across slots and must not be
used as an individual request limit or divided to infer a minimum. Use only
fresh, available samples for the selected model. If several workers match,
take the minimum across workers. Missing or invalid slot information leaves
the per-request limit unknown.
Example data:
{
"observed_at_unix_ms": 1788444000000,
"overall_pressure": "normal",
"topology": "discrete",
"host": {
"capacity_bytes": 68719476736,
"available_bytes": 42949672960,
"managed_bytes": 0,
"reserved_bytes": 0,
"pressure": "normal"
},
"accelerator": {
"capacity_bytes": null,
"available_bytes": null,
"managed_bytes": 0,
"reserved_bytes": 0,
"pressure": "unknown"
},
"last_action_unix_ms": null,
"counters": {
"active_reservations": 0,
"managed_allocations": 0,
"pressure_actions_in_flight": 0,
"completed_demotions": 0,
"completed_evictions": 0,
"failed_releases": 0,
"orphaned_release_bytes": 0,
"telemetry_errors": 0,
"backend_cleanup_failures": 0,
"backend_cleanup_latched": 0
}
}
capacity_bytes and available_bytes are nullable because Werk refuses to
invent missing telemetry. Topology is currently discrete, unified or
unknown; pressure is normal, soft, hard, emergency or unknown.
Only allocations made through an adapter with bounded pre-load requirements
appear as managed/reserved bytes.
failed_releases counts backend cleanup failures for managed allocations.
Werk then retains the allocation in accounting and pins it against policy
movement instead of claiming that capacity was freed. Reservation admission
uses projected utilization: after a rejection, one bounded relief plan may
demote or evict eligible least-recently-used state until the requested bytes
would remain below the hard threshold, even if the current pre-request
pressure is still normal or soft. orphaned_release_bytes is the known
logical size whose backend cleanup could not be confirmed; unknown sizes add
zero rather than an estimate. telemetry_errors records a failed observation
for the current response, while managed accounting remains visible.
backend_cleanup_failures is the service-level cleanup failure count and
backend_cleanup_latched is 1 while further unsafe state mutation is blocked.
A state summary contains:
| Field | Type | Meaning |
|---|---|---|
id |
string | Opaque st_... identifier. |
model_id, backend |
string | Installed model and producing backend. |
tier |
enum | vram, ram, disk, or external. |
status |
enum | ready, loading, moving, unavailable, or quarantined. |
bytes |
integer or null | Logical state size when the backend can report it. |
created_unix_ms, last_accessed_unix_ms |
integer | Unix milliseconds. |
expires_unix_ms |
integer or null | Policy expiry. |
pinned |
Boolean | Protected from policy eviction. |
reusable |
Boolean | Whether reuse is currently valid. |
Backend handles, filesystem paths and compatibility fingerprints are not part of this public summary. Expiry overrides pinning: an expired pinned state is not reusable or visible and is deleted by the next real catalog mutation.
Optional query fields are model_id, tier, limit and opaque cursor.
limit is 1 through the discovered max_page_size (currently 100). Results
are ordered by opaque state ID and return next_cursor when another page
exists.
curl -sS \
-H "Authorization: Bearer $WERK_API_KEY" \
'http://127.0.0.1:11434/werk/v1/states?model_id=my-model&tier=disk&limit=50'
The request fields are:
{
"action": "promote",
"target_tier": "ram",
"dry_run": true,
"allow_experimental": false
}
action is pin, unpin, promote, demote or evict.
target_tier is required only for promote/demote and must move in the stated
direction. external cannot be selected as a local movement target. The
response contains state, changed and dry_run; on a dry run, state
describes the projected result without invoking the mutation. State reads and
dry-run previews do not initialize or reconcile persistent storage, update
access metadata, quarantine corruption or delete expired entries; invalid and
expired entries are omitted or reported unavailable instead.
Prune requires exactly one tagged selector. IDs:
{
"selector": {"kind": "ids", "ids": ["st_opaque-one", "st_opaque-two"]},
"dry_run": true
}
Constrained filter:
{
"selector": {
"kind": "filter",
"model_id": "my-model",
"tier": "disk",
"older_than_unix_ms": 1788444000000
},
"dry_run": true
}
Every visible state:
{
"selector": {"kind": "all", "confirm": true},
"dry_run": true
}
An ID selector requires 1 through max_state_ids_per_operation unique IDs. A
filter requires at least one non-null restriction. all is rejected unless
confirm is true. dry_run defaults to true if omitted. The response reports
matched, removed, nullable logical bytes, and dry_run; a preview has
removed: 0 and does not perform catalog cleanup.
Optional query fields are model_id, tier (vram, ram, or external),
limit, cursor, and allow_experimental (default false). Backend routing
and its capability check are bound to one captured adapter for the operation.
A successful adapter response contains experts and
next_cursor; each expert has an opaque ID, model ID, tier, nullable byte
size, numeric hotness, pin state and optional last-use timestamp.
{
"model_id": "moe-model",
"expert_ids": ["expert_opaque"],
"action": "prefetch",
"target_tier": "vram",
"dry_run": true,
"allow_experimental": false
}
Actions are prefetch, pin, unpin and evict. IDs are explicit and
bounded by max_expert_ids_per_operation. Target handling is strict:
| Action | target_tier |
|---|---|
prefetch |
Required and must be vram or ram; external is rejected. |
pin, unpin, evict |
Must be omitted. |
The response contains experts, changed and dry_run.
The oMLX adapter implements these operations experimentally when
WERK_OMLX_EXPERT_CACHE_MB or a positive werk.omlx.expert_cache_mb chat
option enables SSD expert offload and a loaded worker has
confirmed the cache is active. Start the model through normal generation in
the same persistent Werk service before requesting expert telemetry; these
routes do not load a model. Both list and action requests must explicitly set
allow_experimental: true. Capability discovery remains authoritative.
ComfyUI and n8n text nodes can configure the chat options directly. For a model used with multiple cache budgets, subsequent expert requests target the worker most recently used for that model. A prepared expert operation remains bound to its original worker while it executes.
For this adapter, external denotes experts backed by the local checkpoint
outside the cache, and ram denotes its active Apple unified-memory cache.
The tier is independent of the capability status externally_managed, which
still permits read-only telemetry. oMLX accepts only ram as the prefetch
target; vram is rejected. Pin, unpin and evict omit target_tier. Eviction
releases cache residency and retains the checkpoint source for future loads;
it does not delete model files. Cache capacity also applies to pin and
prefetch. A dry-run does not load, evict or change pins.
Other adapters retain their declared capability and target-tier behavior. Route presence or a model architecture alone is not a support claim.
Text input:
{
"model_id": "my-model",
"input": {"type": "text", "text": "A reusable prefix"},
"policy": {
"mode": "auto",
"reuse": "prefer",
"ttl_seconds": 3600,
"pin": false
},
"allow_experimental": true
}
Message input:
{
"model_id": "my-model",
"input": {
"type": "messages",
"messages": [
{"role": "system", "content": "Answer concisely."},
{"role": "user", "content": "Describe the model."}
]
},
"policy": {"mode": "ephemeral", "reuse": "disabled", "pin": false},
"allow_experimental": true
}
mode is ephemeral, memory, disk or auto; reuse is disabled,
prefer or required. The policy defaults to auto/prefer, no TTL and not
pinned. TTL is 1 through max_ttl_seconds. Text or total message content is
limited to 512 KiB; a message array contains 1 through 256 role/content items.
By default, omitting the top-level policy member uses that protocol default,
and omitting allow_experimental means false. A server started with
werk serve --persistence can instead provide a policy and experimental opt-in
independently for each omitted member. In particular, policy: {} does not
prevent a missing allow_experimental from defaulting to true under server
persistence. The granular server options are
--persistence-mode, --persistence-reuse,
--persistence-ttl-seconds and --persistence-pin.
Request intent has precedence. If policy is present, the entire request
policy wins; this includes {}, whose missing inner fields receive the normal
protocol defaults rather than the server’s granular defaults. An explicitly
supplied allow_experimental value also wins, including false. Server
defaults do not change capability status or bypass backend validation.
This policy-default behavior exists only at POST /werk/v1/prefill.
Separately, the server flag configures local vLLM APC defaults and, with
auto/disk mode and reuse enabled, verified short exact-prefix caching in
local oMLX 0.6.4 workers. Those native caches do not redirect /v1 calls through
Prefill, cache complete answers, persist model weights or guarantee
cross-restart restore. Media inference is unchanged. Named runtime state
remains an opaque backend-owned value governed and compatibility-checked by
Werk. Current named persistence and prefill/decode support is experimental and limited to a
functionally validated Werk-managed llama-server process for the exact
installed GGUF model; clients must still use capability discovery.
Example data:
{
"handoff": "opaque-secret-value",
"state_id": "st_opaque-id",
"prompt_tokens": 42,
"reused": false,
"tier": "disk",
"expires_unix_ms": 1788444900000
}
state_id is null for ephemeral prefill. The handoff is a secret bearer value
for one decode, not a portable state serialization. Do not log it, persist it
in workflow JSON or expose it in a UI string. Live handoffs are bounded to 128
per principal and 1024 across the server. Expired handoffs are discarded first;
if a bound remains full, prefill fails with retryable resource_exhausted
without invalidating any live handoff. Prefill reserves its handoff slot before
backend or persistence work and rolls that reservation back on failure, so it
cannot complete expensive state work and then fail only because another
request filled the registry.
{
"handoff": "opaque-secret-value",
"max_tokens": 256,
"temperature": 0.7,
"top_p": 0.9,
"seed": 42,
"stop": ["</answer>"],
"allow_experimental": true
}
max_tokens is 1 through 32768. Optional temperature is 0 through 2,
optional top_p is greater than 0 through 1, and stop contains at most 16
non-empty strings of at most 1024 bytes each. Backend-returned completion text
is limited to 1 MiB of UTF-8 bytes.
Decode consumes the handoff before backend execution. It never retries on a
different backend. Consumption atomically reserves capacity for a possible
replacement handoff; that reservation is committed only if the backend returns
updated state and is otherwise released. Example data:
{
"text": "Completion text",
"handoff": null,
"completion_tokens": 18,
"finish_reason": "stop"
}
handoff is non-null only if the backend produced a new state for a later
decode.
| Code | HTTP status | Meaning |
|---|---|---|
invalid_request |
400 | The request shape, value or selector is invalid. |
incompatible_protocol |
406 | The request cannot accept the server’s protocol version or JSON representation. |
unauthorized |
401 | Credentials are missing or invalid. |
forbidden |
403 | The operation or state directory is unsafe/forbidden. |
not_found |
404 | A model or state is absent or expired. |
conflict |
409 | The request conflicts with current state, including required reuse with no match. |
incompatible_state |
409 | The producer compatibility envelope does not match. |
expired_handoff |
410 | The handoff is invalid, expired, already consumed or belongs to another principal. |
experimental_opt_in_required |
428 | An experimental operation lacks explicit opt-in. |
resource_exhausted |
429, or 413 for the HTTP body limit | A catalog, memory, request or other bounded resource limit was reached. |
unsupported |
501 | The active adapter does not implement the operation. |
unavailable |
503 | A current process, probe, telemetry or secure-identity prerequisite is missing. |
corrupt_state |
500 | Stored state failed integrity validation. |
internal |
500 | An internal control-plane operation failed. |
The HTTP adapter replaces internal/corrupt-state detail with a stable public
message and does not forward arbitrary backend JSON. For
incompatible_state, it may expose only an allow-listed
details.mismatch_fields string array produced by compatibility validation;
all other arbitrary detail remains redacted. WWW-Authenticate: Bearer
accompanies a 401 response.
werk1112::werk_protocol::WerkProtocolClient exposes typed methods matching
all ten routes:
use werk1112::werk_protocol::{StateListFilter, WerkProtocolClient};
let client = WerkProtocolClient::new(
"http://127.0.0.1:11434",
std::env::var("WERK_API_KEY").ok(),
)?;
let info = client.info()?;
let capabilities = client.capabilities()?;
let states = client.list_states(&StateListFilter::default())?;
The client requires http://, an explicit port and a base URL containing no
path or user information. It has a 30-second default timeout, accepts a custom
timeout, sends no redirects, bounds requests at 1 MiB and responses at 8 MiB,
and validates both content-length and chunked response framing. It sends
Accept: application/json and X-Werk-Protocol-Version: 1.0, then checks the
response header (when supplied) against the envelope. Its debug and error
representations never include the API key.
The runtime CLI is a thin pretty-JSON interface over this client. ComfyUI uses its own strict Python validator and keeps handoffs in an opaque socket; see the custom-node guide.
Capability api.chat.limits (limits) advertises
GET /werk/v1/chat/limits?model=MODEL. Authentication, version headers, no-store
and the standard envelope apply. The default model is used when the query omits
model. Discovery reads manifests/configuration without loading or probing a
model and without running inference or advancing deployment selection.
Example data:
{"schema_version":1,"model":"local-model","context_tokens":16384,"max_output_tokens":16383,"max_concurrent_requests":2}
Fields may be null when unknown. Context is per request. Output is the
minimum of explicit manifest parameter ceilings (max_output_tokens,
max_completion_tokens, max_new_tokens, scalar or { "max": N }) and context
minus one when context is known. Generation defaults are not hard ceilings.
For an alias, known limits use the minimum across replicas; an unknown replica
makes that field unknown. This endpoint does not predict free GPU memory or
guarantee that a particular prompt/output pair fits.
max_concurrent_requests reports configured concurrent inference capacity, not
currently free/busy slots. For running llama.cpp workers it is read from the
actual native launch arguments (--parallel / -np, including trailing
WERK_LLAMA_ARGS overrides), without querying inference or telemetry endpoints.
Unknown/auto capacity and cold models report null. Multiple matching workers
and alias replicas use the conservative minimum, never an assumed sum. An
unknown replica makes the alias capacity unknown. Other backends may leave this
optional field unknown. Clients may refresh it without generating tokens.
Clients must count input and reserve their margin before expanding an initial
output allowance. Chat admission rejects output above the same ceiling with
output_limit_exceeded; the read-only token counter can still count an oversized
request. Reasoning profiles use that ceiling as well as the request’s output
budget. Increasing output does not override an explicit reasoning tier.
File-editing clients can use these ceilings to plan complete sections across multiple calls: reserve input and safety margin, then divide the actual output allowance between reasoning, tool JSON and file content. Resolve numeric reasoning from the selected/default available tier in the reasoning profile for that output allowance. A character target is only a heuristic: the protocol does not advertise a universal tokens-to-characters ratio or a maximum file-edit character count. Unknown numeric reasoning must remain unknown. Paginated tool reads and atomic section writes are client tool behavior, not extra inference slots or server-side continuation of an incomplete tool call.
Capability api.chat.reasoning (profile, tiers) advertises
GET /werk/v1/chat/reasoning?model=MODEL&max_output_tokens=4096. The response uses
the standard authenticated protocol envelope, no-store, and version headers.
The output limit defaults to 256, matching chat requests. Querying does not load
models, probe GPU capabilities, count tokens, or generate text. Only verified
configuration of running workers is reported. Unknown/cold runtimes return
supported:false, an empty tiers list, and default_tier:null. Deployment
aliases require agreeing profiles across replicas; otherwise select an ID.
Example data for thinking enabled, a 2048-token launch budget and 2048 output:
{
"schema_version": 1,
"model": "my-model",
"backend": "llama.cpp",
"supported": true,
"source": "worker_launch_configuration",
"default_tier": "medium",
"default_adjusted_to_limits": true,
"limits": {
"context_tokens": 32768,
"output_tokens": 2048,
"answer_reserve_tokens": 512,
"reasoning_tokens": 1536
},
"tiers": [
{"id":"none","label":"Off","available":true,"reasoning_tokens":0,"minimum_output_tokens":1},
{"id":"low","label":"Low","available":true,"reasoning_tokens":512,"minimum_output_tokens":1024},
{"id":"medium","label":"Medium","available":true,"reasoning_tokens":1024,"minimum_output_tokens":1536},
{"id":"high","label":"High","available":false,"reasoning_tokens":2048,"minimum_output_tokens":2560},
{"id":"max","label":"Max","available":false,"reasoning_tokens":4096,"minimum_output_tokens":4608},
{"id":"ultra","label":"Ultra","available":false,"reasoning_tokens":8192,"minimum_output_tokens":8704}
]
}
Clients send "werk":{"reasoning":{"tier":"medium"}} in both chat and
/werk/v1/chat/count_tokens requests. "default" resolves the current server
default on every request; clients must not freeze the reported default as an
explicit tier. An unsupported runtime preserves its native default for this
sentinel; explicit IDs require a verified profile. Unknown IDs, unavailable tiers
and combinations with native reasoning_effort or werk.omlx thinking/effort
controls fail before generation. OpenAI chat errors use invalid_reasoning_tier.
For llama.cpp with verified budget control, these are Werk-owned budget tiers,
not a claim that every template supports native effort strings with these names.
The resolver uses the smaller of the launch budget (nonnegative values) and the
output limit minus a 512-token answer reserve. It conservatively limits output
to context minus 512 tokens; ordinary prompt/context admission still applies.
A named launch effort matching a tier is retained or reduced to fit. Otherwise
the enabled default is the highest available tier; a disabled launch default
remains none. A tier is a ceiling, not a guarantee that all tokens are used or
that the final answer fits. Unlimited launch budgets remain bounded by output.
Werk forwards the resolved budget and thinking switch to llama.cpp.
Native effort backends use their verified vocabulary, narrowed by model
parameter_constraints.reasoning_effort.enum when present. They report
reasoning_tokens:null and no guaranteed answer reserve; the total output cap
still applies. Explicit native reasoning_effort remains a separate legacy API
and bypasses managed tier resolution. Clients must consume the reported list
rather than hardcode a universal vocabulary or numeric mapping.