Local inference · Multimodal · Rust
Werk1112 brings text, image, video and audio models to one CLI and HTTP service. Discover models, choose a compatible backend and run inference on infrastructure you control.
Runtime
Route typed requests according to the model, task, installed runtime and host capabilities. Optional hosted adapters connect to external providers and remain subject to their access requirements and costs.
Choose compatible models, hardware and runtimes to support digital sovereignty and reduce reliance on paid hosted inference. Suitable local workflows can work offline once models, runtimes and dependencies are provisioned; downloads and hosted APIs require a network.
Import existing files, link external weights or pull compatible Hugging Face repositories into a shared model store. Use typed chat, vision, image, video and audio commands with the required runtimes.
Use automatic or explicit backend selection, werk inspect and
werk doctor to diagnose requirements and routing blockers.
Planning estimates do not guarantee that a model fits or executes.
Retain supported workers with startup preload,
inspect activity with werk top, and connect
Prometheus and Grafana.
Capacity and concurrency depend on the backend and available resources.
Use backend-specific CPU/GPU offload, placement and cache controls where implemented. Weight residency, prefix/KV reuse and SSD expert caches have separate semantics; KV state is not portable between arbitrary runtimes.
Use embedding, reranking and classification endpoints with compatible models and installed pooling runtimes. The embedding API accepts text and float output; reranking and classification use their documented contracts.
Quick start
Install Werk for Linux, macOS or Windows, then provision the matching drivers, runtime and model separately. For a local GGUF chat model, choose a llama.cpp build that supports its architecture from the backend guide.
werk backend list
werk import /absolute/path/to/model.gguf --name local-model
werk doctor --model local-model
werk chat local-model --max-tokens 128Replace the path with your model file. Model, runtime and hardware must be compatible.
Generate a key and replace the placeholder with it before starting the service
(POSIX shell example). Configure compatible OpenAI clients with
http://127.0.0.1:11434/v1 and the same key.
werk auth api-key generate
export WERK_API_KEY="replace-with-generated-key"
werk serve --model local-model
For several installed text models, werk serve --preload model-a model-b performs
ordered, best-effort loading and warmup. Successful workers remain resident until shutdown;
unsupported paths and resource failures are logged and skipped.
This does not generally warm image, video or audio generation.
Hardware & backends
These are documented integration paths. Execution still depends on architecture, checkpoint, platform and installed runtime; discovery covers more models than executable adapters.
Use compatible llama.cpp, Candle or ONNX CPU paths, or llama.cpp CUDA and eligible vLLM/Python runtimes. Linux is the primary CUDA path; Windows needs a matching toolchain, WSL2 is best effort, and local vLLM requires native Linux. See the platform matrix.
Choose compatible llama.cpp Metal, MLX/MLX-VLM or oMLX adapters on macOS. Managed oMLX requires macOS 15+. Unified memory is shared physical capacity; budgets and offload settings do not guarantee model fit. See runtime requirements.
AMD Strix Halo has native Linux ROCm/HIP and Vulkan paths, with no NPU adapter. DGX Spark / GB10 uses a Spark-specific Linux aarch64 profile, without native TensorRT-LLM or NIM adapters. Both integrations still require their real-hardware smoke gates.
Single-host deployment profiles configure supported local CUDA GPU groups, aliases and replicas. Multi-GPU hardware runs remain unverified; these profiles do not provide multi-node clusters or mixed NVIDIA/AMD sharding.
Interfaces
For digital sovereignty initiatives, the documented API subsets let existing compatible clients connect to a self-operated Werk service. Client applications own agent loops and workflow execution; Werk remains an inference runtime and router.
Use Chat Completions at /v1/chat/completions and Messages at
/v1/messages for documented text, vision, streaming and client tool cycles.
These are compatible subsets;
the OpenAI Responses API is not implemented.
Use OpenAI-inspired and Werk-native media, job and file routes, plus small ComfyUI/AUTOMATIC1111 compatibility surfaces. Separate Werk Protocol 1.0 routes expose capability, state and memory controls; inspect support before using optional operations.
Connect optional ComfyUI nodes (Beta) or manually installed n8n nodes (Beta) for native media and runtime controls. Document and file inputs retain their format and runtime limits.
Documentation
The project README is intentionally short. Detailed API contracts, backend and operating-system support, fallback behavior, media tasks and known limitations live in versioned documentation.
All routes, fields, responses, task coverage, authentication, jobs, retention and compatibility gaps.
Versioned runtime control, truthful backend capabilities, opaque state, memory pressure and prefill/decode boundaries.
Routing, fallback policies, managed installation, platform support, Qwen-TTS and cleanup.
Canonical image, video and audio tasks, model layouts, parameters, planning and examples.
Install Werk1112, add a model, run local inference and start the authenticated HTTP service.
Global options, commands, model lifecycle, maintenance, diagnostics, inference and serving semantics.
Imports, external weight links, pulls, manifests, model selection, managed storage and optimized artifacts.
Local CUDA GPU groups, aliases, replicas, admission and concurrency limits, with hardware validation status.
Exact model identities, local and hosted paths, dependencies, access requirements and inference verification.
Ecosystem
Werk1112 focuses on local AI infrastructure and backend routing without becoming a workflow engine.
Werk1112 is developed in the philipbodenbach/werk1112 GitHub repository by Philip Bodenbach.