Providers, profiles & devices reference

Caution

hal0’s source has two unrelated “provider” concepts. This page covers both, clearly separated — do not conflate them.

(A) Local inference-backend provider

hal0.providers.base.Provider — the abstract class that runs a slot’s inference process. Concrete implementations:

Provider	Module	Runs
ContainerProvider	providers/container.py	llama-server, via podman.
FLMProvider	providers/flm.py	NPU inference (FastFlowLM).
ComfyUIProvider	providers/comfyui.py	Image generation.
KokoroProvider	providers/kokoro.py	TTS.
Qwen3TTSProvider	providers/qwen3tts.py	TTS.
MoonshineProvider	providers/moonshine.py	STT (CPU, ONNX).

Every Provider is stateless — no per-slot mutable state lives on the instance. Abstract methods: build_env(), start_cmd(), health(), infer(), container_spec(). Every slot dispatches through ContainerProvider, which builds a RuntimeLaunchPlan (frozen dataclass: image, command/argv, env, typed mounts with read-only/SELinux flags, devices, cap_add, security_opt, port, network_mode, extra_args, optional health check) fed to a single Quadlet renderer.

SlotConfig.provider (the string field on a slot) is deprecated — every slot runs as a podman container now, and this field is a legacy label round-tripped for backwards compatibility and UI display only. The CLI’s SlotProvider enum (llama-server, flm, moonshine, kokoro) names the inference engine binary that serves the slot — this is distinct from the slot’s hardware backend (vulkan/rocm/cpu).

(B) Remote/cloud upstream provider catalog

This is very likely the concept you mean when you say “provider” in a hal0 config context — the built-in template registry for cloud/remote LLM APIs (hal0.upstreams.integrations). The catalog itself is code, never mutated at runtime; user config lives in /etc/hal0/upstreams.toml.

Catalog id	Base URL	Auth style	Category	Capabilities
openai	https://api.openai.com/v1	bearer	cloud	chat, embed, vision, tools, tts, stt
anthropic	https://api.anthropic.com/v1	anthropic (x-api-key + anthropic-version header)	cloud	chat, vision, tools
openrouter	—	—	cloud	—
google_ai_studio	—	—	cloud	—
deepseek	—	—	cloud	—
minimax	—	—	cloud	—
ollama	http://localhost:11434/v1	none	local	—

Note

anthropic’s catalog entry speaks the native /v1/messages shape only — it needs a proxy for OpenAI-shape compatibility. Auth styles overall: bearer, anthropic, google_query, header, none.

Persisted config is a ProviderEntry per upstream (see Config schema → providers.toml): catalog_id, name/base_url overrides, auth_value_env (the name of an environment variable, never the key itself — the actual secret lives in api.env), enabled, models.

Caution

hal0 upstream CLI commands (upstream create, upstream credentials set, etc.) and upstreams.toml/UpstreamEntry are a separate schema from providers.toml/ ProviderEntry above — both represent “remote provider” concepts but are distinct tables with distinct fields (UpstreamEntry adds kind, warmup_strategy, model_filters; see Config schema). Verify which one a given feature actually reads before documenting behavior.

Profiles

hal0.profiles + ProfileConfig — a profile is a reusable, device-agnostic backend tuning template: a bench-tuned flag bundle plus a few classification hints, not a runtime image reference.

Field	Purpose
flags	Bench-tuned llama-server CLI flags.
mtp	Informational only — MTP is model-owned, not profile-owned (see Model roster & benchmarks).
device_class	gpu | cpu | npu | img | null.
backend	rocm | vulkan | cuda | null.
cloned_from	Provenance — set when a profile was cloned from a seed.
intent	Dashboard card headline.
quant	Display chip only — the runtime reads quant from the model, not the profile.

Caution

On the v1.0 branch (Stream C, fix/profile-tuning-only — reviewed PASS, not yet merged to main at time of writing), device_class/backend are being rewritten to inert match-only fit hints. The slot’s own device field (see the device taxonomy below) becomes the single authoritative hardware fact; the profile hints are retained on-disk only because published .hal0profile.json export artifacts carry them and their checksums cover the field. Verify this has landed on main before treating it as unqualified current behavior.

Profiles carry no image field — image resolution is slot-owned (slot.image_pin or RUNNER_IMAGES[slot.binary]).

ProfileCatalog provides CRUD over /etc/hal0/profiles.toml: list(), resolve(name), create(), update() (via a ProfilePatch), delete() (blocked if the profile is still referenced by a slot or by a model’s defaults.profile). Seed profiles are virtual — reapplied from code on every load, so they can’t be deleted or edited, only cloned.

A resolved profile (ResolvedProfile) adds derived fields: runtime_family (llama-server, flm, kokoro, qwen3tts, moonshine, comfyui — classified structurally off name and device_class, not by sniffing an image string), supported_slot_types (e.g. flm → llm, embedding, transcription; kokoro/qwen3tts → tts; moonshine → transcription; comfyui → image; everything else → llm, embedding, reranking), used_by (slot names referencing the profile), and static tps/rtf numbers from the bench table.

The seeded moonshine profile (device cpu) is the STT sibling of the seeded kokoro profile: runtime_family = "moonshine", supported_slot_types = ("transcription",), image ghcr.io/hal0ai/hal0-toolbox-moonshine:v1 (resolved via RUNNER_IMAGES["moonshine"], digest pinned in manifest.json), and self-managed (operator-staged) weights — see Voice: STT and TTS and ADR-0001.

Device taxonomy

See Hardware matrix for the full table. In short: hal0.model_meta.CANONICAL_DEVICES is the single source of truth for the 5 valid device ids (gpu-rocm, gpu-vulkan, gpu-cuda, cpu, npu), consumed by SlotConfig.device, a profile’s (now inert) device_class, and every other call site that used to duplicate this list.