Skip to main content

Developer Reference

VerdictModels Direct API & Certified Local Runtime Guide

https://api.verdictmodels.com/v1 is a stateless OpenAI-compatible Direct API gateway dedicated exclusively to first-party vc/verdict/* models.

1. Authentication & Mandatory llm:chat Scope

Authenticate with an existing Verdict Cloud live API key (Authorization: Bearer vrd_live_...) created at verdictide.com/en/account/api-keys.

Important Scope Rule: Newly minted Verdict keys default to scopes = []. When creating a key for POST /v1/chat/completions, explicitly enable the literal "llm:chat" scope. Keys with empty scopes, wildcard-only scopes lacking "llm:chat", or vrd_test_* keys are rejected with HTTP 403 before dispatch.

2. Canonical 3-Segment Model Naming

Every POST /v1/chat/completions request must pass the full 3-segment vc/verdict/<slug> identifier:

  • vc/verdict/verdict-4-35b-toolterse-v5 — Flagship 35B-A3B MoE coder & tool caller.
  • vc/verdict/verdict-4-30b-terse — 30B-A3B Softmax MoE terse coder (repeat_penalty=1.1).
  • vc/verdict/verdict-4-4b-quill — Compact 4B dense model for rapid bounded edits.
  • Bare slugs (verdict-4-35b-toolterse-v5) or 2-segment strings (verdict/verdict-4-35b-toolterse-v5) return HTTP 400 invalid_model_id with a remediation hint. Non-Verdict third-party models return HTTP 404 model_not_available_on_verdictmodels (use https://api.verdictide.com/v1 for third-party vendors).

3. SDK, Aider, OpenCode & Tool-Use Integration

Sample promptUses standard openai SDK with Idempotency-Key, 160s cold-start safe timeout, and streaming usage.
import os, uuid
from openai import OpenAI

client = OpenAI(
    base_url="https://api.verdictmodels.com/v1",
    api_key=os.environ["VERDICT_API_KEY"],  # vrd_live_* with literal "llm:chat" scope
    timeout=160.0,
)

stream = client.chat.completions.create(
    model="vc/verdict/verdict-4-35b-toolterse-v5",
    temperature=0.2,
    stream=True,
    stream_options={"include_usage": True},
    extra_headers={"Idempotency-Key": str(uuid.uuid4())},
    messages=[
        {"role": "user", "content": "Write a concise Python binary_search(nums, target) function."}
    ],
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

4. Scale-to-Zero Timeouts, 32K Slot Limit & Idempotency Contract

Cold-Start & Generation Deadlines

First-party GPU workers scale to zero when idle. FlashBoot resume takes 8–28s; full cold worker allocation takes 35–67s. Configure SDK timeouts to >= 160s. The gateway enforces a 155s first-meaningful-output budget (keepalive comments and role-only deltas do not mask a stalled cold boot) and a 330s total wall ceiling.

32,768-Token Per-Slot Context

Cloud Option B workers enforce 32,768 tokens per request slot (prompt + completion). Requests exceeding 32K return HTTP 400 exceed_context_size_error without debiting credits. Request bodies are capped at 1 MiB UTF-8 JSON.

Scoped Idempotency-Key & 564d4c Request IDs

Supply a 1–128 printable-ASCII Idempotency-Key on POST /v1/chat/completions. VerdictModels derives a deterministic 564d4c-prefixed RFC 9562 UUIDv8 scoped to (user_id, billing_account_id, key_id, Idempotency-Key) and returns it in X-Verdict-Request-Id:

  • Warm replay (<= 15 min, <= 64 KiB): Identical retries return Idempotent-Replayed: true with zero duplicate wallet debit. Reusing the same key with a different JSON body returns HTTP 409 idempotency_conflict.
  • Post-eviction / cold-instance matrix: Non-streaming → non-streaming replays from upstream RequestState.receipt_json without re-billing. Streaming → non-streaming returns HTTP 409 idempotency_original_was_stream_metadata_only (upstream streaming receipts store metadata/usage only, never choices). Any post-eviction streaming retry returns HTTP 502 upstream_stream_closed_outcome_unknown pre-header (or terminal SSE event: error without [DONE] if keepalive already flushed) with zero duplicate charge.
  • Unknown settlement warning: If a connection drops mid-flight before terminal frames, settlement state is unknown. Public usage endpoints (/en/account/billing) show aggregate ledger totals rather than a per-request terminal state, so absence of a visible charge is inconclusive; minting a fresh Idempotency-Key may incur a new charge.

5. Local Engine Presets (LM Studio & llama.cpp)

Always apply the certified VerdictModels sampler presets when running local GGUF or MLX weights:

  • LM Studio (model.yaml): LM Studio ignores GGUF-embedded sampler metadata and defaults to temperature=0.8, repeatPenalty=1.1. Place the downloaded model.yaml inside ~/.lmstudio/hub/models/verdict/<slug>/model.yaml (never inside ~/Models/verdict/).
  • 4B Quill Flash Attention (VX-2340): Always pass --flash-attn / llm.load.llama.flashAttention: true on verdict-4-4b-quill (reduces 30K-token CPU RSS from 18.64 GiB to 9.42 GiB).
  • 30B Terse vs. 35B/Quill Repeat Penalty: Use repeat_penalty=1.1 (penalty_last_n=64) on verdict-4-30b-terse GGUF, and strict repeat_penalty=1.0 on verdict-4-35b-toolterse-v5, 4b-quill, 1500m-quill, and 500m-quill.
  • Unsupported Runtimes: ollama is not currently certified for Verdict-4 GGUF releases (no Modelfile preset is shipped in v1).