Playbook: llm-verifier & TurboAgent — Use Cases & Implementation

Companion document to blog post. Focus: how to actually use this. 5 use cases, mỗi cái có setup steps, code skeleton, gotchas, và decision criteria.

---

Quick reference

Use case	API chính	Backend min	Effort
1. Best-of-N selection	llm_verifier.select()	DeepSeek or Vertex	30 min
2. Pairwise labeling (RLHF)	llm_verifier.compare()	DeepSeek or Vertex	1 hour
3. Progress tracking	llm_verifier.track() / ProgressTracker	DeepSeek or Vertex	30 min
4. Chat client plugin (Claude Code)	turbo-agent proxy	DeepSeek + Anthropic	15 min
5. Multimodal verification	images= param	Gemini Vertex	30 min

---

Use case 1: Best-of-N selection

Khi nào dùng: Bạn đã có N candidates (từ N prompts, N seeds, hoặc N models) và cần pick winner.

Setup

pip install llm-verifier

.env:

DEEPSEEK_API_KEY=sk-...    # hoặc VERTEX_API_KEY cho Gemini

Minimal code

import llm_verifier

problem = "Write a function that reverses a string."
candidates = [
    "def rev(s): return s[::-1]",
    "def rev(s): return s",
    "def rev(s): return ''.join(sorted(s))",
]

result = llm_verifier.select(
    problem=problem,
    candidates=candidates,
    criteria={"Correctness": "Does the code actually reverse the string?"},
)

print(result.index)   # 0
print(result.scores)  # [0.73104, 0.38446, 0.38449]
print(result.ranking) # [0, 2, 1] sorted by score

Tuning parameters

result = llm_verifier.select(
    problem=problem,
    candidates=candidates,
    criteria=criteria,
    model="deepseek/deepseek-v4-flash",  # verifier model
    n_evaluations=4,    # repeated evaluations per criterion (higher = more stable)
    pivots=2,           # pivot count in tournament (higher = more accurate, more cost)
    seed=42,            # reproducibility
)

Trade-offs:

• n_evaluations: mỗi criterion được đánh giá N lần, lấy average. 4 là default paper. Tăng nếu scores nhiễu.
• pivots: PPT dùng k pivots thay vì full round-robin. pivots=2 = O(N×2) comparisons thay vì O(N²). Trade cost vs accuracy.
• criteria: dict {name: question}. Multiple criteria = multiple reward signals được averaged.

Decision criteria: Khi nào KHÔNG dùng

• N = 1: chỉ có 1 candidate → không có gì để select.
• Candidates gần identical (cùng prompt, cùng seed) → verifier không distinguish được.
• Task có ground truth rõ ràng (unit test, regex match) → dùng rule-based, đỡ tốn cost.

---

Use case 2: Pairwise labeling cho RLHF/RLAIF

Khi nào dùng: Bạn cần preference data (prompt, response_a, response_b, preferred) cho RL training.

Setup

pip install llm-verifier

Code

import llm_verifier

problem = "Explain monads to a beginner."
response_a = "A monad is a monoid in the category of endofunctors..."
response_b = "Think of a monad like a container that wraps values..."

reward_a, reward_b = llm_verifier.compare(
    problem,
    response_a,
    response_b,
    criteria={
        "Clarity": "Is the explanation clear to a beginner?",
        "Accuracy": "Is the explanation technically correct?",
    },
)

# reward_a, reward_b ∈ [0, 1]
# Use as preference signal: P(A > B) = sigmoid(reward_a - reward_b)

Scale up: Batch labeling pipeline

import json
from pathlib import Path

pairs = load_preference_pairs("data/pairs.jsonl")  # [{prompt, a, b}, ...]

labels = []
for pair in pairs:
    r_a, r_b = llm_verifier.compare(
        pair["prompt"], pair["a"], pair["b"],
        criteria={"Overall": "Overall response quality"},
    )
    labels.append({
        "prompt": pair["prompt"],
        "chosen": "a" if r_a > r_b else "b",
        "margin": abs(r_a - r_b),  # confidence signal
        "reward_a": r_a,
        "reward_b": r_b,
    })

Path("data/labels.jsonl").write_text("\n".join(json.dumps(l) for l in labels))

Decision criteria

Dùng khi:

• Bạn đang build RLHF pipeline và cần scale labeling beyond human capacity
• Bạn cần continuous reward (training loss) thay vì discrete label
• Quality bar quan trọng hơn labeling cost

Không dùng khi:

• Chỉ có vài trăm samples — human labeling vẫn faster, more reliable
• Task domain quá specialized mà verifier model không familiar (medical, legal edge cases)

---

Use case 3: Progress tracking cho long-running agents

Khi nào dùng: Agent chạy task phức tạp N steps, bạn muốn biết khi nào nên abort sớm.

Offline tracking (trajectory đã có)

import llm_verifier

steps = [
    'Read the problem statement',
    'Wrote def rev(s): return s',
    'Tested: rev("abc") returned "abc"',
    'Changed to def rev(s): return s[::-1]',
    'Tested: rev("abc") returned "cba"',
]

result = llm_verifier.track(
    problem="Write a function that reverses a string.",
    steps=steps,
    checkpoint_steps=[1, 2, 3, 4, 5],  # score sau mỗi step
    n_evaluations=4,
)

print(result.scores)
# [0.00106, 0.02417, 0.03143, 0.62004, 0.99978]

Online tracking (agent đang chạy)

import llm_verifier

tracker = llm_verifier.ProgressTracker(
    problem=problem,
    n_evaluations=4,
    abort_threshold=0.05,  # tùy chọn: auto-abort nếu score quá thấp
    patience=3,            # cho phép 3 steps dưới threshold trước khi abort
)

for step_text in agent.run(task):
    score = tracker.update(step_text)
    
    print(f"Score after step: {score:.4f}")
    
    if score < 0.05 and consecutive_low >= patience:
        agent.abort()
        break

Gotcha: Multi-turn trajectory

ProgressTracker tích lũy steps. Mỗi update() thêm step mới vào history, verifier thấy full trajectory so far.

# Score sau step 5 dựa trên steps 1-5
# Score sau step 10 dựa trên steps 1-10
# Verifier KHÔNG peek future

Decision criteria

Dùng khi:

• Agent tasks > 10 steps thường xuyên
• Bạn observe wasted compute trên hopeless rollouts
• Bạn cần signal để decide “continue vs resample”

Không dùng khi:

• Tasks ngắn (< 5 steps) — overhead > savings
• Tasks luôn succeed hoặc luôn fail (deterministic) — không cần signal
• Không có logprob backend sẵn — setup cost cao

---

Use case 4: TurboAgent — drop-in proxy cho Claude Code

Khi nào dùng: Bạn muốn mọi Claude Code turn được verified tự động, không cần modify skill files.

Setup

pip install turbo-agent

turbo-agent.yaml:

backend:
  models:
    - name: anthropic/claude-opus-4-5
      api_key: $ANTHROPIC_API_KEY
      num_candidates: 3    # mỗi turn generate 3 candidates

verifier:
  model:
    name: deepseek/deepseek-v4-flash
    api_key: $DEEPSEEK_API_KEY

.env:

ANTHROPIC_API_KEY=sk-ant-...
DEEPSEEK_API_KEY=sk-...

Run

turbo-agent  # starts proxy on port 8888

Terminal khác:

ANTHROPIC_BASE_URL=http://localhost:8888 claude

Visualizer

Mở http://localhost:8888/visualizer — xem DAG pipeline cho mỗi request: context refinement, N candidates, tournament comparisons, final selection.

Gotchas

Anthropic không thể làm verifier. Config này raise error:

verifier:
  model:
    name: anthropic/claude-opus-4-5  # ❌ raises: no logprobs

Latency thay đổi. Verifier mode: response đến 1 burst (sau tournament), không streaming token-by-token. Tool calls vẫn work, chỉ là latency.

Candidates = N × cost. Mỗi turn cost N lần backend inference. Trade-off: quality vs cost.

Decision criteria

Dùng khi:

• Bạn đang dùng Claude Code cho high-stakes work (production code, security patches)
• Bạn chấp nhận latency cao hơn để có quality cao hơn
• Bạn có DeepSeek key sẵn

Không dùng khi:

• Interactive flow cần response nhanh
• Cost là concern chính
• Tasks đơn giản (lint, autocomplete)

---

Use case 5: Multimodal verification

Khi nào dùng: Verify outputs có images (UI screenshots, diagrams, robot camera frames).

Setup

pip install llm-verifier
# Cần multimodal model: Gemini via Vertex AI, hoặc vLLM serving Qwen-VL

Code

import llm_verifier

result = llm_verifier.select(
    problem="Does this UI match the design spec?",
    candidates=["variant_a.png", "variant_b.png", "variant_c.png"],
    criteria={"Visual fidelity": "Matches the design spec?"},
    images="design_spec.png",  # context image
)

# Per-step frames cho tracking
tracker = llm_verifier.ProgressTracker(problem)
for step in robot_episode:
    score = tracker.update(step.description, images=step.camera_frame)

images accept:

• File path: "frame.png"
• URL: "https://..."
• Raw bytes: open("frame.png", "rb").read()
• List: ["a.png", "b.png"]

Per-step images accumulate trong trajectory. Verifier thấy full visual history.

Decision criteria

Dùng khi:

• UI generation tasks (multi-model comparison)
• Robot/embodied AI rollouts (camera frames)
• Design tasks (visual quality subjective)

Không dùng khi:

• Text-only tasks — wasted multimodal capability
• Image quality không phải concern chính

---

Backend selection guide

Backend	Logprobs	Cost	Setup	Match paper
deepseek/deepseek-v4-flash	✅	$	API key only	Self-verification TB 2.1
Gemini via Vertex AI	✅	$$	GCP project, ADC	TB V2, SWE-Bench, MedAgentBench
vLLM/SGLang (self-hosted)	✅	Infra	GPU + serve	Custom
Anthropic	❌	—	—	N/A as verifier
OpenAI GPT-4	❌ (binary logprobs only, không đủ granularity)	—	—	N/A

Recommendation: Bắt đầu với DeepSeek. Cheapest, simplest, matches self-verification numbers. Scale lên Vertex AI nếu cần numbers match paper benchmarks.

---

Common patterns

Pattern 1: Custom criteria cho domain

criteria/ folder trong repo có templates cho Terminal-Bench, SWE-Bench, MedAgentBench. Để customize:

criteria = {
    "Correctness": "Does the code produce the right output for all valid inputs?",
    "Edge cases": "Does it handle empty input, null, boundary values?",
    "Style": "Does it follow the project's code style conventions?",
    "Performance": "Is it efficient for the expected input size?",
}

result = llm_verifier.select(
    problem=problem,
    candidates=candidates,
    criteria=criteria,
)

Mỗi criterion là 1 reward signal. Average over criteria = final score.

Pattern 2: Reproducibility

result = llm_verifier.select(
    problem=problem,
    candidates=candidates,
    criteria=criteria,
    seed=42,  # PPT tournament seed
    n_evaluations=4,  # deterministic per-pair given model + seed
)

Note: full determinism requires backend to be deterministic. DeepSeek và Vertex Gemini support seed param.

Pattern 3: Token accounting

import llm_verifier

llm_verifier.USAGE.reset()
result = llm_verifier.select(problem, trajectories, criteria="terminal_bench")
print(llm_verifier.token_usage())
# {'calls': 24, 'input_tokens': 1512480, 'cached_input_tokens': 1190208,
#  'uncached_input_tokens': 322272, 'output_tokens': 180224,
#  'reasoning_tokens': 145408, 'cache_hit_rate': 0.787}

Dùng để budget verifier calls, measure cache hit rate.

---

Checklist khi triển khai

• Backend logprob verified: turbo-agent check hoặc test 1 call manual
• .env configured với đúng keys
• Criteria drafted (ít nhất 2-3 criteria cho mỗi task domain)
• Pilot test trên 5-10 samples trước khi scale
• Token budget estimated dựa trên pilot
• Cache hit rate measured (target > 70%)
• Failure mode handled: nếu verifier backend down, fallback là gì?

---

Troubleshooting

Verifier scores gần nhau (0.4 vs 0.5): Tăng n_evaluations hoặc refine criteria cụ thể hơn.

Verifier score ngẫu nhiên không ổn định: Set seed, dùng temperature=0 cho verifier.

Cost quá cao: Giảm pivots, giảm n_evaluations, hoặc dùng prefix-cache optimization (criterion ở tail).

Verifier hallucinate scores: Cross-check với rule-based test nếu có. Verifier không thay thế ground truth — nó approximate.

Anthropic không làm verifier được: Đây là constraint cứng. Dùng DeepSeek/Vertex/self-hosted.

---

Khi nào KHÔNG dùng llm-verifier

• Tasks có ground truth rõ ràng (unit test pass/fail) → dùng test, không cần verifier
• Single candidate → không có gì để select
• Real-time interactive flows → latency overhead không acceptable
• Domain quá specialized mà verifier model không biết (highly technical, niche)
• Output quality không phải concern chính (lint suggestions, simple edits)

llm-verifier giải quyết một class bài toán cụ thể: khi bạn cần verifiable quality mà không có cách nào khác để measure. Không phải general-purpose tool.