A cookbook-style walkthrough: classify a ticket, read confidence, and route to auto-queue, confirm, or human — without parsing LLM JSON.
Most “AI support routing” demos stop at a label. Production systems need a second axis: how sure are we?
TypeSafe’s confidence-gated routing pattern is built for that. Here’s a compact cookbook you can adapt. Primitives and patterns: docs.typesafe.ai/patterns/confidence-routing.
Goal
Given an inbound ticket string:
- Ask Jev for department, urgency, and refund_intent
- Auto-route only when confidence clears your bar
- Otherwise escalate — never invent a department string
The questions
from typesafe_sdk import Choice, Noul, TypeSafeClient
client = TypeSafeClient(model="jev-1.13.0") # pin in prod
def classify_ticket(text: str):
return client.system_one(
state=text,
questions={
"department": Choice(
instructions="Which support queue should own this ticket",
criteria={
"billing": "Charges, invoices, subscriptions, refunds as money issues",
"technical": "Bugs, outages, integrations, API failures",
"account": "Login, permissions, profile, access control",
},
),
"urgent": Noul(
instructions="The customer needs attention today because of ongoing business impact",
),
"wants_refund": Noul(
instructions="The customer is requesting money back or a charge reversal",
),
},
)
One call. Three independent judgments. Parallel by design.
The gate (this is the product)
def route(ticket_id: str, text: str):
result = classify_ticket(text)
dept = result.answers["department"]
urgent = result.answers["urgent"].noul
refund = result.answers["wants_refund"].noul
# Choice confidence: act vs review vs escalate
if dept.confidence < 0.55:
return enqueue_human(ticket_id, reason="low_confidence_department")
if dept.confidence < 0.8:
return enqueue_review(ticket_id, suggested=dept.choice, probs=dept.probabilities)
# High confidence path
priority = "p1" if urgent >= 0.85 else "p3"
tags = []
if refund >= 0.7:
tags.append("refund_candidate")
return enqueue_queue(
ticket_id,
queue=dept.choice,
priority=priority,
tags=tags,
audit={
"model": result.model,
"department": dept.choice,
"confidence": dept.confidence,
"urgent": urgent,
"wants_refund": refund,
},
)
Stub enqueue_* with your queue system. What matters is the shape:
- Answer says what
- Confidence says whether to trust the auto path
- Noul thresholds are your policy knobs
Why not one mega Choice?
You could ask “pick an action from {auto_billing_p1, human, …}”. Don’t.
Atomic questions keep each judgment testable. When finance changes refund policy, you tweak refund thresholds — not a tangled action enum. TypeSafe’s build guide leans hard on this: how to build with System One.
Hardening checklist (from real jagged edges)
TypeSafe documents failure modes for jev-1.13 honestly. For a router, watch these:
- Literal instructions. If you mean “business impact,” say that — don’t rely on vibes.
- No math in the model. SLA minutes remaining? Compute in code; ask Jev about language of urgency.
- Trim state. Don’t paste the customer’s entire 40-message history if the last message + three facts suffice.
- Don’t equate Noul and Choice yes/no. They are different instruments; don’t port thresholds blindly (jaggedness).
Stretch goals
Once the router is boring:
- Add a Score for toxicity / abuse before auto-replies fire
- Fan out speculative questions (VIP? churn risk?) in the same call — see speculative fan-out
- Put the same pattern in front of an LLM agent as a guardrail on tool calls (LLM guardrails cookbook)
Fifty lines won’t replace a support org. They will replace the fragile “please reply with JSON only” middleware a surprising number of production systems still ship.
Comments
Loading comments…