Prompt · Support
Ticket classification prompt
Routing inbound messages to the right queue at volume, where the cost of a wrong label is a customer waiting in the wrong place for two days.
Classify the support message below into exactly one category.
<message>
{{message}}
</message>
Categories:
{{categories}}
Rules:
- Choose the category matching the customer's primary problem, not the first topic mentioned.
- If two categories fit, choose the one the customer would consider more urgent.
- If none fit, or the message is too vague to place, return "unclear". Do not force a fit.
- Base the decision only on the message. Do not use assumptions about typical customers.
Return JSON:
{"category": "<one of the listed values, or unclear>", "confidence": "high" | "low", "signal": "<the phrase from the message that decided it>"}
Return "low" confidence whenever you had to choose between two plausible categories.What to fill in
{{message}}- The ticket body. Include the subject line if you have it — it carries a lot of the signal.
{{categories}}- One per line, each with a one-line definition of what belongs in it. Definitions matter more than names: "Billing" means nothing until you say whether a failed payment is billing or technical.
Why it is written this way
Every rule in the prompt is there because of a specific failure it prevents. Knowing which is which is what lets you adapt it instead of only pasting it.
Unclear is a category
Every classifier without an escape hatch achieves 100% coverage by mislabelling the cases it cannot place. Those are exactly the cases a human should see. An "unclear" bucket that receives 5% of traffic is a working system; one that receives 0% is a system hiding its errors.
The tie-break is written down
Real messages fit two categories often. Without a stated rule the model picks inconsistently between runs, which shows up as a queue that behaves differently on Tuesday than on Monday for no reason anyone can find.
It returns the deciding phrase
The `signal` field costs a handful of tokens and turns every misclassification into a five-second diagnosis. Without it, debugging a bad label means re-reading the whole ticket and guessing.
Confidence is defined, not felt
Rather than asking for a number the model has no calibrated basis for, this ties "low" to a specific condition: two plausible categories. That is something the model can actually observe about its own decision.
Primary problem, not first topic
Customers open with context and get to the problem in paragraph two. Without this line, classifiers label on the opening sentence.
The version most people write, and what it costs
Classify this ticket as Billing, Technical, or Account:
{{message}}Everything gets a label, including the messages that fit none of the three. A password reset request that mentions a charge becomes Billing. Nothing in the output indicates the decision was close, so the mislabelled tickets are invisible until a customer complains that nobody answered.
What it still gets wrong
- Category definitions do more work than the model does. A classifier disagreeing with your team usually means the team disagrees with itself about where the boundary is.
- It classifies the message, not the customer. History, plan tier and previous tickets often change the right routing, and none of that is here.
- Confidence labels are heuristic. Treat "low" as a routing signal to human review, not as a probability.
Check it before you ship it
- Evaluation Dataset BuilderCases with expected outputs and difficulty, exported three ways.
- Inter-Rater Agreement Calculator (Cohen's Kappa)Cohen's kappa with disagreement list. Below 0.6, fix the rubric first.
- LLM API Cost CalculatorRequests per month in, dollars out. Input, cached input and output priced separately.
To build one of these from scratch for a task not covered here, the prompt generator assembles the same structure — delimiters, output contract, edge cases — from an expert-authored blueprint, and the prompt review checklist is what to run over the result.