What a Decision Engine Is — and Why “Reply in JSON” Is Not One
Most teams that need a model to make a call in production start the same way: they wrap a chat model in a prompt that says respond only with valid JSON, parse the output, and ship it. It survives staging. Then it meets real traffic — a slightly different phrasing in a ticket, an image with odd lighting, a truncated response, a hallucinated key — and suddenly your routing logic is running behind a try/catch that fires more often than anyone wants to admit. The team ends up building a validation layer, a retry layer, a fallback heuristic, and a small dashboard to watch all of it drift.
A decision engine inverts that contract. Instead of asking a model to generate text that happens to look like an answer, you send state — text, JSON, or up to 16 images or video frames — together with questions you have already defined in code, and you get back a schema-valid, typed answer in a single pass. There is no prose to interpret with a regex, no token stream to buffer, no second call to coerce the output into the shape your application needs.
The difference is architectural, and it comes down to three properties. Shape: answers arrive as one of a small set of primitives — a choice from your own labels, a score on an ordered scale, a truth expressed as a calibrated probability between 0.00 and 1.00, a number on a defined grid, or a point or box locating something inside an image. Confidence: every answer carries a calibrated confidence value, so code can act instantly above a threshold and re-check or take a safe path below it. Determinism: the same request returns the same answer, which means decision logic becomes something you can unit test, replay from logs, and diff between model versions.
That last property is the one teams underestimate. A decision layer that is not reproducible is not really a decision layer — it is a suggestion layer that happens to be called by code. When you cannot replay the exact input that produced a verdict, incident review becomes archaeology. When you can, a regression test for a production misroute is three lines long.
The Output Contract: Typed Answers, Calibrated Confidence, and Replayable Verdicts
The interface is deliberately small. You post state plus questions to one HTTPS endpoint — DecisionNode exposes this as POST /v1/decide — and receive answers that already conform to the schema you declared. Because the questions live in your codebase rather than in a prompt string, changing what you ask is a pull request, not a prompt-engineering session with unclear diffs.
Calibration is what makes thresholds meaningful. A raw model percentage tells you how confident the model sounds; a calibrated probability tells you what fraction of answers at that level have historically been correct. That is the number you can build policy on: auto-approve above 0.95, queue for human review between 0.70 and 0.95, and hand to a deterministic fallback below that. Ops teams can reason about those bands the same way they reason about any other SLO.
Latency and cost are the other half of the contract. DecisionNode ships two models: DecisionNode Flash, at roughly 5 ms server-side and 19 ms end to end, priced at $0.035 per 1M input tokens, and DecisionNode MoE, at roughly 9 ms server-side, priced at $0.042 per 1M input tokens. Output tokens are always free, there is a free daily tier, and batch discounts are available for offline scoring jobs. For loops that exceed ten decisions per second — drones, robots, simulators, machine monitoring — sessions run over WebSocket rather than request/response. Rather than assemble that infrastructure yourself, most teams adopt a decision engine that already enforces the contract end to end.
Client surface matters more than it sounds. SDKs for TypeScript, Python, Go, and Ruby, plus curl, mean the decision call slots into existing services without a new language in the stack. MCP tools and webhooks cover the agent and event-driven cases: an AI agent can call decide before acting, receive a typed verdict with confidence, and branch without hallucinating its own guardrails. Optional boosters — human age estimation, AI-generated image detection, face detection and recognition, OCR, depth and segmentation — can be layered onto the same request when a decision depends on something specific inside an image rather than the image as a whole.
Where Decision Engines Pay Off: Triage, Risk, Gating, and the Physical World
The pattern that justifies the investment is usually a high-volume branch that is currently either hard-coded rules or a slow, inconsistent model call. Ticket routing and triage is the canonical case: a support pipeline receives free-text plus screenshots, and needs a category, a priority score, and a confidence. Rules break on paraphrase; chat models produce plausible categories that occasionally do not exist in the taxonomy. A typed choice constrained to your label set, with a confidence you can threshold, removes both failure modes without removing the model.
Risk and abuse decisions follow the same shape. Fraud and promo-abuse checks need a calibrated probability more than they need an explanation, because the probability is what feeds the rule engine downstream. Order and shipping decisions — split shipments, address confidence, carrier selection — are scoring problems where determinism matters, since two identical orders should not receive different treatment depending on when they were evaluated. Moderation, scoring, and gating all reduce to the same three questions: which label, how confident, and what should happen at the edge of the threshold.
The physical and operational cases are where latency stops being a nice-to-have. Quality control on production lines needs a verdict inside the cycle time of the equipment; a 200 ms round trip means inspecting a sample instead of the whole line. Warehouse and traffic monitoring need box and point outputs over video frames at a rate that request/response cannot sustain, which is exactly the case WebSocket sessions are built for. Machine monitoring and simulation environments apply the same loop: observe state, ask a question, act on a typed answer, repeat thousands of times per minute.
Consider an illustrative migration: a logistics team gating carrier upgrades on damage-risk and address-confidence signals. Today those signals come from a hand-written rule tree that has accumulated 400 branches, none of which anyone can safely delete. Replacing the tree with two typed questions — a score on an ordered risk scale and a truth probability for address deliverability — collapses the logic into a threshold policy that fits on a page. The rule tree becomes a documented set of bands instead of tribal knowledge, and the team can replay last quarter’s traffic through the new engine to compare verdicts before flipping a single production flag. That replay is the real deliverable: not a smarter model, but a decision layer that ops, product, and engineering can all read, test, and change deliberately.
