Part 10 — Integrated Case Studies

Constrained decoding

Diagnosis and correction·Evaluation·6 min read

The case study's cleanest single result: constrained decoding drove structural failures to zero across all six systems, so 100% of outputs were valid — and yet accuracy ranged from 45.5% to 95.5%. This is the book's central thesis, proven in one experiment.

The case study contains the clearest possible demonstration of the book's central thesis, and it's worth its own chapter because it's so decisive. By constraining the model's decoding to the QueryIntent schema, the project drove structural failures to zero: every one of the six systems produced 100% schema-valid output. And in the same experiment, semantic accuracy ranged from 45.5% to 95.5%. Structure completely solved; meaning wide open. That single pair of facts is valid ≠ correct, proven on real systems and real data.

What you will understand by the end

  • What constrained decoding did in the case study, and why it worked so completely.
  • Why 100% validity across all six systems is the structural boundary solved.
  • Why 45.5%–95.5% accuracy on top of it is the thesis in one experiment.
  • What this cleanly separates for the rest of the case study.

Constrained decoding solved the structural boundary

The systems used constrained decoding — restricting the model to only emit tokens that keep the output valid against the QueryIntent schema. The effect was total: no parse failures, no missing fields, no illegal pattern values, across every system and every one of the 88 tasks. The structural failure modes — malformed output, schema violations — simply didn't occur, because the decoding made them unrepresentable.

Key idea

Constrained decoding drove the case study's structural failures to zero: 100% schema-valid output across all six systems. The syntax boundary was completely and reliably solved by construction — which is exactly what constrained decoding is for, and it took an entire class of failures off the table so only meaning remained.

100% valid, 45.5%–95.5% correct

Here is the result the whole book points at. Every system: 100% valid. The systems' semantic accuracy: from 45.5% (Qwen 3B, raw) to 95.5% (the best hosted systems). Two facts, one experiment:

   structural validity:   ████████████████████ 100%   (all six systems)
   semantic accuracy:     █████████░░░░░░░░░░░░  45.5% ── to ── 95.5%  ████████████████████░
                          └── same outputs, same schema, wildly different correctness ──┘

The validity number told you nothing about which system to trust: two systems with identical 100% validity were 50 points apart on meaning. If you had shipped a "100% schema-compliant" dashboard, you'd have had no idea whether your answers were 45% or 95% right.

Watch out

This is the trap in its purest form: a green "100% valid" metric that is completely uncorrelated with correctness. In this experiment the two coexisted — perfect validity, 45.5% accuracy — on the same system. Constrained decoding is genuinely valuable (it removed a whole failure class), but reading its 100% as a quality signal would have hidden a system that got the answer wrong more than half the time. Validity is a syntax health check, never a correctness check.

What this cleanly separates

The completeness of the structural solution is what makes the rest of the case study clean. Because every remaining failure is semantic — a valid intent with the wrong meaning — the 45.5%–95.5% spread is a pure meaning story, with no structural noise mixed in. The diagnosis could focus entirely on the ranking-vs-breakdown boundary; the comparison is a comparison of semantic quality; the guardrail targets a meaning error. Solving the structural boundary completely is what let the experiment be, cleanly, about correctness.

Observed evidence

The dashboard shows both numbers side by side: uniform 100% validity, and an accuracy column spanning 45.5% to 95.5%. That's the entire thesis of the evaluation half of the book, sitting in one leaderboard — structure is solved, meaning is the open question, and only measuring meaning separates a trustworthy system from an untrustworthy one. Read the structured-output-semantics analysis →

Mental model

Constrained decoding drove the case study's structural failures to zero — 100% valid across all six systems — while accuracy ranged 45.5%–95.5%. Perfect validity, uncorrelated with correctness, on the same outputs: valid ≠ correct, proven. Solving the structural boundary completely is both genuinely useful and exactly why the rest of the experiment is a clean story about meaning.

Common mistakes

  • Reading 100% validity as quality. In this experiment it coexisted with 45.5% accuracy on the same system — completely uncorrelated.
  • Undervaluing constrained decoding. It removed a whole failure class; the point is it solves structure, not that it's useless.
  • Mixing structural and semantic failures. Because structure was fully solved here, all remaining failures are semantic — keep them separate.
  • Shipping a validity dashboard as a quality dashboard. It would have hidden a system wrong more than half the time.

Practical guidance

  • Use constrained decoding to solve the structural boundary — it reliably takes malformed and out-of-schema output off the table.
  • Treat its 100% validity as a syntax-health signal, and always report a separate semantic accuracy beside it.
  • Once structure is solved, focus diagnosis and comparison entirely on meaning — that's where the remaining failures live.
  • Remember the numbers: 100% valid and 45.5% correct coexisted — the definitive reason to measure meaning.

Summary

  • Constrained decoding drove the case study's structural failures to zero — 100% valid across all six systems.
  • On top of that, semantic accuracy ranged 45.5%–95.5% — validity told you nothing about which system to trust.
  • The two coexisting is valid ≠ correct proven in one experiment — the evaluation half's thesis.
  • Solving structure completely is what makes the rest of the case study a clean story about meaning.

Knowledge check

All six systems in the case study hit 100% schema validity. A stakeholder concludes the systems are equivalently good. Using the accuracy range, explain the error.

The 100% validity is a structural result — every output is a well-formed, schema-legal QueryIntent — and it's completely uncorrelated with correctness: in this very experiment, semantic accuracy ranged from 45.5% to 95.5%, so two systems with identical 100% validity were 50 points apart on whether their answers actually meant the right thing. Validity measures whether the output parses and fits the schema, not whether it chose the right pattern and filters for the question. Concluding the systems are equivalent from equal validity mistakes a syntax check for a quality check — the exact trap the case study proves, since one of those "valid" systems is right less than half the time.

Given constrained decoding "only" solves structure and does nothing for meaning, was it worth using in the case study?

Yes — solving structure completely is genuinely valuable and it's what makes the rest of the experiment clean. By driving structural failures to zero, constrained decoding removed an entire class of failures (malformed output, missing fields, illegal patterns) so they couldn't confound the results, which means every remaining failure is semantic — a valid intent with the wrong meaning. That let the diagnosis, comparison, and guardrail all focus purely on correctness with no structural noise mixed in. The lesson isn't that constrained decoding is pointless; it's that its 100% validity must never be read as a quality signal — it solves the syntax boundary, and you measure meaning separately.

Related chapters