Small chat models break structured JSON replies in predictable ways

Quirks and fixes

• Qwen flattens nested objects into the top level. Asked for {"text": "...", "tool_define": {"name","code",...}} it returns {"type":"tool_define","name":...,"code":...} with no text. Asked for "remember": {"key","text"} it returns top-level key and text. A tool call comes back as {"type":"tool","name","input"}. Fix: normalise the known flattened shapes into the nested field before validating, and treat a reply containing only an action field as an action rather than empty text. Before the fix the symptom was frequent "Model returned empty text" errors that were really dropped tool builds.
• Log the raw reply on validation failure. The error text alone ("empty text") hid the cause for hours. Logging the first 600 characters of the parsed reply and capturing wrangler tail --format json found it in one capture.
• gpt-5-mini rejects reasoning: {enabled: false}. reasoning: {effort: 'minimal'} works for OpenAI reasoning models. Gemini 2.5/3.x Pro and Claude Sonnet 5.5 on OpenRouter return HTTP 400 "Reasoning is mandatory for this endpoint and cannot be disabled"; {effort: 'low'} works for them. A single test call per new model id is cheap insurance.
• Code fields arrive as whole functions. Asked for a function body, both models often send function f(s) {...} or s => .... Running it as a body first and, if that returns nothing or fails to parse and the text looks like a function expression, retrying as return (CODE)(input); recovers these.
• Word limits are ignored without a retry. gpt-5-mini overran a 14-word limit on most lines until one rewrite retry ("at most N-4 words") was added.
• Low max_tokens silently drops turns. A 300-token cap cut JSON replies mid-string, JSON.parse failed, and the turn vanished. Raising the cap costs nothing extra (billing is per token used); telling the model on the next turn that its reply was cut helps.
• Narrated instead of performed. Models write "I counted with the tool" without filling the tool field. Detecting such claims in the text with no matching field and adding a corrective note on the next turn works. Prompt examples that show narration ("counts the lines") make the behaviour worse.
• Mid-prompt notes get ignored. An event described in the middle of a long prompt was ignored. Placing it as the final "PRIORITY THIS TURN" line made the model act on it immediately.