Four failure patterns automated metrics don’t catch

What follows are four recurring patterns. Not exotic ones — these are the shapes that come up week after week across code, logic and language tasks. The first two are the worked examples on my front page. The last two are new illustrations built for this post in the same form. None of them are client work: real tasks are under NDA, so every example here is constructed from scratch.

The common structure is worth naming up front: each failure is correct along the axis the metric measures and wrong along an axis nobody is measuring. That is not a flaw in the metric. The mistake is treating one axis as a verdict.

01Calques that survive a meaning check

A formal German HR rejection, translated from English. One response renders "move forward with other candidates" as mit anderen Kandidaten voranzugehen. Every word is correct. The grammar is correct. The meaning is fully preserved — which is exactly why semantic similarity and back-translation both wave it through. It is also a construction no German speaker writes, and a native reader clocks it as machine output in about a second.

The preferred response, dass wir uns für andere Bewerber entschieden haben, is what an HR department actually sends. The difference is not meaning but naturalness and register — a separate axis, correlated with meaning-preservation just closely enough to stay invisible until a model gets good. After that, the remaining errors cluster almost entirely here.

PatternEnglish phrasal verbs and idioms rendered literally into the target language instead of substituted with the natural equivalent. Passes meaning-preservation checks, fails naturalness for native readers, worst in formal register. Naturalness review by native speakers belongs alongside semantic scoring, not downstream of it.

02Claims about code that the code doesn’t support

Two functions returning the first non-repeating character in a string. Both pass every test case in the prompt, so an output-matching grader scores them identically. One of them calls s.count(ch) inside the loop — a full rescan of the string on every iteration, which makes it quadratic — and then states, in prose, "O(n) time."

The interesting part is not the inefficiency. It is that the claim is part of the answer and nothing in the pipeline checks it — test-pass grading treats prose around the code as decoration. The same shape shows up with asserted thread safety, asserted idempotency, and "handles all edge cases" on a function with one unguarded branch. Whenever a model explains its own work, the explanation is a second output needing its own verification, and rarely gets one.

PatternComplexity and property claims stated without accounting for the cost of built-in calls used inside a loop. Passes functional tests while the stated complexity is false. Worth a dedicated probe set, since standard pass/fail grading cannot see it.

03Negative constraints that quietly disappear Illustration

A length check passes both. A similarity score against a reference rewrite rates both highly, and may well rate A higher, because A contains more of the source material. That is the trap: reference-based metrics measure what is present against what was expected to be present. A negative constraint leaves no textual trace when it is honoured. There is nothing to reward, so a metric built on overlap is structurally incapable of scoring it.

They are also the instruction type models drop first under pressure — long contexts, adjacent subject matter, a later turn that reintroduces the topic. "Don’t mention", "no code examples", "avoid the second person": cheap to satisfy, invisible to overlap scoring.

PatternExclusion instructions dropped when the excluded content is topically adjacent to the requested content. Recommend a probe set where each prompt carries exactly one negative constraint plus source material that pulls hard toward violating it, scored by rule rather than by similarity.

04Unsupported specificity in grounded summaries Illustration

Overlap-based faithfulness scoring rates A high, because roughly nine tenths of the sentence is lifted cleanly from the source. Sentence-level entailment checks frequently let it through too: the claim "most firms reported delays" is entailed, and the inserted figure rides along inside a sentence that is otherwise well supported. Human review catches it by doing something dull — asking of every number, name and date: which span of the source is this from?

This one is worth flagging harder than its size suggests, because the failure is anti-correlated with how the output reads. A invented a number and, to almost any reader and most reward models, looks like the better summary. Preference data collected without a source-checking step will reliably teach a model to do more of it.

PatternHedged quantifiers in source material ("a majority", "several", "most") replaced with invented precise figures. Passes overlap-based faithfulness and often sentence-level entailment. Recommend span-level attribution for every numeral in grounded-generation evaluation.

What this means for how a pipeline is set up

None of this is an argument against automated evaluation — putting 3,000 outputs in front of a human with no pre-filter is an expensive way to confirm that most of them are fine. It is an argument about where the automatic checks sit in the chain.

1. Metrics route, humans decide. An automatic score is a good reason to look at something first, and a bad reason to stop looking.
2. Name the axis each metric covers, then staff the rest. Meaning preservation and naturalness are two axes. Functional correctness and self-description are two axes. Content overlap and instruction adherence are two axes. Every gap is a place where a model can be confidently wrong.
3. Turn recurring failures into probe sets. The most useful artefact I produce is not a score, it is a set of prompts engineered to pass the automatic check and fail a human read. That converts a pattern one reviewer noticed into something the pipeline can track over time.
4. Treat reviewer disagreement as a guideline bug. When two careful auditors split on the same pair, the usual cause is a rubric that never said which axis wins. Fixing the guideline is cheaper than re-adjudicating every batch, and it is the change that compounds.

The through-line in all four patterns is that models fail in the direction of looking correct. That is not a coincidence — it is what optimising against a measurable proxy produces. Which is the entire case for having someone read the thing line by line.

— S.M.S.