Why ranking, and not simply a better search

Two practical routes were run over the same English passages. A simple comparison-word search is precise but misses most relevant expressions. The two-model workflow searches more broadly, then orders what people inspect. Both analyses were added by this project; neither was part of the original #ReframeCovid dataset release.

How candidates are foundcandidatesworth keepingstrongknown-good items found
Look for comparison words
"feels like", "as if", "like a" — a regular expression
28624.5%313 of 17
Broad proposal, then second-model verification
one model proposes; another checks each phrase
1,66524.6%12317 of 17

Candidate recovery only: 17 of 17 means that every known-good item produced a candidate somewhere in the pool. Four Menu entries named in the Menu-likeness prompt were excluded from all ranking-based reference results, so this does not show that all 17 ranked highly.

Simple but narrow

Searching for comparison words is remarkably precise: judged one at a time, an expression found this way is as likely to be worth keeping as one from the full search — 24.5% against 24.6%. The problem is reach. Only 12% of English passages contain such a marker, and under 4% of Danish ones, because people describing illness rarely announce the comparison. It finds three of the seventeen known-good items and cannot reach the rest at any effort.

Why ordering is the answer

The simple search is limited by how people happen to phrase things: most metaphors do not announce themselves with “like” or “as if”. Broad prompting produces a larger pool. Second-model verification and the recorded screens set its provisional order; reviewers decide what to retain.

One practical note: the comparison-word route needs no model, no GPU and no paid service. A team with no computing resources at all can run it and obtain 70 candidates that pass every check, 31 of them strong — provided everyone understands it will miss four fifths of the material.

← back to the demo