Do known-good metaphors surface?

Before asking people to review a ranked list, we test the ranking itself: entries of the published Metaphor Menu (Lancaster University) — expressions already chosen for a metaphor menu, so known to be good — were hidden among the candidates. If the ranking works, they should come out near the top. This is the same idea as a positive control in a lab: other fields inject synthetic signals into telescope data, spike known molecules into sequencing samples, or mix questions with known answers into crowd work, and judge the pipeline by what it gives back.

planted entries	n	median rank of 1,490	in top 10%	mean score
considered	13	107	77%	5.54
quoted in the scoring prompt — not considered, removed before ranking	4	—	—	—

Why 4 entries are not considered. The scoring prompt shows the model a few Metaphor Menu entries as examples of what a menu-worthy metaphor looks like. 4 of the 17 planted entries are among those examples, so the model would score them partly against themselves. They are removed from the candidate list before ranking — here and in the project's Danish and Dutch pages, where the exclusions differ because the quoted examples do — and only the remaining 13 count. We found this after first reporting results and corrected it.

Why 17 planted here, and 15 in Dutch

The published Metaphor Menu has 17 entries and all 17 were planted in this English pool. The project's Danish pool now plants all 17 too (two — the scary fairground ride and the journey on which you look up and notice the scenery — were missing from the translated set at first and were added later); the Dutch pool plants 15, because in Dutch those two are among the scoring examples and could not count.

How to read it

A typical planted entry surfaces around rank 107 of 1,490 — inside the top 7% — and 77% of them land in the top 10%. Ranking does not replace human judgement (one entry sits at rank 713); it decides what people read first, which is why stage 1 also reviews a concealed sample from deep in the list.