A shortlist of genes is not a boundary between true and false

Nov 14, 2026

A gene just below a software cut-off has not been shown to be irrelevant. A gene just above it has not been proved to explain the biology. Yet a results table can make that distinction look settled: keep these rows, discard the rest.

In 2012, I wrote How Not to Be a Bioinformatician with Segun Fatumo and Reinhard Schneider. We used deliberately bad advice to expose bad practice. One instruction was to draw arbitrary cut-offs through ranked results and treat the division as “Absolute truth above, absolute falsehood below”. We also warned against accepting default settings without exploring alternatives and looking only at the first result. Our commentary, guideline 8.

The joke identifies a specific mistake: turning a rule for selecting what to examine into a claim about what is true.

Consider a hypothetical analysis that ranks candidate genes for further investigation. A candidate gene is simply a gene being considered as a possible explanation. Suppose the researcher has enough time to examine only the highest-ranked candidates. The resulting shortlist is useful. It tells the researcher where to start.

It does not, by itself, explain why the next gene should be dismissed. Perhaps its score is almost the same as the last gene retained. Perhaps the ranking would change under another reasonable setting. Those are questions to investigate, not findings from an experiment I am reporting here. But they show why a practical limit on review time should remain visible as a practical limit.

The wording of the report matters. “These genes were selected for review using this scoring rule” tells the reader what happened. “These are the genes responsible” makes a much stronger claim. The second statement needs evidence beyond the fact that a program placed them at the top of a list.

This does not make every threshold arbitrary. A threshold can have a stated purpose and evidence supporting its use. My objection is to leaving that justification unstated, then asking the reader to treat inclusion as confirmation and exclusion as refutation. The question is what the boundary means in this particular analysis.

I would therefore want a gene shortlist to come with an explanation of the score, the selection rule and what was left outside it. If a small, reasonable change to the settings changes the candidates selected, that deserves attention. It does not automatically invalidate the analysis. It tells us something about how firmly we should hold its interpretation. These are reporting and checking practices I am proposing here, rather than outcomes demonstrated by our paper.

Our 2012 article was a commentary, not a study measuring how often researchers make this mistake or testing a remedy. It also credited Carole Goble’s The Seven Deadly Sins of Bioinformatics beside the cut-off warning. The argument belongs to a wider conversation about scientific practice, not to me alone. Original text and references.

When I read a shortlist, I want to know why those genes were selected and what evidence would change the selection. The boundary in the table should help organise the investigation. It should not quietly become its conclusion.