Ranking

Work Backwards
From the Garbage

Ranking up requires a theory of quality. Filtering out only requires a theory of garbage — and garbage is structurally obvious.

On building an index by exclusion

Every search engine ever built has tried to answer the same question: which of these pages is best? Twenty-five years of ranking signals — links, anchor text, freshness, dwell time, click-through, embeddings — are all attempts at that one question.

It is a very hard question. “Best” depends on who’s asking and why. It’s subjective, contested, and it changes by the hour. And the moment you publish a signal that rewards good pages, an entire industry forms to manufacture whatever that signal measures. You end up in an arms race where your ranking function is public knowledge and your adversary is better resourced than you are.

We build the index around a different question, and it turns out to be a much easier one: which of these pages is garbage?

The asymmetry

Good pages are wildly heterogeneous. A great explainer, a primary source, a forum thread where somebody finally solved the thing — they share almost no measurable properties. There is no feature vector for “good.”

Bad pages are boringly uniform. They are thin, because writing costs money. They are ad-saturated, because that’s the point. They are affiliate-stuffed, templated, tracker-laden, and padded with the same filler phrases. Not because spammers lack imagination, but because the economics that produce them are narrow. If a page exists to arbitrage traffic into ad revenue, its structure is determined by that goal.

Quality is diverse. Garbage converges.

And convergent things are countable. You don’t need to interpret a page to notice it has 180 words and nine ad slots and fourteen affiliate links and a seventy-percent boilerplate ratio. Those are facts you can extract, not judgements you have to make.

What that looks like

The scoring function is three terms, and the order matters:

score = crowdRank          // primary — what people actually chose
      + priors * decay     // cold-start scaffolding; FADES as the crowd grows
      - penalties          // the exclusion floor

Exclusion is a floor, not a weight. Spam, duplicates and slop are dropped regardless of what else they have going for them. A page cannot buy its way past the filter with a strong domain or a lot of inbound links — which is precisely the failure mode that lets content farms outrank the thing they plagiarised.

Some of the actual filters, with their real thresholds:

None of these require understanding the page. They’re counted from features the scraper already extracted, which is deliberate: scraping happens elsewhere, ranking happens here. The ranker never touches the network, so it’s cheap to run, easy to test, and you can replay it over a fixture set to see exactly what a change does.

Every penalty says why

A page doesn’t just lose points. It carries the reason with it — thin content, ad-heavy (SEO farm), affiliate-link saturated. Plain language, attached to the result.

This started as a debugging affordance and turned into something more useful. A ranking you can explain is one you can argue with. If we demote something unfairly, the reason is right there to be disputed — by us, or by the person who wrote the page. An opaque score can only be obeyed.

Why exclusion is harder to game

This is the part that convinced me the tradeoff is right. To defeat a ranking signal you only have to simulate the thing it measures — manufacture the links, pad the dwell time, stuff the entities.

To defeat an exclusion filter, you have to stop doing the thing. Fewer ads. More words. Fewer affiliate links. Less boilerplate. The only way past the filter is to become materially less spammy — at which point the filter has already won, whatever your intent was.

You can still write bad content that passes. Exclusion doesn’t promise good; it promises not obviously garbage. But it moves the cost of gaming from “buy some links” to “change your business model,” and that is a very different price.

What handles the rest

Exclusion can’t tell you which of two decent pages is better. Nothing structural can. That ordering comes from crowd-rank — what people actually chose, boosted, and blocked. Behaviour, not inference.

Which leaves the cold-start problem: on day one there is no crowd. So there are priors — seed membership, domain legitimacy — with one important property: they decay. As real behaviour accumulates, the scaffolding fades out. The priors exist to make the index usable before it has users, not to be the answer forever. An index that leans on authority priors permanently just reproduces the existing hierarchy, which is the thing we’re trying not to do.