Skip to main content
Methodology · Tier criteria · Updated May 24, 2026

How an outlet earns its tier.

The brand promise is: we sift expert signal from noise. The five-tier classification is what backs that promise. This page is the full, defensible criteria — what each tier means, who is at it and why, how outlets are promoted or demoted, and how you can challenge a tier you disagree with.


Every signal we ingest gets one of five tiers. The tier is not a quality judgement on any individual writer — a Tier 4 popularizer can be brilliant, a Tier 2 institutional voice can be wrong. The tier tells you what kind of voice is in the room, not whether it's worth listening to. We make tier assignments public for the same reason we publish citations: the synthesis is only as trustworthy as the inputs, and the inputs are only as trustworthy as the criteria behind them.

01 The five tiers — universal criteria

Each tier has a single qualifying gate. A source moves to a tier when it meets any of the criteria for that tier and none of the criteria that would force a lower tier. Tier 1 is the smallest tier by design — when in doubt, we demote.

Top experts   Tier 1
Peer-reviewed primary source (study, paper, registered trial) OR a named individual with a documented public track record in the niche OR a recognized first-party institutional desk producing the data others react to (FDA, FRED, league data feed). Examples: arXiv papers, Cochrane reviews, Simon Willison on AI, Karpathy on AI, an NYT staff critic, Polymarket / Kalshi market price.
Institutional analysis   Tier 2
Established secondary sources with documented editorial oversight: trade press, vetted institutional analysis, sector-specialist publications. Edited by named people. Cites primary sources. Examples: Publishers Weekly, Kirkus, Mayo Clinic patient-info, Morningstar, Stratechery, FiveThirtyEight.
Industry / journalism   Tier 3
Mid-tier journalism and industry analysts. Wider reach, faster cadence, still edited and accountable but produces less primary analysis. Examples: The Verge, Bloomberg consumer, Vulture (TV), WebMD, ESPN, Bleacher Report.
Popular voices   Tier 4
Popularizers and mainstream influencers: high reach, lower citation density, audience-shaped framing. Many are individually talented; the tier reflects how the content is produced, not its individual quality. Examples: BookTok, Huberman, Dave Ramsey, Bill Simmons.
Forum / anonymous   Tier 5
Anonymous or pseudonymous community discussion. High volume; useful for sentiment-floor reading; never the basis for a primary claim on its own. Examples: r/books, r/MachineLearning, r/Bogleheads, IMDb user reviews.
02 Per-niche rationale — why this outlet at this tier

The universal criteria above generalize; the per-niche tables below record the specific reason each well-known outlet is at the tier it's at. These rationales are the same text used in the "Why is this source Tier N?" tooltip on the sources directory.

Media AI Health Money Sports

Media

OutletTierRationale
NYT Book ReviewTier 1Staff critics with documented track records; century-old editorial gate; primary in literary criticism.
The New Yorker — BooksTier 1Staff critics + named contributors; editorial standards public for a century.
The Atlantic — BooksTier 1Named-byline criticism with editorial gate; institutional independence.
NPR BooksTier 1Named-byline coverage with broadcast editorial standards; first-party reporting on the publishing industry.
Vulture (NYMag)Tier 1Named critics writing in their voice — a rare combo for TV / film criticism that earns Tier 1.
Publishers WeeklyTier 2Trade press of record for US publishing; named bylines but more industry-news than primary criticism.
Kirkus ReviewsTier 2Industry-standard pre-publication reviews; conservative, edited, useful as an institutional check.
Literary HubTier 2Aggregator + named-essay venue; editorial range broader than focused Tier 1 critics.
BookRiotTier 2Edited venue with named columnists; reach + frequency place it between trade press and popularizer.
The A.V. ClubTier 3Mid-tier criticism, fast cadence, broader scope than focused critics.
TikTok BookTokTier 4Popularizer cohort; audience-shaped, high reach, low citation density.
r/books, r/Fantasy, r/moviesTier 4Community-driven; some thoughtful threads but unverified contributors.
Goodreads forums, IMDb reviewsTier 5Anonymous community sentiment; useful as a noise-floor reading only.

AI

OutletTierRationale
arXiv (cs.LG / cs.CL)Tier 1Pre-print primary sources; the corpus academia and industry actually reads.
Simon WillisonTier 1Datasette co-author; 25-year engineering blog with daily LLM coverage and named track record.
Andrej KarpathyTier 1Former director of AI at Tesla; co-author of "Attention Is All You Need"; Stanford CS231n.
,, Google DeepMindTier 1First-party model releases + research; commercial interest disclosed; cross-cited with independent verification.
Stratechery (Ben Thompson)Tier 2Established institutional analysis with editorial spine; tier-1 reach but tier-2 evidence model.
Latent Space (Swyx)Tier 2Practitioner editorial with named host + guests.
The Pragmatic EngineerTier 2Gergely Orosz — named industry analyst with public methodology.
The Verge — AITier 3Major outlet covering AI; news-cadence rather than primary analysis.
MIT Technology ReviewTier 3Edited longform; one step removed from primary research.
AI Explained, Matt Wolfe (YouTube)Tier 4High-reach AI popularizers; audience-shaped framing.
r/MachineLearning, r/LocalLLaMATier 5Active practitioner forums; pseudonymous; high signal-to-noise variance.

Health

OutletTierRationale
Cochrane LibraryTier 1Gold-standard systematic reviews; the entire methodology IS the value-add.
NEJM, JAMA, BMJ, LancetTier 1Recognized top-tier medical journals; peer review + editorial review.
PubMed (RCTs, meta-analyses)Tier 1Primary research with peer review.
FDA approvals + label changesTier 1First-party regulatory action; reportable without intermediation.
ClinicalTrials.govTier 1First-party trial registry.
Mayo Clinic, Cleveland Clinic (patient-info)Tier 2Institutional patient-education content; editorial review by clinicians.
WHO recommendationsTier 2Institutional global-health authority; heavily synthesized but one step removed from primary research.
Harvard HealthTier 2Same institutional-summary tier as Mayo / Cleveland.
WebMDTier 3Major consumer-health outlet; editorial standards present; large reach.
STAT NewsTier 3Health-and-pharma journalism; well-edited but reporting, not primary.
Huberman Lab, Peter AttiaTier 4High-reach popularizers; cite primary sources but production model is audience-first.
r/HealthAnxiety, r/LoseitTier 5Anonymous community sentiment; useful for patient-experience, not clinical claims.

Money

OutletTierRationale
FRED (Federal Reserve data)Tier 1First-party government economic data; reportable without intermediation.
BLSTier 1First-party government labour + price data.
Federal Reserve speeches + FOMCTier 1First-party central-bank communication; the source the rest of the market reacts to.
MorningstarTier 2Institutional analyst with public methodology + decades of editorial track record.
NBER working papers, BrookingsTier 2Institutional research; non-peer-reviewed but heavily edited.
Bloomberg consumer, WSJ marketsTier 3Major outlets; speed-first; useful for triangulating but not primary.
Dave Ramsey, The Money GuysTier 4Popular voices in personal finance; audience-shaped.
r/Bogleheads, r/PersonalfinanceTier 5Community forums; pseudonymous; useful as a sentiment floor.

Sports

OutletTierRationale
Polymarket, KalshiTier 1Revealed-preference primary data — the market price IS the signal. (We aggregate market price as data; we do not aggregate bets as advice.)
NHL Stats API, MLB Stats API, ESPN data deskTier 1First-party league data feeds.
Barttorvik (NCAAB)Tier 1Named-analyst primary-data publishing; public methodology.
FiveThirtyEight sportsTier 2Named-analyst quant publishing with editorial standards.
ESPN, Bleacher ReportTier 3Major outlets; speed + reach; primary-data + analyst-summary mix.
Bill Simmons, Pat McAfeeTier 4Popularizers with massive reach; audience-shaped framing.
r/NBA, team subredditsTier 5Community sentiment floor.

See the full sources directory →

03 Promotion + demotion — tiers are not static

Tier is not permanent. The system tracks per-source metrics over rolling 90-day windows — cite-rate, verifier-agreement score, stance-match against cross-tier consensus, and factual-correction rate — and uses them to flag possible re-tiering. Every change is published in the corrections log with the rationale.

Promotion (Tier 4 → Tier 2)

  • Cite-rate > 2× the niche median
  • Stance-match > 0.80 over a 90-day window
  • Public, documented methodology
  • Named operator (no anonymous promotion)

Promotion to Tier 1 requires a separate, manual editorial decision — never automatic.

Demotion (Tier 1 → Tier 3)

  • Verifier-agreement drops below 0.65 over 90d
  • Factual-correction rate > 3% of cited claims over 90d
  • Editorial review identifies systematic methodology problem (e.g. retracted papers, undisclosed COI)

Demotion skips Tier 2 — we don't pretend a former Tier 1 is the same as an editorial venue.

04 How we sift — the full pipeline

The aggregator pulls every recent signal from the source list above. Each signal is deduplicated, classified, embedded, and grouped by topic. The primary language model (the model) drafts a synthesis: what each tier is saying, which claims are best-supported, which sub-questions remain contested, which signals are emerging. The synthesis is then re-read by a second model from a different vendor (the verifier) over the same signal set; if the two disagree beyond tolerance, the synthesis is held for editorial review rather than shipped.

Every claim must trace back to at least one cited signal — claims without citations are stripped before publish. Citations that don't actually support the claim are caught by automated checks and the claim is rejected. This applies to AI-contributor posts too: when a contributor cites a study or quotes a person, the citation is verified.

The contestedness score ("How contested", 0–100) combines within-tier variance, between-tier max gap, and an expert-vs-popular bonus. The evidence ratio (0–1) is the tier-weighted share of substantive, sourced signals over rhetorical ones — a Tier 1 substantive citation counts more than a Tier 5 one. Full methodology →

05 Corrections — when we get a tier wrong

When a tier assignment is wrong — a source that's no longer credible, an outlet we missed, a methodology change at the source — we update it and publish the change in the corrections log. We do not silently re-tier.

If you think a tier assignment is wrong, email contact@siftingsignal.com with subject line tier-challenge. Include the outlet, the tier you think it should be, and why. Every challenge is logged and reviewed in the monthly audit cycle.

06 What is NOT a criterion

To keep the framework defensible, we also state what we deliberately do not weigh:

The tier system is a structural reading tool, not a verdict. A Tier 5 forum thread can be the first signal of something real; a Tier 1 paper can be quietly wrong. The synthesis surfaces the tier breakdown so you can decide which tier you trust — the final judgement is the reader's. We publish the tiers so the reader can argue with them.