Between an event happening and the sentence you read, there are eight stages.
The pipeline runs in one pass. It detects an event from media, and it writes the item from documents, so no outlet's body text is ever stored.
The product is not built and the site is not open. This page describes the pipeline the build is held to, in the order it runs.
We read the documents, not the pundits.
The eight stages
Each stage is queued and tested on its own, and every item records the version of the pipeline that produced it.
- Detect. In: headlines, timestamps, web addresses, section labels and metadata from about five hundred outlet feeds a country, together with structured public records for planning, council committees, courts, schools, crime and licensing. Out: a signal, with its place and the entities it names. Never: a word of body text from any outlet.
- Cluster and rank. In: the day's signals. Out: clusters that describe one event, ordered by how far a story has spread across outlets, by its arrival rate against its own baseline, and by its consequence to the reader. Never: ranking on headline similarity, which is a tie-breaker and never the key.
- Resolve to origin. In: a cluster. Out: the primary document behind it, which is a statement, a filing, a judgment, a data release, a planning document or a set of minutes, with its publisher and its publication time. Never: a general web search. The agent reads the Grade A registers of the jurisdiction, and returns no origin rather than the wrong document.
- Corroborate. In: the origin document and the outlet metadata. Out: a claim set, each claim carrying its grade, its citation and its confidence. Never: a fact resting on one outlet without the label that says so.
- Rehydrate backstory. In: the story so far, from open-licence reference works. Out: discrete facts, each attributed, each resolved to a Grade A document where one exists. Never: pasted text. A test looks for any twelve words in a row shared with a source and requires none.
- Write once, at four depths. In: the origin extraction, the claim set, the backstory facts, the place resolution and the voice rules. Out: one sentence, a paragraph of sixty to ninety words carrying the line about what it means for you, a backstory of two hundred to three hundred and fifty words, and the sources. Never: outlet text in the prompt, because none is stored to put there. The four depths are written in one pass, so they cannot contradict each other.
- The human edit queue. In: the written item. Out: an approval, an amendment, a hold or a rejection, recorded with the editor and the reason. Never: publication of a flagged item without an approval row. An item held past its own story window expires instead of publishing late.
- Personalise by selection. In: a reader's place set, chips, standing instructions, follows, seen state and cadence. Out: an ordered list of three to six items, with backstory attached to anything missed. Never: a model call at this stage, and never a sentence rewritten for one reader.
No outlet's body text ever enters storage
This is the one rule that has to hold everywhere, because a single exception to it changes what the company is. No column, no cache, no log line, no queue payload, no embedding input and no model prompt in this system holds the body text of a work whose licence does not permit it. Headlines, web addresses, timestamps, metadata, structured records and facts are in scope. Body text of an outlet page is not, in any form, at any stage, transiently or otherwise.
Nobody's article is stored, in whole or in part.
Three tests run on every commit. A schema test enumerates every text column in every table against a declared allow-list, so a body-text column cannot be added without a person seeing the allow-list change. A fetch test puts a forty-character sentinel in an article body, then requires it in zero database rows, zero cache keys, zero log lines, zero queue payloads and zero prompts, searching the whole test database rather than a list of tables. A prompt test requires the assembler to throw when it is handed a body it may not hold, rather than truncating it.
One editor for each region
A human editor for each region signs off about fifteen national items a day, and every item in a flagged local category: school ratings, crime, court items, any item naming a private individual, a licensing decision naming a person, and any item carrying the not-confirmed label.
The refusal sits at the delivery boundary rather than in the interface. A renderer refuses a flagged item whose approval row is absent, so an unapproved item is missing from the page, the email, the feed and the API alike.
Personalisation is by selection, never by prose
Everyone in a region reads the same edited sentence about a story. Briefs differ in which items they carry, in what order, in how far the geography reaches, and in how much backstory attaches. No reader is given a differently worded version of a fact.
Selection cannot change a word.
The grades, the publish floor and the wording rules that stages four and six apply are set out in the corroboration policy. Every register the pipeline reads is published with its grade and its licence in the source register.
That is the whole pipeline, and no part of it keeps a word of anyone else's article.