📘 Keep feedback loop from meaning send count

What you’ll learn

1. The FlywheelState Turck's 2016 data network effect in his terms, tell it from a classic network effect, and refuse to call a click log a moat.You watch a support agent rewrite an AI draft the dashboard still counts as a win. Turck, January 2016: a product gets smarter as it gets more user data, which draws more use — but only if the learning is productized. Classic network effects are other people. A loop map names signal, store, dataset, change, deploy, measure, and the clock. If you cannot walk that path, you have a warehouse.
2. Two Kinds of SignalSplit PAIR's explicit and implicit channels, treat ratings as a biased slice, and treat a rewrite as a labeled pair.PAIR's Feedback + Control chapter: explicit is commentary they meant to give; implicit is behaviour they took for another reason. An accept can mean good enough. People who click a thumb are not a census. A rewrite pairs the wrong output with the right one. Watch the three channels together, and treat disagreement as a finding.
3. Labels and the Cold StartTurn a log into a dataset, treat low annotator agreement as a spec bug, and bootstrap with Turck's data trap rather than a promise.Huyen: raw events are not a dataset until input, output, and schema hold. Labels come from humans, users, or a model-as-judge. Ng: consistency is paramount, so low agreement means rewrite the brief. Turck's cold start is a data trap — ship something already useful, then replace seed and synthetic data as live pairs arrive.
4. Close the Loop, Guard ItMatch a signal to a mechanism and a clock, state Ng's data-centric move, apply Article 5 without reducing it to consent, and name Jiang's degenerate loop.Route feedback to a prompt edit, a retrieval update, a fine-tune, or a retrain, then validate. Ng, 2021: hold the model fixed and improve the data. GDPR Article 5 requires purpose limitation and minimisation, plus a lawful basis — not always consent. Jiang et al., AIES 2019: the system's choices become the next dataset; explore, grow the candidate pool, freeze an eval set, and sample the discards.
5. Design a Reply LoopWrite a one-page loop for a support reply assistant — signals, labels, cold start, close, privacy, guards — and leave with three lines on a loop you already use.The assistant drafts; the agent decides. Star edit size and discards. Launch on a useful default and learn on pairs. Close with a same-day prompt path and a slower fine-tune, behind a test on edit rate and resolution. Minimise ticket fields and honour opt-outs. Tonight: name the signal, the outcome, and the clock on a feature you already touch, then ask what it would stop seeing if it trained only on what it already rewards.

Grounded in trusted sources

• The Power of Data Network Effects — Matt Turck, 4 January 2016 — data network effects, productized learning, cold start, data trap
• People + AI Guidebook: Feedback + Control — Google PAIR, 2019 — implicit vs explicit feedback; interaction is not always a request for more
• Andrew Ng Launches A Campaign For Data-Centric AI — Gil Press, Forbes, 16 June 2021 — hold the model fixed; consistency of data is paramount
• Art. 5 GDPR — Principles relating to processing of personal data — Regulation (EU) 2016/679, Article 5 — purpose limitation, data minimisation, storage limitation
• Degenerate Feedback Loops in Recommender Systems — Jiang, Chiappa, Lattimore, György, Kohli, AIES 2019 — echo chamber vs filter bubble; exploration
• Designing Machine Learning Systems — Chip Huyen, O'Reilly, May 2022 — logs to datasets, hand vs natural vs programmatic labels
• Matt Turck, The Power of Data Network Effects (4 January 2016)
• Google PAIR, People + AI Guidebook: Feedback + Control (2019)

Every Wunder lesson is built from real, reputable sources — never invented.

Related courses