Data Strategy & the Data Flywheel

Every founder in 2026 rents the same frontier models from the same three or four labs. The weights are a commodity you cannot own and cannot hide. What you can own is the proprietary data exhaust your product generates while doing real work for real customers - and the machinery that turns that exhaust back into a better product. This chapter is about building that machinery deliberately, from day one, because a data flywheel that starts spinning in month three is worth more than a model architecture nobody can replicate but everybody can rent.

5.1Data as the AI moat#

Model quality is converging and renting it is trivial. When your competitor can call the same Claude, Gemini, or GPT endpoint you can, the base intelligence is table stakes. The durable asset is proprietary feedback data from real users doing real tasks - data that is expensive to collect, slow to accumulate, and structurally impossible for an outsider to buy or scrape.¹ We treat moat strategy as Chapter 12’s domain; here we care about the one moat made of data, and how you manufacture it.

Not all data is a moat. Draw a hard line between three tiers. Commodity data - public web text, generic images, anything scrapable - confers no advantage, because everyone has it and it is exactly what the labs already trained on. Purchased data - a Surge AI or Scale AI annotation contract - narrows the gap but is available to anyone with a budget, so it buys parity, not defensibility.² Only process-generated data - the signal that exists solely because your specific product mediated a specific real interaction - compounds into a moat. 11A useful test: if a well-funded competitor could recreate your dataset in ninety days with a checkbook, it is not a moat. It is an expense.

The compounding is the whole point. Each turn of the loop makes the product incrementally better, which draws more usage, which yields more proprietary data, which sharpens the next turn. Practitioners peg the time to a defensible flywheel at 18 to 36 months - long enough that late starters cannot out-spend their way in, because they cannot buy back the calendar.¹

Consider Cortex, our running example: an autonomous AI customer-support agent for mid-market SaaS companies. Cortex’s base reasoning comes from a frontier model any competitor can license. Its edge is that every ticket it has ever resolved or escalated - across hundreds of B2B SaaS deployments - carries a label for what a correct resolution looked like in that exact product’s context. No competitor starting today can conjure that corpus, because it exists only as a byproduct of Cortex having done the work.

5.2Acquiring and labeling data, including synthetic#

Before the flywheel spins on its own, you have a cold-start problem: you need enough data to make the product good enough that people will use it and generate more. There are four levers, and mature teams pull all four in sequence.

Bootstrap from the founding team. The first few hundred labeled examples should come from you and your domain experts, by hand. This is not beneath you; it is the highest-leverage labeling you will ever do, because you are simultaneously defining the label schema and discovering what “good” even means. For Cortex, that meant the founders sitting with three design-partner support teams and hand-resolving a few hundred tickets while writing down the decision rules.

Buy expert labels for what you cannot self-generate. The labeling market has bifurcated sharply. Commodity annotation - bounding boxes, basic classification - is being eaten alive by AI-assisted pre-labeling and synthetic generation, and prices are collapsing. The durable, expensive work is expert human data: domain specialists producing the nuanced judgments a model cannot yet make.² The vendor landscape reshuffled violently in 2025: after Meta took a 49% stake in Scale AI for $14.3 billion, rival labs including Google, OpenAI, and xAI pulled contracts within days over conflict-of-interest concerns, and independents captured the flight - Surge AI (bootstrapped, $1.2B 2024 revenue), Mercor, and Turing became the fastest-growing alternatives.² 22The lesson for founders is second-order: your labeling vendor can become your competitor's parent overnight. Never let a single annotation supplier see your full label schema and your task distribution.

33In the wildThe Scale-Meta fracture (June-July 2025) - Meta's $14.3B for 49% of Scale AI triggered an exodus: rival labs cut ties within days, and Scale laid off roughly 14% of staff in July 2025. Surge AI, Mercor, and Turing absorbed the displaced demand. It is a live case study in supplier-concentration risk for any AI startup dependent on outsourced labels.

Generate synthetic data - carefully. By 2026 synthetic data is mainstream: Gartner’s much-cited projection - 75% of businesses using generative AI to create synthetic data by end of 2026, up from under 5% in 2023 - now looks conservative.³ Tooling matured accordingly - NVIDIA acquired Gretel in a reported nine-figure deal, and Gretel Navigator’s “Text to Dataset” lets you type “generate 5,000 flood-damage insurance claims in Florida” and get a populated schema back.³ Synthetic data is superb for rare edge cases, adversarial examples, privacy-safe pipelines, and bootstrapping eval suites (evals are Chapter 6’s turf). But there is a hard failure mode: model collapse, the degenerative drift that occurs when models train on their own outputs across generations. The mitigations are empirical - retain real anchors; even a modest real-data floor, on the order of 10%, sharply reduces perplexity drift.³ For Cortex, synthetic tickets fill the long tail (the once-a-quarter refund-dispute-during-an-outage scenario) but never replace the real, human-verified resolutions that anchor the model.

Use programmatic and weak supervision (Snorkel-style labeling functions) to turn expert heuristics into labels at scale, then have humans review only the low-confidence and disagreement cases. This is the highest-ROI middle path once you have enough hand labels to write good rules.

5.3Designing the data flywheel#

The flywheel is not a metaphor you hope emerges; it is an engineering system you design. The canonical loop has four stages, each of which you must instrument deliberately.

The stages are capture (log production traffic, decisions, and outcomes), curate (filter, label, and select what is worth training on), train (fine-tune, distill, or update routing/retrieval on the curated set), and deploy (ship the improvement and measure the lift). NVIDIA’s open-source Data Flywheel blueprint formalizes exactly this loop - collect production logs, evaluate with an LLM judge, fine-tune, and redeploy - and in NVIDIA’s own internal tests it cut inference costs by up to 98.6% while holding accuracy, largely by distilling a big model’s behavior into a small fine-tuned one once enough production data existed.⁴ That cost collapse is the underrated payoff: a working flywheel doesn’t just improve quality, it lets you replace an expensive frontier call with a cheap specialized model on the tasks you now have data for.

Cortex’s flywheel in motion. Every ticket Cortex touches ends in one of two labeled states: resolved (the customer confirmed the fix, or never reopened it within the SLA window) or escalated (a human agent took over). Each terminal state is a free, high-quality label for what “good” looked like on that ticket. Resolved tickets teach the routing model which retrieval passages and action sequence produced a fix; escalated tickets flag exactly where the agent’s confidence should have been lower - training signal for when to hand off. The retrieval layer (Chapter 8) is re-ranked on which documents actually appeared in successful resolutions. No synthetic corpus and no purchased dataset can produce this, because the label is defined by the outcome of Cortex’s own action inside a specific customer’s live product.

The trap founders fall into is building a flywheel that is technically closed but too slow to matter. If it takes six weeks to get a production signal back into the model, you get roughly eight turns a year. If it takes two days, you get a hundred. Loop latency is a first-class metric - instrument it and drive it down like you would p95 response time.

5.4Capturing feedback and implicit signals#

Training data quality lives or dies at the capture layer, and most teams under-invest here catastrophically. There are two families of signal, and you need both.

Explicit feedback is what the user deliberately tells you: thumbs up/down, a corrected answer, an accepted or rejected suggestion, a star rating. It is high-signal but scarce - people rarely bother. The single highest-leverage UX decision is friction: an inline thumbs up/down, designed into the primary interaction surface rather than hidden behind a click, lifts submission volume by an order of magnitude over a separate modal form. Better still, capture user edits - when a user takes your model’s draft and changes three words before sending, those edits are a gift-wrapped preference pair.

Implicit feedback is the behavioral exhaust: regeneration requests, session abandonment, dwell time, copy events, whether the user re-asked the same question, whether they escalated to a human.⁴ Implicit signals are noisier per event but arrive in enormous volume and cannot be gamed by survey fatigue. A regeneration is a soft thumbs-down; an abandoned session after an answer is a strong negative; a copied code block is a strong positive. The art is mapping each behavior to a confidence-weighted label.

These labels feed preference-tuning pipelines directly. Explicit up/down and implicit positive/negative can be filtered into DPO or KTO training sets, or used to gate which examples enter supervised fine-tuning - a full RLHF pipeline is often overkill for an application startup, and gives only a minor, non-durable edge unless you already have a large engaged base generating the preferences.¹ Your advantage is not the RL algorithm - those are public - it is the proprietary preference data flowing in.

For Cortex, the richest implicit signal is the outcome itself: did the ticket stay closed? A ticket the customer reopens within 48 hours is a delayed, unambiguous negative that no thumbs-down button would have captured, because the user felt satisfied in the moment and only discovered the failure later. Outcome-based labels - tied to what actually happened downstream - are the crown jewels of any flywheel, and they are available only to the company that owns the end-to-end workflow.

44In the wildNVIDIA Data Flywheel Blueprint (2025-26) - NVIDIA open-sourced a reference blueprint (with a Weights & Biases observability layer) that operationalizes the full loop: capture production logs and thumbs/edit/regeneration signals, evaluate candidate models with an LLM judge, run DPO/KTO or filtered fine-tuning, and redeploy. In one internal run it distilled a 70b production model into a fine-tuned 1b for a tool-calling task at comparable accuracy. A concrete template founders can study rather than reinvent.

5.5Privacy, governance and data rights#

Here is where founders quietly sabotage their own flywheel: they build the loop and only later discover they never had the right to train on the data flowing through it. Data you cannot legally use for training is not fuel - it is liability. Governance is a precondition for the flywheel to spin at all, not a compliance afterthought. (The regulatory deep-dive - EU AI Act, GDPR, sector rules - is Chapter 17; here we cover the data-rights mechanics specific to training.)

Your training rights live in the contract, not the homepage. Two controls are routinely conflated and are legally distinct: zero data retention (ZDR), meaning the processor does not store your inputs/outputs beyond the immediate request, and training opt-out, meaning your data is not used to improve someone’s model. A vendor can honor one and violate the other.⁵ You must resolve both directions:

• As a consumer of model APIs: confirm in the DPA/MSA (not the marketing page) that your customers’ data is not used to train the provider’s models, with explicit scope, covered endpoints, and exclusions. Enterprise tiers of the major providers contractually prohibit training on API data - but consumer and free tiers increasingly opt you in by default. GitHub Copilot, for instance, begins using Free/Pro/Pro+ interaction data for training by default on April 24, 2026 unless users opt out, while Business and Enterprise seats are exempt.⁶ 55Read the version-and-tier fine print. "We don't train on your data" is almost always true only of the enterprise SKU, and submitting thumbs-up feedback can explicitly opt that specific conversation back into training even on protected tiers.
• As a producer building on customer data: your own customer contracts must grant you the right to use their data - ideally de-identified and aggregated - to improve your product. If your MSA is silent, assume you cannot, and you will discover this during a security review that stalls a six-figure deal.

For a B2B product like Cortex, the governance design is load-bearing to the sales motion. Cortex’s contracts specify that resolved-ticket data may be used to improve the service in de-identified, aggregated form - customer names, account IDs, and PII stripped at the capture boundary, never at training time. Cross-tenant learning (patterns from Customer A improving Customer B’s routing) is permitted only on abstracted signals - which retrieval strategy worked, not the content - so no customer’s proprietary data leaks into another’s model. Documented, de-identified provenance of this kind also eases the data-governance expectations regulators now attach to higher-risk systems, whose enforcement regime is Chapter 17’s subject.⁷ The startups that treat governance as an enabling design constraint sell into regulated enterprises; the ones that treat it as paperwork get stuck in procurement forever.