You are a Trust and Safety Product Manager at YouTube. How would you measure the success of this?

**Hint / Brief Solution** This is a metrics question with a twist. In Trust and Safety, success is the harm users never see. So don't count your own work. Count what actually reaches the viewer. **Step 1. Clarifying Questions** - Which part of T&S? All of YouTube videos, or one area like comments, Shorts or Live? - All harm types, or one (child safety, hate, scams, misinformation)? - Global or one region? Rules differ by country. - Are we tracking the health of the whole program, or judging one launch like a new classifier? Assume: all of T&S for YouTube videos, all harm types, global. **Step 2. Understand the Product** T&S is a pipeline: Policy → Detection (machines + user flags) → Human review → Action (remove, age-restrict, limit reach, strike) → Appeal. It can fail in two ways. Bad videos stay up. Good videos get taken down. Your metrics must catch both. **Step 3. Stakeholders and Value** - Viewers: don't run into harmful videos. Parents can trust YouTube. - Creators: honest videos stay up. Appeals are fair and fast. - Advertisers: their ads never run next to harmful content. - YouTube: user trust, regulators satisfied, ad revenue protected. Where value overlaps: harmful videos don't get watched, and honest videos stay up. The first half becomes the North Star. The second half becomes the guardrail. **Step 4. North Star Metric** Violative View Rate (VVR) = views on rule-breaking videos ÷ all views. Lower is better. - Measure it by sampling random views and sending them to human reviewers. This way it also counts bad videos your systems missed. - Why not number of videos removed? Removals go up when more bad content is uploaded, or when you remove too much. That measures your effort, not what viewers saw. - Cut it by harm type. One child safety view is far worse than one spam view. - This is the same metric YouTube publishes every quarter. **Step 5. L0 Metrics (they multiply to the North Star)** Follow a bad video's journey: Uploaded → Gets past us → Gets watched. VVR = Violative Upload Rate × Escape Rate × Relative Reach - Violative Upload Rate (what comes in) = bad uploads ÷ all uploads - Escape Rate (what gets past us) = bad videos that got at least one view ÷ bad uploads - Relative Reach (how far it travels) = avg views of an escaped bad video ÷ avg views per video Example (illustrative): 100M uploads and 100B views, so 1,000 views per video on average. - 1% are bad = 1M bad uploads - 40% escape = 400K get seen - Each gets 250 views before removal, so Relative Reach = 250 ÷ 1,000 = 0.25 1% × 40% × 0.25 = 0.1%, which is 10 in every 10,000 views. Check: 400K × 250 = 100M bad views. 100M ÷ 100B = 0.1%. Same answer. **Step 6. L1 Metrics** - What comes in: repeat violator rate, share of bad uploads from new or fake accounts, re-upload attempts of removed videos - What gets past us: detection recall at upload by harm type, % of removals first flagged by machines, recall on Live, Shorts and non-English videos - How far it travels: time to removal, % of videos removed within 10 views, share of bad views coming from recommendations **Step 7. Guardrails (what must not get worse)** - Wrongful removal rate (proxy: % of appealed videos that get reinstated) - Appeal turnaround time - Creator uploads and watch time per DAU Trap to call out: you can push VVR close to zero by removing everything borderline. Creators get hurt, and the guardrails catch it. Interview tip: North Star = harm seen. L0s = where it leaks. Guardrails = don't over-police.