Scoping & Success Metrics: What You'll Refuse to Build

Discovery told you what problem to solve. Scoping decides what you will build, what you will explicitly refuse, and how everyone will know it worked. This is the discipline that prevents six-month death marches.

The client problem: the word "just"

The next morning, Maria's email lands in your inbox:

Read that email the way an FDE reads it: as a scope event — any moment where the boundary of the work shifts. Your job isn't to say yes or no on instinct; it's to run the ask through the boundary you should have drawn before week 3.

Start with the word "just." Unpack what Tom's button actually contains: choosing the columns, handling 40,000-row exports without timing out, scheduling a job that runs before Tom's coffee, managing the email list (personal data Lisa will want to review), handling bounces, and maintaining all of it forever. That "just" was hiding roughly a week of work and a permanent maintenance obligation. Maria's "nothing fancy" prediction is worse: a forecasting model you'd have to stand behind in front of borough presidents, trained on data Dev just told you is unreliable.

The pattern: small asks feel free in the moment and expensive forever. Nobody is being difficult — Tom genuinely needs his Excel, Maria genuinely needs Thursday's demo to land. But every unexamined yes moves the deadline, dilutes the core build, and erodes trust later, when the thing you promised casually arrives late or half-working.

Why it happens (and why it's nobody's fault)

Scope creep isn't a character flaw — it has structural causes you can design against:

1. Nobody wrote down what "done" looks like. Maria's "done" is a Thursday demo that lands. Tom's is his morning Excel. Without one shared, written definition, every conversation redefines the project.

2. Each stakeholder asks locally. Tom can't see Maria's ask or Dev's warning. From his seat, his button is the only new thing. Somebody has to hold the total — that's you.

3. Yes feels like good service. It's warm in the moment — but you're measured at week six, not mid-demo. A yes you can't deliver is a delayed disappointment with your name on it.

4. "Later" has no address. Without a written deferred list, "maybe later" sounds like a promise everyone remembers differently. A refusal list marked deferred respects the idea, parks it, and dates it.

The minimal concept: the boundary and the test

Scoping has two parts. Learn both as plain language first; the jargon comes free afterward.

The boundary: what we will build, and what we will explicitly refuse. Scope is the list of what this engagement includes — with the refusal list carrying equal weight. If it isn't written down as refused, it will be assumed as included. (The formal version is called a statement of work, or SOW. Your one-pager at the end is the SOW in miniature.)

The test: acceptance criteria a non-technical stakeholder can verify. Acceptance criteria are the demo-day test: checkable statements that let Maria say "yes, this works" without opening a terminal. The rule — the Maria test: if Maria can't verify it herself, it isn't an acceptance criterion yet.

Watch the difference. Each pair below says the same thing; only the second one passes the Maria test:

Vague (fails the test)	Verifiable (passes the test)
The dashboard loads quickly.	On Maria's office laptop, on office Wi-Fi, clicking a borough shows results within 3 seconds.
The data is up to date.	Each weekday by 9:00 AM, the dashboard shows yesterday's requests, and the on-screen "last updated" date reads today.
Tom can export data.	Tom filters to his borough, clicks one Download button, and opens the file in Excel — without calling anyone for help.
The pull is reliable.	If the nightly pull fails, the team gets an email by 7:00 AM, and the dashboard shows yesterday's data with a "refresh delayed" note instead of a blank page.

The formula inside the good column: who checks it, what they do, how it's measured, and the threshold that counts as passing. Who + what + measurement + threshold — that's the whole technique, and it's what lets a stakeholder see exactly what they're buying.

Fix it together: can CityOps depend on these two APIs?

CityOps leans on two free, public APIs: the NYC 311 feed (our data) and Open-Meteo (weather, for overlaying storms on complaint spikes). Before any pipeline code, an FDE asks: are we allowed to build on these, and what breaks if they change? That question has a name — build-vs-buy — and a standard checklist. "Build" means implementing it yourself; "buy" means depending on someone else's service. Here "buying" is free, but free is not the same as safe.

Six checks. Plain language first, every time:

1. SLA — a written uptime promise and what happens when it breaks. Free public APIs almost never offer one: you own the failure plan.
2. Rate limits — requests allowed per hour or day. Exceed it and you get an error (usually HTTP 429) instead of data.
3. Auth — how the API knows it's you, and what secret you'd have to store and protect.
4. Compliance — the legal and policy rules around the data: privacy, retention, who may see it.
5. Data rights — who owns the feed, and whether the tap can be turned off tomorrow.
6. Exit cost — how painful switching is if the API disappears. A cheap exit makes a risky dependency acceptable.

Here's what the 311 feed actually looks like — a real record, pulled live with this command:

$ curl "https://data.cityofnewyork.us/resource/erm2-nwe9.json?\$limit=1&\$select=unique_key,created_date,agency,complaint_type,descriptor,incident_address,city,borough,status,open_data_channel_type,latitude,longitude"

(one real record, trimmed for readability)
{
  "unique_key": "70591261",
  "created_date": "2026-10-01T02:05:23.000",
  "agency": "NYPD",
  "complaint_type": "Noise - Street/Sidewalk",
  "descriptor": "Loud Music/Party",
  "incident_address": "963 WOODYCREST AVENUE",
  "city": "BRONX",
  "borough": "BRONX",
  "status": "In Progress",
  "open_data_channel_type": "PHONE",
  "latitude": "40.831669762919574",
  "longitude": "-73.9287544719845"
}

Real data, real fields, no key required — this exact request works from any terminal. Note what's in it: a stable ID (unique_key), timestamps, agency, complaint type, location, status. This is the raw material of CityOps. Now the evaluation:

Check	NYC 311 (Socrata) API	Open-Meteo
SLA — the uptime promise	None. It's a public dataset published on the city's schedule — daily refreshes, not real-time. If the portal is down, we wait.	None on the free tier. Fair-use service; no guaranteed uptime.
Rate limits	Free, no key required for reads. Anonymous requests share a throttled pool — HTTP 429s are common under load. A free app token (sent as an X-App-Token header) raises the ceiling to roughly 1,000 requests per rolling hour.	Free, no key at all. Fair use of roughly 10,000 calls per day. CityOps needs about 24 (one cached call per hour) — nowhere near the cap.
Auth — proving who we are	Nothing to store for reads; the optional app token is a public, rate-limit-only credential — not a secret, but still belongs in config, not code.	Nothing. No key, no signup.
Compliance	The feed is public data — anyone can download it, so storing it isn't the issue. Lisa's standing rule still applies to what we send out: complaint text never goes to an external model or third-party service without her written sign-off, and any internal staff notes stay in our own database.	We send coordinates to a third party. We round them and cache aggressively so we send as little as possible. Free tier is non-commercial, and the data requires attribution ("Weather data by Open-Meteo.com").
Data rights — can the tap close?	The city owns the dataset and publishes it under its open-data terms; it can change the schema or retire the dataset. Mitigation: our pipeline validates the schema on every pull and alerts on unexpected changes (you'll build this in Stage 2).	Open-Meteo can change free-tier terms. Mitigation: weather is an enrichment layer, not the core — the dashboard must work with the weather panel showing "unavailable," never a blank page.
Exit cost — the fallback	Low. The same data is downloadable as bulk CSV from the NYC Open Data portal, same schema — a bad API day becomes a file download, not a crisis.	Low. For US locations, NOAA's api.weather.gov is also free and keyless (it asks for a User-Agent header). One adapter interface, two implementations.

The decision, written the way you'd write it to Maria: Yes to both — with rules. The 311 feed is pulled once daily with the free app token, retries with backoff on 429s, and reconciles record counts on every run. Open-Meteo is called once an hour and cached — never per page load — with a graceful "weather unavailable" state. The one-pager states plainly: data is as fresh as the city's daily publish; we do not promise real-time. Cheap exits make both safe to enter: that's build-vs-buy in practice — six questions, honest answers, failure modes named.

Break it: what vague acceptance criteria cost

Now the failure mode. Imagine we skipped the Maria test and shipped with this acceptance criterion: "The dashboard should be fast and show everything the team needs."

Thursday's demo arrives. Maria says it feels slow — on her laptop, the borough filter takes nine seconds. You measured two on yours; neither of you is wrong, because "fast" was never defined. Tom asks where his Excel button is — "everything the team needs" included it in his head. Dev points out the unresolved count doesn't match his spreadsheet, because nobody specified which requests count as unresolved. Three stakeholders, three definitions of done, one demo in flames.

The repair bill: two weeks of rework, a second demo, and a dent in the trust you'd built in weeks 1–2. The prevention bill: forty-five minutes in week 1, writing the verifiable versions from the table above and getting Maria's signature on them.

The general rule: vagueness doesn't remove disagreement — it schedules it for the worst possible moment. Write the argument down early, while it's still cheap, and resolve it over coffee instead of in front of borough presidents.

Make it production-safe: the scope one-pager

The one-pager is the SOW in miniature: one page, plain language, signed off by the stakeholder who owns the outcome (Maria). Pin it in Slack, link it from the repo README, re-read it at the start of every demo. When Tom's "just a button" arrives in week 3, you don't improvise — you open the page.

Copy this template for every engagement; fill the brackets with the customer:

SCOPE ONE-PAGER
Project:  [name]
Date:     [____]   Owner (engineer): [____]   Signed off by: [____]

1. THE PROBLEM (one sentence, in the customer's words)
   [What hurts.]

2. WHAT WE WILL BUILD (numbered — each item testable)
   1. [...]
   2. [...]
   3. [...]

3. WHAT WE WILL NOT BUILD (the refusal list — as important as section 2)
   - [...]  (deferred to [phase/date] / refused for v1 because [reason])
   - [...]

4. SUCCESS METRICS (how we know it worked — numbers, not adjectives)
   - [...]
   - [...]

5. ACCEPTANCE CRITERIA (the demo-day test — a non-technical person can run it)
   1. [Who] [does what] [measured how] [threshold for pass]
   2. [...]

6. DEPENDENCIES AND ASSUMPTIONS
   - [External API / data / access we rely on — and what happens if it breaks]
   - [What the customer must provide, and by when]

7. WHAT CHANGES COST
   New requests get written down, estimated, and re-signed before work
   starts. Nothing enters the build from a hallway conversation.

Section 7 is the quiet powerhouse. It doesn't say no — it says here's the door new requests walk through. When Maria's email arrives, your answer is a process, not a debate: "Great asks — let me write them up, estimate them against the six weeks, and we'll decide together what moves." One paragraph, scope intact, Maria heard instead of blocked.

Say it to the customer in plain language

Here's how the week-3 conversation actually sounds when you've done the work. Two paragraphs, no jargon:

The refusal is early (before the deadline is at risk), specific (names both asks), respectful (written down, estimated, phase two), and anchored (points back to the signed criteria). You're not the engineer who says no — you're the engineer protecting Thursday.

Add it to CityOps

Time to make it real. Using the discovery note from the previous post, fill in the one-pager for CityOps v1. Here's a worked start:

In v1	Explicitly out of v1 (with reason)
Daily 311 pull with reconciliation (every record accounted for)	Automated emailed Excel reports — deferred to phase 2; needs scheduling, distribution lists, and Lisa's review of the recipient data
Borough / agency / status filters on the dashboard	Next-week hotspot prediction — refused for v1; Dev flagged the underlying ZIP data as unreliable, and we won't present a model we can't stand behind
48-hour storm overlay from Open-Meteo (cached hourly)	Real-time data — refused; the city's feed publishes daily, so promising "live" would be promising something we don't control
Unresolved-request counts Maria asked for	Resident-facing portal — out of scope; this engagement serves the operations team, not the public

Then write the acceptance criteria so each stakeholder can verify their own piece:

Maria picks any five requests on screen, searches their IDs on the city's public 311 portal, and finds the same five. Tom filters to his borough, clicks Download, and opens the file in Excel without calling anyone. Dev reruns the daily pull and the log's record count matches the dashboard. Lisa reviews the repo, finds no keys or credentials, and sees the written rule: complaint text never leaves our systems without her sign-off.

In Stage 1's milestone you'll do this for real: the discovery note and the scope one-pager become the first two artifacts in the CityOps repo — and every later client interrupt gets answered by opening this page first.

Where this skill transfers

Scoping is one of the most portable skills in customer engineering:

You're at the second stop: Scope. Every later stage returns to this diagram — and when things go wrong in production, the first question is usually "what did we agree to back at Scope?"