CityOps Under Fire

CityOps is live — Compose stack, intake pipeline, CityOps API, the Ask CityOps copilot, audit trail, runbook. This is the week it all gets attacked: by its own suppliers, its own growth, and its own users. Six incidents, one trial. Triage, cut scope, redesign, communicate, ship — and at the end, a scorecard that grades judgment, not heroics. The one principle for the whole chapter: production green is not success. Adoption is the next test; customer outcome is the destination.

The rules of the trial

Before the first inject, the rules — because a trial without rules is just suffering. Each incident below follows the same shape:

• The inject — the stakeholder message, verbatim. Read it the way you'd read it at 6 AM: what is actually being asked?
• What "done" looks like — the acceptance line, written before any fix. If you can't state done, you can't stop.
• The worked example — a strong response: triage commands, the fix, the output. This is what disciplined looks like.
• The trap — the fancy fix that loses to the disciplined one. Every incident has one.

The week, as scheduled:

Grading follows the curriculum's milestone scorecard — technical correctness, data reconciliation, reliability, security & compliance, time-to-value, customer outcome, scope discipline, communication, operability, adoption. The full rubric is at the end of this chapter. One rule of the trial, stated up front: the technically fanciest solution sometimes scores worse than the boring one. The scorecard grades the customer outcome, not the architecture.

The minimum concept: the triage protocol

Every incident in this chapter — and, Dev would argue, every incident in the job — runs through one protocol, in this order. The order is the concept:

Three things about this order that the week will keep proving. First, scope before cause: knowing "the API is fine, only new ingestion is delayed" changes what Maria says at the demo more than knowing why it's delayed. Second, data safety before fix: "nothing is lost, it's late" and "records are corrupted" demand completely different responses, and you cannot choose the response until you know which one you're in. Third, containment can precede cause: when data is actively crossing a boundary it shouldn't — PII reaching an external model, corrupt writes landing in a table — you stop the calls first and investigate after. The protocol orders the thinking, not the hands: containment is a reflex, cause is the diagnosis. Don't read the sequence as "never mitigate until the root cause is known." The protocol is a checklist against panic — triage is a protocol, not a personality.

Incident 1: Lisa blocks the model boundary (Monday)

The spot check, reproduced. This is the input Lisa's team sent:

spot = "Call me at (555) 010-0199 — Maria Santos, 44 Elm St, about my trash."
clean, tags = redact_pii(spot)
# clean: "Call me at (555) 010-0199 — Maria Santos, 44 Elm St, about my trash."
# tags:  []   <-- nothing redacted. The parens format, the name, and the
#              address all sailed through the three-pattern teaching redactor.

The honest reading: the Milestone 4 redactor was always labeled a teaching redactor — three patterns, minimize-first as the production principle. The copilot has since moved from a stubbed generator to a real external model API, and the boundary it crosses is now a legal boundary, not just an architectural one. Lisa isn't asking for better regexes. She's asking for a boundary that fails closed.

What "done" looks like (written before any code, agreed with Lisa): only minimized text that passes the approved boundary policy crosses the external model boundary; detected or uncertain sensitive content is refused and queued for human review; every crossing and refusal is audited with a payload hash; the Thursday demo runs on the new boundary. This reduces risk but does not prove an imperfect detector catches every possible identifier. Who decides: Lisa, by spot-check.

Worked example: minimize, then gate, then audit

The redesign has three layers, in priority order — and the order matters more than any one layer:

Layer 1 — minimize. The model needs the question and the retrieved procedures. It does not need the resident's name, address, or the raw complaint thread. The first fix is architectural, not textual: stop sending the complaint. Send only the redacted question plus retrieved chunks:

def prepare_model_input(question, chunks):
    # MINIMIZE first: the external model receives the question and the
    # retrieved procedures — never the raw complaint thread. What never
    # crosses the boundary can't leak across it.
    clean_q, tags = redact_pii(question)
    return clean_q, chunks, tags

Layer 2 — fail closed. Whatever the redactor can't clear must not cross. A broader detector (names, addresses, the phone formats Lisa found) runs after redaction; anything still PII-shaped is refused and queued for human review:

def model_boundary_check(clean_q, chunks, tags):
    leftover = find_pii_shapes(clean_q)  # broader detector: names,
                                         # addresses, phone variants
    if leftover:
        audit("model_boundary_refused",
              {"shapes": sorted(leftover), "redactions": tags})
        queue_for_review(clean_q, leftover)
        raise BoundaryRefused(
            "PII-shaped content blocked from the model boundary; "
            "queued for human review")
    payload = {"question": clean_q, "chunks": chunks}
    audit("model_boundary_crossed",
          {"payload_sha": sha256_of(payload), "redactions": tags})
    return payload

One boundary subtlety worth stating: queue_for_review(clean_q, leftover) deliberately places sensitive text into another system. That's acceptable only because the review queue is an authorized sensitive-data store with access and retention controls — staffed, with an SLA, per Lisa's line at the end of this incident. Refusing model transmission does not mean the data stops being sensitive; it means the data moved to a boundary built to hold it. This is the data-boundary thinking from the security lessons, applied to your own tooling.

The run against Lisa's spot check:

>>> model_boundary_check(*prepare_model_input(spot, chunks))
audit: model_boundary_refused  shapes=['phone_variant', 'person_name', 'address']
BoundaryRefused: PII-shaped content blocked from the model boundary;
                 queued for human review
# And the clean question — "about my trash", no PII — crosses normally:
>>> model_boundary_check(*prepare_model_input("When is yard waste collected?", chunks))
audit: model_boundary_crossed  payload_sha=9f2c…a41d  redactions=[]

Layer 3 — extend the redactor with the formats Lisa's team actually found (parenthesized phones, +1 prefixes), so fewer legitimate questions hit the review queue. This is layer 3, not layer 1: detection improves coverage and throughput; minimization shrinks the problem the detector must solve, and the gate enforces the policy decision on whatever the detector sees.

Be precise about what this proves, because Lisa will ask: the refusal path is proven (the audit event shows the block and the queue entry). The detection is not proven complete — a novel PII shape could still evade the detector and be labeled clean, in which case the gate opens. The gate enforces the decision; it cannot make an imperfect detector omniscient. And the payload hash is evidence of identity, not safety: it proves this exact payload was the one audited — it says nothing about whether the payload contained PII, and a hash is only as durable as the audit store behind it. Immutability comes from the store's protection, not from the hash alone.

Thursday's demo runs on the new boundary. Lisa spot-checks with five new complaints — two refuse-and-queue, three cross clean. She signs off, with one line for the permanent record: "The review queue is a control, not a backlog — staffed, or it's theater." The queue gets an owner and an SLA before the demo ends.

Incident 2: Tom needs the borough pack in 3 days (Tuesday)

Before designing anything, clarify the ask — because "CSVs for the meeting" hides three different products. Tom is asked one question: who opens this file, and what do they do with it? Answer: the borough president's staff, who paste the numbers into a slide deck. They need meeting-ready rollups, not raw rows and not a new dashboard. That answer kills two scope-creep candidates on the spot: no interactive report builder, no scheduled email delivery. One file, the right columns, generated from a repeatable endpoint.

What "done" looks like: one CSV (plus a small detail file) with exactly the columns the meeting needs, produced by a versioned endpoint that reuses the existing filtered-query layer; a tradeoff memo listing what was explicitly refused. Who decides: Tom, by opening the file Thursday.

Worked example: one query, one endpoint, one memo

The pack is two queries over the reports table — the existing query layer supplies the authorized filters, the new parts are only the grouping and the ranking:

-- borough_pack.sql: the meeting rollup in one query.
-- Runs INSIDE the authorized borough scope: the query layer applies the
-- session's borough filter before this aggregation — the SQL below is the
-- aggregation portion only. Aging buckets are cumulative thresholds:
-- over_30d is a subset of over_7d ("older than 7", "older than 30"),
-- which is what the meeting wants — not 8–30 / 31+ bands.
SELECT b.borough,
       r.service_code,
       COUNT(*) FILTER (WHERE r.status = 'open')                 AS open_reports,
       COUNT(*) FILTER (WHERE r.status = 'open'
                         AND r.age_days > 30)                  AS over_30d,
       COUNT(*) FILTER (WHERE r.status = 'open'
                         AND r.age_days > 7)                   AS over_7d
FROM reports r
JOIN boroughs b ON r.borough_id = b.id
GROUP BY b.borough, r.service_code
ORDER BY b.borough, open_reports DESC;

-- borough_pack_top_streets.sql: top-5 streets per borough, as the small
-- detail file. "Top" means most reports — so rank by count first, then
-- take five. Kept separate from the rollup so the main query stays
-- readable and the ranking stays testable on its own.
SELECT borough, street, reports FROM (
  SELECT b.borough AS borough,
         r.street  AS street,
         COUNT(*)  AS reports,
         ROW_NUMBER() OVER (PARTITION BY b.borough
                            ORDER BY COUNT(*) DESC) AS rn
  FROM reports r
  JOIN boroughs b ON r.borough_id = b.id
  WHERE r.status = 'open'
  GROUP BY b.borough, r.street
) ranked
WHERE rn <= 5
ORDER BY borough, reports DESC;

The endpoints reuse the Milestone 3 export machinery (same authorization, same streaming response) — the new code is the queries and the routes:

@app.get("/reports/borough-pack")
def borough_pack(auth=Depends(require_authorization())):
    # Reuse the existing authorization dependency and authorized query
    # scope: borough_ids come from the session, never from the request.
    # The SQL above is the aggregation portion, applied inside that scope.
    rows = run_query(BOROUGH_PACK_SQL, {"auth": auth})
    audit("report_generated", {"report": "borough-pack",
                               "rows": len(rows), "by": auth.user})
    return stream_csv(rows, filename="borough-pack.csv")

@app.get("/reports/borough-pack/top-streets")
def borough_pack_top_streets(auth=Depends(require_authorization())):
    # Same authorized scope, second file: the detail Tom pastes under
    # each borough's headline numbers.
    rows = run_query(BOROUGH_PACK_TOP_STREETS_SQL, {"auth": auth})
    return stream_csv(rows, filename="borough-pack-top-streets.csv")

The output Tom opens Thursday:

borough,service_code,open_reports,over_30d,over_7d
North,POTHOLE,412,88,201
North,MISSED_TRASH,305,41,133
South,POTHOLE,388,102,176
...
# borough-pack-top-streets.csv (the detail file):
borough,street,reports
North,Elm St,96
North,3rd Ave,71
...

And the tradeoff memo — the artifact that makes "no" a deliverable. It lists what was asked for or imagined and explicitly refused, with the reason:

Thursday EOD, Tom downloads the file, pastes the North-borough pothole line into his deck, and the meeting runs on CityOps numbers for the first time. Keep that fact — it matters in Incident 6.

Incident 3: the weather API collapses to 10% (Wednesday AM)

Run the protocol. Scope: the API is fine, 311 ingestion is fine — only the enrichment step is failing. Cause: 429s from the weather provider; quota cut to a tenth, unannounced. Data safety: nothing is lost or wrong — reports are complete except for one advisory field. Which answers Tom's question before it's fully asked: no, you don't pause ingestion. Pausing the product because a garnish failed is the incident, not the response.

What "done" looks like: every 311 report still ingests on schedule; reports carry weather with explicit provenance — weather_status ("fresh", "cached", or "unavailable") plus weather_observed_at — so no consumer can mistake degraded context for clean data; no retry storm burns the remaining 10% quota; the remaining quota is reserved for the workloads the customer has explicitly prioritized (Maria chose the Thursday demo queries); an alert fires if degradation lasts past the provider's stated recovery window. Who decides: the data contract — downstream consumers must be able to tell "no weather data" from "clear skies."

Worked example: degrade on purpose

The enrichment worker gets a degradation policy — three branches, no surprises:

def enrich_with_weather(report, weather_client, cache):
    # Enrichment is ADVISORY. The 311 record is the product; weather is
    # context. If the API is sick, the record still ships — with provenance.
    try:
        w, observed_at = weather_client.get(report["date"], report["borough"])
        return {**report, "weather": w, "weather_status": "fresh",
                "weather_observed_at": observed_at}
    except RateLimited:
        # 10% quota: do NOT burn it retrying. Use a cached observation
        # only if it actually corresponds to this report's date/borough —
        # yesterday's weather is not today's weather, and a stale
        # substitution would make the data wrong, not just degraded.
        hit = cache.matching(report["date"], report["borough"])
        if hit is not None:
            audit("weather_degraded",
                  {"report": report["id"], "fallback": "cached"})
            return {**report, "weather": hit.value,
                    "weather_status": "cached",
                    "weather_observed_at": hit.observed_at}
        audit("weather_degraded",
              {"report": report["id"], "fallback": "unavailable"})
        return {**report, "weather": None, "weather_status": "unavailable",
                "weather_observed_at": None}

The worker log at 7:30, after the deploy:

07:31 enrich: 429 from weather API (quota 10%) -> degraded mode
07:31 enrich: report 10412 -> weather_status=cached, observed_at=2026-10-06
07:31 ingest: 47/47 reports stored, 47 with weather_status != fresh
07:31 alert: weather degradation > 60min -> page Tom (sustained, not transient)

The load-bearing detail is the provenance. A boolean flag can't carry what the consumer needs to know: weather_status distinguishes fresh from cached (with when it was observed) from unavailable — four different facts a single bit would collapse into one. A downstream query asking "pothole reports on rainy days" can now decide: fresh rows for the correlation, cached rows with their observation dates for context, unavailable rows excluded. Without that distinction, "no weather data" silently becomes "clear skies" and someone briefs the borough president on a correlation that doesn't exist. Degraded data must be distinguishable from clean data, or the degradation is a corruption. That's a data-contract rule, and it goes into the contract the Pydantic lesson taught: the schema now documents what each status value means.

Productionize note: the provider restores the quota Thursday. The permanent changes: the weather call gets a quota budget (the limiter that controls the allowed request rate), a cache (fewer calls), and a circuit breaker (stop calling when the dependency is failing — a different job from the budget, not a replacement for it); the weather_status and weather_observed_at columns become permanent, with contract documentation; and the runbook gains a "third-party degradation" page — because the next 10% collapse won't announce itself either.

Incident 4: the evals fall below Maria's line (Wednesday PM)

This is the incident the whole evals lesson was building toward. The instinct — Dev's stated options — is to change the system or change the bar, tonight, under pressure. The protocol says: scope, cause, data safety first. Scope: which eval dimension failed? Only retrieval hit-rate; refusal, citations, safety, and latency are green. Cause: unknown — nothing in the system changed. Data safety: no production impact; the gate did its job and blocked the release.

What "done" looks like: the cause of the regression is identified with evidence (system, eval set, or measuring stick); either the system is fixed through the normal pipeline, or the threshold is renegotiated with Maria with the per-slice evidence on the table — never silently. Who decides: Maria, as the domain owner who signed the original line.

Worked example: version all three, then talk

The evals lesson's durable rule — version the system, the eval set, and the measuring stick — is the diagnosis procedure. Run it:

$ git log --oneline -- evals/golden.json | head -3
a91f3c2  add 40 production-log questions to golden set   # <-- the change
7d44e10  release v3: acceptance 0.85 (Maria sign-off)
$ eval_report --slice original-120
  retrieval hit-rate: 0.88   (floor 0.85 — PASSES)
$ eval_report --slice new-40
  retrieval hit-rate: 0.62   (never reviewed, never signed off)

No regression is observed on the original signed-off slice. The measuring stick changed: 40 new questions, pulled from production logs, were added to the golden set without domain-owner review — and they're harder in a specific way. Initial inspection suggests they use long, conversational phrasings the TF-IDF-era chunker was never tuned for; root-cause work is scheduled after the domain review, because evals detect failure and diagnosis locates cause. The original 120 questions — the set Maria actually signed off on — still pass at 0.88. And note what the new slice might be telling us: production-derived questions could reveal a real capability gap in current production traffic. That's a separate, unresolved question — not a cleared one.

So Dev doesn't tune the retriever tonight (an untested change before a demo, chasing an unreviewed bar), and nobody edits the threshold in CI to make it green. The worked response is a conversation with evidence. Dev brings Maria the slice breakdown and two honest options:

Maria agrees. The demo runs on the signed-off set. The review session goes on the calendar for next week, owned by Maria — because the golden set's definition of correct behavior is owned by the domain expert (she decides which questions and expected outcomes are representative and acceptable), and engineering owns the machinery that maps retrieval metrics onto that contract. That's the evals lesson's split, applied under pressure instead of in a textbook.

And the 40 questions don't disappear: the dashboard gains a second eval line that week — the signed-off set (gating, 0.88) and the production slice (reported, 0.62, unresolved). The 0.62 doesn't gate the release, but it doesn't get to hide either. Running the demo on the signed-off set while reporting the new slice separately is governance; letting the bad number vanish behind a redefined denominator would be the midnight-commit behavior the trap section condemns.

Incident 5: dawn triage — the quarantine spike (Thursday)

It is and it isn't. The Milestone 5 incident was a field rename — category became service_code — and the machinery built after it (tolerant parser, quarantine queue, hourly contract test) is exactly what's working now: nothing crashed, the records were set aside instead of lost, and Tom saw a count instead of a flatline. But the contract test asserts field presence, and this time the field is present. Its values changed.

Run the protocol. Scope: API fine, dashboard fine, 214 reports quarantined — the borough pack Tom built Tuesday used yesterday's data, so the meeting numbers are unaffected. Cause (one sample):

$ psql -c "SELECT raw->>'service_code' AS code, COUNT(*)
           FROM quarantine
           WHERE created_at > now() - interval '12 hours'
           GROUP BY 1 ORDER BY 2 DESC LIMIT 5;"
 code | count
------+-------
 312  |   214
# Yesterday: 'POTHOLE', 'MISSED_TRASH', ...  Today: numeric codes.
# Same field name. Different language. The contract test checks names.

Data safety: late, not lost, not wrong — quarantine did its job. The fix is a mapping, not a rescue.

What "done" looks like: the 214 records mapped and backfilled before the 2 PM demo — 214 quarantined, 214 mapped, 214 accepted, 0 unresolved; the mapping lives in a versioned table with effective dates, not in parser code; the contract test gains a value-domain monitor (a drift signal, not a presence assertion); a noon note (not a war-room postmortem — the machinery worked) on Maria's desk. Who decides: the demo clock.

Worked example: map it, version it, monitor the value domain

The fix is small because the architecture is right. A versioned mapping table — with effective dates, because code 312 means POTHOLE only under codebook v3. If the city ships v4 with 312 reassigned, each record must map under the codebook version in force when it arrived; without effective dates, a versioned table exists but the application can't know which version applies. And the next codebook change shouldn't require a code deploy to understand:

-- service_code_map: the city's vocabulary, versioned with effective dates.
-- v3 added 2026-10-08 after the numeric-code switch; source is the official
-- codebook, confirmed with the named data owner — a phone call alone is not
-- durable mapping authority.
INSERT INTO service_code_map (code, name, version, effective_from, effective_to, source) VALUES
  ('312', 'POTHOLE',      3, '2026-10-08', NULL, 'city codebook 2026-10; confirmed with the data owner'),
  ('318', 'MISSED_TRASH', 3, '2026-10-08', NULL, 'city codebook 2026-10; confirmed with the data owner');
-- Backfill with reconciliation: 214 quarantined → 214 mapped →
-- 214 re-ingested and accepted → 0 unresolved. Queue zero alone proves
-- nothing; the counts must tie.
$ psql -c "SELECT COUNT(*) AS quarantined FROM quarantine
           WHERE reason='unmappable service_code';"
 quarantined
-------------
         214
-- (map + re-ingest)
$ psql -c "SELECT COUNT(*) AS accepted FROM reports
           WHERE service_code_origin='quarantine-backfill-2026-10-08';"
 accepted
----------
      214
$ psql -c "SELECT COUNT(*) AS unresolved FROM quarantine
           WHERE reason='unmappable service_code';"
 unresolved
------------
          0

And the contract test grows a second sense — a value-domain monitor. Presence checks catch renames; the monitor watches the value domain for semantic drift. It raises for investigation; it doesn't assert. A statistical distribution change is a drift signal, not a contract violation:

def value_domain_monitor():
    # A DRIFT SIGNAL, not a contract. Asserting "yesterday's top-10 values
    # must all appear today" would false-positive on normal variation —
    # a quiet graffiti day is not a schema change.
    vals = today_values("service_code")
    unknown_rate = sum(1 for v in vals if v not in KNOWN_DOMAIN) / len(vals)
    coverage = len(set(KNOWN_DOMAIN) & set(vals)) / len(KNOWN_DOMAIN)
    if unknown_rate > UNKNOWN_RATE_ALERT or coverage < COVERAGE_FLOOR:
        raise_for_investigation(
            f"value-domain drift: unknown_rate={unknown_rate:.3f}, "
            f"known_coverage={coverage:.2f}")

By 11:30 the backfill is done and the dashboard is current. The noon note is one page — what changed, what the mapping table is, the new monitor — and Maria forwards it without editing. Note the difference from Milestone 5's postmortem: that one documented a system being brittle to a normal event. This one documents a system absorbing a normal event. The quarantine queue, the versioned mapping table, the drift monitor — that's what "productionize" bought. And one sentence in the noon note is worth its weight: quarantine converted a potentially wrong interpretation into merely late data. That's why the queue matters.

The 2 PM demo runs on current data. Nobody in the room hears about the 214 records — and that's the point worth naming: the best incident response is the one the customer never needs to hear about. Not because it was hidden, but because the system absorbed it before it became their problem.

Incident 6: "The system works. Nobody uses it." (Week 3)

This is the incident with no pager and no stack trace, which is why it's the hardest one in the chapter — and the most valuable. Everything is working. That's the problem. The triage protocol still applies, but the instruments change: instead of logs, you investigate with people.

What "done" looks like: a written diagnosis naming the specific frictions (ranked by evidence, not opinion), one change shipped to remove the top friction, and a re-measurement date. Explicitly not done: a feature roadmap. Who decides: Maria, against the next usage pull.

Worked example: measure the funnel, then ask humans

Start with what the audit trail already knows — the funnel, not the total:

-- Scenario constants: audit-derived, week of 2026-10-19.
-- logins: 41 | dashboard_view: 31 | ask_or_export: 23  (of 128 staff)
SELECT event, COUNT(DISTINCT user_id) AS users
FROM audit
WHERE event IN ('login','dashboard_view','ask','export')
  AND ts > now() - interval '7 days'
GROUP BY 1;
-- The drop is steepest at login -> dashboard_view. 87 of 128 had no
-- recorded login that week: the metric is absence, not friction.
-- The interviews identify the friction.

Then the human instrument: three 20-minute interviews and one shadowed shift. Initial qualitative findings, prioritized using funnel size plus interview evidence:

1. The door is heavy. CityOps needs VPN plus a separate login; the old workflow was a spreadsheet on the shared drive. 87 of 128 staff had no recorded login that week — they may have been off shift, on leave, using another authorized workflow, or simply unaware of CityOps. The metric records absence, not friction. The interviews then identified VPN + separate login as a major access friction for those who did try — and "I'll check it later" is how 18% happens. Evidence first.
2. The meeting runs on slides. The weekly ops meeting — the one place these numbers matter — runs on a slide deck. The borough pack exists (Incident 2 built it), but nobody showed the ops leads the one-click path from dashboard to deck. The export they need is three clicks away and nobody knows it. Workflow mismatch, not missing feature.
3. The copilot's first impression was the pothole miss. The earliest story anyone heard about Ask CityOps was the eval failure from the pilot — "it couldn't answer a pothole question." Nobody saw the refusal behavior, the citations, or the blocked release that followed. Trust debt, not a trust problem.

Notice what the investigation didn't find: nobody asked for a new feature. Every friction is about the path to the existing value, not the value itself. So the plan is one change, not a roadmap: a 20-minute floor session — Dev walks the ops leads through the one-click borough pack (their meeting, their deck, their numbers) and live-demos the copilot refusing an unanswerable question with citations on the answerable ones. The VPN+login simplification goes on the backlog as the next friction, owned and dated — because the door is the biggest drop, but the session is the fastest win.

The sequencing rule worth naming: prioritize not only by impact, but by evidence, reversibility, effort, and speed to learning. The door is the biggest drop in the funnel; the floor session is the fastest validated learning — it attacks the second and third frictions immediately, and a visible win builds the political capital for the slower, riskier auth work. Biggest drop ≠ first change.

The re-measurement is set for two weeks out, with the honest caveat stated up front: a before/after comparison can't isolate the session from everything else that happens in two weeks. The team records the confounders they know about (the borough meeting cycle, a staffing change in the South office) alongside the number. Measure anyway, and stay honest about what the measurement can't prove — that's the difference between evidence and theater.

One more honesty upgrade: "23 touched CityOps" is reach, not adoption. The ladder is reach → activation → meaningful workflow use → repeat use → customer outcome. A login isn't adoption; neither is one dashboard view. Tom's meeting running on CityOps numbers back in Incident 2 is the first genuine customer outcome in this chapter — that's the rung that counts.

Communicate: the Friday briefing

Friday, 4 PM. Dev gets five minutes with Maria — the whole week, compressed into the format a customer can actually use. Not a war story: a status. What broke, what's true now, what changed permanently, what's next:

Study the shape of this briefing, because it's the skill the chapter is really teaching. Every item follows the same grammar: what happened, what's true now, what changed permanently. No item ends at "we fixed it" — each one ends at the mechanism that makes it cheaper next time. And the adoption number is reported with the same neutrality as the 429s: bad news, delivered early, with a plan attached. The briefing is a deliverable — Maria forwards its shape to her leadership, the way she forwarded the postmortem.

The scorecard: grading the trial

The curriculum's milestone scorecard, applied to the week. Each dimension names what "good" looked like this week — and the trap it punishes:

Read the right column as a whole: every trap is the same mistake wearing different clothes — optimizing for impressiveness over outcomes. The scorecard's central lesson, stated one final time: the best solution isn't the most impressive architecture — it's the smallest reliable system that creates the required customer outcome and survives production. This week, that meant a gate instead of a detector, a query instead of a framework, a conversation instead of a midnight commit, a mapping table instead of a dict, and a floor session instead of a feature.

CityOps: the job, experienced

Step back and name what the trial proved. Not that CityOps works — the milestones proved that. That it survives: its suppliers (the weather quota, the codebook), its own growth (the eval set, the model boundary), and its users (the 18%). Each incident converted a lesson into a system property — the boundary gate, the tradeoff memo, the degradation provenance, the versioned golden set, the value-domain monitor, the adoption funnel. Productionize, across the whole chapter, meant the same thing it meant in every lesson: turn the lesson into a mechanism.

And the arc of the full curriculum lands here. Stage 1 taught you to scope the vague ask. Stage 2 taught you to move data without losing it. Stage 3 taught you to integrate without trusting. Stage 4 taught you to put contracts around intelligence. Stage 5 taught you to operate what you shipped. Stage 6 taught you to present the evidence. This chapter was the exam for all of it — and the exam's real subject was never the incidents. It was the judgment: what to fix, what to refuse, what to measure, and what to tell the customer on Friday at 4 PM.

If the whole curriculum had to fit on one card, it would be this — the synthesis of every incident in the chapter:

Five lines to leave with. Triage is a protocol, not a personality. Define "done" before you start fixing. Under pressure, the smallest safe change usually beats the most impressive architecture. Every incident should leave behind a mechanism that makes the next occurrence cheaper. And the last one, upgraded: production green is not success — adoption is the next test, customer outcome is the destination.