SkyWatch #2: Flights Meet Weather

"What was that plane?" is more interesting with "…in 25-knot winds." This post enriches flight snapshots with Open-Meteo weather: per-grid-cell caching, joining two sources with different freshness guarantees, and a dashboard that labels what "live" actually means. The one principle for this post: join on freshness, not just keys.

The customer problem

A flight's story is incomplete without its air. A low approach makes sense in a 30-knot crosswind; a holding pattern makes sense under a storm cell. Priya's readers don't need a meteorology degree — they need one line next to each flight: wind 18 kt from the west, 14°C, raining. The data is free — Open-Meteo needs no API key and answers a single HTTP GET with current conditions for any latitude/longitude. The engineering problem isn't fetching weather. It's joining two live sources that were never designed to agree.

Clarify the ask

• Weather where? Per-flight coordinates would mean one weather call per aircraft — hundreds of calls per poll cycle, burning quota for precision nobody needs. Weather doesn't change meaningfully over 500 meters. So: quantize the map into grid cells (say, 0.5° × 0.5°) and fetch weather per cell, not per flight.
• How fresh? Open-Meteo updates its current conditions roughly every 15 minutes. Fetching more often than that buys nothing but quota burn. The cache TTL should mirror the source's own rhythm.
• What if weather is missing? The flight still shows. Weather is enrichment, not a gate — a dashboard that hides planes because the weather call failed has the dependency backwards.

The minimum concept

One cell, one call, one join. Take the bounding box's center, fetch current weather, attach it to every flight in the snapshot:

import requests

def get_weather(lat, lon):
    # Units are a contract decision, not a default: the dashboard displays
    # knots, so request knots. A reader must never guess the unit.
    r = requests.get("https://api.open-meteo.com/v1/forecast",
                     params={"latitude": lat, "longitude": lon,
                             "current": "temperature_2m,wind_speed_10m,"
                                        "wind_direction_10m,weather_code",
                             "wind_speed_unit": "kn"},
                     timeout=15)
    r.raise_for_status()
    return r.json()["current"]

wx = get_weather(37.6, -122.4)
print(wx["temperature_2m"], wx["wind_speed_10m"], wx["weather_code"])
# 20.1  7.6  2  — degrees C, KNOTS (requested), and a WMO weather code

That works for a demo. It does not work for a dashboard: with 200 aircraft in the box, per-flight weather calls would mean 200 API calls per poll cycle. The grid-cell cache is the whole design:

Build: the enrichment join

# 0.5° at ~38° latitude ≈ 56 km N-S × 44 km E-W. That's a regional
# approximation, not local truth: fine for "what's the weather over
# the bay," wrong for "why did this aircraft go around." For approach
# conditions, airport METAR observations beat a model grid — and surface
# conditions never explain what an aircraft felt at altitude. The grid
# resolution is a use-case decision, and weather enrichment must never
# claim to explain a maneuver.
CELL = 0.5  # degrees; coarse by design — documented, not assumed

def cell_key(lat, lon):
    return (round(lat / CELL) * CELL, round(lon / CELL) * CELL)

def enrich(snapshot, wx_cache):
    """Attach cached weather to every flight. Never block on weather —
    and never let a missing position delete an aircraft."""
    rows = []
    for ac in snapshot.values():
        lat, lon = ac.get("latitude"), ac.get("longitude")
        if lat is None or lon is None or not (-90 <= lat <= 90 and -180 <= lon <= 180):
            # No valid position: keep the aircraft, mark weather unavailable.
            rows.append({**ac, "wx": None, "wx_status": "unavailable",
                         "wx_reason": "no_position"})
            continue
        key = cell_key(lat, lon)
        wx = wx_cache.get(key)  # may be None — that's fine
        rows.append({
            **ac,
            "wx": wx["data"] if wx else None,
            # Two different clocks: the weather SOURCE's observation time
            # (from Open-Meteo's `time` field) vs when WE fetched it.
            # Readers see the observation; operators monitor the fetch.
            "wx_observed_at": wx["data"]["time"] if wx else None,
            "wx_fetched_at": wx["fetched_at"] if wx else None,
            "wx_status": wx_status(wx),
        })
    return rows

# Simplified teaching implementation — production needs: a shared
# requests.Session, single-flight refresh per cell, retry/backoff on 429/5xx,
# a per-cycle cell cap, and schema validation before replacing the cache.
# The invariant that matters: a failed refresh NEVER overwrites a valid entry.
def refresh_weather(cells, wx_cache, ttl=900):
    """Fetch only stale cells. Weather changes slowly; act like it."""
    now = time.time()
    for key in cells:
        entry = wx_cache.get(key)
        if entry and now - entry["fetched_at"] < ttl:
            continue
        lat, lon = key
        try:
            data = get_weather(lat, lon)  # schema-checked before it lands
            validate_weather_schema(data)
            wx_cache[key] = {"data": data, "fetched_at": now}
        except (requests.RequestException, SchemaError) as e:
            log(f"weather fetch failed for {key}: {e}; keeping previous entry")
            # previous entry stays untouched — stale is displayed, not hidden

def wx_status(wx, max_display_age=7200):
    """Display policy: fresh, stale-but-displayable, or unavailable.
    A 45-minute-old observation must never look equivalent to a fresh one."""
    if not wx:
        return "unavailable"
    age = time.time() - wx["fetched_at"]
    if age <= 900:
        return "fresh"
    if age <= max_display_age:
        return "stale"
    return "unavailable"  # too old to show next to a live flight

Two design decisions worth naming:

1. Weather failure never removes flights. The wx field is nullable by design. A dashboard row with wx: null renders as "weather unavailable" — the plane is still there. Enrichment is additive; the day weather is down is not the day the dashboard goes dark.
2. Every enriched row carries two clocks. observed_at (when we saw the flight) and wx_observed_at (when we fetched the weather). The dashboard needs both, because they disagree — and the disagreement is the honest part.

Break: when "live" disagrees with itself

The freshness table for the whole system:

The failure mode this prevents: a reader sees "wind 8 kt" next to a go-around and concludes the pilot erred — when the wind reading is 40 minutes old and the actual gust front arrived 10 minutes ago. Stale enrichment presented as live is misinformation with a dashboard. The fix isn't fresher weather (the source doesn't offer it); it's labeled weather. Every number on the page carries its age, and the page teaches the reader to read the ages.

Productionize: the dashboard contract

The dashboard's data contract, stated the way the validation lesson taught — what the consumer is promised, in writing:

• Every row carries: callsign (nullable), baro_altitude_m (nullable), velocity_ms (nullable), position_observed_at (the flight's source observation time), wx (nullable object), wx_observed_at (the weather source's observation time — from the provider's time field), wx_fetched_at (when we retrieved it), wx_status (fresh / stale / unavailable), wx_grid_cell (the cell used for lookup), wx_source ("open-meteo/forecast"). Three different clocks, each labeled — never one timestamp doing three jobs.
• Rows are never removed because weather is missing. Rows are removed when the aircraft leaves the bounding box or goes stale past a threshold (say, no contact for 15 minutes) — with the threshold documented, not magic.
• WMO weather codes map to plain words ("3" → "overcast") in a versioned lookup table, not inline conditionals — the same "vocabulary is validated, never guessed" rule, applied to a weather API.

Communicate: the reader's mental model

Priya's page gets a legend, not a footnote. Three lines, above the fold:

"Positions update every few minutes from OpenSky's community receivers. Weather refreshes every 15 minutes from Open-Meteo. Each row shows when its data was observed — check the ages before drawing conclusions."

That legend does something most dashboards don't: it teaches the reader how to distrust the page correctly. A reader who knows the wind is 11 minutes old won't blame a pilot for a gust the page couldn't see. Trustworthy software sometimes means teaching the user the limits of what you built.

Prove it: the three-aircraft test

The chapter's promise — every aircraft stays visible, every freshness label honest — is testable. Three fixtures:

def test_freshness_labels():
    rows = enrich(snapshot_three_aircraft(), wx_cache_three_states())
    by_id = {r["icao24"]: r for r in rows}
    assert len(by_id) == 3, "no aircraft may disappear"
    assert by_id["A"]["wx_status"] == "fresh"
    assert by_id["B"]["wx_status"] == "stale"
    assert by_id["C"]["wx"] is None and by_id["C"]["wx_status"] == "unavailable"

All three visible, all three labeled correctly. That's the join done truthfully, not just successfully.

Next: SkyWatch #3 — operating the dashboard. Quotas exhaust, APIs go down, data goes stale, and the 2 AM page doesn't care that the API is free.