Missing Is Not Zero

Game Combiner warns you when a multiplayer game's community has left. Improving that warning produced a version that flagged Overwatch 2 as dead while 61,499 people were in game. The bug is one of the oldest in data work and it still got us.

Recommending an online-only game whose servers are empty is a bad experience. So each game carries a dead_mp flag: multiplayer, not single-player, almost nobody playing, enough reviews that the silence means something. The flag shows a warning and applies a small scoring penalty.

It was firing on 866 games out of 142,000. That seemed low.

Why it barely worked

The flag needs a player count, and our bulk dataset carried one for only 19,689 of 136,712 games. The other 86% sat at zero — not because nobody played them, but because nobody had told us. Games with no player data mostly weren't detected as multiplayer either, so they quietly fell out of the calculation instead of being flagged wrongly. Broken, but harmlessly.

Then we scraped real tags for 36,971 games. Suddenly thousands more games were correctly identified as multiplayer — and they all still had a stored player count of zero.

The result

Recomputing the flag produced 3,288 dead games, up from 866. That looked like a fix. Sorting by review count to sanity-check the biggest ones did not:

Overwatch 2                      415,947 reviews   ccu=0   FLAGGED DEAD
ARC Raiders                      411,327 reviews   ccu=0   FLAGGED DEAD
PEAK                             342,622 reviews   ccu=0   FLAGGED DEAD
THE FINALS                       273,872 reviews   ccu=0   FLAGGED DEAD
Borderlands 4                     82,260 reviews   ccu=0   FLAGGED DEAD

Steam's own API, asked directly, reported 61,499 people playing Overwatch 2 right then.

The code said (ccu or 0) < DEAD_CCU. A missing value became zero, zero is less than the threshold, so the game was declared dead. The database couldn't distinguish "we measured this and nobody is playing" from "we never asked."

The other trap, one layer down

Picking a data source had its own version of the same mistake. SteamSpy exposes a ccu field, and it looked authoritative: for Rust it returned 143,870, matching our stored value exactly. Identical to the digit. That felt like proof.

It was a coincidence of an old, popular, long-stable title. For Overwatch 2, SteamSpy reported ccu=0 and owners: 0..20,000 — confidently, in a well-formed response, while tens of thousands of people were in game.

A single matching data point is not validation. It's one matching data point. We switched to Steam's own GetNumberOfCurrentPlayers, which is authoritative because it's Valve reporting on Valve's own servers.

What actually fixed it

Not a better threshold — a different contract. The recompute now takes a map of games actually measured in this run:

live = measured.get(appid)
if live is None:
    continue        # unmeasured: leave the flag alone, never assume dead

A failed API call now means "we don't know," which is the truth. We polled all 10,542 eligible multiplayer games directly and found 2,769 genuinely empty ones:

FLAGGED     Z1 Battle Royale  6 players    Warface: Clutch 6    Blackwake 2
NOT FLAGGED CS2 642,068       PUBG 344,375  Dota 2 474,699      Overwatch 2 61,499

One nice confirmation that the fail-safe works: Apex Legends' lookup didn't return a valid result, so it went unmeasured — and stayed unflagged rather than being guessed at.

The bit that generalises

This class of bug survives because the wrong behaviour is indistinguishable from the right one until you look at specific rows. The flag count went from 866 to 3,288 and that felt like progress. Only checking which games were flagged, against the world, exposed it.

Then it came back. Wiring the same job into weekly automation, we capped the sweep to keep the run bounded — and the recompute cleared flags for every game it hadn't re-checked. Same mistake, new location: unmeasured being treated as measured-and-fine. The capped run would have silently wiped all 2,769 flags down to whatever it happened to look at that week.

If your schema can't tell absent from zero, some code path will eventually conflate them, and it'll happen most easily right after you improve something else.