Briefing hub for the UN System Data Commons hackathon project
Last updated: 2026-09-14 · Team: Devin, Henry, Olivia · Repo
Eleven Pairs, Three Charts — three comparisons that hold with provenance and grade on every card, then the eight that fail and why. Each card also places NYC among every reporting country in 2019. Public, no sign-in.
The headline finding: NYC ranks #5 of 186 on city air quality, #18 of 196 on road deaths (four times safer than the US), and #85 of 137 on homicide. Against the US, NYC’s homicide trend reads as a success story; against the world it sits in the bottom half. Same number, different comparator, opposite conclusion.
Builders’ Day — Tue 22 Sep 2026, 10:00–18:00, Google NY (HUD315). UN Data Commons goes fully public 17 Sep; we are building against staging until then.
| When | What |
|---|---|
| 10:00–12:30 | Everyone demos. 5–15 min, single track — everyone sees everything. |
| 14:00–16:00 | Facilitated groups: evidence provenance · agent tooling on SDMX · trust & QA gates |
| 16:00–18:00 | Public showcase to senior UN managers, during UNGA week |
~40 builders: Google Data Commons engineers, UN innovation staff, NY tech-for-good and academic teams. The brief is explicit — bring something you built, and show what surprised you.
The afternoon themes are almost exactly what this project has been doing: provenance on every number, and a QA gate that a human judgment can veto.
If you are an AI assistant: this page is the project’s shared memory. Read it in full before answering questions about the project. It is current as of the date above — check the repo’s recent commits if that date looks stale. Deeper material is linked at the bottom with full URLs you can fetch. For what happened when, fetch the activity feed.
This page is current state and gets overwritten. The activity feed is append-only history. Read this to know what is true now; read the feed to know how it got that way.
A live Voluntary Local Review workbench for NYC government staff.
New York City published the world’s first Voluntary Local Review of the SDGs in 2018. It is a PDF, produced every few years, by a small team. Meanwhile NYC Open Data and the Mayor’s Management Report publish the same underlying indicators continuously, and the UN System Data Commons now holds the authoritative global series for the same concepts. Nobody has connected the two for the person who actually needs it: the analyst at DOT, DOHMH, HPD or OMB writing a budget justification, a council testimony, or a program review.
Three layers:
These are probe results against the live deployment, not documentation claims. Full detail and reproducible commands: platform probe findings.
1. The graph is national-level. NYC has no UN data. NYC resolves as an entity
(geoId/3651000, type City) but appears in no variable’s placesWithData. Scope a search
to NYC alone and you get zero variables — even for “total population”. There is no “look up NYC
in UN Data Commons”. The crosswalk is therefore the product, not a feature of it.
2. MCP is governed; REST is federated. Every variable returned by the MCP tools is
undata/-prefixed. The REST endpoint answers for other publishers’ variables without
warning, so it can hand back a non-UN number that is structurally identical to a UN one.
Rule: discover through MCP; use REST only for structural walks from an undata/ id.
3. Series density varies wildly, and it will kill a naive demo. SDG 3.6.1 (road traffic deaths) returns one observation for the US, in 2021. UNICEF population returns 26 annual points. This is why the coverage probe exists.
4. Provenance is first-class and machine-readable. Every observation carries
provenanceUrl, observationPeriod, sourceId and a unit DCID that encodes the
denominator. That makes part of the comparability check automatic rather than hand-curated.
From the latest coverage report — 30 GREEN, 9 AMBER, 5 RED:
| Strong candidates | Why |
|---|---|
| Homicide rate (16.1.1) | 21–31 annual points back to 1990; NYC has excellent matching data |
| Municipal waste recycled (11.6.1) | 19 points 2000–2018; pairs with DSNY diversion rate |
| PM2.5 (11.6.2) | 10 points; and it has a URBANIZATION--DOU_CITY slice |
| Renewable energy share (7.2.1) | 34 points 1990–2023 |
| Unemployment (8.5.2) | ILO series, dense, sub-annual |
The PM2.5 finding is the interesting one: there is no NYC, but there is a “city degree of urbanization” slice. That is the closest the UN data comes to a city-level comparator, and it appears across several variables. Worth exploring — it may let us say “NYC vs US cities as a class” honestly.
Road safety is out as our headline demo. It was the example in our application. One data point cannot carry it.
689 base SDG indicators in the goal framework. Screened against a six-country panel: 442 usable, of which 130 are indicators the United States does not report and 67 of those are city-scoped — mostly municipal and hazardous waste, wastewater and groundwater. A city can be compared internationally on exactly the indicators its own country skips. The 11 hand-built pairs are 1.6% of the corpus and 4% of the usable pool — so the crosswalk is a verified sample, not a representative one.
The probe/ pipeline now enumerates, screens and proposes candidates from the corpus rather
than from intuition. Its matching stage uses embedding search over the full 2,400-dataset NYC catalog: on seven
hand-verified pairs it puts the correct dataset at median rank 23, against 1535 for the
keyword matcher it replaced. Still a shortlist for human review, never mappings — about 30% of
real mappings depend on what is inside a dataset (NYC’s homicide series is offence code 101
inside “NYPD Complaint Data Historic”) and no text method can find those.
11 indicator pairs mapped; 3 can honestly share an axis (PM2.5, homicide, waste recycling). The other 8 fail for reasons worth knowing: mismatched age bands, denominators that live in a different dataset, concepts that share a name and measure nothing alike, ceiling effects, and one Tier 1 indicator with no NYC source. Recording why is the product — a crosswalk showing only the easy pairs would be the thing we are building against.
Road traffic deaths is blocked by the UN side, not ours: NYC has daily Vision Zero data, the UN has one 2021 modelled estimate. For the city’s flagship street-safety programme the international comparison does not exist.
URBANIZATION--DOU_CITY slice lets us
compare NYC to the city aggregate of other nations — same source, method and unit — but a
scan of 5,320 variables found it on only five indicators (PM2.5, built-up area per capita,
population, two food-insecurity measures). They cluster in gridded-geospatial families; the
administrative indicators have no spatial dimension and will not gain one. About eight more
carry a coarser urban/rural cut of little NYC relevance. The general case remains
NYC-vs-nation, so the comparability grade carries most of the weight. See the
activity feed.CENSUS_API_KEY — never
committed; this repo is public. Known hole: no ACS 1-year release for 2020, so 2020
carries no rate, on the year NYC homicides spiked. Left empty, not interpolated.SKILL.md MCP resource, mirroring the platform’s own convention| What | Where |
|---|---|
| Demo — Eleven Pairs, Three Charts | https://sarapis.github.io/undatacommons-nyc/demo/benchmarks.html |
| Activity feed (append-only updates) | https://sarapis.github.io/undatacommons-nyc/activity |
| Crosswalk status (both sides verified) | https://sarapis.github.io/undatacommons-nyc/artifacts/crosswalk-latest |
| Platform probe findings | https://sarapis.github.io/undatacommons-nyc/findings/2026-09-14-platform-probe |
| Latest coverage report | https://sarapis.github.io/undatacommons-nyc/artifacts/coverage-latest |
| Decision log | https://sarapis.github.io/undatacommons-nyc/decisions |
| Probe harness source | https://github.com/sarapis/undatacommons-nyc/tree/main/probe |
| Machine index | https://sarapis.github.io/undatacommons-nyc/llms.txt |
https://unsd-datacommons.gcp.un-icc.cloud/mcp — no auth, stateless HTTPhttps://unsd-datacommons.gcp.un-icc.cloud/core/api/v2/node