加载中…
加载中…
A from-scratch, 148,195-parameter character-level parser that turns a plain-English or Hinglish question about a US ZIP-code table into a structured query, plus a deterministic engine that answers it from the table. The model decides what is being asked; the engine guarantees every fact comes from a row. It never hallucinates a county.
"60614 ka county aur area code batao"
→ parser : intent=lookup wants=[COUNTY_NAME, AERA_CODE] slots=[(ZIP, "60614")]
→ engine : ZIP 60614 is in Cook County and has area code 773.
Trained on CPU (2 cores) in ~2.5 hours. Inference is ~2 ms for the model and ~100 ms for a fuzzy name lookup over 36k place names. Weights: 594 kB.
The table is not included. The model was trained against a licensed USPS-derived dataset (99,214 rows:
ZIP, STATE, STATE_NAME, CITY, CITY_ABB, LATITUDE, LONGITUDE, COUNTY_CODE, COUNTY_NAME, AERA_CODE, ZIP_TYPE, LAST_LINE_INDICATOR, FACILITY_CODE, FINANCE_NUMBER, CBSA, CBSA_TITLE, CBSA_STATUS). Any CSV with those columns works; the free HUD USPS ZIP crosswalk or the Census ZCTA relationship files can be mapped to it. The engine builds its indexes from whatever CSV you give it.
That was the first idea — published as sraivante/zip-county-city-seq2seq-21m — and it is the wrong tool. A ~40k-ZIP table is ~100k rows of digits and proper nouns with no
pattern to generalise from. A small transformer trained to emit "72013 ARECIBO" after "zip 612" is gradient descent used
as a lossy hash map: it confuses 612/613/614 (adjacent numbers, same county — great for generalisation, fatal for exact
recall), invents plausible FIPS codes and coordinates, and has to be retrained on every table refresh.
What people actually need is: understand the question, look up the facts. Those are different problems with different right-sized solutions. Understanding ~7 question shapes over ~12 columns is a 150k-parameter job. Looking up facts is a dictionary.
question (≤96 chars, lower-cased)
│
▼ char embedding (52 symbols → 48-d)
│
▼ 1-layer bidirectional GRU, hidden 128 ─────────────┐
│ │
├─► per-character head → BIO slot tags │ (ZIP, CITY, COUNTY, STATE, CBSA, AREACODE, LAT, LON)
│ │
└─► sentence vector (final states + mean pool) ─────────┤
├─► softmax → intent (lookup, list, count, check, nearest, top, unknown)
└─► sigmoid → wants (COUNTY_NAME, COUNTY_CODE, STATE, CITY, ZIP, AERA_CODE,
LATLON, CBSA_TITLE, CBSA, ZIP_TYPE, ALL — multi-label)
Three heads, one loss (CE + BCE + CE). Char-level because "Mayagez", "mayagüez" and "MAYAGUEZ" must land on the same
embedding and unseen place names must not become <unk>. A 2-layer GRU was 2× slower on CPU for no accuracy gain.
No human-labelled questions exist for this, so the corpus is synthesised from the table — the city, county and state names are the entity vocabulary, so coverage is exhaustive by construction. 1,000,000 examples:
| intent | share | how it is generated |
|---|---|---|
| lookup | 42% | 55 question frames × 11 want-columns × random entity; 20% compound ("county and area code") |
| list | 14% | "zips in X", "neighbourhoods of X", "which states have a Springfield" |
| check | 13% | "is X in Y", "are X and Y in the same county" |
| count | 11% | "how many zips in X", "kitne county hai" |
| unknown | 8% | weather, population, greetings, SQL, gibberish, questions about places in the table that the table can't answer |
| nearest | 6% | "closest city to 18.44, -66.40" with random coordinates in the table's bounding box |
| top | 6% | "which county has the most zips", "top 5 counties in Texas by zip codes", "sabse zyada…" |
Every example is built from labelled segments, so character-level slot tags survive the noise that is then applied: keyboard-neighbour typos, dropped/swapped/doubled letters (35% of entity tokens, 12% of frame tokens), random casing, dropped neighbourhood prefixes ("Urb Monaco 3" → "Monaco 3"), un-padded ZIPs ("603" for "00603"), chatty prefixes and suffixes ("bhai", "pls", "batao"), and Hinglish frames ("{E} kis county me hai", "{E} ka {T} batao").
Three generator decisions turned out to matter more than anything in the model:
IN OR ME OK HI DE LA OH PA MA CO ID AL MO NE MD MS AR MI are
English (or Spanish) words. A model that learns "miami in" → Indiana is worse than useless; the generator only
emits these as full state names, and the parser accepts the 2-letter form only when written in capitals (or, for the
rarer ones like co, ma, immediately after a place name).ALL). Conversely, foreign places ("pin code of Varanasi")
were removed from the unknown templates — the model should not learn that "Varanasi" means unknown, it should
learn that unresolvable means unknown, which the engine decides.python3 prepare_data.py raw_dump.csv table.csv # clean (see §6)
python3 gen_corpus.py table.csv corpus.jsonl 1000000
python3 train.py corpus.jsonl parser.pt 2 # AdamW 2e-3, OneCycle, bs 256, ~75 min/epoch on 2 CPU cores
Validation on a 2% held-out split of the generated corpus:
| epoch | intent acc | wants exact-match | slot tag acc (per char) |
|---|---|---|---|
| 1 | 99.93% | 97.56% | 99.02% |
| 2 | 99.98% | 98.78% | 99.22% |
Those numbers are against generated questions and are mostly a sanity check. The number that matters is below.
eval_full.py holds 48 questions written without looking at the templates — English, Hinglish, typos, compound
asks, ambiguous names, out-of-scope requests — each with the substring the answer must contain. Final score: 48/48.
which county is 90210 in → ZIP 90210 is in Los Angeles County.
Bevrly Hills california zip → Beverly Hills has ZIP codes 90209, 90210, 90211, 90212, 90213.
which county is springfield in → There are 26 places called Springfield (AR, CO, GA, …). Which state do you mean?
county of springfield MA → Springfield is in Hampden County.
what county is las vegas in → (assuming Las Vegas, NV; it also exists in NM) Las Vegas is in Clark County.
county of St. Louis MO → Saint Louis is in Saint Louis City County, Saint Louis County.
Cook County IL me kitne zip hai → Cook County has 228 ZIP codes.
how many states have a washington county → There is a Washington County in 31 states: Alabama (AL), …
top 3 states by number of zip codes → 1. Texas (2598); 2. California (2589); 3. Pennsylvania (2172).
is 10001 in the new york metro → Yes, ZIP 10001 is in the New York-Newark-Jersey City metro area.
county of 09002 → ZIP 09002 is a military (APO/FPO/DPO) address in Armed Forces Europe (AE)…
population of los angeles county → I can only answer questions about this ZIP table: …
Going from a 1,000-row sample to 99,214 rows broke nothing in the parser and almost everything in the engine. This is the central lesson of the project: the model was never the bottleneck.
| problem at scale | fix |
|---|---|
The dump had stray " characters inside CITY_ABB and CBSA_TITLE; a standard CSV parser silently dropped 1,158 rows |
no field contains a comma, so: strip all quotes, split on commas |
CBSA_TITLE was double-encoded mojibake (San GermÃÆ'…) |
repair latin1→utf8 twice, fold to ASCII, override 4 titles by CBSA code |
ZIP stored as int: 00601 → 601 |
zero-pad on load; accept both forms on input |
| 31 Washington Counties, 26 Springfields, 6,543 city names in more than one state | state scoping: a STATE slot narrows; else if one state holds ≥80% of the ZIPs (New York, Los Angeles) assume it and say so; else ask "which state?" |
| Names that are a city and a county and a state (Miami, Vermont, Washington, Nevada) | decided from cue words ("county", "city") and position, not the tagger alone |
| A city called Many (LA) and a city called Tell (TX) | a single-word gazetteer hit is rejected if the word occurs in any question template |
| "st louis" exists literally in MS and TX; Missouri's is "SAINT LOUIS" | st/ste/mt/ft/pt aliases, applied when the state filter comes back empty |
difflib fuzzy matching: ~2 s per query over 36k names |
rapidfuzz WRatio: ~100 ms; plus a ratio≥80 floor so "varanasi" stops matching "Varna" |
| Military APO/FPO rows: no county, coordinates 0,0 | answered as military addresses; skipped by nearest |
| USPS label "cities" (BUSINESS REPLY MAIL, BAR CODE CHURCH STREET…) | kept in the table, excluded from question generation |
| "chicago metro" vs title "Chicago-Naperville-Elgin" | CBSA prefix match wins over the exact city hit |
low.find("or") matched the "or" inside "portland" |
whole-word span expansion |
Row: 60614, IL, Illinois, CHICAGO, 41.9238, -87.6492, 17031, COOK, 773, standard, CBSA 16980 Chicago-Naperville-Elgin.
Each line below was actually run against the published weights.
Field lookups — answered
Same row from the city / county side — answered
Checks — answered
Geography — answered
Aggregations — answered
Follow-ups — answered (one turn of memory)
Refused — not in the table (scope message)
Refused — columns deliberately not exposed (explicit message)
Cannot do — honest limits
in / or / me / ok / hi / de / oh / la / pa are never read as states — "portland or" needs OR or "oregon"| file | role |
|---|---|
model.safetensors, config.json |
the parser (148,195 params) |
model.py / schema.py |
architecture, labels |
ask.py |
QA class: parse → gazetteer backstop → scope guards → engine; CLI and interactive loop |
engine.py |
entity resolution (exact → abbreviation → fuzzy → cross-type), state scoping, the 6 intents, answer templates |
gen_corpus.py, train.py, prepare_data.py |
reproduce everything from a raw dump |
eval_full.py, eval_questions.py |
hand-typed test sets |
example.py |
10-line usage |
pip install torch rapidfuzz safetensors
python3 example.py # expects table.csv next to it
python3 ask.py # interactive
The parser learns question shapes, not facts, so it transfers to any table with these columns unchanged — the
engine rebuilds its indexes from the CSV. Retrain only if your place-name distribution is very different (non-US
scripts, say): gen_corpus.py reads its entity vocabulary from the table, so the four commands in §4 are the whole
recipe. Adding a new question type = a few dozen frames in gen_corpus.py + a method in engine.py + retrain (~2.5 h CPU).
Predecessor: sraivante/zip-county-city-seq2seq-21m, a
31M-parameter seq2seq that tried to memorise the table directly — 12.1% complete answers on fresh prompts at v0.3.0.
This repo replaces it: 200× fewer parameters, 48/48 on hand-typed questions, zero hallucinated facts.
Same recipe as sraivante/rx-intent-1m (drug-name lookup): a tiny
intent parser in front of a deterministic, typo-tolerant lookup engine. The thesis both models test is that for
"answer questions about this dataset", a sub-megabyte model plus a dictionary beats a large model plus RAG on
correctness, latency and cost — as long as you are honest about the question shapes you support and refuse the rest.