tiennm99
00a08d5fab
The Go parser has matched the Rust one field-by-field across all four datasets, so the Rust crate is retired and CI builds the Go binary instead. crawl-baotintuc.js moves to go-parser/scripts/ — it is the only mechanism for refreshing data/2017 and has a documented runbook. check-duplicates.js and diff-datasets.js are dropped: both had been broken since before the repo was unified, and neither had a caller. db-stats.js and verify-parity.js are dropped as superseded by differential-parity.mjs, which compares more and cannot silently skip a dataset. The reader-fidelity oracle is kept and marked frozen. It was produced by the Rust reader so it can no longer be regenerated, but it still fails if any single cell of any of the 299 input files reads differently. Source comments cite the original Rust by file and line; those paths resolve at tag pre-go-parser-removal, recorded in go-parser/README.md.
thptqg
Tra cứu điểm thi THPT Quốc gia — exam-score lookup for Vietnam's national high
school graduation exam. Client-side SQL (sql.js) over a SQLite database built
from the ministry's raw .xls score files by the Rust xlsxread parser.
Live at tiennm99.github.io/thptqg.
| Dataset | Exam | Candidates | Site |
|---|---|---|---|
2016 |
2016 | 877,461 | /2016/ |
2017 |
2017 | 861,068 | /2017/ |
2017-old |
2017 | 847,348 | /2017-old/ |
2017-old2 |
2017 | 679,764 | /2017-old2/ |
The three 2017 datasets are successive publications of the same exam and they disagree; all three are kept so the differences stay inspectable.
Layout
index.html + src/ the frontend — one app serving all four datasets and the hub
datasets.js the four dataset ids and their per-dataset content
router.js pathname → dataset
data/<id>/ raw Excel files, one directory per dataset
parser/ the Rust parser
src/schema.rs canonical 22-column table: DDL, INSERT, subject regexes
configs/<id>.yml per-dataset parse rules only, no SQL
scripts/ database build, crawler, parity verification
scripts/ site assembly
docs/ architecture, data pipeline, deployment
The dataset id is one identifier end to end:
data/2017-old/ → go-parser/configs/2017-old.yml → db/2017-old.db.gz → /thptqg/2017-old/
Build
npm ci
npm run build:go # compile the parser
npm run build:db # build + gzip all four databases (add an id for just one)
npm run build:site # one Vite build, then assemble into _site/
npx serve _site
Pushing to main runs the same steps in
.github/workflows/deploy-pages.yml and publishes to GitHub Pages.
Adding a dataset
- Put the Excel files in
data/<id>/ - Add
go-parser/configs/<id>.yml— sheet mode, column indices, validation guards. No SQL; the schema is canonical. - Add an entry to
DATASETSinsrc/datasets.js
Everything else follows: the build script, the site assembly and the router all read that one list, and the UI adapts to whichever columns the dataset fills.
Docs
See docs/ — overview,
architecture,
data pipeline,
deployment.