mirror of
https://github.com/tiennm99/thptqg.git
synced 2026-10-11 03:13:48 +00:00
tiennm99
99465fda59
* feat(xlsxread): stage 0 scaffold with pinned deps and clap CLI skeleton * feat(xlsxread): stage 1 reader — calamine sheet enumeration and header skip * feat(xlsxread): stage 2 transform — to_ascii, score regex, validation with 38 unit tests * feat(xlsxread): stage 3 writer — SQLite DDL, INSERT OR REPLACE, VACUUM, stats output * feat(xlsxread): stage 4 audit — distinct SBD scan vs DB count, mirrors audit-row-counts.js output * feat(xlsxread): stage 5 golden tests — in-process xlsx fixtures, 8 integration tests pass * chore(xlsxread): commit Cargo.lock for reproducible Rust builds * feat(build): wire root build:db scripts to xlsxread CLI Replace node scripts/build-database*.js invocations with the Rust xlsxread binary. Each build:db* script now calls `pnpm build:rust` (cargo build --release) before invoking the xlsxread build subcommand with the matching per-dataset config. Drop xlsx and better-sqlite3 from devDependencies — no Node script consumes them anymore. sql.js (runtime DB reader in the SPA) is unaffected and remains in dependencies. * ci: build xlsxread before running database build jobs Add dtolnay/rust-toolchain@stable and Swatinem/rust-cache@v2 (workspaces: tools/xlsxread) for warm incremental Rust builds. Replace the single `pnpm build:db:all` step with explicit xlsxread invocations so CI doesn't call pnpm build:rust redundantly three times. The binary is built once, then each of the three datasets is processed in sequence. * chore: remove deprecated xlsx-based build scripts Delete scripts/build-database.js, build-database-old.js, build-database-old2.js, build-lib.js, and audit-row-counts.js. Functionality replaced by the xlsxread Rust CLI configured via tools/xlsxread/configs/*.toml. History preserved in git; one-click revert available via the chore/migration-backup-260519 branch. * docs: update README build instructions for xlsxread pipeline Replace Node.js + xlsx references with Rust + xlsxread workflow. Update requirements (Node 24+, pnpm, Rust stable), quickstart, scripts table, and project layout tree to reflect the current state after the xlsx-based build scripts were removed. * chore(deps): drop xlsx and better-sqlite3 from package.json and lockfile Remove xlsx (SheetJS, vulnerable: GHSA-4r6h-8v6p-xvw6, GHSA-5pgg-2g8v-p4x9) and better-sqlite3 from devDependencies. Both were only used by the now-deleted Node build scripts. The Rust xlsxread CLI vendors SQLite via rusqlite-bundled; no Node-side SQLite dependency is needed. `pnpm audit` returns clean.
Vietnam THPT 2017 Score Lookup
Static React + SQLite site for looking up Vietnamese high school graduation exam scores (2017). The full database (~861k students, 63 provinces) ships to the browser as a compressed SQLite file and queries run client-side via sql.js — no backend.
Live sites
Three deployments, one per dataset:
| URL | Data source | Rows |
|---|---|---|
/thptqg2017/ |
data/ — baotintuc.vn CDN, .xls |
861,068 |
/thptqg2017/old/ |
data-old/ — earlier .xlsx collection from Vietnamese news sites at the time (exact origin no longer known) |
847,348 |
/thptqg2017/old2/ |
data-old2/ — partial re-export (54 provinces) also from contemporary news sites, exact origin unrecorded |
679,764 |
Features
- Diacritics-insensitive name search (
"nguyen"matches"Nguyễn") - Live debounced search with URL deep-link (
?q=49008235) - Single-result detail card with per-subject TFT rarity-tiered scores (6 tiers, ≤1 → 9-10) and all 49 admission blocks (A00 – D15)
- Share button: copies a formatted summary plus deep-link URL
- SQL query tab with grouped presets (rankings, Long An filters, statistics, schema)
- Light + dark mode (follows OS preference)
- Keyboard shortcut
/to focus search
Requirements
- Node.js 24+
- pnpm
- Rust (stable) — installed via rustup; required for
build:db*scripts
Quickstart
pnpm install
pnpm build:db # compile xlsxread + parse data/ → public/thptqg2017.db (~2 min, 159 MB)
gzip -kf -9 public/thptqg2017.db
pnpm dev # http://localhost:5173
The database build pipeline uses the xlsxread Rust CLI located at
tools/xlsxread/. It reads .xls/.xlsx source files, strips
diacritics, parses score text, and writes a SQLite database — replacing
the former Node.js + xlsx (SheetJS) pipeline. See
tools/xlsxread/README.md for CLI invocation details and config schema.
Scripts
| Command | Action |
|---|---|
pnpm dev |
Vite dev server |
pnpm build |
Production build (main variant → dist/) |
pnpm build:old / build:old2 |
Build variant sites to dist/old/, dist/old2/ |
pnpm build:all |
All 3 web variants |
pnpm build:rust |
Compile tools/xlsxread release binary (run once; auto-called by build:db*) |
pnpm build:db |
Build main DB from data/ via xlsxread |
pnpm build:db:old / build:db:old2 |
Build old / old2 variant DBs via xlsxread |
pnpm build:db:all |
All 3 DBs via xlsxread |
pnpm lint |
ESLint |
node scripts/crawl-baotintuc.js |
Re-download all 63 province files from baotintuc.vn |
node scripts/check-duplicates.js |
MD5 + row-content duplicate audit |
node scripts/diff-datasets.js |
Compare public/ vs backup/ DB (when backup present) |
Project layout
.
├── data/ # 63 .xls files (source)
├── data-old/ # 63 .xlsx (previous export)
├── data-old2/ # 54 .xlsx (update/ overrides)
├── public/ # main variant assets + thptqg2017.db.gz
├── public-old/ # old variant assets
├── public-old2/ # old2 variant assets
├── scripts/
│ ├── crawl-baotintuc.js # downloader
│ ├── check-duplicates.js # md5 dup detector
│ └── diff-datasets.js # DB-to-DB comparator
├── src/
│ ├── App.jsx
│ ├── App.css
│ ├── components/{search-form, score-table, student-detail, custom-query}.jsx
│ ├── hooks/use-sqlite.js
│ └── lib/admission-blocks.js # 49 admission-block definitions + score-tier helper
├── tools/
│ └── xlsxread/ # Rust CLI — reads .xls/.xlsx, writes SQLite
│ ├── configs/
│ │ ├── thptqg2017-data.toml # config for data/ (63 .xls, all sheets)
│ │ ├── thptqg2017-data-old.toml # config for data-old/ (63 .xlsx, sheet 0)
│ │ └── thptqg2017-data-old2.toml # config for data-old2/ (54 .xlsx, all sheets)
│ ├── src/ # Rust source
│ └── README.md # CLI reference + config schema
├── docs/ # see docs/README.md
├── index.html
├── vite.config.js
└── package.json
See docs/ for architecture + deployment details.
License
See LICENSE.
Source
data/— scraped in full from the baotintuc.vn 2017 announcement article:https://baotintuc.vn/tuyen-sinh/tra-cuu-diem-thi-thpt-2017-cua-63-tinh-thanh-pho-tren-baotintucvn-20170706073512672.htmReproducible vianode scripts/crawl-baotintuc.js.data-old/anddata-old2/— collected from Vietnamese news sites at the time of the 2017 exam. The specific publisher URLs were not recorded and cannot be recovered now. These datasets are preserved for comparison and historical reference only;data/is the canonical source for the main deployment.
Intended for reference only.