mirror of
https://github.com/tiennm99/thptqg.git
synced 2026-10-16 10:15:26 +00:00
refactor: collapse the two projects into one tree and move to npm
The repo held two near-duplicate projects. 2016/ and 2017/ each carried their
own React frontend, their own copy of the same Rust crate, and their own
package manager setup. 2016/tools/sync-from-thptqg2017.sh existed purely to
copy the parser source between them.
New layout:
index.html + src/ the 2017 frontend, now the only one
data/<id>/ 2016, 2017, 2017-old, 2017-old2
parser/ the single Rust crate, configs renamed to <id>.toml
docs/ both projects' docs, 2016 copies suffixed -2016-legacy
pending the merge pass
<id> is now one identifier end to end: data/<id>/ feeds parser/configs/<id>.toml
and produces db/<id>.db.gz.
pnpm gives way to npm. pnpm-workspace.yaml existed only to whitelist
better-sqlite3's native build, which npm permits by default, so it has no
equivalent and is simply gone. Lockfiles cannot be converted; package-lock.json
is generated fresh. The migration direction is safe — pnpm's strict layout
forbids phantom dependencies, so anything that resolved under pnpm resolves
under npm's flat tree.
Adds parser/scripts/build-db.js and src/datasets.js: the four dataset IDs are
declared once and read by both the build tooling and (from the next phase) the
frontend.
Follow-on fixes the move made necessary:
- eslint's Node-globals override pointed at scripts/, now parser/scripts/
- crawl-baotintuc.js wrote to <root>/data, now data/2017
- golden tests loaded configs by their old thptqg*-data.toml names
Drops the #[ignore]d Rust-vs-Node golden test. It shelled out to pnpm to run
scripts/build-database.js, a file removed when the parser was ported to Rust,
so it could never pass. check-duplicates.js and diff-datasets.js were already
broken before this change and are annotated as such rather than half-fixed.
63 Rust tests pass and clippy is clean from the new location.
This commit is contained in:
1 parent
82db8295bb
commit
6ff2ed99ec
396 files changed
+2753
-8720
No files matched your search
@@ -0,0 +1,97 @@
|
||||
# System Architecture
|
||||
|
||||
Static site. No backend. Browser downloads a compressed SQLite file at boot, then every query runs locally via `sql.js` (WASM).
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Excel files (.xls / .xlsx)
|
||||
│
|
||||
▼ build-database*.js (Node)
|
||||
SQLite DB (public*/thptqg2017.db)
|
||||
│
|
||||
▼ gzip -9
|
||||
thptqg2017.db.gz (~47 MB for main variant)
|
||||
│
|
||||
▼ Vite build copies public*/ into dist/
|
||||
Static site on GitHub Pages
|
||||
│
|
||||
▼ browser loads
|
||||
sql.js (WASM) opens the .db → queries run client-side
|
||||
```
|
||||
|
||||
## Three deployment variants
|
||||
|
||||
One repo → three independent sites, same frontend, different dataset.
|
||||
|
||||
| Variant | Route | Source dir | Public dir | Build cmd |
|
||||
|---|---|---|---|---|
|
||||
| main | `/thptqg2017/` | `data/` | `public/` | `npm run build` |
|
||||
| old | `/thptqg2017/old/` | `data-old/` | `public-old/` | `npm run build:old` |
|
||||
| old2 | `/thptqg2017/old2/` | `data-old2/` | `public-old2/` | `npm run build:old2` |
|
||||
|
||||
`vite.config.js` reads `process.env.VARIANT` and switches `base`, `publicDir`, `outDir` accordingly. `emptyOutDir` is on only for the main build so the variant builds merge into `dist/old/`, `dist/old2/` cleanly.
|
||||
|
||||
## Schema (all 3 DBs share this)
|
||||
|
||||
```sql
|
||||
CREATE TABLE student (
|
||||
so_bao_danh TEXT PRIMARY KEY, -- exam ID (8 digits, first 2 = province)
|
||||
ho_ten TEXT NOT NULL,
|
||||
ho_ten_ascii TEXT NOT NULL, -- NFD-stripped lowercase for accent-insensitive search
|
||||
ngay_sinh TEXT, -- dd/mm/yyyy
|
||||
toan, ngu_van, vat_ly, hoa_hoc, sinh_hoc, khtn,
|
||||
lich_su, dia_ly, gdcd, khxh,
|
||||
tieng_anh, tieng_phap, tieng_nga, tieng_trung REAL
|
||||
);
|
||||
CREATE INDEX idx_ho_ten ON student(ho_ten);
|
||||
CREATE INDEX idx_ho_ten_ascii ON student(ho_ten_ascii);
|
||||
```
|
||||
|
||||
No German / Japanese language columns — neither appears in any source file.
|
||||
|
||||
Each score column is `NULL` when the student didn't take that subject. A student's "admission block" total is only computed when all 3 required subjects are non-null.
|
||||
|
||||
## Parse quirks (why 3 builder scripts)
|
||||
|
||||
Each dataset had a different quirk. Rather than one mega-parser with mode flags, each builder script is ~70 lines and isolates its own workarounds.
|
||||
|
||||
| Dataset | Quirk | Mitigation |
|
||||
|---|---|---|
|
||||
| data/ (baotintuc .xls) | Hanoi + HCM overflow past the 65,536 row-per-sheet .xls limit | Iterate all `wb.SheetNames` |
|
||||
| data-old/ (xlsx) | Single-sheet always; one bogus header row (`SOBAODANH`/`HO_TEN`) leaked into an earlier DB | Strict numeric-SBD guard rejects the leak |
|
||||
| data-old2/ (xlsx) | HCM overflow + many blank trailing rows | Multi-sheet walk + blank-row pre-filter |
|
||||
|
||||
Shared concerns (regex, schema, `toAscii`, header detection) live in `scripts/build-lib.js`.
|
||||
|
||||
## Admission blocks
|
||||
|
||||
Vietnamese universities admit students based on 3-subject combinations called "admission blocks" (khối thi). `src/lib/admission-blocks.js` lists all 49 official 2017 blocks computable from our schema (A00–A11, B00–B08, C00–C20, D01–D15). Each entry is `{ code, subjects: [3 keys], label }`.
|
||||
|
||||
`computeBlocks(student)` returns only blocks where the student has all 3 subject scores, sorted by total desc. Used by the student detail card.
|
||||
|
||||
The SQL preset "Top 10 best-block in Long An" materialises all 49 blocks as a `UNION ALL` CTE, picks each student's best block via `ROW_NUMBER() OVER (PARTITION BY so_bao_danh ORDER BY s DESC, k)`, then ranks across students.
|
||||
|
||||
## Score tiers (UI coding)
|
||||
|
||||
6-level TFT rarity ladder (white → green → blue → purple → gold → prismatic). Defined in `scoreTier()` in `src/lib/admission-blocks.js`.
|
||||
|
||||
| Tier | Range | UI label (Vietnamese) | English |
|
||||
|---|---|---|---|
|
||||
| common | ≤ 1 | Điểm liệt | Disqualifying |
|
||||
| uncommon | < 5 | Chưa đạt | Below passing |
|
||||
| rare | 5–6.5 | Trung bình | Average |
|
||||
| epic | 6.5–8 | Khá | Good |
|
||||
| legendary | 8–9 | Giỏi | Very good |
|
||||
| prismatic | 9–10 | Xuất sắc | Excellent |
|
||||
|
||||
Color + icon + text label — never color-only.
|
||||
|
||||
## Frontend key behaviors
|
||||
|
||||
- Query state lives in `App.jsx` and is synced to URL `?q=...` via `history.replaceState` — bookmarkable, shareable.
|
||||
- `SearchForm` is a controlled component that 300ms-debounces input changes and auto-triggers search once the query is long enough (3+ digits for SBD, 2+ chars for name).
|
||||
- Detection: all-digits → exact `so_bao_danh` match; ASCII-only → folded `ho_ten_ascii LIKE`; has Vietnamese diacritics → search both `ho_ten` and folded column.
|
||||
- Exactly 1 result → `StudentDetail` card; 0 or >1 → `ScoreTable`.
|
||||
- Global `/` key focuses the search input when not already typing.
|
||||
- Share button on the detail card uses Web Share API when present, else clipboard with a pre-formatted summary + deep-link URL.
|
||||
Reference in new issue
Block a user