mirror of
https://github.com/tiennm99/thptqg.git
synced 2026-10-14 04:16:42 +00:00
refactor: collapse the two projects into one tree and move to npm
The repo held two near-duplicate projects. 2016/ and 2017/ each carried their
own React frontend, their own copy of the same Rust crate, and their own
package manager setup. 2016/tools/sync-from-thptqg2017.sh existed purely to
copy the parser source between them.
New layout:
index.html + src/ the 2017 frontend, now the only one
data/<id>/ 2016, 2017, 2017-old, 2017-old2
parser/ the single Rust crate, configs renamed to <id>.toml
docs/ both projects' docs, 2016 copies suffixed -2016-legacy
pending the merge pass
<id> is now one identifier end to end: data/<id>/ feeds parser/configs/<id>.toml
and produces db/<id>.db.gz.
pnpm gives way to npm. pnpm-workspace.yaml existed only to whitelist
better-sqlite3's native build, which npm permits by default, so it has no
equivalent and is simply gone. Lockfiles cannot be converted; package-lock.json
is generated fresh. The migration direction is safe — pnpm's strict layout
forbids phantom dependencies, so anything that resolved under pnpm resolves
under npm's flat tree.
Adds parser/scripts/build-db.js and src/datasets.js: the four dataset IDs are
declared once and read by both the build tooling and (from the next phase) the
frontend.
Follow-on fixes the move made necessary:
- eslint's Node-globals override pointed at scripts/, now parser/scripts/
- crawl-baotintuc.js wrote to <root>/data, now data/2017
- golden tests loaded configs by their old thptqg*-data.toml names
Drops the #[ignore]d Rust-vs-Node golden test. It shelled out to pnpm to run
scripts/build-database.js, a file removed when the parser was ported to Rust,
so it could never pass. check-duplicates.js and diff-datasets.js were already
broken before this change and are annotated as such rather than half-fixed.
63 Rust tests pass and clippy is clean from the new location.
This commit is contained in:
1 parent
82db8295bb
commit
6ff2ed99ec
396 files changed
+2753
-8720
No files matched your search
@@ -0,0 +1,52 @@
|
||||
# Deployment Guide
|
||||
|
||||
Deploys to GitHub Pages via `.github/workflows/deploy.yml`. Every push to `main` rebuilds and redeploys all 3 variants.
|
||||
|
||||
## What the workflow does
|
||||
|
||||
1. Checkout, setup Node 20, `npm ci`
|
||||
2. `npm run build:db:all` — builds 3 SQLite DBs (main, old, old2)
|
||||
3. Gzip each DB at level 9
|
||||
4. `npm run build:all` — builds 3 Vite bundles into `dist/`, `dist/old/`, `dist/old2/`
|
||||
5. Remove uncompressed `.db` files from `dist/` (only `.gz` ships)
|
||||
6. `actions/upload-pages-artifact` + `actions/deploy-pages`
|
||||
|
||||
Total CI time ≈ 4–6 min (DB build dominates).
|
||||
|
||||
## Resulting URLs
|
||||
|
||||
- `https://<user>.github.io/thptqg2017/`
|
||||
- `https://<user>.github.io/thptqg2017/old/`
|
||||
- `https://<user>.github.io/thptqg2017/old2/`
|
||||
|
||||
## Local reproduction
|
||||
|
||||
```bash
|
||||
npm ci
|
||||
npm run build:db:all
|
||||
gzip -kf -9 public/thptqg2017.db public-old/thptqg2017.db public-old2/thptqg2017.db
|
||||
npm run build:all
|
||||
npx serve dist # or any static server
|
||||
```
|
||||
|
||||
## Adding a new variant
|
||||
|
||||
1. Drop source Excel files into a new `data-vN/` folder
|
||||
2. Add `public-vN/` to `.gitignore` patterns for the `.db` + `.db.gz`
|
||||
3. Copy `scripts/build-database-old.js` → `scripts/build-database-vN.js`; update `SRC_DIR` / `DB_PATH`; adjust parse logic for the new quirks (multi-sheet? blank rows? numeric guard?)
|
||||
4. Add variant to `VARIANT_CONFIG` in `vite.config.js`:
|
||||
```js
|
||||
vN: { base: "/thptqg2017/vN/", publicDir: "public-vN", outDir: "dist/vN" }
|
||||
```
|
||||
5. Add `build:db:vN` and `build:vN` npm scripts; wire them into `build:db:all` and `build:all`
|
||||
6. Update `.github/workflows/deploy.yml` — add gzip step + rm step for the new DB
|
||||
|
||||
## Notes
|
||||
|
||||
- **DB gzip is non-cacheable across deploys** — every rebuild produces a new `.db.gz` (not byte-identical due to SQLite page shuffling). First-visit users pay the 47 MB download; subsequent visits hit browser cache until the next deploy.
|
||||
- **GitHub Pages file size limit**: individual files ≤ 100 MB. Main gzipped DB is ~47 MB — safe. Uncompressed 159 MB DB would not fit; that's why we ship `.gz` and the frontend decompresses via `DecompressionStream`.
|
||||
- **No server-side compression assumption.** The app reads the `.gz` file directly (not `Content-Encoding: gzip`). GH Pages does not reliably gzip on-the-fly for arbitrary paths; shipping pre-gzipped bytes is deterministic.
|
||||
|
||||
## Rollback
|
||||
|
||||
Deploys are stateless snapshots. To rollback, revert the commit on `main` and push — the next workflow rebuilds the older state. There's no state to migrate.
|
||||
Reference in new issue
Block a user