mirror of
https://github.com/tiennm99/thptqg.git
synced 2026-10-11 03:13:48 +00:00
101 lines
3.9 KiB
Markdown
101 lines
3.9 KiB
Markdown
# thptqg
|
|
|
|
Tra cứu điểm thi THPT Quốc gia — exam-score lookup for Vietnam's national high
|
|
school graduation exam. Client-side SQL over a SQLite database the browser
|
|
downloads once, keeps, and queries in memory, built
|
|
from the published `.xls`/`.xlsx` score files by the Go `parser` module. Where
|
|
those files come from: [data pipeline](./docs/data-pipeline.md#sources).
|
|
|
|
Live at **[tiennm99.github.io/thptqg](https://tiennm99.github.io/thptqg/)**.
|
|
|
|
| Dataset | Exam | Candidates | Site |
|
|
| --- | --- | --- | --- |
|
|
| `2016` | 2016 | 877,460 | [/2016/](https://tiennm99.github.io/thptqg/2016/) |
|
|
| `2017` | 2017 | 861,068 | [/2017/](https://tiennm99.github.io/thptqg/2017/) |
|
|
|
|
Two earlier 2017 publications (`2017-old`, `2017-old2`) were kept for a while
|
|
because they disagreed with the current one. They have been removed; they remain
|
|
in git history.
|
|
|
|
## Layout
|
|
|
|
The repository is one directory per pipeline stage, plus the two stores they
|
|
pass between them.
|
|
|
|
```
|
|
crawler/ Go — re-fetches the source spreadsheets → data/
|
|
parser/ Go — Excel to SQLite data/ → .sqlite3
|
|
assembler/ Go — verifies, builds and assembles .sqlite3 + web/ → _site/
|
|
web/ npm — the frontend, one SvelteKit app for every dataset
|
|
data/<id>/ raw Excel files, one directory per dataset
|
|
datasets.json the registry: which datasets exist, and their expected size
|
|
docs/ architecture, data pipeline, deployment
|
|
```
|
|
|
|
Each stage runs on its own and hands its output to the next through the stores.
|
|
`web/` is the only npm project; the three stages are independent Go modules.
|
|
|
|
`datasets.json` is the contract between them. It is JSON because Go and the web
|
|
app both read it and neither needs a dependency to do so; presentation stays in
|
|
`web/src/lib/datasets.js`, keyed by id, which fails loudly if the two disagree.
|
|
|
|
The dataset id is one identifier end to end:
|
|
|
|
```
|
|
data/2017/ → parser/configs/2017.yml → db/2017.sqlite3 → /thptqg/2017/
|
|
```
|
|
|
|
## Build
|
|
|
|
```bash
|
|
(cd web && npm ci)
|
|
go -C assembler run ./cmd/assemble # databases, then the site, into _site/
|
|
npx serve _site
|
|
```
|
|
|
|
That one command compiles the parser, builds and verifies each database against
|
|
its registry row count and size, builds the web app and assembles `_site` —
|
|
refusing to continue if a database is short, an artifact looks truncated, or one
|
|
is missing altogether. Sub-steps when iterating:
|
|
|
|
```bash
|
|
go -C assembler run ./cmd/assemble db 2017 # one database
|
|
go -C assembler run ./cmd/assemble site # web build and _site only
|
|
go -C assembler run ./cmd/assemble verify A B # compare two sets of databases
|
|
(cd web && npm run dev) # the app against staged databases
|
|
```
|
|
|
|
The source spreadsheets are committed, so a crawl is only needed to refresh
|
|
them:
|
|
|
|
```bash
|
|
go -C crawler run ./cmd/crawl 2016
|
|
go -C crawler run ./cmd/crawl 2017
|
|
```
|
|
|
|
Each reads the download links out of the article that published the dataset, so
|
|
no link list is kept in the repository. Crawling is idempotent — files already
|
|
present are skipped — and is never part of the build.
|
|
|
|
Pushing to `main` runs the same steps in
|
|
`.github/workflows/deploy-pages.yml` and publishes to GitHub Pages.
|
|
|
|
## Adding a dataset
|
|
|
|
1. Put the Excel files in `data/<id>/`
|
|
2. Add `parser/configs/<id>.yml` — sheet mode, column indices, validation
|
|
guards. No SQL; the schema is canonical.
|
|
3. Add an entry to `datasets.json` with its expected row count and size
|
|
4. Add the matching presentation to `CONTENT` in `web/src/lib/datasets.js`
|
|
|
|
Everything else follows: the assembler, the router and the hub all read the
|
|
registry, and the UI adapts to whichever columns the dataset fills. Steps 3 and 4
|
|
check each other, so forgetting either one fails rather than half-working.
|
|
|
|
## Docs
|
|
|
|
See [`docs/`](./docs/) — [overview](./docs/project-overview.md),
|
|
[architecture](./docs/system-architecture.md),
|
|
[data pipeline](./docs/data-pipeline.md),
|
|
[deployment](./docs/deployment-guide.md).
|