mirror of
https://github.com/tiennm99/thptqg.git
synced 2026-10-14 20:17:50 +00:00
refactor: one directory per pipeline stage, and an assembler to drive them
The repository now reads as the pipeline it is: crawler fetches, parser converts, assembler verifies and publishes, with data/ and web/ as the stores they hand work through. go-parser is renamed parser now that there is no other. The assembler replaces build-db.js and assemble-site.js. It compiles the parser, builds and verifies each database, compresses it, runs the Vite build and assembles _site — one command, and the only place that knows the order. It also closes a real hole: nothing previously asserted that a database reached the site. An empty staging directory assembled happily, so every page rendered, every query 404d and CI stayed green. The row-count and size guards could not catch that, since they only run when a database was built at all. Removing Node from the root forced the dataset list out of web/src/datasets.js, which the assembler cannot import. datasets.json is now the registry both sides read — JSON because Go and the browser both parse it without a dependency — while presentation stays in the web app, keyed by id and cross-checked against the registry so a half-added dataset fails instead of half-working. Guards verified by making each one fail: a missing database, and an expected row count one higher than the truth.
This commit is contained in:
1 parent
b04f9844f9
commit
c359a0b444
61 files changed
+1341
-480
No files matched your search
@@ -0,0 +1,25 @@
|
||||
// Package sqlitedb registers the SQLite driver the parser writes with and names
|
||||
// it in one place.
|
||||
//
|
||||
// modernc.org/sqlite is a pure-Go SQLite, chosen so the whole module stays
|
||||
// cgo-free — grate, excelize and yaml.v3 are pure Go too, so CI can compile with
|
||||
// CGO_ENABLED=0 and no C toolchain.
|
||||
//
|
||||
// It is a machine-transpiled SQLite rather than the upstream C amalgamation that
|
||||
// Rust's rusqlite --bundled vendors (libsqlite3-sys 0.30.1). Two things make that
|
||||
// acceptable: this parser uses only plain SQL — no CTEs, window functions,
|
||||
// triggers or extensions — and the differential gate compares a full-table
|
||||
// SHA-256 plus PRAGMA table_info/index_list against live Rust output.
|
||||
//
|
||||
// Verified on linux/arm64 with v1.56.0 (SQLite 3.53.3): the full DDL including
|
||||
// the partial idx_ten_cum_thi index, INSERT OR REPLACE, and VACUUM.
|
||||
//
|
||||
// Fallback if the differential gate ever implicates the driver: mattn/go-sqlite3
|
||||
// is upstream C at the cost of cgo.
|
||||
package sqlitedb
|
||||
|
||||
// Registers "sqlite" with database/sql.
|
||||
import _ "modernc.org/sqlite"
|
||||
|
||||
// DriverName is the database/sql driver name to pass to sql.Open.
|
||||
const DriverName = "sqlite"
|
||||
Reference in new issue
Block a user