refactor: one directory per pipeline stage, and an assembler to drive them

The repository now reads as the pipeline it is: crawler fetches, parser
converts, assembler verifies and publishes, with data/ and web/ as the stores
they hand work through. go-parser is renamed parser now that there is no other.

The assembler replaces build-db.js and assemble-site.js. It compiles the
parser, builds and verifies each database, compresses it, runs the Vite build
and assembles _site — one command, and the only place that knows the order.

It also closes a real hole: nothing previously asserted that a database reached
the site. An empty staging directory assembled happily, so every page rendered,
every query 404d and CI stayed green. The row-count and size guards could not
catch that, since they only run when a database was built at all.

Removing Node from the root forced the dataset list out of web/src/datasets.js,
which the assembler cannot import. datasets.json is now the registry both sides
read — JSON because Go and the browser both parse it without a dependency —
while presentation stays in the web app, keyed by id and cross-checked against
the registry so a half-added dataset fails instead of half-working.

Guards verified by making each one fail: a missing database, and an expected
row count one higher than the truth.

parser/README.md

@@ -0,0 +1,76 @@
+# parser
+
+Reads the `.xls`/`.xlsx` source spreadsheets in `data/` and writes one SQLite
+database per dataset.
+
+```bash
+go -C parser build -o bin/xlsxread ./cmd/xlsxread   # compile
+go -C parser test ./...                             # unit tests + the reader-fidelity suite
+```
+
+```
+xlsxread build --schema parser/configs/<id>.yml --input data/<id> --output <db>
+xlsxread audit --schema parser/configs/<id>.yml --input data/<id> --db <db>
+```
+
+This stage only produces a database. Verifying it against the expected row
+count, compressing it and publishing it belong to `assembler/`, which compiles
+this binary and drives it per dataset:
+
+```bash
+go -C assembler run ./cmd/assemble db
+```
+
+## Layout
+
+| path | role |
+|---|---|
+| `internal/reader` | spreadsheet reading; the only place that knows about file formats |
+| `internal/ingest` | dataset policy — sheet selection, header skipping, blank rows, the build loop, and the 2016 per-sheet format detection |
+| `internal/transform` | `ToAscii`, score-regex parsing, row validation |
+| `internal/schema` | the canonical 22-column table: DDL, INSERT, subject regexes |
+| `internal/config` | per-dataset YAML parse rules |
+| `internal/writer` | SQLite lifecycle and the stats block |
+| `internal/audit` | source-vs-database SBD comparison |
+
+The reader deliberately knows nothing about datasets: it reports every sheet and
+every row verbatim. All policy lives in `ingest`. That split is what made the
+reader independently verifiable against a hash oracle.
+
+## Provenance
+
+This is a port of a Rust crate that occupied this same path until the Go
+implementation reached full parity, when it was built alongside as `go-parser/`
+and moved back here once the Rust was removed. Source comments cite the original
+by file and line (`parser/src/transform.rs:56` and similar) — those refer to the
+Rust tree and resolve at the tag **`pre-go-parser-removal`**, the last commit
+containing it.
+
+The port was gated on a field-by-field comparison of both implementations across
+the four datasets that existed then — 3,265,641 rows with identical full-table
+SHA-256, identical per-column non-NULL counts, identical schema metadata and
+identical build stdout. `scripts/differential-parity.mjs` is that comparator and
+still runs against any two sets of databases.
+
+Behaviour was matched bug-for-bug, deliberately. Several quirks look like
+defects and are load-bearing for the published data:
+
+- a parsed score of `0` becomes NULL in the 2016 separate-scores layout,
+  replicating a JavaScript falsy check;
+- `ToAscii` strips combining marks in the literal range U+0300–U+036F rather
+  than by Unicode category, which is narrower;
+- gender is a two-value allowlist, and anything else becomes NULL;
+- `diem_thi` is read untrimmed while the other three fields are trimmed;
+- the `"SINH "` header token carries a trailing space.
+
+Each has a test naming it, so none can be tidied away by accident.
+
+## Verification
+
+`testdata/reader-fidelity-hashes.tsv` holds a SHA-256 per input file over a
+canonical dump of every cell of every sheet. It is **frozen**: it was produced
+by the Rust reader, which no longer exists, so it cannot be regenerated. It
+still fails if any single cell of any of the 182 files reads differently.
+
+The assembler refuses to publish a database whose row count does not match
+the known figure, or whose artifact is under 90% of its usual size.

parser/cmd/dumpcells/main.go

@@ -0,0 +1,88 @@
+// Command dumpcells emits the canonical cell rendering of a spreadsheet, for
+// comparison against the Rust/calamine ground truth produced by
+// parser/examples/dump_cells.rs.
+//
+// The canonical stream carries geometry and rendered cell values only. The
+// calamine Data variant is deliberately excluded: Data::Empty and
+// Data::String("") both render "" and both count as blank everywhere
+// downstream, so the distinction cannot affect the database.
+//
+// Usage: dumpcells <spreadsheet> [out-file]
+package main
+
+import (
+	"bufio"
+	"fmt"
+	"os"
+	"strings"
+
+	"github.com/tiennm99/thptqg/parser/internal/reader"
+)
+
+// escape mirrors the Rust dumper so field separators can never break the format.
+func escape(s string) string {
+	var b strings.Builder
+	b.Grow(len(s))
+	for _, ch := range s {
+		switch ch {
+		case '\\':
+			b.WriteString(`\\`)
+		case '\t':
+			b.WriteString(`\t`)
+		case '\n':
+			b.WriteString(`\n`)
+		case '\r':
+			b.WriteString(`\r`)
+		default:
+			b.WriteRune(ch)
+		}
+	}
+	return b.String()
+}
+
+func main() {
+	if len(os.Args) < 2 {
+		fmt.Fprintln(os.Stderr, "usage: dumpcells <spreadsheet> [out-file]")
+		os.Exit(2)
+	}
+	path := os.Args[1]
+
+	out := os.Stdout
+	if len(os.Args) > 2 {
+		f, err := os.Create(os.Args[2])
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "create: %v\n", err)
+			os.Exit(1)
+		}
+		defer f.Close()
+		out = f
+	}
+	w := bufio.NewWriterSize(out, 1<<20)
+	defer w.Flush()
+
+	wb, err := reader.Open(path)
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "open: %v\n", err)
+		os.Exit(1)
+	}
+	defer wb.Close()
+
+	sheets := wb.Sheets()
+	fmt.Fprintf(w, "FILE\t%s\n", escape(path))
+	fmt.Fprintf(w, "SHEETCOUNT\t%d\n", len(sheets))
+
+	for _, sh := range sheets {
+		fmt.Fprintf(w, "SHEET\t%d\t%s\t%d\t%d\n", sh.Index, escape(sh.Name), sh.Height, sh.Width)
+		err := wb.EachRow(sh.Index, func(s reader.Sheet, rowIdx int, row []reader.Cell) error {
+			fmt.Fprintf(w, "ROW\t%d\t%d\t%d\n", s.Index, rowIdx, len(row))
+			for c, cell := range row {
+				fmt.Fprintf(w, "CELL\t%d\t%d\t%d\t%s\n", s.Index, rowIdx, c, escape(cell.Str))
+			}
+			return nil
+		})
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "rows: %v\n", err)
+			os.Exit(1)
+		}
+	}
+}

parser/cmd/xlsxread/main.go

@@ -0,0 +1,115 @@
+// Command xlsxread reads .xls/.xlsx files and builds SQLite databases for the
+// thptqg datasets.
+//
+// The CLI contract is fixed by parser/scripts/build-db.js and must match the
+// Rust binary exactly:
+//
+//	xlsxread build --schema <config.yml> --input <dir> --output <db>
+//	xlsxread audit --schema <config.yml> --input <dir> --db <db>
+//
+// Implemented with the standard flag package rather than a CLI framework: two
+// subcommands with three flags each do not justify a dependency.
+package main
+
+import (
+	"flag"
+	"fmt"
+	"os"
+
+	"github.com/tiennm99/thptqg/parser/internal/audit"
+	"github.com/tiennm99/thptqg/parser/internal/config"
+	"github.com/tiennm99/thptqg/parser/internal/ingest"
+)
+
+func usage() {
+	fmt.Fprint(os.Stderr, `xlsxread — read .xls/.xlsx files and build SQLite databases for thptqg datasets
+
+Usage:
+  xlsxread build --schema <config.yml> --input <dir> --output <db>
+  xlsxread audit --schema <config.yml> --input <dir> --db <db>
+`)
+}
+
+func main() {
+	if len(os.Args) < 2 {
+		usage()
+		os.Exit(2)
+	}
+
+	switch os.Args[1] {
+	case "build":
+		runBuild(os.Args[2:])
+	case "audit":
+		runAudit(os.Args[2:])
+	case "-h", "--help", "help":
+		usage()
+	default:
+		fmt.Fprintf(os.Stderr, "unknown subcommand %q\n\n", os.Args[1])
+		usage()
+		os.Exit(2)
+	}
+}
+
+func runBuild(args []string) {
+	fs := flag.NewFlagSet("build", flag.ExitOnError)
+	schemaPath := fs.String("schema", "", "path to the dataset YAML config file")
+	inputDir := fs.String("input", "", "directory containing the .xls / .xlsx source files")
+	outputPath := fs.String("output", "", "output SQLite database path")
+	fs.Parse(args)
+
+	if *schemaPath == "" || *inputDir == "" || *outputPath == "" {
+		fmt.Fprintln(os.Stderr, "build requires --schema, --input and --output")
+		os.Exit(2)
+	}
+
+	cfg, err := config.Load(*schemaPath)
+	if err != nil {
+		fatalf("Failed to load config: %v", err)
+	}
+
+	// The 2016 dataset selects its column layout per file at runtime; every other
+	// dataset uses the fixed columns: mapping (main.rs:63-67).
+	if cfg.FormatDetection != nil && *cfg.FormatDetection == "thptqg2016" {
+		if err := ingest.Detect2016(cfg, *inputDir, *outputPath); err != nil {
+			fatalf("%v", err)
+		}
+		return
+	}
+	if err := ingest.Standard(cfg, *inputDir, *outputPath); err != nil {
+		fatalf("%v", err)
+	}
+}
+
+func runAudit(args []string) {
+	fs := flag.NewFlagSet("audit", flag.ExitOnError)
+	schemaPath := fs.String("schema", "", "path to the dataset YAML config file")
+	inputDir := fs.String("input", "", "directory containing the .xlsx source files")
+	dbPath := fs.String("db", "", "SQLite database to compare against")
+	fs.Parse(args)
+
+	if *schemaPath == "" || *inputDir == "" || *dbPath == "" {
+		fmt.Fprintln(os.Stderr, "audit requires --schema, --input and --db")
+		os.Exit(2)
+	}
+
+	cfg, err := config.Load(*schemaPath)
+	if err != nil {
+		fatalf("Failed to load config: %v", err)
+	}
+	res, err := audit.Run(*inputDir, *dbPath, cfg)
+	if err != nil {
+		fatalf("%v", err)
+	}
+	audit.PrintReport(res)
+
+	// A mismatch is a non-zero exit even though the audit itself succeeded
+	// (main.rs:46-48) — CI treats it as a failure signal.
+	if !res.Matched {
+		os.Exit(1)
+	}
+}
+
+func fatalf(format string, a ...any) {
+	fmt.Fprintf(os.Stderr, "Error: "+format+"\n", a...)
+	os.Exit(1)
+}

parser/configs/2016.yml

@@ -0,0 +1,32 @@
+# thptqg2016 data/ — 4 .xls + 115 .xlsx mixed files.
+#
+# Three column layouts exist across the 119 files; the binary selects the
+# right one per-file at runtime via format_detection: thptqg2016:
+#
+#   separate-scores  header SBD(0)/HOTEN(1)/TOAN(2)...  — dhhanghai files
+#   mapped           header SOBAODANH|SBD + DIEM_THI     — most provinces
+#   default          no header; positional 6-col layout  — remaining files
+#
+# This is the only dataset that populates ten_cum_thi and gioi_tinh, and the
+# only one whose files carry Tiếng Đức / Tiếng Nhật scores.
+#
+# sheet_mode = "all": several provinces overflow into Sheet2 (65k Excel row cap).
+# strip_blank_rows = false: no blank-row anomaly observed in this dataset.
+#
+# Table shape, INSERT and subject regexes are canonical — see parser/internal/schema/schema.go.
+
+format_detection: thptqg2016
+
+reader:
+  sheet_mode: all
+  strip_blank_rows: false
+
+validation:
+  require_numeric_sbd: false
+  require_nonempty_name: true
+  require_nonempty_sbd: true
+
+header:
+  # Tokens that identify a header row by first-cell content (uppercased).
+  # Covers both SOBAODANH-style and SBD-style headers.
+  tokens: ["SOBAODANH", "SBD", "HO_TEN", "HOTEN", "HỌ TÊN", "STT"]

parser/configs/2017.yml

@@ -0,0 +1,25 @@
+# thptqg2017 data/ — 63 .xls files from baotintuc.vn (current generation).
+#
+# sheet_mode = "all": Hà Nội and HCM overflow into Sheet2 (65k Excel row cap).
+# SBD validation: no numeric guard — build-database.js did not apply ^\d+$.
+# strip_blank_rows = false: no blank-row anomaly in this dataset.
+#
+# Table shape, INSERT and subject regexes are canonical — see parser/internal/schema/schema.go.
+
+reader:
+  sheet_mode: all
+  strip_blank_rows: false
+
+columns:
+  ho_ten: 0
+  ngay_sinh: 1
+  so_bao_danh: 2
+  diem_thi: 3
+
+validation:
+  require_numeric_sbd: false
+  require_nonempty_name: true
+  require_nonempty_sbd: true
+
+header:
+  tokens: ["HO_TEN", "HỌ TÊN", "STT"]

parser/go.mod

@@ -0,0 +1,30 @@
+module github.com/tiennm99/thptqg/parser
+
+go 1.26.5
+
+require (
+	github.com/pbnjay/grate v0.0.0-20231006022435-3f8e65d74a14
+	github.com/xuri/excelize/v2 v2.11.0
+	golang.org/x/text v0.41.0
+	gopkg.in/yaml.v3 v3.0.1
+	modernc.org/sqlite v1.56.0
+)
+
+require (
+	github.com/dustin/go-humanize v1.0.1 // indirect
+	github.com/google/uuid v1.6.0 // indirect
+	github.com/mattn/go-isatty v0.0.24 // indirect
+	github.com/ncruces/go-strftime v1.0.0 // indirect
+	github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
+	github.com/richardlehane/mscfb v1.0.7 // indirect
+	github.com/richardlehane/msoleps v1.0.6 // indirect
+	github.com/tiendc/go-deepcopy v1.7.2 // indirect
+	github.com/xuri/efp v0.0.1 // indirect
+	github.com/xuri/nfp v0.0.2-0.20250530014748-2ddeb826f9a9 // indirect
+	golang.org/x/crypto v0.53.0 // indirect
+	golang.org/x/net v0.56.0 // indirect
+	golang.org/x/sys v0.47.0 // indirect
+	modernc.org/libc v1.74.4 // indirect
+	modernc.org/mathutil v1.7.1 // indirect
+	modernc.org/memory v1.11.0 // indirect
+)

parser/go.sum

@@ -0,0 +1,82 @@
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
+github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
+github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
+github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
+github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
+github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
+github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
+github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
+github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
+github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
+github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
+github.com/pbnjay/grate v0.0.0-20231006022435-3f8e65d74a14 h1:ZfXdW7GIVZT3Z9oejLJ+GHrrQv/ezU2Bwqn0BF37s4g=
+github.com/pbnjay/grate v0.0.0-20231006022435-3f8e65d74a14/go.mod h1:VaZEKQrYbYr2untVA/EFNdC6hM7GyARRNM+k4+5CmA0=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
+github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
+github.com/richardlehane/mscfb v1.0.7 h1:oeoiM0WE79vHwE8RpIYYvIAc8ajTH2mb6UZm55/+EB0=
+github.com/richardlehane/mscfb v1.0.7/go.mod h1:pe0+IUIc0AHh0+teNzBlJCtSyZdFOGgV4ZK9bsoV+Jo=
+github.com/richardlehane/msoleps v1.0.6 h1:9BvkpjvD+iUBalUY4esMwv6uBkfOip/Lzvd93jvR9gg=
+github.com/richardlehane/msoleps v1.0.6/go.mod h1:BWev5JBpU9Ko2WAgmZEuiz4/u3ZYTKbjLycmwiWUfWg=
+github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
+github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
+github.com/tiendc/go-deepcopy v1.7.2 h1:Ut2yYR7W9tWjTQitganoIue4UGxZwCcJy3orjrrIj44=
+github.com/tiendc/go-deepcopy v1.7.2/go.mod h1:4bKjNC2r7boYOkD2IOuZpYjmlDdzjbpTRyCx+goBCJQ=
+github.com/xuri/efp v0.0.1 h1:fws5Rv3myXyYni8uwj2qKjVaRP30PdjeYe2Y6FDsCL8=
+github.com/xuri/efp v0.0.1/go.mod h1:ybY/Jr0T0GTCnYjKqmdwxyxn2BQf2RcQIIvex5QldPI=
+github.com/xuri/excelize/v2 v2.11.0 h1:HxaEFl6sRN2+8J5a8HaKq+0M4FsjBGMnWWtjOCPSG88=
+github.com/xuri/excelize/v2 v2.11.0/go.mod h1:jxFLbzaIwGQ5ufFNvYfUOHqXhfPaNmP14KWfmNz2Uak=
+github.com/xuri/nfp v0.0.2-0.20250530014748-2ddeb826f9a9 h1:+C0TIdyyYmzadGaL/HBLbf3WdLgC29pgyhTjAT/0nuE=
+github.com/xuri/nfp v0.0.2-0.20250530014748-2ddeb826f9a9/go.mod h1:WwHg+CVyzlv/TX9xqBFXEZAuxOPxn2k1GNHwG41IIUQ=
+golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
+golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
+golang.org/x/image v0.38.0 h1:5l+q+Y9JDC7mBOMjo4/aPhMDcxEptsX+Tt3GgRQRPuE=
+golang.org/x/image v0.38.0/go.mod h1:/3f6vaXC+6CEanU4KJxbcUZyEePbyKbaLoDOe4ehFYY=
+golang.org/x/mod v0.38.0 h1:MECBjubtXD7yj4HrhIUcywNaGeNVUdfVnxmPajOk4yk=
+golang.org/x/mod v0.38.0/go.mod h1:V6Xz0pq8TQ3dGqVQ1FVHuelZpAL0uNhSkk9ogYP3c40=
+golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
+golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
+golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
+golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
+golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
+golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
+golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
+golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
+golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE=
+golang.org/x/tools v0.48.0/go.mod h1:08xX0orndb/F7jJxGDicx061tyd5pcMto75YMAXr6lk=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+modernc.org/cc/v4 v4.29.1 h1:MKgdCV3WykTSPqpVrnxdEDS0HEd2FHpKZDzxzU5LyeI=
+modernc.org/cc/v4 v4.29.1/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
+modernc.org/ccgo/v4 v4.34.6 h1:sBgfIwyN0TQ9C5hwIeuqyeAKyMWnbvj2fvpF4L11uzU=
+modernc.org/ccgo/v4 v4.34.6/go.mod h1:SZ8YcN9NG7XVsQYdm6jYBvi8PQP1qi+kqB6OhjqI3Fk=
+modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
+modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
+modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
+modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
+modernc.org/gc/v3 v3.1.4 h1:2g65LGVSmFQrXeITAw97x7hCRvZFcyE1uDP+7Vng7JI=
+modernc.org/gc/v3 v3.1.4/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
+modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
+modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
+modernc.org/libc v1.74.4 h1:fX1Omw4o2/1C2iRkkIsrQTasJQldLhRmuPreXLoWs9k=
+modernc.org/libc v1.74.4/go.mod h1:eeQAS9W3sZeKYMFubydxJpII9ybHWshk+7or7bLG9co=
+modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
+modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
+modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
+modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
+modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
+modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
+modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
+modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
+modernc.org/sqlite v1.56.0 h1:/D8e2RfFqoy/Zc6PuC76U28zFwmI/sYx1Kjm4yEn9e0=
+modernc.org/sqlite v1.56.0/go.mod h1:yCJ2cmAaIkHQ25oXWrF8H4O1lIfPYPR26yCEDj2P3pQ=
+modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
+modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
+modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
+modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=

parser/internal/audit/audit.go

@@ -0,0 +1,148 @@
+// Package audit compares distinct SBDs in the source spreadsheets against the
+// row count in a built database — a port of parser/src/audit.rs.
+//
+// Two deliberate divergences from the build path are preserved, both inherited
+// from audit-row-counts.js:
+//
+//   - only .xlsx files are considered (audit.rs:53-59)
+//   - only sheet 0 is read, regardless of sheet_mode (audit.rs:81-84)
+//
+// These are intentional, not oversights. Do not "fix" them.
+package audit
+
+import (
+	"database/sql"
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+
+	"github.com/tiennm99/thptqg/parser/internal/config"
+	"github.com/tiennm99/thptqg/parser/internal/ingest"
+	"github.com/tiennm99/thptqg/parser/internal/reader"
+	"github.com/tiennm99/thptqg/parser/internal/sqlitedb"
+)
+
+// Result carries the audit counters.
+type Result struct {
+	TotalDataRows uint64
+	BothEmpty     uint64
+	EmptyName     uint64
+	EmptySbd      uint64
+	DistinctSbds  int
+	DBCount       int64
+	Matched       bool
+}
+
+// Run collects distinct SBDs from the .xlsx files in inputDir and compares the
+// total against the student row count in dbPath.
+func Run(inputDir, dbPath string, cfg *config.DatasetConfig) (*Result, error) {
+	entries, err := os.ReadDir(inputDir)
+	if err != nil {
+		return nil, fmt.Errorf("read input dir %s: %w", inputDir, err)
+	}
+	var files []string
+	for _, e := range entries {
+		if e.IsDir() {
+			continue
+		}
+		if strings.EqualFold(filepath.Ext(e.Name()), ".xlsx") {
+			files = append(files, filepath.Join(inputDir, e.Name()))
+		}
+	}
+	sort.Strings(files)
+
+	res := &Result{}
+	seen := make(map[string]struct{})
+
+	// The audit uses positional defaults for format-detection configs, mirroring
+	// audit-row-counts.js's fixed column assumption (audit.rs:104-111).
+	hoTenCol, sbdCol := 1, 0
+	if cfg.Columns != nil {
+		hoTenCol, sbdCol = cfg.Columns.HoTen, cfg.Columns.SoBaoDanh
+	}
+
+	for _, file := range files {
+		wb, err := reader.Open(file)
+		if err != nil {
+			return nil, err
+		}
+		sheets := wb.Sheets()
+		if len(sheets) == 0 {
+			wb.Close()
+			continue
+		}
+
+		firstRow := true
+		err = wb.EachRow(sheets[0].Index, func(_ reader.Sheet, _ int, row []reader.Cell) error {
+			if firstRow {
+				firstRow = false
+				if ingest.IsHeaderRow(row, cfg.Header.Tokens) {
+					return nil
+				}
+			}
+			res.TotalDataRows++
+
+			hoTen := cellAt(row, hoTenCol)
+			sbd := cellAt(row, sbdCol)
+
+			if hoTen == "" && sbd == "" {
+				res.BothEmpty++
+				return nil
+			}
+			if hoTen == "" {
+				res.EmptyName++
+			}
+			if sbd == "" {
+				res.EmptySbd++
+			}
+			if sbd != "" {
+				seen[sbd] = struct{}{}
+			}
+			return nil
+		})
+		wb.Close()
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	// Opened read-only: the audit must never mutate the database it inspects
+	// (audit.rs:139).
+	db, err := sql.Open(sqlitedb.DriverName, "file:"+dbPath+"?mode=ro")
+	if err != nil {
+		return nil, fmt.Errorf("open db %s: %w", dbPath, err)
+	}
+	defer db.Close()
+	if err := db.QueryRow("SELECT COUNT(*) FROM student").Scan(&res.DBCount); err != nil {
+		return nil, fmt.Errorf("count rows: %w", err)
+	}
+
+	res.DistinctSbds = len(seen)
+	res.Matched = int64(res.DistinctSbds) == res.DBCount
+	return res, nil
+}
+
+// PrintReport mirrors audit-row-counts.js:54-62 exactly.
+func PrintReport(r *Result) {
+	fmt.Println("=== Source vs DB ===")
+	fmt.Printf("Source: total data rows across all files: %d\n", r.TotalDataRows)
+	fmt.Printf("Source: rows with empty name AND sbd (skipped): %d\n", r.BothEmpty)
+	fmt.Printf("Source: rows with missing name only: %d\n", r.EmptyName)
+	fmt.Printf("Source: rows with missing sbd only: %d\n", r.EmptySbd)
+	fmt.Printf("Source: distinct SBDs: %d\n", r.DistinctSbds)
+	fmt.Printf("DB:     row count: %d\n", r.DBCount)
+	if r.Matched {
+		fmt.Println("Match:  YES — all unique SBDs accounted for")
+	} else {
+		fmt.Printf("Match:  NO — gap of %d\n", int64(r.DistinctSbds)-r.DBCount)
+	}
+}
+
+func cellAt(row []reader.Cell, idx int) string {
+	if idx < 0 || idx >= len(row) {
+		return ""
+	}
+	return strings.TrimSpace(row[idx].Str)
+}

parser/internal/config/config.go

@@ -0,0 +1,113 @@
+// Package config loads per-dataset parse rules — a port of parser/src/config.rs.
+//
+// The config deliberately carries no SQL. The table shape, the INSERT and the
+// subject regexes are identical for every dataset and live in internal/schema;
+// keeping them here meant four copies of the same DDL, which is how the 2016 and
+// 2017 schemas drifted apart.
+package config
+
+import (
+	"bytes"
+	"fmt"
+	"os"
+
+	"gopkg.in/yaml.v3"
+)
+
+// SheetMode selects which sheets of a workbook are read.
+type SheetMode string
+
+const (
+	// SheetModeAll iterates every sheet, which is what recovers the Hanoi and
+	// HCM rows that overflow past Excel's 65,536-row cap into a second sheet.
+	SheetModeAll SheetMode = "all"
+	// SheetModeFirst reads sheet 0 only.
+	SheetModeFirst SheetMode = "first"
+)
+
+// valid reports whether m is one of the two values the Rust enum accepts.
+//
+// Checked after decoding rather than via a custom unmarshaler: the decoder
+// assigns named string types directly, so a typo would otherwise decode
+// silently and be read as "not all" downstream.
+func (m SheetMode) valid() bool {
+	return m == SheetModeAll || m == SheetModeFirst
+}
+
+// DatasetConfig is the per-dataset parse rule set.
+type DatasetConfig struct {
+	Reader ReaderCfg `yaml:"reader"`
+	// Columns holds fixed column indices. Nil when FormatDetection handles
+	// per-file mapping — a pointer rather than a value because a zero ColumnMap
+	// would silently mean "every column is index 0".
+	Columns    *ColumnMap    `yaml:"columns"`
+	Validation ValidationCfg `yaml:"validation"`
+	Header     HeaderCfg     `yaml:"header"`
+	// FormatDetection, when set to "thptqg2016", enables per-file format
+	// auto-detection: each file's header row is inspected at runtime to choose
+	// the column layout (separate-scores / mapped / default-positional).
+	FormatDetection *string `yaml:"format_detection"`
+}
+
+type ReaderCfg struct {
+	SheetMode SheetMode `yaml:"sheet_mode"`
+	// StripBlankRows skips rows where every cell is empty before counting them
+	// as source rows (a 2017-old2 quirk).
+	StripBlankRows bool `yaml:"strip_blank_rows"`
+}
+
+// ColumnMap holds zero-indexed column positions in the source row. Used by the
+// 2017 configs; 2016 uses runtime format detection instead.
+type ColumnMap struct {
+	HoTen     int `yaml:"ho_ten"`
+	NgaySinh  int `yaml:"ngay_sinh"`
+	SoBaoDanh int `yaml:"so_bao_danh"`
+	DiemThi   int `yaml:"diem_thi"`
+}
+
+type ValidationCfg struct {
+	// RequireNumericSbd mirrors build-database-old.js / -old2.js, which require
+	// so_bao_danh to match ^\d+$.
+	RequireNumericSbd   bool `yaml:"require_numeric_sbd"`
+	RequireNonemptyName bool `yaml:"require_nonempty_name"`
+	RequireNonemptySbd  bool `yaml:"require_nonempty_sbd"`
+}
+
+type HeaderCfg struct {
+	// Tokens are matched against the uppercased first cell to detect a header row.
+	Tokens []string `yaml:"tokens"`
+}
+
+// Parse decodes a config, rejecting any key the struct does not declare.
+//
+// Strictness is load-bearing and has its own test. Rust gets it from serde's
+// deny_unknown_fields; yaml.v3 needs KnownFields(true) explicitly, and Go YAML
+// decoders ignore unknown keys by default. Without it a leftover `schema:`
+// mapping would look effective while internal/schema actually drove the build —
+// the drift that produced two divergent schemas before the unification.
+func Parse(src []byte) (*DatasetConfig, error) {
+	var cfg DatasetConfig
+	dec := yaml.NewDecoder(bytes.NewReader(src))
+	dec.KnownFields(true)
+	if err := dec.Decode(&cfg); err != nil {
+		return nil, fmt.Errorf("decode config: %w", err)
+	}
+	if !cfg.Reader.SheetMode.valid() {
+		return nil, fmt.Errorf("invalid sheet_mode %q (want %q or %q)",
+			cfg.Reader.SheetMode, SheetModeAll, SheetModeFirst)
+	}
+	return &cfg, nil
+}
+
+// Load reads and parses a config file.
+func Load(path string) (*DatasetConfig, error) {
+	src, err := os.ReadFile(path)
+	if err != nil {
+		return nil, fmt.Errorf("read config %s: %w", path, err)
+	}
+	cfg, err := Parse(src)
+	if err != nil {
+		return nil, fmt.Errorf("%s: %w", path, err)
+	}
+	return cfg, nil
+}

parser/internal/config/config_test.go

@@ -0,0 +1,224 @@
+package config
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+// sampleYAML mirrors SAMPLE_YAML in parser/src/config.rs.
+const sampleYAML = `
+reader:
+  sheet_mode: all
+  strip_blank_rows: false
+
+columns:
+  ho_ten: 0
+  ngay_sinh: 1
+  so_bao_danh: 2
+  diem_thi: 3
+
+validation:
+  require_numeric_sbd: false
+  require_nonempty_name: true
+  require_nonempty_sbd: true
+
+header:
+  tokens: ["HO_TEN", "HỌ TÊN", "STT"]
+`
+
+// TestConfigRoundTrip ports config_round_trip (config.rs:115).
+func TestConfigRoundTrip(t *testing.T) {
+	cfg, err := Parse([]byte(sampleYAML))
+	if err != nil {
+		t.Fatalf("parse failed: %v", err)
+	}
+	if cfg.Reader.SheetMode != SheetModeAll {
+		t.Errorf("sheet_mode = %q, want all", cfg.Reader.SheetMode)
+	}
+	if cfg.Reader.StripBlankRows {
+		t.Error("strip_blank_rows should be false")
+	}
+	if cfg.Columns == nil {
+		t.Fatal("columns should be present")
+	}
+	if cfg.Columns.HoTen != 0 {
+		t.Errorf("ho_ten = %d, want 0", cfg.Columns.HoTen)
+	}
+	if cfg.Columns.DiemThi != 3 {
+		t.Errorf("diem_thi = %d, want 3", cfg.Columns.DiemThi)
+	}
+	if cfg.Validation.RequireNumericSbd {
+		t.Error("require_numeric_sbd should be false")
+	}
+	if !cfg.Validation.RequireNonemptyName {
+		t.Error("require_nonempty_name should be true")
+	}
+	if len(cfg.Header.Tokens) != 3 {
+		t.Errorf("tokens = %d, want 3", len(cfg.Header.Tokens))
+	}
+	if cfg.FormatDetection != nil {
+		t.Errorf("format_detection = %v, want nil", *cfg.FormatDetection)
+	}
+}
+
+// TestConfigRejectsLeftoverSQLSections ports config_rejects_leftover_sql_sections
+// (config.rs:132). This is the load-bearing one: Rust gets the behaviour free
+// from serde's deny_unknown_fields, whereas Go YAML decoders ignore unknown keys
+// unless KnownFields(true) is set. Without it a stale schema: mapping would look effective while
+// internal/schema silently drove the build — exactly the drift that produced two
+// divergent schemas before the unification.
+func TestConfigRejectsLeftoverSQLSections(t *testing.T) {
+	withDDL := sampleYAML + "\nschema:\n  ddl: \"CREATE TABLE student (so_bao_danh TEXT);\"\n"
+	if _, err := Parse([]byte(withDDL)); err == nil {
+		t.Fatal("config with a leftover schema: mapping must be rejected")
+	}
+}
+
+// TestConfigRejectsUnknownScalarKey guards the same property for a stray scalar,
+// not just a stray mapping.
+func TestConfigRejectsUnknownScalarKey(t *testing.T) {
+	withKey := sampleYAML + "\nunexpected_key: 1\n"
+	if _, err := Parse([]byte(withKey)); err == nil {
+		t.Fatal("config with an unknown scalar key must be rejected")
+	}
+}
+
+// TestConfigFirstSheetMode ports config_first_sheet_mode (config.rs:140).
+func TestConfigFirstSheetMode(t *testing.T) {
+	src := []byte(replaceAll(sampleYAML, "sheet_mode: all", "sheet_mode: first"))
+	cfg, err := Parse(src)
+	if err != nil {
+		t.Fatalf("parse failed: %v", err)
+	}
+	if cfg.Reader.SheetMode != SheetModeFirst {
+		t.Errorf("sheet_mode = %q, want first", cfg.Reader.SheetMode)
+	}
+}
+
+// TestConfigRejectsUnknownSheetMode: the Rust enum accepts only "all"/"first",
+// so anything else must fail rather than defaulting.
+func TestConfigRejectsUnknownSheetMode(t *testing.T) {
+	src := []byte(replaceAll(sampleYAML, "sheet_mode: all", "sheet_mode: second"))
+	if _, err := Parse(src); err == nil {
+		t.Fatal("unknown sheet_mode must be rejected")
+	}
+}
+
+// TestConfigFormatDetectionField ports config_format_detection_field
+// (config.rs:147): a 2016-style config has no columns: mapping at all.
+func TestConfigFormatDetectionField(t *testing.T) {
+	const src = `
+format_detection: thptqg2016
+
+reader:
+  sheet_mode: all
+  strip_blank_rows: false
+
+validation:
+  require_numeric_sbd: false
+  require_nonempty_name: true
+  require_nonempty_sbd: true
+
+header:
+  tokens: ["SBD", "SOBAODANH", "STT"]
+`
+	cfg, err := Parse([]byte(src))
+	if err != nil {
+		t.Fatalf("parse failed: %v", err)
+	}
+	if cfg.FormatDetection == nil || *cfg.FormatDetection != "thptqg2016" {
+		t.Errorf("format_detection = %v, want thptqg2016", cfg.FormatDetection)
+	}
+	if cfg.Columns != nil {
+		t.Error("columns must be nil when format_detection drives the layout")
+	}
+}
+
+// TestLoadRealConfigs loads the shipped configs and asserts the per-dataset
+// differences recorded during scouting.
+func TestLoadRealConfigs(t *testing.T) {
+	root := repoRoot(t)
+	want := map[string]struct {
+		sheetMode  SheetMode
+		stripBlank bool
+		numericSbd bool
+		hasColumns bool
+		formatDet  string
+		tokenCount int
+	}{
+		"2016": {SheetModeAll, false, false, false, "thptqg2016", 6},
+		"2017": {SheetModeAll, false, false, true, "", 3},
+	}
+	for id, w := range want {
+		t.Run(id, func(t *testing.T) {
+			cfg, err := Load(filepath.Join(root, "parser", "configs", id+".yml"))
+			if err != nil {
+				t.Fatalf("load: %v", err)
+			}
+			if cfg.Reader.SheetMode != w.sheetMode {
+				t.Errorf("sheet_mode = %q, want %q", cfg.Reader.SheetMode, w.sheetMode)
+			}
+			if cfg.Reader.StripBlankRows != w.stripBlank {
+				t.Errorf("strip_blank_rows = %v, want %v", cfg.Reader.StripBlankRows, w.stripBlank)
+			}
+			if cfg.Validation.RequireNumericSbd != w.numericSbd {
+				t.Errorf("require_numeric_sbd = %v, want %v", cfg.Validation.RequireNumericSbd, w.numericSbd)
+			}
+			if (cfg.Columns != nil) != w.hasColumns {
+				t.Errorf("columns present = %v, want %v", cfg.Columns != nil, w.hasColumns)
+			}
+			got := ""
+			if cfg.FormatDetection != nil {
+				got = *cfg.FormatDetection
+			}
+			if got != w.formatDet {
+				t.Errorf("format_detection = %q, want %q", got, w.formatDet)
+			}
+			if len(cfg.Header.Tokens) != w.tokenCount {
+				t.Errorf("tokens = %d, want %d", len(cfg.Header.Tokens), w.tokenCount)
+			}
+			// Every dataset requires a non-empty name and SBD.
+			if !cfg.Validation.RequireNonemptyName || !cfg.Validation.RequireNonemptySbd {
+				t.Error("both non-empty validations should be true for every dataset")
+			}
+		})
+	}
+}
+
+func replaceAll(s, old, new string) string {
+	out := ""
+	for {
+		i := indexOf(s, old)
+		if i < 0 {
+			return out + s
+		}
+		out += s[:i] + new
+		s = s[i+len(old):]
+	}
+}
+
+func indexOf(s, sub string) int {
+	for i := 0; i+len(sub) <= len(s); i++ {
+		if s[i:i+len(sub)] == sub {
+			return i
+		}
+	}
+	return -1
+}
+
+func repoRoot(t *testing.T) string {
+	t.Helper()
+	dir, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	for i := 0; i < 6; i++ {
+		if _, err := os.Stat(filepath.Join(dir, "parser", "configs")); err == nil {
+			return dir
+		}
+		dir = filepath.Dir(dir)
+	}
+	t.Fatal("could not locate repo root")
+	return ""
+}

parser/internal/ingest/detect2016.go

@@ -0,0 +1,382 @@
+package ingest
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"strconv"
+	"strings"
+
+	"github.com/tiennm99/thptqg/parser/internal/config"
+	"github.com/tiennm99/thptqg/parser/internal/reader"
+	"github.com/tiennm99/thptqg/parser/internal/transform"
+	"github.com/tiennm99/thptqg/parser/internal/writer"
+)
+
+// The 2016 dataset's 119 files were produced by inconsistent tooling and use
+// three different column layouts, chosen per sheet at runtime. This is a port of
+// parser/src/format_detect_2016.rs — institutional knowledge encoded as
+// literals, with no abstraction to derive it from, so everything here is copied
+// verbatim rather than rationalised.
+
+// KnownHeaders are the upper-cased first-cell values that identify a header row
+// (format_detect_2016.rs:36-54, mirroring KNOWN_HEADERS in build-database.js).
+//
+// NOTE "SINH " carries a TRAILING SPACE, exactly as in the Rust source. Trimming
+// it would change which rows are recognised as headers.
+var KnownHeaders = []string{
+	"SOBAODANH",
+	"SBD",
+	"HO_TEN",
+	"HOTEN",
+	"HỌ TÊN",
+	"NGAY_SINH",
+	"TEN_CUMTHI",
+	"GIOI_TINH",
+	"DIEM_THI",
+	"STT",
+	"TOAN",
+	"VAN",
+	"LY",
+	"HOA",
+	"SINH ",
+	"SU",
+	"DIA",
+}
+
+func isKnownHeader(s string) bool {
+	for _, h := range KnownHeaders {
+		if h == s {
+			return true
+		}
+	}
+	return false
+}
+
+// IsHeaderRow2016 reports whether row[0] is a known header token
+// (format_detect_2016.rs:58-64).
+//
+// The guard here is len < 2, not the len < 3 used by the 2017 header check.
+func IsHeaderRow2016(row []reader.Cell) bool {
+	if len(row) < 2 {
+		return false
+	}
+	return isKnownHeader(strings.ToUpper(strings.TrimSpace(row[0].Str)))
+}
+
+// FormatKind is one of the three 2016 layouts.
+type FormatKind int
+
+const (
+	// FormatSeparateScores has one column per subject rather than a free-text
+	// DIEM_THI cell — the dhhanghai-style files.
+	FormatSeparateScores FormatKind = iota
+	// FormatMapped resolves column indices from the header by name.
+	FormatMapped
+	// FormatDefault is the positional 6-column layout used when no header is
+	// recognised. It is FormatMapped with fixed indices, not a separate path.
+	FormatDefault
+)
+
+// Format is a detected layout plus, for the mapped case, its column indices.
+type Format struct {
+	Kind      FormatKind
+	Sbd       int
+	HoTen     int
+	NgaySinh  *int
+	TenCumThi *int
+	GioiTinh  *int
+	DiemThi   int
+}
+
+// defaultFormat is FormatMapped with the fixed positional indices
+// (format_detect_2016.rs:295-306).
+func defaultFormat() Format {
+	two, three, four := 2, 3, 4
+	return Format{
+		Kind: FormatDefault, Sbd: 0, HoTen: 1,
+		NgaySinh: &two, TenCumThi: &three, GioiTinh: &four, DiemThi: 5,
+	}
+}
+
+// DetectFormat inspects a header row and decides which layout applies
+// (format_detect_2016.rs:95-145).
+func DetectFormat(headerRow []reader.Cell) Format {
+	cols := make([]string, len(headerRow))
+	for i, c := range headerRow {
+		cols[i] = strings.ToUpper(strings.TrimSpace(c.Str))
+	}
+
+	// Format 1: SBD in col 0 AND TOAN in col 2.
+	if len(cols) > 2 && cols[0] == "SBD" && cols[2] == "TOAN" {
+		return Format{Kind: FormatSeparateScores}
+	}
+
+	// Format 2: resolve indices by header name, order-independent.
+	var sbdIdx, hoTenIdx, ngaySinhIdx, tenCumThiIdx, gioiTinhIdx, diemThiIdx *int
+	for i := range cols {
+		idx := i
+		switch cols[i] {
+		case "SOBAODANH", "SBD":
+			sbdIdx = &idx
+		case "HO_TEN", "HOTEN", "HỌ TÊN":
+			hoTenIdx = &idx
+		case "NGAY_SINH":
+			ngaySinhIdx = &idx
+		case "TEN_CUMTHI":
+			tenCumThiIdx = &idx
+		case "GIOI_TINH":
+			gioiTinhIdx = &idx
+		case "DIEM_THI":
+			diemThiIdx = &idx
+		}
+	}
+
+	if sbdIdx != nil && diemThiIdx != nil {
+		hoTen := 1 // fallback: col 1, present in all known files
+		if hoTenIdx != nil {
+			hoTen = *hoTenIdx
+		}
+		return Format{
+			Kind: FormatMapped, Sbd: *sbdIdx, HoTen: hoTen,
+			NgaySinh: ngaySinhIdx, TenCumThi: tenCumThiIdx, GioiTinh: gioiTinhIdx,
+			DiemThi: *diemThiIdx,
+		}
+	}
+
+	// Unrecognised header, or none at all.
+	return defaultFormat()
+}
+
+// parseFloatCell parses a per-subject score cell.
+//
+// A parsed 0.0 becomes "no score" — this replicates JavaScript's
+// `parseFloat(row[N]) || null`, where 0 is falsy (format_detect_2016.rs:165).
+// It means a genuine zero is indistinguishable from a blank. Not obviously
+// correct, but it is the shipped behaviour and the published data depends on it.
+func parseFloatCell(row []reader.Cell, idx int) (float64, bool) {
+	s := cellAt(row, idx)
+	if s == "" {
+		return 0, false
+	}
+	v, err := strconv.ParseFloat(s, 64)
+	if err != nil || v == 0.0 {
+		return 0, false
+	}
+	return v, true
+}
+
+// processSeparateScoresRow handles the fixed 12-column layout
+// (format_detect_2016.rs:176-216):
+//
+//	0=SBD 1=HOTEN 2=TOAN 3=VAN 4=LY 5=HOA 6=SINH 7=SU 8=DIA
+//	9=NGOAINGUTN 10=NGOAINGUTL 11=NGOAINGU(total -> tieng_anh)
+//
+// tieng_phap / tieng_duc / tieng_nhat / tieng_trung are structurally unreachable
+// in this format, and ngay_sinh / ten_cum_thi / gioi_tinh are always nil.
+// Scores are read as floats directly; this layout has no free-text score cell,
+// so the subject regexes never run.
+func processSeparateScoresRow(row []reader.Cell) *transform.ParsedRow {
+	sbd := cellAt(row, 0)
+	hoTen := cellAt(row, 1)
+	if sbd == "" || hoTen == "" {
+		return nil
+	}
+
+	scores := make(map[string]float64)
+	for field, idx := range map[string]int{
+		"toan": 2, "ngu_van": 3, "vat_ly": 4, "hoa_hoc": 5,
+		"sinh_hoc": 6, "lich_su": 7, "dia_ly": 8,
+		"tieng_anh": 11, // NGOAINGU total
+	} {
+		if v, ok := parseFloatCell(row, idx); ok {
+			scores[field] = v
+		}
+	}
+
+	return &transform.ParsedRow{
+		SoBaoDanh:  sbd,
+		HoTen:      hoTen,
+		HoTenAscii: transform.ToAscii(hoTen),
+		Scores:     scores,
+	}
+}
+
+// processMappedRow handles header-derived column indices
+// (format_detect_2016.rs:226-289).
+func processMappedRow(row []reader.Cell, f Format) *transform.ParsedRow {
+	sbd := cellAt(row, f.Sbd)
+	hoTen := cellAt(row, f.HoTen)
+	if sbd == "" || hoTen == "" {
+		return nil
+	}
+
+	// Leaked-header guard: a row whose SBD or name cell is itself a header token
+	// is a repeated header, not data (format_detect_2016.rs:244-250).
+	if isKnownHeader(strings.ToUpper(sbd)) || isKnownHeader(strings.ToUpper(hoTen)) {
+		return nil
+	}
+
+	optional := func(idx *int) *string {
+		if idx == nil {
+			return nil
+		}
+		if s := cellAt(row, *idx); s != "" {
+			return &s
+		}
+		return nil
+	}
+
+	// Gender is a two-value allowlist, not a general enum: anything other than
+	// exactly "Nam" or "Nữ" becomes nil (format_detect_2016.rs:263-271).
+	var gioiTinh *string
+	if f.GioiTinh != nil {
+		if s := cellAt(row, *f.GioiTinh); s == "Nam" || s == "Nữ" {
+			gioiTinh = &s
+		}
+	}
+
+	// Read untrimmed, matching format_detect_2016.rs:273-276.
+	diemThi := ""
+	if f.DiemThi >= 0 && f.DiemThi < len(row) {
+		diemThi = row[f.DiemThi].Str
+	}
+
+	return &transform.ParsedRow{
+		SoBaoDanh:  sbd,
+		HoTen:      hoTen,
+		HoTenAscii: transform.ToAscii(hoTen),
+		NgaySinh:   optional(f.NgaySinh),
+		TenCumThi:  optional(f.TenCumThi),
+		GioiTinh:   gioiTinh,
+		Scores:     transform.ParseScores(diemThi),
+	}
+}
+
+// ProcessRow2016 dispatches a data row through the detected layout. A nil return
+// means the row is empty or invalid and should be skipped.
+func ProcessRow2016(row []reader.Cell, f Format) *transform.ParsedRow {
+	if f.Kind == FormatSeparateScores {
+		return processSeparateScoresRow(row)
+	}
+	// FormatMapped and FormatDefault share one implementation; Default is just a
+	// fixed index tuple.
+	return processMappedRow(row, f)
+}
+
+// Detect2016 ingests the 2016 dataset — the port of run_build_2016 and
+// process_file_2016 (main.rs:211-377).
+func Detect2016(cfg *config.DatasetConfig, inputDir, outputPath string) error {
+	files, err := InputFiles(inputDir)
+	if err != nil {
+		return err
+	}
+	label := DatasetLabel(inputDir)
+	fmt.Printf("[build:2016] %s/ → %s  (%d files)\n", label, outputPath, len(files))
+
+	db, err := writer.OpenDB(outputPath)
+	if err != nil {
+		return err
+	}
+	defer db.Close()
+
+	tx, err := db.Begin()
+	if err != nil {
+		return fmt.Errorf("begin: %w", err)
+	}
+	ins, err := writer.Prepare(tx)
+	if err != nil {
+		tx.Rollback()
+		return err
+	}
+
+	var st writer.Stats // Skipped stays 0 on this path, matching main.rs:251
+	for _, file := range files {
+		base := filepath.Base(file)
+		fileRows, err := processFile2016(file, cfg, ins, base, &st)
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "  [error] %s: %v\n", base, err)
+			st.Errors++
+			continue
+		}
+		fmt.Printf("  %s: %d rows\n", base, fileRows)
+	}
+
+	if err := ins.Close(); err != nil {
+		tx.Rollback()
+		return err
+	}
+	if err := tx.Commit(); err != nil {
+		return fmt.Errorf("commit: %w", err)
+	}
+	return writer.Finish(db, outputPath, st, label, false)
+}
+
+// processFile2016 detects the layout per SHEET and processes that sheet's rows.
+//
+// Detection is per sheet, not per file (main.rs:344-349): within one workbook,
+// sheet 2 may legitimately detect differently from sheet 1, which matters for
+// the province files that overflow past Excel's row cap.
+func processFile2016(path string, cfg *config.DatasetConfig, ins *writer.Inserter, base string, st *writer.Stats) (uint64, error) {
+	wb, err := reader.Open(path)
+	if err != nil {
+		return 0, err
+	}
+	defer wb.Close()
+
+	sheets := wb.Sheets()
+	if len(sheets) == 0 {
+		return 0, nil
+	}
+	if cfg.Reader.SheetMode == config.SheetModeFirst {
+		sheets = sheets[:1]
+	}
+
+	var fileRows uint64
+	for _, sh := range sheets {
+		var rows [][]reader.Cell
+		if err := wb.EachRow(sh.Index, func(_ reader.Sheet, _ int, row []reader.Cell) error {
+			rows = append(rows, row)
+			return nil
+		}); err != nil {
+			return fileRows, err
+		}
+		if len(rows) == 0 {
+			continue
+		}
+
+		f := defaultFormat()
+		startIdx := 0
+		if IsHeaderRow2016(rows[0]) {
+			f = DetectFormat(rows[0])
+			startIdx = 1
+		}
+
+		for _, row := range rows[startIdx:] {
+			// Rows shorter than 2 cells are dropped BEFORE the counter
+			// (main.rs:351-353), so they never appear in the source total.
+			if len(row) < 2 {
+				continue
+			}
+			st.SourceRows++
+
+			parsed := ProcessRow2016(row, f)
+			if parsed == nil {
+				// Empty or invalid. Note this is NOT counted as skipped — the
+				// 2016 path leaves that counter at zero (main.rs:251), so the
+				// stats block reports insertable == source rows and the Audit
+				// line absorbs the difference.
+				continue
+			}
+			if err := ins.Insert(parsed); err != nil {
+				st.Errors++
+				if st.Errors <= 5 {
+					fmt.Fprintf(os.Stderr, "  [warn] %s: %v\n", base, err)
+				}
+				continue
+			}
+			fileRows++
+		}
+	}
+	return fileRows, nil
+}

parser/internal/ingest/detect2016_test.go

@@ -0,0 +1,252 @@
+package ingest
+
+import (
+	"testing"
+
+	"github.com/tiennm99/thptqg/parser/internal/reader"
+)
+
+// Ports the 11 tests in parser/src/format_detect_2016.rs, plus a guard per quirk.
+//
+// Phase 1 settled the Data -> string translation these fixtures need, so no
+// guessing: Data::String(s) is s verbatim, Data::Float(8.0) renders "8" (Rust's
+// f64 Display drops the .0, matching Go's FormatFloat(v,'f',-1,64)),
+// Data::Float(8.5) renders "8.5", and Data::Empty is "" with IsEmpty true.
+
+// --- header detection ---
+
+func TestIsHeaderRow2016(t *testing.T) {
+	if !IsHeaderRow2016(cells("SBD", "HOTEN", "TOAN")) {
+		t.Error("SBD header not detected")
+	}
+	if !IsHeaderRow2016(cells("sobaodanh", "x")) {
+		t.Error("lowercase SOBAODANH not detected")
+	}
+	if IsHeaderRow2016(cells("Nguyen Van A", "01/01/2000")) {
+		t.Error("data row wrongly detected as header")
+	}
+	// The 2016 guard is len < 2, unlike the 2017 check's len < 3.
+	if IsHeaderRow2016(cells("SBD")) {
+		t.Error("single-cell row must not be a header")
+	}
+	if !IsHeaderRow2016(cells("SBD", "x")) {
+		t.Error("two-cell row with a header token must be a header")
+	}
+}
+
+// TestKnownHeadersHasTrailingSpaceToken pins the "SINH " literal. Trimming it
+// would silently change which rows count as headers.
+func TestKnownHeadersHasTrailingSpaceToken(t *testing.T) {
+	var found bool
+	for _, h := range KnownHeaders {
+		if h == "SINH " {
+			found = true
+		}
+	}
+	if !found {
+		t.Error(`KnownHeaders must contain "SINH " WITH its trailing space`)
+	}
+	if len(KnownHeaders) != 17 {
+		t.Errorf("KnownHeaders has %d tokens, want 17", len(KnownHeaders))
+	}
+}
+
+// --- format detection ---
+
+func TestDetectFormatSeparateScores(t *testing.T) {
+	f := DetectFormat(cells("SBD", "HOTEN", "TOAN", "VAN"))
+	if f.Kind != FormatSeparateScores {
+		t.Errorf("Kind = %v, want FormatSeparateScores", f.Kind)
+	}
+}
+
+func TestDetectFormatMapped(t *testing.T) {
+	f := DetectFormat(cells("STT", "SOBAODANH", "HO_TEN", "NGAY_SINH", "TEN_CUMTHI", "GIOI_TINH", "DIEM_THI"))
+	if f.Kind != FormatMapped {
+		t.Fatalf("Kind = %v, want FormatMapped", f.Kind)
+	}
+	if f.Sbd != 1 || f.HoTen != 2 || f.DiemThi != 6 {
+		t.Errorf("indices sbd=%d ho_ten=%d diem_thi=%d", f.Sbd, f.HoTen, f.DiemThi)
+	}
+	if f.NgaySinh == nil || *f.NgaySinh != 3 || f.TenCumThi == nil || *f.TenCumThi != 4 || f.GioiTinh == nil || *f.GioiTinh != 5 {
+		t.Error("optional indices not resolved")
+	}
+}
+
+// TestDetectFormatMappedIsOrderIndependent: indices are resolved by name.
+func TestDetectFormatMappedIsOrderIndependent(t *testing.T) {
+	f := DetectFormat(cells("DIEM_THI", "HO_TEN", "SBD"))
+	if f.Kind != FormatMapped || f.Sbd != 2 || f.DiemThi != 0 || f.HoTen != 1 {
+		t.Errorf("got %+v", f)
+	}
+}
+
+// TestDetectFormatMappedHoTenFallback ports the col-1 fallback
+// (format_detect_2016.rs:132).
+func TestDetectFormatMappedHoTenFallback(t *testing.T) {
+	f := DetectFormat(cells("SBD", "SOMETHING", "DIEM_THI"))
+	if f.Kind != FormatMapped {
+		t.Fatalf("Kind = %v, want FormatMapped", f.Kind)
+	}
+	if f.HoTen != 1 {
+		t.Errorf("ho_ten = %d, want fallback 1", f.HoTen)
+	}
+}
+
+// TestDetectFormatDefault: a header lacking SBD or DIEM_THI falls back to the
+// positional layout.
+func TestDetectFormatDefault(t *testing.T) {
+	f := DetectFormat(cells("A", "B", "C"))
+	if f.Kind != FormatDefault {
+		t.Errorf("Kind = %v, want FormatDefault", f.Kind)
+	}
+	if f.Sbd != 0 || f.HoTen != 1 || f.DiemThi != 5 {
+		t.Errorf("default indices wrong: %+v", f)
+	}
+}
+
+// --- row processing ---
+
+func TestProcessSeparateScoresRow(t *testing.T) {
+	// 0=SBD 1=HOTEN 2=TOAN 3=VAN 4=LY 5=HOA 6=SINH 7=SU 8=DIA 9,10=NN 11=NN total
+	row := cells("1000", "Nguyễn Văn Đức", "8", "7.5", "", "6", "", "5", "", "", "", "9.25")
+	got := ProcessRow2016(row, Format{Kind: FormatSeparateScores})
+	if got == nil {
+		t.Fatal("row rejected")
+	}
+	if got.SoBaoDanh != "1000" || got.HoTenAscii != "nguyen van duc" {
+		t.Errorf("sbd=%q ascii=%q", got.SoBaoDanh, got.HoTenAscii)
+	}
+	want := map[string]float64{"toan": 8, "ngu_van": 7.5, "hoa_hoc": 6, "lich_su": 5, "tieng_anh": 9.25}
+	if len(got.Scores) != len(want) {
+		t.Errorf("scores = %v, want %v", got.Scores, want)
+	}
+	for k, v := range want {
+		if got.Scores[k] != v {
+			t.Errorf("%s = %v, want %v", k, got.Scores[k], v)
+		}
+	}
+	// These columns are structurally unreachable in this layout.
+	for _, absent := range []string{"tieng_phap", "tieng_duc", "tieng_nhat", "tieng_trung", "khtn", "khxh", "gdcd"} {
+		if _, ok := got.Scores[absent]; ok {
+			t.Errorf("%s must be unreachable in separate-scores", absent)
+		}
+	}
+	if got.NgaySinh != nil || got.TenCumThi != nil || got.GioiTinh != nil {
+		t.Error("ngay_sinh/ten_cum_thi/gioi_tinh are always nil in separate-scores")
+	}
+}
+
+// TestZeroScoreBecomesNull pins the JS falsy quirk: parseFloat(x) || null means
+// a literal 0 is indistinguishable from "no score" (format_detect_2016.rs:165).
+func TestZeroScoreBecomesNull(t *testing.T) {
+	row := cells("1000", "A", "0", "0.0", "1", "", "", "", "", "", "", "")
+	got := ProcessRow2016(row, Format{Kind: FormatSeparateScores})
+	if got == nil {
+		t.Fatal("row rejected")
+	}
+	if _, ok := got.Scores["toan"]; ok {
+		t.Error(`a "0" score must become NULL, not 0`)
+	}
+	if _, ok := got.Scores["ngu_van"]; ok {
+		t.Error(`a "0.0" score must become NULL, not 0`)
+	}
+	if got.Scores["vat_ly"] != 1 {
+		t.Error("a non-zero score must survive")
+	}
+}
+
+// TestGenderAllowlist ports format_detect_2016.rs:263-271 — exactly two values.
+func TestGenderAllowlist(t *testing.T) {
+	f := defaultFormat()
+	for in, want := range map[string]string{"Nam": "Nam", "Nữ": "Nữ"} {
+		row := cells("1", "A", "", "", in, "")
+		got := ProcessRow2016(row, f)
+		if got == nil || got.GioiTinh == nil || *got.GioiTinh != want {
+			t.Errorf("gioi_tinh for %q not preserved", in)
+		}
+	}
+	for _, in := range []string{"nam", "NAM", "Unknown", "M", "F", "nữ", ""} {
+		row := cells("1", "A", "", "", in, "")
+		got := ProcessRow2016(row, f)
+		if got == nil {
+			t.Fatalf("row rejected for %q", in)
+		}
+		if got.GioiTinh != nil {
+			t.Errorf("gioi_tinh for %q = %q, want nil", in, *got.GioiTinh)
+		}
+	}
+}
+
+// TestLeakedHeaderRowSkipped ports format_detect_2016.rs:244-250.
+func TestLeakedHeaderRowSkipped(t *testing.T) {
+	f := defaultFormat()
+	if ProcessRow2016(cells("SBD", "HO_TEN", "", "", "", ""), f) != nil {
+		t.Error("a repeated header row must be skipped")
+	}
+	if ProcessRow2016(cells("123", "SOBAODANH", "", "", "", ""), f) != nil {
+		t.Error("a row whose name cell is a header token must be skipped")
+	}
+	if ProcessRow2016(cells("123", "Nguyen Van A", "", "", "", ""), f) == nil {
+		t.Error("a genuine data row must not be skipped")
+	}
+}
+
+// TestMappedRowEmptyFieldsRejected: either identity field empty rejects the row.
+func TestMappedRowEmptyFieldsRejected(t *testing.T) {
+	f := defaultFormat()
+	if ProcessRow2016(cells("", "A", "", "", "", ""), f) != nil {
+		t.Error("empty sbd must reject")
+	}
+	if ProcessRow2016(cells("1", "", "", "", "", ""), f) != nil {
+		t.Error("empty ho_ten must reject")
+	}
+}
+
+// TestMappedRowPopulates2016OnlyColumns: this is the only dataset that fills
+// ten_cum_thi and gioi_tinh.
+func TestMappedRowPopulates2016OnlyColumns(t *testing.T) {
+	row := cells("123", "Lê Văn Long", "01/01/1998", "Cụm thi số 1", "Nam", "Toán: 7.5  Tiếng Đức: 6")
+	got := ProcessRow2016(row, defaultFormat())
+	if got == nil {
+		t.Fatal("row rejected")
+	}
+	if got.NgaySinh == nil || *got.NgaySinh != "01/01/1998" {
+		t.Errorf("ngay_sinh = %v", got.NgaySinh)
+	}
+	if got.TenCumThi == nil || *got.TenCumThi != "Cụm thi số 1" {
+		t.Errorf("ten_cum_thi = %v", got.TenCumThi)
+	}
+	if got.Scores["toan"] != 7.5 || got.Scores["tieng_duc"] != 6 {
+		t.Errorf("scores = %v", got.Scores)
+	}
+}
+
+// TestDefaultIsMappedWithFixedIndices: FormatDefault must not be a separate code
+// path (format_detect_2016.rs:295-306).
+func TestDefaultIsMappedWithFixedIndices(t *testing.T) {
+	row := cells("123", "A", "01/01/2000", "Cluster", "Nữ", "Toán: 5")
+	viaDefault := ProcessRow2016(row, defaultFormat())
+	two, three, four := 2, 3, 4
+	viaMapped := ProcessRow2016(row, Format{
+		Kind: FormatMapped, Sbd: 0, HoTen: 1,
+		NgaySinh: &two, TenCumThi: &three, GioiTinh: &four, DiemThi: 5,
+	})
+	if viaDefault == nil || viaMapped == nil {
+		t.Fatal("row rejected")
+	}
+	if viaDefault.SoBaoDanh != viaMapped.SoBaoDanh ||
+		*viaDefault.TenCumThi != *viaMapped.TenCumThi ||
+		*viaDefault.GioiTinh != *viaMapped.GioiTinh ||
+		viaDefault.Scores["toan"] != viaMapped.Scores["toan"] {
+		t.Error("FormatDefault must behave identically to the equivalent FormatMapped")
+	}
+}
+
+// TestShortRowGuardIsSeparateFromValidation documents that rows under 2 cells
+// are dropped by the loop before the counter (main.rs:351-353), not here.
+func TestShortRowGuardIsSeparateFromValidation(t *testing.T) {
+	if got := ProcessRow2016([]reader.Cell{{Str: "123"}}, defaultFormat()); got != nil {
+		t.Error("a 1-cell row has no name and must be rejected")
+	}
+}

parser/internal/ingest/ingest.go

@@ -0,0 +1,243 @@
+// Package ingest owns all dataset policy: which sheets to read, which rows are
+// headers, which are blank, and how rows are counted.
+//
+// The reader deliberately has none of this — it reports every sheet and every
+// row exactly as calamine would, which is what made its fidelity independently
+// testable. This package ports the build loop in parser/src/main.rs plus the
+// header/blank helpers in parser/src/reader.rs.
+package ingest
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+
+	"github.com/tiennm99/thptqg/parser/internal/config"
+	"github.com/tiennm99/thptqg/parser/internal/reader"
+	"github.com/tiennm99/thptqg/parser/internal/transform"
+	"github.com/tiennm99/thptqg/parser/internal/writer"
+)
+
+// IsHeaderRow reports whether row is a header, by matching its uppercased first
+// cell against the configured tokens (reader.rs:28-34).
+//
+// Rows shorter than 3 cells are never headers (reader.rs:29) — a 1- or 2-cell
+// row is a stray fragment, not a real header.
+func IsHeaderRow(row []reader.Cell, tokens []string) bool {
+	if len(row) < 3 {
+		return false
+	}
+	first := strings.ToUpper(strings.TrimSpace(row[0].Str))
+	for _, t := range tokens {
+		if strings.ToUpper(t) == first {
+			return true
+		}
+	}
+	return false
+}
+
+// IsAllBlank reports whether every cell is empty or whitespace-only
+// (reader.rs:40-43).
+//
+// Compares on Str only. Cell.IsEmpty is diagnostic: calamine distinguishes
+// Data::Empty from an empty string cell, but both render "" and both count as
+// blank here, so branching on the flag would invent a distinction the Rust
+// original never acts on.
+func IsAllBlank(row []reader.Cell) bool {
+	for _, c := range row {
+		if strings.TrimSpace(c.Str) != "" {
+			return false
+		}
+	}
+	return true
+}
+
+// InputFiles lists a dataset directory's spreadsheets, sorted.
+//
+// The sort is load-bearing, not cosmetic: INSERT OR REPLACE is last-wins, so
+// file order decides which row survives a duplicate SBD. Rust collects read_dir
+// then calls files.sort() (main.rs:82-97) — a bytewise sort on the full path.
+func InputFiles(dir string) ([]string, error) {
+	entries, err := os.ReadDir(dir)
+	if err != nil {
+		return nil, fmt.Errorf("cannot read input dir %s: %w", dir, err)
+	}
+	var out []string
+	for _, e := range entries {
+		if e.IsDir() {
+			continue
+		}
+		switch strings.ToLower(filepath.Ext(e.Name())) {
+		case ".xls", ".xlsx":
+			out = append(out, filepath.Join(dir, e.Name()))
+		}
+	}
+	sort.Strings(out)
+	return out, nil
+}
+
+// DatasetLabel derives the stats-wording key from the input directory basename,
+// matching main.rs:98-101.
+func DatasetLabel(inputDir string) string {
+	base := filepath.Base(strings.TrimRight(inputDir, string(filepath.Separator)))
+	if base == "" || base == "." || base == string(filepath.Separator) {
+		return "data"
+	}
+	return base
+}
+
+// RowFn consumes one data row of one sheet, after header skipping.
+type RowFn func(sheetIdx int, row []reader.Cell)
+
+// ProcessFile applies sheet selection and per-sheet header skipping, invoking fn
+// for every remaining row — the port of reader.rs:54-105.
+//
+// The header check is per SHEET, not per file: first_row resets inside the sheet
+// loop (reader.rs:91), so a workbook whose second sheet repeats the header has
+// it skipped there too.
+func ProcessFile(path string, cfg *config.DatasetConfig, fn RowFn) error {
+	wb, err := reader.Open(path)
+	if err != nil {
+		return err
+	}
+	defer wb.Close()
+
+	sheets := wb.Sheets()
+	if len(sheets) == 0 {
+		return fmt.Errorf("no sheets in %s", path)
+	}
+	if cfg.Reader.SheetMode == config.SheetModeFirst {
+		sheets = sheets[:1]
+	}
+
+	for _, sh := range sheets {
+		firstRow := true
+		err := wb.EachRow(sh.Index, func(s reader.Sheet, _ int, row []reader.Cell) error {
+			if firstRow {
+				firstRow = false
+				if IsHeaderRow(row, cfg.Header.Tokens) {
+					return nil
+				}
+			}
+			fn(s.Index, row)
+			return nil
+		})
+		if err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+// Standard runs the fixed-column path for the 2017-family datasets — the port of
+// run_build_standard (main.rs:74-199).
+func Standard(cfg *config.DatasetConfig, inputDir, outputPath string) error {
+	files, err := InputFiles(inputDir)
+	if err != nil {
+		return err
+	}
+	label := DatasetLabel(inputDir)
+	fmt.Printf("[build] %s/ → %s  (%d files)\n", label, outputPath, len(files))
+
+	db, err := writer.OpenDB(outputPath)
+	if err != nil {
+		return err
+	}
+	defer db.Close()
+
+	isOld2 := strings.Contains(label, "old2")
+	stripBlank := cfg.Reader.StripBlankRows
+
+	// One transaction spans the whole dataset directory (main.rs:120,184).
+	tx, err := db.Begin()
+	if err != nil {
+		return fmt.Errorf("begin: %w", err)
+	}
+	ins, err := writer.Prepare(tx)
+	if err != nil {
+		tx.Rollback()
+		return err
+	}
+
+	var st writer.Stats
+	for _, file := range files {
+		base := filepath.Base(file)
+		var fileRows, fileSkipped, fileErrors uint64
+
+		procErr := ProcessFile(file, cfg, func(_ int, row []reader.Cell) {
+			allBlank := IsAllBlank(row)
+			// 2017-old2: blank rows drop out BEFORE the source-row counter
+			// (main.rs:134-137, ahead of the increment at :140).
+			if stripBlank && allBlank {
+				return
+			}
+			st.SourceRows++
+
+			hoTen, soBaoDanh := "", ""
+			if cols := cfg.Columns; cols != nil {
+				hoTen = cellAt(row, cols.HoTen)
+				soBaoDanh = cellAt(row, cols.SoBaoDanh)
+			}
+
+			switch transform.ValidateRow(hoTen, soBaoDanh, &cfg.Validation, stripBlank, allBlank) {
+			case transform.SkipBlankRow:
+				// Falls through to transform and insert, matching main.rs:150.
+				// Unreachable here: BlankRow requires stripBlank && allBlank,
+				// which returned above. Kept so the two call sites with opposite
+				// outcomes stay visibly distinct.
+			case transform.SkipNone:
+				// proceed
+			default:
+				fileSkipped++
+				return
+			}
+
+			parsed, err := transform.TransformRow(row, cfg)
+			if err != nil {
+				fileErrors++
+				return
+			}
+			if err := ins.Insert(parsed); err != nil {
+				fileErrors++
+				// Only the first five insert warnings print (main.rs:164).
+				if st.Errors+fileErrors <= 5 {
+					fmt.Fprintf(os.Stderr, "  [warn] %s: %v\n", base, err)
+				}
+				return
+			}
+			fileRows++
+		})
+		if procErr != nil {
+			// A file that cannot be read is logged and counted, never fatal
+			// (main.rs:171-177) — one corrupt file must not abandon the batch.
+			fmt.Fprintf(os.Stderr, "  [error] %s: %v\n", base, procErr)
+			fileErrors++
+		}
+
+		st.Skipped += fileSkipped
+		st.Errors += fileErrors
+		fmt.Printf("  %s: %d rows\n", base, fileRows)
+	}
+
+	if err := ins.Close(); err != nil {
+		tx.Rollback()
+		return err
+	}
+	if err := tx.Commit(); err != nil {
+		return fmt.Errorf("commit: %w", err)
+	}
+
+	// VACUUM only after COMMIT — SQLite refuses it inside a transaction.
+	return writer.Finish(db, outputPath, st, label, isOld2)
+}
+
+// cellAt returns the trimmed cell at idx, or "" when out of range — the
+// unwrap_or_default() behaviour of transform.rs:163-167.
+func cellAt(row []reader.Cell, idx int) string {
+	if idx < 0 || idx >= len(row) {
+		return ""
+	}
+	return strings.TrimSpace(row[idx].Str)
+}

parser/internal/ingest/ingest_test.go

@@ -0,0 +1,131 @@
+package ingest
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/tiennm99/thptqg/parser/internal/reader"
+)
+
+// Ports the 7 tests in parser/src/reader.rs:116-197. They were listed under
+// Phase 1 originally, which was wrong: they exercise header and blank-row
+// policy, which lives here rather than in the reader package.
+
+func cells(vals ...string) []reader.Cell {
+	out := make([]reader.Cell, len(vals))
+	for i, v := range vals {
+		out[i] = reader.Cell{Str: v, IsEmpty: v == ""}
+	}
+	return out
+}
+
+var stdTokens = []string{"HO_TEN", "HỌ TÊN", "STT"}
+
+// header_detects_ho_ten (reader.rs:128)
+func TestHeaderDetectsHoTen(t *testing.T) {
+	if !IsHeaderRow(cells("HO_TEN", "NGAY_SINH", "SBD"), stdTokens) {
+		t.Error("HO_TEN header not detected")
+	}
+}
+
+// header_detects_stt (reader.rs:139)
+func TestHeaderDetectsStt(t *testing.T) {
+	if !IsHeaderRow(cells("STT", "B", "C"), stdTokens) {
+		t.Error("STT header not detected")
+	}
+}
+
+// header_detects_ho_ten_unicode (reader.rs:150)
+func TestHeaderDetectsHoTenUnicode(t *testing.T) {
+	if !IsHeaderRow(cells("HỌ TÊN", "B", "C"), stdTokens) {
+		t.Error("HỌ TÊN header not detected")
+	}
+}
+
+// header_rejects_data_row (reader.rs:161)
+func TestHeaderRejectsDataRow(t *testing.T) {
+	if IsHeaderRow(cells("Nguyen Van A", "01/01/2000", "12345678"), stdTokens) {
+		t.Error("data row wrongly detected as header")
+	}
+}
+
+// header_rejects_short_row (reader.rs:172) — the <3 cell guard.
+func TestHeaderRejectsShortRow(t *testing.T) {
+	if IsHeaderRow(cells("HO_TEN", ""), stdTokens) {
+		t.Error("a 2-cell row must never be a header, even with a matching token")
+	}
+}
+
+// header_case_insensitive (reader.rs:179)
+func TestHeaderCaseInsensitive(t *testing.T) {
+	if !IsHeaderRow(cells("ho_ten", "B", "C"), stdTokens) {
+		t.Error("lowercase header not detected")
+	}
+}
+
+// blank_row_detection (reader.rs:190) — note the third cell is an empty *string*
+// cell, not Data::Empty, and must still count as blank.
+func TestBlankRowDetection(t *testing.T) {
+	blank := []reader.Cell{{IsEmpty: true}, {IsEmpty: true}, {Str: ""}}
+	if !IsAllBlank(blank) {
+		t.Error("row of empty cells should be blank")
+	}
+	if IsAllBlank(cells("Nguyen", "", "")) {
+		t.Error("row with content should not be blank")
+	}
+}
+
+// TestIsAllBlankIgnoresIsEmptyFlag pins the rule that blankness is decided by
+// the rendered string, never by Cell.IsEmpty — calamine emits empty-but-not-Empty
+// cells, and treating the flag as authoritative would diverge.
+func TestIsAllBlankIgnoresIsEmptyFlag(t *testing.T) {
+	// Content present but IsEmpty wrongly set: still not blank.
+	if IsAllBlank([]reader.Cell{{Str: "x", IsEmpty: true}}) {
+		t.Error("a cell with content must not be blank regardless of IsEmpty")
+	}
+	// Whitespace only: blank, matching Rust's trim().is_empty().
+	if !IsAllBlank([]reader.Cell{{Str: "   "}, {Str: "\t"}}) {
+		t.Error("whitespace-only cells should be blank")
+	}
+}
+
+// TestDatasetLabel covers the basename derivation that drives the stats wording.
+func TestDatasetLabel(t *testing.T) {
+	for in, want := range map[string]string{
+		"data/2017":       "2017",
+		"data/2017-old2":  "2017-old2",
+		"data/2017-old2/": "2017-old2",
+		"/abs/path/2016":  "2016",
+	} {
+		if got := DatasetLabel(in); got != want {
+			t.Errorf("DatasetLabel(%q) = %q, want %q", in, got, want)
+		}
+	}
+}
+
+// TestInputFilesSortedAndFiltered: the sort decides which duplicate SBD survives
+// INSERT OR REPLACE, so it is behaviour, not presentation.
+func TestInputFilesSortedAndFiltered(t *testing.T) {
+	dir := t.TempDir()
+	for _, name := range []string{"b.xlsx", "a.xls", "c.XLSX", "notes.txt", "d.csv"} {
+		if err := writeEmpty(filepath.Join(dir, name)); err != nil {
+			t.Fatal(err)
+		}
+	}
+	got, err := InputFiles(dir)
+	if err != nil {
+		t.Fatal(err)
+	}
+	want := []string{"a.xls", "b.xlsx", "c.XLSX"}
+	if len(got) != len(want) {
+		t.Fatalf("got %d files %v, want %d", len(got), got, len(want))
+	}
+	for i := range want {
+		if filepath.Base(got[i]) != want[i] {
+			t.Errorf("file %d = %q, want %q", i, filepath.Base(got[i]), want[i])
+		}
+	}
+}
+
+func writeEmpty(path string) error { return os.WriteFile(path, nil, 0o644) }

parser/internal/reader/fidelity_test.go

@@ -0,0 +1,128 @@
+package reader_test
+
+import (
+	"bufio"
+	"crypto/sha256"
+	"encoding/hex"
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"github.com/tiennm99/thptqg/parser/internal/reader"
+)
+
+// TestReaderFidelity asserts the Go reader reproduces calamine byte-for-byte on
+// every real input file.
+//
+// The oracle is a committed SHA-256 per file over a canonical cell dump. The
+// dumps themselves are real student names and birthdates, so only the hashes are
+// committed — regenerate the dumps from the Rust side on demand
+// (parser/examples/dump_cells.rs), which is possible because parser/ still
+// builds.
+//
+// The canonical form carries geometry and rendered cell values. The calamine
+// Data variant is excluded on purpose: Data::Empty and Data::String("") both
+// render "" and both count as blank in is_all_blank and transform, so the
+// distinction cannot reach the database.
+// Runs by default so CI and `go test ./...` keep the full guarantee; skipped
+// under -short, which is how to iterate without paying ~77s to re-read 418 MB.
+func TestReaderFidelity(t *testing.T) {
+	if testing.Short() {
+		t.Skip("-short: skipping the 299-file corpus sweep")
+	}
+	root := repoRoot(t)
+	manifest := filepath.Join(root, "parser", "testdata", "reader-fidelity-hashes.tsv")
+
+	f, err := os.Open(manifest)
+	if err != nil {
+		t.Fatalf("open manifest: %v", err)
+	}
+	defer f.Close()
+
+	var checked int
+	sc := bufio.NewScanner(f)
+	for sc.Scan() {
+		line := sc.Text()
+		if line == "" || strings.HasPrefix(line, "#") {
+			continue
+		}
+		rel, want, ok := strings.Cut(line, "\t")
+		if !ok {
+			t.Fatalf("malformed manifest line: %q", line)
+		}
+		path := filepath.Join(root, rel)
+		if _, err := os.Stat(path); err != nil {
+			t.Skipf("input data not present (%s); skipping fidelity suite", rel)
+		}
+
+		checked++
+		t.Run(rel, func(t *testing.T) {
+			t.Parallel()
+			got, err := canonicalHash(path)
+			if err != nil {
+				t.Fatalf("hash %s: %v", rel, err)
+			}
+			if got != want {
+				t.Errorf("cell dump diverges from calamine\n  want %s\n  got  %s", want, got)
+			}
+		})
+	}
+	if err := sc.Err(); err != nil {
+		t.Fatalf("read manifest: %v", err)
+	}
+	if checked == 0 {
+		t.Fatal("manifest contained no entries")
+	}
+}
+
+// canonicalHash renders one file in the canonical form and hashes it. Kept
+// byte-identical to the awk canonicalisation used to build the manifest.
+func canonicalHash(path string) (string, error) {
+	wb, err := reader.Open(path)
+	if err != nil {
+		return "", err
+	}
+	defer wb.Close()
+
+	h := sha256.New()
+	sheets := wb.Sheets()
+	fmt.Fprintf(h, "SHEETCOUNT\t%d\n", len(sheets))
+	for _, sh := range sheets {
+		fmt.Fprintf(h, "SHEET\t%d\t%s\t%d\t%d\n", sh.Index, escape(sh.Name), sh.Height, sh.Width)
+		err := wb.EachRow(sh.Index, func(s reader.Sheet, rowIdx int, row []reader.Cell) error {
+			fmt.Fprintf(h, "ROW\t%d\t%d\t%d\n", s.Index, rowIdx, len(row))
+			for c, cell := range row {
+				fmt.Fprintf(h, "CELL\t%d\t%d\t%d\t%s\n", s.Index, rowIdx, c, escape(cell.Str))
+			}
+			return nil
+		})
+		if err != nil {
+			return "", err
+		}
+	}
+	return hex.EncodeToString(h.Sum(nil)), nil
+}
+
+func escape(s string) string {
+	return strings.NewReplacer("\\", `\\`, "\t", `\t`, "\n", `\n`, "\r", `\r`).Replace(s)
+}
+
+// repoRoot walks up from the test's working directory to the directory holding
+// the data/ corpus.
+func repoRoot(t *testing.T) string {
+	t.Helper()
+	dir, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	for i := 0; i < 6; i++ {
+		if _, err := os.Stat(filepath.Join(dir, "data")); err == nil {
+			return dir
+		}
+		dir = filepath.Dir(dir)
+	}
+	t.Fatal("could not locate repo root (no data/ directory found)")
+	return ""
+}

parser/internal/reader/reader.go

@@ -0,0 +1,79 @@
+// Package reader wraps the two spreadsheet libraries behind one streaming
+// contract, mirroring parser/src/reader.rs (which wraps calamine's
+// open_workbook_auto for both formats).
+//
+// The contract is deliberately row-streaming rather than whole-workbook
+// materialising: data/2017/ha-noi.xls alone holds 72k rows across two sheets,
+// and the Rust original keeps at most one sheet range live at a time.
+package reader
+
+import (
+	"fmt"
+	"path/filepath"
+	"strings"
+)
+
+// Cell is one spreadsheet cell rendered the way calamine's Data::to_string()
+// renders it.
+//
+// IsEmpty tracks calamine's Data::Empty variant separately from a string cell
+// that happens to be empty. Rust distinguishes them at reader.rs:42, but both
+// render "" and both count as blank in is_all_blank and in transform, so the
+// flag is diagnostic only — never compare on it.
+type Cell struct {
+	Str     string
+	IsEmpty bool
+}
+
+// Sheet carries a sheet's identity and geometry. Height and Width describe the
+// used range, matching calamine's Range::height()/width(); every used range in
+// the corpus starts at (0,0), verified across all 299 files.
+type Sheet struct {
+	Index  int
+	Name   string
+	Height int
+	Width  int
+}
+
+// RowFunc receives each row of each sheet. Rows are padded to the sheet's used
+// width — width is load-bearing because every column read downstream is
+// positional with an unwrap_or_default() equivalent, so a short row silently
+// NULLs its tail columns.
+type RowFunc func(sheet Sheet, rowIdx int, row []Cell) error
+
+// Workbook is one opened spreadsheet.
+type Workbook interface {
+	// Sheets returns sheet identity and geometry in workbook order.
+	Sheets() []Sheet
+	// EachRow streams every row of the given sheet in order.
+	EachRow(sheetIdx int, fn RowFunc) error
+	Close() error
+}
+
+// Open dispatches on file extension, mirroring calamine's open_workbook_auto.
+func Open(path string) (Workbook, error) {
+	switch strings.ToLower(filepath.Ext(path)) {
+	case ".xls":
+		return openXLS(path)
+	case ".xlsx", ".xlsm":
+		return openXLSX(path)
+	default:
+		return nil, fmt.Errorf("unsupported extension: %s", path)
+	}
+}
+
+// padRow extends row to width with empty cells, and truncates if longer.
+func padRow(row []Cell, width int) []Cell {
+	if len(row) == width {
+		return row
+	}
+	if len(row) > width {
+		return row[:width]
+	}
+	out := make([]Cell, width)
+	copy(out, row)
+	for i := len(row); i < width; i++ {
+		out[i] = Cell{IsEmpty: true}
+	}
+	return out
+}

parser/internal/reader/xls.go

@@ -0,0 +1,119 @@
+package reader
+
+import (
+	"fmt"
+
+	"github.com/pbnjay/grate"
+	// Registers the BIFF backend with grate.Open.
+	_ "github.com/pbnjay/grate/xls"
+)
+
+// xlsWorkbook reads legacy BIFF through pbnjay/grate.
+//
+// grate replaced extrame/xls, which was measured against calamine ground truth
+// and found to corrupt 69% of cells and drop a further 28%: undecoded UTF-16LE
+// and BIFF record framing leaked into cell values, content moved between rows
+// and columns, and tail rows came back blank. That was charset-independent and
+// unfixable from the outside. grate reproduces calamine exactly on the same
+// files.
+//
+// The one normalisation grate needs is trailing blank rows: it yields rows past
+// the end of calamine's used range (one for a populated sheet, two for an empty
+// one), so trailing all-blank rows are trimmed. Note this is the opposite of
+// the xlsx path, where excelize already trims and calamine keeps a 1x1 empty
+// range — in both cases the rule is "match calamine's used range".
+type xlsWorkbook struct {
+	sheets []Sheet
+	rows   [][][]Cell
+}
+
+func openXLS(path string) (Workbook, error) {
+	wb, err := grate.Open(path)
+	if err != nil {
+		return nil, fmt.Errorf("grate open %s: %w", path, err)
+	}
+	defer wb.Close()
+
+	names, err := wb.List()
+	if err != nil {
+		return nil, fmt.Errorf("grate list %s: %w", path, err)
+	}
+
+	out := &xlsWorkbook{}
+	for idx, name := range names {
+		sh, err := wb.Get(name)
+		if err != nil {
+			return nil, fmt.Errorf("grate get %s/%s: %w", path, name, err)
+		}
+
+		var raw [][]string
+		for sh.Next() {
+			row := sh.Strings()
+			cp := make([]string, len(row))
+			for i, v := range row {
+				cp[i] = demergeMarker(v)
+			}
+			raw = append(raw, cp)
+		}
+
+		// Trim to calamine's used range.
+		height := len(raw)
+		for height > 0 && rowAllBlank(raw[height-1]) {
+			height--
+		}
+		raw = raw[:height]
+
+		width := 0
+		for _, r := range raw {
+			if len(r) > width {
+				width = len(r)
+			}
+		}
+
+		cells := make([][]Cell, len(raw))
+		for i, r := range raw {
+			row := make([]Cell, len(r))
+			for j, v := range r {
+				row[j] = Cell{Str: v, IsEmpty: v == ""}
+			}
+			cells[i] = padRow(row, width)
+		}
+
+		out.sheets = append(out.sheets, Sheet{Index: idx, Name: name, Height: height, Width: width})
+		out.rows = append(out.rows, cells)
+	}
+	return out, nil
+}
+
+// demergeMarker blanks grate's merged-cell continuation markers.
+//
+// grate fills the cells covered by a merge with sentinel runes; calamine
+// reports them as empty. In this corpus they occur only in the merged title
+// block of the 2016 spreadsheets (19 cells in rows 0-2 of one file). Only an
+// exact whole-value match is blanked, so a real cell that merely contains an
+// arrow is untouched.
+func demergeMarker(v string) string {
+	switch v {
+	case grate.ContinueColumnMerged, grate.EndColumnMerged,
+		grate.ContinueRowMerged, grate.EndRowMerged:
+		return ""
+	}
+	return v
+}
+
+func (w *xlsWorkbook) Sheets() []Sheet { return w.sheets }
+
+func (w *xlsWorkbook) EachRow(sheetIdx int, fn RowFunc) error {
+	if sheetIdx < 0 || sheetIdx >= len(w.sheets) {
+		return fmt.Errorf("sheet index %d out of range", sheetIdx)
+	}
+	sh := w.sheets[sheetIdx]
+	for i, row := range w.rows[sheetIdx] {
+		if err := fn(sh, i, row); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func (w *xlsWorkbook) Close() error { return nil }

parser/internal/reader/xlsx.go

@@ -0,0 +1,105 @@
+package reader
+
+import (
+	"fmt"
+
+	"github.com/xuri/excelize/v2"
+)
+
+// xlsxWorkbook reads OOXML through excelize.
+//
+// Two excelize behaviours must be corrected to match calamine:
+//
+//  1. GetRows applies the cell number format by default, while calamine renders
+//     the underlying value. RawCellValue: true disables that.
+//  2. GetRows trims trailing blank cells, so rows are ragged; calamine returns a
+//     rectangular used range. Rows are padded back out to the sheet width.
+type xlsxWorkbook struct {
+	f      *excelize.File
+	sheets []Sheet
+	rows   [][][]Cell // [sheetIdx][rowIdx][colIdx]
+}
+
+func openXLSX(path string) (Workbook, error) {
+	f, err := excelize.OpenFile(path)
+	if err != nil {
+		return nil, fmt.Errorf("excelize open %s: %w", path, err)
+	}
+
+	wb := &xlsxWorkbook{f: f}
+	crFixups := buildCRFixups(path)
+	for idx, name := range f.GetSheetList() {
+		raw, err := f.GetRows(name, excelize.Options{RawCellValue: true})
+		if err != nil {
+			f.Close()
+			return nil, fmt.Errorf("excelize GetRows %s/%s: %w", path, name, err)
+		}
+
+		// Do NOT trim trailing blank rows: calamine's used range keeps them, and
+		// excelize's GetRows already drops trailing fully-empty rows itself.
+		//
+		// One correction is needed. 63 sheets in 2017-old and 53 in 2017-old2
+		// hold a single empty shared-string cell at A1; calamine reports those
+		// as a 1x1 range, while GetRows returns nothing. A genuinely empty sheet
+		// (230 of them in 2016) is height 0 on both sides. GetCellType tells the
+		// two apart: the empty-shared-string cell exists in the XML and types as
+		// CellTypeSharedString, an absent cell types as CellTypeUnset.
+		if len(raw) == 0 {
+			if t, terr := f.GetCellType(name, "A1"); terr == nil && t != excelize.CellTypeUnset {
+				raw = [][]string{{""}}
+			}
+		}
+		height := len(raw)
+
+		width := 0
+		for _, r := range raw {
+			if len(r) > width {
+				width = len(r)
+			}
+		}
+
+		cells := make([][]Cell, len(raw))
+		for i, r := range raw {
+			row := make([]Cell, len(r))
+			for j, v := range r {
+				if fixed, ok := crFixups[v]; ok {
+					v = fixed
+				} else {
+					v = normalizeNumeric(f, name, j, i, v)
+				}
+				row[j] = Cell{Str: v, IsEmpty: v == ""}
+			}
+			cells[i] = padRow(row, width)
+		}
+
+		wb.sheets = append(wb.sheets, Sheet{Index: idx, Name: name, Height: height, Width: width})
+		wb.rows = append(wb.rows, cells)
+	}
+	return wb, nil
+}
+
+func rowAllBlank(r []string) bool {
+	for _, v := range r {
+		if v != "" {
+			return false
+		}
+	}
+	return true
+}
+
+func (w *xlsxWorkbook) Sheets() []Sheet { return w.sheets }
+
+func (w *xlsxWorkbook) EachRow(sheetIdx int, fn RowFunc) error {
+	if sheetIdx < 0 || sheetIdx >= len(w.sheets) {
+		return fmt.Errorf("sheet index %d out of range", sheetIdx)
+	}
+	sh := w.sheets[sheetIdx]
+	for i, row := range w.rows[sheetIdx] {
+		if err := fn(sh, i, row); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func (w *xlsxWorkbook) Close() error { return w.f.Close() }

parser/internal/reader/xlsx_fixups.go

@@ -0,0 +1,155 @@
+package reader
+
+import (
+	"archive/zip"
+	"bytes"
+	"encoding/xml"
+	"io"
+	"strconv"
+
+	"github.com/xuri/excelize/v2"
+)
+
+// normalizeNumeric reproduces calamine's rendering of a numeric cell.
+//
+// calamine parses a numeric cell to f64 and renders it with Rust's f64 Display,
+// so the stored literal "6.0" becomes "6". RawCellValue hands back the literal.
+//
+// The cell type must be consulted, not guessed: "01063476", "6.00" and "NAN" are
+// all shared strings that survive ParseFloat, and renumbering them would drop a
+// leading zero, drop a trailing zero, or recase NaN. GetCellType is only called
+// when re-rendering would actually change the text, which keeps it off the hot
+// path for the ~99% of cells that are already canonical or plainly non-numeric.
+func normalizeNumeric(f *excelize.File, sheet string, col, row int, v string) string {
+	if v == "" {
+		return v
+	}
+	fv, err := strconv.ParseFloat(v, 64)
+	if err != nil {
+		return v
+	}
+	out := strconv.FormatFloat(fv, 'f', -1, 64)
+	if out == v {
+		return v
+	}
+	axis, err := excelize.CoordinatesToCellName(col+1, row+1)
+	if err != nil {
+		return v
+	}
+	// OOXML omits the t attribute on numeric cells, and excelize has no map
+	// entry for an empty t, so a plain number reports CellTypeUnset rather than
+	// CellTypeNumber. Unset is only reachable here for a cell that exists and
+	// parsed as a float — an absent cell is "" and returned above — so both
+	// values mean "numeric". Shared strings report CellTypeSharedString and are
+	// left alone, which is what protects "01063476", "6.00" and "NAN".
+	if t, err := f.GetCellType(sheet, axis); err == nil &&
+		(t == excelize.CellTypeNumber || t == excelize.CellTypeUnset) {
+		return out
+	}
+	return v
+}
+
+// buildCRFixups maps a shared string's line-ending-normalised form back to its
+// raw form, for the strings that contain a carriage return.
+//
+// Go's encoding/xml performs the line-ending normalisation the XML 1.0 spec
+// mandates (CRLF and lone CR both become LF), so excelize returns "a\nb" where
+// calamine — which reads the raw bytes — returns "a\r\nb". That difference
+// reaches the database: in one 2016 file it affects 2,233 TEN_CUMTHI values,
+// which populate the ten_cum_thi column.
+//
+// The trick is that character references are exempt from that normalisation, so
+// rewriting literal CR bytes to 
 before decoding round-trips them intact.
+//
+// Returns nil when the file has no CR at all, which is the common case.
+func buildCRFixups(path string) map[string]string {
+	zr, err := zip.OpenReader(path)
+	if err != nil {
+		return nil
+	}
+	defer zr.Close()
+
+	var entry *zip.File
+	for _, zf := range zr.File {
+		if zf.Name == "xl/sharedStrings.xml" {
+			entry = zf
+			break
+		}
+	}
+	if entry == nil {
+		return nil
+	}
+	rc, err := entry.Open()
+	if err != nil {
+		return nil
+	}
+	defer rc.Close()
+	data, err := io.ReadAll(rc)
+	if err != nil || !bytes.ContainsRune(data, '\r') {
+		return nil
+	}
+	data = bytes.ReplaceAll(data, []byte{'\r'}, []byte("
"))
+
+	fixups := make(map[string]string)
+	dec := xml.NewDecoder(bytes.NewReader(data))
+	var cur bytes.Buffer
+	inSI, inT := false, false
+	for {
+		tok, err := dec.Token()
+		if err != nil {
+			break
+		}
+		switch t := tok.(type) {
+		case xml.StartElement:
+			switch t.Name.Local {
+			case "si":
+				inSI, cur = true, bytes.Buffer{}
+			case "t":
+				inT = true
+			}
+		case xml.CharData:
+			if inSI && inT {
+				cur.Write(t)
+			}
+		case xml.EndElement:
+			switch t.Name.Local {
+			case "t":
+				inT = false
+			case "si":
+				if inSI {
+					raw := cur.String()
+					if norm := normalizeEOL(raw); norm != raw {
+						fixups[norm] = raw
+					}
+				}
+				inSI = false
+			}
+		}
+	}
+	if len(fixups) == 0 {
+		return nil
+	}
+	return fixups
+}
+
+// normalizeEOL applies XML 1.0 line-ending normalisation: CRLF and lone CR
+// both collapse to LF. This is what encoding/xml does to the raw bytes, so it
+// reproduces the text excelize hands back.
+func normalizeEOL(s string) string {
+	if !bytes.ContainsRune([]byte(s), '\r') {
+		return s
+	}
+	var b bytes.Buffer
+	b.Grow(len(s))
+	for i := 0; i < len(s); i++ {
+		if s[i] == '\r' {
+			if i+1 < len(s) && s[i+1] == '\n' {
+				continue // CRLF: the LF is emitted on the next pass
+			}
+			b.WriteByte('\n') // lone CR
+			continue
+		}
+		b.WriteByte(s[i])
+	}
+	return b.String()
+}

parser/internal/schema/schema.go

@@ -0,0 +1,144 @@
+// Package schema is the single source of truth for the SQL shape of every
+// dataset — a direct port of parser/src/schema.rs.
+//
+// All four datasets (2016, 2017, 2017-old, 2017-old2) write into the same
+// 22-column student table. Columns a dataset has no data for bind NULL.
+//
+// Column provenance:
+//
+//	ten_cum_thi, gioi_tinh, tieng_duc, tieng_nhat  -> 2016 only
+//	khtn, khxh, gdcd, tieng_nga                    -> 2017 datasets only
+//	everything else                                -> both
+//
+// Before this consolidation the DDL, the INSERT and the subject regexes were
+// duplicated across four TOML configs, which is how the 2016 and 2017 schemas
+// drifted apart. The configs now carry only per-dataset parse rules.
+package schema
+
+import "regexp"
+
+// DDL is executed verbatim after the output database is (re)created.
+//
+// idx_ten_cum_thi is partial, so it holds zero entries on the three 2017
+// datasets — where the column is always NULL — while staying useful for the
+// 2016 cluster-grouping queries. Partial indexes are SQLite-specific.
+//
+// Byte-identical to parser/src/schema.rs:26-54; TestDDLMatchesRust enforces it.
+const DDL = `
+CREATE TABLE student (
+  so_bao_danh   TEXT PRIMARY KEY,
+  ho_ten        TEXT NOT NULL,
+  ho_ten_ascii  TEXT NOT NULL,
+  ngay_sinh     TEXT,
+  ten_cum_thi   TEXT,
+  gioi_tinh     TEXT,
+  toan          REAL,
+  ngu_van       REAL,
+  vat_ly        REAL,
+  hoa_hoc       REAL,
+  sinh_hoc      REAL,
+  khtn          REAL,
+  lich_su       REAL,
+  dia_ly        REAL,
+  gdcd          REAL,
+  khxh          REAL,
+  tieng_anh     REAL,
+  tieng_phap    REAL,
+  tieng_nga     REAL,
+  tieng_duc     REAL,
+  tieng_nhat    REAL,
+  tieng_trung   REAL
+);
+CREATE INDEX idx_ho_ten       ON student(ho_ten);
+CREATE INDEX idx_ho_ten_ascii ON student(ho_ten_ascii);
+CREATE INDEX idx_ten_cum_thi  ON student(ten_cum_thi) WHERE ten_cum_thi IS NOT NULL;
+`
+
+// IdentityFields are the identity columns, in INSERT parameter order.
+var IdentityFields = []string{
+	"so_bao_danh",
+	"ho_ten",
+	"ho_ten_ascii",
+	"ngay_sinh",
+	"ten_cum_thi",
+	"gioi_tinh",
+}
+
+// ScoreFields are the subject columns, in INSERT parameter order. Bound NULL
+// when a row has no score for that subject.
+var ScoreFields = []string{
+	"toan",
+	"ngu_van",
+	"vat_ly",
+	"hoa_hoc",
+	"sinh_hoc",
+	"khtn",
+	"lich_su",
+	"dia_ly",
+	"gdcd",
+	"khxh",
+	"tieng_anh",
+	"tieng_phap",
+	"tieng_nga",
+	"tieng_duc",
+	"tieng_nhat",
+	"tieng_trung",
+}
+
+// ParamCount is the total bound parameters per row.
+const ParamCount = 22
+
+// InsertSQL is a positional INSERT matching IdentityFields then ScoreFields.
+//
+// OR REPLACE is a behavioural contract, not an optimisation: a repeated SBD
+// overwrites the earlier row rather than aborting the transaction, so the last
+// file to supply a duplicate wins.
+const InsertSQL = `
+INSERT OR REPLACE INTO student
+  (so_bao_danh, ho_ten, ho_ten_ascii, ngay_sinh, ten_cum_thi, gioi_tinh,
+   toan, ngu_van, vat_ly, hoa_hoc, sinh_hoc, khtn,
+   lich_su, dia_ly, gdcd, khxh,
+   tieng_anh, tieng_phap, tieng_nga, tieng_duc, tieng_nhat, tieng_trung)
+VALUES
+  (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+`
+
+// scorePatternSources holds the regex per subject, applied to the DIEM_THI cell
+// text. Copied verbatim from parser/src/schema.rs:126-143 — the literals contain
+// Vietnamese text and must never be retyped.
+//
+// Every pattern runs against every dataset. A subject absent from a given exam
+// year simply never matches and stays NULL: 2016 files contain no "KHTN:" or
+// "Tiếng Nga:" tokens, and 2017 files contain no "Tiếng Đức:" or "Tiếng Nhật:".
+//
+// Go's regexp and Rust's regex crate are both RE2, and these patterns use no
+// backreferences, lookaround or Unicode classes, so they port with zero risk.
+var scorePatternSources = map[string]string{
+	"toan":        `Toán:\s*(\d+(?:\.\d+)?)`,
+	"ngu_van":     `Ngữ văn:\s*(\d+(?:\.\d+)?)`,
+	"vat_ly":      `Vật lí:\s*(\d+(?:\.\d+)?)`,
+	"hoa_hoc":     `Hóa học:\s*(\d+(?:\.\d+)?)`,
+	"sinh_hoc":    `Sinh học:\s*(\d+(?:\.\d+)?)`,
+	"khtn":        `KHTN:\s*(\d+(?:\.\d+)?)`,
+	"lich_su":     `Lịch sử:\s*(\d+(?:\.\d+)?)`,
+	"dia_ly":      `Địa lí:\s*(\d+(?:\.\d+)?)`,
+	"gdcd":        `GDCD:\s*(\d+(?:\.\d+)?)`,
+	"khxh":        `KHXH:\s*(\d+(?:\.\d+)?)`,
+	"tieng_anh":   `Tiếng Anh:\s*(\d+(?:\.\d+)?)`,
+	"tieng_phap":  `Tiếng Pháp:\s*(\d+(?:\.\d+)?)`,
+	"tieng_nga":   `Tiếng Nga:\s*(\d+(?:\.\d+)?)`,
+	"tieng_duc":   `Tiếng Đức:\s*(\d+(?:\.\d+)?)`,
+	"tieng_nhat":  `Tiếng Nhật:\s*(\d+(?:\.\d+)?)`,
+	"tieng_trung": `Tiếng Trung:\s*(\d+(?:\.\d+)?)`,
+}
+
+// ScorePatterns holds the compiled subject regexes, compiled once at init.
+// Rust compiles them once per run in CompiledPatterns::new; a package-level map
+// is the equivalent for a single-threaded CLI.
+var ScorePatterns = func() map[string]*regexp.Regexp {
+	out := make(map[string]*regexp.Regexp, len(scorePatternSources))
+	for field, src := range scorePatternSources {
+		out[field] = regexp.MustCompile(src)
+	}
+	return out
+}()

parser/internal/schema/schema_test.go

@@ -0,0 +1,150 @@
+package schema
+
+import (
+	"strings"
+	"testing"
+)
+
+// Ports the four tests in parser/src/schema.rs:149-213. Their purpose is to stop
+// the DDL, the INSERT column list and the field-order constants from drifting
+// apart — a drift that silently lands values in the wrong columns.
+
+// TestInsertMatchesFieldOrder ports insert_matches_field_order (schema.rs:156).
+func TestInsertMatchesFieldOrder(t *testing.T) {
+	if ParamCount != 22 {
+		t.Errorf("ParamCount = %d, want 22", ParamCount)
+	}
+	if got := strings.Count(InsertSQL, "?"); got != ParamCount {
+		t.Errorf("INSERT placeholders = %d, want %d", got, ParamCount)
+	}
+
+	open := strings.Index(InsertSQL, "(")
+	closeIdx := strings.Index(InsertSQL, ")")
+	if open < 0 || closeIdx < 0 {
+		t.Fatal("INSERT must contain a column list")
+	}
+	var listed []string
+	for _, c := range strings.Split(InsertSQL[open+1:closeIdx], ",") {
+		if c = strings.TrimSpace(c); c != "" {
+			listed = append(listed, c)
+		}
+	}
+
+	want := append(append([]string{}, IdentityFields...), ScoreFields...)
+	if len(listed) != len(want) {
+		t.Fatalf("INSERT lists %d columns, want %d", len(listed), len(want))
+	}
+	for i := range want {
+		if listed[i] != want[i] {
+			t.Errorf("column %d: INSERT has %q, field order has %q", i, listed[i], want[i])
+		}
+	}
+}
+
+// TestScorePatternsCoverScoreFields ports score_patterns_cover_score_fields
+// (schema.rs:183).
+func TestScorePatternsCoverScoreFields(t *testing.T) {
+	if len(ScorePatterns) != len(ScoreFields) {
+		t.Fatalf("%d patterns for %d score columns", len(ScorePatterns), len(ScoreFields))
+	}
+	inFields := make(map[string]bool, len(ScoreFields))
+	for _, f := range ScoreFields {
+		inFields[f] = true
+	}
+	for field := range ScorePatterns {
+		if !inFields[field] {
+			t.Errorf("pattern %q has no column", field)
+		}
+	}
+	for _, field := range ScoreFields {
+		if _, ok := ScorePatterns[field]; !ok {
+			t.Errorf("column %q has no pattern", field)
+		}
+	}
+}
+
+// TestDDLColumnsMatchInsert ports ddl_columns_match_insert (schema.rs:201).
+func TestDDLColumnsMatchInsert(t *testing.T) {
+	for _, field := range append(append([]string{}, IdentityFields...), ScoreFields...) {
+		if !strings.Contains(DDL, field) {
+			t.Errorf("DDL missing column %q", field)
+		}
+	}
+}
+
+// TestScorePatternsCompile ports score_patterns_compile (schema.rs:208).
+// Compilation happens in the package initialiser, so reaching this point already
+// proves it; the explicit checks guard against an empty or partial table.
+func TestScorePatternsCompile(t *testing.T) {
+	for field, re := range ScorePatterns {
+		if re == nil {
+			t.Errorf("pattern %q is nil", field)
+		}
+	}
+}
+
+// TestDDLMatchesRust asserts the DDL is byte-identical to parser/src/schema.rs.
+// Anything less and the two parsers can produce structurally different databases
+// while every row-level check still passes.
+func TestDDLMatchesRust(t *testing.T) {
+	const want = `
+CREATE TABLE student (
+  so_bao_danh   TEXT PRIMARY KEY,
+  ho_ten        TEXT NOT NULL,
+  ho_ten_ascii  TEXT NOT NULL,
+  ngay_sinh     TEXT,
+  ten_cum_thi   TEXT,
+  gioi_tinh     TEXT,
+  toan          REAL,
+  ngu_van       REAL,
+  vat_ly        REAL,
+  hoa_hoc       REAL,
+  sinh_hoc      REAL,
+  khtn          REAL,
+  lich_su       REAL,
+  dia_ly        REAL,
+  gdcd          REAL,
+  khxh          REAL,
+  tieng_anh     REAL,
+  tieng_phap    REAL,
+  tieng_nga     REAL,
+  tieng_duc     REAL,
+  tieng_nhat    REAL,
+  tieng_trung   REAL
+);
+CREATE INDEX idx_ho_ten       ON student(ho_ten);
+CREATE INDEX idx_ho_ten_ascii ON student(ho_ten_ascii);
+CREATE INDEX idx_ten_cum_thi  ON student(ten_cum_thi) WHERE ten_cum_thi IS NOT NULL;
+`
+	if DDL != want {
+		t.Errorf("DDL diverges from parser/src/schema.rs:26-54\n--- got ---\n%s\n--- want ---\n%s", DDL, want)
+	}
+}
+
+// TestScorePatternsMatchScores exercises each pattern against the shape the
+// DIEM_THI cell actually carries, including the wide runs of spaces seen in the
+// real corpus.
+func TestScorePatternsMatchScores(t *testing.T) {
+	const cell = "Toán:   8.50   Ngữ văn:   7.00   Tiếng Đức:   9   KHXH: 5.58   "
+	cases := map[string]string{
+		"toan":       "8.50",
+		"ngu_van":    "7.00",
+		"tieng_duc":  "9",
+		"khxh":       "5.58",
+		"tieng_nhat": "", // absent from the cell -> no match
+	}
+	for field, want := range cases {
+		re, ok := ScorePatterns[field]
+		if !ok {
+			t.Fatalf("no pattern for %q", field)
+		}
+		m := re.FindStringSubmatch(cell)
+		got := ""
+		if m != nil {
+			got = m[1]
+		}
+		if got != want {
+			t.Errorf("%s: matched %q, want %q", field, got, want)
+		}
+	}
+}

parser/internal/sqlitedb/sqlitedb.go

@@ -0,0 +1,25 @@
+// Package sqlitedb registers the SQLite driver the parser writes with and names
+// it in one place.
+//
+// modernc.org/sqlite is a pure-Go SQLite, chosen so the whole module stays
+// cgo-free — grate, excelize and yaml.v3 are pure Go too, so CI can compile with
+// CGO_ENABLED=0 and no C toolchain.
+//
+// It is a machine-transpiled SQLite rather than the upstream C amalgamation that
+// Rust's rusqlite --bundled vendors (libsqlite3-sys 0.30.1). Two things make that
+// acceptable: this parser uses only plain SQL — no CTEs, window functions,
+// triggers or extensions — and the differential gate compares a full-table
+// SHA-256 plus PRAGMA table_info/index_list against live Rust output.
+//
+// Verified on linux/arm64 with v1.56.0 (SQLite 3.53.3): the full DDL including
+// the partial idx_ten_cum_thi index, INSERT OR REPLACE, and VACUUM.
+//
+// Fallback if the differential gate ever implicates the driver: mattn/go-sqlite3
+// is upstream C at the cost of cgo.
+package sqlitedb
+
+// Registers "sqlite" with database/sql.
+import _ "modernc.org/sqlite"
+
+// DriverName is the database/sql driver name to pass to sql.Open.
+const DriverName = "sqlite"

parser/internal/transform/ascii_crosscheck_test.go

@@ -0,0 +1,65 @@
+package transform_test
+
+import (
+	"database/sql"
+	"os"
+	"testing"
+
+	_ "github.com/tiennm99/thptqg/parser/internal/sqlitedb"
+	"github.com/tiennm99/thptqg/parser/internal/transform"
+)
+
+// TestToAsciiAgainstRustOutput cross-checks ToAscii against Rust on real data.
+//
+// A Rust-built database is its own oracle: every row carries ho_ten alongside
+// the ho_ten_ascii that Rust derived from it, so the whole table is a
+// name -> expected-slug corpus far broader than the 20 hand-picked unit cases.
+//
+// Point it at a Rust-built database:
+//
+//	GO_PARSER_RUST_DB=/tmp/rust-2016.db go test ./internal/transform/
+//
+// Skips when unset, so the default suite stays hermetic.
+func TestToAsciiAgainstRustOutput(t *testing.T) {
+	path := os.Getenv("GO_PARSER_RUST_DB")
+	if path == "" {
+		t.Skip("GO_PARSER_RUST_DB not set; skipping cross-check against Rust output")
+	}
+	if _, err := os.Stat(path); err != nil {
+		t.Skipf("GO_PARSER_RUST_DB=%s not readable: %v", path, err)
+	}
+
+	db, err := sql.Open("sqlite", "file:"+path+"?mode=ro")
+	if err != nil {
+		t.Fatalf("open %s: %v", path, err)
+	}
+	defer db.Close()
+
+	rows, err := db.Query("SELECT ho_ten, ho_ten_ascii FROM student")
+	if err != nil {
+		t.Fatalf("query: %v", err)
+	}
+	defer rows.Close()
+
+	var checked, bad int
+	for rows.Next() {
+		var name, rustAscii string
+		if err := rows.Scan(&name, &rustAscii); err != nil {
+			t.Fatalf("scan: %v", err)
+		}
+		checked++
+		if got := transform.ToAscii(name); got != rustAscii {
+			bad++
+			if bad <= 5 {
+				t.Errorf("ToAscii(%q)\n  rust = %q\n  go   = %q", name, rustAscii, got)
+			}
+		}
+	}
+	if err := rows.Err(); err != nil {
+		t.Fatalf("iterate: %v", err)
+	}
+	if checked == 0 {
+		t.Fatal("database contained no rows")
+	}
+	t.Logf("compared %d real names, %d mismatches", checked, bad)
+}

parser/internal/transform/transform.go

@@ -0,0 +1,223 @@
+// Package transform performs row transformation: ASCII normalisation, score
+// regex parsing, and validation — a port of parser/src/transform.rs.
+//
+// ToAscii replicates build-lib.js toAscii exactly:
+//
+//	str.normalize("NFD").replace(/[̀-ͯ]/g,"").replace(/đ/gi,"d").toLowerCase()
+package transform
+
+import (
+	"errors"
+	"math"
+	"strconv"
+	"strings"
+
+	"golang.org/x/text/unicode/norm"
+
+	"github.com/tiennm99/thptqg/parser/internal/config"
+	"github.com/tiennm99/thptqg/parser/internal/reader"
+	"github.com/tiennm99/thptqg/parser/internal/schema"
+)
+
+// ToAscii normalises a Vietnamese name to an ASCII slug.
+//
+//  1. NFD decompose (splits base + combining diacritics)
+//  2. Drop combining marks in U+0300..U+036F
+//  3. Replace đ/Đ with d (NFD does not decompose them)
+//  4. Lowercase
+//
+// Step 2 filters a LITERAL CODEPOINT RANGE, not a Unicode category. The inline
+// comment at transform.rs:53 says "Unicode category M", but the code at :56 is
+// the specification and it checks '\u{0300}'..='\u{036f}'. unicode.Is(unicode.Mn, r)
+// is strictly broader and would strip marks Rust keeps, silently changing
+// ho_ten_ascii — the column the site's accent-insensitive search runs on.
+//
+// Step order matters and mirrors transform.rs:54-63: the đ/Đ replacement happens
+// before lowercasing.
+func ToAscii(s string) string {
+	if s == "" {
+		return ""
+	}
+	decomposed := norm.NFD.String(s)
+
+	var b strings.Builder
+	b.Grow(len(decomposed))
+	for _, r := range decomposed {
+		if r >= 0x0300 && r <= 0x036F {
+			continue // combining mark, in the range Rust drops
+		}
+		switch r {
+		case 'đ', 'Đ':
+			b.WriteByte('d')
+		default:
+			b.WriteRune(r)
+		}
+	}
+	return strings.ToLower(b.String())
+}
+
+// ParsedRow is one row ready for insertion.
+type ParsedRow struct {
+	SoBaoDanh  string
+	HoTen      string
+	HoTenAscii string
+	NgaySinh   *string
+	// TenCumThi is 2016 only: examination cluster name (TEN_CUMTHI column).
+	TenCumThi *string
+	// GioiTinh is 2016 only: gender, normalised to "Nam"/"Nữ" or nil.
+	GioiTinh *string
+	// Scores maps subject field -> value. Absent subjects are simply missing and
+	// bind NULL.
+	Scores map[string]float64
+}
+
+// SkipReason says why a row was skipped, or SkipNone when it passed.
+//
+// The distinction is load-bearing for the printed counters, and the two
+// non-blank reasons are counted as source rows while BlankRow is not — but note
+// that split lives in the CALLER, not here. parser/src/main.rs has two call
+// sites with opposite outcomes for BlankRow: at :135-137 it returns before the
+// counter at :140, while at :151 it matches Err(BlankRow) => {} and falls
+// through to transform and insert. The build loop must reproduce both.
+type SkipReason int
+
+const (
+	SkipNone SkipReason = iota
+	// SkipBlankRow: row is fully blank (2017-old2 only, checked before the
+	// source-row counter).
+	SkipBlankRow
+	// SkipEmptyField: so_bao_danh or ho_ten empty/missing.
+	SkipEmptyField
+	// SkipNonNumericSbd: so_bao_danh contains non-digit characters
+	// (2017-old / 2017-old2 guard).
+	SkipNonNumericSbd
+)
+
+func (s SkipReason) String() string {
+	switch s {
+	case SkipNone:
+		return "none"
+	case SkipBlankRow:
+		return "blank_row"
+	case SkipEmptyField:
+		return "empty_field"
+	case SkipNonNumericSbd:
+		return "non_numeric_sbd"
+	}
+	return "unknown"
+}
+
+// ValidateRow checks a row against the dataset's validation rules.
+//
+// Signature mirrors transform.rs:101-107, taking stripBlankRows and allBlank
+// explicitly; a shorter signature could not express both blank-row paths.
+func ValidateRow(hoTen, soBaoDanh string, cfg *config.ValidationCfg, stripBlankRows, allBlank bool) SkipReason {
+	// 2017-old2: skip fully blank rows BEFORE counting source rows.
+	if stripBlankRows && allBlank {
+		return SkipBlankRow
+	}
+	if cfg.RequireNonemptySbd && soBaoDanh == "" {
+		return SkipEmptyField
+	}
+	if cfg.RequireNonemptyName && hoTen == "" {
+		return SkipEmptyField
+	}
+	if cfg.RequireNumericSbd && !allASCIIDigits(soBaoDanh) {
+		return SkipNonNumericSbd
+	}
+	return SkipNone
+}
+
+// allASCIIDigits mirrors Rust's chars().all(|c| c.is_ascii_digit()).
+//
+// Deliberately not strconv.Atoi: Atoi accepts a leading sign, so "+123" would
+// pass a check Rust rejects. Empty input returns true, matching Rust's all() on
+// an empty iterator — the empty case is caught earlier by RequireNonemptySbd.
+func allASCIIDigits(s string) bool {
+	for i := 0; i < len(s); i++ {
+		if s[i] < '0' || s[i] > '9' {
+			return false
+		}
+	}
+	return true
+}
+
+// ParseScores extracts subject scores from a DIEM_THI cell.
+//
+// Every one of the 16 patterns runs against every dataset; a subject absent from
+// a given exam year never matches and stays NULL. Matching is unanchored
+// first-match, like Rust's Regex::captures.
+func ParseScores(diemThi string) map[string]float64 {
+	out := make(map[string]float64)
+	if diemThi == "" {
+		return out
+	}
+	for field, re := range schema.ScorePatterns {
+		m := re.FindStringSubmatch(diemThi)
+		if m == nil {
+			continue
+		}
+		v, err := strconv.ParseFloat(m[1], 64)
+		if err != nil {
+			continue
+		}
+		// Unreachable given the pattern shape, but kept for parity with
+		// transform.rs:136 (is_finite).
+		if math.IsInf(v, 0) || math.IsNaN(v) {
+			continue
+		}
+		out[field] = v
+	}
+	return out
+}
+
+// ErrNoColumns is returned when the fixed-column path is used on a config that
+// has no columns: mapping. Rust panics here via .expect() (transform.rs:162);
+// returning an error is the Go-idiomatic equivalent and is unreachable in
+// practice, since only the non-2016 path calls this.
+var ErrNoColumns = errors.New("transform: config has no columns mapping")
+
+// TransformRow extracts one row into a ParsedRow using fixed column indices,
+// the 2017-family path. 2016 uses runtime format detection instead.
+func TransformRow(raw []reader.Cell, cfg *config.DatasetConfig) (*ParsedRow, error) {
+	cols := cfg.Columns
+	if cols == nil {
+		return nil, ErrNoColumns
+	}
+
+	// Trimmed accessor, mirroring the closure at transform.rs:163-167. Out-of-range
+	// indices yield "" rather than an error, matching unwrap_or_default().
+	get := func(idx int) string {
+		if idx < 0 || idx >= len(raw) {
+			return ""
+		}
+		return strings.TrimSpace(raw[idx].Str)
+	}
+
+	hoTen := get(cols.HoTen)
+	ngaySinh := get(cols.NgaySinh)
+	soBaoDanh := get(cols.SoBaoDanh)
+
+	// diem_thi is read WITHOUT trimming (transform.rs:172-175), unlike the three
+	// fields above. Harmless because the score patterns are unanchored, but it is
+	// the shipped behaviour — do not "tidy" it.
+	diemThi := ""
+	if cols.DiemThi >= 0 && cols.DiemThi < len(raw) {
+		diemThi = raw[cols.DiemThi].Str
+	}
+
+	var ngaySinhOpt *string
+	if ngaySinh != "" {
+		ngaySinhOpt = &ngaySinh
+	}
+
+	return &ParsedRow{
+		SoBaoDanh:  soBaoDanh,
+		HoTen:      hoTen,
+		HoTenAscii: ToAscii(hoTen),
+		NgaySinh:   ngaySinhOpt,
+		TenCumThi:  nil,
+		GioiTinh:   nil,
+		Scores:     ParseScores(diemThi),
+	}, nil
+}

parser/internal/transform/transform_test.go

@@ -0,0 +1,283 @@
+package transform
+
+import (
+	"testing"
+
+	"github.com/tiennm99/thptqg/parser/internal/config"
+	"github.com/tiennm99/thptqg/parser/internal/reader"
+)
+
+// Ports every test in parser/src/transform.rs's test module (:201-409) — 29 in
+// total, not the 20 in the :213-315 range, which covers only ToAscii. The nine
+// outside that range are the ParseScores and ValidateRow cases, i.e. exactly the
+// behaviours this package's traps concern.
+
+// --- ToAscii: the 20 cases at transform.rs:213-315 ---
+
+func TestToAscii(t *testing.T) {
+	cases := []struct{ name, in, want string }{
+		{"plain_latin", "Nguyen Van A", "nguyen van a"},
+		{"nguyen_thi_hoa", "Nguyễn Thị Hoa", "nguyen thi hoa"},
+		{"tran_van_duc", "Trần Văn Đức", "tran van duc"},
+		{"le_thi_my_duyen", "Lê Thị Mỹ Duyên", "le thi my duyen"},
+		{"pham_thi_lan", "Phạm Thị Lan", "pham thi lan"},
+		{"bui_thi_thu", "Bùi Thị Thu", "bui thi thu"},
+		{"hoang_van_truong", "Hoàng Văn Trường", "hoang van truong"},
+		{"do_thi_ngan", "Đỗ Thị Ngân", "do thi ngan"},
+		{"nguyen_van_khanh", "Nguyễn Văn Khánh", "nguyen van khanh"},
+		{"trinh_thi_bich_ngoc", "Trịnh Thị Bích Ngọc", "trinh thi bich ngoc"},
+		{"vu_thi_dieu", "Vũ Thị Diệu", "vu thi dieu"},
+		{"nguyen_thi_tuong_vi", "Nguyễn Thị Tường Vi", "nguyen thi tuong vi"},
+		{"lowercase_d_stroke", "đặng thị hằng", "dang thi hang"},
+		{"uppercase_d_stroke", "ĐẶNG THỊ HẰNG", "dang thi hang"},
+		{"mixed_case", "NGUYỄN VĂN AN", "nguyen van an"},
+		{"tran_thi_kim_anh", "Trần Thị Kim Anh", "tran thi kim anh"},
+		{"nguyen_thi_phuong_thao", "Nguyễn Thị Phương Thảo", "nguyen thi phuong thao"},
+		{"le_van_long", "Lê Văn Long", "le van long"},
+		{"vo_thi_xuan_mai", "Võ Thị Xuân Mai", "vo thi xuan mai"},
+		{"empty_string", "", ""},
+	}
+	for _, c := range cases {
+		t.Run(c.name, func(t *testing.T) {
+			if got := ToAscii(c.in); got != c.want {
+				t.Errorf("ToAscii(%q) = %q, want %q", c.in, got, c.want)
+			}
+		})
+	}
+}
+
+// TestToAsciiUsesLiteralRangeNotUnicodeMn guards the highest-value trap in this
+// package. transform.rs:56 filters the literal range U+0300..U+036F; the inline
+// comment at :53 calls it "Unicode category M", but the code is the spec.
+// unicode.Mn is strictly broader, so using it would strip marks Rust keeps.
+// U+0654 (ARABIC HAMZA ABOVE) is in Mn but outside the range: Rust keeps it.
+func TestToAsciiUsesLiteralRangeNotUnicodeMn(t *testing.T) {
+	const in = "aٔb"
+	if got := ToAscii(in); got != in {
+		t.Errorf("ToAscii(%q) = %q — a combining mark outside U+0300..U+036F must survive; "+
+			"stripping it means unicode.Mn was used instead of the literal range", in, got)
+	}
+	// And a mark inside the range must be stripped.
+	if got := ToAscii("áb"); got != "ab" {
+		t.Errorf("ToAscii(\"a\\u0301b\") = %q, want \"ab\"", got)
+	}
+}
+
+// TestToAsciiDStrokeIndependentOfNFD proves the đ/Đ replacement is a separate
+// step: NFD does not decompose them, so relying on the mark filter alone loses
+// the letter entirely.
+func TestToAsciiDStrokeIndependentOfNFD(t *testing.T) {
+	for _, c := range []struct{ in, want string }{
+		{"đ", "d"}, {"Đ", "d"}, {"đĐ", "dd"},
+	} {
+		if got := ToAscii(c.in); got != c.want {
+			t.Errorf("ToAscii(%q) = %q, want %q", c.in, got, c.want)
+		}
+	}
+}
+
+// --- ParseScores: transform.rs:323, :332, :342 ---
+
+func TestParseScoresSingle(t *testing.T) {
+	s := ParseScores("Toán: 8.5")
+	if v, ok := s["toan"]; !ok || v != 8.5 {
+		t.Errorf("toan = %v (present=%v), want 8.5", v, ok)
+	}
+	if _, ok := s["ngu_van"]; ok {
+		t.Error("ngu_van should be absent")
+	}
+}
+
+func TestParseScoresMultiple(t *testing.T) {
+	s := ParseScores("Toán: 7.25  Ngữ văn: 6.0  Vật lí: 9")
+	for field, want := range map[string]float64{"toan": 7.25, "ngu_van": 6.0, "vat_ly": 9.0} {
+		if v, ok := s[field]; !ok || v != want {
+			t.Errorf("%s = %v (present=%v), want %v", field, v, ok, want)
+		}
+	}
+}
+
+func TestParseScoresEmptyCell(t *testing.T) {
+	if s := ParseScores(""); len(s) != 0 {
+		t.Errorf("ParseScores(\"\") = %v, want empty", s)
+	}
+}
+
+// TestParseScoresRealCellShape uses the wide space runs seen in the corpus.
+func TestParseScoresRealCellShape(t *testing.T) {
+	const cell = "Toán:   4.60   Ngữ văn:   5.50   Lịch sử:   4.50   "
+	s := ParseScores(cell)
+	if len(s) != 3 {
+		t.Fatalf("matched %d subjects, want 3: %v", len(s), s)
+	}
+	if s["toan"] != 4.60 || s["ngu_van"] != 5.50 || s["lich_su"] != 4.50 {
+		t.Errorf("got %v", s)
+	}
+}
+
+// --- ValidateRow: transform.rs:359, :365, :374, :383, :393, :400 ---
+
+func defaultValidation() *config.ValidationCfg {
+	return &config.ValidationCfg{
+		RequireNumericSbd:   false,
+		RequireNonemptyName: true,
+		RequireNonemptySbd:  true,
+	}
+}
+
+func TestValidateOK(t *testing.T) {
+	if r := ValidateRow("Nguyen Van A", "12345678", defaultValidation(), false, false); r != SkipNone {
+		t.Errorf("got %v, want SkipNone", r)
+	}
+}
+
+func TestValidateEmptySbd(t *testing.T) {
+	if r := ValidateRow("Nguyen Van A", "", defaultValidation(), false, false); r != SkipEmptyField {
+		t.Errorf("got %v, want SkipEmptyField", r)
+	}
+}
+
+func TestValidateEmptyName(t *testing.T) {
+	if r := ValidateRow("", "12345678", defaultValidation(), false, false); r != SkipEmptyField {
+		t.Errorf("got %v, want SkipEmptyField", r)
+	}
+}
+
+func TestValidateNonNumericSbdRejected(t *testing.T) {
+	v := defaultValidation()
+	v.RequireNumericSbd = true
+	if r := ValidateRow("Nguyen Van A", "12AB5678", v, false, false); r != SkipNonNumericSbd {
+		t.Errorf("got %v, want SkipNonNumericSbd", r)
+	}
+}
+
+func TestValidateNumericSbdAccepted(t *testing.T) {
+	v := defaultValidation()
+	v.RequireNumericSbd = true
+	if r := ValidateRow("Nguyen Van A", "12345678", v, false, false); r != SkipNone {
+		t.Errorf("got %v, want SkipNone", r)
+	}
+}
+
+func TestValidateBlankRowSkipped(t *testing.T) {
+	if r := ValidateRow("", "", defaultValidation(), true, true); r != SkipBlankRow {
+		t.Errorf("got %v, want SkipBlankRow", r)
+	}
+}
+
+// TestValidateNumericSbdIsDigitScanNotAtoi: Rust uses chars().all(is_ascii_digit),
+// which strconv.Atoi does not reproduce — Atoi accepts a leading sign, and would
+// wrongly admit "+123".
+func TestValidateNumericSbdIsDigitScanNotAtoi(t *testing.T) {
+	v := defaultValidation()
+	v.RequireNumericSbd = true
+	for _, sbd := range []string{"+123", "-123", "12 3", "1.0", "ABC123", "１２３"} {
+		if r := ValidateRow("Nguyen Van A", sbd, v, false, false); r != SkipNonNumericSbd {
+			t.Errorf("ValidateRow(sbd=%q) = %v, want SkipNonNumericSbd", sbd, r)
+		}
+	}
+}
+
+// TestValidateBlankRowOnlyWhenStripEnabled: with strip_blank_rows false, an
+// all-blank row falls through to the empty-field checks instead (transform.rs:109).
+func TestValidateBlankRowOnlyWhenStripEnabled(t *testing.T) {
+	if r := ValidateRow("", "", defaultValidation(), false, true); r != SkipEmptyField {
+		t.Errorf("got %v, want SkipEmptyField when strip_blank_rows is off", r)
+	}
+}
+
+// --- TransformRow ---
+
+func fixedColumnCfg() *config.DatasetConfig {
+	return &config.DatasetConfig{
+		Columns:    &config.ColumnMap{HoTen: 0, NgaySinh: 1, SoBaoDanh: 2, DiemThi: 3},
+		Validation: *defaultValidation(),
+	}
+}
+
+func cells(vals ...string) []reader.Cell {
+	out := make([]reader.Cell, len(vals))
+	for i, v := range vals {
+		out[i] = reader.Cell{Str: v, IsEmpty: v == ""}
+	}
+	return out
+}
+
+func TestTransformRow(t *testing.T) {
+	row := cells("Nguyễn Văn Đức", "04/04/1999", "51002167", "Toán: 8.5  Ngữ văn: 7")
+	got, err := TransformRow(row, fixedColumnCfg())
+	if err != nil {
+		t.Fatalf("TransformRow: %v", err)
+	}
+	if got.HoTen != "Nguyễn Văn Đức" || got.HoTenAscii != "nguyen van duc" {
+		t.Errorf("ho_ten=%q ascii=%q", got.HoTen, got.HoTenAscii)
+	}
+	if got.SoBaoDanh != "51002167" {
+		t.Errorf("so_bao_danh = %q", got.SoBaoDanh)
+	}
+	if got.NgaySinh == nil || *got.NgaySinh != "04/04/1999" {
+		t.Errorf("ngay_sinh = %v", got.NgaySinh)
+	}
+	// 2016-only columns are never populated on the fixed-column path.
+	if got.TenCumThi != nil || got.GioiTinh != nil {
+		t.Error("ten_cum_thi and gioi_tinh must stay nil on the 2017 path")
+	}
+	if got.Scores["toan"] != 8.5 || got.Scores["ngu_van"] != 7 {
+		t.Errorf("scores = %v", got.Scores)
+	}
+}
+
+// TestTransformRowEmptyNgaySinhBecomesNil ports transform.rs:179-183.
+func TestTransformRowEmptyNgaySinhBecomesNil(t *testing.T) {
+	got, err := TransformRow(cells("A", "", "1", ""), fixedColumnCfg())
+	if err != nil {
+		t.Fatalf("TransformRow: %v", err)
+	}
+	if got.NgaySinh != nil {
+		t.Errorf("empty ngay_sinh should be nil, got %q", *got.NgaySinh)
+	}
+}
+
+// TestTransformRowShortRowYieldsEmptyFields ports the unwrap_or_default()
+// behaviour at transform.rs:163-176: a row shorter than the configured indices
+// yields empty strings rather than an error.
+func TestTransformRowShortRowYieldsEmptyFields(t *testing.T) {
+	got, err := TransformRow(cells("OnlyName"), fixedColumnCfg())
+	if err != nil {
+		t.Fatalf("TransformRow: %v", err)
+	}
+	if got.HoTen != "OnlyName" || got.SoBaoDanh != "" || got.NgaySinh != nil {
+		t.Errorf("got ho_ten=%q sbd=%q ngay_sinh=%v", got.HoTen, got.SoBaoDanh, got.NgaySinh)
+	}
+}
+
+// TestTransformRowDiemThiIsNotTrimmed pins an asymmetry that is easy to
+// "tidy away": ho_ten, ngay_sinh and so_bao_danh are trimmed through the closure
+// at transform.rs:164-168, but diem_thi is read raw at :172-175.
+func TestTransformRowDiemThiIsNotTrimmed(t *testing.T) {
+	row := cells("  A  ", "  01/01/2000  ", "  123  ", "   Toán: 5   ")
+	got, err := TransformRow(row, fixedColumnCfg())
+	if err != nil {
+		t.Fatalf("TransformRow: %v", err)
+	}
+	if got.HoTen != "A" || got.SoBaoDanh != "123" {
+		t.Errorf("trimmed fields wrong: ho_ten=%q sbd=%q", got.HoTen, got.SoBaoDanh)
+	}
+	if got.NgaySinh == nil || *got.NgaySinh != "01/01/2000" {
+		t.Errorf("ngay_sinh = %v, want trimmed", got.NgaySinh)
+	}
+	// Untrimmed diem_thi still parses — the regexes are unanchored.
+	if got.Scores["toan"] != 5 {
+		t.Errorf("scores = %v", got.Scores)
+	}
+}
+
+// TestTransformRowRequiresColumns: the fixed-column path is only reachable when
+// the config has a columns: mapping. Rust panics via .expect() (transform.rs:162);
+// Go returns an error instead.
+func TestTransformRowRequiresColumns(t *testing.T) {
+	cfg := &config.DatasetConfig{Validation: *defaultValidation()}
+	if _, err := TransformRow(cells("A", "B", "C", "D"), cfg); err == nil {
+		t.Fatal("TransformRow without a columns: mapping must return an error")
+	}
+}

parser/internal/writer/writer.go

@@ -0,0 +1,177 @@
+// Package writer handles SQLite output: DDL setup, INSERT OR REPLACE, VACUUM
+// and the stats block — a port of parser/src/writer.rs.
+//
+// Every dataset writes the same canonical table (internal/schema), so there is
+// exactly one insert path. Columns a dataset carries no data for bind NULL.
+//
+// The stats lines are reproduced verbatim because they are the operator-facing
+// output of the build, and docs/deployment-guide.md points at the per-file row
+// counts for troubleshooting.
+package writer
+
+import (
+	"database/sql"
+	"fmt"
+	"os"
+	"path/filepath"
+
+	"github.com/tiennm99/thptqg/parser/internal/schema"
+	"github.com/tiennm99/thptqg/parser/internal/sqlitedb"
+	"github.com/tiennm99/thptqg/parser/internal/transform"
+)
+
+// OpenDB deletes any existing database at dbPath, recreates it, and executes the
+// canonical DDL.
+//
+// Deleting the file rather than issuing DROP TABLE mirrors build-lib.js:54 via
+// writer.rs:24-30. A consequence worth knowing: a concurrent reader sees the file
+// vanish mid-rebuild rather than a transactional swap.
+func OpenDB(dbPath string) (*sql.DB, error) {
+	if _, err := os.Stat(dbPath); err == nil {
+		if err := os.Remove(dbPath); err != nil {
+			return nil, fmt.Errorf("remove existing db %s: %w", dbPath, err)
+		}
+	}
+	if parent := filepath.Dir(dbPath); parent != "" && parent != "." {
+		if err := os.MkdirAll(parent, 0o755); err != nil {
+			return nil, fmt.Errorf("create %s: %w", parent, err)
+		}
+	}
+
+	db, err := sql.Open(sqlitedb.DriverName, dbPath)
+	if err != nil {
+		return nil, fmt.Errorf("open db %s: %w", dbPath, err)
+	}
+	if _, err := db.Exec(schema.DDL); err != nil {
+		db.Close()
+		return nil, fmt.Errorf("execute DDL: %w", err)
+	}
+	return db, nil
+}
+
+// Inserter wraps a prepared INSERT statement.
+//
+// Rust calls conn.execute(INSERT_SQL, ...) per row (writer.rs:73), re-preparing
+// each time. Preparing once is a performance choice, not a parity requirement —
+// the SQL and its bindings are identical either way.
+type Inserter struct{ stmt *sql.Stmt }
+
+// Prepare compiles the canonical INSERT against tx.
+func Prepare(tx *sql.Tx) (*Inserter, error) {
+	stmt, err := tx.Prepare(schema.InsertSQL)
+	if err != nil {
+		return nil, fmt.Errorf("prepare insert: %w", err)
+	}
+	return &Inserter{stmt: stmt}, nil
+}
+
+// Close releases the prepared statement.
+func (i *Inserter) Close() error { return i.stmt.Close() }
+
+// Insert binds one parsed row and executes the INSERT.
+//
+// Parameter order is IdentityFields then ScoreFields. Subjects absent from
+// row.Scores — and the two identity columns only the 2016 layouts populate —
+// bind NULL.
+func (i *Inserter) Insert(row *transform.ParsedRow) error {
+	args := make([]any, 0, schema.ParamCount)
+	args = append(args,
+		row.SoBaoDanh,
+		row.HoTen,
+		row.HoTenAscii,
+		nullableString(row.NgaySinh),
+		nullableString(row.TenCumThi),
+		nullableString(row.GioiTinh),
+	)
+	for _, field := range schema.ScoreFields {
+		if v, ok := row.Scores[field]; ok {
+			args = append(args, v)
+		} else {
+			args = append(args, nil)
+		}
+	}
+	if len(args) != schema.ParamCount {
+		return fmt.Errorf("built %d params, want %d", len(args), schema.ParamCount)
+	}
+	if _, err := i.stmt.Exec(args...); err != nil {
+		return err
+	}
+	return nil
+}
+
+func nullableString(s *string) any {
+	if s == nil {
+		return nil
+	}
+	return *s
+}
+
+// Stats carries the counters the build loop accumulates.
+type Stats struct {
+	SourceRows uint64
+	Skipped    uint64
+	Errors     uint64
+}
+
+// Finish runs VACUUM and prints the stats block.
+//
+// VACUUM must run AFTER the transaction commits — SQLite refuses it inside one.
+//
+// The wording branches on datasetLabel, which Rust derives from the input
+// directory's basename (main.rs:98-101). That makes the output depend on a
+// filesystem path rather than on config; it is reproduced here for parity, and
+// the caller passes the label explicitly so tests are not at the mercy of a
+// temp-directory name.
+func Finish(db *sql.DB, dbPath string, st Stats, datasetLabel string, isOld2 bool) error {
+	if _, err := db.Exec("VACUUM"); err != nil {
+		return fmt.Errorf("vacuum: %w", err)
+	}
+
+	var dbCount int64
+	if err := db.QueryRow("SELECT COUNT(*) FROM student").Scan(&dbCount); err != nil {
+		return fmt.Errorf("count rows: %w", err)
+	}
+
+	insertable := st.SourceRows - st.Skipped
+
+	fmt.Println()
+	if isOld2 {
+		fmt.Printf("Source non-blank data rows:      %d\n", st.SourceRows)
+		fmt.Printf("  skipped (empty/non-numeric SBD): %d\n", st.Skipped)
+	} else {
+		fmt.Printf("Source data rows (post-header):  %d\n", st.SourceRows)
+		if containsOld(datasetLabel) {
+			fmt.Printf("  skipped (empty/non-numeric SBD): %d\n", st.Skipped)
+		} else {
+			fmt.Printf("  skipped (empty/invalid):        %d\n", st.Skipped)
+		}
+	}
+	fmt.Printf("  insertable:                     %d\n", insertable)
+	fmt.Printf("  insert errors:                  %d\n", st.Errors)
+	fmt.Printf("DB rows (distinct SBD):           %d\n", dbCount)
+
+	if !containsOld(datasetLabel) && st.Errors == 0 {
+		gap := int64(insertable) - dbCount
+		if gap == 0 {
+			fmt.Println("Audit: OK — every source row made it in.")
+		} else {
+			fmt.Printf("Audit: %d row(s) collapsed (duplicate SBDs overwriting).\n", gap)
+		}
+	}
+
+	var size int64
+	if fi, err := os.Stat(dbPath); err == nil {
+		size = fi.Size()
+	}
+	fmt.Printf("Size: %.1f MB\n", float64(size)/1024.0/1024.0)
+	return nil
+}
+
+func containsOld(label string) bool {
+	for i := 0; i+3 <= len(label); i++ {
+		if label[i:i+3] == "old" {
+			return true
+		}
+	}
+	return false
+}

parser/testdata/reader-fidelity-hashes.tsv

@@ -0,0 +1,192 @@
+# Canonical cell-dump SHA-256 per input file, produced by the Rust/calamine
+# ground truth (parser/examples/dump_cells.rs). The Go reader must reproduce
+# each hash exactly. Hashes only - the dumps themselves are real student PII
+# and are never committed, per parser/tests/fixtures/README.md.
+# FROZEN. These hashes were produced by the Rust/calamine parser, which has
+# since been removed, so they cannot be regenerated. They remain a real
+# regression guard: any change to the Go reader that alters a single cell of any
+# of the 299 input files fails the suite. If the inputs themselves ever change,
+# the affected rows must be re-derived deliberately and reviewed, not refreshed
+# in bulk.
+data/2016/023718c7d3cf7ace3a7116fabb12bd9cdhyduoccantho-1468920829104.xlsx	fc3afd8da9ed28b0fc177192fab7b19a06535cb45d86841835d8e8828fc61c1c
+data/2016/08cdeb3636dcb2adaef829d62968274atravinh-1468902443859.xlsx	71f0e61918c63b95f08a16d502f5e9f2054e3c830bd83c725f1285bb1aabead1
+data/2016/0b583c9875443e65ffa449d5ca76fae8ninhbinh-1468899778557.xlsx	b0c2cfb9ab419442f151be5a014439f7f477fa7b31a9ae3f37b83073276bdb59
+data/2016/15c85d901a73dd49f2bae71aadcdbb5cninhthuan-1468920071763.xlsx	f27ceabc10484c0124c7d03f8343248df05719486af81ea2fb387d09a2d11b31
+data/2016/167116af6c8a096c2a9307edfa115f77dhcnthucpham-tphcm-1468910578187.xls	9962f437ca047cafb3add0c9eb07f96cf4168d026b7d5947ecd2dc915117ef11
+data/2016/17843bccde10f362c2b9600bc49cd3f8backan-1468933016076.xlsx	ffdabc85670ef7d5522bb33254d6053682b96deb248406e171bd066749c51cae
+data/2016/178814c33c8b85a77d0fc295fa6d2be1hagiang-1468932847230.xlsx	843d83e11b78862598eb77b1a5448f7b09f0a7f1842687b4f0174e69ae4fff8b
+data/2016/18055d17e46854ecc6c8be585bcb6e5cbaclieu-1468934144150.xlsx	ec257a18650045ec476ebbbdbfed48760b4c51b5aee73fe6456729d7cd8ea365
+data/2016/1c66db0265bf45df237b37a7bcaeaef3hanoi-1468932790746.xlsx	aeb84a0588ce445b852a9328f8a233444a5e5ace6ea0cdff695d96a85cf0015e
+data/2016/2004a8d225fd87524e324d2d519e87a4daclac-1468899960505.xlsx	455f5c3f17397139e50b71329e936ffc1c76a39dc244ee72f49289b3eeb5a47c
+data/2016/20db5eaace927596f1806f151649ee6edhtantrao-1468901296040.xlsx	b25aa4ab69339c095c20d291614fdcd6569f210b33cdebae955e54945e647010
+data/2016/216ee4e385dbc236767be899b068bf98dhtaynguyen-1468940490240.xlsx	88346d3fa7087055f4c602bda8456581d721c9f9f2d4e0a18dbc511c9c8288fe
+data/2016/2208903e1fa12220d100f93e40d5ff3bquangtri-1468933440097.xlsx	49241f1af4fd0eb84e6a2943f3c07d1c0f7bd71c9a8ea407b5adcff194bcbcbd
+data/2016/22e8617886412abd90ae3d33cc6d7db4dhyduocthaibinh-1468941020145.xlsx	9ec94a6c47932bccbf001c431c0d4c480a861cb9aa3f845d9b6e86d4f2688889
+data/2016/238adc19e80c0daef089a307af23edcethanhhoa-1468933304999.xlsx	121b6ed30c60a2979db512f55fd2f8e65194c7323b5bf495d8175a256eb8eb2b
+data/2016/2b33640b0dd154490f2809d96be81797dhdalat-1468902130616.xlsx	201b953d4a19c0a0441693d43be8f37212b1309084428dc6d1367fb2904a92f6
+data/2016/2fa6d9a308bd11bc09f02d787eef2c15lamdong-1468933821082.xlsx	323b5f6551d35d57a995ef04c5d0d569433775fe6e2d685afc88d43442e3d215
+data/2016/357c6aaf59aeaa6c11c3e83c595d38cfdhbachkhoa-tphcm-1468939678829.xlsx	873fa6455a85783e4fe54b420c6cbda91803e0fa36356af859236092019f0083
+data/2016/388fcdfdac18ef6eee6125a2a015714ddhdongthap-1468934545359.xlsx	6db5b9fa0acd509c1b07bd6a03ae07df19487073a3d18495df3e185e85711189
+data/2016/3c0e56abd8c5334f4e1ff7df3ac23a15dienbien-1468920113820.xlsx	cfbadef763402bb8ac1e08b398cb38bebc184a2de7e760c44c2e31ce54349a87
+data/2016/3daa6ee33fdafadc3c112c18bff18c2ddhthainguyen-1468940268065.xlsx	a7d7e20b00b912fdd4658fa73c1fd13efc4a748e01d9bd41ce25d91222e6b29b
+data/2016/412e62265f35ca0078a0d94af5d1ed8abinhdinh-1468899884192.xlsx	06d144fca25ec641e5d8bec34df77b69fcea1946120d050d7679bf282a77b1bb
+data/2016/43213f61074068d9177685eb6c3f2d3bdhbachkhoa-dhdanang-1468938494437.xlsx	381a59addee12fd6efc7951282578d177101fce5a07be8621b1143f1c99a0e29
+data/2016/4334ea476a2b8196d54ba40a341831echvkythuatquansu-1468920342994.xlsx	3705579c6f432e2a8aa053e7f2564f79aad49fde836ad3934d1f9f39375150a4
+data/2016/487c7287c1c9735722b31d877942becddhcongnghieptphcm-1468983373303.xlsx	2c50ae0fc1992ad8b61b7d52ad02f32159bd649f0426ae00d17583b53427b05c
+data/2016/4a3fa95a83fe9b3b2fafab309323fd22dhthuyloi-1468901840919.xlsx	e75239bd4e311ad36e57d42ffb4c2649ce8e599f1ae0bad505957edd69b09a80
+data/2016/4ad416ccf014bd5d01f415238566b210khanhhoa-1468933789968.xlsx	7a4451cba4a76acc50362cb94aefee1ff9cb04e6e2461daddc383ff1601c80da
+data/2016/4b98fdd7300b86a94636e4d15394fc19sonla-1468899319124.xlsx	961407526e9d79d0e2dfcf370eaed7ff7f64d14835f88481ac17361acbea996c
+data/2016/4c2fd846ed2a0670ba6d1200d1d68e1dspkythuatvinhlong-1468902640438.xlsx	88b634010f5888ed6e954807c31712125a280f432219dc652d790d01ef7c10d1
+data/2016/4e0ccee19eb64513d23a6dff96ff7f9cvinhphuc-1468919874814.xlsx	52199086d32401b35629d6c4930b024161b0d6d979f4aebc1bef26a391d4b235
+data/2016/4e9bb192d56c9fa92b3e0ec13d8236c8dhxaydungmientrung-1468940978346.xlsx	dd3d9e4ce4e0c385e753a56f16ce03e7de535a3700e2c62dfe3dce201c73f6f9
+data/2016/505a876dfd1ea321291e29192aebe95ehoabinh-1468983220125.xlsx	b17f8a592f67c1ed8408b145a2241b4acb6088825463708b5f38749fedfd21be
+data/2016/514877c1cfebdc9c2251f651b5f02c7edhspkythuattphcm-1468906568655.xlsx	86cbefa0051faad10505f5d2f5dc4c4690c2c9ccd475fa06713754da9edcc627
+data/2016/51bb2f9b6e9f565c2d705606fd56958bdhgiaothongvantai-1468939107463.xlsx	b2199701dc1126ee509581dd6fd739d9a913ea6a232a55d59a14fced8ab82cc3
+data/2016/538c9676b70f2509573245d58fba93bcdongnai-1468938241935.xlsx	8ff1d5c71cf63b688ecfac852244b24cb6a4321eccf28fe056139f1527f2fb33
+data/2016/53e5e20ebd74d84741513b703fbd13bfdhnonglam-dhthainguyen-1468939023637.xlsx	b3ac4319785a8f33065f4684b947598733050a916ae9cabaa5684023d42f9163
+data/2016/5465cca59495dcd9a48f689e2d38b85ephutho-1468902409682.xlsx	44873acca3ede545505c531b921605fc7b8372c32bff3886572ca072966c898e
+data/2016/54bbcf865c137d58cf62af15733bd376dhsupham-dhdanang-1468938526558.xlsx	7e846a06d99ff5b71dbeb3db833d748c7675a17901e3d41bac84a9023e16601d
+data/2016/575df8a4df74e87abb8f55d807324374dongthap-1468933902699.xlsx	db44859e1fce523fb86907b4928c84f92036e696bdf14a337b1f25f69664389f
+data/2016/5aede4951e7af8c1e1056c747a553386hvnongnghiepvn-1468900610577.xlsx	76e75a3d59c5e850f62191dfa3ce65b6d1347b4eaa4f4870c687e94aea5087fd
+data/2016/5b16d7156a03bb7ed0618796bae87bc3cantho-1468933954714.xlsx	6d9b5f906fd7b5a66390e42d51f06a3445724e92f4a67cb116ab74359d8693e3
+data/2016/5ca241aa1de8d25a3e6ff6db75234025dhcantho-baclieu-1468907071460.xls	290c7e658c3b218befd5b111ad89d8c03f5e5dc1a1815e730da8bc1308660b6c
+data/2016/5fb3a1b66aff077ee3da482be08cd391thainguyen-1468902797020.xlsx	6df75f210b0dc269ceb4ddbbe323640cc32d75157096942ff4eb7565157d4d7a
+data/2016/5fd13ae5287cbf0149bd733b507e0608dhkhxhnv-dhqgtphcm-1468920630193.xlsx	311b90c29f089ffc7bfdc7e5b9c7fdcefaa803bb6d785189cebb6b170b61ee1e
+data/2016/6375fd13557bee08d20f1f3e46dac130dhnhatrang-coso1-1468901350425.xlsx	05dc9a62f844c3a73f0a781066ba9fc3f3bf5faa970cb1b3f0f0f0487769488e
+data/2016/64d2635efd30a0d0db4f7c30b9735261dhvinh-1468940230027.xlsx	0e2fc0f487f3963cd063bcb3fac067d2af1ef5a73e5d3def3523e1ecc30851e2
+data/2016/64e44466ffeff9a587260795644805d0dhkhtunhien-dhqgtphcm-1468921015146.xlsx	4f67d25acd11182b60d8c938ee49102fb30e8c96c85e6ccecd260fca0f5d53aa
+data/2016/6960b6fa493e1365a183af461e945389thaibinh-1468920019273.xlsx	6f10581918d96d82a99eab4c2729672836e091265c4518996f4aad5687038f3a
+data/2016/698b930ab7697a0672bbc39168024c9cgialai-1468933753098.xlsx	3be46dfad3d9d2c81557e1bc4103622bb7d9e9ba4fa18b25d2a4b355bf617264
+data/2016/6badc7cfa258b59ad67e2b64f713b95fthuathienhue-1468983258438.xlsx	1dccf8266a62a50911046837d01cd07f6085d43dc1c3cc953a35bf033c4991b3
+data/2016/70bf8cd2b6e87523219e55149a8524c5dhquynhon-1468938878328.xlsx	874dfbb664c72a3b075be56adc386dba2ac9aea4be2bb7a11d250b67d67b964d
+data/2016/74145979ee30186bf4e40ee3e6a3ac74bentre-1468934010040.xlsx	6db50bd7918d17ab04bd4a8a30859719bb1f3e7c5aa6e3bbe8d4f49d4c2bcf49
+data/2016/7418e83ff9d07b146216a3beafa927bbdhtravinh-1468902716174.xlsx	eadf00e204586e628a12390d335627c65cec48cb0e6ed53f5cc383bac73de402
+data/2016/7435bb054e8e4ec34c173af1b6ad8402kiengiang-1468900147395.xlsx	271fa0d823b0d07f32c0fb9b885513e465eeb4880e8fbcc72eedd20759c7d11d
+data/2016/74e6cd0dc78a3fb7dcf56ae57283b9b9dhtaybac-1468940302052.xlsx	9b0a007cb2f55d5c2a629d1c9bfa7f16547e3811f3d5c256bd1377b2013e7856
+data/2016/76f19098881e9c79f66ac4f1b42a8a55dhsuphamhanoi-1468901144203.xlsx	1a955af97d97f57d9dc20b5c406ca146753760e03050c02c74bef4d62a39ec21
+data/2016/778ab074ed6a95d209fa901fd5d68843dhhungvuong-1468901964127.xlsx	0add20e99589c83ec61e21582a965692148d899b4df0182c4a4d6d2ce7696109
+data/2016/7afd5d7fdff7c170a6342cd8f9474b22langson-1468932984587.xlsx	8f4014d1d2bf4e02b3136d6790297af8352f8148dfecace502c4cbaa59b6ca7d
+data/2016/7b5d74564ab9f6e0e137f6d161fab6f0dhxaydung-1468940561682.xlsx	b95d40c1b9497224dd8d59f6e7aeea31a618deba14e43e57539cf32ae748e61d
+data/2016/7cfc91bd6a1f5ae9d7917e920f469b90haiduong-1468899640007.xlsx	5abbd89e8130078fe42ecee222327add52e4dbd362c2f96576d9e6ed7c441c19
+data/2016/7e0d0b6fb981d407cffdb79d724e8aecbacninh-1468919984998.xlsx	3f32fd1af47a2b1f92175bcc55d95c48be1de1d9544e06f440ac36a8491b1a14
+data/2016/7f12b90225e91f9d00aac3bcdf9974e1nghean-1468920043121.xlsx	a4c5a99a9faa75e36b809d1bfa4b7caab706bde26649256ed1248f429e59abe6
+data/2016/7f1cbe7f35d1f539ca817a0ab8bd00c2dhcantho-haugiang-1468907167921.xls	434417e4765c5b225570c4a19b655c5ac957707209af398f7277a0a9ee90f2b6
+data/2016/7f4362ef567cdf74495368e0fc11072avinhlong-1468934067981.xlsx	de063663b5f31f525a8a07958d436c8e10805d9ec7b27f9bff2b4a2c7ae559f0
+data/2016/82338a4a93ea1e0cde26e0061066ca8adhcongnghiephanoi-1468920154757.xlsx	fed3686286093866bfe03b67d7f72ab7c3aaa1a5ce92d503d1ca24582addee31
+data/2016/86a692bebf9b0b7c58adb9876b9befd5dhnganhangtphcm-1468902244566.xlsx	5a2b2e5e63bb8c12f0e16b802b51fac470650fc050acd9e846275e82a5d33fbc
+data/2016/87ffaf3197ba20f0919f98a0b3f33914dhkhoahoc-dhhue-1468938729472.xlsx	1b46f645bb5c491ba9e9e490f36f1692b055e2d29878dfff1fa4db94fabad72f
+data/2016/8916da7b3c4d58cffc1dd29dc7a8aabfdhkinhte-dhhue-1468938560205.xlsx	332aa93e333430021755667c10795749db3ec3f7c7d3afc4d70a48fbd00bc820
+data/2016/8af8f1b29b6a40e26ccb4a2e19e8eededhkythuatcongnghiep-dhthainguyen-1468901919568.xlsx	94347288ba444f19a160868649528dc29633f4eb660aad74ea9799ee79935432
+data/2016/8f948c2a5ab49a3c5fd03b6ea04887a7dhkinhtequocdanfix-1468902083782.xlsx	f8e4c450509b176716429055e2ed72dd340af7ad5a5704db2d0c60818deae548
+data/2016/919276a0348634459f8937dfcd6c7129dhhue-1468938792115.xlsx	cb91ae55927a40d0eef9ef1bc2bd23a758528de6b36496505dba75e1cccdb849
+data/2016/92045214159cb0335b2836aa1c4bb25ahungyen-1468899732002.xlsx	74536aad02570a17c83933c46d10ce9d2300a5e61a37fa2a67e5d99cffb17632
+data/2016/964c4368131f6be6058fabb1caefba1chaugiang-1468934247324.xlsx	1df076df14fcb51daffa600dca05064be61269fe19f33dc14d03c8f43d8ee1ca
+data/2016/96c95c85c5549a06f2e92ffcacf8ff21hvtaichinh-1468934470924.xlsx	9a7a3841700b27e4265eef8def9addf17406d08c4f007462cda9335f5369c4ff
+data/2016/97aa741a4c2b1276559eb021779daa32yenbai-1468933116055.xlsx	aa79b6e84e95f58704b715b701449507dbd079f9bee9900f20e6c90baf299952
+data/2016/9ba098a7913ad837a66e173b3fc41fd7dhngoaingu-dhdanang-1468902750424.xlsx	8864a16b969ea34dc1f7b8762b8039e17df1599bfea7ca5914dc518983fd36fe
+data/2016/9c730c43bf4309171528c3853ed53b19quangbinh-1468933376032.xlsx	b637ecd24a240e4a73f291cc4c9faef42379c04d40d1952cf84e54e51eb8cdf1
+data/2016/9d78f6dce7965fb4b7a337279920d4b1dhtonducthang-1468902168623.xlsx	1c4f32eb3ed9aa831d2abe41998f5c157ae5ca3d9735b87bb101913870f4c4cc
+data/2016/9e0e564ae801e347bd7649da8b89ad90dhluathn-1468920377978.xlsx	1a1b4b0bed838f2c6043d789dae4ab7f96a904b886cd5990a98f7be3b960caf5
+data/2016/a11e707e6ed5fe4d29a8e32380d7558cdhsphanoi-2-1468902006284.xlsx	9b8060368d09cf89c1e88001a642d1aec413e4b3f8646c8360a0459d3704dfd3
+data/2016/a31b3446bae5a17c517348d8c38c5b7equangninh-1468919920197.xlsx	8b284c18aab150349d49fe596cb96a0a838e8c5618eb1f2d7dd5f0053430c71d
+data/2016/a3aa106104f114c6426deb2d5e5da5fdlaichau-1468932954079.xlsx	0697ae494d6a17e188138e2c9f3d412a1bbd8a462e2eb82060ffff25e9982506
+data/2016/a8ff80d9478747e36d21ec95cc9ecb8fsoctrang-1468902686409.xlsx	3fb1d35e7647d0581f08d2f56b4d7ccd5fab1ab125bdcb3b83c64c65d540a53b
+data/2016/a995de3f7a7055cf10b2f002f0e194f7dhsptphcm-1468920792291.xlsx	9f25b647f9d4d714772ffcb1cba40fa1b338500bd08e39352704de6a9810a8df
+data/2016/ab2f259ef55cb08b3a436d3f15d42cc7dhsupham-dhhue-1468934431828.xlsx	96f40bdf397bdf98e596701bfed957d18483ec0affb7e945f6e8b69453b7015c
+data/2016/af6828f837a80e03d77a65f3d310ec60dh-tai-chinh-marketing-1468924711596.xlsx	297b71a7760cfff612a290f31082f22647c0d7a8528dbe052356885fc620dc5e
+data/2016/b2552e20c9ff4b237244dfbf2e278a47dhthuongmai-1468902876264.xlsx	8c7f9d58fb5ffdfafb1431e9fd51347d338ccba14f547aa26917a2692a515982
+data/2016/b2a739584a8861502573d1c6372fd6e9dhnonglamtphcm-1468939555326.xlsx	d8c8b6e10d3a44f8a429d2a718af55c853bd508dcb79d13522f1e40d9d6e96e7
+data/2016/b34c777942ca8a4de91f23a35cce1c6bdhbachkhoahn-1468901491486.xlsx	b5135f12ac53b3f513b46078a89b6a2663089e6bb3ad97d48a13312b5ac69ca6
+data/2016/b35a3f26a162bec3740e2da5639327afdhcantho-1468907131146.xls	d114bf1c070bb165bd45a64ec79495c764805ee989c740ed95fad06eabd69ec4
+data/2016/b42fd58ead4c54279d7df5f53160a851dhdanang-1468920187203.xlsx	9ce9d1b6f494866f6a39e68babe27b3a9562cd1515477f5f32a8e2a8a57bf13e
+data/2016/b7a0c1ecce7c8445b3a1021091a5e654bacgiang-1468983153932.xlsx	a53266ee75a8d49fe5a017125e198a1368a9bc9259dabe8cdf1cad11551dd439
+data/2016/bacd1b2133b37d0ea9fbf46445265036dhtiengiang-1468940391597.xlsx	bc90ae7bf5611ae0bb3c321e5185afcb530e1331c5a920a40b85b30b47eb1006
+data/2016/bd857add0cfe92c1c247469ff2104527dh-hong-duc-1468900427119.xlsx	96b62a2954d2c1541699d491ea931c240a9b06660d5c27a3cd9055433306aaef
+data/2016/bf259a908ff8a8dcc067a761204459f4daknong-1468934173803.xlsx	2f8fb2dbcd9a06acfd91becba0f728bf56e64f5c9984b40eac060834ea619b44
+data/2016/c174631d90303276ac63ff38b31157bbdhmodiachat-1468900697349.xlsx	c5c4435cfe1e6feede1936fd11e5cd3ff5758b4c4d522df03facf7522a9e7194
+data/2016/c2cfaaf287fb1e7120894e545771b027dhkhoahoc-dhthainguyen-1468900365545.xlsx	9a35e56e631ba5d1056fb149ea01a41b4e62ff075937fa735666d192b1714da2
+data/2016/c5c28959fcd0b4e39dd9fd538db5448adhgtvttphcm-1468939150821.xlsx	17a032a01a9ebe9f577af8fa4666e8700e9ab4e97166703cf6a7a0ff9dea4e9b
+data/2016/c5ecc58a731fd1a54b7e6cc3bc99449ddhlamnghiep-1468939300790.xlsx	8f5f7bf26bafb3a8dd6988ab9858dbf03a14d200a23ed5299a8a5411c84d7f1b
+data/2016/c6ef16bc9f75ce0fd16c1229afad4e71dhhanghai-1468906457479.xlsx	8e43d5e1782e366344c8d615666541931319475e02df0be29ae290c7ab2abda1
+data/2016/cc57dc986acb525086b86e85df54949dkontum-1468933632113.xlsx	2397d3b0fca3888ff34d4166a9d3da70dcd6986e6405bd556471d46fff0ab551
+data/2016/cdc912664d7217c75eb9b67c6ef3797adhkiengiang-1468901222446.xlsx	460eb7e1ce79dde123fe53a18034c5617f0616ad1abce6dedfcd0e4a2957a28d
+data/2016/d035f470c13adbf5785797c6ccdcb204laocai-1468919805337.xlsx	d23348d36bf6fa886d50e30507d4eada114a58f52cb6e615c119980d7753b5ad
+data/2016/d5ecd2574b5ab36c62dffad435262cb8tuyenquang-1468902296807.xlsx	3acb7e4ba9a3c210906b9a32372db747ba2d95d25d276ba3ba45046b941dbbef
+data/2016/d652d24e8bd4fe6a48d634b116f37c27dhluattphcm-1468939398928.xlsx	f519010d9b2631457c9b7784bb77f07979907321199b98b3f97cb45e5b20c2ff
+data/2016/d75282bff0a4d8cd1421c3fb74f789c3dhsaigon-1468900867561.xlsx	ee7734bac44228a58a11b85d490cbd5da27d7a6e07b22c1eb9921a123b3e6acd
+data/2016/ddb631f833af894200ef10cd408d2ef0caobang-1468932877769.xlsx	2bbda20bab1190f42b44d0eeabcf2be7f2203d4e5f8a2ad3d80365770a8e2d9e
+data/2016/deca197a916a8f633f6894b4e13c3ca4dhhaiphong-1468901178368.xlsx	221fe731a4b2de02b3952572baf48d66544fddf87dfdd7851b740325c0864d24
+data/2016/dfa63c5b68a8f638379ca0e9c5a1c556namdinh-1468933234061.xlsx	446cd9626fe197f6509efb01ca5b4cb26730907b5e114c4b9c07399a489b45d1
+data/2016/e693bc9de67acc916032c6a3b2fc9eedhanam-1468933146526.xlsx	b1ef9283b500e965205281d3875eff011be1d418312d187831925860b1ddab74
+data/2016/e80dc028d53fa49ee247391a0653dee5quangngai-1468933592400.xlsx	9c6ba937e8ad772a3782baf9c40ef781121c27ffdeb4519170cbf2b81545a146
+data/2016/ec765c32190773f5a18cd8202a5eaf0dquangnam-1468933501467.xlsx	a566488b611ca04c705f385c47b871792ce75f7c1ca793d0528220352d6b49e1
+data/2016/ec84067268f47d523b1e80b493248377dhngoaithuong-1468920480753.xlsx	5d892f08ec818be3ceb343529c9f0b35e0af0aeb43a229446dccaa87c6d4c846
+data/2016/ee09c723da4e86cdbe26213413450af2dhsupham-dhthainguyen-1468939061872.xlsx	aefac40009116615131c926775e6625eeebc615cd2ff4c7c9af399dd2faab982
+data/2016/f0aa5a19c2c8c1fde8211341f0e98e26dhkinhtetphcm-1468939229043.xlsx	5ce36388e304cbba01c513f11f54b940f4299061e2e8acfa4c4219ee536a01f2
+data/2016/f2846894bd5533e02b122a1e3b4333f6hvnganhang-1468939472850.xlsx	142b39b4f77c1d7a648c2397165b315acdd48ae6d354d914dd674a46b3d7160c
+data/2016/f85f04bb2bbd84457b83ac846728a2a6dhspkythuathungyen-1468920667091.xlsx	f8cc571e94720652f1507576ff48616d5331a4ed1403cb95cdc30124a03c3a68
+data/2016/fc511852659150f6305f6f624445300ddhangiang-1468940189321.xlsx	4e16ea6a122e5bc0c62c7e6e55d9ede3a41242dd3e034d92a450bf29e8fff197
+data/2016/ff877788e43bd84f0119dae026996892dhkinhteluat-dhqgtphcm-1468939860894.xlsx	8c9e7b097d0796750995268aa36a9fa00112c0989b513023ef0a53456afd7051
+data/2017/an-giang.xls	5bde5f6447b9e27540f4f72d9b371b6c94376aeb97af65f4ec5eede43ba9f25c
+data/2017/bac-giang.xls	9cd5c6d858062a1fb30c477b699be467139f183745e66d134746e043d73579a4
+data/2017/bac-kan.xls	9642c00d14c54102ce8dc38408676863a87652aee89651113f4ad3e0a25bd1fa
+data/2017/bac-lieu.xls	e364dd65fa068d40e9a390aa3322e1676171af797c5d1f98949277cadab5369e
+data/2017/bac-ninh.xls	539ed3cb1a2df358ab169e2163face76c949aac2eca0d2acb7ceb51e97811bc5
+data/2017/ba-ria-vung-tau.xls	78dc6d67bbf3ffcb7e4407a6d950e28e39366ed62617691c9a9db610a5079134
+data/2017/ben-tre.xls	b4cce0c5a443062a5e7aef95785fc56f421c21d5ad22f993cbc30ec28e078f4f
+data/2017/binh-dinh.xls	fa1a39cb5134a8159e71dabea41c9ef2a3ec263642f19bed54ddd17e5a5e3d98
+data/2017/binh-duong.xls	ced0d7469186d9e5fbbb8c4323a4bb6a589abb8ef23f01d8d39f947a575af4ba
+data/2017/binh-phuoc.xls	d1ecf9b66ea294eec24ac0f703c614a2011b7d693aeb08c719cfcb73cae7ca47
+data/2017/binh-thuan.xls	3cb0f85889ada81cd257f51ae0b0b6414d2ad8f81d7f94f4b1b38010600243c6
+data/2017/ca-mau.xls	c4bba31f7728f7825c20b3c0b708ef977dcf6f56f0dd5c14d9342cea0467e505
+data/2017/can-tho.xls	b5ba59ff7b87c31859686adb98da8ef2e95de84404daff2ac3df1e00b748b57d
+data/2017/cao-bang.xls	9b0a497ed16840a22af34f7e75e316b00dd60490d6d7f8a2f012507c95fad3e7
+data/2017/dak-lak.xls	aae3e074d91afd98d89cc9d07bc1f8debe4559bd78c409cc5f50f2670a2c337a
+data/2017/dak-nong.xls	f7e30b4cdfe951086aeb1c169e7aad88a17bf9fe07aa33f85654e309899f6beb
+data/2017/da-nang.xls	80963ff2f56d2b6bf331546d08184e325ad0ec7432ba948c0f74845f440e87c8
+data/2017/dien-bien.xls	6bcb1969ce81ccb94c6ee625456530b9661ee255c00eb598dc195611a46205c8
+data/2017/dong-nai.xls	af248453ea27e879da1b8cfa0f2cd345aa068f93c884d1bc98862db0ca2f9faa
+data/2017/dong-thap.xls	186a0c4ac6bf838cc496c4604a7e8979712051c71a53622a9ec72af70491e4f7
+data/2017/gia-lai.xls	3a1b64c8362cad028ee53f4fd18b0923315d919b117df9a2fd83d2f46544db98
+data/2017/ha-giang.xls	a3248393c759a41de424e3ed994d2b9298cb39fda3c6eb9a68f48025c5a2e072
+data/2017/hai-duong.xls	6634715f5a0cbac6a8d2d18be0df74e0abd328641721692224758c730fd86e51
+data/2017/hai-phong.xls	7a8a210986de8c562a9128dccfaed1c6fc6c32df0c5e2e8948bee2f73777ba99
+data/2017/ha-nam.xls	f725e212f976af00e68edd6552721e56b30d5fa7d786f548dfc34721ec8b4426
+data/2017/ha-noi.xls	f0f4bc9216e421acf655a40844a4dcb74e871b188cf6d455982ee6e408170ef7
+data/2017/ha-tinh.xls	b0f5bcd8beff7cfe411b7bd88f9e5b9c5a9a0578a55cf18bb003284aca12e57f
+data/2017/hau-giang.xls	9e8bccd0738bac3d68640768a8d9abc3e7a6a4e97e8c5968039530cf313cafb9
+data/2017/hoa-binh.xls	51b1e910bfc47404b411cb2ab1ff14b522d8fe54d66a1d50479350273c7be1b5
+data/2017/ho-chi-minh.xls	c4e2f920a5e58fa62bd0913d994d44079e472b955adfc504b734c6773c485f64
+data/2017/hung-yen.xls	833584c8ea9337734feb37e068bb06232293108d50b18d88f731d30a576cc29a
+data/2017/khanh-hoa.xls	644ab06abca93d7b966205604313a65d50f3a0a6fb8be509cadb55e0d55fe12e
+data/2017/kien-giang.xls	df452e618544a9b7932d918d26dd2a0f2149f92599609c15a9ac373291cfc440
+data/2017/kon-tum.xls	7d26ff078adaf51dcf4333b074f8a444d328d8083835bd55b1034c1a63e4eb17
+data/2017/lai-chau.xls	b37cd06914cad19bc0e523d0b1cfe87db692e26dfab327f5f9a7686b2b14dae9
+data/2017/lam-dong.xls	59cc802a6c4d622f1bb6233f6b9b1a94f342a6eae1c389235383db1767f34720
+data/2017/lang-son.xls	ae376e75bf615483b6cb3a37daa1cea6a7eec4cf3fdd1186801f56d074eb4ce7
+data/2017/lao-cai.xls	7596c99851c20ff60b95347bc9fae78014873b4c3e1f1f7144c48d9f285f7962
+data/2017/long-an.xls	e52d8e8a4592258e87c5cda04e0907541f60e7e7ef4cb21e9171b65283e0b87a
+data/2017/nam-dinh.xls	80dcda8b058dcf1a7fdcd7e34d126c44c31b2d4987b52eeae790bf6c38c58e40
+data/2017/nghe-an.xls	3f32421eb6ee8bc21fcedd4cc1043df1c9c0381abfaa1006c85c3ca6d6209543
+data/2017/ninh-binh.xls	dc3d08a86cdd145e870486773d95a1d896a003b5400ba6ff8456a5512b10b2b8
+data/2017/ninh-thuan.xls	d69b0ed7192e85cff31ad5bca8942fe3c92322268d1ea707e945f9991bd78115
+data/2017/phu-tho.xls	e352a51109a2373c1efbb6726d262fc490bcbf9229ae90bd473d14a4b6eebcc0
+data/2017/phu-yen.xls	6871aedd90d2c6c87ad6409164a9b26f6f89f97cdd7a1dffbbe4e6879f2348c6
+data/2017/quang-binh.xls	6c5e9b3c6780a5b70a02dd4b4854861ba148613361c048bfc001706e5bb77a61
+data/2017/quang-nam.xls	859258cb8c4b7806101039e4e2484e56971f2640a65720a108a0933245bd3157
+data/2017/quang-ngai.xls	834988a6484fab64bc59fe35a6813038663054553d269d16b7d8200593f468e8
+data/2017/quang-ninh.xls	1b1c0b1ea1d6aadd8bce5faef6b5fdc5181daaa9bd44df6acbf06c9021a0ade9
+data/2017/quang-tri.xls	35e401061962c6d80593d684f179e1f82b05f884a6a470d1bc17b2d31151e05f
+data/2017/soc-trang.xls	668a0edcb624faa954c424616ea6c35d6f06b9e9fc62524723dea01af09490c0
+data/2017/son-la.xls	96bcfeb35e5bc23ccd7dc6a62a533e21dabeb3145473ecc5a039d970019a1129
+data/2017/tay-ninh.xls	aedf4ccab43e7a1fa985721de0d6053988ad471355c0ccb0b8f9c19af02423e2
+data/2017/thai-binh.xls	b20f66cdeb975a64eed7322b616bd291fe34e08e5fbedad3d729d21da0eba627
+data/2017/thai-nguyen.xls	f0b93db9aab966d393b8a3323204ba7f147ddd0a92746355b81df09bc58267b4
+data/2017/thanh-hoa.xls	3f803ae67dac24317fa45dac60f36e39dc222ff84d23a0a3dde7751cdcb17646
+data/2017/thua-thien-hue.xls	71e220ee4ea0287657fa379c587ca93f71f96b4dacc48cef074f8c223e940daa
+data/2017/tien-giang.xls	2e061f8724e1ea8b997d7e6700ac89a60921771a57dffa7297f4728715811406
+data/2017/tra-vinh.xls	59507a1d237ac5e29c1033d42b08da6ac9a7505a5045b791e512e91493102ae7
+data/2017/tuyen-quang.xls	053d293e4acd9dfa6ce9532a047cc64b4d2edee1ff13b387723586c1fae6312a
+data/2017/vinh-long.xls	6e7584a3047b0ba7487dbb8c71f5b594db220f11b747149d41d31ea97e1e4eab
+data/2017/vinh-phuc.xls	9753e2d864584e76aef1e690e4e2de98b28240fbba810e614eab9926668d66d2
+data/2017/yen-bai.xls	fe6824a1254116bc6ed372e0a1fd635a816a09e010b7efde9659baca2e143138