Revert "Merge branch 'fix/httpvfs-chunked-length'"

This reverts commit d4f106f488, reversing
changes made to c5ef1f58ff.

docs/data-pipeline.md

@@ -199,7 +199,7 @@ exists.
 ## Verifying a rebuild
 
 The assembler verifies itself: each database's row count must match the
-figure in the table above, and each `.sqlite30` must be at least 90% of its usual
+figure in the table above, and each `.sqlite3` must be at least 90% of its usual
 size, or the build fails rather than publishing. That guard is the reason a
 truncated dataset cannot reach the site with a green pipeline.
 
@@ -213,8 +213,8 @@ go -C assembler run ./cmd/assemble db   # rebuild
 go -C assembler run ./cmd/assemble verify /tmp/before .build/public/db
 ```
 
-Each side is a directory of `<id>.sqlite30`; a gzipped database from before the
-switch to range requests is still expanded to a temporary file automatically. It exits non-zero on any mismatch,
+Each side is a directory of `<id>.sqlite3`; a database under an older name, or
+a gzipped one, is still opened automatically. It exits non-zero on any mismatch,
 names the first differing rows and columns, and fails rather than skipping when a
 dataset is absent from either side — silently comparing one of two datasets is
 how a gate passes without proving anything.

docs/deployment-guide.md

@@ -106,26 +106,21 @@ from publishing something it did not mean to.
   files committed to a repository; the databases are built in CI and uploaded
   as a Pages artifact, and the documented Pages limits are a 1 GB published
   site and 100 GB/month of bandwidth, with no per-file figure. The two
-  databases are 302 MB and 247 MB.
-- **Total artifact is about 552 MB**, inside the 1 GB site limit but with less
-  headroom than before: a third dataset of this size would not fit. The fallback
-  is `sql.js-httpvfs`'s chunked mode, which splits a database into parts.
-- **Pages does compress the databases, and that is survivable.** The extension
-  is unknown to Pages, so the file is served as `application/octet-stream`,
-  which is marked compressible in `mime-db` and gzipped: a plain request
-  returns `Content-Encoding: gzip` and the compressed length. Ranged reads are
-  not affected, because the Fetch standard makes browsers send
-  `Accept-Encoding: identity` on any request carrying a `Range` header. Only
-  the length probe breaks, so the site supplies the length itself instead of
-  trusting HEAD — see `web/src/lib/db-probe.js` and the chunked-mode note in
-  [system-architecture](./system-architecture.md).
-- **Verify the way a browser asks.** A bare `curl -sI` advertises no encoding
-  and so reports success whatever the host does; it is what let this reach
-  production. Check ranged reads instead, and check the bytes, not the headers:
+  databases are 142 MB and 119 MB.
+- **Total artifact is about 263 MB**, comfortably inside the 1 GB site limit.
+  Dropping the search index the range-request design needed halved both files.
+- **Pages compresses the databases on the wire, which is a benefit here.** The
+  extension is unknown to Pages, so the file is served as
+  `application/octet-stream`, which `mime-db` marks compressible, and the
+  browser decompresses it transparently: 142 MB stored becomes about 31 MB
+  delivered for 2016, and 119 MB becomes about 36 MB for 2017. Note that the
+  smaller database is the larger download: the two compress differently, so
+  neither transfer figure can be derived from the stored size.
+- **Verify the bytes, not the headers.** A bare `curl -sI` reports success
+  whatever the host does. Read the file's first bytes instead:
 
   ```bash
-  curl -s -r 0-15 -H 'Accept-Encoding: identity;q=1, *;q=0' \
-    https://<user>.github.io/thptqg/db/2016.sqlite30 | head -c 16
+  curl -s -r 0-15 https://<user>.github.io/thptqg/db/2016.sqlite3 | head -c 16
   # must print: SQLite format 3
   ```
 
@@ -141,9 +136,8 @@ run rebuilds the older state. There is no data to migrate.
 | Blank page, 404 on assets | `paths.base` in `svelte.config.js` does not match the repo name |
 | `Failed to fetch database: 404` | Dataset id in `datasets.json` does not match the file in `db/` |
 | A route 404s | The site step did not run, or the id is missing from `datasets.json` |
-| `Length of the file not known` | The host gzipped the un-ranged response, so HEAD reports the compressed size. The site supplies `databaseLengthBytes` from a range probe; if this returns, the config is no longer reaching the worker in chunked mode |
-| Database fails to open | A ranged read did not return raw database bytes. The range check above must print `SQLite format 3` |
-| Every query is slow or huge | It is not using an index. `EXPLAIN QUERY PLAN` it: a `SCAN` means the browser is fetching the whole table |
-| Deploy fails on assembly | An uncompressed database artefact reached the output; the error names the files |
+| Download gate never finishes | The file is not being served, or the tab ran out of memory holding it. The check above must print `SQLite format 3` |
+| Tab crashes on a phone | The database needs 142 MB of memory for 2016, 119 MB for 2017; a low-memory device may have the tab killed |
+| Deploy fails on assembly | A stray database artefact reached the output — a journal, a `.gz`, or a name from an earlier design; the error names the files |
 | Missing rows after a data update | Unknown Excel header — check the per-file row counts the parser prints |
 | Visitor still sees old data after a deploy | Their stored copy is keyed by ETag, so this should not happen. Confirm the deployed file's `ETag` header actually changed |

docs/system-architecture.md

@@ -24,11 +24,11 @@ through. `assembler/` sequences everything from the parser onwards.
 data/<id>/*.xls(x)
         │
         ▼  parser/    (Go, one binary, one config per dataset)
-   .build/public/db/<id>.sqlite30
+   .build/public/db/<id>.sqlite3
         │
         ▼  assembler/ — row count and size must match datasets.json
-   .build/public/db/<id>.sqlite30   (uncompressed: ranges of a gzip stream
-        │                            are not ranges of the database)
+   .build/public/db/<id>.sqlite3    (uncompressed: the host gzips it on the
+        │                            wire, so a .gz would decompress twice)
         ▼  assembler/ → npm run build (SvelteKit static, assets = .build/public)
    web/dist/
         │
@@ -71,7 +71,7 @@ figure is reported separately as the transfer size.
 One identifier ties the whole pipeline together:
 
 ```
-data/2017/  →  parser/configs/2017.yml  →  db/2017.sqlite30  →  /thptqg/2017/
+data/2017/  →  parser/configs/2017.yml  →  db/2017.sqlite3   →  /thptqg/2017/
 ```
 
 `datasets.json` at the repository root declares the ids once, with the row count
@@ -204,57 +204,56 @@ total descending.
 
 | Concern | Choice | Rationale |
 | --- | --- | --- |
-| Storage | Static SQLite file, read by range request | No backend; the datasets are frozen, and a lookup needs a few pages of them |
-| Reading mode | `serverMode: "chunked"` over a single chunk | The only mode whose config accepts the file length. In full mode the worker hardcodes it to `undefined` and falls back to a HEAD request, which Pages answers with the gzipped size. One chunk means the index is always 0, hence the published name `<id>.sqlite30` |
-| Compression | None | A byte range of a gzip stream is not a byte range of the database |
-| WASM hosting | Bundled with the app | `sql.js-httpvfs` ships its own build; one less third-party runtime dependency |
-| Diacritics search | Pre-computed `ho_ten_ascii`, indexed word by word | `LOWER(REPLACE(...))` at query time defeats the index, and `LIKE '%x%'` reads the whole table |
-| Row count in the footer | Read from `datasets.json` | `COUNT(*)` scans an index — 20 MB over range requests |
-| Page size | 1 KiB, matched by `requestChunkSize` | One HTTP request is one page; a row fetched by seek costs 1 KB rather than 4 KB, for about 5% more file |
-| SQL safety | Leading-keyword allowlist | `sql.js` is in-memory so writes cannot persist; the allowlist prevents confusion |
+| Storage | Static SQLite file, downloaded whole | No backend; the datasets are frozen, and one large transfer is something browsers and CDNs are both good at |
+| Download gate | Blocking, no dismiss | The page has no answers before the file arrives, and 31 MB of someone's mobile data should be asked for rather than spent silently |
+| Keeping the download | Cache Storage, versioned by ETag | The transfer is paid once per device instead of once per visit; the ETag is what stops a redeploy being answered with last week's data |
+| Searching | On submit, never while typing | A name search scans every row, so a keystroke-per-search would run hundreds of full scans to answer one question |
+| Compression | None published | The host gzips on the wire, so a `.gz` artifact would only be decompressed twice |
+| WASM hosting | Bundled with the app | `sql.js` ships its own build; one less third-party runtime dependency |
+| Diacritics search | Pre-computed `ho_ten_ascii` | `LOWER(REPLACE(...))` at query time is far slower over 877,460 rows than a column computed once at build |
+| Secondary indexes | None | In memory a full scan costs a few hundred milliseconds; an index costs every visitor megabytes of download. The name index alone was 146 MB |
+| Row count in the footer | Read from `datasets.json` | Costs nothing and is the same number the assembler enforces |
+| Page size | 4 KiB | SQLite's default, and nothing on the client depends on it any more |
+| SQL safety | Leading-keyword allowlist | The copy is the visitor's own, so this guards their session against a typo rather than protecting data |
 | Row caps | 100 (lookup), 1000 (SQL) | Keeps DOM render sizes reasonable |
 | Routing | SvelteKit file routes, prerendered | Each dataset gets a real HTML file with its own title |
 | Styling | Tailwind, with tier colours as CSS variables | Tier classes are chosen at runtime, which no utility generator can see |
 
 ### Considered and not taken
 
-- **Splitting the database into several chunks.** The site uses chunked mode,
-  but over one chunk (see above). Real splitting would let a CDN cache each part
-  whole; GitHub Pages serves everything with `Cache-Control: max-age=600`, and
-  every rebuild relays SQLite's pages so the file changes even when the data
-  does not, so that caching is cancelled by the host. Worth revisiting behind a
-  CDN with long TTLs, and it is the fallback if a single 300 MB file ever
-  becomes a problem.
-- **`sqlite-wasm-http`.** Maintained, and built on the official SQLite WASM
-  rather than a 2022 fork, which is the better long-term footing. It does not
-  help here: its worker sizes the file from a HEAD request's `Content-Length`
-  exactly as `sql.js-httpvfs` does, and its `Options` has no field for the
-  length, so on Pages it would silently take the gzipped size instead of
-  failing. Its shared-cache backend needs COOP/COEP, which Pages cannot send,
-  but it ships a fallback backend that does not — so isolation is not the
-  blocker, the missing length option is.
-- **Substring name search.** `LIKE '%x%'` cannot use an index, so it read the
-  whole 127 MB table. `name_word` keeps search by any word of a name without
-  it.
+- **Reading the file over HTTP range requests** (`sql.js-httpvfs`), which this
+  site did until it was measured. Two costs killed it. Reads are serial —
+  the worker uses synchronous XHR — so a name search that touched 390 pages
+  waited 17 seconds to move 608 KB, and roughly one request per result row is a
+  floor no page size removes. And the first visitor after each deploy waited
+  ~26 s for the CDN to fill its cache with a 288 MB object. The download pays
+  once, up front, visibly.
+- **`sqlite-wasm-http`.** Built on the official SQLite WASM and maintained,
+  but it sizes the file from a HEAD request's `Content-Length` and exposes no
+  option to override it, so on a host that gzips it would silently use the
+  compressed size. Same class of problem, less recourse.
+- **DuckDB-WASM over Parquet.** Genuinely maintained, async, parallel range
+  requests. Its binaries are 32–37 MB before the Parquet extension, which is
+  more than the entire database download for a phone looking up one score.
+- **Static pre-generated shards**, one file per exam-number bucket. The most
+  robust option and the fastest single lookup, but it cannot answer arbitrary
+  SQL, and a static file cannot stop early the way `LIMIT` does — a common
+  Vietnamese surname would mean fetching a very large posting list.
 
 ## Risks and limitations
 
-- **Unindexed queries are expensive.** The SQL tab can express a query that
-  walks the table, which over range requests means fetching 100+ MB. A byte
-  budget stops one before it gets that far, and the tab warns before it opens.
-- **`Content-Encoding` on a ranged response would break everything.** A range
-  of a compressed body addresses the wrong bytes. In practice browsers prevent
-  it: the Fetch standard requires `Accept-Encoding: identity` on any request
-  carrying a `Range` header. GitHub Pages *does* gzip the un-ranged response —
-  `application/octet-stream` is compressible in `mime-db` — which is why the
-  file length is probed with a range request and passed as
-  `databaseLengthBytes` rather than left to the library's HEAD.
-  `db-probe.js` checks the returned bytes
-  start with the SQLite magic, so a host that ever compresses a ranged response
-  fails loudly instead of returning nonsense.
-- **`sql.js-httpvfs` is unmaintained** (0.8.12, September 2022) and ships its
-  own SQLite WASM. `sqlite-wasm-http`, on the official build, is the fallback.
-- **Hosted size.** 552 MB for both datasets against the 1 GB GitHub Pages
+- **Memory is the binding constraint.** The database lives in the tab's
+  WebAssembly memory for as long as the page is open: 142 MB for 2016, 119 MB
+  for 2017. A low-memory phone may have the tab killed, which is why the gate
+  states the figure before the download starts.
+- **A deploy costs returning visitors the transfer again.** Every rebuild lays
+  SQLite pages out differently, so the file — and its ETag — changes even when
+  the data does not. The stored copy is then a stale version and is replaced.
+- **The stored copy can be evicted.** Cache Storage is subject to the browser's
+  own storage pressure, so a device short on disk falls back to downloading.
+- **A visitor who will not download cannot use the site.** That is the
+  deliberate shape of the gate, and it makes the first impression a 31 MB ask.
+- **Hosted size.** 526 MB for both datasets against the 1 GB GitHub Pages
   limit; a third dataset of this size would not fit.
 - **Excel format drift.** A new source file with an unseen header layout needs a
   new branch in `parser/internal/ingest/detect2016.go` or a new config.