fix(web): take the database length from a range probe, not the host

GitHub Pages serves .sqlite3 as application/octet-stream, which mime-db
marks compressible, so an un-ranged response comes back gzipped with the
compressed length. sql.js-httpvfs sizes a file with a HEAD request, sees
that length is unusable and refuses to open the database:

  Length of the file not known. It must either be supplied in the config
  or given by the HTTP server.

Page reads were never affected. The Fetch standard requires browsers to
send Accept-Encoding: identity on any request carrying a Range header,
and the live site returns 206 with raw bytes to one. So the length is
probed the same way and passed as fileLength, which is the escape hatch
the library's own message points at.

The probe reads the first 100 bytes, so it also checks the file starts
with the SQLite magic and that its page size matches the request size —
a host that ever compresses a ranged response now fails with a clear
message rather than feeding the library the wrong bytes.

The post-deploy check the docs prescribed could not have caught this: a
bare `curl -sI` advertises no encoding, so it reports success whatever
the host does. It is replaced, in the docs and in CI, by a ranged read
that verifies the bytes.
This commit is contained in:
tiennm99 committed 2026-08-14 15:23:20 +07:00
1 parent 3fed50cf8f
commit 9571fb7d55
7 files changed
+253 -12

No files matched your search

+9 -4
View File
@@ -204,10 +204,15 @@ total descending.
- **Unindexed queries are expensive.** The SQL tab can express a query that
walks the table, which over range requests means fetching 100+ MB. A byte
budget stops one before it gets that far, and the tab warns before it opens.
- **`Content-Encoding` breaks everything.** If the host ever compresses
`<id>.sqlite3` on the wire, ranges address compressed bytes and
`sql.js-httpvfs` refuses to open the file. Verify after a deploy:
`curl -sI …/db/2016.sqlite3` must show no `content-encoding`.
- **`Content-Encoding` on a ranged response would break everything.** A range
of a compressed body addresses the wrong bytes. In practice browsers prevent
it: the Fetch standard requires `Accept-Encoding: identity` on any request
carrying a `Range` header. GitHub Pages *does* gzip the un-ranged response —
`application/octet-stream` is compressible in `mime-db` — which is why the
file length is probed with a range request and passed as `fileLength` rather
than left to the library's HEAD. `db-probe.js` checks the returned bytes
start with the SQLite magic, so a host that ever compresses a ranged response
fails loudly instead of returning nonsense.
- **`sql.js-httpvfs` is unmaintained** (0.8.12, September 2022) and ships its
own SQLite WASM. `sqlite-wasm-http`, on the official build, is the fallback.
- **Hosted size.** 552 MB for both datasets against the 1 GB GitHub Pages