7 Commits
Author SHA1 Message Date
tiennm99 4107167b03 docs: index merged crawlers and point their module paths at mttools 2026-10-03 11:16:25 +07:00
tiennm99 a912c7e084 docs: require Go 1.25 to match go.mod 2026-08-31 15:25:31 +07:00
tiennm99 d17a79445f feat(pdf): export a novel as a phone-readable PDF
Add a pdfout package that renders the fetched chapters to a single PDF
and an export package holding the whole pipeline, so the CLI and an
embedding program share one code path and reject the same inputs.

The default page is 90x160mm rather than A4 with large type: a viewer
scales a whole page to fit the screen, so on a phone the page shape
decides legibility and a wide page just gets shrunk.

Embed DejaVu Sans in the binary. Vietnamese needs the Latin Extended
Additional block, and a headless host often has no fonts installed at
all, so rendering must not depend on finding one. An explicitly named
font is never silently substituted. Font data is passed as bytes because
fpdf resolves a font path against its own font directory.

Keep the original per-chapter text output available behind -txt.
2026-08-24 20:40:01 +07:00
tiennm99 ad8d2649c1 refactor!: generalise single-novel script into a hako library
The crawler took one hardcoded novel URL, wrote per-chapter text files
into ./data, and fetched serially until the site answered 429.

Rename the module to hako-crawler and split fetching, page parsing and
crawl orchestration into a hako package that takes any novel URL.

The chapter body is no longer served as markup: #chapter-content now
carries a shuffled, XOR-obfuscated payload that the site's own script
expands in the browser, so the previous p[id=<digits>] extractor
returned nothing at all. Decode it, keeping the plain-markup path as a
fallback. The scheme is read from the page and an unrecognised one is a
hard error, because the alternative is exporting a book of mojibake.

Space requests globally instead of per-goroutine, so concurrency no
longer raises the request rate, and retry only what a retry can fix.
Cache raw pages so re-runs cost no requests.

BREAKING CHANGE: module path is now github.com/tiennm99/hako-crawler,
and the novel URL is a required argument rather than a constant.
2026-08-24 20:39:50 +07:00
tiennm99 f2b2cb33c5 feat(go): rewrite in Go 2025-11-30 20:22:58 +07:00
tiennm99 29feea38c8 [Init] 2024-10-11 21:03:49 +07:00
Tien Nguyen Minh b9f1f41c18 Initial commit 2024-10-11 19:54:02 +07:00