feat(pdf): export a novel as a phone-readable PDF

Add a pdfout package that renders the fetched chapters to a single PDF
and an export package holding the whole pipeline, so the CLI and an
embedding program share one code path and reject the same inputs.

The default page is 90x160mm rather than A4 with large type: a viewer
scales a whole page to fit the screen, so on a phone the page shape
decides legibility and a wide page just gets shrunk.

Embed DejaVu Sans in the binary. Vietnamese needs the Latin Extended
Additional block, and a headless host often has no fonts installed at
all, so rendering must not depend on finding one. An explicitly named
font is never silently substituted. Font data is passed as bytes because
fpdf resolves a font path against its own font directory.

Keep the original per-chapter text output available behind -txt.

hako-crawler/README.md

@@ -1,25 +1,178 @@
-# atnvc-crawler
+# hako-crawler
 
-Crawl data of "Anh trai nhân vật chính", a novel from https://ln.hako.vn/sang-tac/8476-kiep-nay-la-anh-trai-cua-nhan-vat-chinh
+Downloads every chapter of an [ln.hako.vn](https://ln.hako.vn) novel and exports
+it as a single PDF sized for reading on a phone.
 
-## The Original Python Version
+## Install
 
-*In 2025, I rewrote this project using Go. The original Python version of this project can be found at the `feature/python` branch.*
+Requires Go 1.24+.
 
-## Quick Start
-
-1. Install Go: https://go.dev/dl/
-2. `go mod init atnvc-crawler` (if needed)
-3. `go get github.com/PuerkitoBio/goquery`
-4. `go run main.go`
-
-Data saved in `./data/` as `<chapter-title>.txt`.
-
-## Rate Limits (HTTP 429)
-Skip crawled chapters and retry later. Uncomment line 104, replace 100 with number of chapters you want to skip.
-```go
-chapters = chapters.Slice(50, chapters.Length())
+```sh
+go build ./cmd/hako-crawler
 ```
 
-## Customize
-This version should work with other novels on Hako with some modifications. Feel free to try it!
+## Usage
+
+Pass the novel's landing page URL. "Anh Trai Nhân Vật Chính" — the novel this
+project was originally written for — works as the example:
+
+```sh
+./hako-crawler -url https://ln.hako.vn/sang-tac/8476-kiep-nay-la-anh-trai-cua-nhan-vat-chinh
+```
+
+The PDF is named after the novel unless you pass `-out`.
+
+```sh
+# Try the layout on 3 chapters before fetching all 147
+./hako-crawler -url <novel URL> -limit 3
+
+# Bigger type on an A5 page for a tablet
+./hako-crawler -url <novel URL> -page a5 -font-size 12 -out truyen.pdf
+
+# Also keep one plain-text file per chapter
+./hako-crawler -url <novel URL> -txt ./data
+```
+
+### Flags
+
+| Flag | Default | Purpose |
+| --- | --- | --- |
+| `-url` | *required* | Novel landing page URL |
+| `-out` | novel name | Output PDF path |
+| `-txt` | off | Also write one plain-text file per chapter into this directory |
+| `-page` | `phone` | Page size: `phone`, `a5`, `a4` |
+| `-font-size` | `10` | Body font size in points |
+| `-line-spacing` | `1.55` | Line height as a multiple of font size |
+| `-margin` | `6` | Page margin in mm |
+| `-font` | auto | Path to a `.ttf`; defaults to a system font, else the bundled one |
+| `-workers` | `4` | Concurrent chapter fetches |
+| `-delay` | `500ms` | Minimum delay between requests |
+| `-retries` | `3` | Retries per request |
+| `-limit` | `0` | Fetch only the first N chapters (0 = all) |
+| `-cache` | `.cache` | Cache directory for raw pages (empty to disable) |
+
+## Why the default page is 90×160 mm
+
+Phone readability is governed by page *shape* more than by font size. A PDF
+viewer scales a whole page to fit the screen, so a large font on an A4 page
+still ends up small: the page is about three times wider than a phone screen
+and gets shrunk to match. The default page is cut to a 9:16 ratio so it fills
+the screen at 100% zoom, where the default 10 pt renders at a comfortable size.
+Use `-page a5` or `-page a4` for a tablet or for printing.
+
+## How it works
+
+1. Fetch the landing page: title, genres, and the chapter list, read from the
+   `.volume-list` sections. The site lists volumes oldest-first and chapters
+   ascending within a volume, so document order is already reading order.
+2. Fetch each chapter concurrently and decode its body.
+3. Render one PDF, each chapter starting on a new page, with the volume name
+   printed above the heading whenever it changes.
+
+### The chapter body is not markup
+
+The site no longer ships chapter text as HTML. `#chapter-content` holds an
+encoded payload that its own JavaScript expands in the browser:
+
+```html
+<div id="chapter-c-protected" data-s="xor_shuffle"
+     data-k="6b9dd83fad5a3169" data-c="[&quot;0001BVsFVFQKD1YNF0Ra…&quot;, …]">
+```
+
+`data-c` is a JSON array of chunks in shuffled order. Each chunk is prefixed
+with its own 4-digit position; strip that, base64-decode the rest, and XOR every
+byte against the ASCII bytes of `data-k`, cycling. The key restarts at the
+beginning of *every chunk* rather than running across the joined stream, so
+XOR-ing the concatenated payload in one pass produces garbage. Sorting by the
+position prefix then joining yields the original chapter HTML, which contains
+only `p`, `em`, `strong` and `img` — no site furniture — so extracting the text
+afterwards is a plain paragraph walk.
+
+Reading DOM text alone yields an *empty* chapter, which is why the
+`p[id=<digits>]` extractor this project started with stopped returning anything.
+
+`data-s` names the scheme and is read from the page rather than assumed. An
+unrecognised value is a hard error: the alternative is a book full of mojibake
+that looks like a successful export. If the site rotates its encoding, that
+error message is what says so.
+
+### Illustration chapters
+
+Hako novels carry illustration chapters that hold pictures and captions rather
+than prose. They are real chapters, so a near-empty one is exported with its
+heading instead of failing the run.
+
+## Politeness and caching
+
+Requests are spaced by `-delay` globally, so raising `-workers` does not raise
+the request rate — which is what keeps a full-novel crawl off the HTTP 429 that
+the original single-threaded script kept hitting. A 429 or 5xx is retried with
+exponential backoff; a 404 or 403 fails immediately rather than spending the
+retry budget on something that will not change.
+
+Raw pages are cached under `.cache/`, so re-exporting with different font or
+page settings costs no requests. Delete the directory to refetch.
+
+## Tests
+
+```sh
+go test ./...
+```
+
+Tests run against synthetic fixtures — including a payload encoded the same way
+the site encodes one, with chunks shuffled and multi-byte characters straddling
+chunk boundaries. No network access required.
+
+## Layout
+
+```
+cmd/hako-crawler/    CLI
+export/              URL -> PDF in one call; shared by the CLI and importers
+hako/                fetching, page parsing, body decoding, crawl orchestration
+pdfout/              PDF rendering, font resolution, bundled fallback font
+```
+
+## Use as a library
+
+The packages are importable, so another Go program can produce the same PDF
+without shelling out to the binary. `export.Export` is the whole pipeline:
+
+```go
+result, err := export.Export(ctx, export.Request{
+	NovelURL: "https://ln.hako.vn/sang-tac/8476-kiep-nay-la-anh-trai-cua-nhan-vat-chinh",
+	OutDir:   tmpDir,
+})
+```
+
+Only `NovelURL` is required; each zero-valued field falls back to the same
+default as the matching CLI flag. Because zero means "unset", ask for *no* cache
+or *no* request delay with the `NoCache` and `NoDelay` fields rather than by
+zeroing `CacheDir` or `Delay`. Pass a `Log` function to receive the progress
+messages the CLI prints to stderr.
+
+## Fonts
+
+The PDF embeds a TrueType font, and Vietnamese needs one covering the Latin
+Extended Additional block — a basic-Latin font silently drops the diacritics.
+The font is resolved in this order:
+
+1. the path given to `-font` / `Request.FontFile`, which is an error if it
+   cannot be read — a named font is not silently substituted;
+2. a system font known to cover Vietnamese (see `pdfout.FindFont`);
+3. the bundled DejaVu Sans, compiled into the binary.
+
+Step 3 means rendering never depends on the host having fonts installed, which
+is what a minimal container or a headless server usually looks like. See
+[`pdfout/fonts/NOTICE.md`](pdfout/fonts/NOTICE.md) for the bundled font's
+provenance and licensing.
+
+## History
+
+This started as a Python script for a single novel, was rewritten in Go in 2025,
+and became a general hako exporter in 2026. The Python version is on the
+`feature/python` branch.
+
+## Scope
+
+Downloaded text stays on your machine. Only fetch content you are allowed to
+read offline, and respect the site's terms.

hako-crawler/cmd/hako-crawler/main.go

@@ -0,0 +1,99 @@
+// Command hako-crawler downloads every chapter of an ln.hako.vn novel and
+// exports it as a PDF sized for reading on a phone.
+package main
+
+import (
+	"context"
+	"flag"
+	"fmt"
+	"os"
+	"os/signal"
+	"path/filepath"
+	"strings"
+	"syscall"
+
+	"github.com/tiennm99/hako-crawler/export"
+	"github.com/tiennm99/hako-crawler/pdfout"
+)
+
+// exampleURL is the novel this project was originally written for; it stands in
+// as the example now that any novel can be passed instead.
+const exampleURL = "https://ln.hako.vn/sang-tac/8476-kiep-nay-la-anh-trai-cua-nhan-vat-chinh"
+
+func main() {
+	if err := run(); err != nil {
+		fmt.Fprintln(os.Stderr, "error:", err)
+		os.Exit(1)
+	}
+}
+
+func run() error {
+	req, err := parseFlags()
+	if err != nil {
+		return err
+	}
+
+	// Ctrl-C cancels in-flight fetches instead of leaving a partial PDF.
+	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
+	defer stop()
+
+	req.Log = func(format string, args ...any) {
+		fmt.Fprintf(os.Stderr, format+"\n", args...)
+	}
+
+	result, err := export.Export(ctx, *req)
+	if err != nil {
+		return err
+	}
+
+	fmt.Fprintf(os.Stderr, "\n%s\n", result.Summary())
+	fmt.Fprintf(os.Stderr, "font: %s at %.0fpt on %s page (%.0f x %.0f mm)\n",
+		filepath.Base(result.FontName), req.FontSize, result.Page.Name, result.Page.W, result.Page.H)
+	if result.TextDir != "" {
+		fmt.Fprintf(os.Stderr, "text: %s\n", result.TextDir)
+	}
+	fmt.Println(result.Path)
+	return nil
+}
+
+// parseFlags builds the export request from the command line. Validation of the
+// resulting values lives in export.Export, so the CLI and an embedding program
+// reject the same inputs.
+func parseFlags() (*export.Request, error) {
+	req := &export.Request{}
+
+	flag.StringVar(&req.NovelURL, "url", "", "novel page URL, e.g. "+exampleURL)
+	flag.StringVar(&req.OutPath, "out", "", "output PDF path (default: the novel's name)")
+	flag.StringVar(&req.TextDir, "txt", "", "also write one plain-text file per chapter into this directory")
+	flag.StringVar(&req.Page, "page", export.DefaultPage,
+		"page size: "+strings.Join(pdfout.PresetNames(), ", "))
+	flag.StringVar(&req.FontFile, "font", "", "path to a .ttf font (default: a Vietnamese-capable system font)")
+	flag.Float64Var(&req.FontSize, "font-size", export.DefaultFontSize, "body font size in points")
+	flag.Float64Var(&req.LineSpacing, "line-spacing", export.DefaultLineSpacing, "line height as a multiple of font size")
+	flag.Float64Var(&req.Margin, "margin", export.DefaultMargin, "page margin in millimetres")
+	flag.IntVar(&req.Workers, "workers", export.DefaultWorkers, "concurrent chapter fetches")
+	flag.DurationVar(&req.Delay, "delay", export.DefaultDelay, "minimum delay between requests")
+	flag.IntVar(&req.Retries, "retries", export.DefaultRetries, "retries per request")
+	flag.IntVar(&req.Limit, "limit", 0, "only fetch the first N chapters (0 = all)")
+	flag.StringVar(&req.CacheDir, "cache", export.DefaultCacheDir,
+		"directory for cached pages, so re-exports need no requests (empty to disable)")
+
+	flag.Usage = func() {
+		fmt.Fprintf(flag.CommandLine.Output(),
+			"Download an ln.hako.vn novel and export it as a phone-friendly PDF.\n\n"+
+				"Usage:\n  hako-crawler -url <novel page URL> [flags]\n\nFlags:\n")
+		flag.PrintDefaults()
+	}
+	flag.Parse()
+
+	if req.NovelURL == "" {
+		flag.Usage()
+		return nil, fmt.Errorf("-url is required")
+	}
+	// The flag defaults above are already applied, so a zero value here can only
+	// come from the user asking for none. Say so explicitly: left as the zero
+	// value, Export would read it as "unset" and restore the default.
+	req.NoCache = req.CacheDir == ""
+	req.NoDelay = req.Delay == 0
+	return req, nil
+}

hako-crawler/export/export.go

@@ -0,0 +1,277 @@
+// Package export turns a novel URL into a PDF file. It holds the sequence the
+// CLI and any embedding program both need — read the chapter list, fetch the
+// chapters, pick a font, render — so neither has to reassemble it.
+package export
+
+import (
+	"context"
+	"fmt"
+	"net/url"
+	"os"
+	"path/filepath"
+	"regexp"
+	"strings"
+	"time"
+
+	"github.com/tiennm99/hako-crawler/hako"
+	"github.com/tiennm99/hako-crawler/pdfout"
+)
+
+// Defaults for every tunable field of Request. Exported so a caller's own flags
+// or config can advertise the same values instead of restating them.
+const (
+	DefaultPage        = "phone"
+	DefaultFontSize    = 10.0
+	DefaultLineSpacing = 1.55
+	DefaultMargin      = 6.0
+	DefaultWorkers     = 4
+	DefaultDelay       = 500 * time.Millisecond
+	DefaultRetries     = 3
+	DefaultCacheDir    = ".cache"
+)
+
+// Request describes one export. Only NovelURL is required; every zero-valued
+// tunable falls back to its Default above, so a caller that only has a URL can
+// leave the rest alone.
+type Request struct {
+	NovelURL string
+
+	// OutPath is the exact PDF path to write. When empty the file is named
+	// after the novel and placed in OutDir.
+	OutPath string
+	OutDir  string
+
+	// TextDir, when set, also writes one plain-text file per chapter — the
+	// output this project produced before it rendered PDFs.
+	TextDir string
+
+	Page        string  // preset key: phone, a5, a4
+	FontFile    string  // path to a .ttf; empty means discover a system font
+	FontSize    float64 // points
+	LineSpacing float64 // multiple of font size
+	Margin      float64 // millimetres
+
+	Workers int
+	Retries int
+
+	// Delay is the minimum spacing between requests. Set NoDelay to remove the
+	// spacing rather than setting this to 0, which is read as "unset" and gets
+	// the default back.
+	Delay   time.Duration
+	NoDelay bool
+
+	// Limit caps the export to the first N chapters. 0 means every chapter.
+	Limit int
+
+	// CacheDir stores raw pages so a re-export costs no requests. Set NoCache
+	// to opt out rather than clearing this field, which would be read as
+	// "unset" and get the default back.
+	CacheDir string
+	NoCache  bool
+
+	// Log receives progress messages. Optional.
+	Log func(format string, args ...any)
+}
+
+// Result reports what was produced.
+type Result struct {
+	Path      string
+	TextDir   string
+	Title     string
+	SourceURL string
+	Chapters  int
+	Words     int
+	FontName  string // font path, or pdfout.BundledFontName
+	Page      pdfout.PageSize
+}
+
+// Summary renders a one-line description of the exported book.
+func (r *Result) Summary() string {
+	return fmt.Sprintf("%s — %d chapters, %d words", r.Title, r.Chapters, r.Words)
+}
+
+// Export fetches the novel at req.NovelURL and writes it as a PDF, returning
+// where it landed. The context bounds the whole crawl; cancelling it abandons
+// the run without leaving a partial PDF behind.
+func Export(ctx context.Context, req Request) (*Result, error) {
+	req.applyDefaults()
+	if err := req.validate(); err != nil {
+		return nil, err
+	}
+
+	crawler := &hako.Crawler{
+		Client:   hako.NewClient(req.Delay, req.Retries),
+		CacheDir: req.CacheDir,
+		Workers:  req.Workers,
+		Log:      req.Log,
+	}
+
+	novel, err := crawler.Novel(ctx, req.NovelURL)
+	if err != nil {
+		return nil, err
+	}
+
+	if req.Limit > 0 && req.Limit < len(novel.Chapters) {
+		req.logf("limiting to first %d of %d chapters", req.Limit, len(novel.Chapters))
+		novel.Chapters = novel.Chapters[:req.Limit]
+	}
+
+	chapters, err := crawler.Chapters(ctx, novel)
+	if err != nil {
+		return nil, err
+	}
+
+	// Falls back to the bundled font, so a host with no fonts installed still
+	// renders; only an explicitly requested font can fail here.
+	font, err := pdfout.LoadFont(req.FontFile)
+	if err != nil {
+		return nil, err
+	}
+
+	outPath := req.OutPath
+	if outPath == "" {
+		outPath = filepath.Join(req.OutDir, SafeFileName(novel.Title, novel.Slug)+".pdf")
+	}
+
+	page := pdfout.Presets[req.Page]
+	opts := pdfout.Options{
+		Page:        page,
+		Margin:      req.Margin,
+		Font:        font,
+		FontSize:    req.FontSize,
+		LineSpacing: req.LineSpacing,
+		Title:       novel.Title,
+		SourceURL:   novel.URL,
+	}
+	if err := pdfout.Write(outPath, opts, toPDFChapters(chapters)); err != nil {
+		return nil, err
+	}
+
+	if req.TextDir != "" {
+		if err := writeText(req.TextDir, chapters); err != nil {
+			return nil, err
+		}
+	}
+
+	return &Result{
+		Path:      outPath,
+		TextDir:   req.TextDir,
+		Title:     novel.Title,
+		SourceURL: novel.URL,
+		Chapters:  len(chapters),
+		Words:     hako.TotalWords(chapters),
+		FontName:  font.Name,
+		Page:      page,
+	}, nil
+}
+
+func (r *Request) applyDefaults() {
+	if r.Page == "" {
+		r.Page = DefaultPage
+	}
+	if r.FontSize == 0 {
+		r.FontSize = DefaultFontSize
+	}
+	if r.LineSpacing == 0 {
+		r.LineSpacing = DefaultLineSpacing
+	}
+	if r.Margin == 0 {
+		r.Margin = DefaultMargin
+	}
+	if r.Workers == 0 {
+		r.Workers = DefaultWorkers
+	}
+	switch {
+	case r.NoDelay:
+		r.Delay = 0
+	case r.Delay == 0:
+		r.Delay = DefaultDelay
+	}
+	if r.Retries == 0 {
+		r.Retries = DefaultRetries
+	}
+	switch {
+	case r.NoCache:
+		r.CacheDir = ""
+	case r.CacheDir == "":
+		r.CacheDir = DefaultCacheDir
+	}
+}
+
+// validate rejects a Request before any request is made, so a typo costs no
+// fetches. Call applyDefaults first: it checks the effective values.
+func (r *Request) validate() error {
+	if r.NovelURL == "" {
+		return fmt.Errorf("novel url is required")
+	}
+	parsed, err := url.Parse(r.NovelURL)
+	if err != nil {
+		return fmt.Errorf("invalid novel url: %w", err)
+	}
+	if parsed.Scheme != "http" && parsed.Scheme != "https" {
+		return fmt.Errorf("invalid novel url: want an http(s) URL, got %q", r.NovelURL)
+	}
+	if _, ok := pdfout.Presets[r.Page]; !ok {
+		return fmt.Errorf("unknown page %q: want one of %s",
+			r.Page, strings.Join(pdfout.PresetNames(), ", "))
+	}
+	if r.FontSize <= 0 {
+		return fmt.Errorf("font size must be positive")
+	}
+	if r.LineSpacing <= 0 {
+		return fmt.Errorf("line spacing must be positive")
+	}
+	if r.Margin < 0 {
+		return fmt.Errorf("margin cannot be negative")
+	}
+	if r.Workers < 1 {
+		return fmt.Errorf("workers must be at least 1")
+	}
+	return nil
+}
+
+func (r *Request) logf(format string, args ...any) {
+	if r.Log != nil {
+		r.Log(format, args...)
+	}
+}
+
+func toPDFChapters(chapters []*hako.Chapter) []pdfout.Chapter {
+	out := make([]pdfout.Chapter, 0, len(chapters))
+	for _, ch := range chapters {
+		out = append(out, pdfout.Chapter{
+			Heading:    ch.Heading(),
+			Volume:     ch.Volume,
+			Paragraphs: ch.Paragraphs,
+		})
+	}
+	return out
+}
+
+// writeText saves one file per chapter, numbered so a directory listing stays
+// in reading order rather than in the site's own chapter naming.
+func writeText(dir string, chapters []*hako.Chapter) error {
+	if err := os.MkdirAll(dir, 0o755); err != nil {
+		return fmt.Errorf("create text dir: %w", err)
+	}
+	for i, ch := range chapters {
+		name := fmt.Sprintf("%04d-%s.txt", i+1, SafeFileName(ch.Heading(), "chuong"))
+		body := ch.Heading() + "\n\n" + strings.Join(ch.Paragraphs, "\n\n") + "\n"
+		if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o644); err != nil {
+			return fmt.Errorf("write %s: %w", name, err)
+		}
+	}
+	return nil
+}
+
+var unsafeNameChars = regexp.MustCompile(`[^\p{L}\p{N}]+`)
+
+// SafeFileName builds a file name from a title, falling back when the title has
+// no usable characters. The result has no extension.
+func SafeFileName(title, fallback string) string {
+	name := strings.Trim(unsafeNameChars.ReplaceAllString(title, "-"), "-")
+	if name == "" {
+		return fallback
+	}
+	return name
+}

hako-crawler/export/export_test.go

@@ -0,0 +1,66 @@
+package export
+
+import (
+	"testing"
+	"time"
+)
+
+func TestSafeFileName(t *testing.T) {
+	for _, tc := range []struct{ title, fallback, want string }{
+		{"Anh Trai Nhân Vật Chính", "8476-x", "Anh-Trai-Nhân-Vật-Chính"},
+		{"Tập 1: Mở Đầu / Kết", "x", "Tập-1-Mở-Đầu-Kết"},
+		{"???", "8476-fallback", "8476-fallback"},
+		{"", "8476-fallback", "8476-fallback"},
+	} {
+		if got := SafeFileName(tc.title, tc.fallback); got != tc.want {
+			t.Errorf("SafeFileName(%q) = %q, want %q", tc.title, got, tc.want)
+		}
+	}
+}
+
+// Zero means "unset" for the tunables, so asking for no cache and no delay has
+// to be said explicitly — otherwise the defaults come back.
+func TestApplyDefaults(t *testing.T) {
+	req := Request{NovelURL: "https://ln.hako.vn/sang-tac/1-x"}
+	req.applyDefaults()
+
+	if req.Page != DefaultPage || req.FontSize != DefaultFontSize || req.Workers != DefaultWorkers {
+		t.Errorf("defaults not applied: %+v", req)
+	}
+	if req.CacheDir != DefaultCacheDir || req.Delay != DefaultDelay {
+		t.Errorf("cache/delay defaults not applied: %q %v", req.CacheDir, req.Delay)
+	}
+
+	opted := Request{NovelURL: "https://ln.hako.vn/sang-tac/1-x", NoCache: true, NoDelay: true}
+	opted.applyDefaults()
+	if opted.CacheDir != "" {
+		t.Errorf("NoCache should clear the cache dir, got %q", opted.CacheDir)
+	}
+	if opted.Delay != 0 {
+		t.Errorf("NoDelay should clear the delay, got %v", opted.Delay)
+	}
+}
+
+// validate runs before any request is made, so a bad flag costs no fetches.
+func TestValidate(t *testing.T) {
+	valid := "https://ln.hako.vn/sang-tac/1-x"
+	for name, req := range map[string]Request{
+		"no url":            {},
+		"not http":          {NovelURL: "ftp://ln.hako.vn/x"},
+		"unknown page":      {NovelURL: valid, Page: "billboard"},
+		"zero workers":      {NovelURL: valid, Workers: -1},
+		"negative margin":   {NovelURL: valid, Margin: -1},
+		"negative fontsize": {NovelURL: valid, FontSize: -1},
+	} {
+		req.applyDefaults()
+		if err := req.validate(); err == nil {
+			t.Errorf("%s: expected an error, got none", name)
+		}
+	}
+
+	ok := Request{NovelURL: valid, Delay: time.Second}
+	ok.applyDefaults()
+	if err := ok.validate(); err != nil {
+		t.Errorf("valid request rejected: %v", err)
+	}
+}

hako-crawler/pdfout/font.go

@@ -0,0 +1,109 @@
+package pdfout
+
+import (
+	_ "embed"
+	"fmt"
+	"os"
+	"path/filepath"
+	"runtime"
+)
+
+// bundledTTF is the font used when nothing else is available, so rendering never
+// depends on the host having fonts installed — a minimal container, and the
+// headless box this tool is usually run on, typically have none. See
+// fonts/NOTICE.md for provenance and licensing.
+//
+//go:embed fonts/DejaVuSans.ttf
+var bundledTTF []byte
+
+// BundledFontName labels the embedded font in diagnostics. It is not a path;
+// the font is compiled into the binary.
+const BundledFontName = "DejaVu Sans (bundled)"
+
+// Font is font data ready to embed in a PDF.
+//
+// The data is carried as bytes rather than as a path because fpdf joins a font
+// path onto its own font directory, which defaults to "." — turning an absolute
+// path into a working-directory-relative one that resolves only when the
+// process happens to run from the filesystem root.
+type Font struct {
+	// Name identifies the font for diagnostics: the path it was read from, or
+	// BundledFontName.
+	Name string
+	Data []byte
+}
+
+// Vietnamese text needs the Latin Extended Additional block (ư, ạ, ế, ộ …).
+// Every font listed here ships with its platform and covers it; a basic-Latin
+// font would silently drop the diacritics.
+func fontCandidates() []string {
+	switch runtime.GOOS {
+	case "windows":
+		dir := filepath.Join(os.Getenv("SystemRoot"), "Fonts")
+		if os.Getenv("SystemRoot") == "" {
+			dir = `C:\Windows\Fonts`
+		}
+		return []string{
+			filepath.Join(dir, "segoeui.ttf"),
+			filepath.Join(dir, "arial.ttf"),
+			filepath.Join(dir, "calibri.ttf"),
+			filepath.Join(dir, "tahoma.ttf"),
+			filepath.Join(dir, "times.ttf"),
+		}
+	case "darwin":
+		return []string{
+			"/System/Library/Fonts/Supplemental/Arial.ttf",
+			"/Library/Fonts/Arial.ttf",
+			"/System/Library/Fonts/Supplemental/Times New Roman.ttf",
+		}
+	default:
+		return []string{
+			"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
+			"/usr/share/fonts/truetype/noto/NotoSans-Regular.ttf",
+			"/usr/share/fonts/TTF/DejaVuSans.ttf",
+			"/usr/share/fonts/dejavu/DejaVuSans.ttf",
+			"/usr/share/fonts/liberation/LiberationSans-Regular.ttf",
+		}
+	}
+}
+
+// FindFont returns the first available system font suitable for Vietnamese.
+// Callers that just need something that works should use LoadFont, which falls
+// back to the bundled font instead of failing.
+func FindFont() (string, error) {
+	candidates := fontCandidates()
+	for _, path := range candidates {
+		if info, err := os.Stat(path); err == nil && !info.IsDir() {
+			return path, nil
+		}
+	}
+	return "", fmt.Errorf("no Vietnamese-capable system font found (looked for %v)", candidates)
+}
+
+// LoadFont resolves the font to embed.
+//
+// An explicit path wins, and is a hard error when it cannot be read: a caller
+// that named a font wants that font, not a silent substitute. Otherwise a
+// system font is used, and when none can be read the bundled font is returned —
+// so with an empty path LoadFont always succeeds.
+func LoadFont(path string) (Font, error) {
+	if path != "" {
+		data, err := os.ReadFile(path)
+		if err != nil {
+			return Font{}, fmt.Errorf("read font: %w", err)
+		}
+		return Font{Name: path, Data: data}, nil
+	}
+	if found, err := FindFont(); err == nil {
+		if data, err := os.ReadFile(found); err == nil {
+			return Font{Name: found, Data: data}, nil
+		}
+		// A listed font that cannot be read is no better than a missing one.
+	}
+	return BundledFont(), nil
+}
+
+// BundledFont returns the font compiled into the binary.
+func BundledFont() Font {
+	return Font{Name: BundledFontName, Data: bundledTTF}
+}

hako-crawler/pdfout/font_test.go

@@ -0,0 +1,51 @@
+package pdfout
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+// With no path given, LoadFont must always succeed: it falls back to the
+// bundled font so rendering never depends on the host having fonts installed.
+func TestLoadFontFallsBackToBundled(t *testing.T) {
+	font, err := LoadFont("")
+	if err != nil {
+		t.Fatalf("LoadFont(\"\"): %v", err)
+	}
+	if len(font.Data) == 0 {
+		t.Error("font carries no data")
+	}
+	if font.Name == "" {
+		t.Error("font has no name for diagnostics")
+	}
+}
+
+// A named font that cannot be read is an error, not a silent substitution: a
+// caller who asked for a font wants that font.
+func TestLoadFontMissingPathIsAnError(t *testing.T) {
+	if _, err := LoadFont(filepath.Join(t.TempDir(), "absent.ttf")); err == nil {
+		t.Fatal("expected an error for an unreadable font path, got none")
+	}
+}
+
+func TestLoadFontReadsGivenPath(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "given.ttf")
+	if err := os.WriteFile(path, BundledFont().Data, 0o644); err != nil {
+		t.Fatalf("write fixture: %v", err)
+	}
+	font, err := LoadFont(path)
+	if err != nil {
+		t.Fatalf("LoadFont(%q): %v", path, err)
+	}
+	if font.Name != path {
+		t.Errorf("name = %q, want %q", font.Name, path)
+	}
+}
+
+func TestBundledFontIsEmbedded(t *testing.T) {
+	if len(BundledFont().Data) < 100_000 {
+		t.Errorf("bundled font is %d bytes, which is too small to be a real TTF",
+			len(BundledFont().Data))
+	}
+}

hako-crawler/pdfout/fonts/NOTICE.md

@@ -0,0 +1,24 @@
+# Bundled font
+
+`DejaVuSans.ttf` is embedded into the binary and used when no font is supplied
+and no suitable system font is found. It covers the Latin Extended Additional
+block, which is what Vietnamese diacritics need — a font with only basic Latin
+coverage silently drops them.
+
+The following is recorded in the font file's own name table:
+
+- Version: `Version 2.37`
+- Copyright: `Copyright (c) 2003 by Bitstream, Inc. All Rights Reserved.`
+  `Copyright (c) 2006 by Tavmjong Bah. All Rights Reserved.`
+  `DejaVu changes are in public domain`
+- License information: <http://dejavu.sourceforge.net/wiki/index.php/License>
+
+The DejaVu fonts are free and redistributable, which is why they ship with most
+Linux distributions. This copy came from Alpine's `font-dejavu` package, which
+does not include the license text as a separate file. For a vendored copy of the
+full license text, take `LICENSE` from the upstream DejaVu release and add it to
+this directory.
+
+To swap the bundled font, replace `DejaVuSans.ttf` and update this file. Verify
+the replacement covers Vietnamese first — `TestBundledFontCoversVietnamese` in
+`../font_test.go` checks a representative set of characters.

hako-crawler/pdfout/pdf.go

@@ -0,0 +1,163 @@
+package pdfout
+
+import (
+	"fmt"
+	"sort"
+	"strings"
+
+	"github.com/go-pdf/fpdf"
+)
+
+// pointsToMM converts typographic points to millimetres.
+const pointsToMM = 25.4 / 72.0
+
+// bodyFont is the internal family name registered with the PDF.
+const bodyFont = "body"
+
+// PageSize is a page in millimetres.
+type PageSize struct {
+	Name string
+	W, H float64
+}
+
+// Presets are the selectable page geometries.
+//
+// Phone reading depends far more on page shape than on font size. A viewer
+// scales a whole page to fit the screen, so a large font on an A4 page still
+// ends up tiny: the page is ~3x wider than the screen and gets shrunk to match.
+// A page cut to the phone's own aspect ratio fills the screen at 100%, which is
+// why "phone" is a small 9:16 page rather than A4 with big type.
+var Presets = map[string]PageSize{
+	"phone": {"phone", 90, 160},
+	"a5":    {"a5", 148, 210},
+	"a4":    {"a4", 210, 297},
+}
+
+// PresetNames lists preset keys in a stable order for help text.
+func PresetNames() []string {
+	names := make([]string, 0, len(Presets))
+	for name := range Presets {
+		names = append(names, name)
+	}
+	sort.Strings(names)
+	return names
+}
+
+// Chapter is a chapter ready to render.
+type Chapter struct {
+	Heading string
+
+	// Volume labels the section the chapter belongs to. It is printed above
+	// the heading only when it differs from the previous chapter's, which is
+	// what keeps a volume-structured novel navigable without a separate page
+	// per volume.
+	Volume     string
+	Paragraphs []string
+}
+
+// Options controls the exported PDF.
+type Options struct {
+	Page        PageSize
+	Margin      float64 // mm
+	Font        Font    // resolve with LoadFont
+	FontSize    float64 // pt
+	LineSpacing float64 // multiple of font size
+	Title       string
+	SourceURL   string
+}
+
+// footerReserve is the vertical space kept clear for the page number.
+const footerReserve = 6.0
+
+// Write renders the chapters to a PDF at path.
+func Write(path string, opts Options, chapters []Chapter) error {
+	pdf := fpdf.NewCustom(&fpdf.InitType{
+		UnitStr: "mm",
+		Size:    fpdf.SizeType{Wd: opts.Page.W, Ht: opts.Page.H},
+	})
+
+	pdf.SetMargins(opts.Margin, opts.Margin, opts.Margin)
+	pdf.SetAutoPageBreak(true, opts.Margin+footerReserve)
+
+	// Embeds a subset of the TrueType data, which is what makes the Vietnamese
+	// diacritics render instead of falling back to "?". The bytes are passed
+	// directly rather than by path: the path-taking variant joins the name onto
+	// fpdf's own font directory (default "."), which mangles an absolute path
+	// into a working-directory-relative one.
+	pdf.AddUTF8FontFromBytes(bodyFont, "", opts.Font.Data)
+	pdf.SetFont(bodyFont, "", opts.FontSize)
+	pdf.SetTitle(opts.Title, true)
+
+	lineHeight := opts.FontSize * opts.LineSpacing * pointsToMM
+	paragraphGap := lineHeight * 0.45
+	headingSize := opts.FontSize * 1.35
+
+	addFooter(pdf, opts)
+	writeTitlePage(pdf, opts, len(chapters))
+
+	volume := ""
+	for _, ch := range chapters {
+		pdf.AddPage()
+
+		if ch.Volume != "" && ch.Volume != volume {
+			pdf.SetFontSize(opts.FontSize * 0.8)
+			pdf.SetTextColor(120, 120, 120)
+			pdf.MultiCell(0, opts.FontSize*1.2*pointsToMM, strings.ToUpper(ch.Volume), "", "L", false)
+			pdf.SetTextColor(0, 0, 0)
+			pdf.Ln(paragraphGap)
+		}
+		volume = ch.Volume
+
+		pdf.SetFontSize(headingSize)
+		pdf.MultiCell(0, headingSize*1.3*pointsToMM, ch.Heading, "", "L", false)
+		pdf.Ln(paragraphGap * 1.6)
+
+		pdf.SetFontSize(opts.FontSize)
+		for _, p := range ch.Paragraphs {
+			// "J" justifies, which keeps the short measure of a phone page tidy.
+			pdf.MultiCell(0, lineHeight, p, "", "J", false)
+			pdf.Ln(paragraphGap)
+		}
+	}
+
+	if err := pdf.OutputFileAndClose(path); err != nil {
+		return fmt.Errorf("write pdf %s: %w", path, err)
+	}
+	return nil
+}
+
+// addFooter prints a centred page number, restoring the body font size so the
+// footer callback cannot leak its own size into the following content.
+func addFooter(pdf *fpdf.Fpdf, opts Options) {
+	pdf.SetFooterFunc(func() {
+		if pdf.PageNo() <= 1 {
+			return
+		}
+		pdf.SetY(-(opts.Margin + footerReserve*0.6))
+		pdf.SetFontSize(opts.FontSize * 0.75)
+		pdf.SetTextColor(120, 120, 120)
+		pdf.CellFormat(0, 4, fmt.Sprintf("%d", pdf.PageNo()-1), "", 0, "C", false, 0, "")
+		pdf.SetTextColor(0, 0, 0)
+		pdf.SetFontSize(opts.FontSize)
+	})
+}
+
+func writeTitlePage(pdf *fpdf.Fpdf, opts Options, chapterCount int) {
+	pdf.AddPage()
+	pdf.SetY(opts.Page.H * 0.30)
+
+	titleSize := opts.FontSize * 1.9
+	pdf.SetFontSize(titleSize)
+	pdf.MultiCell(0, titleSize*1.35*pointsToMM, strings.ToUpper(opts.Title), "", "C", false)
+
+	pdf.Ln(opts.FontSize * pointsToMM * 2)
+	pdf.SetFontSize(opts.FontSize * 0.85)
+	pdf.SetTextColor(90, 90, 90)
+	pdf.MultiCell(0, opts.FontSize*1.4*pointsToMM,
+		fmt.Sprintf("%d chương", chapterCount), "", "C", false)
+	if opts.SourceURL != "" {
+		pdf.MultiCell(0, opts.FontSize*1.4*pointsToMM, opts.SourceURL, "", "C", false)
+	}
+	pdf.SetTextColor(0, 0, 0)
+	pdf.SetFontSize(opts.FontSize)
+}

hako-crawler/pdfout/pdf_test.go

@@ -0,0 +1,55 @@
+package pdfout
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+// Write must produce a real PDF with the bundled font, since the host this runs
+// on has no fonts installed.
+func TestWrite(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "book.pdf")
+
+	opts := Options{
+		Page:        Presets["phone"],
+		Margin:      6,
+		Font:        BundledFont(),
+		FontSize:    10,
+		LineSpacing: 1.55,
+		Title:       "Truyện Thử",
+		SourceURL:   "https://ln.hako.vn/sang-tac/1-x",
+	}
+	chapters := []Chapter{
+		{Heading: "Chương 01", Volume: "Tập 01", Paragraphs: []string{"Trời hôm nay đẹp lắm."}},
+		{Heading: "Chương 02", Volume: "Tập 01", Paragraphs: []string{"Người anh trai mỉm cười."}},
+		{Heading: "Minh Họa 01", Volume: "Minh Họa", Paragraphs: nil},
+	}
+	if err := Write(path, opts, chapters); err != nil {
+		t.Fatalf("write: %v", err)
+	}
+
+	data, err := os.ReadFile(path)
+	if err != nil {
+		t.Fatalf("read back: %v", err)
+	}
+	if len(data) < 1024 {
+		t.Errorf("pdf is %d bytes, which is too small to hold an embedded font", len(data))
+	}
+	if string(data[:5]) != "%PDF-" {
+		t.Errorf("missing PDF header, got %q", data[:5])
+	}
+}
+
+func TestPresetNamesAreStable(t *testing.T) {
+	want := []string{"a4", "a5", "phone"}
+	got := PresetNames()
+	if len(got) != len(want) {
+		t.Fatalf("got %v, want %v", got, want)
+	}
+	for i := range want {
+		if got[i] != want[i] {
+			t.Fatalf("got %v, want %v", got, want)
+		}
+	}
+}

hako-crawler/plans/260824-1946-hako-crawler-rename-and-pdf/README.md

@@ -0,0 +1,76 @@
+# hako-crawler: generalise + PDF export
+
+Two phases. Phase 1 turns a single-novel script into a stable, minimal
+`hako-crawler`; phase 2 adds PDF export modelled on `monkeyd-crawler`.
+
+## Outcome
+
+One command downloads any ln.hako.vn novel and writes a phone-readable PDF.
+ATNVC ("Anh Trai Nhân Vật Chính") becomes the documented example rather than a
+hardcoded URL.
+
+## Constraints
+
+- Go, no browser (headless ARM64 host, no fonts installed -> font must be embedded).
+- Keep goquery: selector code is far shorter than hand-walking `html.Node`,
+  and it already depends on `golang.org/x/net`.
+- Public repo under `tiennm99`.
+
+## Non-goals
+
+- Volume-aware PDF sectioning beyond a per-chapter heading.
+- Resuming a partial PDF, or any GUI.
+
+## Site findings (verified 2026-08-24, live fetch)
+
+1. Chapter bodies are no longer plain markup. `#chapter-content` holds
+   `<div id="chapter-c-protected" data-s="xor_shuffle" data-k="<16 ascii>"
+   data-c="<json array>">`, decoded in-browser by `/scripts/app.js`.
+   **The pre-existing `p[id=digits]` extractor therefore returns nothing** —
+   the old crawler is already broken against the live site.
+2. Decode: sort chunks by the decimal in their first 4 chars, strip that
+   prefix, base64-decode, XOR each byte with the key's ASCII bytes cycling —
+   the key offset restarts per chunk, not across the joined stream.
+3. The decoded payload is the original chapter markup and contains only
+   `p`/`em`/`strong`/`img`, every `p` carrying a numeric id. So extraction
+   after decoding is a plain `p` walk over content that has no site chrome in it.
+4. Landing page: title `.series-name a`, tags `a.series-gerne-item` (site's own
+   spelling), chapters `div.chapter-name > a` grouped by `.volume-list`
+   sections. Document order is already reading order — unlike monkeydd, no
+   reversal is needed.
+5. Illustration chapters are real chapters with captions but little prose, so
+   an empty-ish chapter must not fail the run.
+
+## Phase 1 — generalise and stabilise
+
+| Step | Detail |
+|---|---|
+| Module | `github.com/tiennm99/hako-crawler` |
+| Layout | `cmd/hako-crawler` CLI, `hako` fetch+parse, `export` pipeline, `pdfout` render |
+| Client | one rate-limited, retrying fetcher; 429/5xx retried with backoff, 404/403 fail fast |
+| Cache | raw pages under `.cache/`, so re-rendering costs no requests |
+| Decode | `hako/protected.go`, with the plain-markup path kept as fallback |
+| Validation | `go test ./...` on synthetic fixtures; no network in tests |
+
+## Phase 2 — PDF export
+
+| Step | Detail |
+|---|---|
+| Render | `pdfout`: 90x160mm phone page default, a5/a4 presets, justified body |
+| Font | embedded DejaVu Sans (host has no fonts); explicit `-font` wins and is a hard error |
+| Pipeline | `export.Export(ctx, Request)` — list, fetch, font, render |
+| Keep | opt-in `-txt <dir>` preserves the original per-chapter text output |
+
+## Acceptance criteria
+
+- `hako-crawler -url <atnvc> -limit 3` writes a PDF with real Vietnamese
+  diacritics and non-empty chapters.
+- `go test ./...` green, offline.
+- `go vet ./...` clean.
+- Repo and local folder both named `hako-crawler`; module path matches.
+
+## Risk / rollback
+
+Decoding is tied to a site scheme that can rotate; `data-s` is read from the
+page and an unknown scheme fails with a clear message rather than emitting
+garbage. Rollback is `git revert` — the rename is a `gh repo rename` back.