docs(plans): record the thuyvan flood alert research, plan and journal

plans/261002-1205-thuyvan-flood-alert/plan.md

@@ -0,0 +1,50 @@
+# Plan: `/thuyvan` flood alerts for Tân Thuận
+
+Status: implemented, not yet committed (2026-10-02)
+Design: [research report](../reports/research-261002-1205-thuyvan-flood-alert.md)
+
+## Outcome
+
+`/thuyvan` (no parameter) shows Tân Thuận's flood risk: a 5-day tide-peak
+forecast at Phú An and Nhà Bè from the KTTV Nam Bộ bulletin, the rain forecast
+from Open-Meteo, and live VNDMS levels at stations within 30 km. Chats that opt
+in with `/thuyvan_subscribe` get a 10:30 ICT push, but only when some day has a
+tide peak of at least 1.40 m (BĐ I) or at least 50 mm of rain.
+
+## Constraints and non-goals
+
+- The `thoitiet` module is renamed to `weather`. The command names stay the
+  same.
+- Subscriber storage and push fan-out move out of `lol` into a shared helper,
+  and `lol` behaviour stays the same.
+- Non-goals: other locations, tide modelling, or an evening push.
+
+## Phases
+
+1. [x] Rename `internal/modules/thoitiet` to `internal/modules/weather`
+   (package, catalog key, README).
+2. [x] Extract `internal/modules/util/subscription`: the subscriber list, the
+   terminal-error classifier, the per-day claim and fan-out with pruning.
+   Move `lol` onto it.
+3. [x] Tide bulletin: homepage → article → PDF → peak rows, with a strict
+   validator and a real PDF fixture.
+4. [x] VNDMS: parse the popups, tag alarm tiers, keep stations within 30 km of
+   Tân Thuận, using a trimmed fixture.
+5. [x] `/thuyvan`, the subscribe and unsubscribe commands, and the 10:30 ICT
+   cron. Wire up the menu test and the README.
+6. [x] Run `go test ./...`, `go vet ./...` and `golangci-lint run`, then
+   review.
+
+## Acceptance
+
+- `/thuyvan` renders today's bulletin (fixture) with the BĐ tiers. When a
+  source fails, its section is replaced by a note.
+- The push is sent only on risk days, once per ICT day, and prunes dead chats.
+- The `lol` tests pass unchanged in behaviour.
+
+## Risk
+
+- The bulletin PDF layout could drift. The strict row validator turns that
+  into a missing-bulletin note.
+- `MODULES=thoitiet` fails startup after the rename. Check Coolify before
+  pushing.

plans/journals/2026-10-02-thuyvan-flood-alerts-for-tan-thuan.md

@@ -0,0 +1,38 @@
+---
+title: thuyvan flood alerts for Tan Thuan
+date: 2026-10-02
+summary: Added /thuyvan tide/rain/gauge flood view and risk-only push; renamed thoitiet to weather; shared subscription helper
+---
+
+# thuyvan flood alerts for Tan Thuan
+
+## What happened
+
+Added `/thuyvan` to the weather module, which was renamed from `thoitiet`. It
+shows Tân Thuận's flood risk from three sources: the KTTV Nam Bộ 5-day tide
+bulletin PDF (Phú An and Nhà Bè peaks), the Open-Meteo rain forecast, and VNDMS
+river gauges within 30 km. `/thuyvan_subscribe` adds a 10:30 ICT push, sent
+only when a peak reaches báo động I (1.40 m) or rain reaches 50 mm, with a
+12:30 retry. The `lol` subscriber, claim and fan-out code moved to
+`internal/modules/util/subscription`.
+
+## Lessons
+
+- The bulletin PDF's text layer scrambles labels ("P n A ú h"), but each
+  forecast row keeps its values in order after the date token. Keying on the
+  date token instead of X positions handles the swapped-date rows.
+- In some bulletins the station label sits on its own text row, 1 pt from the
+  middle forecast row. All 11 real bulletins from April to October 2026 parse
+  after allowing for that.
+- VNDMS returns 403 without a same-site Referer, and its old `dmc.gov.vn`
+  domains now serve a "domain moved" page.
+- The review caught that a missing or stale bulletin silently turned into "no
+  risk". It now errors and is retried at 12:30.
+
+## Next steps
+
+- Check the Coolify `MODULES` value before pushing: if it lists `thoitiet`,
+  the bot fails at startup after the rename.
+- Commit and push once that check is done.
+
+> Historical work record — not durable authority. Prefer docs/specs/ADRs for current decisions.

plans/reports/research-261002-1205-thuyvan-flood-alert.md

@@ -0,0 +1,217 @@
+# Research + Brainstorm: `/thuyvan` flood alerts for Tân Thuận
+
+Conducted 2026-10-02. Scope: flood (ngập lụt) alerts for Tân Thuận, Quận 7,
+TP.HCM, through a `/thuyvan` command and an opt-in daily push. Both go into the
+current weather module, which is renamed from `thoitiet` to `weather`.
+
+## Recommendation
+
+Flooding in Tân Thuận is driven mostly by high tides (triều cường) on the Sài
+Gòn and Đồng Điền rivers, and sometimes by heavy rain. The nearest gauges are
+Phú An (2.9 km) and Nhà Bè (6.8 km). Both have official alarm levels (báo động,
+BĐ) of BĐ I 1.40 m, BĐ II 1.50 m and BĐ III 1.60 m. The design uses three
+sources:
+
+1. **Tide forecast:** the official 5-day tide bulletin from Đài KTTV Nam Bộ, a
+   daily PDF with the peak forecast at Phú An and Nhà Bè. This is the alert
+   trigger.
+2. **Rain forecast:** the Open-Meteo daily `precipitation_sum` at Tân Thuận,
+   the client the module already has. This is the second trigger.
+3. **Live levels:** VNDMS readings for the stations near Tân Thuận, with their
+   alarm tier. This is context, and the fallback when the bulletin is missing.
+
+An alert fires when a forecast tide peak at Phú An or Nhà Bè is at least
+**1.40 m** (BĐ I), or when the day's rain forecast is at least **50 mm**.
+
+## Decisions (user, 2026-10-02)
+
+- Goal: flood (ngập lụt) alerts at Tân Thuận, not a general water-level tool.
+- Delivery: a `/thuyvan` command plus an opt-in daily push
+  (`/thuyvan_subscribe`, `/thuyvan_unsubscribe`).
+- Trigger: a tide peak at or above BĐ I, OR rain of 50 mm or more.
+- `/thuyvan` takes **no parameter**. It always shows Tân Thuận: the flood
+  forecast plus the live levels at nearby stations.
+- Placement: inside the weather module, renamed `thoitiet` → `weather`.
+- Rejected: Open-Meteo tide modelling and a `[location...]` argument.
+
+## Sources (all verified live on 2026-10-02)
+
+### Tide bulletin: KTTV Nam Bộ
+
+- **Discovery.** The `https://kttvnb.vn/` homepage links today's article, for
+  example
+  `/index.php/100-thong-tin-kttv/thuy-van/29194-ba-n-tin-da-ba-o-tha-y-v-n-tphcm-ra-nga-y-02-10-2026`.
+  The slug varies (one used `khu-va-c-tphcm`), so match
+  `(\d+)-[^"]*tphcm-ra-nga-y-(\d{2})-(\d{2})-(\d{4})` and take the newest date.
+  The `thuy-van` category page is not sorted by date, so do not use it.
+- **PDF.** The article page links
+  `https://kttvnb.vn/attachments/article/<id>/HCMC_TVHN_YYYYMMDD.pdf` (318 KB,
+  issued at about 09:24 ICT). The PDF path needs the article ID, so it cannot
+  be guessed from the date. Plain `http://kttv-nb.org.vn` also serves the files,
+  but its TLS certificate is broken, so use `https://kttvnb.vn`.
+- **Text extraction** with `github.com/ledongthuc/pdf` (`GetTextByRow`, pure
+  Go) works. The labels come out with odd letter spacing (`P h ú  An`), but the
+  forecast number rows are clean and come in a fixed order: 5 rows for Phú An,
+  5 for Nhà Bè, then 5 for Thủ Dầu Một.
+
+  ```text
+  02/ 10 1.33 07.00 1.29 21.00 - 1.79 15.00 - 0.04 01.00
+  ```
+
+  Each row is the date, peak 1 (height and time), peak 2, trough 1 and
+  trough 2. `ct` means no second peak that day. Minus signs may be split from
+  the number (`- 1.79`) or appear after it (`1.78 -`). Only the peaks are
+  needed, and they are always positive, so the parser can drop the trough
+  columns.
+- The same PDF prints the BĐ I/II/III thresholds (1.40/1.50/1.60 m), and VNDMS
+  `detailRain` confirms them for both stations.
+- Today's forecast is a peak of 1.33 m at Phú An on 02/10, falling to 0.68 m by
+  06/10, so no alert. The 28–30/09 peaks reached 1.56–1.60 m, BĐ II–III.
+
+### Live levels: VNDMS
+
+- `GET https://vndms.gov.vn/water_level?lv=0` needs a
+  `Referer: https://vndms.gov.vn/` header, or it returns 403. It gives all 458
+  stations as GeoJSON, with the fields inside `popupInfo` HTML (name, code,
+  province, river, and `Mực nước (1.33(m) 7-02/10)` or `Không có số liệu`).
+  `lv=1..3` returns only the stations at that alarm tier.
+- The response is 32 KB gzipped (432 KB decoded) and takes 0.2–1.2 s.
+- Reporting stations within 30 km of Tân Thuận today: Phú An 2.9 km, Nhà Bè
+  6.8 km, Biên Hòa 25.0 km, Thủ Dầu Một 26.1 km and Bến Lức 29.1 km.
+
+### Rain: Open-Meteo
+
+The module's existing `fetchForecast` already returns daily `precipitation_sum`
+and `precipitation_probability_max`. Call it with a fixed Tân Thuận place.
+
+### Tân Thuận point
+
+Open-Meteo geocoding gives `Tân Thuận`, Quận Bảy, at
+`10.74111, 106.71806`. Store it as a constant, as `hcmPlace` is stored, because
+geocoding "Tan Thuan" by name returns 14 Vietnamese places.
+
+## Design
+
+```text
+/thuyvan ─┬─ tide bulletin (homepage → article → PDF → peak rows)
+          ├─ Open-Meteo forecast @ Tân Thuận (rain, days 0..4)
+          └─ VNDMS lv=0..3 (stations ≤ 30 km, sorted by distance)
+          ─► merge into a per-day risk view ─► Reply
+
+cron 10:30 ICT ─► same fetch ─► any day at risk? ─► push to subscribers
+                                 no  ─► send nothing
+```
+
+- **Fetching.** The three sources run concurrently under one fetch context.
+  Each one fails on its own: the reply omits a failed source with a short note
+  and never shows guessed numbers. If the bulletin is missing (for example
+  before 09:24) or fails to parse, the reply uses yesterday's bulletin when the
+  homepage still links it, labelled with its issue date, or else shows only
+  the live levels and the rain.
+- **Parsing safety.** A bulletin counts as valid only if it yields exactly 5
+  dated rows for each of Phú An and Nhà Bè, with peaks between 0 and 3 m.
+  Anything else is a parse failure, logged with the PDF URL. Tests use the
+  captured `HCMC_TVHN_20261002.pdf` as a fixture.
+- **Risk per day.** Take the highest of the two peaks at Phú An and Nhà Bè and
+  map it to a tier: below BĐ I, BĐ I, BĐ II or BĐ III. Rain of 50 mm or more
+  flags the day too. Rain forecasts beyond about 5 days are not reliable, so
+  only the bulletin's 5 days count.
+- **Push.** A cron at 10:30 ICT, after the bulletin is issued, sends one
+  message per subscribed (chat, topic) only when at least one of the next 5
+  days is at risk. It needs no dedupe state, because it fires once a day.
+  Subscribers are stored like `lol`'s subscriber list, including the pruning
+  of chats that blocked the bot. `lol/subscribers.go` and its pruning code
+  would then have a second user, so move them to a shared
+  `internal/modules/util` package instead of copying them.
+
+Sample `/thuyvan` reply:
+
+```text
+🌊 Thuỷ văn — Tân Thuận (Q.7)
+Dự báo đỉnh triều (Phú An / Nhà Bè):
+02/10: 1.33 / 1.34 m (07:00) — dưới BĐ I
+03/10: 1.18 / 1.19 m (08:00)
+04/10: 1.01 / 1.03 m (09:00)
+05/10: 0.84 / 0.88 m
+06/10: 1.13 / 0.70 m
+Mưa hôm nay: 5.3 mm (70%)
+Mực nước hiện tại:
+Phú An (Sài Gòn, 2.9 km): 1.33 m, 7h 02/10
+Nhà Bè (Đồng Điền, 6.8 km): 1.18 m, 7h 02/10
+⚠️ Thủ Dầu Một (Sài Gòn, 26.1 km): 1.56 m, 7h 02/10 — BĐ II
+BĐ I/II/III: 1.40 / 1.50 / 1.60 m
+Nguồn: Đài KTTV Nam Bộ, VNDMS, Open-Meteo
+```
+
+Sample push, sent only on risk days:
+
+```text
+⚠️ Cảnh báo ngập — Tân Thuận (Q.7)
+29/09: đỉnh triều 1.60 m lúc 17:00 tại Nhà Bè — BĐ III
+30/09: đỉnh triều 1.56 m — BĐ II
+Mưa 29/09: 62 mm
+Nguồn: Đài KTTV Nam Bộ, Open-Meteo
+```
+
+## Changes
+
+- Rename `internal/modules/thoitiet` to `internal/modules/weather`: the
+  package, its doc, the log `module` field, and the `cmd/server/main.go` import
+  and catalog key `"weather"`.
+- The module gains storage for its subscriber list, in the `weather`
+  collection, with no migration. Add the `CollectionName` constant the way the
+  other modules with storage do.
+- New files: `flood.go` (command, cron and risk), `tide_bulletin.go`
+  (discovery and PDF parsing), `water_level.go` (VNDMS), plus tests and
+  fixtures.
+- Add the dependency `github.com/ledongthuc/pdf`. It has no tagged releases,
+  so `go get @latest` records a pseudo-version.
+- Add three public commands: `thuyvan`, `thuyvan_subscribe` and
+  `thuyvan_unsubscribe`. Update `cmd/server/command_menu_test.go` and the
+  README module row.
+- **Breaking:** `modules.Build` rejects unknown module names, so if Coolify's
+  `MODULES` lists `thoitiet`, the bot **fails to start** after the rename.
+  Check that value before pushing. Commit the rename on its own as
+  `refactor(weather)!:`.
+
+## Risks
+
+| Risk | Mitigation |
+|---|---|
+| The bulletin layout changes and breaks the PDF row parsing | Strict row-count and range check, fallback to live levels, a logged URL, fixture tests |
+| The homepage drops the article link or changes the slug | Loose regex; a missing link means "no bulletin" and the fallback |
+| The VNDMS Referer requirement tightens, or the domain moves again (it moved in 2025–26) | URL var, a clean failure note in the reply |
+| 50 mm/day misses short cloudbursts (40 mm in an hour floods streets too) | Use the daily total for v1; the hourly maximum is a follow-up if alerts prove too quiet |
+| The bulletin's 5 days end before a long high-tide spell | Each day's push carries the next 5 days |
+
+## Changes after review (2026-10-02)
+
+- A bulletin issued more than a day before today counts as missing, because
+  a stalled homepage would otherwise produce a valid-looking bulletin whose
+  dates no longer cover the window.
+- If the bulletin is missing and the rain forecast shows no risk, the push
+  returns an error instead of reporting "no risk", since tide is the main
+  signal.
+- A 12:30 ICT retry cron reruns the push. The day is claimed only when an
+  alert goes out, so the retry sends nothing after a successful 10:30 push.
+- `/thuyvan` notes a failed rain forecast as well as a failed bulletin or
+  gauge list.
+- Known limit: `/thuyvan` is not cached, so each call makes 8 upstream
+  requests. That is acceptable at this bot's usage; add a short cache if VNDMS
+  starts rate-limiting.
+
+## Next steps
+
+1. Check the Coolify `MODULES` value for the rename.
+2. Write the plan under `plans/261002-1205-thuyvan-flood-alert/` with phases:
+   the rename, moving the subscriber code to a shared package, the parsers and
+   fixtures, then the command, cron and push.
+3. Run `go test ./internal/modules/... ./cmd/server/...`, `go vet ./...` and
+   `golangci-lint run`.
+
+## Unresolved questions
+
+- Push time 10:30 ICT: is that fine, or do you want a second, evening push
+  before the evening high tides?
+- Should the push go out only when there is risk (as designed), or every day
+  as a summary?