diff --git a/docs/command-parameter-conventions.md b/docs/command-parameter-conventions.md index 94cf149..61a79e9 100644 --- a/docs/command-parameter-conventions.md +++ b/docs/command-parameter-conventions.md @@ -67,6 +67,3 @@ When adding or changing a command: - [Microsoft command-line syntax key](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/command-line-syntax-key) - [Python argparse documentation](https://docs.python.org/3/library/argparse.html) - [docopt usage-pattern grammar](https://github.com/docopt/docopt/blob/master/README.rst) - -The supporting project-specific analysis is in -[`plans/reports/260721-1418-command-parameter-schema-research.md`](../plans/reports/260721-1418-command-parameter-schema-research.md). diff --git a/docs/journals/2026-08-05-lol-pandascore-migration.md b/docs/journals/2026-08-05-lol-pandascore-migration.md deleted file mode 100644 index c119c67..0000000 --- a/docs/journals/2026-08-05-lol-pandascore-migration.md +++ /dev/null @@ -1,32 +0,0 @@ -# Decade-Old Riot API Dies; Twice-Rewritten Transport in One Day - -**Date**: 2026-08-05 morning/afternoon -**Severity**: Critical -**Component**: lol module (Go Telegram bot, LoL esports schedules) -**Status**: Resolved - -## What Happened - -The bot's lol schedule fetch started failing HTTP 403. Investigation revealed Riot revoked the decade-old x-api-key for esports-api.lolesports.com. The new lolesports.com is a Next.js/Netlify site keeping its API key server-side behind /api/gql persisted-query proxies. Official replacement (GRID/formerly Bayes LDP) is paid-only. Two transports were implemented and shipped the same day. - -## The Brutal Truth - -This is incredibly frustrating because a clever same-day reverse-engineering win was immediately exposed as unmaintainable and abandoned. Commit b76d0ca reverse-engineered lolesports.com /api/gql by extracting the persisted-query ID manifest from webpack chunk 29—extracting sha256 hashes failed; the Apollo client uses `generatePersistedQueryIdsFromManifest`, and the manifest URL came from the webpack runtime's chunk-hash map. It worked. We verified it live on a headless ARM64 server without a browser. And then we threw it away. The second rewrite took a fraction of the effort because the ScheduleEvent boundary held—architecture won out over cleverness. - -## Technical Details - -**b76d0ca (reverse-engineering, abandoned)**: Persisted query approach worked on first live probe, pulling real homeEvents data from lolesports.com /api/gql proxy. - -**15f7e50 (PandaScore, shipped)**: REST transport to /lol/matches with Bearer token auth. Persisted ScheduleEvent contract and bson cache untouched. League slugs canonicalized via 12-entry mapping (e.g., `league-of-legends-lck-champions-korea` → `lck`). Results joined by team_id; outcome only set when winner_id present (preserves "score pending" semantics). Code review caught silent page-budget truncation; fixed with warn log and budget increase (3→5). Live probe rendered a real week with LIVE JDG 1–0 LGD—bracket-encoding verified correct. - -## Root Cause Analysis - -Riot's infrastructure changed, not our code. But the reverse-engineered approach depended on webpack chunk IDs and manifest URLs rotating with each frontend deploy—a hidden maintenance tax. The persisted-query ID schema is Riot's internal concern; every redeployment could break the mapping without warning. We should have recognized this immediately instead of celebrating the win. - -## Lessons Learned - -**Data-source sovereignty beats cleverness.** A PandaScore REST endpoint (versioned, documented, intentionally exposed) beats reverse-engineered gql IDs. **Stable boundaries matter.** The ScheduleEvent contract was unchanged between transport rewrites; that boundary made the second pass clean and fast. **Reverse-engineering is a temporary fix.** It solved a crisis, but architecture decides sustainability. - -## Next Steps - -None—migration complete and live-verified. Monitor PandaScore rate limits (1000 req/h free tier). Superseded pending Leaguepedia enrichment plan (260726-0952) since PandaScore results already carry scores. diff --git a/docs/journals/260722-1048-stock-portfolio-currency-title.md b/docs/journals/260722-1048-stock-portfolio-currency-title.md deleted file mode 100644 index eecfa65..0000000 --- a/docs/journals/260722-1048-stock-portfolio-currency-title.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -date: 2026-07-22 -component: internal/modules/stock -status: completed ---- - -# Stock Portfolio Currency Title - -## Context - -The stock portfolio repeated `VND` in every monetary table cell. - -## What Happened - -The renderer now declares `VND` once in the `Stock Portfolio (VND)` title, -labels the balance row `Cash`, and uses suffix-free monetary values in the -portfolio table. Standalone stock replies retain their explicit `VND` suffix. - -## Reflection - -A portfolio-specific display helper kept this presentation change from altering -the exported formatters used by trades, prices, and dividends. - -## Decisions - -- Keep command, storage, and exported formatting contracts unchanged. -- Cover the title, cash label, suffix-free values, and single currency marker in - the portfolio regression test. - -## Next - -No follow-up required. Focused and full tests, vet, and lint passed. diff --git a/docs/journals/260722-1119-compact-portfolio-number-plan.md b/docs/journals/260722-1119-compact-portfolio-number-plan.md deleted file mode 100644 index 9043843..0000000 --- a/docs/journals/260722-1119-compact-portfolio-number-plan.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -date: 2026-07-22 -component: stock-and-coin-portfolios -status: completed ---- - -# Compact Portfolio Number Plan - -## Context - -Full monetary values make the stock and coin Telegram portfolio position tables -wide and difficult to scan. - -## What Happened - -The compact format work is done. Automatic `k/M/B/T` suffixes now apply to -stock and coin position values with at most three trimmed fractional digits. -Stock keeps Vietnamese separators; coin keeps USD separators and `$`. Only the -position `Avg`, `Now`, `Value`, and `Unrealized P&L` fields were compacted. - -- [Brainstorm report](../../plans/reports/260722-1112-compact-portfolio-number-format.md) -- [Implementation plan](../../plans/260722-1114-compact-portfolio-numbers/plan.md) - -## Reflection - -Per-value scaling still beats fixed units per column or unit labels in headers. -It keeps the table readable without changing summaries or non-portfolio output. - -## Decisions - -- Compact only position `Avg`, `Now`, `Value`, and `Unrealized P&L` amounts. -- Keep percentages, summaries, `N/A`, and non-portfolio replies unchanged. -- Focused tests, full tests, `go vet`, and `golangci-lint` all passed. -- A pre-existing extreme-finite overflow in legacy full-value formatters was - observed separately and left as optional follow-up, not part of this change. - -## Next - -No further action required for this change set. diff --git a/docs/journals/260722-1614-stock-dividend-history.md b/docs/journals/260722-1614-stock-dividend-history.md deleted file mode 100644 index a4b11b0..0000000 --- a/docs/journals/260722-1614-stock-dividend-history.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: Stock Dividend History and Notification State -date: 2026-07-22 16:14 -component: stock -status: resolved ---- - -# Stock Dividend History and Notification State - -## Context - -We needed per-user dividend history that survives repeated `/stock_portfolio` calls without turning SSI’s flaky responses into duplicate credits or missing user notifications. The old shape was not enough: it had no stable per-event history, no clean way to separate notifications from processing state, and no safe way to distinguish fresh SSI data from already-accepted dividends. - -## What Happened - -We moved dividend history to `dividends..` so every SSI event has a stable per-user record. We intentionally did not add `dividendCheckedAt`; that timestamp would have encouraged the wrong mental model and still would not have solved idempotency. - -Record-date gating now decides whether the portfolio shows an `Apply` button. Events without a Record date stay informational while the bot keeps refreshing the original publication window for SSI updates. Historical refresh stays active until the event is processed or ages out at 90 days. - -Notifications now repeat on every `/stock_portfolio` response until the event is processed or expired, even if SSI temporarily omits the event. That was the right call because silence from SSI is not proof that the event disappeared. Multiple buttons for the same event are still expected; only the first successful click can mark the per-user record processed. - -We also added a startup migration to initialize the new history layout and preserve existing state. The migration is idempotent and guarded so it can run on every boot without redoing work. - -## Decisions - -We chose stable per-event history over recalculating from raw SSI responses on every request. That gave us deterministic notification state and idempotent processing. - -We accepted the legacy duplicate-credit risk for manual dividend commands versus SSI-backed buttons. That is ugly, but it is explicit: manual commands do not carry SSI event IDs, and trying to retrofit full cross-path deduplication would have been more invasive than the feature warranted. - -We also kept notifications visible until processed or 90 days old, instead of suppressing them after one failed SSI lookup. Suppression would have hidden real work from users and made the bot look broken. - -## Verification - -We verified the history shape and button flow with repeated portfolio refreshes, missing-date refreshes, and repeated clicks on the same event. The key checks were: - -- the same SSI event keeps the same `dividends..` record -- `Apply` only appears for Record-date events -- informational events continue to recheck their original publication window -- repeated portfolio commands keep showing unprocessed notifications -- a second button click does not credit the same event again -- the startup migration runs cleanly more than once - -## Risks/Next - -The remaining risk is mostly operational, not logical: SSI can still omit or reshuffle events, and the bot has to treat that as an external data problem rather than a local delete signal. - -The legacy duplicate-credit risk is still real for manual commands. We are accepting that for now because the alternative was a larger behavioral rewrite across old and new dividend paths. - -Next step is to keep watching for edge cases where SSI history and manual adjustments overlap in surprising ways. If that starts showing up in real usage, the processing model will need a dedicated reconciliation pass instead of more ad hoc checks. diff --git a/docs/journals/260722-1705-mobile-portfolio-columns.md b/docs/journals/260722-1705-mobile-portfolio-columns.md deleted file mode 100644 index dca6114..0000000 --- a/docs/journals/260722-1705-mobile-portfolio-columns.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -title: Mobile Stock and Coin Portfolio Columns -date: 2026-07-22 17:05 -component: stock and coin portfolio UI -status: completed ---- - -# Mobile Stock and Coin Portfolio Columns - -## Context - -The stock and coin position tables were still too wide for Telegram-sized screens. Long headers and a combined unrealized P&L cell made each row harder to scan on mobile. - -## What Happened - -Both position tables now use `Sym`, `Qty`, `Avg`, `Now`, `Val`, `P&L`, and `%`. Stock `Avg` and `Now` use thousand VND as an implicit unit without `k`; coin position money omits `$`. P&L amount and percentage are separate cells without parentheses. Summary formatting remains unchanged. - -## Reflection - -The useful part of this change is that it stayed local to the portfolio renderer. We did not touch command behavior, storage, or trade accounting, so the formatting work did not leak into unrelated output paths. - -The review callout about implicit `VND` was intentional, not a bug. The user explicitly removed `(VND)` from the stock title and accepted stock-domain context as sufficient. Keeping full currency labels in the summary would have widened the output and contradicted that decision. - -## Decisions - -- Keep compact `k/M/B/T` suffixes for position value and P&L amounts. -- Treat stock `Avg` and `Now` as thousand VND without a `k` suffix. -- Treat coin position money as implicit USD without `$`. -- Preserve explicit `VND` in non-portfolio replies where the standalone context is useful. -- Keep existing summary formats unchanged. -- Leave command and storage contracts unchanged. - -## Verification - -We validated the change with focused tests first, then the full suite. The final checks all passed: - -- focused tests -- `go test -count=1 ./...` -- `go build ./...` -- `go vet ./...` -- `golangci-lint run` - -Coverage landed at `76.7%` statement coverage, which is acceptable for this change because the work was formatting-heavy and stayed inside existing rendering paths. - -## Next - -The code change is done, but the commit is still pending user approval. The next step is to package this as a focused commit once that approval lands. diff --git a/docs/journals/260722-1843-retire-stock-dividend-command.md b/docs/journals/260722-1843-retire-stock-dividend-command.md deleted file mode 100644 index 577b193..0000000 --- a/docs/journals/260722-1843-retire-stock-dividend-command.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -title: Retire Redundant Stock Dividend Command -date: 2026-07-22 18:43 -component: stock module -status: completed ---- - -# Retire Redundant Stock Dividend Command - -## Context - -The stock module had one command too many. `/stock_dividend` overlapped with the specialized cash and share dividend commands, and the combined path was doing more harm than good. It blurred intent, made command handling harder to reason about, and created extra surface area for bugs that did not buy us any actual user value. - -## What Happened - -We removed the redundant `/stock_dividend` command and kept the specialized `/stock_cash_dividend` and `/stock_share_dividend` flows. That decision matches the product behavior: cash and share dividends are distinct operations, and forcing them through one combined command was an abstraction tax with no payoff. - -Historical stats were not deleted. They were marked `deleted:true` so the record stays intact for analytics and compatibility while the visible views stop treating the command as active. That is the right compromise for a retired command: preserve history, suppress the live surface. - -We also fixed two implementation traps that showed up while wiring the removal. First, the rolling deploy finding made it clear that a one-time migration was not enough; read/write suppression and every-boot reconciliation are now permanent so mixed-version deployments do not resurrect a dead command. Second, the Mongo path had an N+1 shape, so we replaced the row-by-row updates with `UpdateMany` plus `CountDocuments` instead of pretending the loop was acceptable. - -## Brutal Truth - -This was one of those changes where the codebase was telling us the truth before we were willing to hear it. Keeping a redundant command around because it “might be convenient” would have meant carrying dead complexity forever. The frustrating part is that the bug shape was predictable: once the command surface drifted from the underlying model, every extra path became another place to desync. - -## Technical Details - -- `/stock_dividend` removed from the live command surface -- `/stock_cash_dividend` and `/stock_share_dividend` retained as the explicit paths -- legacy stats rows preserved with `deleted:true` -- exact-prefix matching kept for safety so we do not suppress unrelated command names -- Mongo writes changed from per-document updates to `UpdateMany` plus `CountDocuments` - -## What We Tried - -- Considered leaving `/stock_dividend` as a compatibility alias, but that kept the ambiguity alive. -- Considered deleting stats history, but that would have broken reporting and destroyed useful audit data. -- Considered a row-by-row Mongo rewrite, but the N+1 behavior was the same old inefficiency in a different coat. - -## Root Cause Analysis - -The root cause was command overloading. We tried to make one command cover two different user intents, and the result was a brittle contract that was harder to maintain than the feature justified. The deploy/reconciliation issue was the same class of mistake: assuming a one-time action would stay correct across rolling versions. It will not. - -## Lessons Learned - -- Prefer explicit commands when the operations are semantically different. -- Retain legacy stats, but mark retired commands as deleted instead of pretending they never existed. -- Exact-prefix checks matter when suppressing command behavior; sloppy matching is how unrelated names get caught in the blast radius. -- If a deploy has mixed versions, assume the old behavior will reappear unless read/write suppression and reconciliation are both permanent. - -## Verification - -- focused stock, stats, and server tests passed -- focused race-enabled tests passed -- full `go test -count=1 ./...` passed -- real MongoDB 8 integration ran through Testcontainers and passed -- `go build ./...` and `go vet ./...` passed -- `golangci-lint run` reported zero issues -- total statement coverage measured `77.3%`; the repository has no configured coverage threshold - -## Next Steps - -The code is done and verified. The only remaining step is a commit, pending user approval. After that, the retired-command behavior should be watched in real deployments for any command-menu or stats edge cases. diff --git a/docs/journals/260723-1006-stock-events-command.md b/docs/journals/260723-1006-stock-events-command.md deleted file mode 100644 index 4451ed7..0000000 --- a/docs/journals/260723-1006-stock-events-command.md +++ /dev/null @@ -1,42 +0,0 @@ -# Stock Events Command - -**Date**: 2026-07-23 10:06 -**Severity**: Medium -**Component**: `stock` module, command metadata, docs -**Status**: Resolved - -## What Happened - -We shipped a new read-only `/stock_events [days]` command for SSI iBoard corporate actions. It defaults to 30 days, accepts 1..90, and returns chronological, Telegram-safe chunks. The important design choice was to keep this path additive and raw: the command reads SSI event data directly instead of forcing it through the strict dividend-normalization path used by `/stock_portfolio`. - -That decision mattered because the review found a bad assumption in the original normalization approach: optional SSI fields like ex-right, record, or payment date can be malformed without the event itself being unusable. We stopped trying to force every event into a dividend-shaped class and exposed the raw fields instead. That is the right tradeoff for an undocumented provider. - -## The Brutal Truth - -This was more fragile than it needed to be. We had to correct a command-menu metadata regression after the feature landed, and the review exposed that strict parsing was silently dropping valid SSI events. That is exactly the kind of bug that wastes time because it looks “clean” until you realize you are throwing away real data. - -## Technical Details - -- Feature landed in `feat(stock): add stock events lookup`. -- Command registration had to be kept in sync with Telegram help/menu metadata for ` [days]`. -- Review showed the generic path should not parse raw SSI events into a rigid class. -- Verification gates passed after fixes: focused stock tests, `go test -count=1 ./...`, `go vet ./...`, `go build ./...`, `golangci-lint run`, and `git diff --check`. - -## What We Tried - -- Reused the SSI corporate-action fetcher. -- Initially normalized events like dividends. -- Fixed the menu metadata mismatch in the shared command contract. -- Dropped the rigid event-class mapping and kept the raw provider payload. - -## Root Cause Analysis - -We overfit an undocumented API to our internal dividend model. That made the feature look structured but hid malformed optional fields and risked losing valid events. The real mistake was prioritizing type shape over provider fidelity. - -## Lessons Learned - -When the source API is undocumented, raw field preservation beats premature normalization. Keep user-facing contracts exact, but do not invent structure that the provider does not guarantee. - -## Next Steps - -Monitor for SSI schema drift, keep `/stock_events` best-effort, and expand tests only around the raw fields and command contract. The owner is the stock module maintainer; no follow-up migration work is needed. diff --git a/docs/journals/260723-1225-stock-info-command.md b/docs/journals/260723-1225-stock-info-command.md deleted file mode 100644 index 4edae42..0000000 --- a/docs/journals/260723-1225-stock-info-command.md +++ /dev/null @@ -1,47 +0,0 @@ -# Stock Info Command - -**Date**: 2026-07-23 12:25 -**Severity**: Medium -**Component**: `stock` module, SSI quote client, command docs/tests -**Status**: Resolved - -## What Happened - -We added a new read-only `/stock_info ` command to show a compact SSI iBoard quote snapshot with company, exchange, current price, gain/loss since open, change versus reference price, open/high/low, and volume. `/stock_price` stayed unchanged on purpose. - -The implementation had to be forced into a safer shape after review. The first pass reused the same SSI quote DTO for legacy price paths, which was a bad idea for an undocumented upstream schema. We split the detail response into its own DTO so the old decode path would not get dragged into the new fields, and we kept `/stock_price` on the existing fallback chain. - -## The Brutal Truth - -This was the kind of change that looks simple until it starts biting in review. The upstream API is undocumented, so every assumption becomes a liability. The frustrating part is that we had to spend time proving that the “easy” version would have been brittle: redirects, malformed fields, and bogus math all showed up as real failure modes. That is annoying, but it is also exactly why the extra guardrails were necessary. - -## Technical Details - -- `/stock_info` uses exactly one SSI GET and no KBS/VCI fallback. -- Since-open change is computed from `matchedPrice - openPrice`; reference change remains separate. -- Review blockers fixed: - - separate SSI detail DTO to protect legacy `/stock_price` decoding - - redirect refusal so the command cannot silently fan out into extra requests - - finite-safe change math to avoid `Inf%` - - explicit zero volume preserved as `0` while nil/invalid volume stays `N/A` - - company/exchange text bounded so Telegram replies stay safe -- Best-effort behavior is still required because SSI is undocumented. -- Verification passed: focused stock tests, race tests, `go vet ./...`, `go build ./...`, `golangci-lint run`, and repository test gates. - -## What We Tried - -- Started with a shared quote structure and a direct SSI detail fetch. -- Tightened the handler and formatting tests around the compact output. -- Reworked the client after adversarial review exposed the legacy decode and redirect risks. - -## Root Cause Analysis - -The root mistake was treating an undocumented provider like a stable contract. That made the first design too optimistic: one shared DTO, unchecked redirect behavior, and derived math that assumed normal values. The code worked until review forced the ugly cases into the open. - -## Lessons Learned - -When the upstream schema is not under our control, isolate the new path instead of extending old contracts in place. Keep the user-facing command additive, keep the legacy command stable, and assume the provider will return missing, malformed, or absurd values. - -## Next Steps - -Keep `/stock_info` best-effort and monitor for SSI schema drift. If the upstream shape changes again, update only the detail path and leave `/stock_price` untouched. No migration work is needed; the owner is the stock module maintainer. diff --git a/plans/260722-1114-compact-portfolio-numbers/phase-01-define-compact-formatters.md b/plans/260722-1114-compact-portfolio-numbers/phase-01-define-compact-formatters.md deleted file mode 100644 index f3dcd64..0000000 --- a/plans/260722-1114-compact-portfolio-numbers/phase-01-define-compact-formatters.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -phase: 1 -title: Define Compact Formatters -status: completed -priority: P1 -dependencies: [] -effort: small ---- - -# Phase 1: Define Compact Formatters - -## Overview - -Define private compact monetary formatters for stock VND and coin USD while -preserving the modules' exported formatting contracts. - -## Requirements - -- Functional: select base, `k`, `M`, `B`, or `T` from absolute magnitude. -- Functional: show at most three fractional digits and trim trailing zeroes. -- Functional: promote after rounding rollover, so `999.9999k` becomes `1M`. -- Functional: preserve negative signs and module-native separators. -- Non-functional: keep `FormatVND`, `FormatPnL`, `FormatUSD`, and - `FormatPnLUSD` output unchanged outside position tables. - -## Architecture - -Each module owns a private compact formatter because stock uses dot grouping -plus comma decimals, while coin uses `$`, comma grouping, and dot decimals. -Both follow the same magnitude table: base `< 10^3`, then powers of 1,000 -through `T`. P&L wrappers reuse existing percentage calculation behavior while -substituting only the monetary formatter. - -## Related Code Files - -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/format.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/format_test.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/coin/format.go` -- Create: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/coin/format_test.go` - -## Implementation Steps - -1. Replace the preliminary stock `k`-only helper with automatic suffix selection. -2. Preserve stock grouping, decimal comma, sign, and nearest-VND input rounding. -3. Add the equivalent USD helper with `$` and English separators. -4. Add private compact P&L wrappers that retain signed two-decimal percentages. -5. Test base/`k`/`M`/`B`/`T` boundaries, trimmed fractions, negatives, zero, - rounding rollover, and unchanged exported formatter results. - -## Success Criteria - -- [x] Stock examples include `999`, `1k`, `25,35k`, `126M`, and `1,25B`. -- [x] Coin examples include `$999.99`, `$1k`, `$50k`, and `$1.234M`. -- [x] Boundary rollover never emits `1000k`, `1000M`, or `1000B`. -- [x] Exported stock and coin formatters retain their current test outputs. - -## Risk Assessment - -Main risks: threshold off-by-one errors, locale separator drift, negative-sign -duplication, and rounding into the next suffix. Table-driven boundary tests -mitigate all four. No security or data-protection impact; formatting is local -and receives already-validated numeric values. diff --git a/plans/260722-1114-compact-portfolio-numbers/phase-02-integrate-portfolio-renderers.md b/plans/260722-1114-compact-portfolio-numbers/phase-02-integrate-portfolio-renderers.md deleted file mode 100644 index 50145f0..0000000 --- a/plans/260722-1114-compact-portfolio-numbers/phase-02-integrate-portfolio-renderers.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -phase: 2 -title: Integrate Portfolio Renderers -status: completed -priority: P1 -dependencies: - - 1 -effort: small ---- - -# Phase 2: Integrate Portfolio Renderers - -## Overview - -Wire the approved compact formatters into the four monetary position columns -of both portfolio renderers and lock down user-visible output. - -## Requirements - -- Functional: compact `Avg`, `Now`, `Value`, and position `Unrealized P&L`. -- Functional: keep `Qty`, percentages, title currency markers, and `N/A` intact. -- Functional: keep stock and coin summary tables at their existing full formats. -- Non-functional: do not change quote fetching, valuation, sorting, Telegram - HTML escaping, reply truncation, or dividend notification behavior. - -## Architecture - -Only the position-row construction changes. Stock calls its VND compact helper; -coin calls its USD compact helper. Summary row construction continues using the -existing full-value formatters, isolating the behavior to the requested cells. - -## Related Code Files - -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/handlers.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/stats_test.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/coin/views.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/coin/handlers_test.go` -- Inspect: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/coin/views_reply_budget_test.go` - -## Implementation Steps - -1. Replace stock position monetary cells with automatic compact formatting, - including the missing-price branch's available average. -2. Apply compact USD formatting to the equivalent coin position cells. -3. Keep both summary slices unchanged and verify full values remain present. -4. Strengthen renderer assertions for magnitude suffixes, P&L percentages, - currency conventions, and the absence of compaction in summaries. -5. Confirm partial-price paths still emit `N/A` and unavailable Account P&L. - -## Success Criteria - -- [x] Stock position cells select `k/M/B/T` independently and use VND separators. -- [x] Coin position cells select `k/M/B/T` independently and retain `$`. -- [x] Position P&L amounts compact; percentages remain unchanged. -- [x] Summary values and non-portfolio replies remain byte-for-byte compatible. -- [x] Missing and overflowed quotes retain current `N/A` behavior. -- [x] Telegram reply-budget tests remain valid or improve due to shorter cells. - -## Risk Assessment - -Main regression risk is accidentally compacting summary or standalone output. -Use private position-only wrappers and assertions that check both compact rows -and full summary values. No authorization, concurrency, persistence, or network -boundary changes. diff --git a/plans/260722-1114-compact-portfolio-numbers/phase-03-verify-formatting-contracts.md b/plans/260722-1114-compact-portfolio-numbers/phase-03-verify-formatting-contracts.md deleted file mode 100644 index 5613df8..0000000 --- a/plans/260722-1114-compact-portfolio-numbers/phase-03-verify-formatting-contracts.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -phase: 3 -title: Verify Formatting Contracts -status: completed -priority: P1 -dependencies: - - 2 -effort: small ---- - -# Phase 3: Verify Formatting Contracts - -## Overview - -Verify formatter correctness, both portfolio flows, and repository-wide quality -without expanding the feature scope. - -## Requirements - -- Functional: every approved example and boundary has automated coverage. -- Non-functional: changed Go files are formatted; tests, vet, and lint pass. -- Non-functional: public functions, commands, schemas, and persistence remain - unchanged. - -## Architecture - -Verification proceeds from deterministic formatter tests to renderer tests, -then the full repository gates. A final diff review maps each changed call site -to the requested four columns and checks that unrelated formatting is untouched. - -## Related Code Files - -- Inspect: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock` -- Inspect: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/coin` -- Inspect: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/README.md` -- Inspect: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/docs` - -## Implementation Steps - -1. Run `gofmt` on every changed Go file and `git diff --check`. -2. Run `go test ./internal/modules/stock ./internal/modules/coin`. -3. Run `go test ./...` and `go vet ./...`. -4. Run `golangci-lint run` when available, matching the repository CI gate. -5. Review the final diff for scope, exported-contract stability, secrets, and - accidental summary or standalone-message changes. -6. Decide whether README/docs need an evergreen update; avoid documentation - churn when tests and the approved report sufficiently capture presentation. - -## Success Criteria - -- [x] All focused and repository-wide tests pass. -- [x] `go vet ./...` passes. -- [x] `golangci-lint run` passes when installed. -- [x] `git diff --check` is clean. -- [x] Review finds no changes to commands, storage, migrations, APIs, or quotes. -- [x] Final portfolio examples match the approved brainstorm report. - -## Risk Assessment - -The main risk is incomplete branch coverage for partial or overflowed prices. -Run existing partial/overflow tests and add only focused assertions if a gap is -found. Rollback is a localized renderer/formatter revert; no data migration is -needed. Security impact is none beyond the standard staged-diff secret scan. diff --git a/plans/260722-1114-compact-portfolio-numbers/plan.md b/plans/260722-1114-compact-portfolio-numbers/plan.md deleted file mode 100644 index 7b31172..0000000 --- a/plans/260722-1114-compact-portfolio-numbers/plan.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: Compact Portfolio Number Formatting -description: >- - Compact stock and coin portfolio position values with automatic k/M/B/T - financial suffixes. -status: completed -priority: P2 -branch: main -tags: - - refactor - - stock - - coin - - telegram -blockedBy: [] -blocks: [] -created: '2026-07-22T04:14:02.795Z' -createdBy: 'ck:plan' -source: skill ---- - -# Compact Portfolio Number Formatting - -## Overview - -Replace wide monetary values in stock and coin position tables with automatic -`k/M/B/T` formatting. Apply only to `Avg`, `Now`, `Value`, and position-level -`Unrealized P&L`; preserve summary tables and standalone replies. - -Source: [approved brainstorm report](../reports/260722-1112-compact-portfolio-number-format.md). - -## Phases - -| Phase | Name | Status | -|-------|------|--------| -| 1 | [Define Compact Formatters](./phase-01-define-compact-formatters.md) | Completed | -| 2 | [Integrate Portfolio Renderers](./phase-02-integrate-portfolio-renderers.md) | Completed | -| 3 | [Verify Formatting Contracts](./phase-03-verify-formatting-contracts.md) | Completed | - -## Dependencies - -- No cross-plan dependencies. -- Implementation revised the preliminary stock-only `k` work into the approved - automatic formatter and added matching coin support. -- Go toolchain plus optional `golangci-lint` for verification. - -## Boundaries - -- No command, storage, API, migration, price-fetching, or P&L calculation changes. -- No compact formatting in summary tables or non-portfolio messages. -- No shared cross-module abstraction unless duplication creates real complexity. diff --git a/plans/260722-1356-per-user-dividend-history/phase-01-introduce-dividend-history-and-migration.md b/plans/260722-1356-per-user-dividend-history/phase-01-introduce-dividend-history-and-migration.md deleted file mode 100644 index 185215c..0000000 --- a/plans/260722-1356-per-user-dividend-history/phase-01-introduce-dividend-history-and-migration.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -phase: 1 -title: Introduce Dividend History and Migration -status: completed -priority: P1 -dependencies: [] -effort: large ---- - -# Phase 1: Introduce Dividend History and Migration - -## Overview - -Replace cursor and hashed-ledger persistence with a validated per-user dividend -history, simplify asset lifecycle state, and physically remove obsolete MongoDB -fields through an idempotent startup migration. - -## Requirements - -- Add a persisted dividend record containing normalized kind, provider dates, - amount or share ratio, display details, and `processed`. -- Store records as `Portfolio.Dividends[ticker][rawSSIEventID]`. -- Remove `AssetPosition.DividendCheckedAt` and - `Portfolio.AppliedDividendEvents` from the runtime schema. -- Preserve `openedAt` when buying more and reset it only for a newly opened - position. -- Make manual dividend application independent of discovery state. -- Rewrite legacy user documents once and mark migration completion through - `internal/systemstate`. - -## Related Code Files - -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/portfolio.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/portfolio_test.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/handlers.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/handlers_test.go` -- Create: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/startup.go` -- Create: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/startup_test.go` -- Create: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/startup_mongo_test.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/cmd/server/main.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/cmd/server/main_test.go` - -## Implementation Steps - -1. Add failing table-driven tests for dividend-map initialization, validation, - storage round trips, buy/partial-sell/full-sell independence, and manual - dividend behavior without a cursor. -2. Define `DividendRecord` and nested history maps using existing JSON/BSON - naming conventions and raw provider-ID validation. -3. Update portfolio load, creation, validation, and financial mutation helpers; - remove obsolete cursor error language from manual handlers. -4. Add migration fixtures containing both obsolete fields and pending-action - documents; ensure only `user:` portfolios are rewritten. -5. Implement conflict-aware migration of stock user documents, dropping - `dividendCheckedAt` and `appliedDividendEvents`, initializing the new map, and - recording an idempotent system-state marker with migrated count. -6. Call stock startup maintenance from `cmd/server/main.go` before module build - and cover invocation/error propagation at the server boundary. - -## Success Criteria - -- [x] New and legacy portfolios load with initialized dividend history. -- [x] Assets no longer persist or validate `dividendCheckedAt`. -- [x] Manual dividends change only shares/cash and basis-related portfolio data. -- [x] The legacy hashed ledger is deliberately discarded as approved. -- [x] Startup migration is idempotent, retries version conflicts safely, skips - pending-action keys, and records completion in `system`. -- [x] A MongoDB fixture proves obsolete BSON fields are physically absent after - migration. - -## Risk Assessment - -The migration touches every stock user document and legacy duplicate history is -intentionally removed. Limit the scan to `user:`, validate rewritten documents, -use version-aware writes, and set the completion marker only after the entire -scan succeeds. Never log document contents. diff --git a/plans/260722-1356-per-user-dividend-history/phase-02-synchronize-refresh-and-retain-events.md b/plans/260722-1356-per-user-dividend-history/phase-02-synchronize-refresh-and-retain-events.md deleted file mode 100644 index 9996555..0000000 --- a/plans/260722-1356-per-user-dividend-history/phase-02-synchronize-refresh-and-retain-events.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -phase: 2 -title: Synchronize, Refresh, and Retain SSI Events -status: completed -priority: P1 -dependencies: - - phase-01-introduce-dividend-history-and-migration -effort: large ---- - -# Phase 2: Synchronize, Refresh, and Retain SSI Events - -## Overview - -Build deterministic synchronization around the rolling 30-day publication -window, targeted historical refreshes, provider corrections, and 90-day cleanup. - -## Requirements - -- Fetch an exact `[now-30d, now]` publication window for currently held tickers. -- Upsert by ticker and raw SSI ID while preserving local processing state. -- Re-fetch missing-Record-date events from their stored publication-day window. -- Refresh an event before it becomes actionable. -- Delete history 90 days after Record date, or after publication when Record - date remains missing. -- Continue displaying the portfolio and preserve stored data on provider errors. - -## Related Code Files - -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividend_events.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividend_events_ssi.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividend_events_ssi_test.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividend_notifications.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividend_flow_test.go` -- Create: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividend_history.go` -- Create: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividend_history_test.go` - -## Implementation Steps - -1. Add failing tests for exact boundaries, overlapping SSI day queries, merge - semantics, corrections, missing-date refresh, grouped refresh requests, - partial failures, and both retention clocks. -2. Isolate conversion between `DividendEvent` and the persisted record so raw - SSI data is normalized once and the local processed field cannot be overwritten. -3. Make the caller enforce the exact 30-day `PublishedAt` window after the SSI - adapter's intentional prior-day overlap. -4. Group incomplete refreshes by ticker and original publication day, fetch - bounded windows, and update only matching raw event IDs. -5. Refresh due events before action creation when possible; if SSI omits the - matching event, keep using the retained record until processing or expiry. -6. Prune expired records using Asia/Saigon date boundaries and remove empty - ticker maps before saving. -7. Keep fetch concurrency bounded and preserve the current behavior where one - ticker failure does not suppress successful results for other tickers. - -## Success Criteria - -- [x] Events outside the exact 30-day publication interval are not discovered. -- [x] Repeated fetches are idempotent and preserve `processed`. -- [x] Missing Record dates can be filled after the event leaves the recent feed. -- [x] SSI corrections replace unprocessed provider details before approval. -- [x] Processed state is never reverted by a provider refresh. -- [x] Record-dated and permanently incomplete events are deleted on their - respective 90-day boundaries. -- [x] Provider failures do not delete history or block portfolio rendering. - -## Risk Assessment - -Historical refreshes can amplify SSI traffic. Group requests by ticker/day, -reuse existing pagination limits and timeouts, and refresh only incomplete or -newly due unprocessed records. Boundary tests must use the Asia/Saigon location -to avoid UTC-dependent behavior. diff --git a/plans/260722-1356-per-user-dividend-history/phase-03-gate-notifications-and-approval.md b/plans/260722-1356-per-user-dividend-history/phase-03-gate-notifications-and-approval.md deleted file mode 100644 index 22493e8..0000000 --- a/plans/260722-1356-per-user-dividend-history/phase-03-gate-notifications-and-approval.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -phase: 3 -title: Gate Notifications and Approval by Record Date -status: completed -priority: P1 -dependencies: - - phase-02-synchronize-refresh-and-retain-events -effort: large ---- - -# Phase 3: Gate Notifications and Approval by Record Date - -## Overview - -Split future notices from actionable suggestions and make repeated notifications -safe through idempotent per-user dividend processing. - -## Requirements - -- Send one informational message before Record date after every portfolio request. -- Do not expose approval while Record date is missing. -- At or after the start of Record date in Asia/Saigon, send one separate - actionable message per eligible event after every portfolio request. -- Require a current position whose `openedAt` is no later than Record date. -- Calculate from current quantity and atomically mark the stored event processed. -- Store only event references and Telegram security context in pending actions. - -## Related Code Files - -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividend_notifications.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividend_callback.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/pending_dividend.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividend_flow_test.go` - -## Implementation Steps - -1. Add failing tests for before/on/after Record date, missing Record date, - repeated future notices, failed-send retries, multiple event messages, and - multiple pending buttons guarded by processed-state idempotency. -2. Add callback tests for owner/chat/message binding, sold positions, - post–Record date rebuys, current-quantity calculations, double clicks, - concurrent calls, save failures, and already processed history. -3. Split rendering into informational and actionable event messages while - retaining source dates and the SSI disclaimer. -4. Reload under the per-user lock before every delivery so processed events stop - immediately while unprocessed events repeat on every request. -5. Reduce `PendingDividendAction` to owner/chat/message/token timing plus ticker, - SSI ID, and current-position lifecycle binding. -6. During callback, load the authoritative persisted dividend, revalidate Record - date and eligibility, apply cash/share changes, and set `processed = true` in - the same portfolio save. -7. Keep cleanup, callback acknowledgement, and keyboard removal best-effort - after the financial save, preserving safe retry behavior on save failure. - -## Success Criteria - -- [x] Future events have no approval control and notify after every portfolio - request until processed or expired. -- [x] Missing Record dates never become actionable. -- [x] Every due event receives its own message and button. -- [x] A post–Record date position lifecycle cannot claim an old event. -- [x] Financial values come from portfolio dividend history, not pending data. -- [x] Successful processing and `processed = true` are atomic and idempotent. -- [x] Multiple or recreated pending actions remain safe because only the first - successful callback can process the event. - -## Risk Assessment - -Telegram delivery and portfolio persistence cannot be one transaction. Preserve -the existing conservative sequence, recheck state under the user lock, and test -ambiguous failure paths. Opaque random tokens and exact owner/chat/message -binding remain mandatory. diff --git a/plans/260722-1356-per-user-dividend-history/phase-04-integrate-document-and-verify.md b/plans/260722-1356-per-user-dividend-history/phase-04-integrate-document-and-verify.md deleted file mode 100644 index bf7bb1e..0000000 --- a/plans/260722-1356-per-user-dividend-history/phase-04-integrate-document-and-verify.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -phase: 4 -title: Integrate, Document, and Verify -status: completed -priority: P1 -dependencies: - - phase-03-gate-notifications-and-approval -effort: medium ---- - -# Phase 4: Integrate, Document, and Verify - -## Overview - -Complete handler integration, align user and operations documentation, and run -the repository's full quality gates. - -## Requirements - -- `/stock_portfolio` sends the portfolio first and then runs history sync and - notifications. -- User-facing errors no longer refer to dividend checkpoints. -- Documentation describes the new schema, Record-date gate, refresh policy, - eligibility approximation, pending lifetime, and retention. -- All focused and repository-wide Go checks pass. - -## Related Code Files - -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/handlers.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/handlers_test.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/internal/modules/stock/dividends_test.go` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/README.md` -- Modify: `C:/Users/miti99/Workspaces/tiennm99/miti99bot/docs/deploy-coolify-selfhosted.md` - -## Implementation Steps - -1. Add integration assertions that the portfolio is sent before SSI messages - and remains available during discovery or refresh errors. -2. Update handler error text and remove obsolete cursor expectations from - manual dividend and portfolio tests. -3. Document per-user history, exact recent discovery, historical missing-date - refresh, repeated unprocessed notices, Record-date approval, current-holding - approximation, and 90-day cleanup. -4. Run `gofmt` on all changed Go files. -5. Run focused tests for `internal/modules/stock` and `cmd/server`. -6. Run `go test ./...`, `go vet ./...`, and `go build ./...`. -7. Run `golangci-lint run` when available and resolve all in-scope findings. - -## Success Criteria - -- [x] `/stock_portfolio` preserves portfolio-first and best-effort SSI behavior. -- [x] README and deployment docs contain no cursor or hashed-ledger claims. -- [x] Focused stock/server tests pass. -- [x] Full tests, vet, and build pass. -- [x] CI lint gate passes when locally available. -- [x] `git diff --check` reports no whitespace errors. - -## Risk Assessment - -The main integration risk is documentation or older tests silently preserving -the cursor-era contract. Search the full repository for `dividendCheckedAt`, -`AppliedDividendEvents`, `appliedDividendEvents`, and checkpoint wording before -declaring completion. diff --git a/plans/260722-1356-per-user-dividend-history/plan.md b/plans/260722-1356-per-user-dividend-history/plan.md deleted file mode 100644 index 47561e2..0000000 --- a/plans/260722-1356-per-user-dividend-history/plan.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -title: Per-User Stock Dividend History -description: >- - Replace dividend cursors and the hashed applied ledger with per-user SSI - dividend history, Record-date approval gating, refresh, and retention. -status: completed -priority: P1 -branch: main -tags: - - feature - - stock - - dividends - - migration - - telegram -blockedBy: [] -blocks: [] -created: '2026-07-22T13:56:06+07:00' -createdBy: 'ck:plan' -source: skill ---- - -# Per-User Stock Dividend History - -## Overview - -Persist normalized SSI events under -`portfolio.dividends..`, independent of active positions. -Discover events from the recent 30-day publication window, refresh incomplete -historical records, repeat unprocessed notices on every portfolio request, and -require approval from Record date onward. Remove records after 90 days and migrate obsolete cursor and -hashed-ledger fields out of MongoDB. - -Source: [approved brainstorm report](../reports/260722-1356-per-user-dividend-history-brainstorm.md). - -## Delivery Mode - -Test-first. Each phase adds or revises focused tests before changing financial -state or notification behavior. - -## Phases - -| Phase | Name | Status | -|-------|------|--------| -| 1 | [Introduce Dividend History and Migration](./phase-01-introduce-dividend-history-and-migration.md) | Completed | -| 2 | [Synchronize, Refresh, and Retain SSI Events](./phase-02-synchronize-refresh-and-retain-events.md) | Completed | -| 3 | [Gate Notifications and Approval by Record Date](./phase-03-gate-notifications-and-approval.md) | Completed | -| 4 | [Integrate, Document, and Verify](./phase-04-integrate-document-and-verify.md) | Completed | - -## Dependencies - -- Phases are sequential because notification and callback behavior depend on - the new persisted model and synchronization helpers. -- No cross-plan dependencies. -- SSI remains the existing provider; no new external dependency is required. -- MongoDB integration tests require Docker or `MONGODB_TEST_URL` and may skip - explicitly when neither is available. - -## Data Contract - -- `assets.` retains only quantity, basis, and `openedAt`. -- `dividends..` contains normalized provider details plus - local `processed` state. -- Pending actions reference ticker and SSI ID; they do not duplicate dividend - financial fields. -- Entries expire 90 days after Record date, or after publication when Record - date never becomes available. - -## Boundaries - -- Current quantity remains the calculation basis at approval time. -- The current asset must have opened on or before Record date. -- Manual dividend commands do not create or process SSI history entries. -- Portfolio output remains available when SSI discovery or refresh fails. -- No command registration or parameter changes. - -## Definition of Done - -- All phase success criteria pass. -- Focused stock and server tests pass, followed by `go test ./...` and - `go vet ./...`. -- Changed Go files are formatted with `gofmt`. -- `golangci-lint run` passes when the binary is available. -- README and deployment storage documentation match the new persisted schema - and Record-date behavior. diff --git a/plans/260722-1356-per-user-dividend-history/reports/pm-260722-1611-per-user-dividend-history.md b/plans/260722-1356-per-user-dividend-history/reports/pm-260722-1611-per-user-dividend-history.md deleted file mode 100644 index 3fee537..0000000 --- a/plans/260722-1356-per-user-dividend-history/reports/pm-260722-1611-per-user-dividend-history.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -type: project-status -plan: per-user-dividend-history -generated_at: 2026-07-22T16:11:50+07:00 -status: completed ---- - -# Plan Complete: Per-User Dividend History - -## Summary - -| Metric | Result | -|---|---| -| Phases | 4/4 completed | -| Acceptance checks | 26/26 completed | -| Full tests | 607 passed, 1 intentional helper skip | -| Race test | Stock package passed | -| Static gates | Vet, build, lint, diff check passed | -| Review | Primary and adversarial PASS | - -## Achievements - -- Replaced asset dividend cursors with retained per-user SSI event history. -- Added exact 30-day discovery and historical missing-date refresh. -- Added Record-date approval gating and minimal opaque pending actions. -- Repeated unprocessed notices after every portfolio request while keeping - multiple buttons financially idempotent. -- Added 90-day cleanup and idempotent MongoDB legacy-field migration. -- Updated runtime, deployment, and user-facing documentation. - -## Accepted Risk - -- Legacy hashed applied IDs are intentionally deleted. A recently applied - legacy event could be rediscovered and credited again; the owner explicitly - accepted this before landing. - -## Documentation - -- `README.md` -- `docs/deploy-coolify-selfhosted.md` -- Approved brainstorm report and all implementation phases - -## Unresolved Questions - -- None blocking. Optional future hardening: direct concurrent-callback and - callback-save-failure tests. diff --git a/plans/260722-1705-mobile-portfolio-columns/phase-01-compact-portfolio-columns.md b/plans/260722-1705-mobile-portfolio-columns/phase-01-compact-portfolio-columns.md deleted file mode 100644 index 0f71891..0000000 --- a/plans/260722-1705-mobile-portfolio-columns/phase-01-compact-portfolio-columns.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -phase: 1 -title: Compact Portfolio Columns -status: completed -priority: P1 -dependencies: [] -effort: small ---- - -# Phase 1: Compact Portfolio Columns - -## Overview - -Apply the approved seven-column mobile layout to stock and coin position rows. - -## Requirements - -- Use `Sym | Qty | Avg | Now | Val | P&L | %` in both modules. -- Render stock `Avg` and `Now` in thousand VND without a `k` suffix. -- Omit currency from both titles and omit `$` from coin position money. -- Keep compact suffixes for stock value/P&L and coin position money. -- Preserve full summary formats, P&L calculations, and `N/A` behavior. - -## Architecture - -Only private position formatters and row renderers change. Exported formatters, -portfolio persistence, trading operations, quote providers, summaries, and -dividend workflows remain unchanged. - -## Implementation Checklist - -- [x] Add stock thousand-VND position formatter. -- [x] Split stock position P&L amount and percentage. -- [x] Remove `$` from coin position-only compact formatting. -- [x] Split coin position P&L amount and percentage. -- [x] Update renderer, edge-path, and formatter tests. -- [x] Update README and pass all verification gates. - -## Risks - -- Seven-column `N/A` rows could become misaligned; covered by partial and - overflow tests. -- Removing currency markers is intentionally limited to position displays; - coin summaries retain `$`, while stock uses implicit VND throughout. - diff --git a/plans/260722-1705-mobile-portfolio-columns/plan.md b/plans/260722-1705-mobile-portfolio-columns/plan.md deleted file mode 100644 index 9da66e0..0000000 --- a/plans/260722-1705-mobile-portfolio-columns/plan.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -title: Mobile Portfolio Columns -description: Compact stock and coin position tables for narrow Telegram screens. -status: completed -priority: P1 -effort: small -branch: main -tags: - - stock - - coin - - telegram - - formatting -created: 2026-07-22 ---- - -# Mobile Portfolio Columns - -## Overview - -Shorten stock and coin position tables while preserving calculations, summary -formatting, missing-price behavior, and Telegram reply limits. - -## Phases - -| Phase | Name | Status | Progress | -|---|---|---|---| -| 1 | [Compact Portfolio Columns](./phase-01-compact-portfolio-columns.md) | Completed | 6/6 (100%) | - -## Dependencies - -- Builds on the completed compact-number formatter plan. -- No storage, command, provider, or migration dependency. - -## Success Criteria - -- [x] Both position tables use `Sym`, `Qty`, `Avg`, `Now`, `Val`, `P&L`, `%`. -- [x] Stock `Avg` and `Now` use implicit thousand VND without `k`. -- [x] Coin position money uses implicit USD without `$`. -- [x] P&L amount and signed percentage render in separate columns. -- [x] Summary formatting and unavailable-price behavior remain compatible. -- [x] Focused/full tests, build, vet, lint, and diff checks pass. - diff --git a/plans/260722-1705-mobile-portfolio-columns/reports/pm-260722-1705-mobile-portfolio-columns.md b/plans/260722-1705-mobile-portfolio-columns/reports/pm-260722-1705-mobile-portfolio-columns.md deleted file mode 100644 index aa69eb5..0000000 --- a/plans/260722-1705-mobile-portfolio-columns/reports/pm-260722-1705-mobile-portfolio-columns.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: Mobile Portfolio Columns Completion Report -status: completed -created: 2026-07-22 ---- - -# Mobile Portfolio Columns Completion Report - -## Summary - -| Metric | Result | -|---|---| -| Phases | 1/1 completed | -| Tasks | 6/6 completed | -| Code review | Pass; no remaining findings | -| Focused tests | Pass | -| Full tests | Pass | -| Build / vet / lint | Pass / pass / 0 issues | -| Statement coverage | 76.7% | - -## Achievements - -- Reduced both position tables to concise seven-column mobile layouts. -- Made stock price columns implicit thousand VND. -- Made coin position currency implicit while preserving `$` in summaries. -- Separated P&L amount and percentage without parentheses. -- Preserved missing-price, overflow, truncation, and portfolio behavior. - -## Known Limitations - -- Stock summary VND is intentionally implicit after removing title currency. -- Go coverage reports statement coverage; branch coverage was not generated. - -## Next Step - -- Commit and push after user approval. - diff --git a/plans/260722-1843-retire-stock-dividend-command/phase-01-remove-command-and-retire-stats.md b/plans/260722-1843-retire-stock-dividend-command/phase-01-remove-command-and-retire-stats.md deleted file mode 100644 index ca6a655..0000000 --- a/plans/260722-1843-retire-stock-dividend-command/phase-01-remove-command-and-retire-stats.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -phase: 1 -title: Remove Command and Retire Stats -status: completed -priority: P1 -dependencies: [] -effort: medium ---- - -# Phase 1: Remove Command and Retire Stats - -## Overview - -Delete the generic stock dividend command and add durable stats retirement for -its retained usage history. - -## Requirements - -- Preserve `/stock_cash_dividend` and `/stock_share_dividend` behavior. -- Remove all active `/stock_dividend` command surfaces. -- Retain stats rows rather than deleting usage history. -- Filter retired rows from every memory and Mongo stats view. -- Reconcile late legacy writes on future startup runs. -- Keep Mongo startup bounded with exact-command bulk operations. - -## Architecture - -The stats runtime owns a permanent retired-command rule. New increments for the -retired command are ignored and reads exclude it independently of the stored -flag. Startup physically restores `deleted: true`: memory uses versioned CAS; -Mongo uses indexed `UpdateMany` plus `CountDocuments`. The shared system marker -stores completion status and the current exact-command row count. - -## Implementation Checklist - -- [x] Remove command registration and combined handler. -- [x] Remove command-menu metadata and handler/menu tests. -- [x] Remove README command documentation. -- [x] Add stats/system startup wiring and migration marker. -- [x] Add permanent read/write retirement behavior. -- [x] Add memory and real-Mongo reconciliation tests. -- [x] Pass review, test, race, build, vet, lint, and diff gates. - -## Risks - -- A legacy process can write after migration; runtime filtering hides it and - every later startup physically tombstones it again. -- Unbounded per-row Mongo startup work is avoided by exact indexed bulk calls. -- Marker count is derived from all matching rows, so retries do not undercount. - diff --git a/plans/260722-1843-retire-stock-dividend-command/plan.md b/plans/260722-1843-retire-stock-dividend-command/plan.md deleted file mode 100644 index 775f3ac..0000000 --- a/plans/260722-1843-retire-stock-dividend-command/plan.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Retire Generic Stock Dividend Command -description: Remove /stock_dividend while preserving and hiding historical usage statistics. -status: completed -priority: P1 -effort: medium -branch: main -tags: - - stock - - stats - - migration - - telegram -created: 2026-07-22 ---- - -# Retire Generic Stock Dividend Command - -## Overview - -Remove the combined manual dividend command because users can apply cash and -share dividends through the two specialized commands. Retain historical stats -as deleted legacy records and prevent rolling-deploy writes from exposing them. - -## Phases - -| Phase | Name | Status | Progress | -|---|---|---|---| -| 1 | [Remove Command and Retire Stats](./phase-01-remove-command-and-retire-stats.md) | Completed | 7/7 (100%) | - -## Dependencies - -- Uses the existing stats `deleted` field and shared `system` collection. -- No portfolio schema or dividend-history migration changes. - -## Success Criteria - -- [x] `/stock_dividend` is absent from registration, menu, help, handler, and README. -- [x] Cash and share dividend commands remain available and unchanged. -- [x] Anonymous and per-user historical rows remain stored with `deleted: true`. -- [x] Retired rows remain hidden even after a late legacy-process write. -- [x] Memory reconciliation is idempotent and Mongo reconciliation is bulk/indexed. -- [x] Exact-prefix commands such as `stock_dividend_extra` remain untouched. -- [x] Full, race, Mongo integration, build, vet, and lint checks pass. - diff --git a/plans/260722-1843-retire-stock-dividend-command/reports/pm-260722-1843-retire-stock-dividend-command.md b/plans/260722-1843-retire-stock-dividend-command/reports/pm-260722-1843-retire-stock-dividend-command.md deleted file mode 100644 index b425ada..0000000 --- a/plans/260722-1843-retire-stock-dividend-command/reports/pm-260722-1843-retire-stock-dividend-command.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -title: Retire Stock Dividend Command Completion Report -status: completed -created: 2026-07-22 ---- - -# Retire Stock Dividend Command Completion Report - -## Summary - -| Metric | Result | -|---|---| -| Phases | 1/1 completed | -| Tasks | 7/7 completed | -| Code review | Pass; no remaining findings | -| Focused/full tests | Pass | -| Race tests | Pass | -| Mongo 8 integration | Pass; executed via Testcontainers | -| Build / vet / lint | Pass / pass / 0 issues | -| Statement coverage | 77.3% | - -## Achievements - -- Removed the redundant combined dividend command from every active surface. -- Preserved specialized cash/share dividend workflows. -- Retained historical stats while permanently suppressing the retired command. -- Hardened rolling-deploy behavior against late legacy increments. -- Used bounded Mongo bulk reconciliation with exact-prefix safety. - -## Known Limitations - -- A still-running legacy binary can show its own stale stats view until it is - replaced; new binaries suppress the command immediately. -- No repository coverage threshold is configured; measured total is 77.3%. - -## Next Step - -- Commit and push after user approval. - diff --git a/plans/260723-0852-stock-events-command/phase-01-extend-ssi-corporate-events.md b/plans/260723-0852-stock-events-command/phase-01-extend-ssi-corporate-events.md deleted file mode 100644 index f73cb24..0000000 --- a/plans/260723-0852-stock-events-command/phase-01-extend-ssi-corporate-events.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -phase: 1 -title: Extend SSI Corporate Events -status: completed -priority: P2 -dependencies: [] ---- - -# Phase 1: Extend SSI Corporate Events - -## Overview -Add an additive SSI corporate-action fetch path for read-only event listing. Preserve the current dividend-specific contract used by `/stock_portfolio` and retained dividend history. - -## Requirements -- Functional: fetch all SSI corporate-action rows for one normalized symbol in `(after, through]`, with the same one-day Asia/Saigon overlap and provider page caps already enforced in `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi.go:23` and `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi.go:113`. -- Functional: deduplicate by SSI `CorId`, then return deterministic chronological order by `PublishedAt`, tie-broken by provider ID as today’s dividend path does at `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi.go:160`. -- Functional: preserve SSI raw display fields for the generic command path: symbol, provider ID, SSI type code/label, title/description/name, published/ex/record/payment strings, value, ratio, and source URL. -- Functional: use a private cursor timestamp only for filtering and sorting; do not replace the raw SSI date strings with normalized event dates in the display model. -- Functional: keep `FetchDividendEvents` behavior unchanged for dividend discovery and `/stock_portfolio` notifications (`C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events.go:12`, `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_notifications.go:204`). -- Non-functional: no persistence, no callback/button work, no schema/index/system-state changes. - -## Architecture -`SSIDividendProvider` already owns SSI HTTP access, page fetches, date parsing, and source-link generation in `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi.go:92`, `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi.go:172`, `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi.go:247`, and `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi.go:309`. Phase 1 adds a parallel generic event model plus an additive fetch method that reuses those helpers, but does not widen `DividendEvent` validation. - -Data flow: -1. Symbol and time window enter the generic provider. -2. SSI pages are fetched with the existing timeout/cap logic. -3. Each raw SSI row is copied into a generic display model with minimal validation: matching symbol, valid provider ID, and a private cursor timestamp via `publicDate` or the existing fallback chain. -4. Generic rows are sorted and returned to the command layer with raw SSI strings intact; dividend-only normalization remains on the separate `FetchDividendEvents` path. - -## Related Code Files -- Create: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\stock_events.go` — introduce the generic stock-event model and provider interface shared by the SSI provider and command handler. -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi.go` — add the additive generic SSI fetch method and generic row normalizer while keeping dividend normalization intact. -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi_test.go` — cover generic pagination/dedup/range/order behavior and explicit dividend-regression expectations. - -## Implementation Steps -1. Define a generic read-only event type and provider interface separate from `DividendEvent` so `/stock_portfolio` persistence rules are not broadened. -2. Add an additive SSI fetch method that reuses `fetchPage`, `startOfSaigonDay`, `parseSSIOptionalDate`, `parseSSIDate`, and `eventSourceURL` instead of introducing a second transport/client stack. -3. Normalize every matching SSI row into a generic display model with title fallback order `EventTitle -> EventDescription -> EventName -> EventListCode`, while requiring only data needed for chronological display. -4. Keep `FetchDividendEvents` and `normalizeEvent` behavior additive-only: no relaxed dividend validation for value/ratio fields, no changes to portfolio history serialization. -5. Add provider tests for cross-page deduplication, one-day overlap filtering, deterministic ordering, generic inclusion of non-dividend corporate actions, and a regression assertion that dividend discovery still excludes non-dividend rows. - -## Success Criteria -- [x] Generic SSI fetch returns all eligible corporate actions in deterministic chronological order for one ticker and lookback window. -- [x] Generic fetch propagates upstream/paging/decode errors to the caller instead of silently succeeding with partial pages. -- [x] Existing dividend-provider tests still pass with no behavior drift in `/stock_portfolio` discovery. - -## Risk Assessment -- High: weakening dividend validation would change `/stock_portfolio` suggestions and history merges. Mitigation: additive generic method only; keep dividend-specific tests green before Phase 2. -- Medium: malformed SSI rows could create unstable or unsortable output. Mitigation: require a usable publish timestamp and provider ID; skip rows that cannot be placed chronologically. -- Medium: over-refactoring the proven dividend loop increases blast radius. Mitigation: reuse helper functions first; duplicate only the page-scan control flow if that keeps the dividend path unchanged. -- Rollback: revert the generic provider additions and tests only. No stored data or public command contract exists yet, so rollback is code-only and isolated. diff --git a/plans/260723-0852-stock-events-command/phase-02-add-telegram-command.md b/plans/260723-0852-stock-events-command/phase-02-add-telegram-command.md deleted file mode 100644 index f453363..0000000 --- a/plans/260723-0852-stock-events-command/phase-02-add-telegram-command.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -phase: 2 -title: "Add Telegram Command" -status: completed -priority: P2 -dependencies: [1] ---- - -# Phase 2: Add Telegram Command - -## Overview -Expose `/stock_events [days]` as a public, read-only Telegram command. Keep it independent from sender-specific portfolio state and produce deterministic, Telegram-safe SSI event output. - -## Requirements -- Functional: register public command metadata in `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\stock.go:13` with exact parameters ` [days]` and a concise description compatible with `/help` and Telegram’s native menu (`C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\command_presentation.go:6`, `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\validate.go:14`). -- Functional: accept exactly 1 required arg plus 1 optional days arg; default days to 30; allow only whole numbers `1..90`; reject missing or extra args with exact usage text. -- Functional: normalize ticker with `normalizeStockSymbol` at `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\symbols.go:18`; surface the same unknown-ticker wording used by current stock handlers at `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\handlers.go:93`. -- Functional: do not require `senderInfo`, `LoadPortfolio`, `PendingDividendStore`, or any write path from `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\handlers.go:59` and `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\portfolio.go:64`. -- Functional: respond cleanly for no events, upstream/provider failure, and parse failure. -- Non-functional: preserve provider order, bound every message below the repo’s 4000-char Telegram safety margin used at `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\handlers.go:480` and `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stats\views.go:17`. - -## Architecture -Data flow: -1. Dispatcher routes the public command to `handleStockEvents`. -2. The handler parses args, defaults/validates `days`, normalizes the ticker, and computes `after = now - days*24h`, `through = now`. -3. The handler calls the generic SSI provider from Phase 1 and receives chronologically sorted corporate actions. -4. Each event becomes a plain-text block with symbol, SSI type, bounded title, populated dates, and `SSI event: ` plus the raw source URL when present. -5. Blocks are packed into `<=4000`-char replies without splitting an event block; multi-message output adds deterministic part numbering. If one block still exceeds budget after field truncation, hard-truncate the final line with `…(truncated)`. - -Formatting strategy: -- Reuse the package-level rune-safe truncation helper from `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_notifications.go:363` for long titles and labels. -- Prefer plain-text replies via `chathelper.Reply` instead of HTML tables so raw SSI URLs stay visible and chunking stays simple. -- Keep chronological ordering exactly as returned by the provider; do not re-sort by type or date label in the handler. - -## Related Code Files -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\handlers.go` — add state access to the generic provider if needed, but keep sender/store logic untouched for this command. -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\stock.go` — register `/stock_events` metadata and handler. -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\stock_events.go` — implement args parsing, provider call, event formatting, and reply chunking. -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\handlers_test.go` — add registration, usage, validation, no-sender, no-events, upstream-error, and chunking coverage. - -## Implementation Steps -1. Wire the state to a generic stock-event provider with a zero-write fallback path suitable for tests that do not initialize stores or sender data. -2. Register `stock_events` as `VisibilityPublic` with exact metadata: `Parameters: " [days]"`, description concise enough to stay well under the 256-rune menu limit. -3. Implement the command parser so `/stock_events TCB`, `/stock_events TCB 7`, and only those shapes succeed; anything else returns exact usage or range errors. -4. Implement the read-only handler using `s.now()` and the provider from Phase 1. Do not touch portfolio/dividend history, pending buttons, or locks. -5. Format one event block per SSI row with only required fields. Include date labels only when present so missing `RecordDate` or `PaymentDate` does not generate noisy placeholders. -6. Pack reply blocks into bounded Telegram messages, preserve order across chunks, and keep no-events/failure replies single-message and deterministic. -7. Add handler tests for default 30-day behavior, explicit day range, invalid day values, extra args, senderless execution, provider window propagation, empty results, provider error mapping, and over-budget output chunking. - -## Success Criteria -- [x] `/stock_events [days]` is publicly registered with exact parameters/usage text and no sender requirement. -- [x] The handler produces deterministic chronological output, friendly empty/error replies, and message chunks below the Telegram safety budget. -- [x] The command path is provably read-only: no portfolio loads, pending-action writes, or dividend-history mutations are required for success. - -## Risk Assessment -- High: oversized replies can be rejected by Telegram or split mid-event. Mitigation: chunk by whole event blocks against a 4000-char budget and add an oversized synthetic test. -- Medium: introducing a hidden sender or store dependency would break channel/anonymized use. Mitigation: add a senderless test case and keep the handler isolated from `senderInfo`. -- Medium: metadata drift between handler usage and registration breaks `/help` and the native menu. Mitigation: assert the exact `Parameters` string and usage text in tests. -- Rollback: remove the new command registration and `stock_events` handler changes. Phase 1’s additive provider can remain unused or be reverted separately without data cleanup. diff --git a/plans/260723-0852-stock-events-command/phase-03-verify-and-document.md b/plans/260723-0852-stock-events-command/phase-03-verify-and-document.md deleted file mode 100644 index cce5bf6..0000000 --- a/plans/260723-0852-stock-events-command/phase-03-verify-and-document.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -phase: 3 -title: "Verify and Document" -status: completed -priority: P2 -dependencies: [1, 2] ---- - -# Phase 3: Verify and Document - -## Overview -Lock the public contract, prove dividend behavior did not regress, and update user-facing docs. This phase is the quality gate before implementation is considered done. - -## Requirements -- Functional: document `/stock_events [days]`, default 30 days, valid range `1..90`, and read-only SSI behavior in `C:\Users\miti99\Workspaces\tiennm99\miti99bot\README.md:22`. -- Functional: keep existing dividend event behavior unchanged for `/stock_portfolio`, including SSI overlap/refetch behavior already covered in `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_flow_test.go:94`, `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_flow_test.go:154`, and `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_flow_test.go:491`. -- Non-functional: run the repo gates required by `AGENTS.md`: `gofmt`, focused stock tests, `go test ./...`, `go vet ./...`, `go build ./...`, and `golangci-lint run` when the binary is available. - -## Architecture -This phase does not introduce runtime behavior. It validates that: -1. command metadata matches handler usage and README wording; -2. the new generic SSI path did not alter dividend discovery or notification flows; -3. repo-wide compilation, tests, and vet/lint still pass after the additive command lands. - -Test matrix: -- Unit: generic SSI normalization/order tests, days parser, reply chunking. -- Integration: registration metadata, handler no-events/error/senderless behavior, provider window propagation. -- Regression: dividend SSI/provider tests plus `dividend_flow_test.go` scenarios. -- Repository: full `go test ./...`, `go vet ./...`, `go build ./...`, optional `golangci-lint run`. - -## Related Code Files -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\README.md` — add the new command contract beside the stock command documentation. -- Verify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\handlers_test.go` — command registration/usage/read-only coverage. -- Verify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi_test.go` — generic SSI fetch plus dividend-regression coverage. -- Verify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_flow_test.go` — unchanged `/stock_portfolio` dividend notifications. - -## Implementation Steps -1. Update the README stock section with the exact command syntax, default day window, allowed range, and SSI-source caveat so docs match the public command contract. -2. Run focused stock-module tests first to catch fast feedback on provider normalization, handler parsing, chunking, and dividend regressions. -3. Run `gofmt` on touched Go files, then execute `go test ./...`, `go vet ./...`, and `go build ./...`. -4. Run `golangci-lint run` if the binary is available locally; if it is unavailable, record that explicitly rather than silently skipping it. -5. If any gate fails because of the new command/provider work, fix the owning phase before marking this plan complete. - -## Success Criteria -- [x] README, command metadata, and handler usage text all match `/stock_events [days]` exactly. -- [x] Focused stock tests cover the new command and confirm `/stock_portfolio` dividend behavior is unchanged. -- [x] Repo-wide test, vet, and build gates pass; lint status is either passing or explicitly reported as unavailable/pre-existing. - -## Risk Assessment -- Medium: docs can drift from the exact `Parameters` string and create `/help` confusion. Mitigation: compare README text against the registered `Parameters` and handler usage string before closing. -- Medium: additive SSI changes can still regress dividend behavior indirectly. Mitigation: rerun the provider/dividend-flow suites, not just the new command tests. -- Low: repo-wide gates may surface unrelated pre-existing failures. Mitigation: separate new failures from baseline issues and avoid hiding them behind the feature summary. -- Rollback: revert the README and command changes if validation reveals an unacceptable regression. No data migration or cleanup step is needed. diff --git a/plans/260723-0852-stock-events-command/plan.md b/plans/260723-0852-stock-events-command/plan.md deleted file mode 100644 index 0b9cd1b..0000000 --- a/plans/260723-0852-stock-events-command/plan.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -title: Stock Events Command -description: >- - Add a public /stock_events command for read-only SSI corporate action lookup - with bounded Telegram output. -status: completed -priority: P2 -effort: 8h -branch: main -tags: - - feature - - backend - - telegram - - stock -blockedBy: [] -blocks: [] -created: 2026-07-23T00:00:00.000Z ---- - -# Stock Events Command - -## Overview - -Add public `/stock_events [days]` for read-only SSI corporate-action lookup. Reuse SSI pagination helpers from `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events_ssi.go:113` without changing the dividend-history path used by `/stock_portfolio` (`C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_notifications.go:56`). - -## Scope Challenge - -- Existing code: command metadata/help comes from `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\stock.go:13`, `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\command_presentation.go:6`, and `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\validate.go:14`; ticker normalization already exists at `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\symbols.go:18`. -- Minimum change: add one parallel read-only SSI event path plus one public command; no portfolio writes, no dividend buttons, no stats/storage migration. -- Complexity: 7 touched files, 1 new command, 0 schema changes, 3 sequential phases. - -## Phases - -| Phase | Name | Status | -|-------|------|--------| -| 1 | [Extend SSI Corporate Events](./phase-01-extend-ssi-corporate-events.md) | Completed | -| 2 | [Add Telegram Command](./phase-02-add-telegram-command.md) | Completed | -| 3 | [Verify and Document](./phase-03-verify-and-document.md) | Completed | - -## Dependencies - -- Cross-plan: None. `plans\260722-1114-compact-portfolio-numbers\plan.md`, `plans\260722-1356-per-user-dividend-history\plan.md`, `plans\260722-1705-mobile-portfolio-columns\plan.md`, and `plans\260722-1843-retire-stock-dividend-command\plan.md` are already completed. -- Phase 2 depends on Phase 1's additive SSI corporate-action fetch contract. -- Phase 3 depends on Phases 1-2 for command behavior, regression tests, and docs alignment. - -## Compatibility - -- Keep dividend-only contracts unchanged: `DividendEventProvider` stays specific to `/stock_portfolio` at `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\dividend_events.go:12`, retained history stays under `Portfolio.Dividends` at `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\portfolio.go:44`. -- No Mongo/index/system-state/stats work. Out-of-scope items remain untouched. - -## Validation - -- Unit: SSI generic normalization, overlap/date filtering, deterministic order, days parsing, and reply chunking/truncation. -- Integration: command registration metadata, usage/error replies, senderless read-only execution, no-events path, upstream failure path, and dividend regressions. -- Repo gate: `gofmt`, focused stock tests, `go test ./...`, `go vet ./...`, `go build ./...`, and `golangci-lint run` when available. - -## Open Questions - -None. diff --git a/plans/260723-0852-stock-events-command/reports/pm-260723-1003-stock-events-command.md b/plans/260723-0852-stock-events-command/reports/pm-260723-1003-stock-events-command.md deleted file mode 100644 index 9f59ce2..0000000 --- a/plans/260723-0852-stock-events-command/reports/pm-260723-1003-stock-events-command.md +++ /dev/null @@ -1,42 +0,0 @@ -# PM Completion Report - -Plan: `plans/260723-0852-stock-events-command` - -## Phases - -- Phase 1 `Extend SSI Corporate Events`: completed -- Phase 2 `Add Telegram Command`: completed -- Phase 3 `Verify and Document`: completed - -## Tests - -- Focused stock tests: pass -- Command menu discovery test: pass -- `go test -count=1 ./...`: pass -- `go test -race -count=1 ./internal/modules/stock`: pass -- `go vet ./...`: pass -- `go build ./...`: pass -- `golangci-lint run`: pass -- `git diff --check`: pass - -## Docs - -- `README.md` updated for `/stock_events [days]` -- Plan text updated to reflect raw SSI display fields and private cursor handling -- Phase checklists and status synced to completed - -## Limitation - -- SSI lookup remains best-effort because the upstream corporate-actions endpoint is undocumented and can change without notice. - -## Blockers - -- None - -## Open Questions - -- None - -## Mappings - -- No unresolved task-to-phase mappings. diff --git a/plans/260723-1017-stock-info-command/phase-01-model-ssi-quote-details.md b/plans/260723-1017-stock-info-command/phase-01-model-ssi-quote-details.md deleted file mode 100644 index 9bbfc55..0000000 --- a/plans/260723-1017-stock-info-command/phase-01-model-ssi-quote-details.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -phase: 1 -title: Model SSI Quote Details -status: completed -priority: P2 -dependencies: [] ---- - -# Phase 1: Model SSI Quote Details - -## Overview - -Extend the existing SSI single-stock response with a read-only detailed quote -model and a dedicated one-request fetch method. Do not change `FetchPrice`. - -## Requirements - -- Functional: capture symbol, Vietnamese/English company name, exchange, - matched/open/reference/high/low prices, and normal traded quantity from SSI. -- Functional: perform one GET to `/stock/` with existing SSI headers. -- Functional: reject missing/non-positive matched price as `ErrNoPrice`. -- Non-functional: preserve KBS/VCI/SSI fallback behavior for `FetchPrice` and - batch portfolio pricing. - -## Architecture - -Add an SSI-specific quote DTO and dedicated fetch method on `PriceClient`. -Reuse `newSSIRequest`, `doSSIJSON`, `baseURL`, and the shared HTTP client. -The command receives the single decoded response directly; it never calls -`FetchPrice`. - -## Related Code Files - -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\prices_ssi.go` -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\prices_test.go` - -## Implementation Steps - -1. Extend the SSI quote struct with only the fields used by `/stock_info`. -2. Add a dedicated detailed-quote fetch method that issues one GET. -3. Keep current `fetchSSIPrice`, `FetchPrice`, and `FetchPrices` contracts intact. -4. Test path, headers, one-call count, decoded fields, invalid price, HTTP, and decode errors. - -## Success Criteria - -- [x] Detailed quote fetch uses exactly one SSI GET. -- [x] Required and optional fields decode without inventing fallback requests. -- [x] Existing price and batch provider tests remain unchanged and pass. - -## Risk Assessment - -SSI is undocumented and may omit fields. Keep optional fields nullable by value -and make the handler degrade to `N/A`; isolate the new method from trading and -portfolio callers. diff --git a/plans/260723-1017-stock-info-command/phase-02-add-stock-info-command.md b/plans/260723-1017-stock-info-command/phase-02-add-stock-info-command.md deleted file mode 100644 index bf3b730..0000000 --- a/plans/260723-1017-stock-info-command/phase-02-add-stock-info-command.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -phase: 2 -title: "Add Stock Info Command" -status: completed -priority: P2 -dependencies: [1] ---- - -# Phase 2: Add Stock Info Command - -## Overview - -Register `/stock_info ` and render a compact, sender-independent quote -snapshot from the Phase 1 SSI detail method. - -## Requirements - -- Functional: exact syntax `/stock_info ` and public metadata ``. -- Functional: normalize ticker; handle usage, unknown ticker, no price, upstream, - and decode failures with friendly replies. -- Functional: show Vietnamese company name with English fallback, exchange, - current price, since-open amount/percent, reference amount/percent, - open/high/low, and normal traded quantity. -- Non-functional: read-only, senderless, one SSI request, Telegram-safe text. - -## Architecture - -Create a narrow handler/formatter file. Calculate changes from matched price -against positive open/reference prices; otherwise show `N/A`. Reuse existing -VND, sign, and integer formatting where their output matches the command. - -## Related Code Files - -- Create: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\stock_info.go` -- Create: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\stock_info_test.go` -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\stock.go` -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\handlers_test.go` -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\cmd\server\command_menu_test.go` - -## Implementation Steps - -1. Implement exact argument validation and ticker normalization. -2. Fetch with `chathelper.FetchContext` through the one-call SSI detail method. -3. Format compact fields with signed amount/percent and `N/A` for unavailable comparisons. -4. Register the command and update shared command-discovery expectations. -5. Test calculations, signs, missing fields, company fallback, senderless use, - request count, usage, no-price, and upstream errors. - -## Success Criteria - -- [x] `/stock_info TCB` returns the agreed compact quote in one SSI call. -- [x] Since-open and reference changes are mathematically correct and signed. -- [x] `/stock_price` registration, output, and fallbacks remain unchanged. - -## Risk Assessment - -Avoid calling `FetchPrice` from the new handler because it can trigger multiple -providers. Keep the detailed quote method private to the new command path. diff --git a/plans/260723-1017-stock-info-command/phase-03-verify-and-document.md b/plans/260723-1017-stock-info-command/phase-03-verify-and-document.md deleted file mode 100644 index f45bc22..0000000 --- a/plans/260723-1017-stock-info-command/phase-03-verify-and-document.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -phase: 3 -title: "Verify and Document" -status: completed -priority: P2 -dependencies: [1, 2] ---- - -# Phase 3: Verify and Document - -## Overview - -Align public documentation and prove the new one-call command does not change -existing quote, portfolio, event, or dividend behavior. - -## Requirements - -- Functional: document syntax, compact fields, one-call SSI-only behavior, and - best-effort upstream limitation in README. -- Non-functional: satisfy focused, repository, race, vet, build, lint, and diff gates. - -## Architecture - -No new runtime behavior. Verify command metadata/usage/docs as one contract and -exercise existing `/stock_price` plus portfolio quote callers as regressions. - -## Related Code Files - -- Modify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\README.md` -- Verify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\prices_test.go` -- Verify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\internal\modules\stock\stock_info_test.go` -- Verify: `C:\Users\miti99\Workspaces\tiennm99\miti99bot\cmd\server\command_menu_test.go` - -## Implementation Steps - -1. Update README with `/stock_info ` and its SSI-only behavior. -2. Run `gofmt` and focused stock/server command-discovery tests. -3. Run `go test -count=1 ./...` and stock race tests. -4. Run `go vet ./...`, `go build ./...`, `golangci-lint run`, and `git diff --check`. -5. Review all price callers and public contracts for unintended changes. - -## Success Criteria - -- [x] README, registration metadata, handler usage, and tests match exactly. -- [x] All focused and repository gates pass with no `/stock_price` regression. -- [x] Review confirms one SSI call and no storage/schema/provider-fallback changes. - -## Risk Assessment - -The SSI endpoint is undocumented. Document best-effort behavior and keep the -new surface additive so it can be removed without migration or data cleanup. diff --git a/plans/260723-1017-stock-info-command/plan.md b/plans/260723-1017-stock-info-command/plan.md deleted file mode 100644 index 9e3105f..0000000 --- a/plans/260723-1017-stock-info-command/plan.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -title: Stock Info Command -description: >- - Add a one-call SSI quote-detail command while preserving /stock_price and its - provider fallbacks. -status: completed -priority: P2 -branch: main -tags: - - feature - - stock - - telegram - - api -blockedBy: [] -blocks: [] -created: '2026-07-23T03:17:04.304Z' -createdBy: 'ck:plan' -source: skill ---- - -# Stock Info Command - -## Overview - -Add public `/stock_info ` for a compact SSI quote snapshot. The command -uses exactly one SSI single-ticker GET and reports company, exchange, current -price, since-open and reference changes, open/high/low, and volume. Existing -`/stock_price`, batch quotes, portfolio valuation, and provider fallbacks stay -unchanged. - -## Phases - -| Phase | Name | Status | -|-------|------|--------| -| 1 | [Model SSI Quote Details](./phase-01-model-ssi-quote-details.md) | Completed | -| 2 | [Add Stock Info Command](./phase-02-add-stock-info-command.md) | Completed | -| 3 | [Verify and Document](./phase-03-verify-and-document.md) | Completed | - -## Dependencies - -- No cross-plan dependencies; all existing stock plans are completed. -- Reuse the current SSI request headers, client, timeout, and response envelope. -- No storage, portfolio, stats, event, dividend, migration, or environment changes. - -## Contract - -- Syntax: `/stock_info `. -- One SSI request only; never fall back to KBS or VCI. -- Missing optional quote fields render consistently as `N/A`. -- A missing/non-positive matched price returns a friendly no-info response. - -## Validation - -- Focused provider/handler/metadata tests plus `/stock_price` regression coverage. -- `go test -count=1 ./...`, stock race tests, `go vet ./...`, `go build ./...`, - `golangci-lint run`, and `git diff --check`. - -## Open Questions - -None. diff --git a/plans/260723-1017-stock-info-command/reports/pm-2026-07-23-1224-stock-info-command.md b/plans/260723-1017-stock-info-command/reports/pm-2026-07-23-1224-stock-info-command.md deleted file mode 100644 index ae155ea..0000000 --- a/plans/260723-1017-stock-info-command/reports/pm-2026-07-23-1224-stock-info-command.md +++ /dev/null @@ -1,33 +0,0 @@ -# PM Completion Report - -- Plan: `Stock Info Command` -- Plan status: `completed` -- Date: `2026-07-23 12:24` - -## Delivery - -- Phase 1: completed -- Phase 2: completed -- Phase 3: completed -- Plan summary aligned to verified implementation and harness evidence - -## Evidence - -- Focused stock and server tests passed -- Stock race tests passed -- Full repository test suite passed -- `go vet` passed -- `go build` passed -- `golangci-lint run` passed with 0 issues -- `git diff --check` passed -- Live SSI quote verification passed - -## Reconciliation - -- No unresolved phase mappings -- No blockers -- No docs gap recorded in plan artifacts - -## Next Step - -- Main agent: finish any remaining implementation-plan closure work, then handle commit and push in the normal ship flow if still desired. diff --git a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-01-feasibility-gate.md b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-01-feasibility-gate.md deleted file mode 100644 index abe5df5..0000000 --- a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-01-feasibility-gate.md +++ /dev/null @@ -1,100 +0,0 @@ ---- -phase: 1 -title: "Feasibility Gate" -status: pending -effort: "S" -priority: P1 -dependencies: [] ---- - -# Phase 1: Feasibility Gate - -## Overview - -Answer the questions research could not, because Fandom rate-limited then refused the connection. **This phase can cancel the plan.** No production code until every check below passes. - -## Why This Exists - -Research (`plans/reports/260726-0941-lol-score-source-alternatives.md`) established that Leaguepedia is the only viable free source with independent ingestion, and that its `MatchSchedule` table exposes `Team1Score`/`Team2Score`/`Winner`. It could **not** establish that Leaguepedia actually has the LEC/LCS scores lolesports is missing, nor how fast its editors are. Those are the load-bearing assumptions. - -Leaguepedia certainly *covers* LEC/LCS/LPL/LCK — it is the reference wiki. The open risks are latency, join viability, and access. - -## Checks - -Run against the known-bad matches (lolesports has these as `completed` with `0-0`): - -| startTime (UTC) | league | teams | -|---|---|---| -| 2026-07-25T14:30Z | lec | G2 Esports vs Team Vitality | -| 2026-07-25T17:30Z | lec | Movistar KOI vs Karmine Corp | -| 2026-07-25T19:00Z | cblol-brazil | LOUD vs paiN Gaming | -| 2026-07-25T20:00Z | lcs | FlyQuest vs LYON | -| 2026-07-25T23:00Z | lcs | Dignitas vs Sentinels | - -Baseline query (proven to work during research): - -```bash -curl -s -G "https://lol.fandom.com/api.php" \ - -A "miti99bot/0.1 (+https://github.com/tiennm99/miti99bot)" \ - --data-urlencode "action=cargoquery" \ - --data-urlencode "format=json" \ - --data-urlencode "tables=MatchSchedule=MS" \ - --data-urlencode "fields=MS.DateTime_UTC,MS.Team1,MS.Team2,MS.Team1Score,MS.Team2Score,MS.Winner,MS.BestOf,MS.OverviewPage" \ - --data-urlencode "where=MS.DateTime_UTC >= '2026-07-25 14:00:00' AND MS.DateTime_UTC < '2026-07-26 00:00:00'" \ - --data-urlencode "order_by=MS.DateTime_UTC" \ - --data-urlencode "limit=100" -``` - -### Gate 1 — Data exists (BLOCKING) - -- [ ] At least 4 of the 5 matches above return a row with non-empty `Team1Score`/`Team2Score` -- [ ] Scores are plausible for the Bo3 (2-0/2-1, not 0-0) -- [ ] For `Movistar KOI vs Karmine Corp`, score is `2-0` or `0-2` — VOD evidence from the debug session showed exactly 2 games played, game 3 `unneeded` - -**If fewer than 4 of 5 have scores → STOP. Cancel plan.** Leaguepedia is no faster than Riot and the enrichment buys nothing. - -### Gate 2 — Name join works (BLOCKING) - -Compare Leaguepedia `Team1`/`Team2` against Riot `Team.Name` for the same matches. - -- [ ] Riot `Team.Name` values match Leaguepedia `Team1`/`Team2` exactly, OR differ only by case/punctuation/whitespace -- [ ] Record every mismatch verbatim — these define the normalization rules for Phase 3 -- [ ] Confirm no two matches in a single day-window share the same normalized name pair (would make the join ambiguous) - -Known naming risk to check specifically: `paiN Gaming` (Riot code `PAIN`) and `LYON` — irregular capitalization. - -**If names diverge structurally (e.g. Leaguepedia uses `Movistar KOI (European Team)` disambiguators) → note it; Phase 3 gains a disambiguator-stripping rule. Not fatal, but scope grows.** - -### Gate 3 — Editorial latency (INFORMATIONAL, shapes value) - -- [ ] Note wall-clock lag between match end and Leaguepedia having the score, for at least 2 matches -- [ ] If Leaguepedia is consistently *slower* than Riot's backfill, the feature is near-worthless even if Gates 1-2 pass → report and ask before continuing - -### Gate 4 — Access viability (BLOCKING) - -- [ ] A single cold query succeeds without prior warmup (research only succeeded on retry 3 of 3 after bursts) -- [ ] Determine whether the throttle is per-IP burst-based or a hard quota — space 3 queries 60s apart and confirm all 3 succeed -- [ ] Confirm no API key or login is required for `MatchSchedule` reads - -**If a lone spaced query cannot reliably succeed → STOP or pivot to Liquipedia LPDB** (free tier, needs key + manual approval; repo is public so it likely qualifies). - -### Gate 5 — Licensing - -- [ ] Confirm CC-BY-SA 3.0 attribution requirement and decide placement (Phase 4 handles it — likely a line in `/lol` help or the digest footer) - -## Deliverable - -Append findings to `plans/reports/260726-0941-lol-score-source-alternatives.md` (do not create a second report — the unresolved-questions section there is exactly what this phase closes). Record: raw JSON for one match, the exact name pairs observed, throttle behaviour, and a GO/NO-GO. - -## Success Criteria - -- [ ] Gates 1, 2, 4 pass → GO, proceed to Phase 2 -- [ ] Gate 3 measured and acceptable -- [ ] Any gate fails → NO-GO recorded with evidence; plan marked `cancelled`; report back before writing code -- [ ] Normalization rules for Phase 3 written down from real observed data, not guessed - -## Risk Assessment - -**Biggest risk: doing Phases 2-4 first and discovering Leaguepedia is also stale.** That is the entire reason this phase is a hard gate rather than advisory. - -Second risk: verifying against *this* stall only. One stall is a sample of one. If Gate 3 shows Leaguepedia leading Riot by hours here, that is suggestive, not proof of steady-state behaviour. Note the caveat rather than over-claiming. diff --git a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-02-cargo-client.md b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-02-cargo-client.md deleted file mode 100644 index 2125e27..0000000 --- a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-02-cargo-client.md +++ /dev/null @@ -1,137 +0,0 @@ ---- -phase: 2 -title: "Cargo Client" -status: pending -effort: "M" -priority: P2 -dependencies: [1] ---- - -# Phase 2: Cargo Client - -## Overview - -A minimal read-only client for Leaguepedia's Cargo API returning finished-match scores in a UTC time window. Mirrors the existing `Client` shape in `api_client.go` so tests can inject an `httptest.Server`. - -## Requirements - -Functional: -- Fetch `MatchSchedule` rows for `[from, to)` UTC -- Return team names, series scores, Bo count, scheduled time -- Skip rows without a usable score (unplayed / in-progress future matches) - -Non-functional: -- Disabled by default; single env var enables -- Bounded timeout, no retries (a miss is not worth a second request against a throttling host) -- Descriptive User-Agent with contact URL — required by wiki API etiquette -- Never returns an error the caller must surface to the user; caller treats all failures as "no data" - -## Architecture - -New file `internal/modules/lol/leaguepedia_client.go`. - -```go -// leaguepediaEnabledEnv gates the whole feature. Absent or not "1"/"true" → no -// requests are ever made. -const leaguepediaEnabledEnv = "LOL_LEAGUEPEDIA_ENABLED" - -const ( - leaguepediaURL = "https://lol.fandom.com/api.php" - // leaguepediaUserAgent identifies the bot per wiki API etiquette; Fandom - // throttles anonymous traffic aggressively and an honest UA with contact - // info is the minimum courtesy for a free source. - leaguepediaUserAgent = "miti99bot/0.1 (+https://github.com/tiennm99/miti99bot)" - leaguepediaTimeout = 6 * time.Second - // leaguepediaRowLimit caps a day/week window. A dense week across all - // leagues (incl. amateur tiers we filter out) can exceed 100 rows. - leaguepediaRowLimit = 500 -) - -// SeriesResult is one finished series as Leaguepedia records it. Team names are -// full names ("Movistar KOI"), matching Riot's Team.Name — see Phase 3 join. -type SeriesResult struct { - StartTime time.Time - Team1 string - Team2 string - Team1Wins int - Team2Wins int -} - -type leaguepediaClient struct { - HTTP *http.Client - URL string // test override; empty → leaguepediaURL -} - -// leaguepediaEnabled reports whether the operator opted in. -func leaguepediaEnabled() bool - -// FetchResults returns finished series in [from, to). Returns (nil, nil) when -// disabled — callers treat empty and error identically, so there is no -// "disabled" error to special-case. -func (c *leaguepediaClient) FetchResults(ctx context.Context, from, to time.Time) ([]SeriesResult, error) -``` - -### Cargo response shape - -Rows arrive wrapped, with **spaces** in field names (Cargo replaces `_`): - -```json -{"cargoquery":[{"title":{ - "DateTime UTC":"2026-07-25 17:30:00", - "Team1":"Movistar KOI","Team2":"Karmine Corp", - "Team1Score":"2","Team2Score":"0", - "Winner":"1","BestOf":"3"}}]} -``` - -Two decoding traps, both must be handled: -- `DateTime UTC` and `Team1Score` are **strings**, not numbers → decode into `string` and `strconv.Atoi` -- Errors come back HTTP 200 with `{"error":{"info":"You've exceeded your rate limit..."}}` → must check for an `error` key, not just the status code - -Timestamp format is `2006-01-02 15:04:05`, no zone suffix, always UTC. - -## Related Code Files - -- Create: `internal/modules/lol/leaguepedia_client.go` -- Create: `internal/modules/lol/leaguepedia_client_test.go` -- Reference (patterns to copy, do not modify): `internal/modules/lol/api_client.go` (Client/HTTP/URL shape, `truncate` for log bounding), `internal/modules/misc/wheelofnames_api_client.go:62` (env const + `os.Getenv`) - -## Implementation Steps - -1. Add env gate `leaguepediaEnabled()` — `strings.TrimSpace(os.Getenv(...))` matched against `"1"`/`"true"`, so an empty or garbage value is safely off -2. Add `httpClient()`/`baseURL()` helpers mirroring `api_client.go:133-145` -3. Build the query with `url.Values` — `action=cargoquery`, `format=json`, `tables=MatchSchedule=MS`, the field list, `where` bounded by `from`/`to`, `order_by`, `limit` -4. Set `User-Agent` and `Accept`; issue GET with the request context -5. Decode into a struct with an `Error *struct{ Info string }` field; if present, log at warn with `truncate(...)` and return an error -6. Map rows → `[]SeriesResult`, skipping any row where either score fails to parse or `Team1Wins == 0 && Team2Wins == 0` (unplayed — same "absent vs zero" trap as the original bug; do not import a fabricated 0-0 from a *second* source) -7. Parse `DateTime UTC` with `time.ParseInLocation(..., time.UTC)`; skip unparseable rows - -## Tests - -All via `httptest.Server`, no network: - -- [ ] Happy path: 3 rows → 3 `SeriesResult` with correct scores and UTC times -- [ ] Disabled env → returns `(nil, nil)` and **makes zero HTTP requests** (assert with a request counter on the test server) -- [ ] Rate-limit body (`HTTP 200` + `{"error":{"info":"..."}}`) → error, no partial results -- [ ] Non-2xx status → error -- [ ] Malformed JSON → error -- [ ] Row with `Team1Score:""` → skipped, siblings still returned -- [ ] Row with `Team1Score:"0", Team2Score:"0"` → skipped (unplayed guard) -- [ ] Row with unparseable `DateTime UTC` → skipped, siblings returned -- [ ] Request assertions: `where` contains both bounds, UA is set, `limit` present -- [ ] Context cancellation propagates - -## Success Criteria - -- [ ] `go test ./internal/modules/lol/ -run Leaguepedia` green -- [ ] Zero network access in tests -- [ ] Zero HTTP requests when disabled -- [ ] No change to any existing file — this phase is purely additive -- [ ] `go vet ./...` clean - -## Risk Assessment - -**Silent schema drift.** Cargo field names with spaces are easy to typo into always-empty strings, which would look like "Leaguepedia has no data" rather than a bug. Mitigate by asserting exact parsed values in the happy-path test using a fixture captured verbatim in Phase 1, not hand-written JSON. - -**Importing another fabricated 0-0.** Step 6's guard is the whole point — this plan exists because a 0-0 was trusted once already. - -**Throttling in production.** Out of scope here (client makes one request when asked); Phase 4 owns call-site frequency. diff --git a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-03-score-join.md b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-03-score-join.md deleted file mode 100644 index 6ae0e93..0000000 --- a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-03-score-join.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -phase: 3 -title: "Score Join" -status: pending -effort: "M" -priority: P2 -dependencies: [2] ---- - -# Phase 3: Score Join - -## Overview - -Pure functions that decide which lolesports events need a score, and merge Leaguepedia rows into them. No I/O — this is the correctness core and must be exhaustively unit-tested. - -## Requirements - -- Identify events needing enrichment: `State == "completed" && !scoreIsPublished(t1, t2)` -- Match each to at most one `SeriesResult` -- Fill `GameWins` + `Outcome` so `formatEventLine` renders `✅` naturally, with the winner bolded -- Refuse ambiguous matches rather than guess -- Leave everything else byte-identical - -## Architecture - -New file `internal/modules/lol/score_enrich.go`. Reuses `scoreIsPublished` from `format.go` — that predicate already encodes exactly "has upstream published a usable score", so the trigger needs no new logic. - -```go -// needsScore reports whether an event is a finished series with no published -// score — the only shape this enrichment may touch. -func needsScore(e ScheduleEvent) bool - -// normalizeTeamName folds the cosmetic differences between Riot's Team.Name and -// Leaguepedia's Team1/Team2 into a comparable key. Rules are derived from the -// real pairs recorded in Phase 1, not invented here. -func normalizeTeamName(s string) string - -// mergeScores returns events with missing scores filled from results. Input is -// never mutated; a copy is returned. Unmatched, ambiguous, and already-scored -// events pass through untouched. Reports how many were filled, for logging. -func mergeScores(events []ScheduleEvent, results []SeriesResult) ([]ScheduleEvent, int) -``` - -### Match key - -`(normalizeTeamName(team1), normalizeTeamName(team2), UTC date of StartTime)` - -Date rather than exact instant: Leaguepedia records *scheduled* time, which can drift from Riot's by minutes when a broadcast slips. Same-day plus an exact team pair is specific enough — two teams do not play the same opponent twice in a day in these leagues. - -Orientation: try `(t1, t2)`; if no hit, try `(t2, t1)` and **swap the scores when applying**. Riot and Leaguepedia do not guarantee the same side ordering. Getting this backwards silently inverts every result, so it needs its own test. - -### Ambiguity rule - -Build the index as `map[key][]SeriesResult`. Apply only when `len(candidates) == 1`. Two or more → skip and log at warn. A wrong score is worse than `score pending`; that is the premise of the whole fix. - -### Applying a result - -```go -// Winner gets outcome "win", loser "loss", so formatEventLine bolds correctly. -// Outcome must be set: scoreIsPublished keys off it, so a fill that set only -// GameWins would still render as "score pending". -``` - -Edge case — a genuine draw or an unresolved series cannot occur here, because Phase 2 already dropped `0-0` rows. If both wins are equal and non-zero (a real Bo2 1-1), set both outcomes to `"loss"`... **decide during implementation**: LEC/LCS/LPL Bo2 splits exist. Safer: if `Team1Wins == Team2Wins`, fill `GameWins` on both and set both `Outcome` to `"tie"`. `formatEventLine` bolds only on `== "win"`, so a tie renders `✅ A 1–1 B` unbolded, which is correct. Add a test. - -## Related Code Files - -- Create: `internal/modules/lol/score_enrich.go` -- Create: `internal/modules/lol/score_enrich_test.go` -- Reference (do not modify): `internal/modules/lol/format.go` (`scoreIsPublished`, `declaredOutcome`, `seriesWins`) - -## Implementation Steps - -1. `needsScore` — thin wrapper over `State == "completed"` + `!scoreIsPublished(...)`; guard `len(Teams) >= 2` first (`format.go:126-131` tolerates short slices, so this must too) -2. `normalizeTeamName` — lowercase, trim, collapse internal whitespace, strip punctuation. Add a disambiguator strip (`" (…Team)"` suffix) **only if Phase 1 observed one** -3. Build the candidate index from `results` -4. Copy the events slice; deep-copy only the `Match.Teams` of events being modified (`Team.Result` is a pointer — mutating a shared `*TeamResult` would corrupt the caller's data and, worse, the cached payload) -5. For each event where `needsScore`, look up forward then reversed; apply on a unique hit -6. Return the copy plus a fill count - -## Tests - -- [ ] Fills a `{"outcome":null,"gameWins":0}` event → renders `✅ MKOI 2–0 KC` with `MKOI` via `formatEventLine` -- [ ] Reversed orientation: Leaguepedia has `KC vs MKOI 0-2` → still yields `MKOI 2–0 KC`, winner bolded correctly -- [ ] Already-scored event is **never** touched, even when a conflicting Leaguepedia row exists -- [ ] `unstarted` and `inProgress` events untouched -- [ ] No matching row → unchanged, still `☑️ … score pending` -- [ ] Two candidates for one key → unchanged + fill count 0 -- [ ] Name normalization: `"paiN Gaming"` vs `"PaiN Gaming"`, extra whitespace, punctuation -- [ ] Different UTC date, same teams → no match -- [ ] Tie (`1-1` Bo2) → both filled, neither bolded -- [ ] Event with <2 teams → no panic -- [ ] **Aliasing**: input events unchanged after `mergeScores` (assert the original `*TeamResult` pointers still read `Outcome == ""`) -- [ ] Empty results / empty events → no-op -- [ ] Schedule fields (`StartTime`, `Strategy.Count`, `BlockName`, `League`) identical before/after - -## Success Criteria - -- [ ] All tests green; existing `format_test.go` untouched and still passing -- [ ] `mergeScores` is pure — no I/O, no globals, no clock -- [ ] Input slice and its pointees provably unmutated -- [ ] Ambiguity and orientation both covered by dedicated tests - -## Risk Assessment - -**Pointer aliasing is the sharpest hazard.** `Team.Result` is `*TeamResult`, and enriched events may be the same objects written into the Mongo cache by `GetEventsWithFallback`. In-place mutation would persist a Leaguepedia-derived score into the lolesports cache, making a later cache read indistinguishable from real upstream data. Step 4 + the aliasing test exist for this. - -**Silent orientation inversion** would report every score backwards while looking perfectly healthy. Dedicated test. - -**Name drift over a season.** Rebrands mid-split break the join. Failure mode is benign (falls back to `score pending`), but log unmatched events at debug so it is diagnosable. diff --git a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-04-wire-in.md b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-04-wire-in.md deleted file mode 100644 index 6e91830..0000000 --- a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-04-wire-in.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -phase: 4 -title: "Wire-in" -status: pending -effort: "S" -priority: P2 -dependencies: [3] ---- - -# Phase 4: Wire-in - -## Overview - -Connect client + join to the four commands and the daily push, in one shared place. Add the CC-BY-SA attribution and document the env var. - -## Requirements - -- Every render path benefits: `/lol`, `/lol_tomorrow`, `/lol_this_week`, `/lol_next_week`, daily push -- **Zero** extra HTTP requests when nothing needs enrichment -- No new user-visible error path — enrichment failure is invisible except for the unchanged `☑️ score pending` -- One integration point, not five (DRY) - -## Architecture - -`replyForRange` (`handlers.go:96-124`) already funnels all four commands through fetch → filter → render. `runDailyPush` (`cron.go:174-253`) duplicates that sequence. So there are exactly **two** call sites. - -Insert between filter and render: - -```go -filtered := FilterMajor(events) -filtered = s.enrichScores(ctx, filtered, from, to) // new -text = renderDay(filtered, from, emptyLine) -``` - -```go -// enrichScores fills scores lolesports has not published, from Leaguepedia. -// Best-effort by design: any failure returns events unchanged so the digest -// still sends. Makes no request when nothing needs enrichment. -func (s *state) enrichScores(ctx context.Context, events []ScheduleEvent, from, to time.Time) []ScheduleEvent { - if !leaguepediaEnabled() || !slices.ContainsFunc(events, needsScore) { - return events - } - results, err := s.leaguepedia.FetchResults(ctx, from, to) - if err != nil { - log.Warn("lol_leaguepedia_fetch_fail", "err", err) - return events - } - merged, filled := mergeScores(events, results) - log.Info("lol_leaguepedia_enriched", "filled", filled, "candidates", ...) - return merged -} -``` - -The `ContainsFunc(events, needsScore)` short-circuit is what keeps steady-state cost at zero — on a normal day no event needs a score and no request is made. - -`state` gains `leaguepedia *leaguepediaClient`, wired in `New` (`lol.go:17-22`) beside `client: &Client{}`. - -### Attribution - -CC-BY-SA 3.0 obliges crediting Leaguepedia when its data is displayed. Cheapest honest placement: a footer line appended **only when at least one score was filled**, so unaffected digests stay clean. - -``` -Scores via Leaguepedia (CC BY-SA 3.0) -``` - -This makes attribution conditional on actual use — decide during implementation whether `renderDay`/`renderWeek` take a footer argument or the caller appends. Prefer the caller appending, to keep the renderers pure. - -## Related Code Files - -- Modify: `internal/modules/lol/handlers.go` (one call in `replyForRange`) -- Modify: `internal/modules/lol/cron.go` (one call in `runDailyPush`) -- Modify: `internal/modules/lol/lol.go` (wire `leaguepedia` into `state`) -- Modify: `docs/deploy-coolify-selfhosted.md` (add `LOL_LEAGUEPEDIA_ENABLED` to the env table at line ~28, `Required: optional`) -- Modify: `.env.example` (the docs table links to it, so both must stay in sync) -- Create: `internal/modules/lol/score_enrich_integration_test.go` (or extend `handlers_test.go`) - -## Implementation Steps - -1. Add `leaguepedia *leaguepediaClient` to `state` (`handlers.go:17-29`); wire in `New` -2. Add `enrichScores` method — put it in `score_enrich.go` next to `mergeScores`, not in `handlers.go`, keeping handlers thin -3. Call it in `replyForRange` after `FilterMajor` -4. Call it in `runDailyPush` after `FilterMajor`, **before** `claimDailyPush` — a Leaguepedia hiccup must not consume the day's idempotency claim, matching the existing rationale at `cron.go:195-197` -5. Conditional attribution footer when `filled > 0` -6. Document the env var; note default-off and the rate-limit rationale - -## Tests - -Existing `handlers_test.go` already injects a fake lolesports server; extend that harness with a fake Cargo server. - -- [ ] `/lol` with an unscored event + Leaguepedia having it → reply contains `✅` and the real score -- [ ] Same, Leaguepedia 500 → reply contains `☑️ … score pending`, **no error to user** -- [ ] Same, env disabled → `☑️ … score pending`, zero Cargo requests -- [ ] All events already scored → zero Cargo requests (assert counter == 0) -- [ ] Daily push enriches, and a Cargo failure still sends the digest -- [ ] Daily push: Cargo failure does **not** consume the push claim (second run still sends) -- [ ] Attribution footer present only when a score was filled -- [ ] Week view enriches across multiple days in one Cargo request (assert counter == 1) - -## Success Criteria - -- [ ] `go test ./...` fully green -- [ ] `go vet ./...` clean; changed files gofmt-clean modulo the repo's pre-existing CRLF -- [ ] Feature off by default — behaviour byte-identical to `570ee94` with the env var unset -- [ ] Zero Cargo requests in the steady state -- [ ] Attribution shown when data is used -- [ ] Manual check with the var set against the real API, confirming a real score renders - -## Risk Assessment - -**Regression surface is the four commands + daily push.** Mitigated by the default-off flag: with it unset the diff is inert, so a bad merge cannot degrade production until someone opts in. - -**Latency.** Adds up to `leaguepediaTimeout` (6s) to a `/lol` reply when a gap exists. Telegram tolerates this, and the short-circuit means it only happens on stall days. If it becomes annoying, cache — deliberately deferred per `plan.md` (the TTL index's partial filter is scoped to `matches:`, so a new prefix would never expire and needs its own index). - -**Push ordering.** Step 4 exists because enriching after the claim would let a Cargo failure burn the day's claim and silently skip the digest. diff --git a/plans/260726-0952-lol-leaguepedia-score-enrichment/plan.md b/plans/260726-0952-lol-leaguepedia-score-enrichment/plan.md deleted file mode 100644 index 8dc8d04..0000000 --- a/plans/260726-0952-lol-leaguepedia-score-enrichment/plan.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -title: "LoL Leaguepedia score enrichment" -description: "Fill missing series scores from Leaguepedia when lolesports marks a match completed but publishes no result" -status: cancelled -priority: P2 -branch: "main" -tags: [lol, external-api, enrichment] -blockedBy: [260805-1708-lol-pandascore-replacement] -blocks: [] -created: "2026-07-26T02:54:48.516Z" -createdBy: "ck:plan" -source: skill ---- - -# LoL Leaguepedia score enrichment - -> **Cancelled 2026-08-05.** Superseded by -> [260805-1708-lol-pandascore-replacement](../260805-1708-lol-pandascore-replacement/plan.md): -> the lol module's upstream moved to PandaScore, whose match `results` carry -> final series scores directly, absorbing this plan's enrichment job. The -> lolesports "completed without score" ingestion gap this plan targeted no -> longer applies to the new source. - -## Overview - -lolesports flips `event.state` to `completed` off the broadcast timeline but fills `result.outcome`/`result.gameWins` from a separate per-game ingestion path. When that path stalls, finished matches carry `{"outcome": null, "gameWins": 0}`. Commit `570ee94` made this render honestly as `☑️ … score pending`; this plan restores a real score by filling the gap from Leaguepedia. - -**Additive only.** lolesports stays authoritative for schedule, times, Bo count, block names, and any score it *does* publish. Leaguepedia is consulted solely for events where `scoreIsPublished(t1, t2) == false`. Every failure path — disabled, throttled, network error, ambiguous match, no row — falls through to today's `☑️ … score pending`. - -Evidence base: `plans/reports/260726-0941-lol-score-source-alternatives.md`. - -## Key Design Decisions - -**Join on team name + date, not team codes.** Riot's payload already carries `Team.Name` (`"Movistar KOI"`), and Leaguepedia's `MatchSchedule.Team1`/`Team2` are also full names. No code-mapping table, no `Teams` table query, no per-split `OverviewPage` string to maintain. This removes the largest maintenance risk identified in research. - -**Query by UTC time window, not by tournament.** One Cargo request per render covers every league in the window. Avoids `OverviewPage` naming (`LEC/2026 Season/Split 3`), which drifts every split. - -**No new persistence.** The existing TTL index (`startup.go:38-52`) has a partial filter scoped to `_id` in `[matches:, matches;)`, so any new cache prefix would never expire. Enrichment runs inline per render; the daily push is 1 request/day and `/lol*` commands are user-triggered and low-volume. Revisit only if rate limiting proves to bite. - -**Off by default.** Anonymous Fandom Cargo throttled hard during research (1 success in ~15 burst attempts, then connection refused). Ship behind `LOL_LEAGUEPEDIA_ENABLED`, matching the env-const + `os.Getenv` pattern in `internal/modules/misc/wheelofnames_api_client.go:62`. - -## Phases - -| Phase | Name | Status | -|-------|------|--------| -| 1 | [Feasibility Gate](./phase-01-feasibility-gate.md) | Pending | -| 2 | [Cargo Client](./phase-02-cargo-client.md) | Pending | -| 3 | [Score Join](./phase-03-score-join.md) | Pending | -| 4 | [Wire-in](./phase-04-wire-in.md) | Pending | - -Phase 1 is a **kill switch**, not a formality. It is cheap (a handful of curl calls) and can invalidate Phases 2-4 entirely. Do not write production code before it passes. - -## Architecture - -``` -replyForRange (handlers.go) ─┐ - ├─→ GetEventsWithFallback ─→ FilterMajor ─→ enrichScores ─→ render -runDailyPush (cron.go) ─┘ │ - │ only if any event has - │ state==completed && !scoreIsPublished - ▼ - leaguepediaClient.FetchResults(from, to) - │ - ┌──────────────┴──────────────┐ - 1 match found 0 or 2+ / error - │ │ - fill GameWins+Outcome leave untouched - ▼ ▼ - ✅ MKOI 2–0 KC ☑️ MKOI vs KC · score pending -``` - -Invariants (assert in tests): -- Never overwrites a score lolesports published -- Never mutates a non-`completed` event -- Never changes schedule fields (`StartTime`, `Strategy`, `BlockName`, `League`) -- Any error → input events returned unchanged, digest still sends - -## Acceptance Criteria - -- [ ] `✅ MKOI 2–0 KC · Bo3 (Week 1)` renders for a lolesports-unscored match that Leaguepedia has -- [ ] `☑️ … score pending` still renders when Leaguepedia lacks it, errors, is throttled, or is disabled -- [ ] Scores lolesports *does* publish are never altered -- [ ] Enrichment never blocks or fails a digest — no new error path reaches the user -- [ ] Zero extra HTTP requests when every completed event already has a score -- [ ] `go test ./...` green; no network access in tests (`httptest` only) -- [ ] Feature off by default; single env var flips it - -## Dependencies - -No cross-plan dependencies. All 8 existing plans are `completed` and scoped to the `stock` module. - -External: `lol.fandom.com` Cargo API (CC-BY-SA 3.0 — attribution obligation, see Phase 4). - -## Open Questions - -Tracked in Phase 1; all must be answered before Phase 2 starts. diff --git a/plans/260805-1708-lol-pandascore-replacement/phase-01-feasibility-and-mapping-discovery.md b/plans/260805-1708-lol-pandascore-replacement/phase-01-feasibility-and-mapping-discovery.md deleted file mode 100644 index e787c7a..0000000 --- a/plans/260805-1708-lol-pandascore-replacement/phase-01-feasibility-and-mapping-discovery.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -phase: 1 -title: Feasibility and mapping discovery -status: completed -effort: 'S — curl session only, no production code' ---- - -# Phase 1: Feasibility and mapping discovery - -## Overview - -Answer every open question with the live token before writing code. **Done -2026-08-05** — all questions answered, no assumption broke badly enough to -reshape Phase 2. ~7 quota requests used. Findings below supersede the -original question list. - -## Findings (verified live 2026-08-05) - -1. **Auth + quota.** Bearer header works; `HTTP 200`; - `x-rate-limit-remaining: 999` after first call → 1000/h free tier - confirmed. Token in repo `.env` as `LOL_PANDASCORE_TOKEN` (51 chars); - `.env` is NOT shell-sourceable (unquoted `&` on line 11) — Go reads env - directly, irrelevant for production. -2. **Window recipe.** - `GET /lol/matches?range[begin_at]=,&sort=begin_at&per_page=100` - → 77 matches for a 2-day window, ascending. **Upper range bound is - INCLUSIVE** (event at exactly `to` returned) → client-side `[from,to)` - filter is mandatory, keep it. Pagination: `Link` headers rel=next/last - AND `page=N` both available → walk `page` while `len==per_page`. - Response is a **top-level JSON array** (no envelope). -3. **Status vocabulary** (observed): `not_started`, `running`, `finished`. - Map to `unstarted`/`inProgress`/`completed`; **drop any other status** - (canceled, postponed) defensively. -4. **League slug mapping** (from `/lol/leagues`, 133 leagues, 3 pages): - - | canonical (format.go) | PandaScore slug | id | - |---|---|---| - | lck | league-of-legends-lck-champions-korea | 293 | - | lpl | league-of-legends-lpl-china | 294 | - | lec | league-of-legends-lec | 4197 | - | lcs | league-of-legends-lcs | 4198 | - | lcs | league-of-legends-lta-north | 5345 | (LTA-era NA top flight; slug canonicalized, display name passes through) | - | worlds | league-of-legends-world-championship | 297 | - | msi | league-of-legends-mid-invitational | 300 | - | first_stand | league-of-legends-first-stand | 5369 | - | ewc_lol | league-of-legends-esports-world-cup | 5262 | - | lcp | league-of-legends-lcp | 5351 | - | cblol-brazil | league-of-legends-cblol-brazil | 302 | - | emea_masters | league-of-legends-emea-masters | 4996 | - - Active majors verified in live windows: lck, lpl, lec, cblol-brazil, lcp. -5. **Field shapes.** - - `begin_at`: RFC3339 `Z` — parses with existing `time.RFC3339`. - - `results[]`: `{team_id, score}` present in ALL states (0-0 upcoming, - 1-1 running, final on finished) → live 🔴 scores work. Join to - opponents strictly by `team_id`. - - `winner_id`: non-null only when finished → Outcome win/loss derived - from `status==finished && winner_id != nil`; finished without winner_id - → no outcome → existing `☑️ score pending` path (semantics preserved). - - `number_of_games` → Strategy.Count; `opponents[].opponent.{acronym,name,image_url}` - → Team.Code/Name/Image. - - **BlockName ← `tournament.name`** ("Regular Season", "Group Ascend", - "Playoffs") — closest analogue to old block labels. - - `name` is "MCN vs BAN" — redundant, unused. -6. **TBD.** 0 matches with <2 opponents in 100-match upcoming sample; - mapping must still tolerate `len(opponents)<2` (distant playoffs) — - `formatEventLine`/`teamLabel` already render "TBD" for missing teams. -7. **ToS.** Docs state Fixtures-Only plan free for all users; no explicit - attribution requirement found in developer docs. Low risk; see open - question in plan.md. - -## Success Criteria - -- [x] 200 with token; quota headers recorded; free-tier headroom confirmed -- [x] Date-window recipe verified incl. pagination behavior -- [x] Status vocabulary + mapping decided -- [x] League mapping table fully filled -- [x] BlockName source + TBD shape decided -- [x] ToS/attribution answer recorded (docs level; full ToS text unread) -- [x] Phase 2 adjustments: none structural — confirmed top-level array, - inclusive upper bound, results-in-all-states, lta-north dual mapping - -## Risk Assessment (resolved) - -- Worlds/MSI are ordinary leagues (297/300) — no serie-based lookup needed. -- EWC/First Stand exist as leagues (5262/5369) — no gaps. diff --git a/plans/260805-1708-lol-pandascore-replacement/phase-02-pandascore-client-rewrite.md b/plans/260805-1708-lol-pandascore-replacement/phase-02-pandascore-client-rewrite.md deleted file mode 100644 index 1e9be26..0000000 --- a/plans/260805-1708-lol-pandascore-replacement/phase-02-pandascore-client-rewrite.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -phase: 2 -title: PandaScore client rewrite -status: completed -effort: 'M — transport swap + fixtures, same blast radius as b76d0ca' -dependencies: - - 1 ---- - -# Phase 2: PandaScore client rewrite - -## Overview - -Swap the transport in `internal/modules/lol/api_client.go` from the gql -persisted-query client to PandaScore REST, keeping `ScheduleEvent` and every -consumer (`format.go`, `handlers.go`, `cron.go`, cache) unchanged. Mirrors -the b76d0ca rewrite: transport + mapping only. - -## Requirements - -- Functional: same rendered output for day/week windows; scores on finished - series; major-league filtering intact via slug mapping from Phase 1. -- Non-functional: no live calls in tests; bounded timeouts (keep 8s); token - never logged; stale-cache fallback (60 min) preserved. - -## Architecture - -``` -GetEventsWithFallback / GetEventsLive (unchanged) - │ - ▼ -fetchEventsInRange(ctx, from, to, maxPages) (page-walk, [from,to) filter — unchanged shape) - │ - ▼ -fetchEventsPage(ctx, from, to, page) (NEW: GET api.pandascore.co/lol/matches - │ range[begin_at]=from,to & sort=begin_at - ▼ & per_page=100 & page=N -psMatch.toScheduleEvent() Authorization: Bearer $LOL_PANDASCORE_TOKEN) - │ - ▼ -ScheduleEvent (existing struct, bson cache shape untouched) -``` - -Mapping (finalize against Phase 1 findings): - -| ScheduleEvent | PandaScore | -|---|---| -| StartTime | `begin_at` (RFC3339) | -| State | `not_started→unstarted`, `running→inProgress`, `finished→completed`; drop canceled/postponed per Phase 1 | -| BlockName | source decided in Phase 1 (candidate `tournament.name`) | -| League.Slug/Name/Image | PandaScore league mapped → canonical slug via table; name/image passthrough | -| Match.Strategy.Count | `number_of_games` (Type: "bestOf") | -| Match.Teams[].Code/Name/Image | `opponents[].opponent.acronym/name/image_url` | -| Match.Teams[].Result | `results[]` matched to opponent by `team_id`; GameWins=score; Outcome win/loss from `winner_id` when `finished` | - -## Related Code Files - -- Modify: `internal/modules/lol/api_client.go` (transport, psMatch structs, - mapping, league slug table, token from env) -- Modify: `internal/modules/lol/api_client_test.go` (fixtures → PandaScore - shape; keep test matrix: cache-first, live-first, stale fallback, hard - failure, page walk, exact-window filter, non-JSON error; add: token-missing - error, canceled-match dropped, results/winner mapping) -- Modify: `internal/modules/lol/handlers_test.go` (`todayBody`/`futureBody` - fixtures → PandaScore shape) -- No changes: `format.go`, `handlers.go`, `cron.go`, `subscribers.go`, - `lol.go` wiring (Client zero-value still works; token read at request time) - -## Implementation Steps - -1. Define `psMatch`/`psOpponent`/`psResult`/`psLeague` structs + - `toScheduleEvent()` with the mapping table above. -2. Replace `fetchEventsPage`: build URL with `range[begin_at]`, `sort`, - `per_page`, `page`; Bearer header from `LOL_PANDASCORE_TOKEN` (read via - `os.Getenv` at call time, `strings.TrimSpace`d); keep `truncate`d body - logging on non-2xx; distinct log key `lol_token_missing` when env empty - (return error without calling upstream). -3. Rework `fetchEventsInRange` pagination: increment `page` while a full page - (`len==per_page`) returned, capped by maxPages; keep exact `[from,to)` - client-side filter (PandaScore range bounds semantics per Phase 1). -4. League slug mapping table (PandaScore slug → canonical) as package-level - map from Phase 1 findings; unmapped leagues pass through their PandaScore - slug (FilterMajor drops them naturally). -5. Update package doc comment: endpoint, auth, quota, mapping rationale, - token env var. -6. Rewrite test fixtures in PandaScore shape (raw JSON arrays — note: - `/lol/matches` returns a top-level array, not an envelope, per docs; - confirm in Phase 1). Keep httptest pattern; assert Bearer header + query - params in a request-inspection test. -7. Handle 401/403 (bad token) and 429 (quota) with clear log keys - (`lol_fetch` with status suffices; verify stale cache covers). -8. `gofmt`, `go test ./internal/modules/lol/`, then full gates. - -## Success Criteria - -- [ ] All existing test scenarios pass with PandaScore fixtures -- [ ] New tests: token-missing, canceled dropped, score/winner mapping, Bearer - header + range params asserted -- [ ] `rg lolesports internal/modules/lol` → only doc-comment history note or - nothing -- [ ] `go test ./...`, `go vet ./...`, `golangci-lint run` green - -## Risk Assessment - -- Top-level array vs envelope mismatch → caught by Phase 1 recipe. -- Opponent order vs results order mismatch → always join `results` by - `team_id`, never by index; test covers reversed order. -- Token leakage → never log request URL if token-in-URL auth is used; use - Bearer header only. diff --git a/plans/260805-1708-lol-pandascore-replacement/phase-03-wire-in-cleanup-and-supersede.md b/plans/260805-1708-lol-pandascore-replacement/phase-03-wire-in-cleanup-and-supersede.md deleted file mode 100644 index 798517e..0000000 --- a/plans/260805-1708-lol-pandascore-replacement/phase-03-wire-in-cleanup-and-supersede.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -phase: 3 -title: Wire-in cleanup and supersede -status: completed -effort: 'S — env docs, live smoke, plan bookkeeping' -dependencies: - - 2 ---- - -# Phase 3: Wire-in cleanup and supersede - -## Overview - -Deployment plumbing, live verification with the real token, documentation, -and cross-plan bookkeeping (cancel the superseded Leaguepedia plan). - -## Implementation Steps - -1. **Env plumbing.** Add `LOL_PANDASCORE_TOKEN` to the deployment environment - (wherever `GOLD_VNAPP_API_KEY` lives — user applies the secret; never - commit it). Confirm the bot process picks it up. -2. **Docs.** Update README / docs env-var table if one exists (check - `README.md`, `docs/`) with `LOL_PANDASCORE_TOKEN` (required for lol module - fetches; without it /lol replies degrade to fetch-error message). - Attribution note if Phase 1 ToS check requires it. -3. **Live smoke (temporary probe test, delete after):** today + this-week - windows through the real client; verify major leagues present, scores on - finished series, ICT rendering; check quota headers after the run. -4. **Supersede plan 260726-0952:** set `status: cancelled` in its `plan.md` - frontmatter + add one-line note: "Superseded by - 260805-1708-lol-pandascore-replacement — PandaScore results carry final - scores, absorbing this enrichment." Do not delete files. -5. **Full gates:** `go test ./...`, `go vet ./...`, `golangci-lint run`. -6. **Commit** (on user request, conventional message, e.g. - `feat(lol): replace schedule source with PandaScore API`). - -## Success Criteria - -- [ ] Live /lol and /lol_this_week verified with real token on the server -- [ ] Env var documented; secret nowhere in repo or logs -- [ ] Plan 260726-0952 marked cancelled with supersede pointer -- [ ] All gates green; probe test removed -- [ ] Commit created (user-approved) - -## Risk Assessment - -- Daily push (08:00 ICT cron) is the first unattended consumer — watch first - push after deploy; stale cache + error message paths already tested. -- Rollback: revert commit → b76d0ca gql client returns (its persisted ID may - need re-extraction per that commit's package doc). diff --git a/plans/260805-1708-lol-pandascore-replacement/plan.md b/plans/260805-1708-lol-pandascore-replacement/plan.md deleted file mode 100644 index be5033a..0000000 --- a/plans/260805-1708-lol-pandascore-replacement/plan.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -title: Replace LoL schedule source with PandaScore -description: >- - Swap the lol module's upstream from the lolesports.com gql persisted-query - client to PandaScore's REST API as the sole schedule/results source -status: completed -priority: P2 -branch: main -tags: - - lol - - external-api - - migration -blockedBy: [] -blocks: [] -created: '2026-08-05T10:12:46.240Z' -createdBy: 'ck:plan' -source: skill ---- - -# Replace LoL schedule source with PandaScore - -## Overview - -Replace the lolesports.com `/api/gql` persisted-query transport (commit `b76d0ca`) -with PandaScore's REST API as the **only** upstream for the lol module. User -decision (2026-08-05): full replacement, not fallback — accepting the trade-off -that official-source data and the gql client are dropped in exchange for a -stable, versioned, documented API contract that does not break on Riot frontend -deploys. Free tier: 1000 req/h, Bearer token (user already has one). - -Public contract unchanged: `ScheduleEvent` shape, formatters, handlers, cron, -subscriber flows, and the 60-min stale-cache fallback all stay. Only the -transport + response mapping in `api_client.go` change, mirroring the b76d0ca -rewrite pattern. - -Supersedes plan `260726-0952-lol-leaguepedia-score-enrichment`: PandaScore's -match `results` carry final series scores directly, absorbing the -score-enrichment job. That plan is marked cancelled in Phase 3. - -Evidence: `plans/reports/research-260805-1627-stable-free-lol-schedule-source-report.md`. - -## Key design decisions - -- **One mapping table, client-side.** PandaScore league slugs differ from - Riot's; the client maps PandaScore league → existing canonical slug so - `format.go`'s `majorLeagueSlugs` allowlist and `leagueOrder` stay untouched. - Phase 1 discovers the real slugs with the live token. -- **Token via env** `LOL_PANDASCORE_TOKEN`, matching the - `GOLD_VNAPP_API_KEY` pattern (`internal/modules/gold/vnappmob_client.go:52`). - Missing token → fetch error at call time (stale cache may still serve); - never panic at startup. -- **Single window endpoint.** `GET /lol/matches?range[begin_at]=,` - covers past+running+upcoming in one call; page-walk `per_page=100`. - Exact `[from,to)` filtering stays client-side (proven pattern). -- **Status mapping**: `not_started→unstarted`, `running→inProgress`, - `finished→completed`; `canceled`/`postponed` dropped (Phase 1 verifies the - full status vocabulary). - -## Phases - -| Phase | Name | Status | -|-------|------|--------| -| 1 | [Feasibility and mapping discovery](./phase-01-feasibility-and-mapping-discovery.md) | Completed | -| 2 | [PandaScore client rewrite](./phase-02-pandascore-client-rewrite.md) | Completed | -| 3 | [Wire-in cleanup and supersede](./phase-03-wire-in-cleanup-and-supersede.md) | Completed | - -Phase 1 is a cheap kill/adjust gate (curl only, needs `LOL_PANDASCORE_TOKEN` -exported); Phases 2-3 must not start before its questions are answered. - -## Acceptance criteria - -- [x] `/lol`, `/lol_tomorrow`, `/lol_this_week`, `/lol_next_week` render from - PandaScore data with identical message format — live probe rendered a - real week: `🔴 LIVE JDG 1–0 LGD · Bo3 (Group Ascend)`, ICT day grouping -- [x] Major-league filter keeps working via slug mapping (LCK/LPL verified - live; minor leagues dropped) -- [x] Finished series show real scores from `results`; winner bolded - (unit-tested; outcome gated on `winner_id`) -- [x] No references to lolesports.com remain in module code beyond - doc-comment history; gql client fully removed -- [x] Missing/invalid token degrades to fetch error + stale cache - (`lol_token_missing`, no upstream call — unit-tested) -- [x] `go test ./...`, `go vet`, `golangci-lint` green; tests use `httptest` - fixtures in PandaScore shape, no live calls -- [x] Live smoke: today+tomorrow and this-week windows returned plausible - events incl. a live LPL match; quota 1000/h confirmed in Phase 1 -- [x] Plan 260726-0952 marked cancelled/superseded with pointer here - -Code review (code-reviewer subagent): all 9 criteria PASS, -DONE_WITH_CONCERNS; both concerns fixed same day (page-budget warn log + -live budget 3→5; bracket encoding verified by live probe; phantom test -removed). - -## Dependencies - -- Supersedes: `260726-0952-lol-leaguepedia-score-enrichment` (marked in Phase 3). -- External: PandaScore REST API, free tier, `LOL_PANDASCORE_TOKEN` (user holds token). - -## Open questions - -- PandaScore full ToS text unreviewed (developer docs show no attribution - requirement for the free Fixtures plan) — low risk, revisit if the bot's - audience grows. -- Deployment env (`Coolify`) needs `LOL_PANDASCORE_TOKEN` set by the owner - before the next deploy; local `.env` already has it. diff --git a/plans/reports/260710-0606-telegram-tgs-bot-support.md b/plans/reports/260710-0606-telegram-tgs-bot-support.md deleted file mode 100644 index 0238432..0000000 --- a/plans/reports/260710-0606-telegram-tgs-bot-support.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -type: researcher -date: 2026-07-10 -conducted_at: 2026-07-10T06:06:00Z ---- - -# Research Report: Telegram Bot `.tgs` Support - -## Navigation - -- [Summary](#summary) -- [Methodology](#methodology) -- [Findings](#findings) -- [Implementation Recommendation](#implementation-recommendation) -- [References](#references) -- [Next Steps](#next-steps) -- [Unresolved Questions](#unresolved-questions) - -## Summary - -Yes. Telegram Bot API `sendSticker` accepts a new animated `.TGS` sticker as a -multipart upload. It also accepts an existing Telegram `file_id`, which is the -recommended path after first upload. A `.TGS` URL is not accepted for an -animated sticker. - -This repo already has everything needed. `github.com/go-telegram/bot` v1.20.0 -provides `SendSticker` and `models.InputFileUpload`; no dependency upgrade or -new package required. - -## Methodology - -- Sources consulted: Telegram Bot API, Telegram sticker specification, pinned - Go module source, current repo implementation -- Research date: 2026-07-10 -- Terms: `sendSticker`, `.TGS`, `InputFile`, multipart upload, animated sticker -- Boundary: send an existing `.TGS`; sticker-set creation excluded - -## Findings - -### API Support - -Use `sendSticker`, with the `sticker` parameter as an `InputFile` multipart -upload. Telegram explicitly lists `.WEBP`, `.TGS`, and `.WEBM` as supported -uploads. For `.TGS`, do not pass an HTTP URL; upload bytes or reuse a `file_id`. -The file does not need to be added to a sticker set first. - -`emoji` is optional and applies to a newly uploaded sticker. Telegram returns a -`Message`; its sticker contains the reusable `file_id`. - -### `.TGS` Requirements - -Telegram's animated-sticker requirements: - -- 512 x 512 canvas -- maximum 3 seconds -- looped animation -- maximum 64 KB after rendering -- 60 FPS -- supported Bodymovin-TG/Lottie features only - -An arbitrary Lottie JSON renamed to `.tgs` is not sufficient. Telegram `.TGS` -is its constrained, gzip-compressed Lottie format. - -### Repo Compatibility - -The repo pins `github.com/go-telegram/bot` v1.20.0. It currently sends stickers -by `models.InputFileString` in `internal/modules/loldle/handlers.go`, and already -uses `models.InputFileUpload` for animation bytes in -`internal/modules/misc/wheelofnames_command.go`. Direct `.TGS` upload combines -those existing patterns. - -### `sendSticker` vs `sendDocument` - -`sendSticker` makes Telegram render the file as a sticker. `sendDocument` can -transport the `.tgs` as a generic downloadable file (currently up to 50 MB), -but that is not the right method when sticker rendering is desired. A document -`file_id` cannot later be used with `sendSticker`, because Telegram does not -allow changing the media type represented by a `file_id`. - -### Security and Performance - -- Treat local/embedded `.TGS` assets as untrusted input until validated against - Telegram's format and size constraints. -- Avoid fetching user-supplied URLs server-side; `.TGS` sticker URLs are not - supported by `sendSticker` anyway. -- Upload once, persist the returned bot-scoped `file_id`, then resend by - `file_id` to avoid repeated multipart transfers. - -## Implementation Recommendation - -For in-memory bytes, matching current repo conventions: - -```go -sent, err := b.SendSticker(ctx, &bot.SendStickerParams{ - ChatID: msg.Chat.ID, - MessageThreadID: msg.MessageThreadID, - Sticker: &models.InputFileUpload{ - Filename: "celebration.tgs", - Data: bytes.NewReader(tgsData), - }, - Emoji: "🎉", -}) -if err != nil { - return err -} - -fileID := sent.Sticker.FileID -``` - -For later sends, use the repo's existing pattern: - -```go -Sticker: &models.InputFileString{Data: fileID} -``` - -Common pitfalls: - -- Passing an HTTPS `.tgs` URL instead of multipart-uploading it -- Using `sendAnimation`; `.TGS` is an animated sticker, not a GIF-style - animation payload -- Sending invalid or oversized Lottie data -- Omitting `MessageThreadID` in forum-topic replies -- Assuming another bot's `file_id` is reusable - -## References - -- [Telegram Bot API: `sendSticker`](https://core.telegram.org/bots/api#sendsticker) -- [Telegram Bot API: sending files](https://core.telegram.org/bots/api#sending-files) -- [Telegram Bot API: `sendDocument`](https://core.telegram.org/bots/api#senddocument) -- [Telegram animated sticker requirements](https://core.telegram.org/stickers#animated-stickers-and-emoji) -- [`SendStickerParams` in go-telegram/bot v1.20.0](https://github.com/go-telegram/bot/blob/v1.20.0/methods_params.go) -- [`InputFileUpload` in go-telegram/bot v1.20.0](https://github.com/go-telegram/bot/blob/v1.20.0/models/input_file.go) - -## Next Steps - -1. Add the `.tgs` asset only if a concrete command/feature needs it. -2. Validate it against Telegram's limits. -3. Upload with `SendSticker`; retain the returned `file_id` for repeated sends. -4. Add focused handler and multipart-call tests if implementing a command. - -## Unresolved Questions - -- Which command or event should send the sticker? -- Should the asset ship with the binary or be configured by `file_id`? diff --git a/plans/reports/260710-0614-wheelofnames-tgs-document-feasibility.md b/plans/reports/260710-0614-wheelofnames-tgs-document-feasibility.md deleted file mode 100644 index a382fde..0000000 --- a/plans/reports/260710-0614-wheelofnames-tgs-document-feasibility.md +++ /dev/null @@ -1,291 +0,0 @@ ---- -type: researcher -date: 2026-07-10 -conducted_at: 2026-07-10T06:14:00Z ---- - -# Research Report: Wheel of Names as a TGS Document - -## Navigation - -- [Executive Summary](#executive-summary) -- [Research Methodology](#research-methodology) -- [Current Architecture](#current-architecture) -- [Key Findings](#key-findings) -- [Comparative Analysis](#comparative-analysis) -- [Timing Recommendation](#timing-recommendation) -- [Implementation Impact](#implementation-impact) -- [Recommendation](#recommendation) -- [Resources and References](#resources-and-references) -- [Next Steps](#next-steps) -- [Unresolved Questions](#unresolved-questions) - -## Executive Summary - -Sending `wheelofnames.tgs` through Telegram `sendDocument` is technically -possible. Bots may multipart-upload general files up to 50 MB, and this repo's -Go library already supports `SendDocument` plus `InputFileUpload`. - -The exact proposal is not attractive for the current user experience. Telegram -defines `sendDocument` as general-file delivery and does not promise inline or -auto-playing TGS document previews. Inline TGS animation is part of the sticker -contract. Therefore users may see a downloadable `.tgs` attachment rather than -the wheel animation. - -Producing TGS is also not a GIF conversion setting. The existing -`tiennm99/wheelofnames` renderer uses React/Remotion, Chromium-rendered PNG -frames, and Remotion's GIF encoder. Remotion does not emit Lottie/TGS. A TGS -version needs a second vector renderer/serializer. Dynamic arbitrary labels are -the hardest part because Telegram TGS disallows text layers; glyphs would need -conversion to vector paths, threatening the 64 KB sticker-format target. - -Verdict: **transmission feasible; useful inline wheel experience not reliably -feasible with `sendDocument`; engineering/value ratio poor.** If the goal is a -smaller inline animation with caption, H.264 MP4 through the existing -`sendAnimation` method is the strongest option to benchmark first. - -## Research Methodology - -- Conducted: 2026-07-10 UTC -- Sources: current `miti99bot`, `tiennm99/wheelofnames` at commit - `f13055651e7a18a59b024f7b22266611cc843389`, Telegram Bot API and sticker - specification, Remotion renderer documentation -- Criteria: Telegram UX, file size, renderer compatibility, timing quality, - implementation scope, operational cost -- Boundary: research only; no implementation or live Telegram client test -- Search terms: `sendDocument`, TGS, Lottie, Remotion codec, 3-second wheel, - Telegram document preview - -## Current Architecture - -```text -miti99bot - POST /api/gif (options, winner, 6000ms spin, 1000ms hold, 20 FPS, 512px) - -> wheelofnames React/Remotion composition - -> Chromium renders PNG frames - -> Remotion encodes GIF - sendAnimation(GIF + spoiler caption) - -> Telegram inline animation -``` - -Current facts: - -- Bot requests a 6.0s spin plus 1.0s result hold: 7.0s total. -- Bot requests 20 FPS and 512x512. -- Renderer accepts only 12, 15, or 20 FPS. -- Renderer schema currently requires at least 3.0s spin and 0.5s hold, so its - shortest accepted total is 3.5s. -- Renderer supports up to 32 options with 40 characters each. -- Recorded 384px/12 FPS/3.5s GIFs are about 298-316 KB. A production-equivalent - 512px/20 FPS/7s benchmark is not recorded. -- Current bot sends the result as a spoiler caption attached to the animation. - -The proposed architecture is materially different: - -```text -miti99bot - POST /api/tgs (new contract and timing) - -> new vector wheel model - -> new Lottie JSON serializer - -> validate Telegram-supported features - -> gzip as .tgs - sendDocument(TGS + spoiler caption) - -> Telegram document attachment - -> inline playback not guaranteed -``` - -## Key Findings - -### 1. Telegram Can Transport the File - -`sendDocument` accepts an uploaded file of any type up to 50 MB and returns a -message whose media is a `Document`. The pinned Go dependency exposes all -required fields, including document upload, caption, thread ID, and optional -content-type detection control. - -The animated-sticker restrictions (3s, 64 KB, 512x512, 60 FPS) are not Bot API -limits on a generic document. The document limit is 50 MB. Following the TGS -restrictions still makes sense if the artifact should remain a valid Telegram -animation or may later be sent as a sticker. - -### 2. `sendDocument` Does Not Promise TGS Playback - -Telegram documents inline animation for `Animation` (GIF or silent H.264 MP4) -and `Sticker` (`.WEBP`, `.TGS`, `.WEBM`) message types. It documents -`sendDocument` only as sending general files. - -Inference from the official message contracts: a `.tgs` uploaded as a document -cannot be relied on to auto-play or render like a sticker. Client behavior may -vary and is not an API guarantee. This is the proposal's decisive risk. A -downloadable file users must open elsewhere is worse than the current GIF. - -### 3. TGS Requires a New Renderer - -The current renderer's output path is frame-based: - -```text -React DOM/CSS -> Chromium -> PNG frames -> GIF encoder -``` - -TGS is gzip-compressed Lottie vector-animation JSON. There is no reliable -GIF-to-TGS conversion: raster frames do not contain the vector geometry, -transforms, or typography needed for compact Lottie output. Remotion's media -codecs include GIF and video formats such as H.264; TGS is not an output codec. - -A TGS renderer would need to independently serialize: - -- wedge vector paths and colors -- hub and pointer vector paths -- wheel rotation keyframes/easing -- winner indication using supported vector properties -- every label as glyph outlines, or omit labels -- JSON normalization, gzip, and TGS validation - -The current CSS shadow, drop-shadow winner glow, HTML text, and font fallback -cannot be transferred directly because Telegram's TGS requirements disallow -layer effects and text. - -### 4. Dynamic Labels Undermine the Size Advantage - -The wheel's value comes from arbitrary user labels, including Vietnamese and -other Unicode text. Telegram-compatible TGS cannot use text layers. Converting -every unique glyph to vector paths requires font shaping, path generation, and -font coverage. Up to 32 x 40 characters creates a large worst case. - -Consequences: - -- 64 KB is plausible only for a simplified wheel with few short labels, or no - labels; it is not a safe general contract without empirical prototypes. -- Font files and fallback behavior become renderer concerns. -- Unicode shaping and emoji are substantially harder than current browser text. -- Repeated glyph/path data can erase much of TGS's size advantage. - -If sent only as a document, exceeding 64 KB is allowed, but then the proposal -loses both guaranteed playback and its strongest size target. - -### 5. Operational and Security Considerations - -- Keep current option-count and character-count bounds; vector path generation - expands attacker-controlled text into CPU and memory work. -- Validate generated JSON, compressed and uncompressed sizes, layer count, - duration, frame rate, and unsupported features before upload. -- Generate TGS internally; do not accept user-provided TGS payloads without - decompression limits and schema validation. -- A second render path duplicates layout logic unless wheel geometry is first - extracted into a renderer-neutral model. - -## Comparative Analysis - -| Option | Telegram UX | Renderer effort | Expected size | Caption | Main risk | -|---|---|---:|---:|---|---| -| Current GIF + `sendAnimation` | Inline/autoplay | Existing | Largest | Yes | Bandwidth/render time | -| 3s GIF + `sendAnimation` | Inline/autoplay | Low | Lower than current | Yes | Still GIF compression | -| TGS + `sendDocument` | Generic attachment; playback not guaranteed | High | Unknown; potentially small | Yes | Users may not see animation | -| TGS + `sendSticker` | Inline sticker | High | Must target 64 KB | No sticker caption | Labels/features may fail limits | -| H.264 MP4 + `sendAnimation` | Inline/autoplay | Low-medium | Likely much smaller than GIF; benchmark needed | Yes | Encoding/client tuning | - -H.264 is a closer fit to current architecture: Remotion already renders H.264, -and Telegram explicitly accepts silent H.264/MPEG-4 AVC through -`sendAnimation`. It preserves the spoiler caption and browser-rendered labels. -No size claim should be finalized until identical 3s fixtures are benchmarked. - -## Timing Recommendation - -If a 3-second-compatible animation prototype is created, use: - -| Phase | Duration | 60 FPS frames | Behavior | -|---|---:|---:|---| -| Spin/decelerate | 2300ms | 138 | Three full turns, cubic ease-out to winner | -| Winner hold | 650ms | 39 | Stationary wheel; simple winner emphasis | -| Total | 2950ms | 177 | 50ms safety margin below 3 seconds | - -Why three turns: the current wheel uses seven turns over six seconds with a -cubic ease-out. Compressing seven turns into 2.3s would start near nine -rotations/second and be visually noisy. Three turns preserves roughly the -current initial angular speed while making deceleration readable. - -The 650ms hold is short but sufficient for confirmation because the caption -also states the result. If no caption is present, favor 2200ms spin plus 750ms -hold. Timing has little UX value when TGS is sent as a non-previewed document. - -Required renderer changes even for a GIF/MP4 timing experiment: - -- lower `durationMs` minimum below 3000 -- permit total duration below 3500ms -- add 60 FPS only if targeting valid TGS; GIF/MP4 need not use 60 FPS -- reduce `fullTurns` from 7 to 3 for the short profile -- verify the winner lands exactly after frame rounding - -## Implementation Impact - -If this exact TGS-document option is later chosen: - -### `tiennm99/wheelofnames` - -- Add a renderer-neutral wheel geometry/timing model. -- Add `/api/tgs` and a response MIME/validation contract. -- Implement Lottie vector serialization and gzip packaging. -- Convert supported text to glyph paths or define label restrictions. -- Simplify unsupported shadow/glow effects. -- Add format, timing, size, Unicode, and winner-alignment tests. -- Benchmark sparse and worst-case option sets. - -### `miti99bot` - -- Change the API URL/Accept/content-type and magic validation from GIF to TGS. -- Change the filename and response byte limit. -- Replace `SendAnimationParams` with `SendDocumentParams`, retaining caption, - parse mode, thread ID, and text fallback. -- Update recording-bot assertions and all wheel handler/client tests. -- Update user-facing GIF wording and deployment docs/config contract. - -No dependency change is necessary in the bot. - -## Recommendation - -Do not implement **TGS via `sendDocument`** as the primary wheel response unless -a real Telegram Android/iOS/Desktop client matrix proves acceptable inline -playback. The Bot API provides no such guarantee. - -Decision order: - -1. Reduce the existing animation to the 2.3s spin + 0.65s hold profile. -2. Benchmark identical GIF and silent H.264 MP4 outputs at 512px. -3. Prefer H.264 with `sendAnimation` if it materially reduces bytes while - retaining inline playback and caption. -4. Prototype TGS only if sticker-style delivery or vector rendering is itself - a product requirement, and first test a no-label/few-label wheel against - Telegram's validator and clients. - -## Resources and References - -### Official Documentation - -- [Telegram Bot API: `sendDocument`](https://core.telegram.org/bots/api#senddocument) -- [Telegram Bot API: `sendAnimation`](https://core.telegram.org/bots/api#sendanimation) -- [Telegram Bot API: `sendSticker`](https://core.telegram.org/bots/api#sendsticker) -- [Telegram animated TGS requirements](https://core.telegram.org/stickers#animated-stickers-and-emoji) -- [Remotion `renderMedia()`](https://www.remotion.dev/docs/renderer/render-media) - -### Current Implementations - -- [`wheelofnames` GIF renderer](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/src/render/render-gif.js) -- [`wheelofnames` request limits](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/src/schemas/wheel-request.js) -- [`wheelofnames` Remotion composition](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/src/remotion/WheelComposition.jsx) -- [`wheelofnames` recorded benchmarks](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/docs/render-benchmarks.md) - -## Next Steps - -No implementation now. If the idea advances: - -1. Decide whether inline/autoplay is mandatory. -2. Run a Telegram client-matrix spike with one hand-authored TGS document. -3. Benchmark 2.95s GIF versus H.264 before building a new vector renderer. -4. Only then estimate TGS implementation from a few-label vector prototype. - -## Unresolved Questions - -- Must the wheel auto-play inline on Telegram clients? -- Must all option labels remain visible inside the wheel? -- Is the target specifically under 64 KB, or only smaller than the current GIF? -- Is silent H.264 MP4 acceptable if it preserves the current message UX? diff --git a/plans/reports/260710-0632-inhouse-wheel-rendering-gif-video.md b/plans/reports/260710-0632-inhouse-wheel-rendering-gif-video.md deleted file mode 100644 index c328847..0000000 --- a/plans/reports/260710-0632-inhouse-wheel-rendering-gif-video.md +++ /dev/null @@ -1,476 +0,0 @@ ---- -type: researcher -date: 2026-07-10 -conducted_at: 2026-07-10T06:32:00Z ---- - -# Research Report: In-House Wheel Rendering with GIF or Video - -## Navigation - -- [Executive Summary](#executive-summary) -- [Research Scope](#research-scope) -- [Current Constraints](#current-constraints) -- [Technical Options](#technical-options) -- [Format Decision](#format-decision) -- [Recommended Architecture](#recommended-architecture) -- [Performance and Quality](#performance-and-quality) -- [Security and Reliability](#security-and-reliability) -- [Implementation Impact](#implementation-impact) -- [Validation Plan](#validation-plan) -- [Resources and References](#resources-and-references) -- [Next Steps](#next-steps) -- [Unresolved Questions](#unresolved-questions) - -## Executive Summary - -Best technical direction: **keep the existing React/Remotion wheel composition, -render silent H.264 MP4 locally, and continue sending it with Telegram -`sendAnimation`.** This preserves the current geometry, colors, shadows, -winner glow, fonts, label layout, easing, caption, and inline/autoplay behavior. -It removes the renderer HTTP service and its URL/token, while avoiding a risky -visual rewrite. - -"All in house" should mean one deployed container, no renderer HTTP listener, -no `WHEELOFNAMES_API_URL`, no renderer authentication token, no runtime browser -download, and no external render service. The Go bot starts a local persistent -Node/Remotion worker and communicates through stdin/stdout plus controlled -temporary files. Telegram upload remains the only media network step. - -H.264 is preferred over GIF. Telegram explicitly accepts silent H.264/MPEG-4 -AVC as an animation. H.264 preserves far more color detail than GIF's -256-color palette and should compress the mostly-static wheel frames much more -efficiently. Exact size and latency remain benchmark gates; no successful -H.264 benchmark exists yet. - -Main trade-off: exact visual reuse requires Node, Chromium, fonts, and Remotion -inside the bot container. The current tiny distroless/static image cannot run -them. Expect a much larger image and materially higher render-time CPU/RAM. - -## Research Scope - -- Conducted: 2026-07-10 UTC -- Goal: local rendering, GIF or video, current visual fidelity, no renderer API -- Sources: current `miti99bot`; `tiennm99/wheelofnames` at commit - `f13055651e7a18a59b024f7b22266611cc843389`; Telegram Bot API; Remotion - renderer docs; Go `image/gif` docs -- Compared: pure-Go GIF, Go frames plus FFmpeg, local Remotion worker -- Boundary: research only; no project implementation - -### Definition of "No API" - -Recommended interpretation: - -- no HTTP call to a wheel renderer -- no renderer TCP port or public/private service -- no external rendering vendor -- no runtime download of code, fonts, Chromium, or media tools -- local child-process IPC is allowed - -If "no API" instead means one Go process with no child runtime, exact parity -with the current Remotion composition is not realistic. The pure-Go option can -approximate it but becomes a separate renderer requiring visual maintenance. - -## Current Constraints - -### Bot Deployment - -The current image is: - -```text -Go 1.26 static build -> distroless/static non-root runtime -> /server only -``` - -It contains no shell, Node, Chromium, fonts, or FFmpeg. It cannot execute the -current renderer locally without a Docker runtime redesign. - -### Wheel Renderer - -The current `wheelofnames` project uses: - -- React 19 and Remotion 4.0.485 -- Chromium-rendered DOM/CSS frames -- Remotion `renderMedia()` -- PNG frame rendering, then GIF encoding -- Noto fonts, including Vietnamese and color emoji -- one render at a time by default - -Current bot profile: - -| Setting | Value | -|---|---:| -| Canvas | 512x512 | -| Spin | 6000ms | -| Winner hold | 1000ms | -| Total | 7000ms | -| FPS | 20 | -| Frames | 140 | -| Full turns | 7 | - -Recorded renderer measurements cover only a smaller fixture: - -| Fixture | Output | Bytes | Render time | -|---|---:|---:|---:| -| 384px, 12 FPS, 3.5s cold | GIF | 315,661 | 13.9s | -| 384px, 12 FPS, 3.5s warm service | GIF | 298,374 | 4.1s | - -No production-equivalent 512px/20 FPS/7s GIF measurement or H.264 measurement -is recorded. - -## Technical Options - -### Option A: Pure-Go Raster Renderer and GIF - -Flow: - -```text -Go wheel geometry + Go font/layout + raster frames + image/gif -> sendAnimation -``` - -Advantages: - -- retains one static Go binary and distroless image -- no child process or browser -- simplest production runtime -- deterministic, fully offline renderer - -Disadvantages: - -- reimplement every current visual and animation behavior -- browser-quality Unicode shaping, fallback fonts, and color emoji are hard -- must recreate anti-aliased wedges, radial text, shadows, glow, and CSS effects -- GIF allows at most 256 palette colors per frame -- palette quantization/dithering can damage text edges, shadows, and glow -- standard `image/gif.EncodeAll` holds all paletted frames before writing -- duplicates visual logic now owned by the Remotion composition - -Verdict: good only if minimal runtime matters more than exact visual parity. - -### Option B: Pure-Go Frames Piped to Local FFmpeg H.264 - -Flow: - -```text -Go wheel geometry + Go font/layout -> raw frames -> local FFmpeg -> MP4 -``` - -Advantages: - -- video size/color advantages -- raw frames can stream to FFmpeg with bounded memory -- no Chromium or Node -- no HTTP API - -Disadvantages: - -- still requires the full visual rewrite -- adds FFmpeg executable and Linux runtime libraries -- loses current distroless/static simplicity -- child-process lifecycle, cancellation, temp/output bounds still required -- font shaping remains the largest fidelity risk - -Verdict: inferior to reusing Remotion; it pays both rewrite and runtime costs. - -### Option C: Existing Remotion Composition as a Local Worker - -Flow: - -```text -Go bot -> local JSON-lines worker -> Remotion/Chromium -> H.264 MP4 - <- result metadata + controlled temp path -Go bot -> Telegram sendAnimation -``` - -Advantages: - -- highest visual parity; reuses the current composition directly -- browser keeps current Unicode/font behavior -- H.264 is already a supported Remotion codec -- no renderer HTTP surface, URL, token, or network failure mode -- one composition can still render GIF during benchmarks/debugging -- current text fallback remains available if rendering fails - -Disadvantages: - -- combined runtime needs Node 24, Chromium, fonts, and browser libraries -- image size grows from a tiny static runtime to likely hundreds of MB -- render CPU/RAM now shares the bot container's cgroup -- local worker protocol and restart supervision must be implemented -- Remotion bundle/browser cold start remains expensive unless reused - -Verdict: best match for "current beauty + no renderer API." - -### Option D: One-Shot Local Remotion CLI per Command - -This also removes HTTP, but it repeats Node startup, bundling, and browser -launch per command. Existing measurements show a large cold/warm gap. Remotion -officially supports reusing an open browser to speed multiple renders. - -Verdict: acceptable prototype, poor production architecture. - -## Format Decision - -### GIF versus H.264 MP4 - -| Criterion | GIF | Silent H.264 MP4 | -|---|---|---| -| Telegram method | `sendAnimation` | `sendAnimation` | -| Inline/autoplay UX | Yes | Yes | -| Caption/spoiler | Yes | Yes | -| Colors | 256 per frame | Full-color input, compressed video output | -| Shadows/glow | Palette banding/dither risk | Better gradients; lossy artifact risk | -| Temporal compression | Limited | Strong | -| Expected bytes | Larger | Smaller; must benchmark | -| Current Remotion support | Existing | Native `codec: "h264"` | -| Pure-Go production | Possible | Not with standard library | - -**Choose H.264 MP4 as the production format.** Keep GIF only as a temporary -benchmark/debug option, not an automatic production fallback. Re-rendering GIF -after an MP4 failure doubles latency and resource use; retain the existing text -winner fallback instead. - -Suggested first parity profile: - -```js -await renderMedia({ - codec: 'h264', - composition, - crf: 18, - imageFormat: 'png', - inputProps, - muted: true, - outputLocation, - overwrite: true, - pixelFormat: 'yuv420p', - puppeteerInstance: sharedBrowser, - serveUrl, -}); -``` - -Notes: - -- Preserve 512px, 20 FPS, 6s spin, 1s hold, seven turns for the first parity - release. Shortening is a separate UX decision. -- Start at CRF 18; compare CRF 18, 20, and 22 visually and by bytes. -- PNG source frames preserve browser rendering before H.264 encoding. -- `yuv420p` is the conservative compatibility target; inspect small radial text - and saturated slice edges for chroma-subsampling artifacts. -- Do not add an audio track. - -## Recommended Architecture - -```text -┌──────────────────── one Coolify container ────────────────────┐ -│ │ -│ Go /server │ -│ command handler │ -│ │ bounded request │ -│ ▼ │ -│ render supervisor ── JSON lines over pipes ──► Node worker │ -│ ▲ │ │ -│ │ metadata + temp path ▼ │ -│ │ cached Remotion bundle │ -│ │ shared Chromium instance │ -│ │ H.264 render, concurrency=1 │ -│ │ │ │ -│ └──────── read + validate + delete ◄── temp MP4 │ -│ │ -│ Go SendAnimation(MP4 + existing spoiler caption) ─► Telegram │ -│ │ -│ no renderer HTTP listener; no renderer network call │ -└───────────────────────────────────────────────────────────────┘ -``` - -### Worker Lifecycle - -1. Go starts one Node worker lazily or warms it asynchronously after startup. -2. Worker bundles Remotion once and opens one reusable browser. -3. Go serializes render requests through a capacity-one queue. -4. Worker renders to a unique directory under a controlled temp root. -5. Worker returns request ID, path, MIME, dimensions, duration, and byte count. -6. Go verifies path prefix, MP4 signature/metadata, and maximum size. -7. Go reads/sends the file and always removes the temp directory. -8. On timeout/crash, Go kills and restarts the worker, then sends text result. - -Use stdout only for machine protocol and stderr for renderer logs. Never send -binary/base64 through JSON lines; base64 adds memory and about 33% transfer -overhead. - -### Container Design - -Recommended multi-stage build: - -```text -stage 1: golang:1.26.5 -> build static /server -stage 2: node:24-bookworm-slim -> pnpm production install + browser download -runtime: node:24-bookworm-slim + Chromium libs + Noto fonts - copy /server, renderer bundle/source, node_modules, pinned browser -entrypoint: /server (starts local Node worker) -``` - -Requirements: - -- install the same Chromium libraries/fonts as current `wheelofnames` Dockerfile -- download/pin the Remotion browser during image build, never at runtime -- run bot and worker as non-root -- keep port 8080 only for bot health; expose no renderer port -- allow controlled writable temp space; keep application/root filesystem - read-only where practical -- retain one bot replica and one render at a time - -This removes the current distroless security/minimal-size benefit. That is the -principal architecture cost. - -## Performance and Quality - -### What Is Verified - -- Telegram accepts GIF and silent H.264/MPEG-4 AVC through `sendAnimation`, up - to 50 MB. -- Remotion `renderMedia()` supports `codec: "h264"`, in-memory or file output, - cancellation, CRF, pixel format, and reusable browser instances. -- Current warm GIF fixture render is 4.1s; cold is 13.9s. -- The current production container lacks all browser/media runtime pieces. - -### What Is Not Yet Verified - -- H.264 bytes and render latency for this wheel -- production 512px/20 FPS/7s GIF bytes and latency -- Telegram-side appearance after upload/transcoding on Android, iOS, Desktop -- combined container image size and idle/peak memory - -A temporary local benchmark was attempted. Dependency installation and browser -preparation succeeded, but Chromium could not launch because the research host -lacks the shared libraries listed in the renderer Dockerfile. No codec numbers -were fabricated from that failed run. - -The temporary checkout plus installed dependencies occupied about 600 MB, and -the downloaded headless browser executable alone was about 188 MB. These are -research-host figures, not a final compressed container-image estimate, but -they confirm the runtime increase is material. - -### Expected Behavior Requiring Measurement - -H.264 should beat GIF for this animation because most pixels remain spatially -and temporally related while only wheel rotation and winner emphasis change. -GIF stores palette-based images and has much weaker temporal compression. The -size win must still be measured on real fixtures. - -## Security and Reliability - -- Preserve option count/length limits before invoking the worker. -- Never interpolate option text into shell commands; send structured JSON over - an already-open pipe. -- Use request IDs and reject unsolicited/out-of-order worker responses. -- Enforce render timeout, output byte cap, temp-root path containment, and file - signature/ffprobe-style metadata checks. -- Keep concurrency at one on the recommended 1-2 vCPU/1-2 GB starting profile. -- Cancel render on Telegram/request context cancellation where practical. -- Restart the worker after crash or repeated render failures. -- Avoid worker stdout logging that can corrupt the IPC stream. -- Pin Node, pnpm lock, Remotion, and browser versions. -- Run without renderer network access if container policy supports it; the - worker needs local files only. -- Preserve current plain-text winner fallback on any render/send failure. - -An out-of-memory kill can still terminate the whole container even though the -renderer is a child process. Resource limits, serial rendering, and production -peak-memory measurement are mandatory. - -## Implementation Impact - -If selected later, expected scope: - -### Repository/runtime - -- bring the required `wheelofnames` composition into this repository or a - version-pinned build input -- add Node renderer package/lock files and local worker entry point -- replace distroless runtime with combined Node/Chromium runtime -- remove wheel API URL/token configuration and deployment instructions - -### Go bot - -- replace HTTP wheel client with a local render supervisor -- preserve command parsing, local winner selection, spoiler caption, thread ID, - and text fallback -- change upload filename from `.gif` to `.mp4` -- keep `SendAnimationParams`; no Telegram method change -- validate local worker output and clean temporary files - -### Tests - -- unit-test supervisor framing, timeouts, crash/restart, output validation, and - cleanup with a small fake child executable -- retain handler tests for caption, thread ID, and text fallback -- test renderer composition and winner alignment in Node -- add deterministic smoke renders for GIF/H.264 comparison -- add Docker smoke test proving browser launch under the final non-root runtime -- perform Telegram client visual matrix before rollout - -### Maintainability Boundary - -Extract renderer-neutral request/timing constants, but do not rewrite wheel -geometry in Go. The Remotion composition remains the sole visual source of -truth. This avoids GIF/video visual drift. - -## Validation Plan - -Before choosing implementation: - -1. Build the combined runtime prototype with required browser libraries. -2. Render identical fixtures as GIF and H.264: - - 2, 8, 16, and 32 options - - ASCII, Vietnamese, long labels, emoji - - all themes - - current 512px/20 FPS/7s profile -3. Record output bytes, cold/warm latency, CPU time, peak RSS, and temp bytes. -4. Compare H.264 CRF 18/20/22 at 512px. -5. Inspect labels, slice edges, shadows, glow, easing, and winner frame. -6. Upload representative files to a development bot and inspect Android, iOS, - Desktop, slow network, reduced-motion/autoplay settings. -7. Require the selected H.264 profile to be visually equivalent and materially - smaller than GIF before removing GIF production output. - -Suggested acceptance gates: - -- no renderer network socket or runtime download -- exact winner and caption behavior -- no obvious text/chroma artifacts at normal Telegram display size -- warm render finishes within existing 30s command timeout -- output remains well below Telegram's 50 MB limit and current 12 MB bot cap -- worker crash always yields text fallback and leaves no temp files -- bot stays responsive during a render - -## Resources and References - -### Official Documentation - -- [Telegram Bot API: `sendAnimation`](https://core.telegram.org/bots/api#sendanimation) -- [Remotion `renderMedia()`](https://www.remotion.dev/docs/renderer/render-media) -- [Remotion server-side rendering](https://www.remotion.dev/docs/ssr) -- [Go `image/gif`](https://pkg.go.dev/image/gif) - -### Current Source and Evidence - -- [`wheelofnames` GIF renderer](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/src/render/render-gif.js) -- [`wheelofnames` composition](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/src/remotion/WheelComposition.jsx) -- [`wheelofnames` container runtime](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/Dockerfile) -- [`wheelofnames` benchmarks](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/docs/render-benchmarks.md) - -## Next Steps - -No implementation now. If approved later: - -1. Build a disposable combined-container benchmark first. -2. Measure identical GIF and H.264 outputs. -3. Confirm H.264 Telegram client quality. -4. Only then plan the local persistent-worker integration. - -## Unresolved Questions - -- Does "all in house" allow a local persistent Node child process, or require - one Go process only? -- Is the larger Node/Chromium container acceptable on the Coolify host? -- Must the first version retain the current 7-second timing exactly, or use the - previously researched 2.95-second profile? diff --git a/plans/reports/260720-1608-dividend-notice-ratio-research.md b/plans/reports/260720-1608-dividend-notice-ratio-research.md deleted file mode 100644 index 0bf1d09..0000000 --- a/plans/reports/260720-1608-dividend-notice-ratio-research.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -type: research-report -topic: dividend-notice-ratio-examples -conducted_at: 2026-07-20T16:08:21+07:00 -status: complete ---- - -# Research Report: Dividend Notice Ratio Examples - -## Summary - -Official VSDC notices support ratio-based share-dividend input. Notices express -the ratio as existing shares or rights to new shares, such as `4:1` and -`100:10`. Resulting fractional shares are rounded down and discarded. - -Keep cash input as VND per existing share. Although notices also state a cash -percentage, that percentage is based on par value; accepting the actual VND per -share avoids needing par-value data. - -## Methodology - -- Sources: 2 recent official VSDC notices -- Notice dates: 2026-05-05 and 2026-06-26 -- Terms: `trả cổ tức bằng tiền`, `trả cổ tức bằng cổ phiếu`, `tỷ lệ thực hiện` -- Scope: input semantics and rounding for the three proposed Telegram commands - -## Findings - -### IDC mixed dividend - -- Cash: 15% per share, explicitly `1 share receives 1,500 VND`. -- Shares: `100:10`, meaning 100 existing rights receive 10 new shares. -- Example: 139 existing shares produce `139 / 100 * 10 = 13.9`, rounded down - to 13 new shares; the 0.9 fraction is discarded. - -Source: [IDC: cash and stock dividends for 2025](https://www.vsd.vn/vi/ad/197421) - -### PTB mixed dividend - -- Cash: 5% per share, explicitly `1 share receives 500 VND`. -- Shares: `4:1`, meaning 4 existing shares receive 1 new share. -- Example: 2,026 existing shares produce `2,026 / 4 * 1 = 506.5`, rounded down - to 506 new shares; the 0.5 fraction is discarded. - -Source: [PTB: cash and stock dividends for 2025](https://vsd.vn/vi/ad/195203) - -## Comparative Analysis - -| Input | Benefit | Problem | -|---|---|---| -| Absolute new-share quantity | Matches current `/stock_bonus` behavior | User must calculate entitlement manually | -| Percentage | Familiar for rates such as 10% | Ambiguous parsing and less faithful to notice wording | -| `owned:new` ratio | Mirrors VSDC notices; easy to audit | Requires integer parsing and floor rounding | - -Recommendation: accept the `owned:new` ratio. - -## Command Recommendation - -```text -/stock_cash_dividend -/stock_share_dividend -/stock_dividend -``` - -Examples: - -```text -/stock_cash_dividend 1500 IDC -/stock_share_dividend 100:10 IDC -/stock_dividend 1500 100:10 IDC - -/stock_cash_dividend 500 PTB -/stock_share_dividend 4:1 PTB -/stock_dividend 500 4:1 PTB -``` - -Calculation: - -```text -new_shares = floor(existing_shares * new / owned) -cash_vnd = existing_shares * vnd_per_share -``` - -For a mixed dividend, calculate both values from the same pre-event holding, -then save cash and shares atomically. - -## Validation and Pitfalls - -- Require positive whole-number ratio parts; reject `0:1`, `4:0`, negatives, - decimals, missing colon, and extra colons. -- Reduce equivalent ratios internally if useful, but preserve the entered ratio - in the reply for auditability. -- Use integer arithmetic and floor division for shares; avoid floating-point - rounding. -- Reject events producing zero new shares, or explicitly confirm zero payout; - recommended behavior: reject with the minimum holding required. -- Calculate cash before adding new shares so newly distributed shares do not - receive cash from the same event. -- Keep cash input in VND/share. Do not derive it from a percentage unless the - portfolio also stores the security's par value. - -## Next Steps - -1. Confirm zero-share-result behavior. -2. Finalize the three command contracts and reply wording. -3. Plan command renames, combined atomic update, stats migrations, and tests. - -## Unresolved Questions - -- When a valid ratio yields zero new shares, should the command reject the - event or record a zero-share result while still applying cash in the combined - command? diff --git a/plans/reports/260721-1418-command-parameter-schema-research.md b/plans/reports/260721-1418-command-parameter-schema-research.md deleted file mode 100644 index fbda0c6..0000000 --- a/plans/reports/260721-1418-command-parameter-schema-research.md +++ /dev/null @@ -1,193 +0,0 @@ ---- -type: research-report -conducted_at: 2026-07-21T14:18:00+07:00 -scope: Telegram command discovery syntax -status: complete ---- - -# Research Report: Minimal Command Parameter Schema - -## Summary - -Keep the project's current lightweight schema. It already matches familiar CLI -synopsis conventions closely enough: - -```text - required value -[name] optional value -[name...] optional remaining text -a | b alternatives inside one optional group -``` - -Do not add types such as ``. Common CLI usage lines describe -structure, not a type system. Use a descriptive name (``, -``) and put validation details in the summary or usage error. - -Only one current notation is needlessly heavy. After project-owner review, the -approved compact replacement is: - -```text - -> -``` - -This keeps the literal comma visible without putting prose inside the -placeholder. Keep `` for the same reason: the separator is -essential input syntax and the concise shape prevents a likely user error. - -## Contents - -- [Scope and method](#scope-and-method) -- [Evidence](#evidence) -- [Recommended schema](#recommended-schema) -- [Current command audit](#current-command-audit) -- [Rejected alternatives](#rejected-alternatives) -- [Next steps](#next-steps) -- [References](#references) -- [Unresolved questions](#unresolved-questions) - -## Scope and Method - -Evaluated only the syntax displayed after `/command` in Telegram's native menu -and `/help`. Criteria, in order: - -1. Understandable without a legend -2. Minimal visual noise on a phone -3. Accurate to the parser -4. Consistent across commands -5. Familiar to terminal users - -Sources span POSIX.1-2024, current GNU guidance, Python 3.14 documentation, -Microsoft's command-line syntax key, docopt's usage grammar, and Telegram's -current Bot API. The configured Gemini path was disabled. Web search returned -an authorization error, so primary sources were retrieved directly over HTTPS. - -## Evidence - -### Strong consensus - -- Square brackets mean optional material. POSIX, Microsoft, argparse, and - docopt all use this convention. -- Angle brackets are a recognized way to mark a value the user must replace. - POSIX explicitly permits ``; Microsoft defines angle-bracket - text as a placeholder. -- A vertical bar separates mutually exclusive alternatives. Microsoft and - docopt explicitly document this. -- An ellipsis indicates repetition. POSIX uses `[operand ...]`; docopt uses - `FILE ...` for one or more and `[FILE ...]` for zero or more. -- Metavariable names should explain the role of the value. Python argparse uses - `metavar` for this purpose and generates the structural punctuation itself. - -### Important nuance - -Traditional terminal help often uses uppercase metavariables such as `FILE` or -`FOO`. Lowercase angle-bracket names such as `` remain unambiguous and -are calmer in Telegram's compact UI. Changing case would add churn without -improving comprehension. - -POSIX's spaced ellipsis (`operand ...`) means repeated operands. This bot's -`[target...]` means one optional free-text remainder. That is a small local -extension, but it communicates the behavior better than `[target]`, which can -look limited to one word. Document it once and keep it consistent. - -## Recommended Schema - -Use only constructs the project currently needs: - -| Meaning | Format | Example | -|---|---|---| -| Required value | `` | `` | -| Required comma-separated values | `` | `` | -| Optional value | `[name]` | `[date]` | -| Optional remaining text | `[name...]` | `[target...]` | -| Optional alternatives | `[literal | literal ]` | `[users | user ]` | - -Naming rules: - -- Lowercase `snake_case`. -- Prefer one plain noun: ``, ``, ``. -- Include a unit when it prevents ambiguity: ``, ``. -- Do not encode primitive types: avoid `` and ``. -- Do not put format prose inside a placeholder. -- Include a compact value shape only when users must type its punctuation - exactly: ``, ``. -- Keep literal subcommands bare: `users`, `user`, `cmd`. -- Put spaces around `|`; readability is worth the two characters. - -This is presentation metadata, not a machine-validated schema language. Avoid -adding an AST, parser, enums, or type annotations until the bot needs generated -validation or completion. - -## Current Command Audit - -| Commands | Current | Recommendation | Reason | -|---|---|---|---| -| `/random`, `/wheelofnames` | `` | `` | Shows the literal delimiter without prose. | -| `/stats` | `[users | user | cmd ]` | Keep | Accurate compact grammar; `command_name` avoids confusing the value with a literal command. | -| `/trongtruonghop`, `/tth` | `[target...]` | Keep | Clearly signals optional multi-word remainder. | -| `/lol` | `[date]` | Keep | Minimal; accepted date shapes belong in the summary/error. | -| `/loldle`, `/wordle` | `[champion]`, `[word]` | Keep | Optional value accurately reflects start-without-argument behavior. | -| Coin commands | ``, ``, ` `, ` ` | Keep | Names expose currency and transaction intent without type noise. | -| Gold commands | ``, `` | Keep | Units are essential and concise. | -| Stock price/trade commands | ``, ``, ` ` | Keep | Conventional positional metavariables. | -| Stock dividend commands | ``, ``, `` combinations | Keep | The ratio shape is valuable syntax, not redundant prose. | - -Result: change two registrations and their matching usage/tests; leave every -other public parameter string unchanged. - -## Rejected Alternatives - -### Uppercase metavariables - -```text - -``` - -Familiar in man pages, but redundant when angle brackets already mark values. -It is visually louder and would touch every command for no behavior gain. - -### Inline type annotations - -```text - -``` - -Not a common shell synopsis convention. Longer, falsely formal, and still -cannot express real constraints such as positive integers or known tickers. - -### Generic names everywhere - -```text - -``` - -Short but ambiguous. Minimalism should remove redundancy, not meaning. - -### Full docopt grammar - -Docopt can represent required groups, nested alternatives, and repetition -precisely. The bot does not need that complexity. Adopt only its familiar -surface notation when a real command requires it. - -## Next Steps - -1. Change `/random` and `/wheelofnames` parameter metadata to ``. -2. Change their handler usage strings and exact-string tests at the same time. -3. Update README/AGENTS guidance: value format belongs in the summary or usage - error unless punctuation is essential, as with ``. -4. Keep current registration validation: non-empty descriptions, single-line - parameters, and Telegram's length limit. -5. Do not build a schema parser. - -## References - -- [POSIX.1-2024, Utility Conventions](https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap12.html) -- [GNU Standards, Command-Line Interfaces](https://www.gnu.org/prep/standards/html_node/Command_002dLine-Interfaces.html) -- [Microsoft command-line syntax key](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/command-line-syntax-key) -- [Python 3 argparse documentation](https://docs.python.org/3/library/argparse.html) -- [docopt usage-pattern grammar](https://github.com/docopt/docopt/blob/master/README.rst) -- [Telegram Bot API](https://core.telegram.org/bots/api#botcommand) - -## Unresolved Questions - -None. The recommendation is intentionally small and implementable without -changing command parsing or persisted data. diff --git a/plans/reports/260721-1654-vietnam-stock-dividend-api-research.md b/plans/reports/260721-1654-vietnam-stock-dividend-api-research.md deleted file mode 100644 index 970406f..0000000 --- a/plans/reports/260721-1654-vietnam-stock-dividend-api-research.md +++ /dev/null @@ -1,151 +0,0 @@ ---- -type: research-report -topic: vietnam-stock-dividend-event-api -conducted_at: 2026-07-21T16:54:00+07:00 -status: complete ---- - -# Research Report: Vietnam Stock Dividend Event API - -## Executive Summary - -Use SSI iBoard's live corporate-actions JSON endpoint for the first free -implementation, isolated behind a provider interface. It supplies symbol/date -filters, pagination, stable event IDs, cash values, share ratios, and relevant -dates. It is undocumented and has no SLA, so it is suitable for a replaceable -prototype adapter—not a permanent production contract. - -VSDC remains the authoritative verification source, but no documented public -JSON API was found. SSI FastConnect's documented API omits corporate actions. -For contractual reliability, FiinGroup's commercial API Datafeed is the better -future provider. - -## Methodology - -- Conducted: 2026-07-21 (Asia/Saigon) -- Sources checked: VSDC, SSI iBoard, SSI FastConnect, VNDirect Finfo, TCBS, - FiinGroup -- Validation: official documentation plus direct live HTTP requests -- Scope: Vietnamese listed-stock cash/share dividends; no automatic portfolio - mutation, entitlement ownership, tax, or coin events - -## Findings - -### SSI iBoard - -Live endpoint: - -```http -GET https://iboard-api.ssi.com.vn/statistics/company/corporate-actions - ?symbol=ACB - &fromDate=01/06/2026 - &toDate=30/06/2026 - &page=1 - &pageSize=50 -``` - -The verified response returned HTTP 200 without authentication and contained -`data` plus `paging`. Useful fields: - -| Field | Use | -|---|---| -| `CorId` | Stable provider event ID | -| `symbol` | Ticker | -| `eventListCode` | Broad event type | -| `eventName`, `eventTitle`, `eventDescription` | Classification and user display | -| `exrightDate` | Ex-right date | -| `recordDate` | Record date | -| `issueDate` | Payment/trading date when supplied | -| `publicDate` | Publication time | -| `value` | Cash VND per share for `DIV` | -| `ratio` | Decimal event ratio | - -Observed mappings: - -- `DIV`: cash dividend; `value` is VND/share. -- `ISS`: issuance; accept only when title/description explicitly identifies a - share dividend. Other issuances must not be treated as dividends. - -Verified ACB examples included a cash event (`CorId=2612974`, `value=700`, -`ratio=0.07`) and share event (`CorId=2612975`, `ratio=0.13`) with ex-right and -record dates in June 2026. - -Limitations: undocumented endpoint, no published SLA/rate limits/terms, and -server-side `eventListCode` filtering was ignored. Filtering must be local. - -### Other providers - -| Provider | Result | Decision | -|---|---|---| -| VSDC | Official notices contain the full legal event text; no documented JSON API found | Verification/fallback HTML source | -| SSI FastConnect | Documented, authenticated market-data APIs omit corporate actions | Not sufficient | -| VNDirect Finfo | Unfiltered `/v4/events` responded, but filtered requests failed and no current official contract was found | Do not depend on it | -| TCBS | Public guidance points users to VSDC; no documented corporate-action API found | Not suitable | -| FiinGroup | Commercial standardized securities API/Datafeed | Production option requiring contract | - -## Recommended Design - -Define a replaceable provider contract returning normalized events: - -```text -DividendEvent - providerID - symbol - kind: cash | shares | mixed | ambiguous - exDate - recordDate - paymentDate - vndPerShare - ownedShares - newShares - title - sourceURL -``` - -SSI adapter behavior: - -1. Query ticker and date window with bounded pages. -2. Require every page to succeed. -3. Deduplicate by `CorId`. -4. Accept `DIV`; accept `ISS` only with explicit dividend wording. -5. Validate dates, positive values, and ratios before presenting an event. -6. Display events for user confirmation; never apply automatically. -7. Keep malformed/ambiguous items informational only. - -Use a one-day overlap around the stored cursor to avoid boundary loss. The -project owner chose the existing behavior where applying a manual stock -dividend advances `dividendCheckedAt`; the fetch feature must preserve that -contract unless separately redesigned. - -## Risks - -- SSI can change or remove the endpoint without notice. -- Decimal ratios require exact conversion to the bot's `owned:new` integer - format; do not use binary floating-point for entitlement math. -- An `ISS` event is not necessarily a dividend. -- Publication, ex-right, record, payment, and tradable-share dates are distinct. -- Multiple historical events can exist in one cursor window; the UI must show - stable IDs and dates so users can apply the correct event manually. - -## References - -- [SSI iBoard live corporate-actions example](https://iboard-api.ssi.com.vn/statistics/company/corporate-actions?symbol=ACB&fromDate=01%2F06%2F2026&toDate=30%2F06%2F2026&page=1&pageSize=50) -- [SSI FastConnect API documentation](https://guide.ssi.com.vn/ssi-products/fastconnect-data/api-specs) -- [VSDC official notices](https://www.vsd.vn/vi/) -- [TCBS dividend guidance](https://help.tcbs.com.vn/tra-cuu-co-tuc/) -- [FiinGroup API Datafeed](https://fiingroup.vn/vi/giai-phap-du-lieu-chung-khoan-api-datafeed-microsite.html) - -## Next Steps - -1. Approve SSI iBoard as the initial replaceable provider. -2. Specify the Telegram command and event-selection interaction. -3. Implement exact ratio normalization, pagination, overlap, deduplication, - fixtures, and failure behavior. -4. Reassess a licensed feed if this becomes production-critical. - -## Unresolved Questions - -- Which command should fetch events, and should it inspect one ticker or every - stock asset? -- Should selecting an event prefill guidance only, or execute the existing - manual dividend command after explicit confirmation? diff --git a/plans/reports/260722-1112-compact-portfolio-number-format.md b/plans/reports/260722-1112-compact-portfolio-number-format.md deleted file mode 100644 index 996649d..0000000 --- a/plans/reports/260722-1112-compact-portfolio-number-format.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -title: Compact portfolio number format -status: approved -created: 2026-07-22 -tags: [stock, coin, formatting, telegram] ---- - -# Compact Portfolio Number Format - -## Summary - -Stock and coin position tables need shorter monetary cells without hiding each -value's magnitude. Use automatic per-cell financial suffixes for the `Avg`, -`Now`, `Value`, and position-level `Unrealized P&L` columns. - -## Problem - -Full monetary values make Telegram's monospace portfolio tables wide and harder -to scan. A single fixed scale fails because stock and coin prices and position -values span several orders of magnitude. - -Evidence is direct user feedback on the rendered portfolio. Success means the -four position columns become visibly narrower while their currencies, -magnitudes, signs, and P&L percentages remain understandable. - -## Requirements - -- Scale absolute values automatically: base below 1,000, then `k`, `M`, `B`, - and `T` at successive powers of 1,000. -- Show at most three fractional digits and trim trailing zeroes. -- Preserve each module's convention: stock uses dot grouping and comma decimal; - coin uses `$`, comma grouping, and dot decimal. -- Keep signs and P&L percentages unchanged. -- Apply compact formatting only to position-table `Avg`, `Now`, `Value`, and - `Unrealized P&L` monetary amounts. -- Keep summary tables and standalone trade, price, top-up, and dividend replies - unchanged. -- Keep `N/A` behavior unchanged when a price or valuation is unavailable. - -## Evaluated Approaches - -### Automatic magnitude per value — selected - -Examples: stock `25,35k`, `126M`, `1,25B`; coin `$950`, `$50k`, `$1.234M`. -Each value carries its scale, so mixed rows remain unambiguous. Three fractional -digits balance width and display precision. - -### Fixed unit per column - -Predictable alignment, but produces awkward small values such as `0,125M` and -cannot fit the full range of coin prices cleanly. - -### Unit in headers - -Produces the narrowest cells, but one header scale cannot represent mixed -magnitudes without exceptions and ambiguity. - -## Decision - -Use the common financial suffix set `k/M/B/T`. Do not use strict SI `G` for -billion because it is unfamiliar in financial displays. Do not use Vietnamese -`tr/tỷ` because the requested stock-and-coin style should remain uniform and -compact. - -## Implementation Considerations - -- Keep compact helpers private to their modules so exported formatting - contracts remain stable. -- Select a suffix from the absolute value, then preserve the original sign. -- Round to at most three scaled fractional digits before trimming zeroes. -- Add boundary tests around 1,000, 1M, 1B, 1T, negatives, and rounding rollover. -- Update renderer tests for both fully priced and unavailable-price paths. - -## Success Criteria - -- `999` remains unscaled; `1,000` becomes `1k`; `1,000,000` becomes `1M`. -- Stock examples use `25,35k` and `126M`. -- Coin examples use `$50k` and `$1.234M`. -- Position P&L amounts compact while percentages remain exact to two decimals. -- Summary and non-portfolio messages retain current output. -- Focused tests, full tests, vet, and lint pass. - -## Next Steps - -Create an implementation plan, revise the preliminary stock-only formatter, -add the coin formatter and renderer integration, then verify both modules. - -## Unresolved Questions - -None. diff --git a/plans/reports/260722-1356-per-user-dividend-history-brainstorm.md b/plans/reports/260722-1356-per-user-dividend-history-brainstorm.md deleted file mode 100644 index 1ae6d57..0000000 --- a/plans/reports/260722-1356-per-user-dividend-history-brainstorm.md +++ /dev/null @@ -1,143 +0,0 @@ ---- -type: brainstorm-report -topic: per-user-dividend-history -conducted_at: 2026-07-22T13:56:06+07:00 -status: approved ---- - -# Brainstorm Report: Per-User Dividend History - -## Problem - -Stock dividend discovery currently depends on `assets..dividendCheckedAt` -and a global hashed `appliedDividendEvents` ledger. That model offers SSI events -immediately, loses the discovery cursor on a full exit, duplicates trusted event -data in pending buttons, and cannot defer approval reliably until Record date. - -The desired behavior is to check SSI when `/stock_portfolio` runs, notify users -about future events, expose approval only from Record date, retain enough event -data to refresh incomplete notices, and remove old history after 90 days. - -## Constraints Discovered - -- SSI queries are filtered by publication date, not Record date. An event can - leave a rolling 30-day feed before its Record date arrives. -- SSI `corId` lookup is not reliable enough to make ID-only persistence safe. -- Full sales delete `assets.`, so nesting processed history there would - allow the same event to be rediscovered after a sell and rebuy. -- Existing applied-event keys are one-way hashes without ticker identity and - cannot be losslessly converted to raw SSI IDs. -- The bot does not persist historical lots. Eligibility can only be bounded by - the current position and its `openedAt` lifecycle marker. - -## Approaches Considered - -### Per-user dividend history - -Persist full normalized SSI details under -`portfolio.dividends..`, separate from active assets. -This keeps approval self-contained, survives full exits, and gives every user -an explicit notification and processing lifecycle. - -### Global event cache with per-user references - -Normalize SSI events once globally and keep only user state in each portfolio. -This reduces duplicated provider data but adds cross-document consistency, -retention, and callback failure modes that are unnecessary at current scale. - -### Event IDs only with SSI refetch - -Persist only IDs and retrieve financial details at approval time. This is the -smallest schema but is unsafe because old publication windows move out of the -recent feed and SSI does not provide dependable lookup by ID. - -## Approved Design - -Use per-user dividend history: - -```text -portfolio -|- assets. -| |- quantity -| |- base -| `- openedAt -`- dividends.. - |- normalized SSI kind, dates, amount/ratio, title, and source URL - `- processed -``` - -The SSI event ID is the dynamic map key rather than a duplicated record field. -Raw IDs are already constrained to BSON-safe characters by the SSI adapter. - -## Discovery and Refresh Flow - -On `/stock_portfolio`: - -1. Render the portfolio even if SSI later fails. -2. Fetch events published in the exact preceding 30 days for held tickers. -3. Upsert normalized details while preserving local notification and processed - state. -4. Re-query the original publication-date window for stored events whose Record - date is missing. Before creating an actionable message, refresh the event - once more so SSI corrections are captured. -5. Send an informational message for every future unprocessed event after each - portfolio request. -6. At or after the start of Record date in Asia/Saigon, send a separate approval - message for every unprocessed eligible event after each portfolio request. - -An event with no Record date remains informational-only until SSI fills the -date. Re-fetches match the original ticker and raw SSI event ID. - -## Approval and Idempotency - -Pending actions retain only the opaque token bindings and references needed to -locate the user dividend record. Financial values are read from the trusted -portfolio record during callback handling. - -Approval requires a current position, `openedAt` no later than Record date, a -valid owner/chat/message-bound token, and `processed == false`. Calculations use -the current quantity because dated holdings are outside scope. Portfolio -mutation and `processed = true` are persisted together under the existing -per-user lock. Repeated requests may create multiple buttons; after the first -successful approval, all later buttons are rejected by the processed marker. - -## Retention and Migration - -- Delete every dividend 90 days after its Record date, processed or not. -- If Record date remains missing, delete it 90 days after publication so - malformed provider rows cannot persist forever. -- Remove `dividendCheckedAt` from asset positions and manual dividend behavior. -- Remove the old hashed `appliedDividendEvents` field; the owner explicitly - accepts dropping that legacy duplicate ledger. -- Run an idempotent startup migration over `user:` stock documents and record - completion in the shared `system` collection, ensuring obsolete MongoDB fields - are physically removed rather than waiting for organic portfolio writes. - -## Boundaries - -- Manual dividend commands remain independent because they have no SSI ID. -- No dated-lot or legal-entitlement accounting is introduced. -- No automatic dividend application occurs. -- No command names or parameters change. -- SSI remains a best-effort, replaceable provider. - -## Risks and Mitigations - -- Provider corrections: refresh incomplete records and refresh again before an - actionable message. -- Duplicate processing: allow repeated notifications but use the persisted - processed marker to ensure only the first approved button mutates finances. -- Post–Record date rebuy: reject when the current position opened after Record - date. -- Partial provider failure: keep the portfolio response and existing stored - history intact; retry on a later request. -- Concurrent callbacks: recheck state under the user lock and save processing - atomically with the financial mutation. - -## Approval - -The project owner selected per-user dividend history and approved storing full -event details outside `assets`, historical refetch for missing Record dates, -Record-date approval gating, and 90-day retention. The owner later revised the -delivery rule so every portfolio request repeats every unprocessed event until -processing or expiry, even if SSI later omits the retained event. diff --git a/plans/reports/260726-0941-lol-score-source-alternatives.md b/plans/reports/260726-0941-lol-score-source-alternatives.md deleted file mode 100644 index 3f7afcf..0000000 --- a/plans/reports/260726-0941-lol-score-source-alternatives.md +++ /dev/null @@ -1,80 +0,0 @@ -# Research: Alternative LoL Result Source (restore real scores) - -Conducted 2026-07-26 09:41 ICT. Trigger: lolesports marks matches `completed` without publishing `result.gameWins`/`outcome`, so the digest now shows `☑️ … score pending` (commit `570ee94`) instead of a score. - -Goal: source that has real series scores for finished matches, so `✅ MKOI 2–0 KC` returns. - -## Executive Summary - -**Any wrapper of lolesports inherits the bug.** That eliminates most "alternative LoL API" search hits — they proxy the same upstream store. Verified empirically: `getCompletedEvents` returns the identical `MKOI 0-0 KC`, so the gap is store-wide at Riot, not endpoint-specific. - -Only genuinely independent ingestion helps: community wikis (Leaguepedia, Liquipedia) or commercial providers (PandaScore, Abios, GRID). - -**Recommendation: don't build it yet.** The schedule half of lolesports is reliable; only results are broken, and they self-heal. A second provider means a new HTTP dep + team-name→code mapping + cache + failure handling — real complexity for a cosmetic win on an intermittent upstream failure. Current behavior is honest. Wait, measure how often/long Riot stalls, and build the enrichment only if chronic. - -If it does prove chronic → **Leaguepedia Cargo, as opportunistic enrichment, not migration.** - -## Empirical Findings (this session) - -| Probe | Result | -|---|---| -| `getSchedule` | 6 events `completed` w/ `{"outcome":null,"gameWins":0}`; clean cutoff — all ≤`2026-07-25T11:30Z` scored, all ≥`14:30Z` not | -| `getEventDetails` (match `115548681803406271`) | games 1&2 `state:unstarted` but **7 VODs each**; game 3 `unstarted`/0 VODs → played + broadcast, results never ingested | -| `getEventDetails` (scored control) | games `completed`/8 VODs, game 3 `unneeded` | -| `getCompletedEvents?tournamentId=115548681802226458` | **same staleness** — `MKOI 0-0 KC`. Rules out a free same-provider fix | -| `getStandings` | active stage returns `Playoffs` only; no useful Week-1 series data | -| Leaguepedia Cargo `MatchSchedule` | API works, returns populated `Team1Score`/`Team2Score`/`Winner` for finished matches. **Heavily throttled anonymously — 1 success in ~15 attempts** | - -Notable: the one successful Cargo query returned scored results for matches at `2026-07-25 12:00–14:00Z` — inside the window Riot has failed on (cutoff `11:30Z`). Suggestive that Leaguepedia is ahead of Riot there, but those were amateur/university leagues with different editors, so **not** proof for LEC. - -## Provider Comparison - -| Provider | Cost | Auth | Independent ingestion? | Verdict | -|---|---|---|---|---| -| **Leaguepedia** (Fandom Cargo) | free | none | ✅ community-edited | **Best fit.** `MatchSchedule.Team1Score/Team2Score/Winner/BestOf`. Harsh anon rate limit; uses team *names* not codes; CC-BY-SA attribution required | -| **Liquipedia** LPDB | free tier | API key | ✅ | 60 req/hr. Free tier needs non-commercial + **open-sourced project** + custom UA + attribution. Repo is public → likely qualifies. Must apply for key | -| **PandaScore** | free tier 1k req/hr (schedules+results); paid from €400/mo/game | key | ✅ | Free tier may suffice for a daily digest. Verify results are in free scope — pricing implies historical is paid. Betting-adjacent ToS restrictions on stats plans | -| **Abios / GRID / Sportradar** | $2k–10k/mo | key | ✅ | Absurd for a Telegram bot. **Rule out** | -| lolesportsapi.com | free 500/mo | key | ❌ wraps lolesports | **Useless** — inherits the bug | -| Pupix/lol-esports-api | free | — | ❌ wrapper | **Useless** | -| Apify LoL scraper | paid compute | token | ❌ scrapes lolesports | **Useless** | - -## Recommended Design (if/when built) - -Enrichment layer, not a migration. lolesports stays authoritative for schedule, times, Bo count, block names — all correct today. - -``` -GetEventsWithFallback (lolesports) → events - └─ for events where state==completed && !scoreIsPublished(t1,t2): - └─ 1 batched Leaguepedia Cargo query for the day window - ├─ hit → fill gameWins + outcome → renders ✅ with real score - └─ miss → unchanged ☑️ … score pending -``` - -Properties: -- `scoreIsPublished` (already shipped) is the exact trigger — no new detection logic -- 1 extra request per digest, only when a gap exists; cache alongside existing `cacheRecord` -- Degrades to current behavior on any failure — strictly additive, no regression path -- Never overwrites a Riot-published score; fills gaps only - -Work required: -1. Team identity mapping — Leaguepedia returns `Movistar KOI`, we display `MKOI`. Cargo `Teams` table has a `Short` field; needs a cached lookup or a static map for the ~12 allowlisted leagues -2. `OverviewPage` naming per league/split (e.g. `LEC/2026 Season/Split 3`) — brittle, changes each split. Alternatively match on team names + UTC timestamp, which avoids page naming entirely -3. Rate-limit discipline — cache aggressively, custom UA w/ contact, single daily query -4. CC-BY-SA attribution somewhere user-visible - -## Cost/Benefit - -Against: new external dep on a community wiki (editorial lag, schema drift, throttling); name→code mapping is ongoing maintenance; ~150–250 LOC + tests; benefit is cosmetic — current output is already truthful. - -For: restores the informative digest; useful independently if Riot's stalls become routine. - -**Verdict: defer.** Cheapest next step is observation, not code — the current fix already fails safe. - -## Unresolved Questions - -1. **Could not verify** Leaguepedia holds the specific missing LEC/LCS scores (`MKOI vs KC`, `FLY vs LYON`, `DIG vs SEN`) — rate-limited on every targeted attempt; WebFetch of the wiki page returned HTTP 402. **This is the single blocking unknown**; retry the Cargo query later before committing to Leaguepedia. -2. Is Riot's stall a one-off or recurring? No history collected. Determines whether any of this is worth building. -3. Does PandaScore's free tier actually include completed-match results, or schedules only? Pricing page ambiguous. -4. Does Liquipedia grant LPDB keys to a public hobby Telegram bot? Terms say non-commercial + open-source, which fits, but approval is manual. -5. Leaguepedia editorial latency for major leagues specifically — unmeasured. If editors lag Riot, the enrichment adds nothing. diff --git a/plans/reports/algorithm-comparison-260808-2257-amlich-truncated-vs-meeus49-report.md b/plans/reports/algorithm-comparison-260808-2257-amlich-truncated-vs-meeus49-report.md deleted file mode 100644 index 52f23c1..0000000 --- a/plans/reports/algorithm-comparison-260808-2257-amlich-truncated-vs-meeus49-report.md +++ /dev/null @@ -1,43 +0,0 @@ -# Amlich Converter: Algorithm Comparison & Decision - -**Date:** 2026-08-08 | **Verdict: KEEP current implementation** (Hồ Ngọc Đức truncated-Meeus port, `internal/modules/amlich/lunar.go`) - -## Process - -1. Two parallel research agents: - - Algorithms: `plans/reports/researcher-260808-2252-lunar-conversion-algorithms-report.md` - - Calendar rules: `plans/reports/researcher-260808-2252-vietnamese-lunisolar-rules-report.md` -2. Implemented candidate B manually from the rules spec: Meeus *Astronomical Algorithms* ch. 49 full new-moon series (J2000 epoch, 25 periodic + 14 planetary terms), apparent solar longitude with nutation+aberration (ch. 25), Espenak–Meeus piecewise ΔT, month 11 anchored by bisecting the winter-solstice instant. -3. Diffed A (current) vs B over every day of 1800–2199 (146,097 days); validated both against external truth (Tết dates, leap-month tables incl. 2033 leap month 11, national holidays, Tết 2007 = Feb 17, Tết 1968 = Jan 29). - -## Results - -- B internally sound: full-range round-trip + day-continuity pass; all external vectors pass; leap-month table identical to A. -- A vs B differ on **270 days (0.18%)** = **9 lunations**, every one a new moon within **±2 min of 17:00 UT (= UTC+7 midnight)**: 1944-06-20, 1967-07-07, 2072-12-09, 2077-11-15, 2130-05-07, 2150-05-26, 2159-05-17, 2175-01-22, 2199-01-26. Espenak's phase catalog lists the two historical ones at exactly 17:00 UT. - -## Adjudication of the 9 disputes - -- **20/6/1944**: published VN calendar = 30/4 nhuận (new month starts Jun 21). A ✓, B ✗. -- **7/7/1967**: published VN calendar = 30/5 Đinh Mùi (month 6 starts Jul 8). A ✓, B ✗. - - Independent confirmation: pre-1968 Vietnam used the Chinese-calendar convention (UTC+8; reform decree 8/8/1967 effective 1968). At UTC+8 these ~17:00 UT new moons land unambiguously on the next civil day (01:00 local) — HKO tables agree. A's output coincides with the historical record; B's sharper instant (16:58–17:00 UT) lands on the wrong side. -- **Remaining 7 (2072–2199)**: unverifiable. ΔT extrapolation uncertainty (tens of seconds to ±100 s) exceeds the 1–2 min margins, so neither engine can claim those days. No demonstrated benefit for B. - -## Decision rationale - -1. A matches published Vietnamese calendars on **all** verifiable dates, including both razor-edge historical disputes; B fails those two. -2. A is the de-facto standard (HND algorithm) used by essentially all Vietnamese calendar apps/sites — bot output agrees with what users cross-check against. -3. B's precision advantage only manifests in 2072+ where truth is unknowable (ΔT); YAGNI. -4. A is simpler (no bisection, fewer series terms); KISS. - -Candidate B (`lunar_rules.go`) deleted after evaluation; its value is captured as regression tests. - -## Changes kept in repo - -- `lunar_test.go`: two new `knownDates` entries pinning 20/6/1944 → 30/4 nhuận and 7/7/1967 → 30/5 (guards against future "higher-precision" rewrites flipping them). -- Earlier this session (prior task): lunation-overshoot fix in `solarToLunar` + full-range round-trip/continuity tests + leap-month table test. Full suite green. - -## Unresolved questions - -1. Should the module model pre-1968 UTC+8 convention explicitly? A matches the published record on the two known razor-edge cases by numeric coincidence, not by rule. A true UTC+8 pre-1968 mode would change ~1/24 of pre-1968 month boundaries vs current output and diverge from the HND JS standard everyone else uses — recommend NO unless a user reports a concrete mismatch. -2. 2072+ razor-edge lunations (7 dates listed above) are inherently uncertain; if the state calendar bureau publishes tables past 2100, revisit. -3. xemlicham.com's engine may itself derive from HND (partial circularity); mitigated by HKO cross-check and the pre-1968 UTC+8 argument, but a scan of Trần Tiến Bình's printed 1901–2100 tables would be the gold-standard confirmation. diff --git a/plans/reports/debug-260805-1419-lol-api-403-root-cause-report.md b/plans/reports/debug-260805-1419-lol-api-403-root-cause-report.md deleted file mode 100644 index 9d63ce3..0000000 --- a/plans/reports/debug-260805-1419-lol-api-403-root-cause-report.md +++ /dev/null @@ -1,76 +0,0 @@ -# Debug: LoL schedule fetch 403 Forbidden — root cause - -Date: 2026-08-05 | Module: `internal/modules/lol` | Status: root cause confirmed, no code changed - -## Symptom - -``` -WARN lol_fetch status=403 body={"message":"Forbidden"} -ERROR lol_fetch_fail err="lol API HTTP 403" -``` - -Every call to `https://esports-api.lolesports.com/persisted/gw/getSchedule` fails; stale cache (60 min) expires, so `/lol` commands and daily push break. - -## Root cause - -Riot revoked/rotated the long-standing public web-client API key hardcoded at `internal/modules/lol/api_client.go:34` (`0TvQnueqKa5mxJntVWt0w4LpLfEkrV1Ta8rQBb9Z`). No replacement public key exists: the rebuilt lolesports.com no longer ships a key to the browser at all. - -## Evidence - -1. Reproduced from this host: `getSchedule` with the key → HTTP 403, body `{"message":"Forbidden"}`, headers `x-amzn-errortype: ForbiddenException` + `x-amz-apigw-id` present. Request reached AWS API Gateway and Gateway itself rejected the key — not a CloudFront/WAF/IP edge block (an edge block would not carry `x-amz-apigw-id`). -2. Same 403 with no key, and with browser UA + `Origin`/`Referer: lolesports.com` — rules out UA/referer/origin gating; the key itself is dead. -3. Downloaded all 49 JS chunks from current lolesports.com (Next.js on Netlify). No `esports-api.lolesports.com` reference, no embedded key. Bundle shows Apollo client with `uri: () => ESPORTS_EDGE_URL ?? "/api/gql"` and key injected server-side from `process.env.ESPORTS_API_KEY` — key moved behind their BFF, never sent to browsers. -4. Probed `POST https://lolesports.com/api/gql` anonymously → HTTP 400 `PERSISTED_QUERY_ID_REQUIRED`: proxy is publicly reachable but accepts only pre-registered persisted-query IDs, no freeform GraphQL. - -## Why now - -Riot migrated lolesports.com to a new Next.js/GraphQL stack; the legacy `persisted/gw` REST API's public key was revoked as part of that migration. The code comment at `api_client.go:5-7` ("if Riot ever rotates it, lift the new value from their public JS bundle") no longer works — there is nothing to lift. - -## Fix options (not implemented) - -1. **Call `lolesports.com/api/gql` with persisted-query IDs** — sniff the ID + variables the live site sends for its schedule query (browser devtools), replay with `extensions.persistedQuery.sha256Hash`. Anonymous access works. Fragile: IDs change on each frontend deploy; needs re-sniff on breakage. Response shape differs from current `schedulePage` → parser rewrite. -2. **Community mirror APIs** (e.g. Leaguepedia/Fandom cargo API, third-party esports APIs) — stable contracts, but different data model and licensing/ToS review needed. -3. **Official Riot esports data** — Riot has no public esports schedule API in the developer portal; production key does not cover lolesports data. -4. **Keep module, degrade gracefully** — if no replacement chosen, surface "schedule source unavailable" to users instead of daily error logs; disable daily push until fixed. - -Recommendation: option 1 for continuity (accept fragility, keep 60-min stale-cache fallback and add alerting on repeated 403/400), else option 4 short-term. - -## Follow-up: RGAPI key + LoL Esports Data Portal (checked 2026-08-05) - -- User's `RGAPI-…` dev-portal key: valid (200 on `vn2.api.riotgames.com/lol/status/v4`), still 403 on `esports-api.lolesports.com` with both `x-api-key` and `X-Riot-Token`. Dev portal keys only cover `*.api.riotgames.com`; no esports schedule product there. -- Riot dev diary "Introducing the New LoL Esports Data Portal": LDP built with Bayes Esports, replaced LEGs/ACS; targets pro teams/partners; community access "planned". -- Bayes Esports acquired by GRID (bayesesports.com 301 → grid.gg). LoL data now behind GRID's paid portal (`grid.gg/get-league-of-legends/`). GRID Open Access (free tier) covers CS2/Dota2 only, and Series Events (schedules) is paid-tier even there. -- Net: the official replacement is commercially gated; not viable for the bot. Fix options unchanged (persisted-query proxy / Leaguepedia / graceful degrade). - -## Solution options (validated 2026-08-05) - -1. **lolesports.com `/api/gql` persisted queries** — recommended primary. Anonymous access verified; needs sniffed sha256Hash + variables from site frontend; parser rewrite; alert on `PERSISTED_QUERY_NOT_FOUND`; keep stale cache. -2. **PandaScore free tier** — `/lol/matches/upcoming`, token auth, ~1k req/h hobby tier, non-commercial. Stable contract; naming differs from Riot. Verify signup still self-serve. -3. **Leaguepedia Cargo API** (`MatchSchedule` table) — live test from this host: immediate `ratelimited` (anon datacenter IP throttled). Needs Fandom bot-password auth + CC BY-SA attribution. -4. **GRID** — official successor, paid for LoL; Open Access free tier = CS2/Dota2 only, schedules excluded. Not viable now; recheck quarterly. -5. **Scrape `lolesports.com/schedule` RSC payload** — fallback only; more fragile than option 1, no advantage. -6. **Graceful degrade** — regardless of source: user-facing "source unavailable" message + pause daily push on repeated failure. - -Recommendation: 1 primary, 2 as designed fallback. - -## Implemented (2026-08-05): option 1, gql persisted-query client - -No browser available (headless ARM64 server) — persisted-query ID extracted statically instead of via devtools: - -1. Bundle analysis: lolesports.com Apollo client uses `generatePersistedQueryIdsFromManifest` — IDs come from a build manifest, NOT computed sha256 (both `__meta__.hash` and sha256(print(doc)) are rejected by the gateway). -2. Manifest = webpack chunk 29 (`loadManifest:()=>o.e(29)…` in bundle); URL resolved from webpack runtime chunk-hash map → `/_next/static/chunks/29..js`; contains `{name, id, body}` per operation. -3. Operation chosen: `homeEvents` — date-window filter (`eventDateStart/End`), pagination (`pages.newer` + `pageToken`), events carry startTime/state/type/blockName/league/match.strategy/matchTeams(result.outcome/gameWins). Same semantic contract as dead REST API. States verified identical: unstarted/inProgress/completed; pending result = `{outcome:null,gameWins:0}` — `scoreIsPublished` logic still correct. -4. Gateway requirements discovered: `apollographql-client-name` + `-version` headers mandatory ("No client headers set"); `sport` enum lowercase `["lol"]`; Date params `YYYY-MM-DD`. - -Code changes (`internal/modules/lol/`): -- `api_client.go`: transport rewritten — POST /api/gql persisted op `homeEvents` (ID const + refresh procedure in package doc). Removed dead public API key. `gqlEvent→ScheduleEvent` mapping keeps module contract + bson cache shape unchanged. Date window padded ±1 day, exact [from,to) filtered client-side (gateway date tz semantics unspecified). In-band GraphQL errors (HTTP 200 + errors[]) surface as fetch errors; `PERSISTED_QUERY_NOT_IN_LIST` logged as `lol_persisted_query_rotated` ERROR with refresh hint. Older-page walk removed (window is query-bounded; newer-walk only). -- `api_client_test.go`, `handlers_test.go`: fixtures → gateway shape; new tests for gql-error-as-fetch-error and exact-window filtering. -- format.go/handlers.go/cron.go/subscribers.go untouched. - -Verification: module tests green; live probe against real gateway returned 72 events for today+tomorrow incl. an in-progress LCK match, FilterMajor→10; `go test ./...`, `go vet`, `golangci-lint` all clean. Stale-cache fallback (60 min) retained. - -## Unresolved questions - -- Whether Riot will publish an official replacement API (watch RiotGames/developer-relations). -- Persisted-query ID churn rate on lolesports.com deploys (determines option 1 maintenance cost). -- PandaScore free-tier availability/terms as of now (not yet verified with a signup). diff --git a/plans/reports/known-issues-260808-2319-amlich-post-1970-report.md b/plans/reports/known-issues-260808-2319-amlich-post-1970-report.md deleted file mode 100644 index b7c7059..0000000 --- a/plans/reports/known-issues-260808-2319-amlich-post-1970-report.md +++ /dev/null @@ -1,31 +0,0 @@ -# Amlich Known Issues — Dates ≥1970 (Future Reference) - -**Date:** 2026-08-08 | Scope: `internal/modules/amlich`, usage restricted to dates from 1970 onward. -Context: follows `algorithm-comparison-260808-2257-amlich-truncated-vs-meeus49-report.md` (decision: keep HND truncated-Meeus). - -Restricting to ≥1970 removes the pre-1968 UTC+8 convention issue entirely (UTC+7 official since 1968 reform). - -## Real risks - -1. **Razor-edge lunations ≥2072** — new moon within ~2 min of UTC+7 midnight; month boundary (±~30 days of output) may differ ±1 day from future official tables. Dates: 09/12/2072, 15/11/2077, 07/05/2130, 26/05/2150, 17/05/2159, 22/01/2175, 26/01/2199. Inherently undecidable today (see 2); not fixable, only documentable. -2. **ΔT extrapolation drift** — two-branch polynomial (and NASA's) diverge from actual Earth rotation (ΔT ≈ 69 s, flat since ~2016 vs predicted growth). Minutes-level error by 2100+; only matters in case-1 situations. Everything ≥ ~2070 = best-effort prediction. -3. **Solar-term precision** — `sunLongitude` = true longitude, no nutation/aberration (~10 min systematic in term timing). Solstice/trung khí within ~10 min of midnight could flip month numbering / leap-month placement for a whole lunar year. A/B diff 1800–2199 found zero occurrences (all diffs were case-1 new moons); low probability, high impact. -4. **No official-override hook** — pure astronomy; a state-decreed deviation or rule change would silently diverge. None post-1968 to date. - -## Support noise (correct, looks like bugs) - -5. **VN–China divergence years** (~2030, ~2053; historical 1985, 2007): bot differs from Chinese-calendar sources by design; expect user reports. -6. **`/duonglich` default in leap months** (2028 leap 5, 2031 leap 3, …): bare day arg fills current month as regular month, leap flag requires explicit `nhuan`. UX ambiguity, not wrong output. - -## Already handled - -- Overshoot day-0 dates ≥1970 (07/05/2054, 09/04/2062): fixed, regression-tested (`TestSolarToLunar_LunationOvershoot`). -- 2033 leap month 11: correct, covered by `TestLeapMonthTable`. -- Gregorian 2100 non-leap: handled by JD math. -- Bounds 1800–2199: clean rejection outside range. - -## Unresolved questions - -1. Should the 7 razor-edge lunations be surfaced to users (e.g., a caveat in bot reply for affected months ≥2072)? Currently silent. -2. Revisit ΔT model if IERS trends persist (ΔT flat/decreasing); a data-driven update only pays off near case-1 boundaries. -3. Monitor state calendar bureau publications past 2100 for future ground truth. diff --git a/plans/reports/research-260805-1627-stable-free-lol-schedule-source-report.md b/plans/reports/research-260805-1627-stable-free-lol-schedule-source-report.md deleted file mode 100644 index 74f45dc..0000000 --- a/plans/reports/research-260805-1627-stable-free-lol-schedule-source-report.md +++ /dev/null @@ -1,65 +0,0 @@ -# Research: stable free LoL esports schedule source (replace/back up gql client) - -Date: 2026-08-05 16:27 ICT | Sources: 5 web searches + live probes from bot's server | Context: commit b76d0ca (gql persisted-query client) works but breaks when Riot rotates the persisted-query ID. - -## Executive summary - -No free source beats the current gql client on data quality (official, live state, scores, league slugs). The stability gain is available two ways: **Leaguepedia** (no signup, proven decade-old MediaWiki API, works anon from this server — verified live today) or **PandaScore free tier** (most stable API contract, 1000 req/h, but adds account+token dependency). Recommendation: keep gql client primary, add Leaguepedia as the fallback source; skip PandaScore unless the user prefers a commercial-grade contract over zero-dependency. - -## Candidates - -### 1. Leaguepedia Cargo API — recommended fallback -- Endpoint: `https://lol.fandom.com/api.php?action=cargoquery`, table `MatchSchedule` (Team1, Team2, DateTime_UTC, BestOf, Winner, scores, OverviewPage, Stream…). -- **Verified live from the bot's server (2026-08-05):** proper descriptive UA + single paced request → HTTP 200 with correct data. Earlier `ratelimited` was the anon throttle (~1 req/min); burst of 2 rapid requests trips it. -- Cross-validated against lolesports gateway: same match, same start time (Estral vs Ei Nerd 00:00 UTC) — data agrees. -- Auth: Fandom account + bot password raises limits (exact numbers undocumented; anon ~1/min per mediawiki-api list thread). Anon is enough for this bot: 1 daily push + user commands behind 60-min cache, if requests are serialized. -- License: CC BY-SA — attribution required (one line in /help or bot bio suffices). -- Cons: different data model — full team names not codes, league via `OverviewPage` string not slug (needs mapping for FilterMajor), no live `inProgress` state (has Winner/score for finished). Community-entered data (major leagues near-instant, minor leagues can lag). -- Stability: MediaWiki+Cargo API unchanged for ~a decade; immune to Riot frontend deploys. - -### 2. PandaScore free tier — most stable contract -- 1000 req/h free, no credit card (pricing page). Versioned REST (`/lol/matches/upcoming`, `/lol/matches/past`), token auth, real docs. -- Cons: signup + API token to manage (new secret in env); team/league naming differs; free-tier terms can change unilaterally; third-party (not official Riot data, though sourced professionally). -- Best choice if a long-lived stable contract matters more than zero-dependency. - -### 3. Liquipedia LPDB API — viable but stricter -- Free tier 60 req/h; **requires open-source project + non-commercial**; CC-BY-SA attribution; custom UA with contact mandatory; MediaWiki api.php capped 1 req/2s. -- No advantage over Leaguepedia for LoL specifically; stricter terms. Skip. - -### 4. Community iCal feeds (zlypher/lol-events, olol/lolesp-cal) -- Free, zero auth, but single-maintainer projects (staleness risk), ICS carries no scores/state. Emergency backup grade only. zlypher feed last updated 2026-07 and itself sources Liquipedia. - -### 5. lolesportsapi.com -- Commercial wrapper, free 500 calls/month; unknown operator that itself scrapes Riot. Adds a middleman with less stability than our own gql client. Skip. - -## Comparative summary - -| Source | Signup | Quota (free) | Live state | Scores | Deploy-proof | Data model fit | -|---|---|---|---|---|---|---| -| Current gql client | none | ample | ✅ | ✅ | ❌ ID rotates | ✅ native | -| Leaguepedia | none | ~1/min anon | ❌ | ✅ | ✅ | ⚠️ mapping needed | -| PandaScore | account+token | 1000/h | ✅ | ✅ | ✅ | ⚠️ mapping needed | -| Liquipedia | none | 60/h LPDB | ❌ | ✅ | ✅ | ⚠️ + strict terms | -| iCal feeds | none | n/a | ❌ | ❌ | ⚠️ maintainer | ❌ | - -## Recommendation - -1. Keep gql client primary (richest data, official). -2. Implement Leaguepedia as fallback in `GetEventsWithFallback`'s failure path (after stale cache): serialize requests, ≥60s spacing, descriptive UA, map OverviewPage→league slug for the major-league allowlist, render without live-state features. Zero new secrets or accounts. -3. Reconsider PandaScore only if Leaguepedia's anon limit proves painful in practice. -4. Regardless: alert on `lol_persisted_query_rotated` so ID refresh happens fast (fallback covers the gap). - -## Sources - -- [Leaguepedia rate limit thread (mediawiki-api list)](https://lists.wikimedia.org/hyperkitty/list/mediawiki-api@lists.wikimedia.org/thread/A6VWUYRHLGGJWZ3USGEBQJSDMX6A4YCM/) -- [PandaScore pricing](https://www.pandascore.co/pricing) -- [Liquipedia API terms](https://liquipedia.net/api-terms-of-use), [usage guidelines](https://liquipedia.net/commons/Liquipedia:API_Usage_Guidelines) -- [zlypher/lol-events iCal](https://github.com/zlypher/lol-events), [olol/lolesp-cal](https://olol.github.io/lolesp-cal/) -- [river-mwclient (Leaguepedia wrapper by its dev)](https://pypi.org/project/river-mwclient) -- Live probes: Leaguepedia cargoquery 200 from bot host (2026-08-05), cross-checked vs lolesports gateway. - -## Unresolved questions - -- Leaguepedia authenticated (bot-password) rate limit exact number — undocumented; ask on their Discord if anon 1/min ever binds. -- PandaScore free-tier ToS details (commercial-use clause) — not verified beyond "no credit card"; check at signup if pursued. -- Whether the bot repo being public matters to the user (required only for Liquipedia, not Leaguepedia/PandaScore). diff --git a/plans/reports/researcher-260808-2252-lunar-conversion-algorithms-report.md b/plans/reports/researcher-260808-2252-lunar-conversion-algorithms-report.md deleted file mode 100644 index 7475a1a..0000000 --- a/plans/reports/researcher-260808-2252-lunar-conversion-algorithms-report.md +++ /dev/null @@ -1,210 +0,0 @@ -# Research: Vietnamese Lunar Calendar Conversion Algorithms (1800–2199) - -**Date:** 2026-08-08 | **Scope:** Go implementation correctness for amlich module -**Recommendation:** RETAIN truncated-Meeus (current), UPGRADE ΔT handling, ADD test vectors - ---- - -## Executive Summary - -The current **Hồ Ngọc Đức truncated-Meeus algorithm** is algorithmically **sound and sufficient** for day-level Vietnamese calendar accuracy across 1800–2199. Accuracy: ±1–3 minutes for new moon timing, ±10 arcminutes for solar terms—both acceptable for calendar day boundaries. Higher-precision VSOP87/ELP2000-82 would add ~1000× complexity for <1% improvement and is **not justified** for calendar conversion. - -**Trade-off decision:** Keep current algorithm; improve ΔT polynomial to use NASA piecewise fits (1800–2200 coverage); document ±100s uncertainty spike at 2100+. - ---- - -## Algorithm Landscape & Accuracy Tiers - -### 1. Truncated-Meeus (Current Implementation) ⭐ RECOMMENDED -**Used by:** amlich.js, lunar.go (current), most Vietnamese calendar apps -**Formulas:** -- New moon k-index: `JD = 2415020.75933 + 29.53058868k + 0.0001178T² - 0.000000155T³ + [~12 perturbation terms]` - - Source: Jean Meeus, *Astronomical Algorithms* 2nd ed., Ch. 49, truncated ELP-2000/82 (Chapront) -- Sun longitude: Mean + ΔL (simplified), 1–2 degree-series coefficients - - Source: Meeus Ch. 25, low-precision form -- ΔT: Two-branch polynomial (T < –11 vs. ≥ –11, UTC decades from 1900) - -**Accuracy:** -- New moon: ±2–4 minutes (RMS) vs. full ELP2000-82 -- Solar terms: ±10 arcmin (solar disk width ~32 arcmin → 20 min uncertainty) -- **Sufficient for day-level granularity** (day boundaries shift only if NM/ST within 3–5 hours of midnight UTC+7) - -**Computational cost:** ~2–3 μs per conversion (negligible) - -**Known issues:** -- Day 0 returned for ~10 edge dates (1877-04-13, 2062-04-09) when NM falls just before local midnight; **mitigated in code** via loop in `solarToLunar` -- ΔT polynomial discontinuous; NASA piecewise approach better-calibrated 1800–2150 - ---- - -### 2. Full ELP2000-82 (Chapront) ✗ OVERKILL -**Used by:** NASA eclipse predictions, professional ephemerides -**Source:** Bureau des Longitudes; 37,862 periodic terms for Moon (full) -**Accuracy:** ±0.1–0.5 seconds (new moon), ±1 arcsecond (position) -**Cost:** ~100× slower, requires 60–100 term series even when truncated -**Verdict:** Unnecessary; lunar calendar only needs ±hours, not ±seconds. **Abandon.** - ---- - -### 3. VSOP87 + ELP2000/82 (Go Astro library) ✗ OVERKILL -**Used by:** github.com/Starainrt/astro (Chinese lunar calendar, Go) -**Accuracy:** ±0.1–0.5 arcseconds (Sun), ±0.5–2 arcseconds (Moon) -**Date range:** -104 CE to 3000 CE -**Cost:** ~100× code complexity -**Verdict:** Mathematically sound, but **1000× improvement for calendar-day use is waste**. Suitable only if implementing ephemeris, not calendrics. **Not justified for this scope.** - ---- - -## ΔT (TT−UT) Handling: Current vs. Best Practice - -### Current Implementation (lunar.go lines 100–106) -```go -if T < -11 { - deltat = 0.001 + 0.000839*T + 0.0002261*T2 - 0.00000845*T3 - 0.000000081*T*T3 -} else { - deltat = -0.000278 + 0.000265*T + 0.000262*T2 -} -``` -**Issue:** Two-branch fit is crude; NASA provides **piecewise polynomials by era** (1620–1900, 1900–1920, 1920–2005, 2005–2050, 2050–2150). - -### NASA Recommended (NASA Espenak ΔT polyomials, 2004+) -**For 1800–2200 coverage:** -- **1800–1860:** 7th-degree polynomial (Meeus pre-fit) -- **1860–1900, 1900–1920, …, 2005–2050:** 4th–5th degree fits (±0.6–4 second accuracy) -- **2050–2150:** Blended formula addressing discontinuity -- **2150–2200:** Extrapolation with tidal braking (ΔT ≈ +200 to +400 sec) - -**Uncertainty for 2100–2200:** ±100 seconds (dominated by tidal braking model uncertainty, **not algorithm accuracy**). - -**Verdict:** **Upgrade ΔT polynomial.** Current two-branch fit adequate for most dates but discontinuous; NASA piecewise improves edge-case calibration. Effort: ~50 lines, impact: +0–2 second accuracy mid-range, removes systematic bias 1850–1900. - ---- - -## Test Vectors: Verified Discrepancies - -### Known Vietnam–China Calendar Divergence (UTC+7 vs. UTC+8) -These occur when winter solstice or new moon straddles local midnight boundary. - -| Date Range | Event | Vietnam | China | Cause | -|-------------|-------|---------|-------|-------| -| 1985-01-20/21 | Winter solstice → lunar month 11 | Jan 21 (NM on Dec 21 HN time) | Feb 20 (NM on Dec 22 BJ time) | ΔT offset, solstice on different calendar days UTC+7 vs UTC+8 | -| 2007-02-17/18 | Lunar New Year (month 1, year 2007) | Feb 17 | Feb 18 | Similar; NM at UTC 23:02, Dec 20 HN (next day) vs. Dec 21 BJ | -| 2030, 2053 | Predicted future divergence | — | — | Rare alignment pattern continues ~23 years | - -**Sources:** -- Ho Ngoc Duc documentation (https://www.xemamlich.uhm.vn/): "The Winter Solstice 1984 falls on 21/12/1984 Hanoi time, but on 22/12/1984 Beijing time." -- Wikipedia Tết article: confirms 2007 divergence and rarity (3 times in 21st century) - -### Edge Cases for Testing -- **1877-04-13 (Gregorian):** Reference implementation returns day=0; current code loops to correct. Test that fix works. -- **2062-04-09:** Similar edge case documented in code comment. -- **1985-01-20/21:** Verify lunar month 11 starts on Jan 20 or 21 (test both directions). -- **2007-02-17:** Verify Tết Year 2007 calculated on Feb 17, not Feb 18. - ---- - -## Reference Publications: Authority & Coverage - -### Primary Sources -1. **Jean Meeus, *Astronomical Algorithms*, 2nd ed. (1998)** - - Ch. 49: Lunar phase (new moon), ~60-term truncated series from ELP-2000/82 - - Ch. 25: Solar coordinates, mean longitude + perturbations - - Ch. 53–54: ΔT polynomials, limited to 1620–2000 - - **Available:** Amazon, archive.org (1991 1st ed. has fewer chapters) - -2. **Hồ Ngọc Đức, "How to compute the Vietnamese lunar calendar"** - - https://www.informatik.uni-leipzig.de/~duc/amlich/calrules_en.html [**Note:** URL returned 404 in fetch; may be intermittent. Mirror at https://www.xemamlich.uhm.vn/calrules_en.html confirmed live.] - - Truncated coefficients for new moon (Ch. 49 of Meeus) - - Sun longitude simplification (Ch. 25 low-precision) - - Does NOT document ΔT formula explicitly; implementation inferred from JS reference - -3. **Trần Tiến Bình, "Lịch Việt Nam thế kỷ XX–XXI: 1901–2100"** (Culture & Info Publishing House, Hanoi, 2005) - - **Official Government calendar** per Decree 121/CP (VN Government) - - 799 pages; covers **1901–2100 only** (not 1800–1900 or 2101–2199) - - Published calendar dates serve as ground truth for 20th–21st centuries - - **Source of truth for verification**, but does not extend to 1800–2200 full range - -4. **NASA ΔT Polynomials** (Espenak, 2004+) - - https://eclipse.gsfc.nasa.gov/SEcat5/deltatpoly.html - - Piecewise formulas 1620–2150 CE - - Extrapolations to 2200 with ±100s uncertainty band - - **Authoritative for ΔT calibration** - -### Secondary Sources (Reference Implementations) -- **github.com/codeaholicguy/amlich.js** (JavaScript port of Hồ Ngọc Đức, 50+ forks, widely used) - - No explicit test cases in repo; 0 open issues (maintenance status unclear) -- **github.com/hungtrd/amlich** (Go port, 15+ stars) - - Direct port of truncated-Meeus; no test cases documented -- **github.com/Starainrt/astro** (Go Chinese lunar, VSOP87+ELP; production use) - - Covers -104 CE to 3000 CE; Chinese only, not Vietnamese; ~0.1" accuracy - ---- - -## Correctness Assessment: Truncated-Meeus Sufficiency - -### Error Budget for Day-Level Calendar -Goal: Determine if NM/solar term falls on day D or D+1 (local midnight UTC+7). - -**Worst case:** New moon occurs 12 hours before/after local midnight. -- Truncated-Meeus error: ±2–4 minutes (99th percentile) → **safe margin ≫ 12 hours** -- Solar term (sun longitude ±30°): ±10 arcmin (~10 minutes in time) → **also safe** - -**Conclusion:** Truncated-Meeus is **correct within required tolerance for 1800–2199.** - -### Known Limitations -1. **Pre-1900 ΔT:** Simple polynomial misses high-order variations in tidal braking 1750–1850. Impact: ±1–2 second error in NM, negligible for day-level. -2. **Post-2100 ΔT:** Extrapolation model breaks down; uncertainty grows ±100 seconds. **Does not affect day boundary** (only matters if NM within 100 sec of midnight, rare). -3. **Edge case (NM within minutes of midnight):** Documented and mitigated in code (loop in `solarToLunar`). Test coverage advised but implementation sound. - -### Caveats -- **1800–1900 coverage:** Current implementation correct but not verified against Trần Tiến Bình tables (only covers 1901+). -- **Official calendar deviations:** Vietnamese government might announce lunar date corrections for rare years; no automated inference possible. Implementation cannot detect deliberate political/religious overrides. - ---- - -## Recommendation: Algorithm Choice for Go Implementation - -### Primary Recommendation: RETAIN & ENHANCE -✅ **Use truncated-Meeus (current)** -- Proven, widely deployed, sufficient accuracy -- Computational cost negligible - -🔧 **Enhancements (Priority Order):** -1. **Upgrade ΔT polynomial** to NASA piecewise fit for 1800–2150. (Impact: ±0–2 sec, removes 1850–1900 systematic bias. Effort: ~50 lines.) -2. **Add test vectors** for 1985 (Vietnam vs. China), 2007, and 1877/2062 edge cases. (Effort: ~10 test cases, validate against online converters or Trần Tiến Bình tables for 1901–2100.) -3. **Document limitations:** Note ±100 second uncertainty band for 2150–2200 (ΔT extrapolation only, not lunar physics). -4. **Optional: Extend lower bound to 1600** if users ask (requires different ΔT polynomial, but current UTC+7 interpretation only valid from ~1850 onward—before that, Hanoi used local solar time). - -### Rejected Alternatives -❌ **Do NOT switch to VSOP87/ELP2000-82:** Overkill (1000× complexity for 0.01× improvement in seconds-level precision when calendar needs only hour-level). - -❌ **Do NOT implement Chinese-style high-precision:** Hồ Ngọc Đức's algorithm IS the Vietnamese standard; forking it gains nothing. - ---- - -## Unresolved Questions - -1. **Pre-1901 ground truth:** Trần Tiến Bình's official tables cover 1901–2100 only. No published reference exists for 1800–1900. Are there historical Vietnamese government records (e.g., imperial court calendars) that could serve as test vectors? - -2. **Meeus Ch. 49 vs. Hồ Ngọc Đức coefficients:** Do they differ in the last decimal place? If so, which is more accurate? (Hồ Ngọc Đức may have transcribed with rounding; original ELP-2000/82 is Chapront's work, not Meeus's.) - -3. **Day 0 edge case frequency:** How many dates in 1800–2199 trigger the documented day-0 bug? Can a deterministic formula predict them (to pre-compute, rather than loop)? - -4. **2100–2200 ΔT extrapolation validity:** NASA's +442 sec estimate for 2200 assumes linear tidal braking. If lunar mantle dynamics change, prediction breaks. Should document this risk explicitly? - -5. **Official government calendar overrides:** Does Vietnam ever publish Lunar New Year date corrections for rare edge cases? If so, should implementation have a manual override table? - ---- - -## Sources - -- [Hồ Ngọc Đức Vietnamese Lunar Calendar](https://www.xemamlich.uhm.vn/calrules_en.html) -- [NASA Delta T Polynomial Expressions](https://eclipse.gsfc.nasa.gov/SEcat5/deltatpoly.html) -- [NASA Uncertainty in Delta T](https://eclipse.gsfc.nasa.gov/SEcat5/uncertainty.html) -- [Jean Meeus Astronomical Algorithms 2nd ed.](https://www.amazon.com/Astronomical-Algorithms-Meeus-Jean/dp/0943396611) -- [Trần Tiến Bình: Lịch Việt Nam thế kỷ XX-XXI](https://newshop.vn/lich-viet-nam-the-ki-xx-xxi-1901-2100-va-nien-bieu-lich-su-viet-nam.html) -- [1985 Vietnamese–Chinese Calendar Discrepancy](https://blackpony.org/tet2.pdf) -- [Astro Go Library (VSOP87+ELP2000/82 reference)](https://github.com/Starainrt/astro) -- [Truncated ELP-82 Implementation Guide](https://celestialprogramming.com/meeus-elp82.html) -- [Wikipedia Vietnamese Calendar](https://en.wikipedia.org/wiki/Vietnamese_calendar) -- [amlich.js Reference Implementation](https://github.com/codeaholicguy/amlich.js) diff --git a/plans/reports/researcher-260808-2252-vietnamese-lunisolar-rules-report.md b/plans/reports/researcher-260808-2252-vietnamese-lunisolar-rules-report.md deleted file mode 100644 index 2e9ea4e..0000000 --- a/plans/reports/researcher-260808-2252-vietnamese-lunisolar-rules-report.md +++ /dev/null @@ -1,442 +0,0 @@ -# Vietnamese Lunisolar Calendar Rules: Implementation Specification -**Research Report** | Date: 2026-08-08 | Range: 1800–2199 - ---- - -## EXECUTIVE SUMMARY - -Vietnamese lunisolar calendar (âm lịch) follows the **shíxiàn rule** (Chinese-derived system). Month boundaries anchored to astronomical new moons computed at UTC+7 (Hanoi, ~105°E meridian); month 11 defined as month containing winter solstice (☉ longitude 270°). Leap month is first month without a major solar term (trung khí) between consecutive month-11s in 13-lunation years. Year naming uses 60-year can-chi cycle. Rules uniquely stable since 1645 calendar reform due to apparent solar longitude calculations (with nutation/aberration per Meeus formulas). - ---- - -## 1. MONTH-BUILDING RULES - -### 1.1 Lunar Day/Month Boundaries - -**Rule M1:** The first day of a lunar month is the calendar day (civil day at midnight boundary) in which the astronomical conjunction of the Moon (new moon) occurs, computed at UTC+7. - -**Computation:** -- New moon instant = JD(k) from Meeus truncated series (Jean Meeus, *Astronomical Algorithms*, accurate to ~0.01 days) -- Convert JD to civil day: `day_number = floor(JD + 0.5 + UTC_offset/24)` -- UTC offset for Vietnamese calendar: **+7.0 hours** (see Sec. 3: Timezone Policy) -- Month length: 29 or 30 days (determined by next new moon day minus current) - -**Critical Detail:** "New moon day" = the solar (Gregorian) calendar day number at UTC+7, not the astronomical day. This causes month boundaries to shift when astronomical new moon occurs near UTC+7 midnight. - -### 1.2 Month Numbering & Year Anchor - -**Rule M2:** Month 11 is defined as the lunar month containing the winter solstice (Đông chí), i.e., the month in which sun's apparent ecliptic longitude passes 270° (solar longitude index = 9). - -**Algorithm:** -1. For a given Gregorian year Y, find all new moons ≥ Dec 31, Y−1 -2. For each new moon day D, compute `solar_index(D) = floor(sunLongitude(D) / 30°)` (range 0–11) -3. Month 11 of year Y = the first month whose start day D has `solar_index(D) ≥ 9` AND the month contains the winter solstice - -**Year Transition Rule:** -- If months 11–12 of Gregorian year Y both occur before the next month 1, they belong to **lunar year Y** -- If month 1 of Gregorian year Y+1 occurs after month 12 of year Y, those months 11–12 belong to **lunar year Y−1** -- Lunar year increments at month 1 (Tết), not at month 11 (i.e., months 11–12 of Gregorian year N may belong to lunar year N−1) - -### 1.3 Leap Month (Tháng Nhuận) Determination - -**Rule M3:** In a year containing 13 lunar months between month 11 of year Y and month 11 of year Y+1, the first month lacking a major solar term (trung khí) becomes the leap month. - -**Definition of "Major Solar Term":** A month contains a major solar term if the sun's ecliptic longitude transitions across a multiple of 30° (i.e., `solar_index` changes) at any point during that month. - -**Identification Algorithm:** -1. Count new moons between start of month 11(Y) and start of month 11(Y+1): if ≤12, no leap month -2. If 13 moons, iterate months (m=1, 2, ...) after month 11(Y) -3. For each month m, check if `solar_index(day_start) ≠ solar_index(day_end)` (term transition occurred) -4. First month with no transition = leap month (moniker: month(m) nhuận, e.g., "tháng 6 nhuận") - -**Leap Month Properties:** -- Occurs every 2–3 years (19-year Metonic cycle: 7 leap years in 19 years) -- Can theoretically follow any month 1–10, 11, or 12 (rare: month 11 leap last occurred ~1645) -- In a 13-lunation span, exactly one month lacks a major term (guaranteed by solar year ≈12.37 lunations) - -**Tie-Breaking (Solar Term at Month Boundary):** -- If a major solar term occurs exactly at a month's start or end (rare), the Meeus algorithm precision is ~0.01 days; month boundaries are sharp -- No documented Vietnamese rule for this case; assume apparent geocentric longitude computation resolves it - ---- - -## 2. SOLAR TERM DEFINITIONS - -**Rule S1:** The 12 major solar terms (trung khí, or zhōngqì in Chinese) correspond to sun's apparent ecliptic geocentric longitude at multiples of 30°: - -| Index | Longitude | Name (Vietnamese) | Name (Chinese) | Gregorian Window | -|-------|-----------|-------------------|-----------------|------------------| -| 0 | 0° | Xuân Phân | Chun Fen (Spring Equinox) | Mar 19–22 | -| 1 | 30° | Thanh Minh / Cốc Vũ | Qingming | Apr 4–6 | -| 2 | 60° | Lập Hạ | Lixia (Start of Summer) | May 5–7 | -| 3 | 90° | Hạ Chí | Xiazhi (Summer Solstice) | Jun 20–22 | -| 4 | 120° | Đại Thử | Daxu | Jul 6–8 | -| 5 | 150° | Chính Thu | Zhengxiu | Aug 22–24 | -| 6 | 180° | Thu Phân | Qiu Fen (Autumn Equinox) | Sep 22–24 | -| 7 | 210° | Sương Giáng | Shuangjiang | Oct 8–9 | -| 8 | 240° | Lập Đông | Lidong (Start of Winter) | Nov 6–8 | -| 9 | 270° | Đông Chí (Winter Solstice) | Dongzhi | Dec 21–23 | -| 10 | 300° | Tiểu Tuyết | Xiaoxue | Dec 6–8 | -| 11 | 330° | Đại Tuyết | Daxue | Dec 6–8 | - -**Computation:** `solar_index = floor(sunLongitude(JD) × 180/π / 30) mod 12` - -**Apparent vs. Mean Longitude:** -- Vietnamese/Chinese calendars use **apparent geocentric solar longitude** (Meeus algorithm) -- Includes nutation and aberration corrections (relative precision: ~0.005°, order 10 minutes of arc) -- NOT mean longitude (would shift all term dates ~20–30 minutes, causing occasional month misalignment) -- Validated: Meeus algorithm matches observed solar term dates ±1 day historically - ---- - -## 3. TIMEZONE HISTORY & POLICY RECOMMENDATION - -### 3.1 Historical Timezone Evolution - -| Period | Region | Offset | Justification | -|--------|--------|--------|---------------| -| Pre-1906 | French Indochina | ~UTC+7:06:30 | 105°E meridian (historical estimate) | -| 1906–1945 | French Indochina | UTC+7:06:30 | Formal adoption (106°37'30"E), then UTC+7 | -| 1945–1954 | Vietnam (unified) | UTC+7 | Post-WWII, pre-division | -| 1954–1967 | **North Vietnam** | **UTC+8 → UTC+7 (1967-08-08)** | Aligned with China; official switch 1968-01-01? | -| 1954–1959 | **South Vietnam** | **UTC+7** | Initially Indochina standard | -| 1959–1975 | **South Vietnam** | **UTC+8** (from 1960-01-01) | Alignment with regional allies (Philippines, Malaysia) | -| 1975–present | **Unified Vietnam** | **UTC+7** | Post-unification standard; Vietnam Standard Time (VST) | - -**Critical Impact — Tet Mậu Thân 1968:** -- North (UTC+7): Tết 1968 = Jan 29–30 -- South (UTC+8): Tết 1968 = Jan 30–31 -- New moon at UTC: Jan 29, ~17:38 (JD 2439884.235) -- At UTC+7: Jan 30 (midnight +7:00:00); at UTC+8: Jan 30 (midnight +8:00:00) — **same civil day by coincidence, but different calendar conventions** -- Confusion propagated to Vietcong units on different timezone calculations - -### 3.2 Timezone Policy Recommendation (1800–2199) - -**For Historical Years (1800–1966) & Modern (1967+):** - -**RECOMMENDATION: Use UTC+7 throughout 1800–2199 range.** - -**Rationale:** -1. **Hồ Ngọc Đức's algorithm** (de facto Vietnamese reference) uses UTC+7 -2. **Published Vietnamese almanacs** ("Lịch Việt Nam thế kỷ XX", tables by Ho Ngoc Duc) computed at UTC+7 -3. **Historical justification:** French Indochina meridian (105–106°E) maps to UTC+7 (with modest precision loss vs. UTC+7:06:30, ~10 min) -4. **Operational consistency:** Pre-1967 Vietnamese documents don't specify sub-minute precision; UTC+7 matches expectations -5. **Test vectors:** Ho Ngoc Duc's published tables (1800–2199) derived at UTC+7; converting to other timezones breaks validation - -**Exception for North Vietnam 1954–1967:** -- North Vietnam used UTC+8 until ~1967–1968 -- **No separate conversion algorithm needed:** Use UTC+7 regardless; the divergence (1 hour) is absorbed in the historical record -- If precise 1954–1967 North Vietnamese calendar required, source original Hanoi publications; algorithm remains UTC+7 - -**Modern Standard:** Vietnam officially uses UTC+7 (ICT/Indochina Time) with no DST. All conversions post-1975 trivially use UTC+7. - ---- - -## 4. YEAR NUMBERING & CAN-CHI CYCLE - -### 4.1 Can-Chi Sexagenary Naming - -**Rule Y1:** Lunar year N is named by combining: -- **Heavenly Stem (Thiên Can):** 10-element cycle: Giáp (0), Ất (1), Bính (2), Đinh (3), Mậu (4), Kỷ (5), Canh (6), Tân (7), Nhâm (8), Quý (9) -- **Earthly Branch (Địa Chi):** 12-element animal cycle: Tý (0), Sửu (1), Dần (2), Mão (3), Thìn (4), Tỵ (5), Ngọ (6), Mùi (7), Thân (8), Dậu (9), Tuất (10), Hợi (11) - -**Indexing Formula:** -``` -can_index = (lunar_year + 6) % 10 -chi_index = (lunar_year + 8) % 12 -name = can_names[can_index] + " " + chi_names[chi_index] -``` - -**Example:** Lunar year 2024 → (2024+6)%10=0 (Giáp), (2024+8)%12=4 (Thìn) → **"Giáp Thìn"** - -**Cycle Property:** 60-year period (lcm(10,12)=60); can-chi repeats every 60 years. Year Y has same can-chi as Y±60, Y±120, etc. - -### 4.2 Lunar Year vs. Gregorian Year - -**Rule Y2:** Lunar year N runs from month 1 (Tết) of Gregorian year G to month 12 of Gregorian year G' (typically G' = G+1, sometimes G+2). - -**Why months 11–12 "belong" to previous lunar year:** -- Month 11 = month containing winter solstice of year Y -- Month 12 immediately follows -- Both occur in Gregorian year N (December) -- Lunar year N+1 starts at month 1 of Gregorian year N+1 (January/February) → months 11–12 of Gregorian year N belong to **lunar year N, not N+1** - -**Example (Lunar Year 2026, Thìn → 2027, Tỵ):** -- Month 1 (Tết) 2026: Feb 17, 2026 (Gregorian) -- Month 11 (winter solstice): Dec 21–22, 2026 -- Month 12: Jan 2027 -- Lunar year 2026 ends: Feb 2027 (lunar year 2027 begins) - ---- - -## 5. EDGE CASES & RESOLUTIONS - -### 5.1 13-Lunation Years with Two No-Term Months - -**Case 5.1a: Standard Handling** -- In 13-lunation span, one month lacks a major solar term -- Rare: two months without major terms (would require extreme lunar/solar alignment) -- **Rule:** If two months lack major terms, designate the **first** as leap month - -### 5.1b: The 2033 Problem (Chinese/Vietnamese Shared) - -**Year 2033 (Year of the Rooster, Quý Dậu):** -- 13 new moons between month 11(2032) and month 11(2033) -- Months follow sequence: 11, 12, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, ... -- **Leap month occurs after month 11** (extraordinarily rare; last occurrence ~1645) -- Computation reveals: month after month 11 has no major solar term → leap month 11 (tháng 11 nhuận) - -**Implication for Vietnamese Calendar:** Same issue applies. Historical almanacs (Ho Ngoc Duc) correctly place leap month 11 in 2033. Simplified rules assuming "leap month only in 1–10" **fail for 2033** and ~5 other years in 1800–2199 range. - -**Test Vector:** -``` -Month 11, 2033: Nov 22 – Dec 21 (no major term; index 8→9 boundary at winter solstice Dec 22) -Month 11 nhuận, 2033: Dec 22 – Jan 20 (repeat month 11, part of lunar year 2032 technically, but often counted separately) -Month 12, 2033: Jan 21 – Feb 19 -Lunar year 2033 starts: Feb 20, 2034 (month 1) -``` - -**Handling in Code:** Do NOT special-case; compute leap month for every 13-lunation year via solar-term transition rule. - -### 5.2 Month Length Variation - -**Rule L1:** A lunar month contains either 29 (đủ) or 30 (thiếu) days, determined by the number of calendar days between successive new moon days. - -- 29-day month (thiếu, "lacking"): when next new moon occurs 29 days after current start -- 30-day month (đủ, "full"): when next new moon occurs 30 days after current start -- **No "31-day lunar month" exists** (synodic month ≈29.53 days) - -**Distribution:** In a 19-year Metonic cycle (235 lunations), ~50% are 29-day, ~50% are 30-day. No predictable pattern; must compute for each month. - -### 5.3 Leap Month as Month 12 - -**Possibility:** Can month 12 be a leap month? Yes, but **exceedingly rare** (no occurrence recorded post-1645 in published almanacs). Rule M3 applies: if month 12 is the first no-term month in the span, it is designated leap month 12 (tháng 12 nhuận). - -### 5.4 Officially Decreed Deviations - -**Finding:** No documented case where Vietnam officially overrode astronomical calculations for calendar dates post-1645. France may have imposed administrative calendars (1906–1954), but the lunisolar calendar always followed astronomical rules. - -**Exception:** Vietnam's official Gregorian calendar (since 1954) takes precedence for government/legal purposes; lunisolar calendar used for cultural/holiday purposes only. No conflict in algorithms—two parallel systems. - ---- - -## 6. PSEUDOCODE SPECIFICATION - -### 6.1 Core Functions - -``` -function newMoonJD(k): - // Jean Meeus algorithm (2415020.75933 epoch = JD of Jan 0.5, 1900) - T = k / 1236.85 // Julian centuries from 1900.0 - jd1 = 2415020.75933 + 29.53058868*k + O(T^2, T^3) terms - // Apply Meeus corrections for sun/moon anomalies, f-argument - return jd1 + corrections - -function getNewMoonDay(k): - jd = newMoonJD(k) - return floor(jd + 0.5 + 7/24) // Convert to civil day at UTC+7 - -function sunLongitude(jdn): - // Meeus algorithm: apparent ecliptic longitude - T = (jdn - 2451545) / 36525 // J2000 centuries - M = mean anomaly of sun - L0 = mean longitude - DL = corrections (nutation, aberration, etc.) - lambda = L0 + DL (mod 2π) - return lambda - -function getSolarIndex(dayNumber): - // Compute which 30° sector sun is in at local midnight - jdn = dayNumber - 0.5 - 7/24 // Adjust to UT for sun computation - lambda = sunLongitude(jdn) - return floor(lambda / (π/6)) mod 12 // [0..11] - -function getLunarMonth11(year): - // Find month 11 (winter solstice month) of lunar year 'year' - off = jdFromDate(31, 12, year) - 2415021 - k = floor(off / 29.530588853) - - // Step back to find month containing winter solstice (solar index 9) - nm = getNewMoonDay(k) - while getSolarIndex(nm) >= 9: - k-- - nm = getNewMoonDay(k) - return nm - -function getLeapMonthOffset(month11_start): - // Return offset (1..13) from month 11 to leap month, or 0 if no leap - k = floor((month11_start - 2415021.076998695) / 29.530588853 + 0.5) - lastIndex = getSolarIndex(getNewMoonDay(k + 1)) - - for i = 2 to 13: - currentIndex = getSolarIndex(getNewMoonDay(k + i)) - if currentIndex == lastIndex: - return i - 1 // No transition in month i-1 → leap month - lastIndex = currentIndex - return 0 // No leap month (≤12 lunations) - -function solarToLunar(day, month, year): - // Gregorian to lunar conversion - dayNum = jdFromDate(day, month, year) - k = floor((dayNum - 2415021.076998695) / 29.530588853) - - // Find the new moon on or before dayNum - monthStart = getNewMoonDay(k + 1) - while monthStart > dayNum: - k-- - monthStart = getNewMoonDay(k + 1) - - lunDay = dayNum - monthStart + 1 - - // Determine lunar month/year via month-11 anchoring - a11 = getLunarMonth11(year) - b11 = a11 - - if a11 >= monthStart: - lunYear = year - a11 = getLunarMonth11(year - 1) - else: - lunYear = year + 1 - b11 = getLunarMonth11(year + 1) - - // Month offset from month 11 - monthDiff = (monthStart - a11) / 29 // approx - lunMonth = monthDiff + 11 - isLeap = false - - if b11 - a11 > 365: // 13 lunations → leap year - leapOff = getLeapMonthOffset(a11) - if monthDiff >= leapOff: - lunMonth = monthDiff + 10 - if monthDiff == leapOff: - isLeap = true - - if lunMonth > 12: - lunMonth -= 12 - - // Months 11–12 of Gregorian year N may belong to lunar year N-1 - if lunMonth >= 11 and monthDiff < 4: - lunYear-- - - return (lunDay, lunMonth, lunYear, isLeap) -``` - -### 6.2 Validation Test Vectors - -| Gregorian | → | Lunar | Can-Chi | Notes | -|-----------|---|-------|---------|-------| -| 2024-02-10 | → | 1/1/2024 | Giáp Thìn (Dragon) | Tết 2024 | -| 1968-01-30 | → | 1/1/1968 | Mậu Thân (Monkey) | Tet Offensive; South Vietnam calendar | -| 1968-01-29 | → | 1/1/1968 | Mậu Thân | North Vietnam calendar (UTC+7 vs UTC+8) | -| 2033-11-22 | → | 11 nhuận/2032 | Quý Dậu | **Leap month 11** (edge case) | -| 2033-12-22 | → | 12/2032 | Quý Dậu | After leap month 11 nhuận | -| 2034-02-20 | → | 1/1/2034 | Giáp Tỵ | Lunar year 2034 starts | - ---- - -## 7. SOURCE CITATIONS & CREDIBILITY ASSESSMENT - -### Primary Sources (Authoritative) - -1. **Hồ Ngọc Đức's Algorithm** (http://www.informatik.uni-leipzig.de/~duc/amlich/) - - **Type:** Published computational algorithm + tables - - **Coverage:** Vietnamese calendar 1800–2199 (matches published almanacs) - - **Credibility:** De facto Vietnamese reference; basis for all major converter libraries (Node.js, iOS, Android) - - **Access:** xemamlich.uhm.vn mirrors; original at U. Leipzig - -2. **Jean Meeus, *Astronomical Algorithms*, 2nd ed. (1998)** - - **Type:** Academic textbook, widely adopted for calendar computations - - **Coverage:** New moon times, solar longitude, formulae with ±0.01 day precision - - **Credibility:** Standard reference for astronomical calculations globally (cited by ISO 8601 calendar work, Reingold & Dershowitz's *Calendrical Calculations*) - -3. **Vietnamese Official Sources** - - **Ban Lịch Nhà nước (State Calendar Bureau) / Viện Hàn lâm KHXH (Vietnam Academy of Social Sciences):** Publishes annual lunar calendar (Lịch Vạn Niên) - - **Credibility:** Official government authority; used for public holidays - - **Note:** Decrees on timezone adoption (e.g., 1967-08-08 North Vietnam adoption of UTC+7) not readily accessible in English; cited from timezone database (ICANN tz-announce archive) - -### Secondary & Reference Sources (High-Confidence) - -4. **Helmer Aslaksen, "The Mathematics of the Chinese Calendar"** (*Mathematics Magazine* 2010; accessible at www.math.nus.edu.sg/aslaksen/calendar/) - - **Type:** Peer-reviewed academic article - - **Coverage:** Lunisolar calendar rules, solar terms, leap-month determination, 2033 edge case - - **Credibility:** Cited in academic literature; resolves 2033 "problem" definitively - - **Note:** Chinese calendar rules ≈ Vietnamese (same astronomical basis); diverges only via timezone (Beijing UTC+8 vs. Hanoi UTC+7) - -5. **ytliu0's "Rules for the Chinese Calendar"** (https://ytliu0.github.io/ChineseCalendar/rules.html) - - **Type:** Educational synthesis with computational examples - - **Coverage:** Month structure, solar terms, leap-month algorithm, 2033 examples - - **Credibility:** High-quality exposition verified against published sources; used by educators - - **Cross-check:** Matches Aslaksen and Meeus formulations - -### Secondary Sources (Medium-Confidence) - -6. **Wikipedia entries:** Vietnamese calendar, Chinese calendar, Solar term, Lunisolar calendar - - **Type:** Crowdsourced encyclopedia - - **Credibility:** Generally accurate for rule statements; useful for high-level overview; not authoritative for exact algorithms - - **Use:** Validation cross-reference only - -7. **TimeZone Database (ICANN tz project)** — Vietnam timezone history - - **Type:** Authoritative system database - - **Coverage:** UTC offset transitions (1906–present) - - **Note:** Does not cite Vietnamese government decrees explicitly; inferred from historical records - -### Limitations & Gaps - -- **Pre-1906 Vietnamese calendar computation:** No accessible historical source specifies the exact meridian or offset used. The 105–106°E estimate is inferred from French Indochina records. -- **North Vietnam 1954–1967 decree text:** The 1967-08-08 UTC+7 adoption (or 1968-01-01 effective date) lacks primary documentation in English. Inferred from timezone databases and the Tet 1968 divergence evidence. -- **"Fake leap months" (months without solar terms but not leap-designated):** Aslaksen identifies this as a rare modern phenomenon (post-1645); Vietnamese impact unconfirmed. No documented Vietnamese case found; algorithm robustness assumed sufficient. -- **Official Vietnamese government decree numbers (Quyết định / Nghị định):** Historical legislative citations for calendar rules (if any exist) are not available in English sources consulted. - ---- - -## 8. UNRESOLVED QUESTIONS - -1. **Pre-1906 Meridian Precision:** What exact longitude/offset did Vietnamese astronomers use for calendar computations during the Chinese-rule era (pre-1858) and French colonial period (1858–1906)? Was it 105°E precisely, or a different standard? - -2. **North Vietnam 1967–1968 Transition:** What was the exact effective date of North Vietnam's UTC+7 adoption—1967-08-08 or 1968-01-01? Did calendar conversions for Tet 1968 officially recognize the UTC+8→UTC+7 shift, or was it treated as a discontinuity? - -3. **"Ban Lịch Nhà nước" Official Algorithm:** Does Vietnam's State Calendar Bureau publish its own algorithm (potentially diverging from Hồ Ngọc Đức in minor ways) for post-1975 lunar dates? If so, does it match Hồ Ngọc Đức's UTC+7 specification? - -4. **Fake Leap Months in Vietnamese History:** Aslaksen (2010) identifies that in certain 13-lunation years, two months may lack major solar terms under modern calculations. Has this occurred in Vietnamese calendar history (1800–2199), or is the shíxiàn rule perfectly stable? - -5. **Leap Month 11/12 Precedent:** Beyond 2033, are there documented cases in Vietnamese lunar calendar history (published almanacs) where month 12 or month 11 (other than 2033) is designated as the leap month? If so, what are the years? - -6. **Apparent vs. Mean Longitude Confirmation:** Does any Vietnamese authoritative source (Ban Lịch Nhà nước, published almanac) explicitly state that apparent (vs. mean) geocentric solar longitude is used? Hồ Ngọc Đức's algorithm uses apparent; confirmation from official Vietnamese source would be valuable. - ---- - -## 9. RECOMMENDATIONS FOR IMPLEMENTATION - -1. **Adopt UTC+7 throughout 1800–2199** for all computations. It matches published references and is historically plausible for pre-1967 dates. - -2. **Use Meeus algorithm as-is** for new moon and solar longitude. Precision ~0.01 days (new moon) and ~0.01° (solar terms) is sufficient for 300-year span. - -3. **Compute leap month via solar-term transition rule** (no special-casing except for edge case 2033 validation): iterate through consecutive months, identify first without a major-term transition, mark as leap. - -4. **Handle 2033 explicitly as test case:** Month 11 nhuận in 2033 is not a bug; it is the correct output of the shíxiàn rule. Almanac references confirm it. - -5. **Maintain month-11 anchoring invariant:** Verify that computed month 11 always contains the winter solstice (solar index 9 at month's start day or within the month). If violated, algorithm has a bug. - -6. **Validate against Ho Ngoc Duc's tables** (1800–2199) for random years across the range. Mismatches in lunar month number or leap designation indicate algorithmic error. - ---- - -## 10. SUMMARY TABLE: Rules at a Glance - -| Rule Category | Key Formula/Condition | -|---|---| -| **Month Start** | Civil day containing astronomical new moon at UTC+7 | -| **Month 11 Anchor** | Month containing winter solstice (☉ longitude 270°) | -| **Leap Month** | First month (in 13-lunation year) without major solar-term transition | -| **Solar Index** | `floor(☉ longitude / 30°) mod 12` → term 0–11 | -| **Can-Chi Year** | `can = (year+6)%10; chi = (year+8)%12` → name string | -| **Year Increment** | At month 1 (Tết), not month 11 | -| **Months 11–12 Year Belonging** | Belong to the lunar year whose month 11 precedes them | -| **Timezone** | UTC+7 (Hanoi meridian ~105°E) for all 1800–2199 | -| **New Moon Epoch** | JD 2415021.076998695 (Jan 1900.0) | -| **Synodic Month** | 29.530588853 days (mean; actual 29–30) | - ---- - -**Report Status:** Complete. All six research questions addressed. Test vectors and pseudocode provided for implementation. -