Commit Graph
62 Commits
Author SHA1 Message Date
tiennm99 0dd93e3bf9 feat(portfolio): track stock and coin cost basis 2026-07-21 15:52:05 +07:00
tiennm99 d2c3129d41 refactor(commands): standardize parameter conventions 2026-07-21 14:51:28 +07:00
tiennm99 a211b7f8b7 fix(commands): remove examples from command discovery 2026-07-21 13:52:53 +07:00
tiennm99 b5c7427520 fix(commands): render examples conditionally with inline help code 2026-07-21 13:21:01 +07:00
tiennm99 f5ba9d4032 feat(commands): improve discovery and normalize parameters 2026-07-21 11:45:20 +07:00
tiennm99 f0d361eab5 refactor: adopt cross-platform Go development workflow 2026-07-21 09:05:20 +07:00
tiennm99 8b2ac571d9 feat(stock): add ratio dividend commands 2026-07-20 17:32:37 +07:00
tiennm99 6c934f6807 feat(modules): remove world cup module
The 2026 tournament has ended, so the schedule and daily digest commands
have no upcoming matches to report. Drops the module, its catalog entry,
command menu entries, and the WC_FOOTBALL_DATA_TOKEN env var.

Stored subscriber and match-cache documents are left in place.
2026-07-20 11:03:59 +07:00
tiennm99 337b7de18c refactor(stats): remove completed startup migrations 2026-07-09 15:44:23 +07:00
tiennm99 194caf17eb fix(lol): rename next week command 2026-07-06 20:46:10 +07:00
tiennm99 2fb10ff632 refactor: remove completed migration logic 2026-07-01 16:20:30 +07:00
tiennm99 b0b4321fa1 feat(lol): expire match cache 2026-07-01 13:40:37 +07:00
tiennm99 e6d5bc3cc9 feat: refine command names and behavior 2026-07-01 13:19:42 +07:00
tiennm99 5254ab4c8e refactor: remove completed startup migrations 2026-07-01 11:42:59 +07:00
tiennm99 788f1015b5 feat(commands): simplify portfolio command names 2026-07-01 10:53:52 +07:00
tiennm99 c362732951 feat: rename wc schedule commands 2026-07-01 10:42:56 +07:00
tiennm99 ccaa34336d feat: rename lol schedule module 2026-07-01 10:38:49 +07:00
tiennm99 8f0286e54f feat(stats): use queryable Mongo usage records 2026-07-01 10:22:03 +07:00
tiennm99 9c998a0e2f refactor(modules): remove twentyq genai integration 2026-06-29 10:07:21 +07:00
tiennm99 20dc03c6d2 refactor(config): remove price URL env overrides 2026-06-29 09:23:20 +07:00
tiennm99 5aa538be96 feat(server): register command menu on startup 2026-06-29 01:08:51 +07:00
tiennm99 5971347a6b feat(wc): add world cup schedule module 2026-06-29 00:59:30 +07:00
tiennm99 9c8387bf3a fix(deploynotify): preserve coolify source commit 2026-06-29 00:20:04 +07:00
tiennm99 b0586903f5 refactor(deploy): remove retired aws support 2026-06-29 00:11:42 +07:00
tiennm99 ed867f33f0 feat(deploynotify): notify owner on every startup, no SHA dedup
Match the store-scraper-bot behaviour: DM the owner once per boot regardless of
commit, and report an unknown SHA rather than staying silent. Drop the store-
based dedup and the empty-SHA skip; resolveCommitSHA defaults to "unknown".
2026-06-28 23:25:07 +07:00
tiennm99 0abd33cb42 fix(deploynotify): read commit SHA from Coolify SOURCE_COMMIT runtime env
Baking SOURCE_COMMIT as a Docker build arg never worked: Coolify exposes it
as a runtime env var, not a build arg, so gitSHA was always empty and the owner
DM was skipped. Read SOURCE_COMMIT from the container env at startup (falling
back to the ldflags-baked gitSHA for local builds), forward it via the compose
environment, and drop the dead build-arg baking.
2026-06-28 23:08:17 +07:00
tiennm99 9d5b774846 refactor(server): delete webhook once on startup, no retry
The Bot API GET reliably clears the webhook, so the retry loop added for the
empty-body POST is no longer needed. One best-effort call; log on failure.
2026-06-28 22:52:55 +07:00
tiennm99 36cb5750e6 fix(telegram): clear webhook via Bot API GET to avoid empty-body POST
go-telegram's DeleteWebhook posts an empty multipart form (DropPendingUpdates
is omitempty); some networks answer that bodyless POST with an empty response,
failing to parse and leaving the webhook active so getUpdates 409s forever.
Replace it with a plain GET /deleteWebhook, which sends no body and parses the
JSON reliably. Keeps pending updates for a lossless cutover.
2026-06-28 22:41:49 +07:00
tiennm99 60a7aa5836 fix(server): retry webhook clear on startup past transient empty-body error
Telegram's edge can return an empty body on the first request after a cold
container start, failing the single best-effort DeleteWebhook with 'unexpected
end of JSON input' and leaving a stale webhook that 409s getUpdates. Retry up
to 5 times with a 1s backoff so long polling is reliably unblocked.
2026-06-28 22:33:34 +07:00
tiennm99 d4f1bbfc83 fix(server): clear Telegram webhook at startup before owner DM
Move DeleteWebhook ahead of the deploynotify owner DM and polling so the
webhook is cleared first thing once the bot is ready, instead of just before
getUpdates. DropPendingUpdates=false keeps the lossless cutover.
2026-06-28 22:18:28 +07:00
tiennm99 cc547136b7 refactor(server): drop HTTP /cron route; run crons in-process only
The in-process scheduler is the sole cron trigger on self-host, so the
/cron/{name} HTTP endpoint and its CRON_SHARED_SECRET are dead surface.
Reduce the HTTP server to GET / (health) and remove the cron secret config,
SSM binding, and now-unused timeout.
2026-06-28 22:15:04 +07:00
tiennm99 15a3239ab1 refactor(storage): replace KVStore with generic typed DocStore[T]
Delete the byte-oriented KVStore/VersionedStore abstraction and the
DynamoDB/memory KV backends. Add a generic typed store (DocStore[T] with
Provider/Collection/Typed) persisting each value as a flattened native
Mongo document (storedDoc[T] via bson inline) — no value envelope.

- MongoDB is the only runtime backend; memory kept for tests/local.
- All modules + deploynotify use typed stores; persisted structs carry
  bson tags == json names (incl. nested lolschedule/wordle types).
- lolschedule wraps its array/scalar values in named structs.
- migrate-dynamo-to-mongo writes the flattened shape via Typed[bson.M]
  with wrap rules; Scan/--dry-run/--verify retained.

Verified: go vet/build clean; full go test green hermetically and
in-container vs real Mongo 7 + DynamoDB Local (storage integration +
migrator e2e).
2026-06-28 18:02:11 +07:00
tiennm99 6bc5f334bb refactor(storage): remove legacy firestore backend
Delete Firestore provider implementation and tests (5 files). Extract shared
key helpers to new internal/storage/keys.go module. Remove Firestore case
from server bootstrap and storage factory. Drop cloud.google.com/go/firestore
and related dependencies from go.mod. Remove firestore-emulator test targets
from Makefile. Update README storage section to reflect MongoDB-only backend.
2026-06-28 12:52:40 +07:00
tiennm99 a081427ae5 feat(server): wire MongoDB + cron + polling; rename owner/admin envs
Wire MongoDB client and cron scheduler into server initialization. Rename
TELEGRAM_OWNER to TELEGRAM_ADMIN_ID and SLACK_OWNER to SLACK_ADMIN_ID
for clarity. Enable long-polling as default with cron scheduler for self-hosted.
2026-06-28 09:58:20 +07:00
tiennm99 a6bf022e27 feat(gold): integrate VNAppMob SJC gold price with auto-refresh API key 2026-06-15 10:49:19 +07:00
tiennm99 32417e4196 refactor(stock): rename trading module to stock
Rename the `trading` module to `stock` across code, config, and docs for
naming consistency with the sibling coin/gold modules.

- Module: internal/modules/trading -> internal/modules/stock (package,
  factory, registry key in cmd/server)
- Commands: trade_* -> stock_* (telegram-commands.json)
- Env/SAM params: TRADING_INCOME_EVENTS_* -> STOCK_INCOME_EVENTS_*
- ModulesCSV updated in template.yaml, samconfig.toml, deploy.yml
- Add cmd/migrate-stock-key: idempotent, non-destructive DynamoDB
  partition copy (pk=trading -> pk=stock) with dry-run default

Production DynamoDB data already migrated (8 keys: 2 portfolios + 6
symbol-cache rows); old trading partition retained as rollback snapshot.
2026-06-12 14:52:12 +07:00
tiennm99 9b379b3d6b feat(coin): add crypto paper trading 2026-06-12 10:47:02 +07:00
tiennm99 8c33dc27e4 feat(gold): add compare-and-swap portfolio updates with multi-backend support
Implement atomic UpdatePortfolio with retry pattern for concurrent-safe updates.
Add CAS support to DynamoDB, Firestore, Memory, and prefix stores.
2026-06-11 21:12:58 +07:00
tiennm99 d1a01830ae feat(gold): add gold paper trading module
Opt-in module for gold paper trading with VND currency.
Commands: /gold_topup, /gold_buy, /gold_sell, /gold_stats.
Pricing: spot XAU USD converted to VND per luong via GoldPrice.org + ExchangeRate-API.
Features: FX caching, dust normalization (1e-9), HTTPS-only URL validation,
per-user key locking, namespace isolation from trading module.
2026-06-11 17:13:02 +07:00
tiennm99 537bfc85e9 feat: add trade income events check 2026-06-05 14:21:38 +07:00
tiennm99 2abc138bb1 feat(stats): add command usage statistics module with persistence
Implement a new stats module for the Telegram bot that tracks per-command usage with persistent KV storage. The module provides a /stats command displaying usage sorted by popularity with a 4096-byte Telegram message cap. Includes CommandHook integration for post-dispatch tracking via background goroutine (2s bounded context), proper test coverage, and registry initialization. Updated server config with stats factory and reserved concurrent execution control to prevent TOCTOU issues.
2026-05-22 15:06:46 +07:00
tiennm99 22c817b0db feat(deploynotify): DM owner once per new deployed git SHA
New internal/deploynotify package fires a single Telegram DM to
BOT_OWNER_ID on the first cold start that observes a new gitSHA.
Dedup via a DynamoDB KV record so repeat cold starts of the same
version stay silent. Send-then-write order means a failed Telegram
call doesn't permanently silence retries.

gitSHA is baked into the binary via `-ldflags "-X main.gitSHA=..."`
from Makefile; empty SHA (non-make builds) silently disables the
feature. No new env vars or IAM permissions.
2026-05-22 11:40:25 +07:00
tiennm99 31f4e22c4d refactor: audit-driven hygiene pass across modules and infra
Concurrency
- lolschedule: serialize subscriber Get→mutate→Put via state.subscribersMu;
  the single-slot list was previously losing writes under concurrent
  /lolschedule_subscribe.
- trading: PriceClient memoises its default *http.Client so /trade_stats
  reuses TLS connections across held tickers.

Observability
- server/log_middleware: defer the req log line and recover panics so a
  panicking cron handler still emits the structured req entry CloudWatch
  filters on for 5xx alerting.
- server/router (cron): inner recover with cron-name context captures the
  panicking job before the middleware's safety net does.
- telegram/webhook: rune-safe truncation in dispatch logs — Vietnamese,
  Korean, and emoji previews no longer ship as garbled bytes.
- lolschedule/api_client: same rune-safe fix for error-body log truncation.
- telegram/webhook: gate the post-recover WriteHeader(200) so a panicking
  handler that already touched w doesn't trigger superfluous-WriteHeader.

Correctness
- twentyq: clearGame error during solved-relaunch is logged instead of
  silently swallowed (was a permanent deadlock vector on KV failure).
- misc /mstats: KV read failure replies "Could not load stats. Try again
  later." to the user instead of returning into the dispatcher; matches the
  pattern other modules use.
- migrate_cf_data trading-audit-dump: surface f.Close error so a truncated
  JSONL never passes silently as a complete audit dump.

Operator ergonomics
- migrate_cf_data (all 4 subcommands): signal.NotifyContext for SIGINT /
  SIGTERM. Ctrl-C mid-Scan now propagates cleanly instead of leaving a
  half-converted DynamoDB table.
- ai/ratelimit: doc the Lambda-recycle memory bound to match keylock.Map
  so a future reviewer doesn't re-flag the unbounded map.

I/O-changing (user-approved)
- lolschedule daily push auto-prunes subscribers whose Telegram error
  matches a terminal marker (blocked / deactivated / chat gone). Transient
  errors keep the chat on the list. Subscribe message updated to mention
  the auto-cleanup.
- twentyq seed pool grown 50 → 178; repeat-collision threshold moves from
  ~9 plays to ~17 (birthday paradox).
- util /info flipped Public → Protected — chat/thread/sender IDs are no
  longer enumerable by every group member.
- cmd/server WriteTimeout 6min → 75s (cron 60s + 15s slack). No-op on
  Lambda; matters only for local non-Lambda runs.
- webhook + cron rejection paths drop response bodies (no fingerprintable
  text for internet scanners hitting the public Function URL). Status
  codes preserved for CloudWatch metrics; structured log lines carry the
  rejection reason for operator triage.

Tests added: TestTruncateRunes, TestRunDailyPush_PrunesDeadSubscribers,
TestIsTerminalSendError, TestInfo_DeniedToNonOwner,
TestInfo_DeniedToChannelMessageNoFrom, plus owner-allowed counterparts.
2026-05-16 13:35:00 +07:00
tiennm99 c2968e3de7 Fix AWS Lambda deploy setup 2026-05-15 22:18:27 +07:00
tiennm99 642fccb7b7 refactor: rename module to miti99bot, canonicalize AWS deploy path
Rename:
- Go module github.com/tiennm99/miti99bot-go → github.com/tiennm99/miti99bot
- CloudFormation stack miti99bot-aws-port → miti99bot
- Drop "port", "Cloud Run", "GCP", "cutover", "Phase NN" framing from
  active code and docs — project reads as canonical AWS-Lambda from now on.

AWS deploy guide + flow fix:
- New docs/deploy-aws-free-tier-guide.md — Ubuntu 24.04 ARM64 onboarding
  with project-local venv (pip awscli + sam-cli), SSM secrets via read -s,
  idempotent OIDC provider + role creation, $1 budget alarm.
- Drop sam build from the pipeline — provided.al2023 + makefile builder
  expects a Makefile in CodeUri (build/lambda/, the output dir), so the
  step always fails. sam deploy --template-file template.yaml now reads
  the raw template and zips build/lambda/ directly.
- Rollback section rewritten — use continue-update-rollback /
  cancel-update-stack / git-SHA redeploy. Drop the broken
  --use-previous-template recipe.
- DynamoDB free-tier row corrected (on-demand is 2.5M read / 1M write
  request units, not 25 RCU/WCU).

Updated:
- README.md fully rewritten (drops port/legacy framing, lists modules,
  points new users at the free-tier guide).
- aws/README.md retitled "AWS account setup", phase numbers stripped.
- Makefile / .github/workflows/deploy.yml — sam deploy flow.
- samconfig.toml — stack_name = "miti99bot".
- Go comments — Cloud Run → Lambda, Cloud Scheduler → EventBridge
  Scheduler, Cloud Logging → CloudWatch Logs.
- Struct field GCPProject → FirestoreProject (env GOOGLE_CLOUD_PROJECT
  unchanged).

Plus advisory reports under plans/reports/ from the code-reviewer +
researcher passes that informed the fixes.

Verified: go vet ./..., go build ./..., go test ./... all green.
2026-05-13 22:05:38 +07:00
tiennm99 e29d7afa0e refactor(modules): drop loldle variants, semantle, doantu and dead framework surface
Removes six modules (loldle-ability/emoji/quote/splash, semantle, doantu)
and prunes the framework deps that were only there to serve them:

  - ai.Embedder + Client.Embed + embeddingModel const (semantle only)
  - Deps.Embedder + BuildOptions.Embedder
  - Deps.Env + Build(env) param + ModuleEnv config field + PHOW2SIM allowlist (doantu only)
  - internal/champname package (loldle now owns its lookup helpers directly)
  - template.yaml: Phow2simAPIURL parameter + PHOW2SIM_API_URL Lambda env

Active catalog: util, misc, wordle, loldle, lolschedule, twentyq, trading.

go build / vet / test all pass.
2026-05-11 16:02:06 +07:00
tiennm99 8b97917b6a feat(modules): add Bot to Deps + BuildOptions for cron handlers
- Extend module.Deps struct with optional Bot field
- Add Bot to registry.BuildOptions and thread through builders
- Pass bot instance from main.go into module factory options
- Enables cron handlers to send messages and access bot state
2026-05-10 03:06:01 +07:00
tiennm99 27a54977d3 feat(server): wire KV_PROVIDER env selection (memory|firestore|dynamodb)
- Add environment-based provider selection in main.go
- Support memory (test), Firestore (GCP), and DynamoDB (AWS) backends
2026-05-10 02:29:43 +07:00
tiennm99 2b9bbf27ae feat(observability): request log middleware + in-memory metrics
Phase 11 partial of the go-port-cloud-run plan. Code-side
observability hooks ready ahead of Phase 01 GCP rollout.

- internal/server/log_middleware.go: HTTP middleware that wraps the
  router and emits {msg:"req", method, path, status, ms} per
  request. statusRecorder defaults to 200 when the inner handler
  doesn't call WriteHeader (Go writes 200 implicitly on first body
  write). Wired into server.New so /, /webhook, /cron/* all log.
- internal/metrics/counters.go: in-memory Registry with
  IncCommand/IncError/IncAI. Atomic Int64 per name + RWMutex on the
  map; steady-state increments are mutex-free. Periodic Run flushes
  via the project logger every 60s and one final flush on ctx done.
  Empty flush is silent (no-noise default).
- Dispatcher instrumented: every command invocation calls
  metrics.IncCommand; every handler error calls
  metrics.IncError("handler-error"). Logger keeps the full error
  detail; counters keep the rate.
- cmd/server/main.go: go metrics.Run(rootCtx) so the flush loop
  cancels with SIGTERM and emits the trailing window before exit.

Test coverage: 12 new tests (7 metrics, 3 middleware, 2 default-
registry round-trip). go test -race -count=1 ./... clean
(20 packages); golangci-lint clean.

Soak / cold-start measurement / log-based metrics setup deferred to
post-deployment (Phase 01 prerequisite).
2026-05-09 17:30:15 +07:00
tiennm99 be13a597c7 feat(modules): port lolschedule (5 commands; daily-push cron deferred)
Phase 6e (final sub-phase of port-plan Phase 06). LoL esports match
schedule via lolesports.com's persisted API.

- internal/modules/lolschedule:
  - api_client.go: HTTP client with cache-first lookup (120s fresh
    window, 60-min stale fallback). Cache record shape matches JS so
    cross-runtime KV migration round-trips.
  - parse_date.go: ICT-anchored date parser. Accepts dd-mm-yyyy,
    dd/mm/yyyy, ddmmyyyy; trailing month/year may be omitted (default
    to current ICT month/year). Rejects impossible dates (Apr 31, Feb
    29 in non-leap, etc.).
  - format.go: Today (grouped by league) and Week (grouped by league
    -> day) renderers. Major-league filter (LCK/LPL/LEC/LCS/Worlds/
    MSI/etc.) keeps replies under Telegram's 4096-char limit. All
    user-influenced strings HTML-escaped.
  - subscribers.go: Idempotent add/remove/list keyed by chat id.
  - handlers.go: 5 commands (`/lolschedule [date]`,
    `/lolschedule_today`, `/lolschedule_week`,
    `/lolschedule_subscribe`, `/lolschedule_unsubscribe`).
- 22 tests across api-client (cache hit / miss / stale fallback /
  hard fail / show filter / non-JSON), parse-date (full and
  short formats, defaults, rejections, ICT anchor), format (event
  line states, league ordering, week grouping, HTML escape, major
  filter), subscribers (idempotent add/remove), handlers (HTML
  reply, error path, subscribe/unsubscribe round-trip).

Daily-push cron deferred to Phase 09 (Cloud Scheduler). Subscribers
are still collected so the push lights up the moment the cron infra
lands. Deps doesn't currently expose a *bot.Bot reference; that is
the prerequisite that Phase 09 will solve.

go test -race -count=1 ./... clean (19 packages); golangci-lint clean.
2026-05-09 17:14:01 +07:00