Commit Graph
244 Commits
Author SHA1 Message Date
tiennm99 4bfb961423 docs(selfhost): long polling, minimal env surface, AWS decommission scope
- Telegram transport: long polling only (delete webhook code path)
- Mongo auto-selected from MONGO_URL; in-process cron always-on
  (drop KV_PROVIDER, CRON_MODE, PORT from required env)
- Drop stock/coin/gold API URL overrides (use coded providers);
  remove stock income-events feature + FireAnt token
- Gold VNAppMob key auto-fetched/cached in DB (no env)
- Rename OWNER_ID, ADMIN_IDS; add Phase 5 AWS full decommission
- Final env: TELEGRAM_BOT_TOKEN, MONGO_URL, MONGO_DATABASE, MODULES,
  OWNER_ID, ADMIN_IDS (+ optional GEMINI_API_KEY)
- Red-team (15 findings) + validation (5 sessions) logged
2026-06-27 21:35:12 +07:00
tiennm99 b1171e7cbe docs(selfhost): plan Coolify + MongoDB Atlas self-host with AWS decommission
Add 5-phase plan to self-host on Coolify (docker-compose) with a MongoDB
Atlas backend, an in-process cron scheduler, DynamoDB->Atlas migration,
and full AWS teardown. Red-teamed and validated; decommission scope
verified against live AWS/Cloudflare accounts. Include free-tier audit
and S3-elimination research reports. Ignore wrangler local cache.
2026-06-27 20:46:43 +07:00
tiennm99 c107489f89 refactor(stock): drop symbol cache 2026-06-25 17:30:57 +07:00
tiennm99 8343146c28 docs(stock): add price provider research 2026-06-25 17:17:08 +07:00
tiennm99 412d25b19e fix(stock): drop yahoo price fallback 2026-06-25 17:16:54 +07:00
tiennm99 006ad2b676 fix(stock): expose ssi quote response errors 2026-06-25 16:12:12 +07:00
tiennm99 fca5b8961f docs(stock): record ssi provider research 2026-06-25 15:52:43 +07:00
tiennm99 78e51157d7 fix(stock): use ssi price provider 2026-06-25 15:51:33 +07:00
tiennm99 c0cfcfb8d5 fix(modules): fetch stats prices sequentially to reuse pooled connections
Parallelizing the per-symbol fetches opened N simultaneous TLS handshakes into
an empty connection pool. On the memory-constrained Lambda (256MB ~0.15 vCPU)
those CPU-bound handshakes thrashed and each exceeded the per-fetch timeout, so
every ticker rendered "(no price)". Sequential fetches reuse the price client's
keep-alive connection (one handshake), which is why the code was sequential by
design.

- revert stock and coin stats loops to sequential (keep the reply-budget
  sub-context and 3s per-fetch timeout)
- log per-symbol fetch failures instead of silently swallowing them
2026-06-25 14:28:25 +07:00
tiennm99 958da76d33 fix(modules): reserve reply budget and cap price-fetch timeouts in stats handlers
Stats handlers reused the single bounded update context for both upstream
price fetches and the final Telegram reply, while per-upstream HTTP timeouts
equalled the whole-handler budget. One slow upstream drained the deadline and
the reply failed with "context deadline exceeded" (observed on /stock_stats).

- add chathelper.FetchContext: fetches run under a child context that reserves
  a tail of the deadline for the reply, which is sent on the original context
- cap kbs/coin/gold/vnappmob HTTP timeouts at 3s so one hung upstream cannot
  consume the reply budget
- fetch held stock/coin prices concurrently (bounded) so total latency tracks
  the slowest single fetch instead of their sum; per-symbol failures degrade to
  a "no price" line and the summary still sends
2026-06-25 14:07:59 +07:00
tiennm99 eb8c2e0064 docs(plan): add coin sell message plan 2026-06-20 09:52:35 +07:00
tiennm99 6aeca59b98 fix(coin): improve sell insufficient message 2026-06-20 09:52:23 +07:00
tiennm99 71f3336387 fix(coin): sell command accepts USD amount with dust and zero-price guards
- Change /coin_sell semantics from quantity to USD amount

- Add price-validity and dust-quantity guards to buy/sell

- Improve usage text and error messages

- Add tests for invalid amount, dust, zero price, and insufficient holdings
2026-06-16 17:27:39 +07:00
tiennm99 224a27cfd4 fix(coin): use Binance public market-data endpoint data-api.binance.vision 2026-06-15 14:09:11 +07:00
tiennm99 fd2e84f8fd docs(plans): add VNAppMob key parsing bug-fix plan 2026-06-15 12:20:44 +07:00
tiennm99 b2e8385706 fix(gold): parse VNAppMob JWT object wrapper 2026-06-15 12:20:40 +07:00
tiennm99 e0b0418173 fix(gold): use GET for VNAppMob key refresh endpoint
VNAppMob /api/request_api_key returns 405 for POST and accepts GET.
2026-06-15 11:17:50 +07:00
tiennm99 12dc515047 feat(gold): remove XAU/USD fallback and use VNAppMob SJC only 2026-06-15 11:04:06 +07:00
tiennm99 966e4afd51 chore(plans): add VNAppMob integration plan and completion report 2026-06-15 10:49:19 +07:00
tiennm99 ae92539043 docs(gold): document VNAppMob SJC pricing and env vars 2026-06-15 10:49:19 +07:00
tiennm99 a6bf022e27 feat(gold): integrate VNAppMob SJC gold price with auto-refresh API key 2026-06-15 10:49:19 +07:00
tiennm99 8c2c9a7c84 chore: remove one-time stock-key migration tool
Migration complete and verified in production; the trading->stock
DynamoDB partition copy is no longer needed and the old trading rows
have been deleted.
2026-06-12 15:46:03 +07:00
tiennm99 6d0a30c4c0 fix(migrate): check fmt.Fprintf return to satisfy errcheck lint 2026-06-12 15:12:07 +07:00
tiennm99 32417e4196 refactor(stock): rename trading module to stock
Rename the `trading` module to `stock` across code, config, and docs for
naming consistency with the sibling coin/gold modules.

- Module: internal/modules/trading -> internal/modules/stock (package,
  factory, registry key in cmd/server)
- Commands: trade_* -> stock_* (telegram-commands.json)
- Env/SAM params: TRADING_INCOME_EVENTS_* -> STOCK_INCOME_EVENTS_*
- ModulesCSV updated in template.yaml, samconfig.toml, deploy.yml
- Add cmd/migrate-stock-key: idempotent, non-destructive DynamoDB
  partition copy (pk=trading -> pk=stock) with dry-run default

Production DynamoDB data already migrated (8 keys: 2 portfolios + 6
symbol-cache rows); old trading partition retained as rollback snapshot.
2026-06-12 14:52:12 +07:00
tiennm99 62555dee23 fix(deploy): include coin module in ModulesCSV overrides
coin was registered in the catalog and template default but absent from
the ModulesCSV override in both deploy.yml and samconfig.toml, so the
deployed Lambda never loaded it and all coin_* commands went unregistered.

Add a registration checklist to docs/deploy-aws.md to prevent recurrence.
2026-06-12 11:22:16 +07:00
tiennm99 49a6e7091a fix(coin): accept symbol-first trades 2026-06-12 11:07:27 +07:00
tiennm99 b234b54b2e feat(coin): register telegram commands 2026-06-12 10:53:19 +07:00
tiennm99 9b379b3d6b feat(coin): add crypto paper trading 2026-06-12 10:47:02 +07:00
tiennm99 c9809e716a feat(gold): add gold commands to Telegram menu 2026-06-11 22:28:24 +07:00
tiennm99 8ba575ab1d fix(deploy): include gold module in CI ModulesCSV override
CI's --parameter-overrides replaces samconfig.toml values, so every
push reverted the stack to the pre-gold module list.
2026-06-11 22:21:19 +07:00
tiennm99 98947f7bee feat(gold): add multi-provider gold price fallback chain
Replace goldprice.org (403s datacenter IPs) with a provider chain:
gold-api.com -> Swissquote public quotes -> NBP daily fixing (PLN/gram
converted via the shared FX table). Per-provider failures are logged;
a full-chain outage surfaces as a retryable error instead of the
silent no-price reply.
2026-06-11 22:11:30 +07:00
tiennm99 fdbf240b0e fix(deploy): enable gold module in ModulesCSV 2026-06-11 21:57:05 +07:00
tiennm99 645fa4f1ab fix(gold): drop redundant embedded selectors in portfolio tests 2026-06-11 21:49:03 +07:00
tiennm99 63d257878f fix(storage): report missing CAS support as unsupported instead of conflict
- prefix.go: return errors.ErrUnsupported when inner store lacks CompareAndSwap, enabling fail-fast instead of infinite retries
- Adds comprehensive CAS semantics test coverage for all KV backends:
  * memory_kv_test.go: new tests for basic operations and CAS failures
  * prefix_test.go: tests for wrapped CAS errors and unsupported operations
  * dynamodb_kv_test.go, firestore_kv_test.go: CAS failure scenarios
- portfolio_test.go: test retry exhaustion, business-error short-circuit, concurrent updates, and fail-fast on unsupported CAS
2026-06-11 21:40:16 +07:00
tiennm99 8c33dc27e4 feat(gold): add compare-and-swap portfolio updates with multi-backend support
Implement atomic UpdatePortfolio with retry pattern for concurrent-safe updates.
Add CAS support to DynamoDB, Firestore, Memory, and prefix stores.
2026-06-11 21:12:58 +07:00
tiennm99 2501deb4f8 feat(gold): add /gold_price command showing spot price in USD and VND
Read-only price lookup. Displays XAU USD/oz, USD/VND rate, and VND/luong.
No arguments, no portfolio mutation, no keylock needed.
2026-06-11 18:13:39 +07:00
tiennm99 d1a01830ae feat(gold): add gold paper trading module
Opt-in module for gold paper trading with VND currency.
Commands: /gold_topup, /gold_buy, /gold_sell, /gold_stats.
Pricing: spot XAU USD converted to VND per luong via GoldPrice.org + ExchangeRate-API.
Features: FX caching, dust normalization (1e-9), HTTPS-only URL validation,
per-user key locking, namespace isolation from trading module.
2026-06-11 17:13:02 +07:00
tiennm99 5536c24219 fix: simplify trade income event fetcher 2026-06-05 14:26:57 +07:00
tiennm99 537bfc85e9 feat: add trade income events check 2026-06-05 14:21:38 +07:00
tiennm99 3a628eab97 feat: register trade income commands 2026-06-05 13:58:22 +07:00
tiennm99 55ed77aed1 feat(trading): show remaining VND after buy/sell success 2026-06-05 08:48:45 +07:00
tiennm99 ec77a24db7 chore: drop CF→AWS migration tooling and stale JS-port references
The CF→AWS data migration (closed 2026-05-16) is long done and the
tooling isn't wired into any production path. Remove the one-shot binary,
its support package, and the migration runbook.

In live code, replace 'JS-parity' / 'same shape as JS' / 'cross-runtime
KV migration' comments with the real, stable reason for each behavior
(wire-format invariant, null-vs-zero distinction, CloudWatch alarm field
name, etc.). 24 files touched across lolschedule, loldle, wordle, twentyq,
trading, misc, util, server, metrics, ai, keylock.

- delete cmd/migrate_cf_data/
- delete internal/migration/
- delete docs/cf-to-aws-migration-runbook.md
2026-05-25 09:39:17 +07:00
tiennm99 c901f3ad40 fix(lolschedule): route daily push to originating forum topic
Subscribers previously stored only chatID; the daily-push cron sent
without MessageThreadID so Telegram routed to General regardless of the
topic the user subscribed from.

- Storage shape: []Subscriber{ChatID, ThreadID} keyed by (ChatID, ThreadID)
  so the same chat can subscribe in multiple topics independently.
- Decoder accepts legacy []int64 rows (ThreadID=0); successful mutations
  rewrite the slot in the new shape, no manual migration needed.
- Cron forwards MessageThreadID on every send.
- Auto-prune classifies terminal errors: chat-wide kills (bot blocked,
  deactivated, kicked, deleted, group upgraded) wipe every entry for the
  ChatID; topic-only "have no rights to send" prunes just (ChatID, ThreadID).
2026-05-25 09:39:01 +07:00
tiennm99 1c50023261 feat(stats): add per-user analytics with /stats subcommands
Extend the stats module to track per-user counts in addition to the
existing per-command totals. CommandHook now receives the originating
*models.Update so the hook can attribute invocations to a user; only
the stats module consumes this hook today.

Schema (sort keys under pk="stats"):
  count:<cmd>            existing, per-command total
  user:<id>              new, per-user total with cached username
  pair:<cmd>:<id>        new, per (command, user) pair

Subcommands (all public):
  /stats                 top commands (unchanged)
  /stats users           top users overall
  /stats user <name>     top commands for that user
  /stats cmd <name>      top users of that command

When the sender has no Telegram username the per-user/pair writes are
skipped — the global per-command counter still increments. View helpers
fan out KV reads in parallel to stay inside the webhook deadline. The
existing read-modify-write race is unchanged; closing it would need
atomic UpdateItem ADD on the KV interface.
2026-05-22 18:01:12 +07:00
tiennm99 cf7b8366b3 fix(deploy): include stats in CI ModulesCSV override
SAM CLI's --parameter-overrides replaces samconfig.toml's overrides
rather than merging them, so every CI deploy reset the stack's
ModulesCSV to the prior value (which predated the stats module).
The live Lambda's MODULES env was missing stats, so /stats was
dispatched but never handled — the module factory was never called.
2026-05-22 17:27:26 +07:00
tiennm99 819d4a985e fix(stats): fan out per-key GetItem to avoid handler deadline timeout
Sequential List + N GetItem made /stats latency scale with the number
of tracked commands. On a cold Lambda container with ~25 commands the
cumulative DynamoDB round-trips can exceed the 10s webhook handler
deadline; the trailing chathelper.Reply -> b.SendMessage then fails on
a cancelled ctx and the dispatcher only logs the error, leaving the
user with no reply at all.

Fan the per-key GetJSONs out into goroutines joined by sync.WaitGroup.
Wall-clock latency collapses to ~1 round-trip while preserving the
per-key error isolation (a single GetJSON failure still drops only its
own row). Storage interface unchanged; no race regression vs. the prior
per-item write pattern.
2026-05-22 16:36:57 +07:00
tiennm99 ebdef77883 fix(deploy): hardcode non-secret CFN params instead of sed-parsing samconfig
The sed-based extraction from samconfig.toml worked locally (BusyBox sed)
but failed in CI with GNU sed:
  sed: -e expression #1, char 40: invalid reference \1 on 's' command's RHS

After YAML literal and bash single-quote unescaping, the sed pattern's
parens lacked the BRE \(...\) escaping, so there was no capture group
and \1 was invalid. Rather than fight escape layers for a non-secret
value, just inline the IDs in the workflow.
2026-05-22 15:43:07 +07:00
tiennm99 704e74ca69 fix(deploy): pass BotOwnerID and AdminUserIDs from samconfig.toml in CI
CI's --parameter-overrides replaces (not merges with) samconfig.toml's
parameter_overrides, so BOT_OWNER_ID was empty in the deployed Lambda.
deploynotify silently skipped sending the post-deploy DM, and all
Protected/Private commands were denied.

Extract the values from samconfig.toml at deploy time so the file
remains the single source of truth for non-secret deploy params.

Also add 'stats' to samconfig.toml's ModulesCSV so local `sam deploy`
doesn't regress the stats module.
2026-05-22 15:39:42 +07:00
tiennm99 7f71e3cf6f fix(stats): unblock deploy and register /stats in Telegram command menu
- Remove ReservedConcurrentExecutions: 1 (account lacks unreserved
  concurrency headroom; deploy rolled back). Accept best-effort
  counter semantics as documented.
- Add /stats to aws/telegram-commands.json so it appears in the
  Telegram client's / autocomplete after the next deploy.
2026-05-22 15:30:28 +07:00
tiennm99 4f7a6d17f0 fix(dispatcher): place nolint:gosec on go statement, not inside goroutine 2026-05-22 15:14:07 +07:00