Commit Graph
10 Commits
Author SHA1 Message Date
tiennm99 1ca421b3d5 refactor: rename project to store-scraper-bot
GitHub repo rename: js-store-scraper-bot becomes the canonical store-scraper-bot. The Java reference impl is renamed to legacy-store-scraper-bot in parallel.

Updates: package.json name + description, README header + reference link, wrangler.toml worker name (file slated for removal in cleanup phase but kept consistent in the interim).
2026-05-09 17:18:50 +07:00
tiennm99 fbf2e95a52 docs: lock Vercel + Upstash consolidation plan
Brainstorm + research concluded: bot moves off CF Workers + Vercel split-architecture onto a single Vercel deployment with Upstash Redis storage. 7-phase implementation plan added.

Supersedes the in-progress CF KV migration plan; code-level KV migration already shipped, deploy phases never executed.
2026-05-09 17:18:39 +07:00
tiennm99 c4c6a93e06 docs: add Phase 06 (Atlas → KV data migration) to KV plan
Adds the phase doc that drove the migration script and updates plan.md
with the new phase, dependency edges (02 → 06 → 04), and removes the
prior "Out of scope: Data migration" line.
2026-05-05 21:15:27 +07:00
tiennm99 f3a235de00 feat: add Atlas → KV one-shot migration script
Operator runs `npm run migrate` (reads admin + group docs from Atlas)
followed by `npm run migrate:bulk` (uploads via wrangler kv bulk put).
Cache collections are skipped by default since they auto-rebuild from
upstream APIs; --include-cache flag migrates them with TTL preserved.

- mongodb is added as a devDependency only — never enters the Worker
  bundle, the Worker still talks to KV exclusively.
- scripts/.atlas-export.json is gitignored (contains exported state).
- README documents the one-time runbook.
2026-05-05 21:15:18 +07:00
tiennm99 af8f1516fa docs: add KV migration plan, supersede Atlas deploy plan
Adds the 260505-1425 plan that drove the KV swap (Phase 01 already
applied in the preceding commit; Phases 02-05 are operator-driven).
Marks 260426-2327-cloudflare-deploy-and-smoke as superseded since
the KV pivot was taken pre-emptively rather than after a hard-gate trip.
2026-05-05 20:39:12 +07:00
tiennm99 067d463b6a feat: replace MongoDB driver with Cloudflare KV storage
Drops the mongodb dependency entirely; all four logical collections
(admin singleton, group, apple_app, google_app) now live in a single KV
namespace bound as STORE_KV with prefixed keys. Cache TTL is delegated
to KV via expirationTtl (clamped to the 60s minimum). Document shape,
field names, and Java parity at the doc level are preserved.

- Adds src/repository/kv.js helper (getJson/putJson/del with TTL clamp)
- Rewrites all four *-repository.js modules on top of KV
- Removes src/repository/mongodb.js and the MONGODB_URI env requirement
- Adds an early STORE_KV-binding guard in src/index.js
- Bumps to 0.3.0
2026-05-05 20:39:01 +07:00
tiennm99 e3e375d203 docs: add Java vs JS parity reports from /ck:xia
researcher report maps 47 Java files to 35 JS files (11 PARITY, 2
MINOR, 0 GAP, 3 EXTRA). xia synthesis records source manifest, decision
matrix, and risk score. No port work needed; remaining items are
operational.
2026-04-29 16:07:17 +07:00
tiennm99 cbe08d6617 chore: archive code-port plan, add plans/todo.md index
Code port (commit bff1d32) is done; move plan to plans/archive/.
New plans/todo.md is the forward-looking index pointing at the active deploy
plan, pre-flight checks, hard gates, and open questions for the operator.
2026-04-26 23:46:22 +07:00
tiennm99 bff1d324f5 feat: Cloudflare Workers code port (deploy pending)
Refactors source to be Worker-shaped. No live deploy yet — sister deploy plan
runs Atlas provisioning + smoke later.

- wrangler.toml with nodejs_compat_v2, daily UTC 0 cron (= 7am Asia/Ho_Chi_Minh)
- package.json: drop node-telegram-bot-api, node-cron, dotenv, pino,
  pino-pretty; add wrangler devDep; bump to 0.2.0
- src/bot/telegram-api.js: raw fetch wrapper for Telegram Bot API
- src/bot/dispatch.js: per-message dispatcher extracted from polling loop
- src/repository/mongodb.js: memoized MongoClient per warm isolate, typed
  MongoUnavailable error, fast-fail timeouts
- src/repository/store.js: factory binding env once
- All 4 repositories converted to factory shape
- src/api/{apple,google}-scraper.js: take store instead of importing repos
- src/index.js: Worker entry exporting { fetch, scheduled }; webhook validates
  X-Telegram-Bot-Api-Secret-Token; ack-then-waitUntil pattern
- src/scheduler/scheduler.js: trimmed; runDailyCheck only (no node-cron)
- src/config.js, src/logger.js: env-driven, console.log JSON output
- scripts/register-webhook.js: setWebhook + setMyCommands; --dry-run supported
- scripts/check-secret-leaks.js: lint blocks console.log(env.<SECRET>)
- plans/260426-2015-cloudflare-worker-code-port: this code port plan
- plans/260426-2327-cloudflare-deploy-and-smoke: sister deploy plan

Validated via node --check on all 32 source files; lint clean. Real deploy
gates (bundle size, cold-start CPU) run in deploy plan.
2026-04-26 23:36:39 +07:00
tiennm99 a656a18ce8 feat: initial JavaScript port of store-scraper-bot
Node.js 20+ ESM port mirroring Java/Go implementations.

- 13 Telegram commands matching Java identifiers
- MongoDB schema parity (common, group, apple_app, google_app collections)
- Apple/Google scrapers calling store-scraper.vercel.app with 10-min cache
- Daily 7am Vietnam-time cron with weekend-silent mode
- HTML table renderer matching Java/Go output
- Docker + Compose (prod and dev)

Untested end-to-end against live Telegram or upstream API.
2026-04-26 20:16:00 +07:00