tiennm99/store-scraper-bot
1
0 Fork 0
mirror of https://github.com/tiennm99/store-scraper-bot.git synced 2026-10-20 02:18:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
797f0405743e44d8eb2de2e4c5fde4358358dd78
store-scraper-bot/plans/260426-2015-cloudflare-worker-code-port/plan.md
T
tiennm99 797f040574 feat: Cloudflare Workers code port (deploy pending) 
Refactors source to be Worker-shaped. No live deploy yet — sister deploy plan
runs Atlas provisioning + smoke later.

- wrangler.toml with nodejs_compat_v2, daily UTC 0 cron (= 7am Asia/Ho_Chi_Minh)
- package.json: drop node-telegram-bot-api, node-cron, dotenv, pino,
  pino-pretty; add wrangler devDep; bump to 0.2.0
- src/bot/telegram-api.js: raw fetch wrapper for Telegram Bot API
- src/bot/dispatch.js: per-message dispatcher extracted from polling loop
- src/repository/mongodb.js: memoized MongoClient per warm isolate, typed
  MongoUnavailable error, fast-fail timeouts
- src/repository/store.js: factory binding env once
- All 4 repositories converted to factory shape
- src/api/{apple,google}-scraper.js: take store instead of importing repos
- src/index.js: Worker entry exporting { fetch, scheduled }; webhook validates
  X-Telegram-Bot-Api-Secret-Token; ack-then-waitUntil pattern
- src/scheduler/scheduler.js: trimmed; runDailyCheck only (no node-cron)
- src/config.js, src/logger.js: env-driven, console.log JSON output
- scripts/register-webhook.js: setWebhook + setMyCommands; --dry-run supported
- scripts/check-secret-leaks.js: lint blocks console.log(env.<SECRET>)
- plans/260426-2015-cloudflare-worker-code-port: this code port plan
- plans/260426-2327-cloudflare-deploy-and-smoke: sister deploy plan

Validated via node --check on all 32 source files; lint clean. Real deploy
gates (bundle size, cold-start CPU) run in deploy plan.
2026-04-26 23:36:39 +07:00

5.7 KiB
Raw Blame History

title, description, status, priority, effort, branch, tags, created, blockedBy, blocks
title description status priority effort branch tags created blockedBy blocks
Deploy js-store-scraper-bot to Cloudflare Workers Port Telegram bot from Node.js polling/Mongo-driver/node-cron to Workers webhook + nodejs_compat_v2 + scheduled triggers. MongoDB Atlas M0 free tier retains schema parity with Java/Go ports. pending P1 10h dev
cloudflare-workers
telegram
mongodb
atlas
deployment
2026-04-26

Plan: Cloudflare Workers Deployment

Port the Node.js polling bot to a Workers fetch (Telegram webhook) + scheduled (daily check) handler. Free tier only. Storage stays MongoDB (Atlas M0) via nodejs_compat_v2 — schema parity with store-scraper-bot (Java) and go-store-scraper-bot preserved.

Reference Implementation

miti99bot — sister project on the same stack (CF Workers + Atlas via nodejs_compat_v2 + mongodb driver). Code-complete, operator-pending. Lift validated patterns; do not re-research.

Key files to crib:

  • wrangler.toml — compatibility_flags = ["nodejs_compat_v2"], cron triggers, observability block
  • scripts/check-secret-leaks.js — lint that grep-blocks token leaks
  • docs/using-mongodb.md — operational runbook

Constraints (locked)

  • Free plan only. 100K req/day, 10ms CPU/req baseline (50ms cold start), 3 MiB bundle cap.
  • MongoDB Atlas M0 (free, region aws-ap-southeast-1). Schema parity with Java/Go ports.
  • Telegram webhook (NOT polling). Auth via secret_token header (Telegram's first-class mechanism).
  • nodejs_compat_v2 (gives node:net + node:tls for the official mongodb driver).
  • Drop these npm deps: node-telegram-bot-api, node-cron, dotenv, pino, pino-pretty. Workers ships its own equivalents (env bindings, scheduled handler, console.log).
  • Keep: mongodb (only addition), business logic in src/bot/commands/*.js, models, util.

Hard Gates

Gate Where Threshold On fail
Bundle size Phase 01 wrangler deploy --dry-run ≤ 2.7 MiB (10% headroom under 3 MiB Free cap) Pivot to Upstash Redis fallback (out-of-plan)
Cold-start CPU Phase 01 /__mongo-ping cold cycles well under 50ms CPU Escalate to Workers Paid OR pivot to Upstash
Auto-pause behavior Phase 01 Paused M0 yields catchable MongoServerSelectionError within 5s, not a hang Document catch path in Phase 03

If any gate trips, do NOT continue downstream phases — escalate to user with measured numbers.

Phases

# Phase Status Effort Owner files
01 Atlas + Wrangler + bundle gate pending 2h wrangler.toml, package.json, .dev.vars.example, scripts/check-secret-leaks.js, docs/using-mongodb.md
02 Worker-native Telegram client pending 1.5h src/bot/telegram-api.js, src/bot/bot.js
03 MongoDB Worker adapter pending 1.5h src/repository/mongodb.js, src/repository/*-repository.js
04 Worker entry — fetch + scheduled pending 2h src/index.js, src/bot/dispatch.js, src/scheduler/scheduler.js
05 Webhook registration + secrets pending 1h scripts/register-webhook.js, package.json
06 Deploy + smoke + docs pending 2h README.md, docs/cloudflare-deployment.md

Critical dependencies

  • 01 → all (wrangler config + Atlas creds + bundle gate are prerequisites).
  • 02 + 03 → 04 (entry handler wires both into the dispatch + scheduled paths).
  • 04 → 05 (need a deployable fetch handler before setWebhook makes sense).
  • 05 → 06 (deploy + smoke is end-to-end; needs webhook live).

Out of scope (explicit)

  • Node-runtime variant retained alongside Workers variant. No. Single deployment target. The Node polling code is removed in Phase 04, not preserved with branching.
  • Dual-write to KV. Not applicable — no existing CF deployment to migrate from.
  • Backfill scripts. Not applicable — no existing data; greenfield collections.
  • Tests. The original Node port has none and the user did not request them; revisit if user asks.
  • Worker observability tuning beyond wrangler.toml defaults from miti99bot.

Fallback path (if Phase 01 hard gates trip)

Upstash Redis (HTTP-native, ~10K req/day free tier):

  • Replace src/repository/* with thin wrappers over @upstash/redis.
  • Lose Mongo schema parity with Java/Go ports (acceptable — JS becomes its own deployment).
  • Bundle stays tiny (no driver), cold start essentially free.
  • Spec lives in phase-07-alt-upstash-pivot.md (write only if Phase 01 trips).

Questions for user (before Phase 01 starts)

  1. Atlas account: do you have one, or do you need to create it? (M0 free tier — no credit card required.)
  2. Cloudflare account: do you have a Workers free plan attached to a domain? Workers can deploy without a custom domain (*.workers.dev); confirm that's acceptable for Telegram webhook URL.
  3. Bot username for the webhook: same TELEGRAM_BOT_USERNAME as the existing .env.example?
  4. Any existing data in MongoDB you want imported, or greenfield?

References

Reference in new issue View Git Blame Copy Permalink