mirror of
https://github.com/tiennm99/store-scraper-bot.git
synced 2026-10-11 03:13:50 +00:00
tiennm99
971c9c311f
Adds the 260505-1425 plan that drove the KV swap (Phase 01 already applied in the preceding commit; Phases 02-05 are operator-driven). Marks 260426-2327-cloudflare-deploy-and-smoke as superseded since the KV pivot was taken pre-emptively rather than after a hard-gate trip.
js-store-scraper-bot
JavaScript (Node.js) port of store-scraper-bot.
⚠️ Preview / unstable — use at your own risk. This port was produced largely with AI assistance and has not been tested end-to-end against a live Telegram bot or the upstream Java implementation. Behavior parity is intended but unverified. Do not run against a production database.
The Java version remains the reference implementation.
Status
- Mongo schema matches Java/Go (collections:
common,group,apple_app,google_app; string_id;classdiscriminator). - Telegram command identifiers match Java exactly:
/info,/addgroup,/delgroup,/listgroup,/addapple,/delapple,/addgoogle,/delgoogle,/listapp,/checkapp,/checkappscore,/rawappleapp,/rawgoogleapp. - HTML parse mode; weekend-silent daily report; configurable API cache (default 10 min).
Requirements
- Node.js 20+ (uses built-in
fetch) - MongoDB 4.4+
Configuration
See .env.example:
| Name | Notes |
|---|---|
TELEGRAM_BOT_TOKEN |
Telegram bot token (required) |
TELEGRAM_BOT_USERNAME |
Bot username (required) |
MONGODB_CONNECTION_STRING |
Preferred (Java parity); falls back to MONGO_URI |
MONGO_DATABASE |
Optional; inferred from URI path if omitted |
ADMIN_IDS |
Comma-separated Telegram user IDs (required) |
ENV |
DEVELOPMENT or PRODUCTION |
SOURCE_COMMIT |
Optional; shown on startup |
APP_CACHE_SECONDS |
Cache TTL for upstream API responses (default 600) |
NUM_DAYS_WARNING_NOT_UPDATED |
Threshold for daily warning (default 30) |
SCHEDULE_CHECK_APP_TIME |
Cron expression in Vietnam timezone (default 0 7 * * *) |
Run
npm install
cp .env.example .env # then edit credentials
npm start
Or via Docker Compose:
docker compose up --build
Project Layout
src/
├── index.js # entry point: wire up config, mongo, scrapers, bot, scheduler
├── config.js
├── logger.js
├── api/
│ ├── apple-scraper.js
│ └── google-scraper.js
├── models/ # plain object factories matching Mongo docs
├── repository/ # Mongo collection wrappers (admin / group / cached app)
├── bot/
│ ├── bot.js # Telegram polling, command dispatch, sender
│ └── commands/ # one file per /command
├── scheduler/scheduler.js # daily 7am Vietnam-time check
└── util/ # table renderer, time helpers
Differences vs Go / Java
- Group / admin / chat IDs are JS
numbers. Telegram chat IDs fit in safe-int range, so this is intentional and matches Telegram's documented limits. - Pino logging instead of Java/Go's structured loggers; semantics equivalent.
- HTTP via Node 20's built-in
fetch(no extra dependency).