feat: initial JavaScript port of store-scraper-bot

Node.js 20+ ESM port mirroring Java/Go implementations.

- 13 Telegram commands matching Java identifiers
- MongoDB schema parity (common, group, apple_app, google_app collections)
- Apple/Google scrapers calling store-scraper.vercel.app with 10-min cache
- Daily 7am Vietnam-time cron with weekend-silent mode
- HTML table renderer matching Java/Go output
- Docker + Compose (prod and dev)

Untested end-to-end against live Telegram or upstream API.
This commit is contained in:
tiennm99 committed 2026-04-26 20:16:00 +07:00
1 parent ca1a409610
commit 3c97553c69
41 files changed
+1852

No files matched your search

+82
View File
@@ -0,0 +1,82 @@
# js-store-scraper-bot
JavaScript (Node.js) port of [store-scraper-bot](https://github.com/tiennm99/store-scraper-bot).
> ⚠️ **Preview / unstable — use at your own risk.**
> This port was produced largely with AI assistance and has **not** been tested
> end-to-end against a live Telegram bot or the upstream Java implementation.
> Behavior parity is intended but unverified. Do not run against a production database.
The Java version remains the reference implementation.
## Status
- Mongo schema matches Java/Go (collections: `common`, `group`, `apple_app`,
`google_app`; string `_id`; `class` discriminator).
- Telegram command identifiers match Java exactly: `/info`, `/addgroup`,
`/delgroup`, `/listgroup`, `/addapple`, `/delapple`, `/addgoogle`,
`/delgoogle`, `/listapp`, `/checkapp`, `/checkappscore`, `/rawappleapp`,
`/rawgoogleapp`.
- HTML parse mode; weekend-silent daily report; configurable API cache (default 10 min).
## Requirements
- Node.js 20+ (uses built-in `fetch`)
- MongoDB 4.4+
## Configuration
See `.env.example`:
| Name | Notes |
|---|---|
| `TELEGRAM_BOT_TOKEN` | Telegram bot token (required) |
| `TELEGRAM_BOT_USERNAME` | Bot username (required) |
| `MONGODB_CONNECTION_STRING` | Preferred (Java parity); falls back to `MONGO_URI` |
| `MONGO_DATABASE` | Optional; inferred from URI path if omitted |
| `ADMIN_IDS` | Comma-separated Telegram user IDs (required) |
| `ENV` | `DEVELOPMENT` or `PRODUCTION` |
| `SOURCE_COMMIT` | Optional; shown on startup |
| `APP_CACHE_SECONDS` | Cache TTL for upstream API responses (default 600) |
| `NUM_DAYS_WARNING_NOT_UPDATED` | Threshold for daily warning (default 30) |
| `SCHEDULE_CHECK_APP_TIME` | Cron expression in Vietnam timezone (default `0 7 * * *`) |
## Run
```sh
npm install
cp .env.example .env # then edit credentials
npm start
```
Or via Docker Compose:
```sh
docker compose up --build
```
## Project Layout
```
src/
├── index.js # entry point: wire up config, mongo, scrapers, bot, scheduler
├── config.js
├── logger.js
├── api/
│ ├── apple-scraper.js
│ └── google-scraper.js
├── models/ # plain object factories matching Mongo docs
├── repository/ # Mongo collection wrappers (admin / group / cached app)
├── bot/
│ ├── bot.js # Telegram polling, command dispatch, sender
│ └── commands/ # one file per /command
├── scheduler/scheduler.js # daily 7am Vietnam-time check
└── util/ # table renderer, time helpers
```
## Differences vs Go / Java
- Group / admin / chat IDs are JS `number`s. Telegram chat IDs fit in safe-int
range, so this is intentional and matches Telegram's documented limits.
- Pino logging instead of Java/Go's structured loggers; semantics equivalent.
- HTTP via Node 20's built-in `fetch` (no extra dependency).