mirror of
https://github.com/tiennm99/store-scraper-bot.git
synced 2026-10-11 03:13:50 +00:00
tiennm99
3c97553c69
Node.js 20+ ESM port mirroring Java/Go implementations. - 13 Telegram commands matching Java identifiers - MongoDB schema parity (common, group, apple_app, google_app collections) - Apple/Google scrapers calling store-scraper.vercel.app with 10-min cache - Daily 7am Vietnam-time cron with weekend-silent mode - HTML table renderer matching Java/Go output - Docker + Compose (prod and dev) Untested end-to-end against live Telegram or upstream API.
83 lines
2.8 KiB
Markdown
83 lines
2.8 KiB
Markdown
# js-store-scraper-bot
|
|
|
|
JavaScript (Node.js) port of [store-scraper-bot](https://github.com/tiennm99/store-scraper-bot).
|
|
|
|
> ⚠️ **Preview / unstable — use at your own risk.**
|
|
> This port was produced largely with AI assistance and has **not** been tested
|
|
> end-to-end against a live Telegram bot or the upstream Java implementation.
|
|
> Behavior parity is intended but unverified. Do not run against a production database.
|
|
|
|
The Java version remains the reference implementation.
|
|
|
|
## Status
|
|
|
|
- Mongo schema matches Java/Go (collections: `common`, `group`, `apple_app`,
|
|
`google_app`; string `_id`; `class` discriminator).
|
|
- Telegram command identifiers match Java exactly: `/info`, `/addgroup`,
|
|
`/delgroup`, `/listgroup`, `/addapple`, `/delapple`, `/addgoogle`,
|
|
`/delgoogle`, `/listapp`, `/checkapp`, `/checkappscore`, `/rawappleapp`,
|
|
`/rawgoogleapp`.
|
|
- HTML parse mode; weekend-silent daily report; configurable API cache (default 10 min).
|
|
|
|
## Requirements
|
|
|
|
- Node.js 20+ (uses built-in `fetch`)
|
|
- MongoDB 4.4+
|
|
|
|
## Configuration
|
|
|
|
See `.env.example`:
|
|
|
|
| Name | Notes |
|
|
|---|---|
|
|
| `TELEGRAM_BOT_TOKEN` | Telegram bot token (required) |
|
|
| `TELEGRAM_BOT_USERNAME` | Bot username (required) |
|
|
| `MONGODB_CONNECTION_STRING` | Preferred (Java parity); falls back to `MONGO_URI` |
|
|
| `MONGO_DATABASE` | Optional; inferred from URI path if omitted |
|
|
| `ADMIN_IDS` | Comma-separated Telegram user IDs (required) |
|
|
| `ENV` | `DEVELOPMENT` or `PRODUCTION` |
|
|
| `SOURCE_COMMIT` | Optional; shown on startup |
|
|
| `APP_CACHE_SECONDS` | Cache TTL for upstream API responses (default 600) |
|
|
| `NUM_DAYS_WARNING_NOT_UPDATED` | Threshold for daily warning (default 30) |
|
|
| `SCHEDULE_CHECK_APP_TIME` | Cron expression in Vietnam timezone (default `0 7 * * *`) |
|
|
|
|
## Run
|
|
|
|
```sh
|
|
npm install
|
|
cp .env.example .env # then edit credentials
|
|
npm start
|
|
```
|
|
|
|
Or via Docker Compose:
|
|
|
|
```sh
|
|
docker compose up --build
|
|
```
|
|
|
|
## Project Layout
|
|
|
|
```
|
|
src/
|
|
├── index.js # entry point: wire up config, mongo, scrapers, bot, scheduler
|
|
├── config.js
|
|
├── logger.js
|
|
├── api/
|
|
│ ├── apple-scraper.js
|
|
│ └── google-scraper.js
|
|
├── models/ # plain object factories matching Mongo docs
|
|
├── repository/ # Mongo collection wrappers (admin / group / cached app)
|
|
├── bot/
|
|
│ ├── bot.js # Telegram polling, command dispatch, sender
|
|
│ └── commands/ # one file per /command
|
|
├── scheduler/scheduler.js # daily 7am Vietnam-time check
|
|
└── util/ # table renderer, time helpers
|
|
```
|
|
|
|
## Differences vs Go / Java
|
|
|
|
- Group / admin / chat IDs are JS `number`s. Telegram chat IDs fit in safe-int
|
|
range, so this is intentional and matches Telegram's documented limits.
|
|
- Pino logging instead of Java/Go's structured loggers; semantics equivalent.
|
|
- HTTP via Node 20's built-in `fetch` (no extra dependency).
|