mirror of
https://github.com/tiennm99/store-scraper-bot.git
synced 2026-10-11 03:13:50 +00:00
feat: initial JavaScript port of store-scraper-bot
Node.js 20+ ESM port mirroring Java/Go implementations. - 13 Telegram commands matching Java identifiers - MongoDB schema parity (common, group, apple_app, google_app collections) - Apple/Google scrapers calling store-scraper.vercel.app with 10-min cache - Daily 7am Vietnam-time cron with weekend-silent mode - HTML table renderer matching Java/Go output - Docker + Compose (prod and dev) Untested end-to-end against live Telegram or upstream API.
This commit is contained in:
1 parent
ca1a409610
commit
3c97553c69
41 files changed
+1852
No files matched your search
@@ -0,0 +1,82 @@
|
||||
# js-store-scraper-bot
|
||||
|
||||
JavaScript (Node.js) port of [store-scraper-bot](https://github.com/tiennm99/store-scraper-bot).
|
||||
|
||||
> ⚠️ **Preview / unstable — use at your own risk.**
|
||||
> This port was produced largely with AI assistance and has **not** been tested
|
||||
> end-to-end against a live Telegram bot or the upstream Java implementation.
|
||||
> Behavior parity is intended but unverified. Do not run against a production database.
|
||||
|
||||
The Java version remains the reference implementation.
|
||||
|
||||
## Status
|
||||
|
||||
- Mongo schema matches Java/Go (collections: `common`, `group`, `apple_app`,
|
||||
`google_app`; string `_id`; `class` discriminator).
|
||||
- Telegram command identifiers match Java exactly: `/info`, `/addgroup`,
|
||||
`/delgroup`, `/listgroup`, `/addapple`, `/delapple`, `/addgoogle`,
|
||||
`/delgoogle`, `/listapp`, `/checkapp`, `/checkappscore`, `/rawappleapp`,
|
||||
`/rawgoogleapp`.
|
||||
- HTML parse mode; weekend-silent daily report; configurable API cache (default 10 min).
|
||||
|
||||
## Requirements
|
||||
|
||||
- Node.js 20+ (uses built-in `fetch`)
|
||||
- MongoDB 4.4+
|
||||
|
||||
## Configuration
|
||||
|
||||
See `.env.example`:
|
||||
|
||||
| Name | Notes |
|
||||
|---|---|
|
||||
| `TELEGRAM_BOT_TOKEN` | Telegram bot token (required) |
|
||||
| `TELEGRAM_BOT_USERNAME` | Bot username (required) |
|
||||
| `MONGODB_CONNECTION_STRING` | Preferred (Java parity); falls back to `MONGO_URI` |
|
||||
| `MONGO_DATABASE` | Optional; inferred from URI path if omitted |
|
||||
| `ADMIN_IDS` | Comma-separated Telegram user IDs (required) |
|
||||
| `ENV` | `DEVELOPMENT` or `PRODUCTION` |
|
||||
| `SOURCE_COMMIT` | Optional; shown on startup |
|
||||
| `APP_CACHE_SECONDS` | Cache TTL for upstream API responses (default 600) |
|
||||
| `NUM_DAYS_WARNING_NOT_UPDATED` | Threshold for daily warning (default 30) |
|
||||
| `SCHEDULE_CHECK_APP_TIME` | Cron expression in Vietnam timezone (default `0 7 * * *`) |
|
||||
|
||||
## Run
|
||||
|
||||
```sh
|
||||
npm install
|
||||
cp .env.example .env # then edit credentials
|
||||
npm start
|
||||
```
|
||||
|
||||
Or via Docker Compose:
|
||||
|
||||
```sh
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
## Project Layout
|
||||
|
||||
```
|
||||
src/
|
||||
├── index.js # entry point: wire up config, mongo, scrapers, bot, scheduler
|
||||
├── config.js
|
||||
├── logger.js
|
||||
├── api/
|
||||
│ ├── apple-scraper.js
|
||||
│ └── google-scraper.js
|
||||
├── models/ # plain object factories matching Mongo docs
|
||||
├── repository/ # Mongo collection wrappers (admin / group / cached app)
|
||||
├── bot/
|
||||
│ ├── bot.js # Telegram polling, command dispatch, sender
|
||||
│ └── commands/ # one file per /command
|
||||
├── scheduler/scheduler.js # daily 7am Vietnam-time check
|
||||
└── util/ # table renderer, time helpers
|
||||
```
|
||||
|
||||
## Differences vs Go / Java
|
||||
|
||||
- Group / admin / chat IDs are JS `number`s. Telegram chat IDs fit in safe-int
|
||||
range, so this is intentional and matches Telegram's documented limits.
|
||||
- Pino logging instead of Java/Go's structured loggers; semantics equivalent.
|
||||
- HTTP via Node 20's built-in `fetch` (no extra dependency).
|
||||
Reference in new issue
Block a user