From 546e6fcde20976ec45eb1dba27c44420fe3de4ac Mon Sep 17 00:00:00 2001 From: tiennm99 Date: Fri, 26 Jun 2026 13:40:29 +0700 Subject: [PATCH] refactor(store): replace Redis with MongoDB datastore Swap the Redis datastore for MongoDB via mongo-driver v2. New internal/mongostore mirrors the old redisstore type/constant names so consumers change only the import qualifier; document model collapses the multi-key subscriber/settings and per-event delivery sets into single-doc operations and a TTL-indexed delivery collection. Config now requires MONGODB_URI (Atlas) with MONGODB_DATABASE selecting dev vs prod on one cluster; REDIS_URL removed. Compose files run bot-only against Atlas. Default tests stay Docker-free (poller/bot use fakes, mongostore unit tests are pure-logic); a gated //go:build integration suite covers the real backend via testcontainers. --- .env.example | 3 +- README.md | 32 +- cmd/openai-status-bot/main.go | 27 +- docker-compose.bot.yml | 3 + docker-compose.yml | 18 +- docs/component-id-monitoring-report.md | 16 +- docs/setup-guide.md | 26 +- docs/system-architecture.md | 35 +- go.mod | 64 +++- go.sum | 185 ++++++++- internal/bot/bot.go | 16 +- internal/bot/bot_test.go | 30 +- internal/bot/commands.go | 24 +- internal/bot/helpers.go | 4 +- internal/bot/subscription_format.go | 6 +- internal/config/config.go | 36 +- internal/config/config_test.go | 54 +-- internal/mongostore/checkpoint.go | 185 +++++++++ internal/mongostore/indexes.go | 30 ++ internal/mongostore/store.go | 50 +++ internal/mongostore/store_test.go | 358 ++++++++++++++++++ internal/mongostore/subscriber.go | 195 ++++++++++ .../subscriber_normalization.go | 2 +- internal/mongostore/subscriber_test.go | 66 ++++ internal/poller/delivery.go | 4 +- internal/poller/event_collection.go | 14 +- internal/poller/poller.go | 12 +- internal/poller/poller_test.go | 58 +-- internal/redisstore/checkpoint.go | 171 --------- internal/redisstore/store.go | 27 -- internal/redisstore/store_test.go | 243 ------------ internal/redisstore/subscriber.go | 181 --------- internal/redisstore/subscriber_settings.go | 99 ----- internal/telegram/client.go | 4 +- .../phase-01-suitability-decision.md | 65 ++++ .../phase-02-store-package-connection.md | 59 +++ .../phase-03-implement-store-methods.md | 67 ++++ .../phase-04-config-wiring-deployment.md | 51 +++ .../phase-05-cutover-tests.md | 69 ++++ plans/260626-1057-redis-to-mongodb/plan.md | 73 ++++ 40 files changed, 1704 insertions(+), 958 deletions(-) create mode 100644 internal/mongostore/checkpoint.go create mode 100644 internal/mongostore/indexes.go create mode 100644 internal/mongostore/store.go create mode 100644 internal/mongostore/store_test.go create mode 100644 internal/mongostore/subscriber.go rename internal/{redisstore => mongostore}/subscriber_normalization.go (98%) create mode 100644 internal/mongostore/subscriber_test.go delete mode 100644 internal/redisstore/checkpoint.go delete mode 100644 internal/redisstore/store.go delete mode 100644 internal/redisstore/store_test.go delete mode 100644 internal/redisstore/subscriber.go delete mode 100644 internal/redisstore/subscriber_settings.go create mode 100644 plans/260626-1057-redis-to-mongodb/phase-01-suitability-decision.md create mode 100644 plans/260626-1057-redis-to-mongodb/phase-02-store-package-connection.md create mode 100644 plans/260626-1057-redis-to-mongodb/phase-03-implement-store-methods.md create mode 100644 plans/260626-1057-redis-to-mongodb/phase-04-config-wiring-deployment.md create mode 100644 plans/260626-1057-redis-to-mongodb/phase-05-cutover-tests.md create mode 100644 plans/260626-1057-redis-to-mongodb/plan.md diff --git a/.env.example b/.env.example index 5f67d54..7c08a23 100644 --- a/.env.example +++ b/.env.example @@ -1,5 +1,6 @@ TELEGRAM_BOT_TOKEN= -REDIS_URL=redis://localhost:6379/0 +MONGODB_URI=mongodb+srv://:@/ +MONGODB_DATABASE=openai_status_bot POLL_INTERVAL=1m HTTP_TIMEOUT=10s LOG_LEVEL=info diff --git a/README.md b/README.md index 887a97e..48b20c2 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # openai-status-bot -Telegram bot that watches [OpenAI Status](https://status.openai.com/) every minute and sends updates to subscribed chats. State and subscriptions are stored in Redis. +Telegram bot that watches [OpenAI Status](https://status.openai.com/) every minute and sends updates to subscribed chats. State and subscriptions are stored in MongoDB. ## Features @@ -8,12 +8,12 @@ Telegram bot that watches [OpenAI Status](https://status.openai.com/) every minu - Notifies subscribers about new incident updates - Notifies subscribers when component status changes - Supports incident-only, component-only, and component-filtered subscriptions -- Uses Redis for subscribers, delivery state, component checkpoints, and seen incident update versions +- Uses MongoDB for subscribers, delivery state, component checkpoints, and seen incident update versions - Supports Telegram supergroup topics via `message_thread_id` - Clears an existing Telegram webhook before long polling, for migration from webhook deployments - Registers the Telegram command menu on startup - Exposes a local health endpoint for container health checks -- Includes Docker Compose for local Redis + bot runtime +- Includes Docker Compose for the bot runtime (datastore is managed MongoDB Atlas) ## Bot Commands @@ -32,17 +32,24 @@ Telegram bot that watches [OpenAI Status](https://status.openai.com/) every minu ## Quick Start +There is no local datastore. Both Compose files run only the bot and connect to +a managed MongoDB Atlas cluster, so a reachable `MONGODB_URI` is required to +start. The development and production setups share one cluster and differ only +by database name (`MONGODB_DATABASE`). + +Development runtime (targets the `openai_status_bot_dev` database): + ```bash cp .env.example .env -# edit .env and set TELEGRAM_BOT_TOKEN +# edit .env and set TELEGRAM_BOT_TOKEN and MONGODB_URI (Atlas connection string) docker compose up --build ``` -To deploy only the bot when Redis is hosted elsewhere: +Production runtime (uses the `MONGODB_DATABASE` from `.env`, default `openai_status_bot`): ```bash cp .env.example .env -# edit .env and set TELEGRAM_BOT_TOKEN plus REDIS_URL for the external Redis host +# edit .env and set TELEGRAM_BOT_TOKEN, MONGODB_URI, and MONGODB_DATABASE docker compose -f docker-compose.bot.yml up -d --build ``` @@ -58,17 +65,24 @@ go run ./cmd/openai-status-bot | Variable | Default | Description | |----------|---------|-------------| | `TELEGRAM_BOT_TOKEN` | required | Telegram bot token from BotFather | -| `REDIS_URL` | `redis://localhost:6379/0` | Redis connection URL, e.g. `redis://:password@localhost:6379/0` | +| `MONGODB_URI` | required | MongoDB connection string, e.g. an Atlas `mongodb+srv://user:pass@cluster/` URI | +| `MONGODB_DATABASE` | `openai_status_bot` | Database name; use a separate name (e.g. `openai_status_bot_dev`) to split dev and prod on one cluster | | `POLL_INTERVAL` | `1m` | Status check interval, valid `5s`-`1h` | | `HTTP_TIMEOUT` | `10s` | HTTP request timeout, valid `1s`-`5m` | | `LOG_LEVEL` | `info` | `debug`, `info`, `warn`/`warning`, or `error` | -Percent-encode Redis usernames or passwords that contain URL-reserved characters such as `@`, `:`, `/`, `#`, or `%`. +Percent-encode MongoDB usernames or passwords that contain URL-reserved characters such as `@`, `:`, `/`, `#`, or `%`. The bot always reads OpenAI status from `https://status.openai.com`. ## Notes -The first successful poll seeds Redis and does not send historical incidents. Notifications start from later changes. +The first successful poll seeds the database and does not send historical incidents. Notifications start from later changes. + +Switching from a prior Redis deployment starts from empty state: there is no data migration, so subscribers must re-issue `/start` and component checkpoints reseed on the first poll. Incident update dedupe tracks the update content/version, so edited Statuspage updates can notify again. Delivery is checkpointed after successful fan-out; retryable Telegram failures may be retried on a later poll without advancing the global checkpoint. + +A 7-day TTL index on the `delivery` collection expires per-event delivery markers automatically; the bot creates required indexes on startup. + +Integration tests for the MongoDB store run a real `mongod` via testcontainers and are excluded from the default `go test ./...`. Run them with Docker available: `go test -tags=integration ./internal/mongostore/...`. diff --git a/cmd/openai-status-bot/main.go b/cmd/openai-status-bot/main.go index 151099e..12a4c1c 100644 --- a/cmd/openai-status-bot/main.go +++ b/cmd/openai-status-bot/main.go @@ -7,14 +7,15 @@ import ( "os/signal" "syscall" - "github.com/redis/go-redis/v9" "github.com/tiennm99/openai-status-bot/internal/bot" "github.com/tiennm99/openai-status-bot/internal/config" "github.com/tiennm99/openai-status-bot/internal/health" + "github.com/tiennm99/openai-status-bot/internal/mongostore" openai "github.com/tiennm99/openai-status-bot/internal/openai" "github.com/tiennm99/openai-status-bot/internal/poller" - "github.com/tiennm99/openai-status-bot/internal/redisstore" "github.com/tiennm99/openai-status-bot/internal/telegram" + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" ) func main() { @@ -34,14 +35,26 @@ func main() { go health.Run(ctx, logger) - redisClient := redis.NewClient(cfg.RedisOptions) - if err := redisClient.Ping(ctx).Err(); err != nil { - logger.Error("connect redis", "network", cfg.RedisOptions.Network, "addr", cfg.RedisOptions.Addr, "db", cfg.RedisOptions.DB, "tls", cfg.RedisOptions.TLSConfig != nil, "error", err) + mongoClient, err := mongo.Connect(options.Client().ApplyURI(cfg.MongoURI)) + if err != nil { + logger.Error("connect mongodb", "database", cfg.MongoDatabase, "error", err) + os.Exit(1) + } + defer func() { + if err := mongoClient.Disconnect(context.Background()); err != nil { + logger.Warn("disconnect mongodb", "error", err) + } + }() + if err := mongoClient.Ping(ctx, nil); err != nil { + logger.Error("ping mongodb", "database", cfg.MongoDatabase, "error", err) os.Exit(1) } - defer redisClient.Close() - store := redisstore.New(redisClient) + store := mongostore.New(mongoClient, cfg.MongoDatabase) + if err := store.EnsureIndexes(ctx); err != nil { + logger.Error("ensure mongodb indexes", "database", cfg.MongoDatabase, "error", err) + os.Exit(1) + } statusClient := openai.NewClient(cfg.HTTPTimeout) telegramClient := telegram.NewClient(cfg.TelegramBotToken, cfg.HTTPTimeout) if err := telegramClient.DeleteWebhook(ctx); err != nil { diff --git a/docker-compose.bot.yml b/docker-compose.bot.yml index 23b803b..119bb27 100644 --- a/docker-compose.bot.yml +++ b/docker-compose.bot.yml @@ -1,3 +1,6 @@ +# Production runtime: bot only, connecting to the shared Atlas cluster via .env. +# Uses the production database (MONGODB_DATABASE in .env, default +# openai_status_bot). There is no local datastore. services: bot: build: . diff --git a/docker-compose.yml b/docker-compose.yml index 855218d..3e5f91f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,21 +1,11 @@ +# Development runtime: bot only, connecting to the shared Atlas cluster. +# There is no local datastore; set MONGODB_URI in .env. This file targets the +# development database (the MONGODB_DATABASE below overrides any .env value). services: - redis: - image: redis:7-alpine - ports: - - "127.0.0.1:6379:6379" - volumes: - - redis-data:/data - command: ["redis-server", "--appendonly", "yes"] - bot: build: . env_file: - .env environment: - REDIS_URL: redis://redis:6379/0 - depends_on: - - redis + MONGODB_DATABASE: openai_status_bot_dev restart: unless-stopped - -volumes: - redis-data: diff --git a/docs/component-id-monitoring-report.md b/docs/component-id-monitoring-report.md index f2ff5da..be069ee 100644 --- a/docs/component-id-monitoring-report.md +++ b/docs/component-id-monitoring-report.md @@ -18,9 +18,9 @@ Current decision: monitor next incidents/component updates before changing stora ## Findings -- Component checkpoints are stored in Redis as `component_id -> last_status`. -- Redis key: `openai-status:component-statuses`. -- Component names are not stored in that checkpoint hash. +- Component checkpoints are stored in MongoDB as `component_id -> last_status`. +- MongoDB collection: `component_statuses` (document ID is component ID, `status` field holds the last seen status). +- Component names are not stored in that checkpoint document. - Component names come from live OpenAI `summary.json` during each poll. - The bot compares component status by `component.ID`, then formats the alert with `component.Name`. - Incident `page_id` is not used to detect components. @@ -35,21 +35,21 @@ If component ID changed: - Bot treats the component as new. - Previous status becomes `unknown`. -- Old Redis component ID remains unused. +- Old MongoDB component ID remains unused. - Component-filtered subscriptions using old ID may stop matching. - Unfiltered component subscriptions still receive updates. Other possible causes: -- Redis checkpoint was cleared or partially lost. -- `openai-status:initialized` exists while `openai-status:component-statuses` is missing entries. +- MongoDB checkpoint was cleared or partially lost. +- `meta` collection has `initialized` document while `component_statuses` collection is missing entries. - OpenAI added a new component. ## Current State - No code change now. - Monitor upcoming incidents and component updates. -- Compare current OpenAI component IDs with stored Redis component IDs. +- Compare current OpenAI component IDs with stored MongoDB component IDs (in `component_statuses` collection). - Decide later whether ID-only storage is enough. ## Future Options @@ -62,5 +62,5 @@ Other possible causes: ## Unresolved Questions - Did OpenAI actually change the component ID? -- Are old component IDs present in Redis but absent from current `summary.json`? +- Are old component IDs present in MongoDB but absent from current `summary.json`? - Should component subscriptions survive ID changes by matching component name? diff --git a/docs/setup-guide.md b/docs/setup-guide.md index ec3d737..e55eff5 100644 --- a/docs/setup-guide.md +++ b/docs/setup-guide.md @@ -3,7 +3,7 @@ ## Prerequisites - Go 1.24+ -- Redis 7+ +- MongoDB Atlas cluster (managed, no local MongoDB service required) - Telegram bot token from BotFather ## Local Docker Run @@ -11,40 +11,38 @@ ```bash cp .env.example .env # set TELEGRAM_BOT_TOKEN in .env -# REDIS_URL is overridden to redis://redis:6379/0 by docker compose +# set MONGODB_URI to your MongoDB Atlas connection string +# docker compose sets MONGODB_DATABASE=openai_status_bot_dev for local development docker compose up --build ``` -The bundled Redis service is for local development and binds to `127.0.0.1:6379` on the host. +The Docker Compose service connects to MongoDB Atlas; there is no bundled local MongoDB. Ensure `MONGODB_URI` is set in `.env`. ## Bot-Only Docker Run -Use this when Redis is hosted outside Docker Compose: +Use this for production deployment: ```bash cp .env.example .env # set TELEGRAM_BOT_TOKEN in .env -# set REDIS_URL to the external Redis URL, not localhost +# set MONGODB_URI to your MongoDB Atlas connection string +# MONGODB_DATABASE defaults to openai_status_bot (production) docker compose -f docker-compose.bot.yml up -d --build ``` ## Local Go Run -Start Redis: - -```bash -docker run --rm -p 127.0.0.1:6379:6379 redis:7-alpine -``` - -Run bot: - ```bash cp .env.example .env +# set TELEGRAM_BOT_TOKEN in .env +# set MONGODB_URI to your MongoDB Atlas connection string # export TELEGRAM_BOT_TOKEN or source .env with your shell workflow -# REDIS_URL defaults to redis://localhost:6379/0 +# MONGODB_DATABASE defaults to openai_status_bot go run ./cmd/openai-status-bot ``` +MongoDB Atlas connection is required; there is no local MongoDB fallback. + ## Verification ```bash diff --git a/docs/system-architecture.md b/docs/system-architecture.md index 425c276..e4b8c63 100644 --- a/docs/system-architecture.md +++ b/docs/system-architecture.md @@ -7,39 +7,36 @@ - Telegram long polling loop for user commands. - OpenAI status polling loop, default every minute. -Redis stores subscribers, subscription settings, polling checkpoints, delivery retry state, and the Telegram update offset. +MongoDB stores subscribers, subscription settings, polling checkpoints, delivery retry state, and the Telegram update offset. ## Data Flow 1. User sends `/start` in Telegram. -2. Bot stores chat ID, optional topic thread ID, and default subscription settings in Redis. +2. Bot stores chat ID, optional topic thread ID, and default subscription settings in MongoDB. 3. Users can adjust settings with `/subscribe` and inspect them with `/info`. 4. Poller fetches OpenAI status JSON: - `GET https://status.openai.com/api/v2/summary.json` - `GET https://status.openai.com/api/v2/incidents.json` -5. Poller compares fetched state with Redis checkpoints and builds notification events without mutating checkpoints. +5. Poller compares fetched state with MongoDB checkpoints and builds notification events without mutating checkpoints. 6. Events are sent to eligible subscribers, respecting incident/component preferences and component ID filters. 7. Component and incident checkpoints are written only after delivery succeeds or terminal subscriber failures are removed. -## Redis Keys +## MongoDB Collections -| Key | Type | Purpose | -|-----|------|---------| -| `openai-status:subscribers` | set | Telegram chat or topic subscribers | -| `openai-status:subscriber-settings` | hash | Subscription types and component ID filters by subscriber key | -| `openai-status:component-statuses` | hash | Last seen component status by component ID | -| `openai-status:pending-component-events` | hash | Component changes saved before fan-out so retryable delivery failures can be resumed | -| `openai-status:incident-updates` | set | Legacy seen incident update IDs, retained as write-through metadata; version hash is authoritative | -| `openai-status:incident-update-versions` | hash | Seen incident update version by update ID | -| `openai-status:event-delivery:` | set | Temporary per-event subscriber delivery state for retry isolation | -| `openai-status:telegram-offset` | string | Last processed Telegram update offset | -| `openai-status:initialized` | string | Baseline seed marker | +| Collection | Document ID | Fields | Purpose | +|-----------|------------|--------|---------| +| `subscribers` | `chatID` or `chatID:threadID` | chatID, threadID, types, components | Telegram chat or topic subscribers with subscription settings | +| `component_statuses` | component ID | status | Last seen component status by component ID | +| `pending_component_events` | component ID | — | Component changes saved before fan-out so retryable delivery failures can be resumed | +| `incident_update_versions` | update ID | version | Seen incident update version by update ID | +| `delivery` | `eventKey\|subscriber` | eventKey, subscriber, expiresAt (TTL 7 days) | Temporary per-event subscriber delivery state for retry isolation | +| `meta` | `initialized` or `telegramOffset` | value | Baseline seed marker and last processed Telegram update offset | -Subscriber set members are `chatID` or `chatID:threadID`. Missing settings default to incident and component notifications for backward compatibility with older Redis state. +Subscriber document IDs are `chatID` or `chatID:threadID`. Each subscriber document includes subscription types and component ID filters as fields within the document. ## Runtime -The service uses Telegram `getUpdates`, so it does not need a public webhook URL. On startup it starts a local health endpoint at `127.0.0.1:8080/healthz`, calls `deleteWebhook` before long polling, registers the Telegram command menu, and then starts polling. Redis is configured with `REDIS_URL`. The OpenAI status source is fixed to `https://status.openai.com`. Docker Compose starts Redis and the bot for local development; the bundled Redis port is bound to `127.0.0.1` on the host. +The service uses Telegram `getUpdates`, so it does not need a public webhook URL. On startup it starts a local health endpoint at `127.0.0.1:8080/healthz`, calls `deleteWebhook` before long polling, registers the Telegram command menu, and then starts polling. MongoDB is configured with `MONGODB_URI` and `MONGODB_DATABASE` (default `openai_status_bot`). The OpenAI status source is fixed to `https://status.openai.com`. Docker Compose starts the bot connecting to a managed MongoDB Atlas cluster; there is no bundled local MongoDB service. ## Failure Behavior @@ -49,5 +46,5 @@ The service uses Telegram `getUpdates`, so it does not need a public webhook URL - Successful per-subscriber deliveries are tracked temporarily, so retrying one failed subscriber does not resend to already-delivered subscribers. - Pending component events are stored before delivery and removed only after successful delivery or terminal subscriber cleanup. - Telegram 403 and selected terminal 400 errors remove the unreachable subscriber, then delivery continues. -- Malformed subscriber keys are removed from Redis and surfaced as an error for the current poll instead of being skipped silently. -- Redis connection failure at startup exits the process. +- Malformed subscriber document IDs are removed from MongoDB and surfaced as an error for the current poll instead of being skipped silently. +- MongoDB connection failure at startup exits the process. diff --git a/go.mod b/go.mod index 5436a14..0237d0f 100644 --- a/go.mod +++ b/go.mod @@ -1,14 +1,68 @@ module github.com/tiennm99/openai-status-bot -go 1.24 +go 1.25.0 require ( - github.com/alicebob/miniredis/v2 v2.38.0 - github.com/redis/go-redis/v9 v9.17.0 + github.com/testcontainers/testcontainers-go/modules/mongodb v0.43.0 + go.mongodb.org/mongo-driver/v2 v2.7.0 ) require ( + dario.cat/mergo v1.0.2 // indirect + github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect + github.com/Microsoft/go-winio v0.6.2 // indirect + github.com/cenkalti/backoff/v4 v4.3.0 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect - github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect - github.com/yuin/gopher-lua v1.1.1 // indirect + github.com/containerd/errdefs v1.0.0 // indirect + github.com/containerd/errdefs/pkg v0.3.0 // indirect + github.com/containerd/log v0.1.0 // indirect + github.com/containerd/platforms v0.2.1 // indirect + github.com/cpuguy83/dockercfg v0.3.2 // indirect + github.com/davecgh/go-spew v1.1.1 // indirect + github.com/distribution/reference v0.6.0 // indirect + github.com/docker/go-connections v0.6.0 // indirect + github.com/docker/go-units v0.5.0 // indirect + github.com/ebitengine/purego v0.10.0 // indirect + github.com/felixge/httpsnoop v1.0.4 // indirect + github.com/go-logr/logr v1.4.3 // indirect + github.com/go-logr/stdr v1.2.2 // indirect + github.com/go-ole/go-ole v1.2.6 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/klauspost/compress v1.18.5 // indirect + github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect + github.com/magiconair/properties v1.8.10 // indirect + github.com/moby/docker-image-spec v1.3.1 // indirect + github.com/moby/go-archive v0.2.0 // indirect + github.com/moby/moby/api v1.54.2 // indirect + github.com/moby/moby/client v0.4.0 // indirect + github.com/moby/patternmatcher v0.6.1 // indirect + github.com/moby/sys/sequential v0.6.0 // indirect + github.com/moby/sys/user v0.4.0 // indirect + github.com/moby/sys/userns v0.1.0 // indirect + github.com/moby/term v0.5.2 // indirect + github.com/opencontainers/go-digest v1.0.0 // indirect + github.com/opencontainers/image-spec v1.1.1 // indirect + github.com/pmezard/go-difflib v1.0.0 // indirect + github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect + github.com/shirou/gopsutil/v4 v4.26.5 // indirect + github.com/sirupsen/logrus v1.9.4 // indirect + github.com/stretchr/testify v1.11.1 // indirect + github.com/testcontainers/testcontainers-go v0.43.0 // indirect + github.com/tklauser/go-sysconf v0.3.16 // indirect + github.com/tklauser/numcpus v0.11.0 // indirect + github.com/xdg-go/pbkdf2 v1.0.0 // indirect + github.com/xdg-go/scram v1.2.0 // indirect + github.com/xdg-go/stringprep v1.0.4 // indirect + github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect + github.com/yusufpapurcu/wmi v1.2.4 // indirect + go.opentelemetry.io/auto/sdk v1.2.1 // indirect + go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 // indirect + go.opentelemetry.io/otel v1.41.0 // indirect + go.opentelemetry.io/otel/metric v1.41.0 // indirect + go.opentelemetry.io/otel/trace v1.41.0 // indirect + golang.org/x/crypto v0.51.0 // indirect + golang.org/x/sync v0.20.0 // indirect + golang.org/x/sys v0.45.0 // indirect + golang.org/x/text v0.37.0 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect ) diff --git a/go.sum b/go.sum index 6479ead..7647d94 100644 --- a/go.sum +++ b/go.sum @@ -1,14 +1,175 @@ -github.com/alicebob/miniredis/v2 v2.38.0 h1:nZAzCR+Lj+Vxk4ZXzm2NuKq2O33RXj1XxJ2e2uP9jiw= -github.com/alicebob/miniredis/v2 v2.38.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM= -github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs= -github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c= -github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA= -github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0= +dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8= +dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA= +github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk= +github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8= +github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg= +github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= +github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= +github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= +github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8= +github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78= -github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc= -github.com/redis/go-redis/v9 v9.17.0 h1:K6E+ZlYN95KSMmZeEQPbU/c++wfmEvfFB17yEAq/VhM= -github.com/redis/go-redis/v9 v9.17.0/go.mod h1:u410H11HMLoB+TP67dz8rL9s6QW2j76l0//kSOd3370= -github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M= -github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw= +github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI= +github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M= +github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE= +github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk= +github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I= +github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo= +github.com/containerd/platforms v0.2.1 h1:zvwtM3rz2YHPQsF2CHYM8+KtB5dvhISiXh5ZpSBQv6A= +github.com/containerd/platforms v0.2.1/go.mod h1:XHCb+2/hzowdiut9rkudds9bE5yJ7npe7dG/wG+uFPw= +github.com/cpuguy83/dockercfg v0.3.2 h1:DlJTyZGBDlXqUZ2Dk2Q3xHs/FtnooJJVaad2S9GKorA= +github.com/cpuguy83/dockercfg v0.3.2/go.mod h1:sugsbF4//dDlL/i+S+rtpIWp+5h0BHJHfjj5/jFyUJc= +github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s= +github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= +github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= +github.com/docker/go-connections v0.6.0 h1:LlMG9azAe1TqfR7sO+NJttz1gy6KO7VJBh+pMmjSD94= +github.com/docker/go-connections v0.6.0/go.mod h1:AahvXYshr6JgfUJGdDCs2b5EZG/vmaMAntpSFH5BFKE= +github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= +github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= +github.com/ebitengine/purego v0.10.0 h1:QIw4xfpWT6GWTzaW5XEKy3HXoqrJGx1ijYHzTF0/ISU= +github.com/ebitengine/purego v0.10.0/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ= +github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= +github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= +github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= +github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= +github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY= +github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0= +github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE= +github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 h1:6E+4a0GO5zZEnZ81pIr0yLvtUWk2if982qA3F3QD6H4= +github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0/go.mod h1:zJYVVT2jmtg6P3p1VtQj7WsuWi/y4VnjVBn7F8KPB3I= +github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE= +github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0= +github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0= +github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo= +github.com/moby/go-archive v0.2.0 h1:zg5QDUM2mi0JIM9fdQZWC7U8+2ZfixfTYoHL7rWUcP8= +github.com/moby/go-archive v0.2.0/go.mod h1:mNeivT14o8xU+5q1YnNrkQVpK+dnNe/K6fHqnTg4qPU= +github.com/moby/moby/api v1.54.2 h1:wiat9QAhnDQjA7wk1kh/TqHz2I1uUA7M7t9SAl/JNXg= +github.com/moby/moby/api v1.54.2/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs= +github.com/moby/moby/client v0.4.0 h1:S+2XegzHQrrvTCvF6s5HFzcrywWQmuVnhOXe2kiWjIw= +github.com/moby/moby/client v0.4.0/go.mod h1:QWPbvWchQbxBNdaLSpoKpCdf5E+WxFAgNHogCWDoa7g= +github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x084U= +github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc= +github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU= +github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko= +github.com/moby/sys/user v0.4.0 h1:jhcMKit7SA80hivmFJcbB1vqmw//wU61Zdui2eQXuMs= +github.com/moby/sys/user v0.4.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs= +github.com/moby/sys/userns v0.1.0 h1:tVLXkFOxVu9A64/yh59slHVv9ahO9UIev4JZusOLG/g= +github.com/moby/sys/userns v0.1.0/go.mod h1:IHUYgu/kao6N8YZlp9Cf444ySSvCmDlmzUcYfDHOl28= +github.com/moby/term v0.5.2 h1:6qk3FJAFDs6i/q3W/pQ97SX192qKfZgGjCQqfCJkgzQ= +github.com/moby/term v0.5.2/go.mod h1:d3djjFCrjnB+fl8NJux+EJzu0msscUP+f8it8hPkFLc= +github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= +github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= +github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040= +github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU= +github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE= +github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= +github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= +github.com/shirou/gopsutil/v4 v4.26.5 h1:RPcBXkpz7kOj9PqGFQOlBPZHsyaPvPVQc098y9RmCNM= +github.com/shirou/gopsutil/v4 v4.26.5/go.mod h1:LZ6ewCSkBqUpvSOf+LsTGnRinC6iaNUNMGBtDkJBaLQ= +github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w= +github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g= +github.com/stretchr/objx v0.5.3 h1:jmXUvGomnU1o3W/V5h2VEradbpJDwGrzugQQvL0POH4= +github.com/stretchr/objx v0.5.3/go.mod h1:rDQraq+vQZU7Fde9LOZLr8Tax6zZvy4kuNKF+QYS+U0= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/testcontainers/testcontainers-go v0.43.0 h1:oEQx5MW2DGd9z3AeEQfB2lPM0eLs7ztyaGRu75bFo5A= +github.com/testcontainers/testcontainers-go v0.43.0/go.mod h1:+VxkT2NQnKOZPKi6praMuMKYHYyOGXr0XSBSlSMCzFo= +github.com/testcontainers/testcontainers-go/modules/mongodb v0.43.0 h1:RJlAAz8dv61icyHvFmrr1xZHJEWZdvBfVBAU6wnmGPM= +github.com/testcontainers/testcontainers-go/modules/mongodb v0.43.0/go.mod h1:1zaDEZyV1SK+mpz83tmB2KthF6DHaS4Y+lRvy3d2XiE= +github.com/tklauser/go-sysconf v0.3.16 h1:frioLaCQSsF5Cy1jgRBrzr6t502KIIwQ0MArYICU0nA= +github.com/tklauser/go-sysconf v0.3.16/go.mod h1:/qNL9xxDhc7tx3HSRsLWNnuzbVfh3e7gh/BmM179nYI= +github.com/tklauser/numcpus v0.11.0 h1:nSTwhKH5e1dMNsCdVBukSZrURJRoHbSEQjdEbY+9RXw= +github.com/tklauser/numcpus v0.11.0/go.mod h1:z+LwcLq54uWZTX0u/bGobaV34u6V7KNlTZejzM6/3MQ= +github.com/xdg-go/pbkdf2 v1.0.0 h1:Su7DPu48wXMwC3bs7MCNG+z4FhcyEuz5dlvchbq0B0c= +github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI= +github.com/xdg-go/scram v1.2.0 h1:bYKF2AEwG5rqd1BumT4gAnvwU/M9nBp2pTSxeZw7Wvs= +github.com/xdg-go/scram v1.2.0/go.mod h1:3dlrS0iBaWKYVt2ZfA4cj48umJZ+cAEbR6/SjLA88I8= +github.com/xdg-go/stringprep v1.0.4 h1:XLI/Ng3O1Atzq0oBs3TWm+5ZVgkq2aqdlvP9JtoZ6c8= +github.com/xdg-go/stringprep v1.0.4/go.mod h1:mPGuuIYwz7CmR2bT9j4GbQqutWS1zV24gijq1dTyGkM= +github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 h1:ilQV1hzziu+LLM3zUTJ0trRztfwgjqKnBWNtSRkbmwM= +github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78/go.mod h1:aL8wCCfTfSfmXjznFBSZNN13rSJjlIOI1fUNAtF7rmI= +github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= +github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0= +github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0= +go.mongodb.org/mongo-driver/v2 v2.7.0 h1:RO+zqavD2/GCL3cxOMyZhx6R9Irzr8/6gsoqx5tcY/c= +go.mongodb.org/mongo-driver/v2 v2.7.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0= +go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= +go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 h1:sbiXRNDSWJOTobXh5HyQKjq6wUC5tNybqjIqDpAY4CU= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0/go.mod h1:69uWxva0WgAA/4bu2Yy70SLDBwZXuQ6PbBpbsa5iZrQ= +go.opentelemetry.io/otel v1.41.0 h1:YlEwVsGAlCvczDILpUXpIpPSL/VPugt7zHThEMLce1c= +go.opentelemetry.io/otel v1.41.0/go.mod h1:Yt4UwgEKeT05QbLwbyHXEwhnjxNO6D8L5PQP51/46dE= +go.opentelemetry.io/otel/metric v1.41.0 h1:rFnDcs4gRzBcsO9tS8LCpgR0dxg4aaxWlJxCno7JlTQ= +go.opentelemetry.io/otel/metric v1.41.0/go.mod h1:xPvCwd9pU0VN8tPZYzDZV/BMj9CM9vs00GuBjeKhJps= +go.opentelemetry.io/otel/sdk v1.35.0 h1:iPctf8iprVySXSKJffSS79eOjl9pvxV9ZqOWT0QejKY= +go.opentelemetry.io/otel/sdk v1.35.0/go.mod h1:+ga1bZliga3DxJ3CQGg3updiaAJoNECOgJREo9KHGQg= +go.opentelemetry.io/otel/sdk/metric v1.35.0 h1:1RriWBmCKgkeHEhM7a2uMjMUfP7MsOF5JpUCaEqEI9o= +go.opentelemetry.io/otel/sdk/metric v1.35.0/go.mod h1:is6XYCUMpcKi+ZsOvfluY5YstFnhW0BidkR+gL+qN+w= +go.opentelemetry.io/otel/trace v1.41.0 h1:Vbk2co6bhj8L59ZJ6/xFTskY+tGAbOnCtQGVVa9TIN0= +go.opentelemetry.io/otel/trace v1.41.0/go.mod h1:U1NU4ULCoxeDKc09yCWdWe+3QoyweJcISEVa1RBzOis= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= +golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI= +golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8= +golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= +golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4= +golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY= +golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= +golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= +golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4= +golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= +golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ= +golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc= +golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38= +golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= +golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q= +gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA= +pgregory.net/rapid v1.2.0 h1:keKAYRcjm+e1F0oAuU5F5+YPAWcyxNNRK2wud503Gnk= +pgregory.net/rapid v1.2.0/go.mod h1:PY5XlDGj0+V1FCq0o192FdRhpKHGTRIWBgqjDBTrq04= diff --git a/internal/bot/bot.go b/internal/bot/bot.go index b8526c9..20b6b8c 100644 --- a/internal/bot/bot.go +++ b/internal/bot/bot.go @@ -8,7 +8,7 @@ import ( "time" openai "github.com/tiennm99/openai-status-bot/internal/openai" - "github.com/tiennm99/openai-status-bot/internal/redisstore" + "github.com/tiennm99/openai-status-bot/internal/mongostore" "github.com/tiennm99/openai-status-bot/internal/telegram" ) @@ -23,13 +23,13 @@ type StatusClient interface { } type Store interface { - AddSubscriber(ctx context.Context, sub redisstore.Subscriber) error - GetSubscriber(ctx context.Context, sub redisstore.Subscriber) (redisstore.Subscriber, bool, error) - RemoveSubscriber(ctx context.Context, sub redisstore.Subscriber) error + AddSubscriber(ctx context.Context, sub mongostore.Subscriber) error + GetSubscriber(ctx context.Context, sub mongostore.Subscriber) (mongostore.Subscriber, bool, error) + RemoveSubscriber(ctx context.Context, sub mongostore.Subscriber) error SaveTelegramOffset(ctx context.Context, offset int64) error TelegramOffset(ctx context.Context) (int64, error) - UpdateSubscriberSettings(ctx context.Context, sub redisstore.Subscriber, types, components []string) (bool, error) - UpdateSubscriberTypes(ctx context.Context, sub redisstore.Subscriber, types []string) (bool, error) + UpdateSubscriberSettings(ctx context.Context, sub mongostore.Subscriber, types, components []string) (bool, error) + UpdateSubscriberTypes(ctx context.Context, sub mongostore.Subscriber, types []string) (bool, error) } type Bot struct { @@ -125,7 +125,7 @@ func (b *Bot) handleMessage(ctx context.Context, message telegram.Message) { } func (b *Bot) subscribe(ctx context.Context, message telegram.Message) { - sub := redisstore.NewSubscriber(message.Chat.ID, message.MessageThreadID) + sub := mongostore.NewSubscriber(message.Chat.ID, message.MessageThreadID) if err := b.store.AddSubscriber(ctx, sub); err != nil { b.logger.Error("subscribe", "error", err) b.reply(ctx, message, "Could not subscribe right now.") @@ -135,7 +135,7 @@ func (b *Bot) subscribe(ctx context.Context, message telegram.Message) { } func (b *Bot) unsubscribe(ctx context.Context, message telegram.Message) { - sub := redisstore.NewSubscriber(message.Chat.ID, message.MessageThreadID) + sub := mongostore.NewSubscriber(message.Chat.ID, message.MessageThreadID) if err := b.store.RemoveSubscriber(ctx, sub); err != nil { b.logger.Error("unsubscribe", "error", err) b.reply(ctx, message, "Could not unsubscribe right now.") diff --git a/internal/bot/bot_test.go b/internal/bot/bot_test.go index 2fad3b9..61dd9c3 100644 --- a/internal/bot/bot_test.go +++ b/internal/bot/bot_test.go @@ -9,7 +9,7 @@ import ( "time" openai "github.com/tiennm99/openai-status-bot/internal/openai" - "github.com/tiennm99/openai-status-bot/internal/redisstore" + "github.com/tiennm99/openai-status-bot/internal/mongostore" "github.com/tiennm99/openai-status-bot/internal/telegram" ) @@ -44,23 +44,23 @@ func (f fakeBotStatusClient) FetchIncidents(context.Context) (openai.IncidentsRe } type fakeBotStore struct { - sub redisstore.Subscriber + sub mongostore.Subscriber subscribed bool types []string components []string offsetErr error } -func (f *fakeBotStore) AddSubscriber(_ context.Context, sub redisstore.Subscriber) error { +func (f *fakeBotStore) AddSubscriber(_ context.Context, sub mongostore.Subscriber) error { f.sub = sub f.subscribed = true - f.types = redisstore.DefaultSubscriptionTypes() + f.types = mongostore.DefaultSubscriptionTypes() return nil } -func (f *fakeBotStore) GetSubscriber(context.Context, redisstore.Subscriber) (redisstore.Subscriber, bool, error) { +func (f *fakeBotStore) GetSubscriber(context.Context, mongostore.Subscriber) (mongostore.Subscriber, bool, error) { if !f.subscribed { - return redisstore.Subscriber{}, false, nil + return mongostore.Subscriber{}, false, nil } sub := f.sub sub.Types = append([]string{}, f.types...) @@ -68,7 +68,7 @@ func (f *fakeBotStore) GetSubscriber(context.Context, redisstore.Subscriber) (re return sub, true, nil } -func (f *fakeBotStore) RemoveSubscriber(context.Context, redisstore.Subscriber) error { +func (f *fakeBotStore) RemoveSubscriber(context.Context, mongostore.Subscriber) error { f.subscribed = false return nil } @@ -76,7 +76,7 @@ func (f *fakeBotStore) RemoveSubscriber(context.Context, redisstore.Subscriber) func (f *fakeBotStore) SaveTelegramOffset(context.Context, int64) error { return nil } func (f *fakeBotStore) TelegramOffset(context.Context) (int64, error) { return 0, f.offsetErr } -func (f *fakeBotStore) UpdateSubscriberTypes(_ context.Context, _ redisstore.Subscriber, types []string) (bool, error) { +func (f *fakeBotStore) UpdateSubscriberTypes(_ context.Context, _ mongostore.Subscriber, types []string) (bool, error) { if !f.subscribed { return false, nil } @@ -84,7 +84,7 @@ func (f *fakeBotStore) UpdateSubscriberTypes(_ context.Context, _ redisstore.Sub return true, nil } -func (f *fakeBotStore) UpdateSubscriberSettings(_ context.Context, _ redisstore.Subscriber, types, components []string) (bool, error) { +func (f *fakeBotStore) UpdateSubscriberSettings(_ context.Context, _ mongostore.Subscriber, types, components []string) (bool, error) { if !f.subscribed { return false, nil } @@ -96,15 +96,15 @@ func (f *fakeBotStore) UpdateSubscriberSettings(_ context.Context, _ redisstore. func TestSubscribeComponentStoresComponentID(t *testing.T) { bot, store, tg := newTestBot([]openai.Component{{ID: "c-api", Name: "API", Status: "operational"}}) store.subscribed = true - store.sub = redisstore.NewSubscriber(123, nil) - store.types = []string{redisstore.SubscriptionTypeIncident} + store.sub = mongostore.NewSubscriber(123, nil) + store.types = []string{mongostore.SubscriptionTypeIncident} bot.handleMessage(context.Background(), messageText("/subscribe component api")) if len(store.components) != 1 || store.components[0] != "c-api" { t.Fatalf("components = %v, want [c-api]", store.components) } - if !containsComponent(store.types, redisstore.SubscriptionTypeComponent) { + if !containsComponent(store.types, mongostore.SubscriptionTypeComponent) { t.Fatalf("types = %v, want component enabled", store.types) } if len(tg.replies) != 1 || !strings.Contains(tg.replies[0], "Subscribed to component") { @@ -115,8 +115,8 @@ func TestSubscribeComponentStoresComponentID(t *testing.T) { func TestSubscribeComponentAllEnablesComponentType(t *testing.T) { bot, store, _ := newTestBot(nil) store.subscribed = true - store.sub = redisstore.NewSubscriber(123, nil) - store.types = []string{redisstore.SubscriptionTypeIncident} + store.sub = mongostore.NewSubscriber(123, nil) + store.types = []string{mongostore.SubscriptionTypeIncident} store.components = []string{"c-api"} bot.handleMessage(context.Background(), messageText("/subscribe component all")) @@ -124,7 +124,7 @@ func TestSubscribeComponentAllEnablesComponentType(t *testing.T) { if len(store.components) != 0 { t.Fatalf("components = %v, want cleared", store.components) } - if !containsComponent(store.types, redisstore.SubscriptionTypeComponent) { + if !containsComponent(store.types, mongostore.SubscriptionTypeComponent) { t.Fatalf("types = %v, want component enabled", store.types) } } diff --git a/internal/bot/commands.go b/internal/bot/commands.go index ca6bcfe..a9a3ca0 100644 --- a/internal/bot/commands.go +++ b/internal/bot/commands.go @@ -6,12 +6,12 @@ import ( "strings" "github.com/tiennm99/openai-status-bot/internal/poller" - "github.com/tiennm99/openai-status-bot/internal/redisstore" + "github.com/tiennm99/openai-status-bot/internal/mongostore" "github.com/tiennm99/openai-status-bot/internal/telegram" ) func (b *Bot) replySubscribe(ctx context.Context, message telegram.Message, fields []string) { - sub := redisstore.NewSubscriber(message.Chat.ID, message.MessageThreadID) + sub := mongostore.NewSubscriber(message.Chat.ID, message.MessageThreadID) current, subscribed, err := b.store.GetSubscriber(ctx, sub) if err != nil { b.logger.Error("get subscription", "error", err) @@ -25,19 +25,19 @@ func (b *Bot) replySubscribe(ctx context.Context, message telegram.Message, fiel args := fields[1:] arg := strings.ToLower(args[0]) - if arg == redisstore.SubscriptionTypeComponent && len(args) > 1 { + if arg == mongostore.SubscriptionTypeComponent && len(args) > 1 { b.updateComponentFilter(ctx, message, sub, strings.Join(args[1:], " "), subscribed) return } var types []string switch arg { - case redisstore.SubscriptionTypeIncident: - types = []string{redisstore.SubscriptionTypeIncident} - case redisstore.SubscriptionTypeComponent: - types = []string{redisstore.SubscriptionTypeComponent} + case mongostore.SubscriptionTypeIncident: + types = []string{mongostore.SubscriptionTypeIncident} + case mongostore.SubscriptionTypeComponent: + types = []string{mongostore.SubscriptionTypeComponent} case "all": - types = redisstore.DefaultSubscriptionTypes() + types = mongostore.DefaultSubscriptionTypes() default: b.reply(ctx, message, formatSubscribeUsage(currentOrTarget(current, sub, subscribed), subscribed)) return @@ -56,7 +56,7 @@ func (b *Bot) replySubscribe(ctx context.Context, message telegram.Message, fiel b.reply(ctx, message, fmt.Sprintf("Subscription updated: %s", escape(strings.Join(types, ", ")))) } -func (b *Bot) updateComponentFilter(ctx context.Context, message telegram.Message, sub redisstore.Subscriber, componentArg string, subscribed bool) { +func (b *Bot) updateComponentFilter(ctx context.Context, message telegram.Message, sub mongostore.Subscriber, componentArg string, subscribed bool) { if !subscribed { b.reply(ctx, message, "Not subscribed yet. Use /start first.") return @@ -73,7 +73,7 @@ func (b *Bot) updateComponentFilter(ctx context.Context, message telegram.Messag b.reply(ctx, message, "Not subscribed yet. Use /start first.") return } - updated, err := b.store.UpdateSubscriberSettings(ctx, sub, withSubscriptionType(current.Types, redisstore.SubscriptionTypeComponent), nil) + updated, err := b.store.UpdateSubscriberSettings(ctx, sub, withSubscriptionType(current.Types, mongostore.SubscriptionTypeComponent), nil) if err != nil { b.logger.Error("clear component filter", "error", err) b.reply(ctx, message, "Could not update component filter right now.") @@ -118,7 +118,7 @@ func (b *Bot) updateComponentFilter(ctx context.Context, message telegram.Messag if !containsComponent(components, component.ID) { components = append(components, component.ID) } - types := withSubscriptionType(current.Types, redisstore.SubscriptionTypeComponent) + types := withSubscriptionType(current.Types, mongostore.SubscriptionTypeComponent) updated, err := b.store.UpdateSubscriberSettings(ctx, sub, types, components) if err != nil { b.logger.Error("update component filter", "error", err) @@ -161,7 +161,7 @@ func (b *Bot) replyUptime(ctx context.Context, message telegram.Message) { } func (b *Bot) replyInfo(ctx context.Context, message telegram.Message) { - sub := redisstore.NewSubscriber(message.Chat.ID, message.MessageThreadID) + sub := mongostore.NewSubscriber(message.Chat.ID, message.MessageThreadID) current, subscribed, err := b.store.GetSubscriber(ctx, sub) if err != nil { b.logger.Error("get subscription info", "error", err) diff --git a/internal/bot/helpers.go b/internal/bot/helpers.go index 68699a9..060201a 100644 --- a/internal/bot/helpers.go +++ b/internal/bot/helpers.go @@ -6,7 +6,7 @@ import ( "strings" "github.com/tiennm99/openai-status-bot/internal/poller" - "github.com/tiennm99/openai-status-bot/internal/redisstore" + "github.com/tiennm99/openai-status-bot/internal/mongostore" "github.com/tiennm99/openai-status-bot/internal/telegram" ) @@ -51,7 +51,7 @@ func (b *Bot) reply(ctx context.Context, message telegram.Message, text string) } } -func currentOrTarget(current, target redisstore.Subscriber, subscribed bool) redisstore.Subscriber { +func currentOrTarget(current, target mongostore.Subscriber, subscribed bool) mongostore.Subscriber { if subscribed { return current } diff --git a/internal/bot/subscription_format.go b/internal/bot/subscription_format.go index dc34187..aee117b 100644 --- a/internal/bot/subscription_format.go +++ b/internal/bot/subscription_format.go @@ -5,10 +5,10 @@ import ( "strconv" "strings" - "github.com/tiennm99/openai-status-bot/internal/redisstore" + "github.com/tiennm99/openai-status-bot/internal/mongostore" ) -func formatInfo(sub redisstore.Subscriber, subscribed bool) string { +func formatInfo(sub mongostore.Subscriber, subscribed bool) string { chatID := strconv.FormatInt(sub.ChatID, 10) if sub.ThreadID != nil { chatID = fmt.Sprintf("%s:%d", chatID, *sub.ThreadID) @@ -28,7 +28,7 @@ func formatInfo(sub redisstore.Subscriber, subscribed bool) string { ) } -func formatSubscribeUsage(sub redisstore.Subscriber, subscribed bool) string { +func formatSubscribeUsage(sub mongostore.Subscriber, subscribed bool) string { current := "none (use /start first)" components := "all" if subscribed { diff --git a/internal/config/config.go b/internal/config/config.go index d530afc..a394c45 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -6,25 +6,23 @@ import ( "os" "strings" "time" - - "github.com/redis/go-redis/v9" ) type Config struct { TelegramBotToken string - RedisOptions *redis.Options + MongoURI string + MongoDatabase string PollInterval time.Duration HTTPTimeout time.Duration LogLevel slog.Level } const ( - minPollInterval = 5 * time.Second - maxPollInterval = time.Hour - minHTTPTimeout = time.Second - maxHTTPTimeout = 5 * time.Minute - minRedisDB = 0 - maxRedisDB = 15 + minPollInterval = 5 * time.Second + maxPollInterval = time.Hour + minHTTPTimeout = time.Second + maxHTTPTimeout = 5 * time.Minute + defaultMongoDatabase = "openai_status_bot" ) func LoadFromEnv() (Config, error) { @@ -34,12 +32,13 @@ func LoadFromEnv() (Config, error) { if cfg.TelegramBotToken == "" { return Config{}, fmt.Errorf("TELEGRAM_BOT_TOKEN is required") } - var err error - cfg.RedisOptions, err = parseRedisURL("REDIS_URL", getEnv("REDIS_URL", "redis://localhost:6379/0")) - if err != nil { - return Config{}, err + cfg.MongoURI = strings.TrimSpace(os.Getenv("MONGODB_URI")) + if cfg.MongoURI == "" { + return Config{}, fmt.Errorf("MONGODB_URI is required") } + cfg.MongoDatabase = getEnv("MONGODB_DATABASE", defaultMongoDatabase) + var err error cfg.PollInterval, err = parseDurationEnv("POLL_INTERVAL", time.Minute, minPollInterval, maxPollInterval) if err != nil { return Config{}, err @@ -78,17 +77,6 @@ func parseDurationEnv(key string, fallback, minValue, maxValue time.Duration) (t return parsed, nil } -func parseRedisURL(key, value string) (*redis.Options, error) { - options, err := redis.ParseURL(value) - if err != nil { - return nil, fmt.Errorf("%s must be a valid Redis URL", key) - } - if options.DB < minRedisDB || options.DB > maxRedisDB { - return nil, fmt.Errorf("%s database must be between %d and %d", key, minRedisDB, maxRedisDB) - } - return options, nil -} - func parseLogLevel(value string) (slog.Level, error) { switch strings.ToLower(strings.TrimSpace(value)) { case "debug": diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 7f4bae3..0b020bb 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -10,6 +10,7 @@ import ( func TestLoadFromEnvDefaultsAndTrimming(t *testing.T) { setMinimalEnv(t) t.Setenv("TELEGRAM_BOT_TOKEN", " token ") + t.Setenv("MONGODB_URI", " mongodb+srv://user:pass@cluster.example.net/ ") cfg, err := LoadFromEnv() if err != nil { @@ -18,39 +19,37 @@ func TestLoadFromEnvDefaultsAndTrimming(t *testing.T) { if cfg.TelegramBotToken != "token" { t.Fatalf("TelegramBotToken = %q", cfg.TelegramBotToken) } - if cfg.RedisOptions == nil || cfg.RedisOptions.Addr != "localhost:6379" || cfg.RedisOptions.DB != 0 { - t.Fatalf("unexpected Redis options: %+v", cfg.RedisOptions) + if cfg.MongoURI != "mongodb+srv://user:pass@cluster.example.net/" { + t.Fatalf("MongoURI = %q", cfg.MongoURI) + } + if cfg.MongoDatabase != defaultMongoDatabase { + t.Fatalf("MongoDatabase = %q, want %q", cfg.MongoDatabase, defaultMongoDatabase) } if cfg.PollInterval != time.Minute || cfg.HTTPTimeout != 10*time.Second { t.Fatalf("unexpected defaults: %+v", cfg) } } -func TestLoadFromEnvIgnoresLegacyRedisVariables(t *testing.T) { +func TestLoadFromEnvAcceptsCustomMongoDatabase(t *testing.T) { setMinimalEnv(t) - t.Setenv("REDIS_ADDR", "legacy-host:6380") - t.Setenv("REDIS_PASSWORD", "legacy-password") - t.Setenv("REDIS_DB", "5") + t.Setenv("MONGODB_DATABASE", " openai_status_bot_dev ") cfg, err := LoadFromEnv() if err != nil { t.Fatalf("LoadFromEnv returned error: %v", err) } - if cfg.RedisOptions == nil || cfg.RedisOptions.Addr != "localhost:6379" || cfg.RedisOptions.Password != "" || cfg.RedisOptions.DB != 0 { - t.Fatalf("legacy Redis variables affected options: %+v", cfg.RedisOptions) + if cfg.MongoDatabase != "openai_status_bot_dev" { + t.Fatalf("MongoDatabase = %q, want openai_status_bot_dev", cfg.MongoDatabase) } } -func TestLoadFromEnvAcceptsRedisURLCredentials(t *testing.T) { +func TestLoadFromEnvRequiresMongoURI(t *testing.T) { setMinimalEnv(t) - t.Setenv("REDIS_URL", " redis://user:redis%40pass%3Awith%2Freserved@localhost:6380/2 ") + t.Setenv("MONGODB_URI", "") - cfg, err := LoadFromEnv() - if err != nil { - t.Fatalf("LoadFromEnv returned error: %v", err) - } - if cfg.RedisOptions == nil || cfg.RedisOptions.Addr != "localhost:6380" || cfg.RedisOptions.Username != "user" || cfg.RedisOptions.Password != "redis@pass:with/reserved" || cfg.RedisOptions.DB != 2 { - t.Fatalf("unexpected Redis options: %+v", cfg.RedisOptions) + _, err := LoadFromEnv() + if err == nil || !strings.Contains(err.Error(), "MONGODB_URI is required") { + t.Fatalf("error = %v, want MONGODB_URI required", err) } } @@ -63,22 +62,6 @@ func TestLoadFromEnvIgnoresOpenAIStatusBaseURL(t *testing.T) { } } -func TestLoadFromEnvDoesNotLeakRedisURLCredentialsInParseErrors(t *testing.T) { - setMinimalEnv(t) - t.Setenv("REDIS_URL", "redis://:secret%zz@localhost:6379/0") - - _, err := LoadFromEnv() - if err == nil { - t.Fatal("expected error") - } - if strings.Contains(err.Error(), "secret") || strings.Contains(err.Error(), "secret%zz") { - t.Fatalf("error leaked Redis URL credentials: %q", err.Error()) - } - if !strings.Contains(err.Error(), "REDIS_URL must be a valid Redis URL") { - t.Fatalf("error = %q, want generic Redis URL error", err.Error()) - } -} - func TestLoadFromEnvAcceptsWarningLogAlias(t *testing.T) { setMinimalEnv(t) t.Setenv("LOG_LEVEL", "warning") @@ -99,10 +82,6 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) { value string wantErr string }{ - {name: "redis url missing scheme", key: "REDIS_URL", value: "localhost:6379", wantErr: "REDIS_URL must be a valid Redis URL"}, - {name: "redis url invalid db", key: "REDIS_URL", value: "redis://localhost:6379/not-a-number", wantErr: "REDIS_URL must be a valid Redis URL"}, - {name: "redis url db below range", key: "REDIS_URL", value: "redis://localhost:6379/-1", wantErr: "REDIS_URL database must be between 0 and 15"}, - {name: "redis url db above range", key: "REDIS_URL", value: "redis://localhost:6379/16", wantErr: "REDIS_URL database must be between 0 and 15"}, {name: "poll interval too small", key: "POLL_INTERVAL", value: "1ns", wantErr: "POLL_INTERVAL must be between 5s and 1h0m0s"}, {name: "http timeout too small", key: "HTTP_TIMEOUT", value: "500ms", wantErr: "HTTP_TIMEOUT must be between 1s and 5m0s"}, {name: "invalid log level", key: "LOG_LEVEL", value: "verbose", wantErr: "LOG_LEVEL must be one of"}, @@ -127,7 +106,8 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) { func setMinimalEnv(t *testing.T) { t.Helper() t.Setenv("TELEGRAM_BOT_TOKEN", "token") - t.Setenv("REDIS_URL", "") + t.Setenv("MONGODB_URI", "mongodb+srv://user:pass@cluster.example.net/") + t.Setenv("MONGODB_DATABASE", "") t.Setenv("POLL_INTERVAL", "") t.Setenv("HTTP_TIMEOUT", "") t.Setenv("LOG_LEVEL", "") diff --git a/internal/mongostore/checkpoint.go b/internal/mongostore/checkpoint.go new file mode 100644 index 0000000..a12b617 --- /dev/null +++ b/internal/mongostore/checkpoint.go @@ -0,0 +1,185 @@ +package mongostore + +import ( + "context" + "errors" + "time" + + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" +) + +// deliveryTTL is the retention window for per-event delivery markers; enforced +// by the TTL index in EnsureIndexes. +const deliveryTTL = 7 * 24 * time.Hour + +type PendingComponentEvent struct { + ComponentID string `bson:"_id"` + ComponentName string `bson:"componentName"` + Status string `bson:"status"` + UpdatedAt string `bson:"updatedAt"` + Position int `bson:"position"` + PreviousStatus string `bson:"previousStatus"` + DeliveryKey string `bson:"deliveryKey"` +} + +func (s *Store) IsInitialized(ctx context.Context) (bool, error) { + err := s.meta.FindOne(ctx, bson.M{"_id": metaInitializedID}).Err() + if errors.Is(err, mongo.ErrNoDocuments) { + return false, nil + } + if err != nil { + return false, err + } + return true, nil +} + +func (s *Store) SetInitialized(ctx context.Context) error { + _, err := s.meta.UpdateOne(ctx, bson.M{"_id": metaInitializedID}, + bson.M{"$set": bson.M{"value": true}}, options.UpdateOne().SetUpsert(true)) + return err +} + +func (s *Store) ComponentStatuses(ctx context.Context) (map[string]string, error) { + cursor, err := s.componentStatuses.Find(ctx, bson.M{}) + if err != nil { + return nil, err + } + defer cursor.Close(ctx) + + statuses := map[string]string{} + for cursor.Next(ctx) { + var doc struct { + ID string `bson:"_id"` + Status string `bson:"status"` + } + if err := cursor.Decode(&doc); err != nil { + return nil, err + } + statuses[doc.ID] = doc.Status + } + if err := cursor.Err(); err != nil { + return nil, err + } + return statuses, nil +} + +func (s *Store) SaveComponentStatus(ctx context.Context, componentID, status string) error { + _, err := s.componentStatuses.UpdateOne(ctx, bson.M{"_id": componentID}, + bson.M{"$set": bson.M{"status": status}}, options.UpdateOne().SetUpsert(true)) + return err +} + +func (s *Store) PendingComponentEvents(ctx context.Context) (map[string]PendingComponentEvent, error) { + cursor, err := s.pendingComponentEvents.Find(ctx, bson.M{}) + if err != nil { + return nil, err + } + defer cursor.Close(ctx) + + events := map[string]PendingComponentEvent{} + for cursor.Next(ctx) { + var event PendingComponentEvent + if err := cursor.Decode(&event); err != nil { + return nil, err + } + events[event.ComponentID] = event + } + if err := cursor.Err(); err != nil { + return nil, err + } + return events, nil +} + +func (s *Store) SavePendingComponentEvent(ctx context.Context, event PendingComponentEvent) error { + _, err := s.pendingComponentEvents.ReplaceOne(ctx, bson.M{"_id": event.ComponentID}, event, + options.Replace().SetUpsert(true)) + return err +} + +func (s *Store) RemovePendingComponentEvent(ctx context.Context, componentID string) error { + _, err := s.pendingComponentEvents.DeleteOne(ctx, bson.M{"_id": componentID}) + return err +} + +func (s *Store) HasIncidentUpdateVersion(ctx context.Context, updateID, version string) (bool, error) { + var doc struct { + Version string `bson:"version"` + } + err := s.incidentUpdateVersions.FindOne(ctx, bson.M{"_id": updateID}).Decode(&doc) + if errors.Is(err, mongo.ErrNoDocuments) { + return false, nil + } + if err != nil { + return false, err + } + return doc.Version == version, nil +} + +func (s *Store) MarkIncidentUpdateVersion(ctx context.Context, updateID, version string) error { + _, err := s.incidentUpdateVersions.UpdateOne(ctx, bson.M{"_id": updateID}, + bson.M{"$set": bson.M{"version": version}}, options.UpdateOne().SetUpsert(true)) + return err +} + +// DeliveredSubscribers returns the subscriber keys already notified for +// eventKey. One Find replaces a per-subscriber membership check. +func (s *Store) DeliveredSubscribers(ctx context.Context, eventKey string) (map[string]bool, error) { + cursor, err := s.delivery.Find(ctx, bson.M{"eventKey": eventKey}) + if err != nil { + return nil, err + } + defer cursor.Close(ctx) + + delivered := map[string]bool{} + for cursor.Next(ctx) { + var doc struct { + Subscriber string `bson:"subscriber"` + } + if err := cursor.Decode(&doc); err != nil { + return nil, err + } + delivered[doc.Subscriber] = true + } + if err := cursor.Err(); err != nil { + return nil, err + } + return delivered, nil +} + +func (s *Store) MarkDelivered(ctx context.Context, eventKey, subscriberKey string) error { + id := eventKey + "|" + subscriberKey + update := bson.M{"$set": bson.M{ + "eventKey": eventKey, + "subscriber": subscriberKey, + "expiresAt": time.Now().Add(deliveryTTL), + }} + _, err := s.delivery.UpdateOne(ctx, bson.M{"_id": id}, update, options.UpdateOne().SetUpsert(true)) + return err +} + +func (s *Store) ClearDelivery(ctx context.Context, eventKey string) error { + _, err := s.delivery.DeleteMany(ctx, bson.M{"eventKey": eventKey}) + return err +} + +func (s *Store) TelegramOffset(ctx context.Context) (int64, error) { + var doc struct { + Value int64 `bson:"value"` + } + err := s.meta.FindOne(ctx, bson.M{"_id": metaTelegramOffsetID}).Decode(&doc) + if errors.Is(err, mongo.ErrNoDocuments) { + return 0, nil + } + if err != nil { + return 0, err + } + return doc.Value, nil +} + +func (s *Store) SaveTelegramOffset(ctx context.Context, offset int64) error { + _, err := s.meta.UpdateOne(ctx, bson.M{"_id": metaTelegramOffsetID}, + bson.M{"$set": bson.M{"value": offset}}, options.UpdateOne().SetUpsert(true)) + return err +} diff --git a/internal/mongostore/indexes.go b/internal/mongostore/indexes.go new file mode 100644 index 0000000..f9702b9 --- /dev/null +++ b/internal/mongostore/indexes.go @@ -0,0 +1,30 @@ +package mongostore + +import ( + "context" + + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" +) + +// deliveryTTLSeconds is the TTL index expiry for delivery markers, in seconds +// (7 days). Mongo's TTL monitor sweeps expired docs roughly once a minute, +// which is irrelevant against a 7-day window. +const deliveryTTLSeconds = int32(deliveryTTL / 1e9) + +// EnsureIndexes creates the indexes the store relies on. It is idempotent: +// re-creating an existing identical index is a no-op. +func (s *Store) EnsureIndexes(ctx context.Context) error { + _, err := s.delivery.Indexes().CreateMany(ctx, []mongo.IndexModel{ + { + Keys: bson.D{{Key: "expiresAt", Value: 1}}, + Options: options.Index().SetExpireAfterSeconds(deliveryTTLSeconds), + }, + { + // Speeds DeliveredSubscribers/ClearDelivery lookups by eventKey. + Keys: bson.D{{Key: "eventKey", Value: 1}}, + }, + }) + return err +} diff --git a/internal/mongostore/store.go b/internal/mongostore/store.go new file mode 100644 index 0000000..906b19e --- /dev/null +++ b/internal/mongostore/store.go @@ -0,0 +1,50 @@ +package mongostore + +import "go.mongodb.org/mongo-driver/v2/mongo" + +// Collection names. Each former Redis key becomes a MongoDB collection; the +// document model collapses the subscribers set + settings hash into one +// collection and replaces the per-event delivery sets with a single TTL-indexed +// collection. +const ( + subscribersCollection = "subscribers" + componentStatusesCollection = "component_statuses" + pendingComponentEventsCollection = "pending_component_events" + incidentUpdateVersionsCollection = "incident_update_versions" + deliveryCollection = "delivery" + metaCollection = "meta" +) + +// meta document IDs. +const ( + metaInitializedID = "initialized" + metaTelegramOffsetID = "telegramOffset" +) + +const ( + SubscriptionTypeIncident = "incident" + SubscriptionTypeComponent = "component" +) + +type Store struct { + db *mongo.Database + subscribers *mongo.Collection + componentStatuses *mongo.Collection + pendingComponentEvents *mongo.Collection + incidentUpdateVersions *mongo.Collection + delivery *mongo.Collection + meta *mongo.Collection +} + +func New(client *mongo.Client, dbName string) *Store { + db := client.Database(dbName) + return &Store{ + db: db, + subscribers: db.Collection(subscribersCollection), + componentStatuses: db.Collection(componentStatusesCollection), + pendingComponentEvents: db.Collection(pendingComponentEventsCollection), + incidentUpdateVersions: db.Collection(incidentUpdateVersionsCollection), + delivery: db.Collection(deliveryCollection), + meta: db.Collection(metaCollection), + } +} diff --git a/internal/mongostore/store_test.go b/internal/mongostore/store_test.go new file mode 100644 index 0000000..464ec5b --- /dev/null +++ b/internal/mongostore/store_test.go @@ -0,0 +1,358 @@ +//go:build integration + +// Package mongostore integration tests run against a real mongod via +// testcontainers and are excluded from the default `go test ./...`. Run them +// with Docker available: +// +// go test -tags=integration ./internal/mongostore/... +package mongostore + +import ( + "context" + "strings" + "sync" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" + + tcmongo "github.com/testcontainers/testcontainers-go/modules/mongodb" +) + +var ( + sharedClientOnce sync.Once + sharedClient *mongo.Client + sharedClientErr error +) + +// testMongo starts the shared mongod container once, then hands each test a +// Store bound to a unique database that is dropped on cleanup. +func testMongo(t *testing.T) *Store { + t.Helper() + ctx := context.Background() + + sharedClientOnce.Do(func() { + container, err := tcmongo.Run(ctx, "mongo:7") + if err != nil { + sharedClientErr = err + return + } + uri, err := container.ConnectionString(ctx) + if err != nil { + sharedClientErr = err + return + } + // The container is intentionally left running for the whole package + // run; testcontainers' reaper removes it after the process exits. + _ = container + sharedClient, sharedClientErr = mongo.Connect(options.Client().ApplyURI(uri)) + }) + if sharedClientErr != nil { + t.Fatalf("start mongo container: %v", sharedClientErr) + } + + dbName := "test_" + strings.NewReplacer("/", "_", " ", "_").Replace(t.Name()) + db := sharedClient.Database(dbName) + t.Cleanup(func() { + _ = db.Drop(context.Background()) + }) + + store := New(sharedClient, dbName) + if err := store.EnsureIndexes(ctx); err != nil { + t.Fatalf("EnsureIndexes: %v", err) + } + return store +} + +func TestAddAndGetSubscriber(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + sub := NewSubscriber(123, nil) + if err := store.AddSubscriber(ctx, sub); err != nil { + t.Fatalf("AddSubscriber: %v", err) + } + + got, exists, err := store.GetSubscriber(ctx, sub) + if err != nil || !exists { + t.Fatalf("GetSubscriber exists=%v err=%v", exists, err) + } + if len(got.Types) != 2 { + t.Fatalf("default Types = %v, want both defaults", got.Types) + } + if len(got.Components) != 0 { + t.Fatalf("default Components = %v, want empty", got.Components) + } +} + +func TestAddSubscriberPreservesSettingsOnReStart(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + sub := NewSubscriber(123, nil) + if err := store.AddSubscriber(ctx, sub); err != nil { + t.Fatalf("AddSubscriber: %v", err) + } + if _, err := store.UpdateSubscriberSettings(ctx, sub, []string{SubscriptionTypeComponent}, []string{"c-api"}); err != nil { + t.Fatalf("UpdateSubscriberSettings: %v", err) + } + if err := store.AddSubscriber(ctx, sub); err != nil { + t.Fatalf("re-AddSubscriber: %v", err) + } + + got, _, err := store.GetSubscriber(ctx, sub) + if err != nil { + t.Fatalf("GetSubscriber: %v", err) + } + if len(got.Types) != 1 || got.Types[0] != SubscriptionTypeComponent { + t.Fatalf("Types = %v, want preserved [component]", got.Types) + } + if len(got.Components) != 1 || got.Components[0] != "c-api" { + t.Fatalf("Components = %v, want preserved [c-api]", got.Components) + } +} + +func TestGetSubscriberMissingReturnsFalse(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + _, exists, err := store.GetSubscriber(ctx, NewSubscriber(999, nil)) + if err != nil { + t.Fatalf("GetSubscriber: %v", err) + } + if exists { + t.Fatal("exists = true, want false for missing subscriber") + } +} + +func TestListSubscribersSelfHealsMalformedKey(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + good := NewSubscriber(1, nil) + if err := store.AddSubscriber(ctx, good); err != nil { + t.Fatalf("AddSubscriber: %v", err) + } + if _, err := store.subscribers.InsertOne(ctx, bson.M{"_id": "abc:def:ghi", "types": []string{}, "components": []string{}}); err != nil { + t.Fatalf("insert malformed subscriber: %v", err) + } + + if _, err := store.ListSubscribers(ctx); err == nil { + t.Fatal("expected malformed key error on first list") + } + // Malformed doc was deleted; the next list succeeds with only the good one. + subs, err := store.ListSubscribers(ctx) + if err != nil { + t.Fatalf("second ListSubscribers: %v", err) + } + if len(subs) != 1 || subs[0].Key() != good.Key() { + t.Fatalf("subscribers = %v, want only %q", subs, good.Key()) + } +} + +func TestUpdateSubscriberSettingsMatchedVsMissing(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + sub := NewSubscriber(1, nil) + + updated, err := store.UpdateSubscriberSettings(ctx, sub, []string{SubscriptionTypeComponent}, []string{"c-api"}) + if err != nil { + t.Fatalf("UpdateSubscriberSettings: %v", err) + } + if updated { + t.Fatal("updated = true for missing subscriber, want false") + } + + if err := store.AddSubscriber(ctx, sub); err != nil { + t.Fatalf("AddSubscriber: %v", err) + } + updated, err = store.UpdateSubscriberSettings(ctx, sub, []string{SubscriptionTypeComponent}, []string{"c-api"}) + if err != nil { + t.Fatalf("UpdateSubscriberSettings: %v", err) + } + if !updated { + t.Fatal("updated = false for existing subscriber, want true") + } +} + +func TestComponentStatusRoundTrip(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + if err := store.SaveComponentStatus(ctx, "c1", "degraded_performance"); err != nil { + t.Fatalf("SaveComponentStatus: %v", err) + } + if err := store.SaveComponentStatus(ctx, "c1", "operational"); err != nil { + t.Fatalf("SaveComponentStatus update: %v", err) + } + statuses, err := store.ComponentStatuses(ctx) + if err != nil { + t.Fatalf("ComponentStatuses: %v", err) + } + if statuses["c1"] != "operational" { + t.Fatalf("status = %q, want operational", statuses["c1"]) + } +} + +func TestPendingComponentEventRoundTrip(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + event := PendingComponentEvent{ + ComponentID: "c1", + ComponentName: "API", + Status: "degraded_performance", + UpdatedAt: "2026-01-01T00:00:00Z", + Position: 2, + PreviousStatus: "operational", + DeliveryKey: "component:c1:degraded_performance:2026-01-01T00:00:00Z", + } + if err := store.SavePendingComponentEvent(ctx, event); err != nil { + t.Fatalf("SavePendingComponentEvent: %v", err) + } + events, err := store.PendingComponentEvents(ctx) + if err != nil { + t.Fatalf("PendingComponentEvents: %v", err) + } + if got := events["c1"]; got != event { + t.Fatalf("event = %+v, want %+v", got, event) + } + if err := store.RemovePendingComponentEvent(ctx, "c1"); err != nil { + t.Fatalf("RemovePendingComponentEvent: %v", err) + } + events, err = store.PendingComponentEvents(ctx) + if err != nil { + t.Fatalf("PendingComponentEvents after remove: %v", err) + } + if len(events) != 0 { + t.Fatalf("events = %v, want empty after remove", events) + } +} + +func TestIncidentUpdateVersionDedup(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + seen, err := store.HasIncidentUpdateVersion(ctx, "u1", "v1") + if err != nil { + t.Fatalf("HasIncidentUpdateVersion: %v", err) + } + if seen { + t.Fatal("unseen update reported as seen") + } + if err := store.MarkIncidentUpdateVersion(ctx, "u1", "v1"); err != nil { + t.Fatalf("MarkIncidentUpdateVersion: %v", err) + } + seen, err = store.HasIncidentUpdateVersion(ctx, "u1", "v1") + if err != nil || !seen { + t.Fatalf("same version seen=%v err=%v, want seen", seen, err) + } + // An edited update (new version) must notify again. + seen, err = store.HasIncidentUpdateVersion(ctx, "u1", "v2") + if err != nil { + t.Fatalf("HasIncidentUpdateVersion v2: %v", err) + } + if seen { + t.Fatal("edited update version reported as seen") + } +} + +func TestDeliveryDedupAndClear(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + if err := store.MarkDelivered(ctx, "evt1", "sub-a"); err != nil { + t.Fatalf("MarkDelivered: %v", err) + } + if err := store.MarkDelivered(ctx, "evt1", "sub-b"); err != nil { + t.Fatalf("MarkDelivered: %v", err) + } + delivered, err := store.DeliveredSubscribers(ctx, "evt1") + if err != nil { + t.Fatalf("DeliveredSubscribers: %v", err) + } + if !delivered["sub-a"] || !delivered["sub-b"] || len(delivered) != 2 { + t.Fatalf("delivered = %v, want sub-a and sub-b", delivered) + } + if err := store.ClearDelivery(ctx, "evt1"); err != nil { + t.Fatalf("ClearDelivery: %v", err) + } + delivered, err = store.DeliveredSubscribers(ctx, "evt1") + if err != nil { + t.Fatalf("DeliveredSubscribers after clear: %v", err) + } + if len(delivered) != 0 { + t.Fatalf("delivered = %v, want empty after clear", delivered) + } +} + +func TestDeliveryTTLIndexExists(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + cursor, err := store.delivery.Indexes().List(ctx) + if err != nil { + t.Fatalf("list indexes: %v", err) + } + var indexes []bson.M + if err := cursor.All(ctx, &indexes); err != nil { + t.Fatalf("decode indexes: %v", err) + } + found := false + for _, idx := range indexes { + if expire, ok := idx["expireAfterSeconds"]; ok { + if asInt(expire) == 604800 { + found = true + } + } + } + if !found { + t.Fatalf("no TTL index with expireAfterSeconds=604800 in %v", indexes) + } +} + +func asInt(v any) int64 { + switch n := v.(type) { + case int32: + return int64(n) + case int64: + return n + case float64: + return int64(n) + default: + return -1 + } +} + +func TestTelegramOffsetRoundTrip(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + offset, err := store.TelegramOffset(ctx) + if err != nil { + t.Fatalf("TelegramOffset: %v", err) + } + if offset != 0 { + t.Fatalf("default offset = %d, want 0", offset) + } + if err := store.SaveTelegramOffset(ctx, 42); err != nil { + t.Fatalf("SaveTelegramOffset: %v", err) + } + offset, err = store.TelegramOffset(ctx) + if err != nil { + t.Fatalf("TelegramOffset: %v", err) + } + if offset != 42 { + t.Fatalf("offset = %d, want 42", offset) + } +} + +func TestInitializedFlag(t *testing.T) { + ctx := context.Background() + store := testMongo(t) + initialized, err := store.IsInitialized(ctx) + if err != nil { + t.Fatalf("IsInitialized: %v", err) + } + if initialized { + t.Fatal("fresh store reported initialized") + } + if err := store.SetInitialized(ctx); err != nil { + t.Fatalf("SetInitialized: %v", err) + } + initialized, err = store.IsInitialized(ctx) + if err != nil || !initialized { + t.Fatalf("IsInitialized after set = %v err=%v, want true", initialized, err) + } +} diff --git a/internal/mongostore/subscriber.go b/internal/mongostore/subscriber.go new file mode 100644 index 0000000..38bbd2f --- /dev/null +++ b/internal/mongostore/subscriber.go @@ -0,0 +1,195 @@ +package mongostore + +import ( + "context" + "errors" + "fmt" + "strconv" + "strings" + + "go.mongodb.org/mongo-driver/v2/bson" + "go.mongodb.org/mongo-driver/v2/mongo" + "go.mongodb.org/mongo-driver/v2/mongo/options" +) + +type Subscriber struct { + ChatID int64 + ThreadID *int + Types []string + Components []string +} + +// subscriberDoc is the stored shape of a subscriber. Identity lives in _id (the +// subscriber key), mirroring the Redis design where the key held identity and a +// separate hash held settings; chatID/threadID are denormalized for inspection. +type subscriberDoc struct { + ID string `bson:"_id"` + ChatID int64 `bson:"chatID"` + ThreadID *int `bson:"threadID,omitempty"` + Types []string `bson:"types"` + Components []string `bson:"components"` +} + +func NewSubscriber(chatID int64, threadID *int) Subscriber { + var copiedThreadID *int + if threadID != nil { + value := *threadID + copiedThreadID = &value + } + return Subscriber{ + ChatID: chatID, + ThreadID: copiedThreadID, + Types: DefaultSubscriptionTypes(), + Components: []string{}, + } +} + +func (s Subscriber) Key() string { + if s.ThreadID == nil { + return strconv.FormatInt(s.ChatID, 10) + } + return fmt.Sprintf("%d:%d", s.ChatID, *s.ThreadID) +} + +func ParseSubscriberKey(value string) (Subscriber, error) { + parts := strings.Split(value, ":") + if len(parts) != 1 && len(parts) != 2 { + return Subscriber{}, fmt.Errorf("invalid subscriber key %q", value) + } + + chatID, err := strconv.ParseInt(parts[0], 10, 64) + if err != nil { + return Subscriber{}, fmt.Errorf("invalid chat ID: %w", err) + } + if len(parts) == 1 { + return NewSubscriber(chatID, nil), nil + } + + threadID, err := strconv.Atoi(parts[1]) + if err != nil { + return Subscriber{}, fmt.Errorf("invalid thread ID: %w", err) + } + return NewSubscriber(chatID, &threadID), nil +} + +func DefaultSubscriptionTypes() []string { + return []string{SubscriptionTypeIncident, SubscriptionTypeComponent} +} + +func (s Subscriber) Accepts(eventType, componentID, componentName string) bool { + if !containsFold(s.Types, eventType) { + return false + } + if eventType != SubscriptionTypeComponent || len(s.Components) == 0 { + return true + } + return containsFold(s.Components, componentID) || containsFold(s.Components, componentName) +} + +// AddSubscriber upserts the subscriber. An existing subscriber keeps its +// settings on re-/start (only chatID/threadID are refreshed); a new subscriber +// is seeded with default types and no component filter. +func (s *Store) AddSubscriber(ctx context.Context, sub Subscriber) error { + // Identity lives in _id; chatID/threadID are denormalized for inspection. + // Omit threadID when absent so the stored shape matches subscriberDoc's + // omitempty decode (no explicit null). + set := bson.M{"chatID": sub.ChatID} + if sub.ThreadID != nil { + set["threadID"] = *sub.ThreadID + } + update := bson.M{ + "$set": set, + "$setOnInsert": bson.M{ + "types": DefaultSubscriptionTypes(), + "components": []string{}, + }, + } + _, err := s.subscribers.UpdateOne(ctx, bson.M{"_id": sub.Key()}, update, options.UpdateOne().SetUpsert(true)) + return err +} + +func (s *Store) RemoveSubscriber(ctx context.Context, sub Subscriber) error { + _, err := s.subscribers.DeleteOne(ctx, bson.M{"_id": sub.Key()}) + return err +} + +func (s *Store) ListSubscribers(ctx context.Context) ([]Subscriber, error) { + cursor, err := s.subscribers.Find(ctx, bson.M{}) + if err != nil { + return nil, err + } + defer cursor.Close(ctx) + + var subscribers []Subscriber + for cursor.Next(ctx) { + var doc subscriberDoc + if err := cursor.Decode(&doc); err != nil { + return nil, err + } + sub, err := s.subscriberFromDoc(ctx, doc) + if err != nil { + return nil, err + } + subscribers = append(subscribers, sub) + } + if err := cursor.Err(); err != nil { + return nil, err + } + return subscribers, nil +} + +func (s *Store) GetSubscriber(ctx context.Context, sub Subscriber) (Subscriber, bool, error) { + var doc subscriberDoc + if err := s.subscribers.FindOne(ctx, bson.M{"_id": sub.Key()}).Decode(&doc); err != nil { + if errors.Is(err, mongo.ErrNoDocuments) { + return Subscriber{}, false, nil + } + return Subscriber{}, false, err + } + resolved, err := s.subscriberFromDoc(ctx, doc) + if err != nil { + return Subscriber{}, false, err + } + return resolved, true, nil +} + +func (s *Store) UpdateSubscriberTypes(ctx context.Context, sub Subscriber, types []string) (bool, error) { + return s.setSubscriberFields(ctx, sub.Key(), bson.M{"types": normalizeTypes(types)}) +} + +func (s *Store) UpdateSubscriberComponents(ctx context.Context, sub Subscriber, components []string) (bool, error) { + return s.setSubscriberFields(ctx, sub.Key(), bson.M{"components": normalizeComponents(components)}) +} + +func (s *Store) UpdateSubscriberSettings(ctx context.Context, sub Subscriber, types, components []string) (bool, error) { + return s.setSubscriberFields(ctx, sub.Key(), bson.M{ + "types": normalizeTypes(types), + "components": normalizeComponents(components), + }) +} + +// setSubscriberFields applies a $set only when the subscriber exists. +// MatchedCount==0 means "not subscribed", replacing the Redis check-then-set +// Lua script. +func (s *Store) setSubscriberFields(ctx context.Context, key string, fields bson.M) (bool, error) { + result, err := s.subscribers.UpdateOne(ctx, bson.M{"_id": key}, bson.M{"$set": fields}) + if err != nil { + return false, err + } + return result.MatchedCount > 0, nil +} + +// subscriberFromDoc derives identity from the document _id (so a malformed key +// self-heals as it did under Redis) and normalizes the stored settings. +func (s *Store) subscriberFromDoc(ctx context.Context, doc subscriberDoc) (Subscriber, error) { + sub, err := ParseSubscriberKey(doc.ID) + if err != nil { + if _, removeErr := s.subscribers.DeleteOne(ctx, bson.M{"_id": doc.ID}); removeErr != nil { + return Subscriber{}, fmt.Errorf("remove malformed subscriber key %q: %w", doc.ID, removeErr) + } + return Subscriber{}, fmt.Errorf("malformed subscriber key %q: %w", doc.ID, err) + } + sub.Types = normalizeTypes(doc.Types) + sub.Components = normalizeComponents(doc.Components) + return sub, nil +} diff --git a/internal/redisstore/subscriber_normalization.go b/internal/mongostore/subscriber_normalization.go similarity index 98% rename from internal/redisstore/subscriber_normalization.go rename to internal/mongostore/subscriber_normalization.go index 7bc7da9..20dfa86 100644 --- a/internal/redisstore/subscriber_normalization.go +++ b/internal/mongostore/subscriber_normalization.go @@ -1,4 +1,4 @@ -package redisstore +package mongostore import "strings" diff --git a/internal/mongostore/subscriber_test.go b/internal/mongostore/subscriber_test.go new file mode 100644 index 0000000..0258542 --- /dev/null +++ b/internal/mongostore/subscriber_test.go @@ -0,0 +1,66 @@ +package mongostore + +import "testing" + +func TestSubscriberKeyRoundTripWithoutThread(t *testing.T) { + sub := NewSubscriber(12345, nil) + parsed, err := ParseSubscriberKey(sub.Key()) + if err != nil { + t.Fatalf("ParseSubscriberKey returned error: %v", err) + } + if parsed.ChatID != sub.ChatID || parsed.ThreadID != nil { + t.Fatalf("parsed subscriber = %+v, want %+v", parsed, sub) + } +} + +func TestSubscriberKeyRoundTripWithThreadZero(t *testing.T) { + threadID := 0 + sub := NewSubscriber(-10012345, &threadID) + parsed, err := ParseSubscriberKey(sub.Key()) + if err != nil { + t.Fatalf("ParseSubscriberKey returned error: %v", err) + } + if parsed.ChatID != sub.ChatID || parsed.ThreadID == nil || *parsed.ThreadID != threadID { + t.Fatalf("parsed subscriber = %+v, want thread ID %d", parsed, threadID) + } +} + +func TestParseSubscriberKeyRejectsInvalidValue(t *testing.T) { + if _, err := ParseSubscriberKey("abc:def:ghi"); err == nil { + t.Fatal("expected invalid key error") + } +} + +func TestSubscriberAcceptsComponentIDAndLegacyNameFilters(t *testing.T) { + sub := NewSubscriber(12345, nil) + sub.Components = []string{"component-id", "Legacy API Name"} + if !sub.Accepts(SubscriptionTypeComponent, "component-id", "API") { + t.Fatal("expected component ID filter to match") + } + if !sub.Accepts(SubscriptionTypeComponent, "other-id", "legacy api name") { + t.Fatal("expected legacy component name filter to match") + } + if sub.Accepts(SubscriptionTypeComponent, "other-id", "Other") { + t.Fatal("unexpected component filter match") + } +} + +func TestNormalizeTypesDedupesAndDefaults(t *testing.T) { + got := normalizeTypes([]string{"incident", "component", "incident"}) + if len(got) != 2 || got[0] != SubscriptionTypeIncident || got[1] != SubscriptionTypeComponent { + t.Fatalf("normalizeTypes = %v", got) + } + if def := normalizeTypes(nil); len(def) != 2 { + t.Fatalf("normalizeTypes(nil) = %v, want defaults", def) + } + if def := normalizeTypes([]string{"bogus"}); len(def) != 2 { + t.Fatalf("normalizeTypes(bogus) = %v, want defaults", def) + } +} + +func TestNormalizeComponentsTrimsAndDedupes(t *testing.T) { + got := normalizeComponents([]string{" api ", "API", ""}) + if len(got) != 1 || got[0] != "api" { + t.Fatalf("normalizeComponents = %v, want [api]", got) + } +} diff --git a/internal/poller/delivery.go b/internal/poller/delivery.go index 43d774e..1beb98e 100644 --- a/internal/poller/delivery.go +++ b/internal/poller/delivery.go @@ -4,7 +4,7 @@ import ( "context" "fmt" - "github.com/tiennm99/openai-status-bot/internal/redisstore" + "github.com/tiennm99/openai-status-bot/internal/mongostore" "github.com/tiennm99/openai-status-bot/internal/telegram" ) @@ -44,7 +44,7 @@ func (e *deliveryError) addAll(other *deliveryError) { e.count += other.count } -func (r *Runner) notifySubscribers(ctx context.Context, event notificationEvent, subscribers []redisstore.Subscriber, removed, failed map[string]bool) (*deliveryError, error) { +func (r *Runner) notifySubscribers(ctx context.Context, event notificationEvent, subscribers []mongostore.Subscriber, removed, failed map[string]bool) (*deliveryError, error) { deliveryFailures := &deliveryError{} var delivered map[string]bool if event.deliveryKey != "" { diff --git a/internal/poller/event_collection.go b/internal/poller/event_collection.go index 9420f3e..40cf21b 100644 --- a/internal/poller/event_collection.go +++ b/internal/poller/event_collection.go @@ -8,7 +8,7 @@ import ( "strings" openai "github.com/tiennm99/openai-status-bot/internal/openai" - "github.com/tiennm99/openai-status-bot/internal/redisstore" + "github.com/tiennm99/openai-status-bot/internal/mongostore" ) func (r *Runner) collectEvents(ctx context.Context, summary openai.Summary, incidents openai.IncidentsResponse, initialized bool) ([]notificationEvent, []checkpoint, []checkpoint, error) { @@ -55,7 +55,7 @@ func (r *Runner) collectComponentEvents(ctx context.Context, summary openai.Summ pendingEvent := pendingEvent component := pendingComponent(pendingEvent) events = append(events, notificationEvent{ - eventType: redisstore.SubscriptionTypeComponent, + eventType: mongostore.SubscriptionTypeComponent, componentID: pendingEvent.ComponentID, componentName: pendingEvent.ComponentName, deliveryKey: pendingEvent.DeliveryKey, @@ -92,7 +92,7 @@ func (r *Runner) collectComponentEvents(ctx context.Context, summary openai.Summ } deliveryKey := fmt.Sprintf("component:%s:%s:%s", component.ID, component.Status, component.UpdatedAt) - pendingEvent := redisstore.PendingComponentEvent{ + pendingEvent := mongostore.PendingComponentEvent{ ComponentID: component.ID, ComponentName: component.Name, Status: component.Status, @@ -105,7 +105,7 @@ func (r *Runner) collectComponentEvents(ctx context.Context, summary openai.Summ return r.store.SavePendingComponentEvent(ctx, pendingEvent) }) events = append(events, notificationEvent{ - eventType: redisstore.SubscriptionTypeComponent, + eventType: mongostore.SubscriptionTypeComponent, componentID: component.ID, componentName: component.Name, deliveryKey: deliveryKey, @@ -151,7 +151,7 @@ func (r *Runner) collectIncidentEvents(ctx context.Context, response openai.Inci deliveryKey := fmt.Sprintf("incident:%s:%s", update.ID, version) if initialized && !seen { events = append(events, notificationEvent{ - eventType: redisstore.SubscriptionTypeIncident, + eventType: mongostore.SubscriptionTypeIncident, deliveryKey: deliveryKey, sortTime: incidentUpdateSortTime(update), text: FormatIncidentUpdate(incident, update), @@ -165,7 +165,7 @@ func (r *Runner) collectIncidentEvents(ctx context.Context, response openai.Inci return events, checkpoints, nil } -func pendingComponent(event redisstore.PendingComponentEvent) openai.Component { +func pendingComponent(event mongostore.PendingComponentEvent) openai.Component { return openai.Component{ ID: event.ComponentID, Name: event.ComponentName, @@ -175,7 +175,7 @@ func pendingComponent(event redisstore.PendingComponentEvent) openai.Component { } } -func componentsForDuplicateLabels(components []openai.Component, pending map[string]redisstore.PendingComponentEvent) []openai.Component { +func componentsForDuplicateLabels(components []openai.Component, pending map[string]mongostore.PendingComponentEvent) []openai.Component { result := make([]openai.Component, 0, len(components)+len(pending)) seen := map[string]bool{} add := func(component openai.Component) { diff --git a/internal/poller/poller.go b/internal/poller/poller.go index 31b1888..cbed597 100644 --- a/internal/poller/poller.go +++ b/internal/poller/poller.go @@ -6,7 +6,7 @@ import ( "time" openai "github.com/tiennm99/openai-status-bot/internal/openai" - "github.com/tiennm99/openai-status-bot/internal/redisstore" + "github.com/tiennm99/openai-status-bot/internal/mongostore" ) type StatusClient interface { @@ -20,19 +20,19 @@ type Store interface { DeliveredSubscribers(ctx context.Context, eventKey string) (map[string]bool, error) HasIncidentUpdateVersion(ctx context.Context, updateID, version string) (bool, error) IsInitialized(ctx context.Context) (bool, error) - ListSubscribers(ctx context.Context) ([]redisstore.Subscriber, error) + ListSubscribers(ctx context.Context) ([]mongostore.Subscriber, error) MarkDelivered(ctx context.Context, eventKey, subscriberKey string) error MarkIncidentUpdateVersion(ctx context.Context, updateID, version string) error - PendingComponentEvents(ctx context.Context) (map[string]redisstore.PendingComponentEvent, error) + PendingComponentEvents(ctx context.Context) (map[string]mongostore.PendingComponentEvent, error) RemovePendingComponentEvent(ctx context.Context, componentID string) error - RemoveSubscriber(ctx context.Context, sub redisstore.Subscriber) error - SavePendingComponentEvent(ctx context.Context, event redisstore.PendingComponentEvent) error + RemoveSubscriber(ctx context.Context, sub mongostore.Subscriber) error + SavePendingComponentEvent(ctx context.Context, event mongostore.PendingComponentEvent) error SaveComponentStatus(ctx context.Context, componentID, status string) error SetInitialized(ctx context.Context) error } type Notifier interface { - SendMessage(ctx context.Context, sub redisstore.Subscriber, text string) error + SendMessage(ctx context.Context, sub mongostore.Subscriber, text string) error } type Runner struct { diff --git a/internal/poller/poller_test.go b/internal/poller/poller_test.go index 52957bb..e14ac29 100644 --- a/internal/poller/poller_test.go +++ b/internal/poller/poller_test.go @@ -9,7 +9,7 @@ import ( "time" openai "github.com/tiennm99/openai-status-bot/internal/openai" - "github.com/tiennm99/openai-status-bot/internal/redisstore" + "github.com/tiennm99/openai-status-bot/internal/mongostore" "github.com/tiennm99/openai-status-bot/internal/telegram" ) @@ -30,14 +30,14 @@ type fakePollerStore struct { initialized bool componentStatuses map[string]string incidentVersions map[string]string - subscribers []redisstore.Subscriber + subscribers []mongostore.Subscriber delivered map[string]map[string]bool hasDeliveredErr error markDeliveredErr error removeErr error savedComponents map[string]string markedVersions map[string]string - pendingComponents map[string]redisstore.PendingComponentEvent + pendingComponents map[string]mongostore.PendingComponentEvent removed []string } @@ -48,7 +48,7 @@ func newFakePollerStore() *fakePollerStore { delivered: map[string]map[string]bool{}, savedComponents: map[string]string{}, markedVersions: map[string]string{}, - pendingComponents: map[string]redisstore.PendingComponentEvent{}, + pendingComponents: map[string]mongostore.PendingComponentEvent{}, } } @@ -77,7 +77,7 @@ func (f *fakePollerStore) IsInitialized(context.Context) (bool, error) { return f.initialized, nil } -func (f *fakePollerStore) ListSubscribers(context.Context) ([]redisstore.Subscriber, error) { +func (f *fakePollerStore) ListSubscribers(context.Context) ([]mongostore.Subscriber, error) { return f.subscribers, nil } @@ -98,15 +98,15 @@ func (f *fakePollerStore) MarkIncidentUpdateVersion(_ context.Context, updateID, return nil } -func (f *fakePollerStore) PendingComponentEvents(context.Context) (map[string]redisstore.PendingComponentEvent, error) { - result := make(map[string]redisstore.PendingComponentEvent, len(f.pendingComponents)) +func (f *fakePollerStore) PendingComponentEvents(context.Context) (map[string]mongostore.PendingComponentEvent, error) { + result := make(map[string]mongostore.PendingComponentEvent, len(f.pendingComponents)) for key, value := range f.pendingComponents { result[key] = value } return result, nil } -func (f *fakePollerStore) SavePendingComponentEvent(_ context.Context, event redisstore.PendingComponentEvent) error { +func (f *fakePollerStore) SavePendingComponentEvent(_ context.Context, event mongostore.PendingComponentEvent) error { f.pendingComponents[event.ComponentID] = event return nil } @@ -116,7 +116,7 @@ func (f *fakePollerStore) RemovePendingComponentEvent(_ context.Context, compone return nil } -func (f *fakePollerStore) RemoveSubscriber(_ context.Context, sub redisstore.Subscriber) error { +func (f *fakePollerStore) RemoveSubscriber(_ context.Context, sub mongostore.Subscriber) error { if f.removeErr != nil { return f.removeErr } @@ -142,7 +142,7 @@ type fakeNotifier struct { messages []string } -func (f *fakeNotifier) SendMessage(_ context.Context, sub redisstore.Subscriber, text string) error { +func (f *fakeNotifier) SendMessage(_ context.Context, sub mongostore.Subscriber, text string) error { f.sends = append(f.sends, sub.Key()) f.messages = append(f.messages, text) if queued := f.errorQueue[sub.Key()]; len(queued) > 0 { @@ -160,7 +160,7 @@ func TestCheckOnceDoesNotCheckpointWhenSendFails(t *testing.T) { store := newFakePollerStore() store.initialized = true store.componentStatuses["c1"] = "operational" - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil)} + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil)} notifier := &fakeNotifier{errors: map[string]error{"1": errors.New("network")}} runner := NewRunner(fakeStatusClient{summary: summaryWithComponent("c1", "API", "degraded_performance")}, store, notifier, time.Minute, slog.Default()) @@ -176,7 +176,7 @@ func TestCheckOnceSkipsAlreadyDeliveredSubscriberOnRetry(t *testing.T) { store := newFakePollerStore() store.initialized = true store.componentStatuses["c1"] = "operational" - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil), redisstore.NewSubscriber(2, nil)} + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil), mongostore.NewSubscriber(2, nil)} notifier := &fakeNotifier{errors: map[string]error{"2": errors.New("rate limit")}} runner := NewRunner(fakeStatusClient{summary: summaryWithComponent("c1", "API", "degraded_performance")}, store, notifier, time.Minute, slog.Default()) @@ -204,7 +204,7 @@ func TestCheckOnceContinuesFanOutAfterRetryableSendFailures(t *testing.T) { store.initialized = true store.componentStatuses["c1"] = "operational" store.componentStatuses["c2"] = "operational" - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil), redisstore.NewSubscriber(2, nil)} + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil), mongostore.NewSubscriber(2, nil)} notifier := &fakeNotifier{errors: map[string]error{"1": errors.New("rate limit")}} runner := NewRunner(fakeStatusClient{summary: openai.Summary{Components: []openai.Component{ {ID: "c1", Name: "API", Status: "degraded_performance", UpdatedAt: "2026-01-01T00:00:00Z"}, @@ -227,15 +227,15 @@ func TestCheckOnceStopsOnDeliveryStateReadError(t *testing.T) { store.initialized = true store.componentStatuses["c1"] = "operational" store.componentStatuses["c2"] = "operational" - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil)} - store.hasDeliveredErr = errors.New("redis read failed") + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil)} + store.hasDeliveredErr = errors.New("store read failed") notifier := &fakeNotifier{} runner := NewRunner(fakeStatusClient{summary: openai.Summary{Components: []openai.Component{ {ID: "c1", Name: "API", Status: "degraded_performance", UpdatedAt: "2026-01-01T00:00:00Z"}, {ID: "c2", Name: "ChatGPT", Status: "partial_outage", UpdatedAt: "2026-01-01T00:01:00Z"}, }}}, store, notifier, time.Minute, slog.Default()) - if err := runner.CheckOnce(context.Background()); err == nil || !strings.Contains(err.Error(), "redis read failed") { + if err := runner.CheckOnce(context.Background()); err == nil || !strings.Contains(err.Error(), "store read failed") { t.Fatalf("CheckOnce error = %v, want delivery state read error", err) } if len(notifier.sends) != 0 { @@ -248,15 +248,15 @@ func TestCheckOnceStopsWhenTerminalSubscriberRemovalFails(t *testing.T) { store.initialized = true store.componentStatuses["c1"] = "operational" store.componentStatuses["c2"] = "operational" - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil)} - store.removeErr = errors.New("redis remove failed") + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil)} + store.removeErr = errors.New("store remove failed") notifier := &fakeNotifier{errors: map[string]error{"1": &telegram.APIError{StatusCode: 200, ErrorCode: 403, Description: "Forbidden: bot was blocked by the user"}}} runner := NewRunner(fakeStatusClient{summary: openai.Summary{Components: []openai.Component{ {ID: "c1", Name: "API", Status: "degraded_performance", UpdatedAt: "2026-01-01T00:00:00Z"}, {ID: "c2", Name: "ChatGPT", Status: "partial_outage", UpdatedAt: "2026-01-01T00:01:00Z"}, }}}, store, notifier, time.Minute, slog.Default()) - if err := runner.CheckOnce(context.Background()); err == nil || !strings.Contains(err.Error(), "redis remove failed") { + if err := runner.CheckOnce(context.Background()); err == nil || !strings.Contains(err.Error(), "store remove failed") { t.Fatalf("CheckOnce error = %v, want subscriber remove error", err) } if got, want := notifier.sends, []string{"1"}; !equalStrings(got, want) { @@ -269,7 +269,7 @@ func TestCheckOnceSkipsLaterEventsForSubscriberAfterRetryableFailure(t *testing. store.initialized = true store.componentStatuses["c1"] = "operational" store.componentStatuses["c2"] = "operational" - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil), redisstore.NewSubscriber(2, nil)} + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil), mongostore.NewSubscriber(2, nil)} notifier := &fakeNotifier{errorQueue: map[string][]error{"1": {errors.New("rate limit")}}} runner := NewRunner(fakeStatusClient{summary: openai.Summary{Components: []openai.Component{ {ID: "c1", Name: "API", Status: "degraded_performance", UpdatedAt: "2026-01-01T00:00:00Z"}, @@ -288,7 +288,7 @@ func TestCheckOnceRemovesTerminalSubscriberAndCheckpoints(t *testing.T) { store := newFakePollerStore() store.initialized = true store.componentStatuses["c1"] = "operational" - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil), redisstore.NewSubscriber(2, nil)} + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil), mongostore.NewSubscriber(2, nil)} notifier := &fakeNotifier{errors: map[string]error{"2": &telegram.APIError{StatusCode: 200, ErrorCode: 403, Description: "Forbidden: bot was blocked by the user"}}} runner := NewRunner(fakeStatusClient{summary: summaryWithComponent("c1", "API", "degraded_performance")}, store, notifier, time.Minute, slog.Default()) @@ -308,7 +308,7 @@ func TestIncidentUpdateVersionEditSendsAgain(t *testing.T) { store := newFakePollerStore() store.initialized = true store.incidentVersions["u1"] = IncidentUpdateVersion(update) - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil)} + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil)} updated := openai.IncidentUpdate{ID: "u1", Body: "new", UpdatedAt: "2026-01-01T00:01:00Z"} notifier := &fakeNotifier{} runner := NewRunner(fakeStatusClient{incidents: openai.IncidentsResponse{Incidents: []openai.Incident{{ID: "i1", Name: "incident", Impact: "minor", IncidentUpdates: []openai.IncidentUpdate{updated}}}}}, store, notifier, time.Minute, slog.Default()) @@ -328,7 +328,7 @@ func TestPendingComponentEventSurvivesReturnToPreviousStatus(t *testing.T) { store := newFakePollerStore() store.initialized = true store.componentStatuses["c1"] = "operational" - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil), redisstore.NewSubscriber(2, nil)} + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil), mongostore.NewSubscriber(2, nil)} notifier := &fakeNotifier{errors: map[string]error{"2": errors.New("rate limit")}} runner := NewRunner(fakeStatusClient{summary: summaryWithComponent("c1", "API", "degraded_performance")}, store, notifier, time.Minute, slog.Default()) @@ -371,7 +371,7 @@ func TestPendingComponentLabelDoesNotCountCurrentComponentAsDuplicate(t *testing store := newFakePollerStore() store.initialized = true store.componentStatuses["c1"] = "operational" - store.pendingComponents["c1"] = redisstore.PendingComponentEvent{ + store.pendingComponents["c1"] = mongostore.PendingComponentEvent{ ComponentID: "c1", ComponentName: "API", Status: "degraded_performance", @@ -379,7 +379,7 @@ func TestPendingComponentLabelDoesNotCountCurrentComponentAsDuplicate(t *testing PreviousStatus: "operational", DeliveryKey: "component:c1:degraded_performance:2026-01-01T00:00:00Z", } - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil)} + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil)} notifier := &fakeNotifier{} runner := NewRunner(fakeStatusClient{summary: summaryWithComponent("c1", "API", "degraded_performance")}, store, notifier, time.Minute, slog.Default()) @@ -395,7 +395,7 @@ func TestPendingComponentLabelDoesNotCountCurrentComponentAsDuplicate(t *testing } func TestPendingComponentDuplicateLabelsIncludeCurrentRename(t *testing.T) { - pending := map[string]redisstore.PendingComponentEvent{ + pending := map[string]mongostore.PendingComponentEvent{ "c1": { ComponentID: "c1", ComponentName: "Old API", @@ -420,8 +420,8 @@ func TestCheckOnceContinuesWhenMarkDeliveredFailsAfterSend(t *testing.T) { store := newFakePollerStore() store.initialized = true store.componentStatuses["c1"] = "operational" - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil)} - store.markDeliveredErr = errors.New("redis write failed") + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil)} + store.markDeliveredErr = errors.New("store write failed") notifier := &fakeNotifier{} runner := NewRunner(fakeStatusClient{summary: summaryWithComponent("c1", "API", "degraded_performance")}, store, notifier, time.Minute, slog.Default()) @@ -451,7 +451,7 @@ func TestCheckOnceDoesNotRemarkSeenIncidentVersion(t *testing.T) { store := newFakePollerStore() store.initialized = true store.incidentVersions["u1"] = IncidentUpdateVersion(update) - store.subscribers = []redisstore.Subscriber{redisstore.NewSubscriber(1, nil)} + store.subscribers = []mongostore.Subscriber{mongostore.NewSubscriber(1, nil)} notifier := &fakeNotifier{} runner := NewRunner(fakeStatusClient{incidents: openai.IncidentsResponse{Incidents: []openai.Incident{{ID: "i1", Name: "incident", Impact: "minor", IncidentUpdates: []openai.IncidentUpdate{update}}}}}, store, notifier, time.Minute, slog.Default()) diff --git a/internal/redisstore/checkpoint.go b/internal/redisstore/checkpoint.go deleted file mode 100644 index 3f7e1dd..0000000 --- a/internal/redisstore/checkpoint.go +++ /dev/null @@ -1,171 +0,0 @@ -package redisstore - -import ( - "context" - "crypto/sha256" - "encoding/hex" - "encoding/json" - "fmt" - "strconv" - "time" - - "github.com/redis/go-redis/v9" -) - -func (s *Store) IsInitialized(ctx context.Context) (bool, error) { - count, err := s.client.Exists(ctx, initializedKey).Result() - return count > 0, err -} - -func (s *Store) SetInitialized(ctx context.Context) error { - return s.client.Set(ctx, initializedKey, "1", 0).Err() -} - -func (s *Store) ComponentStatuses(ctx context.Context) (map[string]string, error) { - return s.client.HGetAll(ctx, componentStatusesKey).Result() -} - -func (s *Store) SaveComponentStatus(ctx context.Context, componentID, status string) error { - return s.client.HSet(ctx, componentStatusesKey, componentID, status).Err() -} - -type PendingComponentEvent struct { - ComponentID string `json:"component_id"` - ComponentName string `json:"component_name"` - Status string `json:"status"` - UpdatedAt string `json:"updated_at"` - Position int `json:"position"` - PreviousStatus string `json:"previous_status"` - DeliveryKey string `json:"delivery_key"` -} - -func (s *Store) PendingComponentEvents(ctx context.Context) (map[string]PendingComponentEvent, error) { - values, err := s.client.HGetAll(ctx, pendingComponentsKey).Result() - if err != nil { - return nil, err - } - events := make(map[string]PendingComponentEvent, len(values)) - for componentID, value := range values { - event, err := decodePendingComponentEvent(componentID, value) - if err != nil { - return nil, err - } - events[componentID] = event - } - return events, nil -} - -func decodePendingComponentEvent(componentID, value string) (PendingComponentEvent, error) { - var event PendingComponentEvent - if err := json.Unmarshal([]byte(value), &event); err != nil { - return PendingComponentEvent{}, fmt.Errorf("decode pending component event %q: %w", componentID, err) - } - if event.ComponentID == "" { - event.ComponentID = componentID - } - return event, nil -} - -func (s *Store) SavePendingComponentEvent(ctx context.Context, event PendingComponentEvent) error { - payload, err := json.Marshal(event) - if err != nil { - return err - } - return s.client.HSet(ctx, pendingComponentsKey, event.ComponentID, payload).Err() -} - -func (s *Store) RemovePendingComponentEvent(ctx context.Context, componentID string) error { - return s.client.HDel(ctx, pendingComponentsKey, componentID).Err() -} - -func (s *Store) HasIncidentUpdate(ctx context.Context, updateID string) (bool, error) { - return s.client.SIsMember(ctx, incidentUpdatesKey, updateID).Result() -} - -func (s *Store) MarkIncidentUpdate(ctx context.Context, updateID string) error { - return s.client.SAdd(ctx, incidentUpdatesKey, updateID).Err() -} - -func (s *Store) HasIncidentUpdateVersion(ctx context.Context, updateID, version string) (bool, error) { - stored, err := s.client.HGet(ctx, incidentVersionsKey, updateID).Result() - if err == nil { - return stored == version, nil - } - if err != redis.Nil { - return false, err - } - legacySeen, err := s.HasIncidentUpdate(ctx, updateID) - if err != nil { - return false, err - } - if legacySeen { - if err := s.client.HSet(ctx, incidentVersionsKey, updateID, version).Err(); err != nil { - return false, err - } - return true, nil - } - return false, nil -} - -func (s *Store) MarkIncidentUpdateVersion(ctx context.Context, updateID, version string) error { - pipe := s.client.TxPipeline() - pipe.HSet(ctx, incidentVersionsKey, updateID, version) - pipe.SAdd(ctx, incidentUpdatesKey, updateID) - _, err := pipe.Exec(ctx) - return err -} - -// DeliveredSubscribers returns the set of subscriber keys already notified for -// eventKey. Callers fan a single event out to every subscriber, so one SMEMBERS -// plus in-memory lookups replaces one SISMEMBER per subscriber. -func (s *Store) DeliveredSubscribers(ctx context.Context, eventKey string) (map[string]bool, error) { - members, err := s.client.SMembers(ctx, deliveryStateKey(eventKey)).Result() - if err != nil { - return nil, err - } - delivered := make(map[string]bool, len(members)) - for _, member := range members { - delivered[member] = true - } - return delivered, nil -} - -func (s *Store) MarkDelivered(ctx context.Context, eventKey, subscriberKey string) error { - key := deliveryStateKey(eventKey) - pipe := s.client.TxPipeline() - pipe.SAdd(ctx, key, subscriberKey) - pipe.Expire(ctx, key, 7*24*time.Hour) - _, err := pipe.Exec(ctx) - return err -} - -func (s *Store) ClearDelivery(ctx context.Context, eventKey string) error { - return s.client.Del(ctx, deliveryStateKey(eventKey)).Err() -} - -func (s *Store) TelegramOffset(ctx context.Context) (int64, error) { - value, err := s.client.Get(ctx, telegramOffsetKey).Result() - if err == redis.Nil { - return 0, nil - } - if err != nil { - return 0, err - } - offset, err := strconv.ParseInt(value, 10, 64) - if err != nil { - if deleteErr := s.client.Del(ctx, telegramOffsetKey).Err(); deleteErr != nil { - return 0, fmt.Errorf("clear invalid telegram offset: %w", deleteErr) - } - return 0, nil - } - return offset, nil -} - -func (s *Store) SaveTelegramOffset(ctx context.Context, offset int64) error { - return s.client.Set(ctx, telegramOffsetKey, strconv.FormatInt(offset, 10), 0).Err() -} - -func deliveryStateKey(eventKey string) string { - sum := sha256.Sum256([]byte(eventKey)) - return "openai-status:event-delivery:" + hex.EncodeToString(sum[:]) -} diff --git a/internal/redisstore/store.go b/internal/redisstore/store.go deleted file mode 100644 index e9ceed8..0000000 --- a/internal/redisstore/store.go +++ /dev/null @@ -1,27 +0,0 @@ -package redisstore - -import "github.com/redis/go-redis/v9" - -const ( - subscribersKey = "openai-status:subscribers" - subscriberSettingsKey = "openai-status:subscriber-settings" - componentStatusesKey = "openai-status:component-statuses" - pendingComponentsKey = "openai-status:pending-component-events" - incidentUpdatesKey = "openai-status:incident-updates" - incidentVersionsKey = "openai-status:incident-update-versions" - initializedKey = "openai-status:initialized" - telegramOffsetKey = "openai-status:telegram-offset" -) - -const ( - SubscriptionTypeIncident = "incident" - SubscriptionTypeComponent = "component" -) - -type Store struct { - client redis.UniversalClient -} - -func New(client redis.UniversalClient) *Store { - return &Store{client: client} -} diff --git a/internal/redisstore/store_test.go b/internal/redisstore/store_test.go deleted file mode 100644 index 59a781d..0000000 --- a/internal/redisstore/store_test.go +++ /dev/null @@ -1,243 +0,0 @@ -package redisstore - -import ( - "context" - "testing" - - "github.com/alicebob/miniredis/v2" - "github.com/redis/go-redis/v9" -) - -func TestSubscriberKeyRoundTripWithoutThread(t *testing.T) { - sub := NewSubscriber(12345, nil) - parsed, err := ParseSubscriberKey(sub.Key()) - if err != nil { - t.Fatalf("ParseSubscriberKey returned error: %v", err) - } - if parsed.ChatID != sub.ChatID || parsed.ThreadID != nil { - t.Fatalf("parsed subscriber = %+v, want %+v", parsed, sub) - } -} - -func TestSubscriberKeyRoundTripWithThreadZero(t *testing.T) { - threadID := 0 - sub := NewSubscriber(-10012345, &threadID) - parsed, err := ParseSubscriberKey(sub.Key()) - if err != nil { - t.Fatalf("ParseSubscriberKey returned error: %v", err) - } - if parsed.ChatID != sub.ChatID || parsed.ThreadID == nil || *parsed.ThreadID != threadID { - t.Fatalf("parsed subscriber = %+v, want thread ID %d", parsed, threadID) - } -} - -func TestParseSubscriberKeyRejectsInvalidValue(t *testing.T) { - if _, err := ParseSubscriberKey("abc:def:ghi"); err == nil { - t.Fatal("expected invalid key error") - } -} - -func TestSubscriberAcceptsComponentIDAndLegacyNameFilters(t *testing.T) { - sub := NewSubscriber(12345, nil) - sub.Components = []string{"component-id", "Legacy API Name"} - if !sub.Accepts(SubscriptionTypeComponent, "component-id", "API") { - t.Fatal("expected component ID filter to match") - } - if !sub.Accepts(SubscriptionTypeComponent, "other-id", "legacy api name") { - t.Fatal("expected legacy component name filter to match") - } - if sub.Accepts(SubscriptionTypeComponent, "other-id", "Other") { - t.Fatal("unexpected component filter match") - } -} - -func TestDecodePendingComponentEventBackfillsComponentID(t *testing.T) { - event, err := decodePendingComponentEvent("c1", `{"component_name":"API","status":"operational"}`) - if err != nil { - t.Fatalf("decodePendingComponentEvent returned error: %v", err) - } - if event.ComponentID != "c1" { - t.Fatalf("ComponentID = %q, want c1", event.ComponentID) - } -} - -func TestDecodePendingComponentEventRejectsCorruptedPayload(t *testing.T) { - if _, err := decodePendingComponentEvent("c1", `{bad json`); err == nil { - t.Fatal("expected corrupted pending event error") - } -} - -func TestDecodeSubscriberSettingsNormalizesValues(t *testing.T) { - settings, err := decodeSubscriberSettings("123", `{"types":["incident","component","incident"],"components":[" api ","API",""]}`) - if err != nil { - t.Fatalf("decodeSubscriberSettings returned error: %v", err) - } - if len(settings.Types) != 2 || settings.Types[0] != SubscriptionTypeIncident || settings.Types[1] != SubscriptionTypeComponent { - t.Fatalf("Types = %v", settings.Types) - } - if len(settings.Components) != 1 || settings.Components[0] != "api" { - t.Fatalf("Components = %v", settings.Components) - } -} - -func TestDecodeSubscriberSettingsRejectsCorruptedPayload(t *testing.T) { - if _, err := decodeSubscriberSettings("123", `{bad json`); err == nil { - t.Fatal("expected corrupted subscriber settings error") - } -} - -func TestListSubscribersRecoversCorruptSettingsWithoutBlockingOthers(t *testing.T) { - ctx := context.Background() - store, client := newTestStore(t) - good := NewSubscriber(1, nil) - corrupt := NewSubscriber(2, nil) - if err := client.SAdd(ctx, subscribersKey, good.Key(), corrupt.Key()).Err(); err != nil { - t.Fatalf("SAdd subscribers: %v", err) - } - if err := client.HSet(ctx, subscriberSettingsKey, good.Key(), `{"types":["incident"],"components":[]}`).Err(); err != nil { - t.Fatalf("HSet good settings: %v", err) - } - if err := client.HSet(ctx, subscriberSettingsKey, corrupt.Key(), `{bad json`).Err(); err != nil { - t.Fatalf("HSet corrupt settings: %v", err) - } - - subscribers, err := store.ListSubscribers(ctx) - if err != nil { - t.Fatalf("ListSubscribers returned error: %v", err) - } - byKey := map[string]Subscriber{} - for _, subscriber := range subscribers { - byKey[subscriber.Key()] = subscriber - } - if len(byKey) != 2 { - t.Fatalf("subscribers = %v, want two subscribers", subscribers) - } - if got := byKey[good.Key()].Types; len(got) != 1 || got[0] != SubscriptionTypeIncident { - t.Fatalf("good Types = %v, want [incident]", got) - } - if got := byKey[corrupt.Key()].Types; len(got) != 2 { - t.Fatalf("corrupt Types = %v, want default types", got) - } - exists, err := client.HExists(ctx, subscriberSettingsKey, corrupt.Key()).Result() - if err != nil { - t.Fatalf("HExists corrupt settings: %v", err) - } - if exists { - t.Fatal("corrupt settings field still exists") - } -} - -func TestUpdateSubscriberSettingsSavesTypesAndComponentsTogether(t *testing.T) { - ctx := context.Background() - store, _ := newTestStore(t) - sub := NewSubscriber(1, nil) - if err := store.AddSubscriber(ctx, sub); err != nil { - t.Fatalf("AddSubscriber returned error: %v", err) - } - - updated, err := store.UpdateSubscriberSettings(ctx, sub, []string{SubscriptionTypeComponent}, []string{"c-api"}) - if err != nil { - t.Fatalf("UpdateSubscriberSettings returned error: %v", err) - } - if !updated { - t.Fatal("UpdateSubscriberSettings updated = false, want true") - } - got, exists, err := store.GetSubscriber(ctx, sub) - if err != nil { - t.Fatalf("GetSubscriber returned error: %v", err) - } - if !exists { - t.Fatal("subscriber missing after update") - } - if len(got.Types) != 1 || got.Types[0] != SubscriptionTypeComponent { - t.Fatalf("Types = %v, want [component]", got.Types) - } - if len(got.Components) != 1 || got.Components[0] != "c-api" { - t.Fatalf("Components = %v, want [c-api]", got.Components) - } -} - -func TestUpdateSubscriberSettingsReturnsFalseForMissingSubscriber(t *testing.T) { - ctx := context.Background() - store, client := newTestStore(t) - sub := NewSubscriber(1, nil) - - updated, err := store.UpdateSubscriberSettings(ctx, sub, []string{SubscriptionTypeComponent}, []string{"c-api"}) - if err != nil { - t.Fatalf("UpdateSubscriberSettings returned error: %v", err) - } - if updated { - t.Fatal("UpdateSubscriberSettings updated = true, want false") - } - exists, err := client.HExists(ctx, subscriberSettingsKey, sub.Key()).Result() - if err != nil { - t.Fatalf("HExists settings: %v", err) - } - if exists { - t.Fatal("settings were written for missing subscriber") - } -} - -func TestHasIncidentUpdateVersionMigratesLegacySeenUpdate(t *testing.T) { - ctx := context.Background() - store, client := newTestStore(t) - if err := client.SAdd(ctx, incidentUpdatesKey, "u1").Err(); err != nil { - t.Fatalf("SAdd legacy incident update: %v", err) - } - - seen, err := store.HasIncidentUpdateVersion(ctx, "u1", "v1") - if err != nil { - t.Fatalf("HasIncidentUpdateVersion returned error: %v", err) - } - if !seen { - t.Fatal("legacy incident update was not treated as seen") - } - stored, err := client.HGet(ctx, incidentVersionsKey, "u1").Result() - if err != nil { - t.Fatalf("HGet migrated incident version: %v", err) - } - if stored != "v1" { - t.Fatalf("stored version = %q, want v1", stored) - } - - seen, err = store.HasIncidentUpdateVersion(ctx, "u1", "v2") - if err != nil { - t.Fatalf("HasIncidentUpdateVersion changed version returned error: %v", err) - } - if seen { - t.Fatal("changed incident update version should not be treated as seen") - } -} - -func TestTelegramOffsetClearsInvalidStoredValue(t *testing.T) { - ctx := context.Background() - store, client := newTestStore(t) - if err := client.Set(ctx, telegramOffsetKey, "not-an-offset", 0).Err(); err != nil { - t.Fatalf("Set invalid telegram offset: %v", err) - } - - offset, err := store.TelegramOffset(ctx) - if err != nil { - t.Fatalf("TelegramOffset returned error: %v", err) - } - if offset != 0 { - t.Fatalf("offset = %d, want 0", offset) - } - exists, err := client.Exists(ctx, telegramOffsetKey).Result() - if err != nil { - t.Fatalf("Exists telegram offset: %v", err) - } - if exists != 0 { - t.Fatal("invalid telegram offset key was not cleared") - } -} - -func newTestStore(t *testing.T) (*Store, *redis.Client) { - t.Helper() - server := miniredis.RunT(t) - client := redis.NewClient(&redis.Options{Addr: server.Addr()}) - t.Cleanup(func() { - _ = client.Close() - }) - return New(client), client -} diff --git a/internal/redisstore/subscriber.go b/internal/redisstore/subscriber.go deleted file mode 100644 index ab7efa2..0000000 --- a/internal/redisstore/subscriber.go +++ /dev/null @@ -1,181 +0,0 @@ -package redisstore - -import ( - "context" - "fmt" - "strconv" - "strings" -) - -type Subscriber struct { - ChatID int64 - ThreadID *int - Types []string - Components []string -} - -func NewSubscriber(chatID int64, threadID *int) Subscriber { - var copiedThreadID *int - if threadID != nil { - value := *threadID - copiedThreadID = &value - } - return Subscriber{ - ChatID: chatID, - ThreadID: copiedThreadID, - Types: DefaultSubscriptionTypes(), - Components: []string{}, - } -} - -func (s Subscriber) Key() string { - if s.ThreadID == nil { - return strconv.FormatInt(s.ChatID, 10) - } - return fmt.Sprintf("%d:%d", s.ChatID, *s.ThreadID) -} - -func ParseSubscriberKey(value string) (Subscriber, error) { - parts := strings.Split(value, ":") - if len(parts) != 1 && len(parts) != 2 { - return Subscriber{}, fmt.Errorf("invalid subscriber key %q", value) - } - - chatID, err := strconv.ParseInt(parts[0], 10, 64) - if err != nil { - return Subscriber{}, fmt.Errorf("invalid chat ID: %w", err) - } - if len(parts) == 1 { - return NewSubscriber(chatID, nil), nil - } - - threadID, err := strconv.Atoi(parts[1]) - if err != nil { - return Subscriber{}, fmt.Errorf("invalid thread ID: %w", err) - } - return NewSubscriber(chatID, &threadID), nil -} - -func DefaultSubscriptionTypes() []string { - return []string{SubscriptionTypeIncident, SubscriptionTypeComponent} -} - -func (s Subscriber) Accepts(eventType, componentID, componentName string) bool { - if !containsFold(s.Types, eventType) { - return false - } - if eventType != SubscriptionTypeComponent || len(s.Components) == 0 { - return true - } - return containsFold(s.Components, componentID) || containsFold(s.Components, componentName) -} - -func (s *Store) AddSubscriber(ctx context.Context, sub Subscriber) error { - key := sub.Key() - settings, err := s.loadSubscriberSettings(ctx, key) - if err != nil { - return err - } - if err := s.client.SAdd(ctx, subscribersKey, key).Err(); err != nil { - return err - } - return s.saveSubscriberSettings(ctx, key, settings) -} - -func (s *Store) RemoveSubscriber(ctx context.Context, sub Subscriber) error { - key := sub.Key() - pipe := s.client.TxPipeline() - pipe.SRem(ctx, subscribersKey, key) - pipe.HDel(ctx, subscriberSettingsKey, key) - _, err := pipe.Exec(ctx) - return err -} - -func (s *Store) ListSubscribers(ctx context.Context) ([]Subscriber, error) { - keys, err := s.client.SMembers(ctx, subscribersKey).Result() - if err != nil { - return nil, err - } - - // Fetch every subscriber's settings in one HGETALL instead of one HGET per - // key. The settings hash mirrors the subscribers set, so its size is the - // subscriber count rather than unbounded history. - settingsByKey, err := s.client.HGetAll(ctx, subscriberSettingsKey).Result() - if err != nil { - return nil, err - } - - subscribers := make([]Subscriber, 0, len(keys)) - for _, key := range keys { - sub, err := ParseSubscriberKey(key) - if err != nil { - if removeErr := s.client.SRem(ctx, subscribersKey, key).Err(); removeErr != nil { - return nil, fmt.Errorf("remove malformed subscriber key %q: %w", key, removeErr) - } - return nil, fmt.Errorf("malformed subscriber key %q: %w", key, err) - } - value, present := settingsByKey[key] - settings, err := s.resolveSubscriberSettings(ctx, key, value, present) - if err != nil { - return nil, err - } - sub.Types = settings.Types - sub.Components = settings.Components - subscribers = append(subscribers, sub) - } - return subscribers, nil -} - -func (s *Store) GetSubscriber(ctx context.Context, sub Subscriber) (Subscriber, bool, error) { - key := sub.Key() - exists, err := s.client.SIsMember(ctx, subscribersKey, key).Result() - if err != nil || !exists { - return Subscriber{}, false, err - } - return s.loadSubscriber(ctx, key) -} - -func (s *Store) UpdateSubscriberTypes(ctx context.Context, sub Subscriber, types []string) (bool, error) { - key := sub.Key() - settings, exists, err := s.loadExistingSubscriberSettings(ctx, key) - if err != nil || !exists { - return false, err - } - settings.Types = normalizeTypes(types) - return s.saveExistingSubscriberSettings(ctx, key, settings) -} - -func (s *Store) UpdateSubscriberComponents(ctx context.Context, sub Subscriber, components []string) (bool, error) { - key := sub.Key() - settings, exists, err := s.loadExistingSubscriberSettings(ctx, key) - if err != nil || !exists { - return false, err - } - settings.Components = normalizeComponents(components) - return s.saveExistingSubscriberSettings(ctx, key, settings) -} - -func (s *Store) UpdateSubscriberSettings(ctx context.Context, sub Subscriber, types, components []string) (bool, error) { - key := sub.Key() - return s.saveExistingSubscriberSettings(ctx, key, subscriberSettings{ - Types: types, - Components: components, - }) -} - -func (s *Store) loadSubscriber(ctx context.Context, key string) (Subscriber, bool, error) { - sub, err := ParseSubscriberKey(key) - if err != nil { - if removeErr := s.client.SRem(ctx, subscribersKey, key).Err(); removeErr != nil { - return Subscriber{}, false, fmt.Errorf("remove malformed subscriber key %q: %w", key, removeErr) - } - return Subscriber{}, false, fmt.Errorf("malformed subscriber key %q: %w", key, err) - } - settings, err := s.loadSubscriberSettings(ctx, key) - if err != nil { - return Subscriber{}, false, err - } - sub.Types = settings.Types - sub.Components = settings.Components - return sub, true, nil -} diff --git a/internal/redisstore/subscriber_settings.go b/internal/redisstore/subscriber_settings.go deleted file mode 100644 index 4f27801..0000000 --- a/internal/redisstore/subscriber_settings.go +++ /dev/null @@ -1,99 +0,0 @@ -package redisstore - -import ( - "context" - "encoding/json" - "fmt" - - "github.com/redis/go-redis/v9" -) - -type subscriberSettings struct { - Types []string `json:"types"` - Components []string `json:"components"` -} - -var updateSubscriberSettingsScript = redis.NewScript(` -if redis.call("SISMEMBER", KEYS[1], ARGV[1]) == 0 then - return 0 -end -redis.call("HSET", KEYS[2], ARGV[1], ARGV[2]) -return 1 -`) - -func (s *Store) loadExistingSubscriberSettings(ctx context.Context, key string) (subscriberSettings, bool, error) { - exists, err := s.client.SIsMember(ctx, subscribersKey, key).Result() - if err != nil || !exists { - return subscriberSettings{}, false, err - } - settings, err := s.loadSubscriberSettings(ctx, key) - return settings, true, err -} - -func (s *Store) loadSubscriberSettings(ctx context.Context, key string) (subscriberSettings, error) { - value, err := s.client.HGet(ctx, subscriberSettingsKey, key).Result() - if err == redis.Nil { - return s.resolveSubscriberSettings(ctx, key, "", false) - } - if err != nil { - return subscriberSettings{}, err - } - return s.resolveSubscriberSettings(ctx, key, value, true) -} - -// resolveSubscriberSettings turns a raw settings hash value into decoded -// settings. It lets callers that already hold the value (e.g. a batched -// HGETALL in ListSubscribers) reuse the same decode/default/self-heal logic as -// the single-key HGET path. Absent or corrupt values fall back to defaults, -// dropping the corrupt field so it heals on next write. -func (s *Store) resolveSubscriberSettings(ctx context.Context, key, value string, present bool) (subscriberSettings, error) { - if !present { - return defaultSubscriberSettings(), nil - } - settings, err := decodeSubscriberSettings(key, value) - if err == nil { - return settings, nil - } - if removeErr := s.client.HDel(ctx, subscriberSettingsKey, key).Err(); removeErr != nil { - return subscriberSettings{}, fmt.Errorf("clear corrupt subscriber settings %q: %w", key, removeErr) - } - return defaultSubscriberSettings(), nil -} - -func decodeSubscriberSettings(key, value string) (subscriberSettings, error) { - var settings subscriberSettings - if err := json.Unmarshal([]byte(value), &settings); err != nil { - return subscriberSettings{}, fmt.Errorf("decode subscriber settings %q: %w", key, err) - } - settings.Types = normalizeTypes(settings.Types) - settings.Components = normalizeComponents(settings.Components) - return settings, nil -} - -func (s *Store) saveSubscriberSettings(ctx context.Context, key string, settings subscriberSettings) error { - settings.Types = normalizeTypes(settings.Types) - settings.Components = normalizeComponents(settings.Components) - payload, err := json.Marshal(settings) - if err != nil { - return err - } - return s.client.HSet(ctx, subscriberSettingsKey, key, payload).Err() -} - -func (s *Store) saveExistingSubscriberSettings(ctx context.Context, key string, settings subscriberSettings) (bool, error) { - settings.Types = normalizeTypes(settings.Types) - settings.Components = normalizeComponents(settings.Components) - payload, err := json.Marshal(settings) - if err != nil { - return false, err - } - updated, err := updateSubscriberSettingsScript.Run(ctx, s.client, []string{subscribersKey, subscriberSettingsKey}, key, string(payload)).Int() - if err != nil { - return false, err - } - return updated == 1, nil -} - -func defaultSubscriberSettings() subscriberSettings { - return subscriberSettings{Types: DefaultSubscriptionTypes(), Components: []string{}} -} diff --git a/internal/telegram/client.go b/internal/telegram/client.go index a4e0a87..95750db 100644 --- a/internal/telegram/client.go +++ b/internal/telegram/client.go @@ -10,7 +10,7 @@ import ( "strings" "time" - "github.com/tiennm99/openai-status-bot/internal/redisstore" + "github.com/tiennm99/openai-status-bot/internal/mongostore" ) type Client struct { @@ -105,7 +105,7 @@ func (c *Client) GetUpdates(ctx context.Context, offset int64, timeoutSeconds in return updates, nil } -func (c *Client) SendMessage(ctx context.Context, sub redisstore.Subscriber, text string) error { +func (c *Client) SendMessage(ctx context.Context, sub mongostore.Subscriber, text string) error { return c.SendText(ctx, sub.ChatID, sub.ThreadID, text) } diff --git a/plans/260626-1057-redis-to-mongodb/phase-01-suitability-decision.md b/plans/260626-1057-redis-to-mongodb/phase-01-suitability-decision.md new file mode 100644 index 0000000..d63a145 --- /dev/null +++ b/plans/260626-1057-redis-to-mongodb/phase-01-suitability-decision.md @@ -0,0 +1,65 @@ +--- +phase: 1 +title: "Suitability & Decision" +status: complete +effort: "research-only (no code)" +--- + +# Phase 1: Suitability & Decision + +## Overview + +Decide whether MongoDB is the right datastore for this bot before writing code. This phase is the answer to "is it suitable for our case." No code; output is the recorded verdict below. + +## Workload facts (from codebase) + +- State is tiny: one set of subscribers (count = number of Telegram chats), per-subscriber settings, ~dozens of component statuses, a handful of pending component events, incident-update version markers, one `initialized` flag, one telegram offset, and short-lived per-event delivery markers. +- Access pattern: one poll/minute (default `POLL_INTERVAL=1m`) + occasional command handling. No hot path, no high QPS, no large values. +- Already abstracted behind `poller.Store` / `bot.Store` interfaces (`internal/poller/poller.go:17`, `internal/bot/bot.go:25`). +- Redis-specific mechanisms in use: a Lua script for atomic check-then-set (`internal/redisstore/subscriber_settings.go:16`), `EXPIRE` for 7-day delivery dedup TTL (`internal/redisstore/checkpoint.go:137`), and `TxPipeline` for multi-key atomicity (RemoveSubscriber, MarkDelivered, MarkIncidentUpdateVersion). + +## Redis → MongoDB capability mapping + +| Redis usage | MongoDB equivalent | Notes | +|---|---|---| +| `subscribers` SET + `subscriber-settings` HASH | single `subscribers` collection, one doc per subscriber (`_id` = subscriber key, `types`/`components` fields) | **Collapses two keys into one doc**; eliminates the SRem+HDel `TxPipeline` in RemoveSubscriber | +| Lua check-then-set on settings | `UpdateOne(filter{_id}, $set)`, read `MatchedCount` | `MatchedCount==0` ⇒ not subscribed; replaces the Lua script | +| `component-statuses` HASH | `component_statuses` collection, doc per component | upsert via `UpdateOne(SetUpsert)` | +| `pending-component-events` HASH (JSON) | `pending_component_events` collection, doc per component | native BSON fields, no manual JSON marshal needed | +| `incident-update-versions` HASH + legacy `incident-updates` SET | `incident_update_versions` collection, doc per updateID | **legacy SET + migration fallback (`checkpoint.go:89-108`) dropped** — fresh DB, no legacy data | +| `event-delivery:` SET + `EXPIRE 7d` | `delivery` collection, doc per (eventKey, subscriber) + **TTL index** on `expiresAt` | `MarkDelivered` becomes one upsert (no SAdd+Expire pair); sha256 key-hashing dropped (Mongo handles arbitrary string values) | +| `initialized` flag, `telegram-offset` | `meta` collection, doc per key | trivial FindOne/upsert | + +All driver specifics confirmed against MongoDB Go driver **v2** (`go.mongodb.org/mongo-driver/v2`, latest v2.7.0, 2026): `ApplyURI` parses `mongodb+srv://` Atlas URIs; TTL via `IndexModel` + `SetExpireAfterSeconds`; `UpdateOne`/`MatchedCount` and `SetUpsert(true)` cover conditional set and upsert. + +## Verdict + +**Suitable: YES. Justified: YES** (given the stated motivation — team already runs MongoDB, so dropping Redis consolidates ops onto one datastore). + +Functionally, MongoDB does everything Redis does here, and the document model **simplifies** the code: three multi-key `TxPipeline` sequences collapse into single-document operations, the Lua script disappears, and the legacy incident-dedup migration path is deleted (no data to migrate). Latency and TTL-sweep granularity (~60s) are irrelevant at one poll/minute with a 7-day dedup window. + +### Trade-offs to accept (be honest) + +1. **Testing regression (the real cost).** `miniredis` is pure-Go, in-memory, zero-binary, instant. MongoDB has **no equivalent** — every option runs a real `mongod`: `testcontainers-go` (needs Docker in CI, ~2-3s/suite) or `memongo` (downloads a ~100MB `mongod` binary, UNIX-only, primary repo stale — prefer the `tryvium-travels/memongo` fork). CI gains a Docker/binary dependency and slows down. This is a genuine downgrade in test ergonomics and the main argument *against* switching absent the ops motivation. +2. Heavier runtime dependency and image than the tiny `go-redis` client (acceptable; not a hot path). +3. Loss of Redis's trivial single-process local dev (`redis:7-alpine`); replaced by a **required Atlas connection** (no local DB — both dev and prod use Atlas, separated by database name). + +### When NOT to switch +If the only driver were "exploring" with no infra reason, staying on Redis would be the KISS choice — Redis is the better technical fit for this exact workload. The switch is recommended **only because** MongoDB is already operated and consolidation has real ops value. + +## Implementation Steps + +1. Confirm the motivation still holds (already-run MongoDB / managed external host) before committing to Phases 2-5. +2. Accept the testing trade-off. **Decided (Session 1):** testcontainers-go, gated behind `//go:build integration` and run manually later — default `go test` stays Docker-free (Phase 5). +3. Proceed to Phase 2. + +## Success Criteria + +- [x] Verdict reviewed and motivation reconfirmed (already-run MongoDB / Atlas). +- [x] Test backend decided: testcontainers, gated + deferred (Session 1). +- [x] Testing trade-off accepted. + +## Risk Assessment + +- Risk: team underestimates CI impact of losing `miniredis`. Mitigation: Phase 5 makes the test backend explicit and a gating decision here. +- Risk: managed MongoDB connectivity/auth from the bot's deploy environment. Mitigation: validate `MONGODB_URI` reachability early (Phase 2 ping on startup, same as current Redis ping in `main.go:38`). diff --git a/plans/260626-1057-redis-to-mongodb/phase-02-store-package-connection.md b/plans/260626-1057-redis-to-mongodb/phase-02-store-package-connection.md new file mode 100644 index 0000000..b2b1d26 --- /dev/null +++ b/plans/260626-1057-redis-to-mongodb/phase-02-store-package-connection.md @@ -0,0 +1,59 @@ +--- +phase: 2 +title: "Store Package & Connection" +status: complete +effort: "S" +--- + +# Phase 2: Store Package & Connection + +## Overview + +Create `internal/mongostore` with the connection plumbing, collection handles, index bootstrap, and the storage-agnostic domain types/logic moved over verbatim from `redisstore`. No consumer wiring yet (Phase 4); `redisstore` still exists in parallel until Phase 5. + +## Requirements + +- Functional: open a MongoDB client from a URI, ping on startup, expose a `Store` holding collection handles, ensure required indexes exist idempotently. +- Non-functional: keep exported type/constant names identical to `redisstore` so Phase 5 is a qualifier swap, not a rewrite of consumers. + +## Architecture + +Collections (database name from config, default `openai_status_bot`): + +- `subscribers` — `{_id: ""|"", chatID, threadID, types[], components[]}` +- `component_statuses` — `{_id: componentID, status}` +- `pending_component_events` — `{_id: componentID, componentName, status, updatedAt, position, previousStatus, deliveryKey}` +- `incident_update_versions` — `{_id: updateID, version}` +- `delivery` — `{_id: "|", eventKey, subscriber, expiresAt}` (TTL index on `expiresAt`) +- `meta` — `{_id: "initialized"|"telegramOffset", value}` + +Indexes ensured on startup: +- `delivery`: TTL index `{expiresAt:1}` with `ExpireAfterSeconds(604800)` (7 days). The compound key is encoded in `_id`, so no extra unique index needed; an additional non-unique `{eventKey:1}` index speeds `DeliveredSubscribers`/`ClearDelivery` lookups. +- All other collections use `_id` only — no extra indexes. + +## Related Code Files + +- Create: `internal/mongostore/store.go` — `Store` struct, `New(client *mongo.Client, dbName string) *Store`, collection handles, collection-name constants (replacing the `*Key` constants in `redisstore/store.go`), `SubscriptionType*` constants. +- Create: `internal/mongostore/indexes.go` — `EnsureIndexes(ctx) error` (TTL + eventKey index). +- Create: `internal/mongostore/subscriber.go` — move `Subscriber`, `NewSubscriber`, `ParseSubscriberKey`, `Key()`, `Accepts`, `DefaultSubscriptionTypes` **verbatim** (pure logic, storage-agnostic). +- Create: `internal/mongostore/subscriber_normalization.go` — move `normalizeTypes`, `normalizeComponents`, `containsFold` **verbatim**. +- Reference (driver, Phase 4 adds to go.mod): `go.mongodb.org/mongo-driver/v2/mongo`, `.../mongo/options`, `.../bson`. + +## Implementation Steps + +1. `go get go.mongodb.org/mongo-driver/v2@latest` (adds to go.mod; full dep cleanup in Phase 4). +2. Write `store.go`: define collection-name constants, `Store` struct holding `*mongo.Database` + per-collection `*mongo.Collection` handles, and `New(client, dbName)`. +3. Write `subscriber.go` and `subscriber_normalization.go` by copying the storage-agnostic code from the matching `redisstore` files unchanged except the package name. +4. Write `indexes.go` with `EnsureIndexes` creating the TTL index (`options.Index().SetExpireAfterSeconds(604800)`) and the `{eventKey:1}` index via `Indexes().CreateMany`. Idempotent — re-creating an existing index is a no-op/ignored error. +5. `go build ./internal/mongostore/...` to confirm the package compiles in isolation. + +## Success Criteria + +- [ ] `internal/mongostore` compiles standalone. +- [ ] Exported names match `redisstore` (`Subscriber`, `PendingComponentEvent` [added Phase 3], `NewSubscriber`, `DefaultSubscriptionTypes`, `SubscriptionTypeIncident/Component`). +- [ ] `EnsureIndexes` creates the TTL index with 604800s expiry. + +## Risk Assessment + +- Risk: driver v2 import path differs from v1 examples. Mitigation: pin `go.mongodb.org/mongo-driver/v2`; `mongo.Connect(options.Client().ApplyURI(uri))` (no separate `ctx` arg in v2 `Connect`). +- Risk: index creation race on multi-instance deploy. Mitigation: single-instance bot; `CreateMany` is idempotent regardless. diff --git a/plans/260626-1057-redis-to-mongodb/phase-03-implement-store-methods.md b/plans/260626-1057-redis-to-mongodb/phase-03-implement-store-methods.md new file mode 100644 index 0000000..7ebfef6 --- /dev/null +++ b/plans/260626-1057-redis-to-mongodb/phase-03-implement-store-methods.md @@ -0,0 +1,67 @@ +--- +phase: 3 +title: "Implement Store Methods" +status: complete +effort: "M" +--- + +# Phase 3: Implement Store Methods + +## Overview + +Implement every method required by `poller.Store` and `bot.Store` against MongoDB, matching the existing semantics exactly (return values, "exists" booleans, defaults, self-heal-on-corrupt behavior where applicable). + +## Requirements + +Satisfy the full interface surface (from `internal/poller/poller.go:17` and `internal/bot/bot.go:25`): + +Subscribers/settings: `AddSubscriber`, `RemoveSubscriber`, `ListSubscribers`, `GetSubscriber`, `UpdateSubscriberTypes`, `UpdateSubscriberComponents`, `UpdateSubscriberSettings`. +Checkpoint/state: `ComponentStatuses`, `SaveComponentStatus`, `PendingComponentEvents`, `SavePendingComponentEvent`, `RemovePendingComponentEvent`, `HasIncidentUpdateVersion`, `MarkIncidentUpdateVersion`, `DeliveredSubscribers`, `MarkDelivered`, `ClearDelivery`, `IsInitialized`, `SetInitialized`, `TelegramOffset`, `SaveTelegramOffset`. + +## Architecture / semantics mapping + +- `PendingComponentEvent` struct moves to `mongostore` with BSON tags; stored as native fields (drop the manual `json.Marshal`/`Unmarshal` from `checkpoint.go:58-75`). +- **AddSubscriber**: `UpdateOne({_id:key}, {$set:{chatID,threadID}, $setOnInsert:{types:defaults, components:[]}}, upsert)`. Preserves existing settings on re-`/start` (matches current load-then-save behavior at `subscriber.go:73-83`). +- **RemoveSubscriber**: `DeleteOne({_id:key})` — single op (was SRem+HDel TxPipeline). +- **ListSubscribers**: `Find({})` + `cursor.All`. Decode straight into `Subscriber` (was SMEMBERS + HGETALL); keep the malformed-key self-heal: on decode/parse failure for a doc, `DeleteOne` it and continue or error (preserve current behavior at `subscriber.go:108-125`). +- **GetSubscriber**: `FindOne({_id:key})`; `ErrNoDocuments` ⇒ `(zero,false,nil)`. +- **UpdateSubscriberTypes/Components/Settings**: `UpdateOne({_id:key}, {$set:{...}})`; `MatchedCount==0` ⇒ `(false,nil)` (replaces the Lua check-then-set). Apply `normalizeTypes`/`normalizeComponents` before write, same as today. +- **ComponentStatuses**: `Find({})` → `map[string]string{_id: status}`. +- **SaveComponentStatus**: `UpdateOne({_id}, {$set:{status}}, upsert)`. +- **PendingComponentEvents**: `Find({})` → `map[string]PendingComponentEvent`. +- **SavePendingComponentEvent / RemovePendingComponentEvent**: upsert / `DeleteOne` by `_id=componentID`. +- **HasIncidentUpdateVersion**: `FindOne({_id:updateID})`; compare stored `version`. **Drop the legacy `incident-updates` SET fallback** (`checkpoint.go:97-106`) — no legacy data on a fresh DB. +- **MarkIncidentUpdateVersion**: `UpdateOne({_id:updateID}, {$set:{version}}, upsert)` — single op (was HSet+SAdd TxPipeline). +- **DeliveredSubscribers**: `Find({eventKey})` → `map[string]bool{subscriber:true}`. +- **MarkDelivered**: `UpdateOne({_id: eventKey+"|"+subscriber}, {$set:{eventKey,subscriber,expiresAt: now+7d}}, upsert)` — single op with TTL (was SAdd+Expire TxPipeline). `now` from `time.Now()`. +- **ClearDelivery**: `DeleteMany({eventKey})`. Drop sha256 key-hashing (`deliveryStateKey`, `checkpoint.go:168`). +- **IsInitialized / SetInitialized**: `meta` doc `_id:"initialized"` — `FindOne` exists / upsert `{value:true}`. +- **TelegramOffset / SaveTelegramOffset**: `meta` doc `_id:"telegramOffset"` — `FindOne` (default 0 on `ErrNoDocuments`, and clear-on-invalid like `checkpoint.go:155-160` is no longer needed since BSON stores an int natively) / upsert `{value:offset}`. + +## Related Code Files + +- Create: `internal/mongostore/subscriber.go` (extend Phase 2 file with the subscriber CRUD methods). +- Create: `internal/mongostore/subscriber_settings.go` — `subscriberSettings` decode/normalize helpers (drop the Lua script; keep normalization), or fold into subscriber.go if small (KISS). +- Create: `internal/mongostore/checkpoint.go` — `PendingComponentEvent` (BSON tags) + all checkpoint/delivery/meta/component methods. +- Reference: `internal/poller/poller.go:17`, `internal/bot/bot.go:25` (interface contracts to satisfy). + +## Implementation Steps + +1. Add `PendingComponentEvent` with BSON tags to `checkpoint.go`. +2. Implement subscriber CRUD + settings methods; verify against `bot.Store` signatures. +3. Implement checkpoint/delivery/meta/component methods; verify against `poller.Store` signatures. +4. Add a compile-time assertion file or inline `var _ poller.Store = (*Store)(nil)` / `var _ bot.Store = (*Store)(nil)` (only after Phase 5 imports align — or assert against locally redeclared interfaces during this phase) to catch signature drift. +5. `go build ./internal/mongostore/...`. + +## Success Criteria + +- [ ] All listed methods implemented with matching signatures and semantics. +- [ ] No `TxPipeline`/Lua equivalents needed — each former multi-key op is a single document op. +- [ ] Legacy incident-dedup fallback and sha256 delivery-key hashing removed. +- [ ] Package compiles. + +## Risk Assessment + +- Risk: `MatchedCount` vs `ModifiedCount` confusion in conditional updates. Mitigation: use `MatchedCount` for "did the subscriber exist" (a no-op `$set` to identical values still matches but may not modify). +- Risk: telegram offset stored as BSON int32 vs int64. Mitigation: store/read as `int64` explicitly in BSON. +- Risk: decode of self-healed malformed subscriber docs. Mitigation: replicate current delete-and-continue/error path; cover in Phase 5 tests. diff --git a/plans/260626-1057-redis-to-mongodb/phase-04-config-wiring-deployment.md b/plans/260626-1057-redis-to-mongodb/phase-04-config-wiring-deployment.md new file mode 100644 index 0000000..2bf89fc --- /dev/null +++ b/plans/260626-1057-redis-to-mongodb/phase-04-config-wiring-deployment.md @@ -0,0 +1,51 @@ +--- +phase: 4 +title: "Config Wiring & Deployment" +status: complete +effort: "S" +--- + +# Phase 4: Config Wiring & Deployment + +## Overview + +Replace Redis configuration, connection bootstrap, and deployment artifacts with MongoDB equivalents. After this phase `main.go` builds the Mongo client; consumer type swap is Phase 5. + + + +## Requirements + +- New env: `MONGODB_URI` (Atlas `mongodb+srv://...`, **required**, no localhost default — there is no local DB), `MONGODB_DATABASE` (default `openai_status_bot`; dev env sets a separate name e.g. `openai_status_bot_dev`). Remove `REDIS_URL`. +- `main.go` connects, pings, ensures indexes, constructs `mongostore.New(client, dbName)`. +- Deployment (validated): **Atlas for both prod and dev** — single cluster, two databases differentiated by `MONGODB_DATABASE`. **No local `mongo` service** in any compose file. + +## Architecture + +`config.Config`: replace `RedisOptions *redis.Options` with `MongoURI string` + `MongoDatabase string`. Drop `parseRedisURL`, `minRedisDB`/`maxRedisDB`. Validation: `MONGODB_URI` **required** (error if empty, like `TELEGRAM_BOT_TOKEN` at `config.go:35-38` — no localhost fallback since both envs use Atlas); `MONGODB_DATABASE` defaults to `openai_status_bot`. Keep validation minimal; the driver validates the URI on connect/ping. + +## Related Code Files + +- Modify: `internal/config/config.go` — swap Redis fields/parsing for Mongo fields; keep `getEnv` default pattern. +- Modify: `internal/config/config_test.go` — replace Redis URL cases with Mongo URI/DB cases. +- Modify: `cmd/openai-status-bot/main.go` — replace `redis.NewClient`+`Ping` (`main.go:37-42`) with `mongo.Connect(options.Client().ApplyURI(cfg.MongoURI))`, `client.Ping`, `defer client.Disconnect(ctx)`; call `store.EnsureIndexes(ctx)`; `store := mongostore.New(client, cfg.MongoDatabase)`. Update the connect-error log fields (drop redis-specific `Network/Addr/DB/TLS`). +- Modify: `.env.example` — `MONGODB_URI=mongodb+srv://:@/` (placeholder), `MONGODB_DATABASE=openai_status_bot`; remove `REDIS_URL`. +- Modify: `docker-compose.yml` — **remove the `redis` service and `redis-data` volume entirely** (no local DB). Becomes bot-only, connecting to Atlas via `.env`; this is the **development** path, so set `MONGODB_DATABASE` to the dev database (e.g. `openai_status_bot_dev`). No `depends_on`. +- Modify: `docker-compose.bot.yml` — bot-only, connects to Atlas via `.env` with the **production** `MONGODB_DATABASE` (`openai_status_bot`). With no local DB, this and `docker-compose.yml` differ only by database name; keep both for the dev/prod split. Optionally collapse to one file — but keeping both matches the existing two-file convention (YAGNI: don't restructure beyond the DB-name change). +- Modify: `README.md` — Configuration table (`MONGODB_URI`, `MONGODB_DATABASE` rows; drop `REDIS_URL` and the percent-encoding note, or rewrite the note for Atlas SRV URIs), feature bullet "Uses Redis for…" → MongoDB, Compose/Quick-Start description (Atlas, no local DB; dev vs prod database), intro line. Note that no local datastore runs in Compose — an Atlas URI is required to start. + +## Implementation Steps + +1. Edit `config.go` + `config_test.go`; `go test ./internal/config/`. +2. Edit `main.go` for Mongo client lifecycle + `EnsureIndexes`. (Will not fully build until Phase 5 swaps `redisstore`→`mongostore` in poller/bot — acceptable; Phases 4-5 land together.) +3. Update `.env.example`, both compose files, README. + +## Success Criteria + +- [ ] `config` package + tests pass with Mongo env vars; no Redis symbols remain in config. +- [ ] `main.go` opens/pings Mongo, ensures indexes, builds `mongostore`. +- [ ] Compose, `.env.example`, README reference only MongoDB. + +## Risk Assessment + +- Risk: managed URIs carry credentials/`+srv`. Mitigation: `ApplyURI` handles `mongodb+srv://`; never log the URI (log db name only); keep `.env` out of git (already in `.gitignore`). +- Risk: dropping `REDIS_URL` breaks existing deploys silently. Mitigation: README "Notes" + this plan flag the env rename; no compatibility shim (clean cutover, intended). diff --git a/plans/260626-1057-redis-to-mongodb/phase-05-cutover-tests.md b/plans/260626-1057-redis-to-mongodb/phase-05-cutover-tests.md new file mode 100644 index 0000000..96a4f32 --- /dev/null +++ b/plans/260626-1057-redis-to-mongodb/phase-05-cutover-tests.md @@ -0,0 +1,69 @@ +--- +phase: 5 +title: "Cutover & Tests" +status: complete +effort: "M" +--- + +# Phase 5: Cutover & Tests + +## Overview + +Swap all consumers from `redisstore` to `mongostore`, delete `redisstore`, purge the Redis dependencies, and re-establish the test suite on a MongoDB-backed harness (replacing `miniredis`). After this phase the project builds and tests green with zero Redis. + +## Requirements + +- All `redisstore.X` references → `mongostore.X` (type names identical; only the qualifier changes). +- `internal/redisstore` deleted; `go-redis` and `miniredis` removed from go.mod/go.sum. +- Tests pass against a real `mongod` test harness. + +## Architecture: test backend + + + +`miniredis` (pure-Go, in-memory, no binary) has **no MongoDB equivalent** — real-backend tests must run a real `mongod`. + +**Decision (validated): `testcontainers-go/modules/mongodb`, but gated and deferred.** The `mongostore` integration suite is written behind a build tag (`//go:build integration`) so it is **NOT part of default `go test ./...`** in this phase. Default test runs must stay Docker-free. The user runs `go test -tags=integration ./internal/mongostore/...` later, when Docker is available. + +- `internal/mongostore/store_test.go` → tag `//go:build integration`; a `testMongo(t)` helper starts a `mongo:7` container once, returns a `*mongostore.Store` against a unique per-test database, and `t.Cleanup` drops it. +- **Poller/bot tests use an in-test fake `Store`** (hand-written, satisfies the interface) — fast, Docker-free, runs in default `go test`. No real DB in those suites. +- So in this phase: `go test ./...` (no tag) passes with zero Docker — poller/bot/config/etc. on fakes, `mongostore` integration tests skipped by the build tag. CI stays green without Docker; real-backend validation is a deliberate manual/later step. + +## Related Code Files + +- Modify (qualifier swap `redisstore.` → `mongostore.`, update imports): + - `internal/poller/poller.go`, `internal/poller/event_collection.go`, `internal/poller/delivery.go` + - `internal/bot/bot.go`, `internal/bot/commands.go`, `internal/bot/helpers.go`, `internal/bot/subscription_format.go` + - `internal/telegram/client.go` (`Subscriber` param at `client.go:108`) +- Create: `internal/mongostore/store_test.go` — `//go:build integration` tag; port the 243-line `redisstore/store_test.go` to the testcontainers harness (same assertions: subscriber CRUD, settings normalization/self-heal, delivery dedup + TTL-index presence, incident version dedup, offset, initialized). Excluded from default `go test`. +- Modify: `internal/poller/poller_test.go`, `internal/bot/bot_test.go` — swap `redisstore` import; replace any `miniredis` usage with a lightweight **in-test fake** of the `Store` interface (Docker-free, fast). Do not point these at the real harness. +- Create (if not already present in those tests): a shared fake `Store` (e.g. an in-memory map-backed type in a test helper) satisfying both `poller.Store` and `bot.Store`. +- Delete: `internal/redisstore/` (all 6 files). +- Modify: `go.mod`/`go.sum` — remove `github.com/redis/go-redis/v9`, `github.com/alicebob/miniredis/v2` (+ transitive `gopher-lua`, `go-rendezvous`, `xxhash` if now unused); add `go.mongodb.org/mongo-driver/v2` and the chosen test dep. Run `go mod tidy`. + +## Implementation Steps + +1. Add compile-time interface assertions in `mongostore` (`var _ poller.Store`, `var _ bot.Store`) — now imports resolve both ways; fix any signature drift from Phase 3. +2. Swap qualifiers across poller/bot/telegram (grep `redisstore\.` → confirm each maps to an identical `mongostore` symbol). +3. `rm -rf internal/redisstore`; `go build ./...` and fix remaining references. +4. Build the in-test fake `Store`; swap poller/bot tests onto it. Port `store_test.go` to the testcontainers harness under `//go:build integration`. +5. `go mod tidy`; confirm Redis deps gone (`grep redis go.mod` empty). +6. `go test ./...` (no tag) — must pass **without Docker** (poller/bot on fakes, integration suite excluded). `go vet ./...`. Optionally `go build -tags=integration ./...` to confirm the gated suite compiles. + +## Success Criteria + +- [ ] `go build ./...` clean; no `redisstore` references; `internal/redisstore` deleted. +- [ ] `grep -ri redis` finds no code/config/doc references (outside this plan). +- [ ] `go test ./...` (no tag) green **without Docker**; gated `mongostore` suite compiles under `-tags=integration` (run later by the user). +- [ ] `go mod tidy` removed `go-redis` + `miniredis`; added mongo driver + testcontainers (test-only). + +## Risk Assessment + +- Risk: CI lacks Docker for testcontainers. Resolved by decision: integration suite is behind `//go:build integration` and excluded from default `go test`; user runs it manually later. Document the `-tags=integration` + Docker requirement in README. +- Risk: in-test fake `Store` drifts from real `mongostore` semantics (the fake passes but Mongo behaves differently). Mitigation: the gated integration suite is the real-behavior backstop; user must run it before trusting the cutover in production. +- Risk: TTL behavior not directly assertable (sweep ~60s). Mitigation: assert the TTL **index exists** with `ExpireAfterSeconds==604800` rather than waiting for expiry; trust MongoDB to enforce. +- Risk: hidden semantic drift between fake `Store` and real `mongostore` in poller/bot tests. Mitigation: keep at least the `mongostore` suite exercising real behavior end-to-end for the store contract. + +## Post-cutover docs + +Update the three docs that reference Redis: `docs/system-architecture.md`, `docs/setup-guide.md`, `docs/component-id-monitoring-report.md` (verify each via `grep -li redis docs/`). Add a README note that switching from a prior Redis deploy starts from empty state (no migration; subscribers re-`/start`). diff --git a/plans/260626-1057-redis-to-mongodb/plan.md b/plans/260626-1057-redis-to-mongodb/plan.md new file mode 100644 index 0000000..a0ed9df --- /dev/null +++ b/plans/260626-1057-redis-to-mongodb/plan.md @@ -0,0 +1,73 @@ +--- +title: "Migrate datastore from Redis to MongoDB" +description: "" +status: complete +priority: P2 +branch: "main" +tags: [] +blockedBy: [] +blocks: [] +created: "2026-06-26T04:22:22.185Z" +createdBy: "ck:plan" +source: skill +--- + +# Migrate datastore from Redis to MongoDB + +## Overview + +Replace Redis with MongoDB as the bot's datastore. No data migration: cutover is a clean reseed (first poll re-seeds state, subscribers re-issue `/start`). Motivation: team already operates MongoDB; goal is removing the Redis dependency to consolidate on one datastore. Hosting (validated): **managed Atlas for both environments** — one cluster with two databases (production + development) selected via `MONGODB_DATABASE`. **No local MongoDB service**; both compose files connect to Atlas, differing only by database name. + +**Verdict (Phase 1, full reasoning there):** Suitable and justified. Every Redis op maps cleanly to MongoDB, and the document model actually *simplifies* several multi-key sequences. One real regression to accept: unit tests lose the pure-Go in-memory `miniredis`; MongoDB has no equivalent, so tests need a real `mongod` (testcontainers needs Docker, or `memongo` downloads a binary). CI gets slower and gains a Docker/binary dependency. + +## Approach (straight replace, no dual backend) + +`poller.Store` and `bot.Store` are already interfaces, and the only cross-package coupling is the domain types `redisstore.Subscriber`, `redisstore.PendingComponentEvent`, the `SubscriptionType*` constants, `NewSubscriber`, and `DefaultSubscriptionTypes` (~90 references, type *names* not bodies). Plan: create `internal/mongostore` exporting the **same type/constant names**, move the storage-agnostic domain logic verbatim, reimplement only the client-touching methods against MongoDB, swap consumer qualifiers `redisstore.` → `mongostore.`, then delete `internal/redisstore`. YAGNI: no pluggable-backend abstraction since only one backend is wanted. + +## Phases + +| Phase | Name | Status | +|-------|------|--------| +| 1 | [Suitability & Decision](./phase-01-suitability-decision.md) | Done | +| 2 | [Store Package & Connection](./phase-02-store-package-connection.md) | Done | +| 3 | [Implement Store Methods](./phase-03-implement-store-methods.md) | Done | +| 4 | [Config Wiring & Deployment](./phase-04-config-wiring-deployment.md) | Done | +| 5 | [Cutover & Tests](./phase-05-cutover-tests.md) | Done | + +## Acceptance Criteria + +- [x] Bot builds and runs with no `go-redis` / `miniredis` dependency. +- [x] All store operations (subscribers, settings, component statuses, pending component events, incident-update versions, delivery dedup w/ 7-day expiry, initialized flag, telegram offset) work against MongoDB. +- [x] `poller.Store` and `bot.Store` interfaces satisfied by `mongostore`; existing poller/bot/telegram tests pass against the new backend. +- [x] Config, `.env.example`, compose files, and README reference `MONGODB_URI`/`MONGODB_DATABASE`; no Redis references remain. +- [x] TTL index on delivery collection enforces 7-day expiry; conditional "update-only-if-subscribed" preserved via `MatchedCount`. + +## Dependencies + +None (no cross-plan dependencies; no in-repo data migration). + +## Validation Log + +### Session 1 — 2026-06-26 + +**Verification Results** +- Claims checked: 7 anchors + driver facts (Full tier, 5 phases) +- Verified: 7 | Failed: 0 | Unverified: 0 +- Evidence: `poller.go:17` & `bot.go:25` (Store interfaces), `subscriber_settings.go:16` (Lua script), `checkpoint.go:137` (`Expire` 7d), `telegram/client.go:108` (`redisstore.Subscriber` param), 92 `redisstore.` refs across `internal`+`cmd`, `.env` gitignored. Driver = `go.mongodb.org/mongo-driver/v2` (v2.7.0). + +**Decisions confirmed** +1. **Test backend** → testcontainers-go chosen, but **gated/deferred**: integration suite behind a build tag (`//go:build integration`), NOT run in Phase 5. Default `go test ./...` must stay Docker-free; user runs the integration suite later. (Phase 5) +2. **Hosting** → Atlas for both envs; **one cluster, two databases** (prod + dev) selected by `MONGODB_DATABASE`. **No local `mongo` service** in any compose file. (Phase 4) +3. **Cutover** → clean break: remove `REDIS_URL`, no compat shim, state resets (subscribers re-`/start`, checkpoints reseed). (Phase 4/5) +4. **Poller/bot tests** → in-test fake `Store` (Docker-free); real-backend coverage stays in the gated `mongostore` suite. (Phase 5) + +### Whole-Plan Consistency Sweep +- Re-read `plan.md` + all 5 phase files after propagation. Local-`mongo`-service references removed (Overview, Phase 4); test-gating note added (Phase 5). No stale "local mongo" / "docker-compose mongo service" terms remain. Verdict (Phase 1) unaffected by these deployment/test decisions. **0 unresolved contradictions.** + +### Implementation — 2026-06-26 (all phases complete) +- New `internal/mongostore` (store/subscriber/subscriber_normalization/checkpoint/indexes) on driver `go.mongodb.org/mongo-driver/v2 v2.7.0`; `internal/redisstore` deleted; `go-redis`+`miniredis` removed via `go mod tidy`. +- Consumers (`poller`, `bot`, `telegram`) swapped by qualifier only (`redisstore.`→`mongostore.`); identical exported names. Interface conformance enforced at compile time via `main.go` wiring (no `var _` assertion — would cycle). +- `config` now requires `MONGODB_URI`, defaults `MONGODB_DATABASE=openai_status_bot`; `main.go` connects/pings/`EnsureIndexes`. `.env.example`, both compose files (bot-only, Atlas, dev/prod by DB name), README, and 3 docs updated; no Redis refs remain. +- Tests: poller/bot/config on Docker-free fakes; `mongostore` has pure-logic unit tests (default) + gated `//go:build integration` testcontainers suite (run via `go test -tags=integration ./internal/mongostore/...`). `go build`/`go test`/`go vet ./...` all green without Docker. +- `code-reviewer`: DONE, 0 Critical/High/Medium; one cosmetic note (subscriber `threadID` null-vs-omit) applied. +- Deferred: user runs the gated integration suite with Docker before trusting the cutover in production.