refactor(storage): replace KVStore with generic typed DocStore[T]

Delete the byte-oriented KVStore/VersionedStore abstraction and the
DynamoDB/memory KV backends. Add a generic typed store (DocStore[T] with
Provider/Collection/Typed) persisting each value as a flattened native
Mongo document (storedDoc[T] via bson inline) — no value envelope.

- MongoDB is the only runtime backend; memory kept for tests/local.
- All modules + deploynotify use typed stores; persisted structs carry
  bson tags == json names (incl. nested lolschedule/wordle types).
- lolschedule wraps its array/scalar values in named structs.
- migrate-dynamo-to-mongo writes the flattened shape via Typed[bson.M]
  with wrap rules; Scan/--dry-run/--verify retained.

Verified: go vet/build clean; full go test green hermetically and
in-container vs real Mongo 7 + DynamoDB Local (storage integration +
migrator e2e).
This commit is contained in:
tiennm99 committed 2026-06-28 18:02:11 +07:00
1 parent b4910c2289
commit 15a3239ab1
104 files changed
+4036 -2712

No files matched your search

+5 -3
View File
@@ -31,9 +31,11 @@ test: ## Unit tests (no emulator required)
# Run DynamoDB integration tests against DynamoDB Local.
# Override DDB_PORT if 8001 is taken on your host.
DDB_PORT ?= 8001
test-dynamodb: dynamodb-local ## Run DynamoDB tests against DynamoDB Local
DYNAMODB_LOCAL_URL=http://localhost:$(DDB_PORT) LOG_LEVEL=error \
go test -race -count=1 ./internal/storage/...
test-dynamodb: dynamodb-local mongo-local ## Run the DynamoDB→Mongo migrator e2e against local emulators
DYNAMODB_LOCAL_URL=http://localhost:$(DDB_PORT) \
MONGODB_TEST_URL=mongodb://127.0.0.1:$(MONGO_PORT) \
MONGO_DATABASE=migrate_test LOG_LEVEL=error \
go test -race -count=1 ./cmd/migrate-dynamo-to-mongo/...
# Run MongoDB integration tests against a local Mongo container.
# Override MONGO_PORT if 27017 is taken on your host.
+6 -6
View File
@@ -28,7 +28,7 @@ internal/server/ HTTP routes (/ health, /cron/{name} manual trigger)
internal/telegram/ Telegram long-polling bot wrapper
internal/cron/ in-process cron scheduler (replaces EventBridge)
internal/modules/ Module framework, registry, dispatchers, modules
internal/storage/ KVProvider interface; memory + dynamodb + mongodb (values stored as native BSON documents)
internal/storage/ typed DocStore[T] (Provider + Typed); mongodb runtime + memory (tests). Values persist as flattened native BSON root documents
internal/ai/ Gemini client (used by twentyq)
docker-compose.yml Coolify self-host stack (single bot service)
docs/deploy-coolify-selfhosted.md Self-host onboarding + cutover runbook
@@ -58,12 +58,11 @@ MONGO_DATABASE=miti99bot_dev \
go run ./cmd/server
```
For storage integration tests (each skips when its emulator env var is unset):
For integration tests (each skips when its emulator env var is unset):
```sh
make dynamodb-local # docker run amazon/dynamodb-local on :8001
make test-dynamodb # internal/storage tests against DynamoDB Local
make mongo-local # docker run mongo:7 on :27017
make test-mongo # internal/storage tests against local MongoDB
make test-mongo # internal/storage typed-store tests against local MongoDB
make test-dynamodb # DynamoDB→Mongo migrator e2e against DynamoDB Local + local Mongo
```
## Test
@@ -71,7 +70,8 @@ make test-mongo # internal/storage tests against local MongoDB
```sh
make vet # go vet
make test # full unit suite (no emulator)
make test-dynamodb # storage tests against DynamoDB Local (requires Docker)
make test-mongo # typed-store integration tests against local Mongo (requires Docker)
make test-dynamodb # migrator e2e against DynamoDB Local + Mongo (requires Docker)
```
## Deploy
+14 -9
View File
@@ -7,20 +7,25 @@ app writes, then verifies per-module parity. Idempotent and re-runnable.
## What it does
- Full-table `Scan` of DynamoDB (the table is a small KV) → group by `pk`.
- Writes each item through `storage.MongoKVStore.Put`, so the `value` encoding
is **byte-identical** to what the running bot writes (BSON binary). This is
what keeps re-runs idempotent and future CompareAndSwap correct — a raw
`UpdateOne` that stored `value` as a BSON string would silently break both.
- `Put` runs `validateKey` on each sort key and upserts by `_id`, so a re-run
produces no duplicates and an invalid key fails loud (no key the app's read
path can't load is ever written).
- Writes each item through the typed Mongo store (`storage.Typed[bson.M]`) as a
**flattened native document** — the value's JSON fields are hoisted to the
document root alongside `_id`/`version`/`updatedAt`, with no `value` envelope.
This is the exact shape the running bot writes, so the app reads migrated docs
directly. Integers keep int64 fidelity (decoded with `UseNumber`).
- The two non-object values are wrapped into named root fields to match the
module's typed shape: lolschedule `subscribers` (a JSON array) → `{subscribers: [...]}`
and `daily_push:last_date` (a bare date string) → `{date: "..."}`. Any other
non-object value fails loud so a missing wrap rule is obvious (see `encode.go`).
- Writing through the store validates the module/collection name and key and
upserts by `_id`, so a re-run produces no duplicates and bad input fails loud.
| DynamoDB | MongoDB |
|---|---|
| `pk` (module name) | collection name |
| `sk` (user key) | document `_id` |
| `value` (JSON string) | `value` field (BSON binary, same bytes) |
| `updatedAt` | restamped to migration time (write-only field; nothing reads it) |
| `value` (JSON object) | payload fields hoisted to the document root (no `value` field) |
| `value` (array/scalar, lolschedule) | wrapped in a named root field (`subscribers` / `date`) |
| — | `version` = 1, `updatedAt` = migration time (write-only; nothing reads it) |
## Usage
+109
View File
@@ -0,0 +1,109 @@
package main
import (
"bytes"
"encoding/json"
"fmt"
"go.mongodb.org/mongo-driver/v2/bson"
)
// reservedRootFields are owned by the storage layer (see internal/storage); a
// flattened object payload must not collide with them.
var reservedRootFields = map[string]bool{"_id": true, "version": true, "updatedAt": true}
// wrapRule names the root field a non-object DynamoDB value must be wrapped in
// to match the typed Mongo store's named-struct shape. rawString treats the
// DynamoDB value as a bare (non-JSON) string; otherwise it is JSON-decoded.
type wrapRule struct {
field string
rawString bool
}
// wrapRules covers the only KV entries whose value is not a JSON object. They
// mirror the named-struct wrappers the lolschedule module persists:
// - subscribers: a JSON array → {subscribers: [...]}
// - daily_push:last_date: a bare date string → {date: "..."}
//
// Any other non-object value fails loud in payloadForItem so a missing rule is
// obvious rather than silently dropped.
var wrapRules = map[[2]string]wrapRule{
{"lolschedule", "subscribers"}: {field: "subscribers"},
{"lolschedule", "daily_push:last_date"}: {field: "date", rawString: true},
}
// payloadForItem converts one migrated KV row's value into the flattened payload
// map the typed Mongo store stores at the document root. The store adds _id,
// version, and updatedAt; this returns only the payload fields.
func payloadForItem(module, key string, value []byte) (bson.M, error) {
if rule, ok := wrapRules[[2]string{module, key}]; ok {
if rule.rawString {
return bson.M{rule.field: string(value)}, nil
}
decoded, err := decodeJSONNumber(value)
if err != nil {
return nil, fmt.Errorf("%s/%s: decode value: %w", module, key, err)
}
return bson.M{rule.field: decoded}, nil
}
decoded, err := decodeJSONNumber(value)
if err != nil {
return nil, fmt.Errorf("%s/%s: decode value: %w", module, key, err)
}
obj, ok := decoded.(bson.M)
if !ok {
return nil, fmt.Errorf("%s/%s: value is not a JSON object (type %T) and has no wrap rule — add one to wrapRules", module, key, decoded)
}
for k := range obj {
if reservedRootFields[k] {
return nil, fmt.Errorf("%s/%s: payload key %q collides with a reserved root field — add a wrap rule", module, key, k)
}
}
return obj, nil
}
// decodeJSONNumber decodes JSON into a BSON-native value, preserving integral
// numbers as int64 (UseNumber) so migrated numbers keep int64 fidelity, matching
// the live store's value codec.
func decodeJSONNumber(value []byte) (any, error) {
dec := json.NewDecoder(bytes.NewReader(value))
dec.UseNumber()
var v any
if err := dec.Decode(&v); err != nil {
return nil, err
}
if dec.More() {
return nil, fmt.Errorf("trailing data after JSON value")
}
return numberToBSON(v), nil
}
// numberToBSON walks a json.Unmarshal(UseNumber) tree into bson.M/bson.A,
// converting json.Number to int64 when integral, else float64.
func numberToBSON(v any) any {
switch t := v.(type) {
case map[string]any:
m := make(bson.M, len(t))
for k, val := range t {
m[k] = numberToBSON(val)
}
return m
case []any:
a := make(bson.A, len(t))
for i, val := range t {
a[i] = numberToBSON(val)
}
return a
case json.Number:
if i, err := t.Int64(); err == nil {
return i
}
if f, err := t.Float64(); err == nil {
return f
}
return t.String()
default:
return v
}
}
@@ -0,0 +1,87 @@
package main
import (
"testing"
"go.mongodb.org/mongo-driver/v2/bson"
)
func TestPayloadForItem_Object(t *testing.T) {
got, err := payloadForItem("coin", "user:1", []byte(`{"bal":100,"meta":{"createdAt":5}}`))
if err != nil {
t.Fatalf("payloadForItem: %v", err)
}
if got["bal"] != int64(100) {
t.Errorf("bal = %v (%T), want int64(100)", got["bal"], got["bal"])
}
if _, ok := got["value"]; ok {
t.Error("payload must not contain a 'value' envelope field")
}
meta, ok := got["meta"].(bson.M)
if !ok || meta["createdAt"] != int64(5) {
t.Errorf("nested meta = %v, want {createdAt: int64(5)}", got["meta"])
}
}
func TestPayloadForItem_LolscheduleSubscribers(t *testing.T) {
got, err := payloadForItem("lolschedule", "subscribers", []byte(`[{"chatId":1},{"chatId":2}]`))
if err != nil {
t.Fatalf("payloadForItem: %v", err)
}
arr, ok := got["subscribers"].(bson.A)
if !ok || len(arr) != 2 {
t.Fatalf("subscribers = %v (%T), want 2-element bson.A", got["subscribers"], got["subscribers"])
}
}
func TestPayloadForItem_LolscheduleLastPushRawString(t *testing.T) {
// last-push date is stored as a bare (non-JSON) string in DynamoDB.
got, err := payloadForItem("lolschedule", "daily_push:last_date", []byte(`2026-06-28`))
if err != nil {
t.Fatalf("payloadForItem: %v", err)
}
if got["date"] != "2026-06-28" {
t.Errorf("date = %v, want 2026-06-28", got["date"])
}
}
func TestPayloadForItem_UnknownNonObjectFailsLoud(t *testing.T) {
if _, err := payloadForItem("coin", "user:1", []byte(`"a-bare-string"`)); err == nil {
t.Error("non-object value with no wrap rule must fail loud")
}
if _, err := payloadForItem("coin", "user:1", []byte(`[1,2,3]`)); err == nil {
t.Error("array value with no wrap rule must fail loud")
}
}
func TestPayloadForItem_ReservedKeyCollision(t *testing.T) {
if _, err := payloadForItem("coin", "user:1", []byte(`{"version":7}`)); err == nil {
t.Error("payload key colliding with reserved root field must fail loud")
}
}
// TestPayloadForItem_CamelCaseFidelity guards the invariant that lets migration
// work: the migrator preserves the original (camelCase) JSON keys, so a typed
// store struct must declare bson tags matching those names. A struct whose bson
// tags drifted to the driver's lowercased default would read these back empty.
func TestPayloadForItem_CamelCaseFidelity(t *testing.T) {
payload, err := payloadForItem("lolschedule", "events", []byte(`{"startTime":"t","gameWins":3,"blockName":"Week 1"}`))
if err != nil {
t.Fatalf("payloadForItem: %v", err)
}
raw, err := bson.Marshal(payload)
if err != nil {
t.Fatalf("bson.Marshal: %v", err)
}
var got struct {
StartTime string `bson:"startTime"`
GameWins int `bson:"gameWins"`
BlockName string `bson:"blockName"`
}
if err := bson.Unmarshal(raw, &got); err != nil {
t.Fatalf("bson.Unmarshal: %v", err)
}
if got.StartTime != "t" || got.GameWins != 3 || got.BlockName != "Week 1" {
t.Fatalf("camelCase round-trip lost data: %+v", got)
}
}
+20 -14
View File
@@ -12,16 +12,18 @@
// exactly `dynamodb:Scan` on the table ARN (nothing else, no write actions).
// For local testing, set DYNAMODB_LOCAL_URL to point at DynamoDB Local.
//
// - default: scan + write every item through MongoKVStore.Put (same value
// encoding as the live app — JSON objects stored as native BSON), then
// report per-module counts.
// - default: scan + write every item through the typed Mongo store as a
// flattened native document (payload fields hoisted to the root, no `value`
// envelope) — the exact shape the live app writes — then report per-module
// counts.
// - --dry-run: scan + report counts, write nothing.
// - --verify: tally DynamoDB per pk via Scan vs Mongo CountDocuments per
// collection; print a table and exit non-zero on any mismatch.
//
// Note: writing through Put gives each doc the app's native-BSON value encoding
// and a fresh updatedAt/version; nothing in the app reads updatedAt (write-only
// today), and --verify compares counts, so this is intentional and harmless.
// Note: each migrated doc gets a fresh updatedAt and version=1; nothing in the
// app reads updatedAt, and --verify compares counts, so this is intentional and
// harmless. The two non-object values (lolschedule subscribers array and
// last-push date) are wrapped into named root fields (see encode.go).
package main
import (
@@ -142,11 +144,13 @@ func sortedKeys(m map[string]int) []string {
}
// runMigrate scans the table, then (unless dry-run) writes every item through
// MongoKVStore.Put so the value uses the same native-BSON encoding the live app
// writes — guaranteeing idempotent re-runs and correct version-based CAS. Put
// also runs validateKey on each sk and upserts by _id, so a re-run produces no
// duplicates and an invalid key fails loud rather than writing data
// the app's read path cannot load.
// the typed Mongo store as a flattened native document (payload fields hoisted
// to the root, no `value` envelope) — the exact shape the live app writes.
// payloadForItem wraps the two non-object values (lolschedule subscribers array
// and last-push date) into named-struct fields. Writing through
// storage.Typed[bson.M] reuses the store's module/key validation and version
// semantics, so a re-run produces no duplicates (upsert by _id) and an invalid
// module name or key fails loud rather than writing data the app cannot load.
func runMigrate(ctx context.Context, ddb *dynamodb.Client, mdb *mongo.Database, table string, dryRun bool) error {
items, err := scanTable(ctx, ddb, table)
if err != nil {
@@ -160,9 +164,11 @@ func runMigrate(ctx context.Context, ddb *dynamodb.Client, mdb *mongo.Database,
if dryRun {
continue
}
// For invalid module names For returns a store that errors on Put;
// Put validates the key. Either way an error aborts loudly.
if err := provider.For(it.pk).Put(ctx, it.sk, it.value); err != nil {
payload, err := payloadForItem(it.pk, it.sk, it.value)
if err != nil {
return err
}
if err := storage.Typed[bson.M](provider.Collection(it.pk)).Put(ctx, it.sk, payload); err != nil {
return fmt.Errorf("put %s/%s: %w", it.pk, it.sk, err)
}
}
+6 -4
View File
@@ -10,6 +10,7 @@ import (
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"github.com/tiennm99/miti99bot/internal/storage"
@@ -95,15 +96,16 @@ func TestMigrateAndVerify(t *testing.T) {
t.Fatalf("runVerify: %v", err)
}
// Spot-check byte-identical round trip through the app's read path.
// Spot-check the migrated value round-trips through the typed store as a
// flattened native doc: bal hoisted to the root, preserved as int64.
db := mongoDatabase(t, ctx, mongoURL, mongoDB)
provider := storage.NewMongoProvider(db)
got, err := provider.For("coin").Get(ctx, "user:1")
got, _, err := storage.Typed[bson.M](provider.Collection("coin")).Get(ctx, "user:1")
if err != nil {
t.Fatalf("Get migrated value: %v", err)
}
if string(got) != `{"bal":100}` {
t.Errorf("migrated value = %q, want %q", got, `{"bal":100}`)
if got["bal"] != int64(100) {
t.Errorf("migrated value bal = %v (%T), want int64(100)", got["bal"], got["bal"])
}
}
+10 -31
View File
@@ -60,10 +60,6 @@ func factories() map[string]modules.Factory {
}
}
// dynamodbInitTimeout caps DynamoDB client construction at startup. Lambda
// has a 10s init phase; we want to leave headroom for module wiring.
const dynamodbInitTimeout = 5 * time.Second
// mongodbInitTimeout caps MongoDB connect+ping at startup (and Disconnect at
// shutdown). Atlas SRV DNS + TLS handshake can take a couple seconds on a cold
// container; 10s leaves headroom without hiding a wedged cluster.
@@ -153,7 +149,7 @@ func main() {
deploynotify.Run(rootCtx, deploynotify.Config{
Bot: b,
KV: provider.For("deploynotify"),
Store: deploynotify.NewStore(provider.Collection("deploynotify")),
OwnerID: cfg.BotOwnerID,
GitSHA: gitSHA,
})
@@ -210,16 +206,16 @@ func main() {
}
// buildProvider picks the storage backend. Selection order:
// 1. Explicit KV_PROVIDER env (memory|dynamodb|mongodb) wins.
// 1. Explicit KV_PROVIDER env (memory|mongodb) wins.
// 2. Auto-detect: MONGO_URL set → mongodb; otherwise memory.
//
// The self-host default is mongodb (just set MONGO_URL + MONGO_DATABASE — no
// KV_PROVIDER needed). dynamodb remains reachable only via an explicit
// KV_PROVIDER, kept for the data migrator and integration tests. The legacy
// firestore backend and the AWS_LAMBDA_FUNCTION_NAME auto-detect are removed.
// KV_PROVIDER needed). The memory backend is for tests and local no-database
// runs (MODULES=). DynamoDB is no longer a runtime backend — it survives only
// as the one-off migration source (cmd/migrate-dynamo-to-mongo).
//
// Returned closer is always non-nil and safe to call exactly once.
func buildProvider(ctx context.Context, cfg config) (storage.KVProvider, func(), error) {
func buildProvider(ctx context.Context, cfg config) (storage.Provider, func(), error) {
backend := strings.ToLower(strings.TrimSpace(cfg.KVProvider))
if backend == "" {
if cfg.MongoURL != "" {
@@ -261,25 +257,10 @@ func buildProvider(ctx context.Context, cfg config) (storage.KVProvider, func(),
log.Info("storage backend", "backend", "mongodb", "database", cfg.MongoDatabase)
return storage.NewMongoProvider(db), closer, nil
case "dynamodb":
if cfg.DynamoDBTable == "" {
return nil, func() {}, errors.New("KV_PROVIDER=dynamodb requires DYNAMODB_TABLE")
}
initCtx, cancel := context.WithTimeout(ctx, dynamodbInitTimeout)
defer cancel()
client, err := storage.NewDynamoDBClient(initCtx, storage.DynamoDBEndpointFromEnv())
if err != nil {
return nil, func() {}, err
}
log.Info("storage backend",
"backend", "dynamodb",
"table", cfg.DynamoDBTable,
"endpoint_override", storage.DynamoDBEndpointFromEnv() != "")
// DynamoDB SDK v2 client has no Close; HTTP client uses the default pool.
return storage.NewDynamoDBProvider(client, cfg.DynamoDBTable), func() {}, nil
default:
return nil, func() {}, fmt.Errorf("unknown KV_PROVIDER %q (want memory|dynamodb|mongodb)", backend)
// DynamoDB is no longer a runtime backend — it survives only as the
// one-off migration source (cmd/migrate-dynamo-to-mongo).
return nil, func() {}, fmt.Errorf("unknown KV_PROVIDER %q (want memory|mongodb)", backend)
}
}
@@ -298,8 +279,7 @@ type config struct {
Modules []string
BotOwnerID int64
AdminUserIDs map[int64]bool
KVProvider string // empty = auto-detect; or "memory"|"dynamodb"|"mongodb"
DynamoDBTable string // required when KVProvider=dynamodb
KVProvider string // empty = auto-detect; or "memory"|"mongodb"
MongoURL string // required when KVProvider=mongodb (Atlas SRV connection string; SECRET — never log)
MongoDatabase string // required when KVProvider=mongodb
TelegramBotTokenParam string
@@ -341,7 +321,6 @@ func loadConfig() config {
BotOwnerID: parseInt64(envMap["OWNER_ID"]),
AdminUserIDs: parseInt64Set(envMap["ADMIN_IDS"]),
KVProvider: envMap["KV_PROVIDER"],
DynamoDBTable: envMap["DYNAMODB_TABLE"],
MongoURL: envMap["MONGO_URL"],
MongoDatabase: envMap["MONGO_DATABASE"],
TelegramBotTokenParam: strings.TrimSpace(envMap["TELEGRAM_BOT_TOKEN_PARAMETER_NAME"]),
+8 -7
View File
@@ -63,13 +63,14 @@ and the `STOCK/COIN/GOLD *_API_URL` overrides (modules use coded defaults).
4. Copy the `mongodb+srv://…` connection string into `MONGO_URL` and put the
db name in `MONGO_DATABASE`.
> Storage layout: one collection per module; each document is
> `{ _id: <user key>, value: <native BSON document>, version, updatedAt }`.
> Values are stored as native BSON (objects/arrays expand and are queryable in
> Compass); non-JSON values (e.g. a date guard) are plain strings. Concurrency
> uses the `version` field (optimistic lock). If you migrated data with an older
> build that stored `value` as a string/blob, re-run the migrator (idempotent)
> so values become native.
> Storage layout: one collection per module; each document is a flattened native
> document — `{ _id: <user key>, ...payload fields, version, updatedAt }` with no
> `value` envelope. Payload fields are hoisted to the document root so they
> expand and are queryable in Compass. The two non-object values are wrapped in a
> named field: lolschedule subscribers under `subscribers` (array) and the daily
> push date under `date`. Concurrency uses the `version` field (optimistic lock);
> `updatedAt` is a BSON Date. The DynamoDB→Mongo migrator writes this shape
> directly, and is idempotent.
## 2. Coolify
+30 -24
View File
@@ -6,7 +6,7 @@
// (see Makefile). An empty gitSHA — e.g. `go run` or a build without the
// ldflags — is treated as a signal to stay silent.
//
// Dedup: a single KV record (key=last_notified_sha) holds the most recently
// Dedup: a single store record (key=last_notified_sha) holds the most recently
// notified SHA. On match we return early; on miss we send first, then write
// — so a transient Telegram failure doesn't permanently silence retries.
package deploynotify
@@ -23,10 +23,10 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
// kvKey is the single KV slot this package owns.
const kvKey = "last_notified_sha"
// storeKey is the single store slot this package owns.
const storeKey = "last_notified_sha"
// defaultTimeout caps the whole Run path (KV read + Telegram send + KV write)
// defaultTimeout caps the whole Run path (store read + Telegram send + store write)
// so a misbehaving network can never block Lambda init past its 10s budget.
const defaultTimeout = 3 * time.Second
@@ -34,7 +34,7 @@ const defaultTimeout = 3 * time.Second
// nil, Run falls back to bot.SendMessage.
type Config struct {
Bot *bot.Bot
KV storage.KVStore
Store storage.DocStore[notifyRecord]
OwnerID int64
GitSHA string
Timeout time.Duration
@@ -43,11 +43,18 @@ type Config struct {
Sender func(ctx context.Context, chatID int64, text string) error
}
// notifyRecord is the KV value shape. At is informational only — useful for
// eyeballing in the DynamoDB console; not consulted by the code path.
// notifyRecord is the store value shape. At is informational only — useful for
// eyeballing in the database console; not consulted by the code path.
type notifyRecord struct {
SHA string `json:"sha"`
At int64 `json:"at"`
SHA string `json:"sha" bson:"sha"`
At int64 `json:"at" bson:"at"`
}
// NewStore builds the typed store this package needs from a module Collection.
// notifyRecord is unexported, so callers cannot name DocStore[notifyRecord]
// directly — this is the seam that lets cmd/server wire the store.
func NewStore(c storage.Collection) storage.DocStore[notifyRecord] {
return storage.Typed[notifyRecord](c)
}
// Run is the entry point. Fire-and-forget — never returns an error and
@@ -64,12 +71,12 @@ func Run(ctx context.Context, cfg Config) {
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
notify, err := shouldNotify(ctx, cfg.KV, cfg.GitSHA)
notify, err := shouldNotify(ctx, cfg.Store, cfg.GitSHA)
if err != nil {
// Treat KV errors as "not notified yet": worst case is one extra
// Treat store errors as "not notified yet": worst case is one extra
// DM on the next cold start, which is far better than going
// silent on every deploy because DynamoDB threw a transient.
log.Warn("deploynotify kv read failed; will attempt send anyway", "err", err)
// silent on every deploy because the store threw a transient error.
log.Warn("deploynotify store read failed; will attempt send anyway", "err", err)
notify = true
}
if !notify {
@@ -80,23 +87,23 @@ func Run(ctx context.Context, cfg Config) {
log.Warn("deploynotify telegram send failed", "err", err, "owner", cfg.OwnerID)
return
}
if err := markNotified(ctx, cfg.KV, cfg.GitSHA); err != nil {
log.Warn("deploynotify kv write failed (owner was notified)", "err", err)
if err := markNotified(ctx, cfg.Store, cfg.GitSHA); err != nil {
log.Warn("deploynotify store write failed (owner was notified)", "err", err)
return
}
log.Info("deploynotify sent", "sha", cfg.GitSHA, "owner", cfg.OwnerID)
}
// skipReason returns a non-empty short string when Run should no-op without
// touching KV or Telegram. Empty string ⇒ proceed.
// touching the store or Telegram. Empty string ⇒ proceed.
func skipReason(cfg Config) string {
switch {
case cfg.GitSHA == "":
return "empty gitSHA (build without -ldflags)"
case cfg.OwnerID == 0:
return "no OWNER_ID configured"
case cfg.KV == nil:
return "no KV configured"
case cfg.Store == nil:
return "no Store configured"
case cfg.Bot == nil && cfg.Sender == nil:
return "no bot or sender configured"
}
@@ -105,9 +112,8 @@ func skipReason(cfg Config) string {
// shouldNotify reports whether sha differs from the last notified value.
// A missing record (ErrNotFound) is treated as "yes, notify".
func shouldNotify(ctx context.Context, kv storage.KVStore, sha string) (bool, error) {
var prev notifyRecord
err := kv.GetJSON(ctx, kvKey, &prev)
func shouldNotify(ctx context.Context, store storage.DocStore[notifyRecord], sha string) (bool, error) {
prev, _, err := store.Get(ctx, storeKey)
if errors.Is(err, storage.ErrNotFound) {
return true, nil
}
@@ -117,9 +123,9 @@ func shouldNotify(ctx context.Context, kv storage.KVStore, sha string) (bool, er
return prev.SHA != sha, nil
}
// markNotified writes the SHA + current timestamp to KV.
func markNotified(ctx context.Context, kv storage.KVStore, sha string) error {
return kv.PutJSON(ctx, kvKey, notifyRecord{
// markNotified writes the SHA + current timestamp to the store.
func markNotified(ctx context.Context, store storage.DocStore[notifyRecord], sha string) error {
return store.Put(ctx, storeKey, notifyRecord{
SHA: sha,
At: time.Now().UTC().UnixMilli(),
})
+37 -32
View File
@@ -9,23 +9,28 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
// newNotifyStore returns a fresh in-memory typed notifyRecord store for tests.
func newNotifyStore() storage.DocStore[notifyRecord] {
return storage.Typed[notifyRecord](storage.NewMemoryProvider().Collection("deploynotify"))
}
func TestShouldNotify_FirstRun(t *testing.T) {
kv := storage.NewMemoryKVStore()
got, err := shouldNotify(context.Background(), kv, "abc123")
store := newNotifyStore()
got, err := shouldNotify(context.Background(), store, "abc123")
if err != nil {
t.Fatalf("err: %v", err)
}
if !got {
t.Errorf("first run with empty KV → want true, got false")
t.Errorf("first run with empty store → want true, got false")
}
}
func TestShouldNotify_SameSHA(t *testing.T) {
kv := storage.NewMemoryKVStore()
if err := markNotified(context.Background(), kv, "abc123"); err != nil {
store := newNotifyStore()
if err := markNotified(context.Background(), store, "abc123"); err != nil {
t.Fatal(err)
}
got, err := shouldNotify(context.Background(), kv, "abc123")
got, err := shouldNotify(context.Background(), store, "abc123")
if err != nil {
t.Fatalf("err: %v", err)
}
@@ -35,11 +40,11 @@ func TestShouldNotify_SameSHA(t *testing.T) {
}
func TestShouldNotify_DifferentSHA(t *testing.T) {
kv := storage.NewMemoryKVStore()
if err := markNotified(context.Background(), kv, "old111"); err != nil {
store := newNotifyStore()
if err := markNotified(context.Background(), store, "old111"); err != nil {
t.Fatal(err)
}
got, err := shouldNotify(context.Background(), kv, "new222")
got, err := shouldNotify(context.Background(), store, "new222")
if err != nil {
t.Fatalf("err: %v", err)
}
@@ -66,10 +71,10 @@ func (r *recorder) send(_ context.Context, chatID int64, text string) error {
}
func TestRun_SkipsWhenSHAEmpty(t *testing.T) {
kv := storage.NewMemoryKVStore()
store := newNotifyStore()
rec := &recorder{}
Run(context.Background(), Config{
KV: kv,
Store: store,
OwnerID: 42,
GitSHA: "",
Sender: rec.send,
@@ -77,17 +82,17 @@ func TestRun_SkipsWhenSHAEmpty(t *testing.T) {
if rec.called {
t.Errorf("empty SHA must not send; got call with text=%q", rec.text)
}
// And no KV write either.
if _, err := kv.Get(context.Background(), kvKey); !errors.Is(err, storage.ErrNotFound) {
t.Errorf("empty SHA must not write KV; got err=%v", err)
// And no store write either.
if _, _, err := store.Get(context.Background(), storeKey); !errors.Is(err, storage.ErrNotFound) {
t.Errorf("empty SHA must not write store; got err=%v", err)
}
}
func TestRun_SkipsWhenNoOwner(t *testing.T) {
kv := storage.NewMemoryKVStore()
store := newNotifyStore()
rec := &recorder{}
Run(context.Background(), Config{
KV: kv,
Store: store,
OwnerID: 0,
GitSHA: "abc123",
Sender: rec.send,
@@ -95,22 +100,22 @@ func TestRun_SkipsWhenNoOwner(t *testing.T) {
if rec.called {
t.Errorf("zero owner must not send")
}
if _, err := kv.Get(context.Background(), kvKey); !errors.Is(err, storage.ErrNotFound) {
t.Errorf("zero owner must not write KV; got err=%v", err)
if _, _, err := store.Get(context.Background(), storeKey); !errors.Is(err, storage.ErrNotFound) {
t.Errorf("zero owner must not write store; got err=%v", err)
}
}
func TestRun_SendsAndPersistsOnFirstRun(t *testing.T) {
kv := storage.NewMemoryKVStore()
store := newNotifyStore()
rec := &recorder{}
Run(context.Background(), Config{
KV: kv,
Store: store,
OwnerID: 42,
GitSHA: "abc123",
Sender: rec.send,
})
if !rec.called {
t.Fatalf("first run with fresh KV must send")
t.Fatalf("first run with fresh store must send")
}
if rec.chatID != 42 {
t.Errorf("chatID = %d, want 42", rec.chatID)
@@ -118,10 +123,10 @@ func TestRun_SendsAndPersistsOnFirstRun(t *testing.T) {
if !strings.Contains(rec.text, "abc123") {
t.Errorf("message %q missing SHA", rec.text)
}
// KV must now hold the SHA so the next Run is silent.
var got notifyRecord
if err := kv.GetJSON(context.Background(), kvKey, &got); err != nil {
t.Fatalf("post-send KV read: %v", err)
// Store must now hold the SHA so the next Run is silent.
got, _, err := store.Get(context.Background(), storeKey)
if err != nil {
t.Fatalf("post-send store read: %v", err)
}
if got.SHA != "abc123" {
t.Errorf("persisted SHA = %q, want abc123", got.SHA)
@@ -129,13 +134,13 @@ func TestRun_SendsAndPersistsOnFirstRun(t *testing.T) {
}
func TestRun_SilentOnSecondRunSameSHA(t *testing.T) {
kv := storage.NewMemoryKVStore()
if err := markNotified(context.Background(), kv, "abc123"); err != nil {
store := newNotifyStore()
if err := markNotified(context.Background(), store, "abc123"); err != nil {
t.Fatal(err)
}
rec := &recorder{}
Run(context.Background(), Config{
KV: kv,
Store: store,
OwnerID: 42,
GitSHA: "abc123",
Sender: rec.send,
@@ -146,10 +151,10 @@ func TestRun_SilentOnSecondRunSameSHA(t *testing.T) {
}
func TestRun_DoesNotPersistOnSendFailure(t *testing.T) {
kv := storage.NewMemoryKVStore()
store := newNotifyStore()
rec := &recorder{err: errors.New("telegram is down")}
Run(context.Background(), Config{
KV: kv,
Store: store,
OwnerID: 42,
GitSHA: "abc123",
Sender: rec.send,
@@ -158,8 +163,8 @@ func TestRun_DoesNotPersistOnSendFailure(t *testing.T) {
t.Fatalf("sender should have been called")
}
// Send failed → SHA must NOT be persisted, so the next cold start retries.
if _, err := kv.Get(context.Background(), kvKey); !errors.Is(err, storage.ErrNotFound) {
t.Errorf("failed send must not write KV; got err=%v", err)
if _, _, err := store.Get(context.Background(), storeKey); !errors.Is(err, storage.ErrNotFound) {
t.Errorf("failed send must not write store; got err=%v", err)
}
}
+1 -1
View File
@@ -2,7 +2,7 @@
// (typically a chat / user / subject identifier) across goroutines.
//
// Why a separate package: every game module needs a per-subject mutex to
// turn KVStore's single-op atomicity into safe Get→mutate→Put. The bot
// turn the store's single-op atomicity into safe Get→mutate→Put. The bot
// dispatcher runs each Telegram update in its own goroutine, so without
// explicit per-subject serialisation two updates to the same game could
// race and drop one write.
+5 -2
View File
@@ -1,11 +1,14 @@
package coin
import "github.com/tiennm99/miti99bot/internal/modules"
import (
"github.com/tiennm99/miti99bot/internal/modules"
"github.com/tiennm99/miti99bot/internal/storage"
)
// New is the coin paper-trading module factory. It is opt-in through MODULES
// and keeps its portfolio state separate from stock and gold modules.
func New(deps modules.Deps) modules.Module {
s := newState(deps.KV)
s := newState(storage.Typed[Portfolio](deps.Store))
return modules.Module{
Commands: []modules.Command{
{
+3 -3
View File
@@ -49,7 +49,7 @@ func (s *state) handleTopup(ctx context.Context, b *bot.Bot, update *models.Upda
return chathelper.Reply(ctx, b, update.Message, "Amount must be a positive finite USD number within the supported range.")
}
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
p.AddUSD(amount)
p.Meta.Invested += amount
return nil
@@ -94,7 +94,7 @@ func (s *state) handleBuy(ctx context.Context, b *bot.Bot, update *models.Update
}
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
var insufficientBalance *float64
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
ok, balance := p.DeductUSD(amount)
if !ok {
insufficientBalance = &balance
@@ -149,7 +149,7 @@ func (s *state) handleSell(ctx context.Context, b *bot.Bot, update *models.Updat
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
var insufficientHeldQty float64
var insufficientHeld bool
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
ok, held := p.DeductAsset(coin.Symbol, qty)
if !ok {
insufficientHeldQty = held
+2 -2
View File
@@ -19,7 +19,7 @@ func TestHandleBuyAcceptsCoinFirstOrder(t *testing.T) {
}
rb.AssertSentText(t, "Bought 0.0002 BTC")
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
if p.USD != 990 || p.Assets["BTC"] != 0.0002 {
t.Fatalf("after coin-first buy = %+v", p)
}
@@ -38,7 +38,7 @@ func TestHandleSellAcceptsCoinFirstOrder(t *testing.T) {
}
rb.AssertSentText(t, "Sold 0.01 BTC")
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
if p.USD != 1000 || len(p.Assets) != 0 {
t.Fatalf("after coin-first sell = %+v", p)
}
+14 -9
View File
@@ -31,9 +31,14 @@ func (f fakePriceFetcher) FetchUSD(_ context.Context, coin CoinSymbol) (CoinPric
return price, nil
}
// newCoinStore returns a fresh in-memory typed portfolio store for tests.
func newCoinStore() Store {
return storage.Typed[Portfolio](storage.NewMemoryProvider().Collection("coin"))
}
func newTestState(prices map[string]CoinPrice, err error) *state {
return &state{
kv: storage.NewMemoryKVStore(),
store: newCoinStore(),
prices: fakePriceFetcher{prices: prices, err: err},
nowFn: func() time.Time { return time.UnixMilli(123) },
}
@@ -100,7 +105,7 @@ func TestHandleTopup(t *testing.T) {
t.Fatalf("handleTopup: %v", err)
}
rb.AssertSentText(t, "Topped up $1,000.00")
p, err := LoadPortfolio(ctx, s.kv, 7, 999)
p, err := LoadPortfolio(ctx, s.store, 7, 999)
if err != nil {
t.Fatalf("LoadPortfolio: %v", err)
}
@@ -119,7 +124,7 @@ func TestHandleBuyAndSell(t *testing.T) {
t.Fatalf("handleBuy: %v", err)
}
rb.AssertSentText(t, "Bought 0.01 BTC")
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
if p.USD != 500 || p.Assets["BTC"] != 0.01 {
t.Fatalf("after buy = %+v", p)
}
@@ -128,7 +133,7 @@ func TestHandleBuyAndSell(t *testing.T) {
t.Fatalf("handleSell: %v", err)
}
rb.AssertSentText(t, "Sold 0.01 BTC")
p, _ = LoadPortfolio(ctx, s.kv, 7, 999)
p, _ = LoadPortfolio(ctx, s.store, 7, 999)
if p.USD != 1000 || len(p.Assets) != 0 {
t.Fatalf("after sell = %+v", p)
}
@@ -205,7 +210,7 @@ func TestHandleSellInsufficientCoinWithHoldings(t *testing.T) {
t.Fatalf("insufficient sell message used ambiguous have wording: %q", text)
}
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
if p.USD != 500 || p.Assets["BTC"] != 0.01 {
t.Fatalf("portfolio mutated on failed sell = %+v", p)
}
@@ -224,7 +229,7 @@ func TestHandleSellRejectsDustQuantity(t *testing.T) {
}
rb.AssertSentText(t, "too small")
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
if p.USD != 999 || len(p.Assets) != 1 {
t.Fatalf("portfolio mutated on dust sell = %+v", p)
}
@@ -242,7 +247,7 @@ func TestHandleBuyRejectsDustQuantity(t *testing.T) {
}
rb.AssertSentText(t, "too small")
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
if p.USD != 1000 || len(p.Assets) != 0 {
t.Fatalf("portfolio mutated on dust buy = %+v", p)
}
@@ -276,7 +281,7 @@ func TestPriceErrorDoesNotMutatePortfolio(t *testing.T) {
t.Fatalf("handleBuy: %v", err)
}
rb.AssertSentText(t, "Could not fetch coin price")
p, err := LoadPortfolio(ctx, s.kv, 7, 999)
p, err := LoadPortfolio(ctx, s.store, 7, 999)
if err != nil {
t.Fatalf("LoadPortfolio: %v", err)
}
@@ -310,5 +315,5 @@ func TestStatsWithAndWithoutPrice(t *testing.T) {
}
func modDepsForTest() modules.Deps {
return modules.Deps{KV: storage.NewMemoryKVStore()}
return modules.Deps{Store: storage.NewMemoryProvider().Collection("coin")}
}
+3 -4
View File
@@ -14,7 +14,6 @@ import (
"github.com/tiennm99/miti99bot/internal/keylock"
"github.com/tiennm99/miti99bot/internal/log"
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
"github.com/tiennm99/miti99bot/internal/storage"
)
type priceFetcher interface {
@@ -22,14 +21,14 @@ type priceFetcher interface {
}
type state struct {
kv storage.KVStore
store Store
prices priceFetcher
locks keylock.Map
nowFn func() time.Time
}
func newState(kv storage.KVStore) *state {
return &state{kv: kv, prices: NewPriceClientFromEnv()}
func newState(store Store) *state {
return &state{store: store, prices: NewPriceClientFromEnv()}
}
func (s *state) now() time.Time {
+19 -41
View File
@@ -2,7 +2,6 @@ package coin
import (
"context"
"encoding/json"
"errors"
"fmt"
"math"
@@ -14,15 +13,18 @@ import (
const coinDustEpsilon = 1e-9
const portfolioUpdateAttempts = 5
// Store is the coin module's typed portfolio store.
type Store = storage.DocStore[Portfolio]
type Portfolio struct {
USD float64 `json:"usd"`
Assets map[string]float64 `json:"assets"`
Meta PortfolioMeta `json:"meta"`
USD float64 `json:"usd" bson:"usd"`
Assets map[string]float64 `json:"assets" bson:"assets"`
Meta PortfolioMeta `json:"meta" bson:"meta"`
}
type PortfolioMeta struct {
Invested float64 `json:"invested"`
CreatedAt int64 `json:"createdAt"`
Invested float64 `json:"invested" bson:"invested"`
CreatedAt int64 `json:"createdAt" bson:"createdAt"`
}
func NewPortfolio(now int64) Portfolio {
@@ -33,42 +35,26 @@ func portfolioKey(userID int64) string {
return "user:" + strconv.FormatInt(userID, 10)
}
func LoadPortfolio(ctx context.Context, kv storage.KVStore, userID int64, now int64) (Portfolio, error) {
var p Portfolio
err := kv.GetJSON(ctx, portfolioKey(userID), &p)
switch {
case err == nil:
p.normalize()
if p.Assets == nil {
p.Assets = map[string]float64{}
}
if p.Meta.CreatedAt == 0 {
p.Meta.CreatedAt = now
}
return p, nil
case errors.Is(err, storage.ErrNotFound):
return NewPortfolio(now), nil
default:
func LoadPortfolio(ctx context.Context, store Store, userID int64, now int64) (Portfolio, error) {
p, _, err := loadPortfolioForUpdate(ctx, store, portfolioKey(userID), now)
if err != nil {
return Portfolio{}, fmt.Errorf("coin: load portfolio %d: %w", userID, err)
}
return p, nil
}
func SavePortfolio(ctx context.Context, kv storage.KVStore, userID int64, p Portfolio) error {
func SavePortfolio(ctx context.Context, store Store, userID int64, p Portfolio) error {
p.normalize()
if err := kv.PutJSON(ctx, portfolioKey(userID), p); err != nil {
if err := store.Put(ctx, portfolioKey(userID), p); err != nil {
return fmt.Errorf("coin: save portfolio %d: %w", userID, err)
}
return nil
}
func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now int64, mutate func(*Portfolio) error) (Portfolio, error) {
vs, ok := kv.(storage.VersionedStore)
if !ok {
return Portfolio{}, fmt.Errorf("coin: storage does not support versioned portfolio updates")
}
func UpdatePortfolio(ctx context.Context, store Store, userID int64, now int64, mutate func(*Portfolio) error) (Portfolio, error) {
key := portfolioKey(userID)
for attempt := 0; attempt < portfolioUpdateAttempts; attempt++ {
p, version, err := loadPortfolioForUpdate(ctx, vs, key, now)
p, version, err := loadPortfolioForUpdate(ctx, store, key, now)
if err != nil {
return Portfolio{}, fmt.Errorf("coin: load portfolio %d: %w", userID, err)
}
@@ -76,11 +62,7 @@ func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now
return p, err
}
p.normalize()
next, err := json.Marshal(p)
if err != nil {
return Portfolio{}, fmt.Errorf("coin: save portfolio %d: json encode: %w", userID, err)
}
if err := vs.PutVersioned(ctx, key, version, next); err == nil {
if err := store.PutVersioned(ctx, key, version, p); err == nil {
return p, nil
} else if !errors.Is(err, storage.ErrConflict) {
return Portfolio{}, fmt.Errorf("coin: save portfolio %d: %w", userID, err)
@@ -89,14 +71,10 @@ func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now
return Portfolio{}, fmt.Errorf("coin: save portfolio %d: %w", userID, storage.ErrConflict)
}
func loadPortfolioForUpdate(ctx context.Context, vs storage.VersionedStore, key string, now int64) (Portfolio, int64, error) {
raw, version, err := vs.GetVersioned(ctx, key)
func loadPortfolioForUpdate(ctx context.Context, store Store, key string, now int64) (Portfolio, int64, error) {
p, version, err := store.Get(ctx, key)
switch {
case err == nil:
var p Portfolio
if err := json.Unmarshal(raw, &p); err != nil {
return Portfolio{}, 0, fmt.Errorf("json decode: %w", err)
}
p.normalize()
if p.Assets == nil {
p.Assets = map[string]float64{}
+13 -14
View File
@@ -10,7 +10,7 @@ import (
)
func TestLoadPortfolioFirstTimeUser(t *testing.T) {
p, err := LoadPortfolio(context.Background(), storage.NewMemoryKVStore(), 42, 123)
p, err := LoadPortfolio(context.Background(), newCoinStore(), 42, 123)
if err != nil {
t.Fatalf("LoadPortfolio: %v", err)
}
@@ -67,32 +67,31 @@ func TestNormalizeAmountSpecialValues(t *testing.T) {
}
}
// conflictOnceStore wraps a real typed store and forces exactly one write
// conflict (after committing a competing value) before delegating, to exercise
// UpdatePortfolio's optimistic-lock retry.
type conflictOnceStore struct {
storage.KVStore
Store
conflicted bool
}
func (s *conflictOnceStore) GetVersioned(ctx context.Context, key string) ([]byte, int64, error) {
return s.KVStore.(storage.VersionedStore).GetVersioned(ctx, key)
}
func (s *conflictOnceStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error {
func (s *conflictOnceStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val Portfolio) error {
if !s.conflicted {
s.conflicted = true
competing := NewPortfolio(1)
competing.AddUSD(10)
if err := s.PutJSON(ctx, key, competing); err != nil {
if err := s.Store.Put(ctx, key, competing); err != nil {
return err
}
return storage.ErrConflict
}
return s.KVStore.(storage.VersionedStore).PutVersioned(ctx, key, expectedVersion, val)
return s.Store.PutVersioned(ctx, key, expectedVersion, val)
}
func TestUpdatePortfolioRetriesAfterWriteConflict(t *testing.T) {
ctx := context.Background()
kv := &conflictOnceStore{KVStore: storage.NewMemoryKVStore()}
got, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
store := &conflictOnceStore{Store: newCoinStore()}
got, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
p.AddUSD(5)
return nil
})
@@ -106,14 +105,14 @@ func TestUpdatePortfolioRetriesAfterWriteConflict(t *testing.T) {
func TestUpdatePortfolioMutateErrorDoesNotPersist(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
_, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
store := newCoinStore()
_, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
return errInsufficientUSD
})
if !errors.Is(err, errInsufficientUSD) {
t.Fatalf("got %v, want errInsufficientUSD", err)
}
if _, err := kv.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
if _, _, err := store.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
t.Fatalf("failed mutate must not persist, Get = %v", err)
}
}
+1 -1
View File
@@ -17,7 +17,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
if !ok {
return chathelper.Reply(ctx, b, update.Message, "Cannot identify user - /coin_stats needs a sender.")
}
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
if err != nil {
log.Error("coin_load_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not load coin portfolio. Try again later.")
+2 -2
View File
@@ -20,9 +20,9 @@ type compositePriceFetcher struct {
// NewCompositePriceFetcherFromEnv builds the production price fetcher using
// only the env-driven VNAppMob client.
func NewCompositePriceFetcherFromEnv(kv storage.KVStore) priceFetcher {
func NewCompositePriceFetcherFromEnv(coll storage.Collection) priceFetcher {
return &compositePriceFetcher{
vnappmob: NewVNAppMobClientFromEnv(kv),
vnappmob: NewVNAppMobClientFromEnv(coll),
}
}
@@ -92,7 +92,7 @@ func TestGoldPriceLines(t *testing.T) {
}
func TestNewCompositePriceFetcherFromEnv(t *testing.T) {
f, ok := NewCompositePriceFetcherFromEnv(storage.NewMemoryKVStore()).(*compositePriceFetcher)
f, ok := NewCompositePriceFetcherFromEnv(storage.NewMemoryProvider().Collection("gold")).(*compositePriceFetcher)
if !ok {
t.Fatalf("expected *compositePriceFetcher, got %T", f)
}
+4 -2
View File
@@ -1,11 +1,13 @@
package gold
import "github.com/tiennm99/miti99bot/internal/modules"
import (
"github.com/tiennm99/miti99bot/internal/modules"
)
// New is the gold paper-trading module factory. It is opt-in through MODULES
// and keeps its portfolio state separate from the stock module.
func New(deps modules.Deps) modules.Module {
s := newState(deps.KV)
s := newState(deps.Store)
return modules.Module{
Commands: []modules.Command{
{
+4 -4
View File
@@ -60,7 +60,7 @@ func (s *state) handleTopup(ctx context.Context, b *bot.Bot, update *models.Upda
}
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
p.AddVND(amount)
p.Meta.Invested += amount
return nil
@@ -98,7 +98,7 @@ func (s *state) handleBuy(ctx context.Context, b *bot.Bot, update *models.Update
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
var insufficientBalance *float64
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
ok, balance := p.DeductVND(cost)
if !ok {
insufficientBalance = &balance
@@ -145,7 +145,7 @@ func (s *state) handleSell(ctx context.Context, b *bot.Bot, update *models.Updat
return chathelper.Reply(ctx, b, update.Message, "Trade value is too large.")
}
var insufficientHeld *float64
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
ok, held := p.DeductLuong(qty)
if !ok {
insufficientHeld = &held
@@ -173,7 +173,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
return chathelper.Reply(ctx, b, update.Message,
"Cannot identify user - /gold_stats needs a sender.")
}
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
if err != nil {
log.Error("gold_load_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not load gold portfolio. Try again later.")
+10 -10
View File
@@ -38,7 +38,7 @@ func (f fakePriceFetcher) FetchPrice(context.Context) (GoldPrice, error) {
func newTestState(price float64, err error) *state {
return &state{
kv: storage.NewMemoryKVStore(),
store: newGoldStore(),
prices: fakePriceFetcher{price: price, err: err},
nowFn: func() time.Time { return time.UnixMilli(123) },
}
@@ -77,7 +77,7 @@ func TestHandleTopup(t *testing.T) {
t.Fatalf("handleTopup: %v", err)
}
rb.AssertSentText(t, "Topped up 5.000.000 VND")
p, err := LoadPortfolio(ctx, s.kv, 7, 999)
p, err := LoadPortfolio(ctx, s.store, 7, 999)
if err != nil {
t.Fatalf("LoadPortfolio: %v", err)
}
@@ -98,7 +98,7 @@ func TestHandleBuyAndSell(t *testing.T) {
t.Fatalf("buy: %v", err)
}
rb.AssertSentText(t, "Bought 1.25 luong gold")
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
if p.Luong != 1.25 || p.VND != 2_500_000 {
t.Fatalf("after buy: %+v", p)
}
@@ -107,7 +107,7 @@ func TestHandleBuyAndSell(t *testing.T) {
t.Fatalf("sell: %v", err)
}
rb.AssertSentText(t, "Sold 1.25 luong gold")
p, _ = LoadPortfolio(ctx, s.kv, 7, 999)
p, _ = LoadPortfolio(ctx, s.store, 7, 999)
if p.Luong != 0 || p.VND != 5_000_000 {
t.Fatalf("after sell: %+v", p)
}
@@ -139,7 +139,7 @@ func TestPriceErrorDoesNotMutatePortfolio(t *testing.T) {
t.Fatalf("buy: %v", err)
}
rb.AssertSentText(t, "Could not fetch gold price")
p, err := LoadPortfolio(ctx, s.kv, 7, 999)
p, err := LoadPortfolio(ctx, s.store, 7, 999)
if err != nil {
t.Fatalf("LoadPortfolio: %v", err)
}
@@ -201,7 +201,7 @@ func (f spreadPriceFetcher) FetchPrice(context.Context) (GoldPrice, error) {
}
func modDepsForTest() modules.Deps {
return modules.Deps{KV: storage.NewMemoryKVStore()}
return modules.Deps{Store: storage.NewMemoryProvider().Collection("gold")}
}
func TestHandlePrice(t *testing.T) {
@@ -239,7 +239,7 @@ func TestHandlePriceFetchError(t *testing.T) {
func TestHandleBuyUsesSellPrice(t *testing.T) {
ctx := context.Background()
s := &state{
kv: storage.NewMemoryKVStore(),
store: newGoldStore(),
prices: spreadPriceFetcher{buy: 90_000_000, sell: 91_000_000},
nowFn: func() time.Time { return time.UnixMilli(123) },
}
@@ -253,7 +253,7 @@ func TestHandleBuyUsesSellPrice(t *testing.T) {
if !strings.Contains(text, "91.000.000 VND/luong") {
t.Fatalf("expected sell price 91.000.000 in %q", text)
}
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
if p.VND != 409_000_000 {
t.Fatalf("balance after buy at 91M: got %v, want 409000000", p.VND)
}
@@ -262,7 +262,7 @@ func TestHandleBuyUsesSellPrice(t *testing.T) {
func TestHandleSellUsesBuyPrice(t *testing.T) {
ctx := context.Background()
s := &state{
kv: storage.NewMemoryKVStore(),
store: newGoldStore(),
prices: spreadPriceFetcher{buy: 90_000_000, sell: 91_000_000},
nowFn: func() time.Time { return time.UnixMilli(123) },
}
@@ -277,7 +277,7 @@ func TestHandleSellUsesBuyPrice(t *testing.T) {
if !strings.Contains(text, "90.000.000 VND/luong") {
t.Fatalf("expected buy price 90.000.000 in %q", text)
}
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
if p.VND != 499_000_000 {
t.Fatalf("balance after sell at 90M: got %v, want 499000000", p.VND)
}
+6 -3
View File
@@ -24,14 +24,17 @@ type priceFetcher interface {
}
type state struct {
kv storage.KVStore
store PortfolioStore
prices priceFetcher
locks keylock.Map
nowFn func() time.Time
}
func newState(kv storage.KVStore) *state {
return &state{kv: kv, prices: NewCompositePriceFetcherFromEnv(kv)}
func newState(coll storage.Collection) *state {
return &state{
store: storage.Typed[Portfolio](coll),
prices: NewCompositePriceFetcherFromEnv(coll),
}
}
func (s *state) now() time.Time {
+19 -38
View File
@@ -2,7 +2,6 @@ package gold
import (
"context"
"encoding/json"
"errors"
"fmt"
"math"
@@ -14,15 +13,18 @@ import (
const goldDustEpsilon = 1e-9
const portfolioUpdateAttempts = 5
// PortfolioStore is the gold module's typed portfolio store.
type PortfolioStore = storage.DocStore[Portfolio]
type Portfolio struct {
VND float64 `json:"vnd"`
Luong float64 `json:"luong"`
Meta PortfolioMeta `json:"meta"`
VND float64 `json:"vnd" bson:"vnd"`
Luong float64 `json:"luong" bson:"luong"`
Meta PortfolioMeta `json:"meta" bson:"meta"`
}
type PortfolioMeta struct {
Invested float64 `json:"invested"`
CreatedAt int64 `json:"createdAt"`
Invested float64 `json:"invested" bson:"invested"`
CreatedAt int64 `json:"createdAt" bson:"createdAt"`
}
func NewPortfolio(now int64) Portfolio {
@@ -33,39 +35,26 @@ func portfolioKey(userID int64) string {
return "user:" + strconv.FormatInt(userID, 10)
}
func LoadPortfolio(ctx context.Context, kv storage.KVStore, userID int64, now int64) (Portfolio, error) {
var p Portfolio
err := kv.GetJSON(ctx, portfolioKey(userID), &p)
switch {
case err == nil:
p.normalize()
if p.Meta.CreatedAt == 0 {
p.Meta.CreatedAt = now
}
return p, nil
case errors.Is(err, storage.ErrNotFound):
return NewPortfolio(now), nil
default:
func LoadPortfolio(ctx context.Context, store PortfolioStore, userID int64, now int64) (Portfolio, error) {
p, _, err := loadPortfolioForUpdate(ctx, store, portfolioKey(userID), now)
if err != nil {
return Portfolio{}, fmt.Errorf("gold: load portfolio %d: %w", userID, err)
}
return p, nil
}
func SavePortfolio(ctx context.Context, kv storage.KVStore, userID int64, p Portfolio) error {
func SavePortfolio(ctx context.Context, store PortfolioStore, userID int64, p Portfolio) error {
p.normalize()
if err := kv.PutJSON(ctx, portfolioKey(userID), p); err != nil {
if err := store.Put(ctx, portfolioKey(userID), p); err != nil {
return fmt.Errorf("gold: save portfolio %d: %w", userID, err)
}
return nil
}
func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now int64, mutate func(*Portfolio) error) (Portfolio, error) {
vs, ok := kv.(storage.VersionedStore)
if !ok {
return Portfolio{}, fmt.Errorf("gold: storage does not support versioned portfolio updates")
}
func UpdatePortfolio(ctx context.Context, store PortfolioStore, userID int64, now int64, mutate func(*Portfolio) error) (Portfolio, error) {
key := portfolioKey(userID)
for attempt := 0; attempt < portfolioUpdateAttempts; attempt++ {
p, version, err := loadPortfolioForUpdate(ctx, vs, key, now)
p, version, err := loadPortfolioForUpdate(ctx, store, key, now)
if err != nil {
return Portfolio{}, fmt.Errorf("gold: load portfolio %d: %w", userID, err)
}
@@ -73,11 +62,7 @@ func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now
return p, err
}
p.normalize()
next, err := json.Marshal(p)
if err != nil {
return Portfolio{}, fmt.Errorf("gold: save portfolio %d: json encode: %w", userID, err)
}
if err := vs.PutVersioned(ctx, key, version, next); err == nil {
if err := store.PutVersioned(ctx, key, version, p); err == nil {
return p, nil
} else if !errors.Is(err, storage.ErrConflict) {
return Portfolio{}, fmt.Errorf("gold: save portfolio %d: %w", userID, err)
@@ -86,14 +71,10 @@ func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now
return Portfolio{}, fmt.Errorf("gold: save portfolio %d: %w", userID, storage.ErrConflict)
}
func loadPortfolioForUpdate(ctx context.Context, vs storage.VersionedStore, key string, now int64) (Portfolio, int64, error) {
raw, version, err := vs.GetVersioned(ctx, key)
func loadPortfolioForUpdate(ctx context.Context, store PortfolioStore, key string, now int64) (Portfolio, int64, error) {
p, version, err := store.Get(ctx, key)
switch {
case err == nil:
var p Portfolio
if err := json.Unmarshal(raw, &p); err != nil {
return Portfolio{}, 0, fmt.Errorf("json decode: %w", err)
}
p.normalize()
if p.Meta.CreatedAt == 0 {
p.Meta.CreatedAt = now
+58 -80
View File
@@ -12,9 +12,13 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
// newGoldStore returns a fresh in-memory typed portfolio store for tests.
func newGoldStore() PortfolioStore {
return storage.Typed[Portfolio](storage.NewMemoryProvider().Collection("gold"))
}
func TestLoadPortfolio_FirstTimeUser(t *testing.T) {
kv := storage.NewMemoryKVStore()
p, err := LoadPortfolio(context.Background(), kv, 42, 123)
p, err := LoadPortfolio(context.Background(), newGoldStore(), 42, 123)
if err != nil {
t.Fatalf("LoadPortfolio: %v", err)
}
@@ -24,15 +28,15 @@ func TestLoadPortfolio_FirstTimeUser(t *testing.T) {
}
func TestSaveAndLoadRoundTrip(t *testing.T) {
kv := storage.NewMemoryKVStore()
store := newGoldStore()
p := NewPortfolio(1)
p.AddVND(5_000_000)
p.AddLuong(1.25)
p.Meta.Invested = 5_000_000
if err := SavePortfolio(context.Background(), kv, 42, p); err != nil {
if err := SavePortfolio(context.Background(), store, 42, p); err != nil {
t.Fatalf("Save: %v", err)
}
got, err := LoadPortfolio(context.Background(), kv, 42, 999)
got, err := LoadPortfolio(context.Background(), store, 42, 999)
if err != nil {
t.Fatalf("Load: %v", err)
}
@@ -71,32 +75,31 @@ func TestNormalizeAmountSpecialValues(t *testing.T) {
}
}
// conflictOnceStore wraps a real typed store and forces exactly one write
// conflict (after committing a competing value) before delegating, to exercise
// UpdatePortfolio's optimistic-lock retry.
type conflictOnceStore struct {
storage.KVStore
PortfolioStore
conflicted bool
}
func (s *conflictOnceStore) GetVersioned(ctx context.Context, key string) ([]byte, int64, error) {
return s.KVStore.(storage.VersionedStore).GetVersioned(ctx, key)
}
func (s *conflictOnceStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error {
func (s *conflictOnceStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val Portfolio) error {
if !s.conflicted {
s.conflicted = true
competing := NewPortfolio(1)
competing.AddVND(10)
if err := s.PutJSON(ctx, key, competing); err != nil {
if err := s.PortfolioStore.Put(ctx, key, competing); err != nil {
return err
}
return storage.ErrConflict
}
return s.KVStore.(storage.VersionedStore).PutVersioned(ctx, key, expectedVersion, val)
return s.PortfolioStore.PutVersioned(ctx, key, expectedVersion, val)
}
func TestUpdatePortfolioRetriesAfterWriteConflict(t *testing.T) {
ctx := context.Background()
kv := &conflictOnceStore{KVStore: storage.NewMemoryKVStore()}
got, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
store := &conflictOnceStore{PortfolioStore: newGoldStore()}
got, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
p.AddVND(5)
return nil
})
@@ -106,7 +109,7 @@ func TestUpdatePortfolioRetriesAfterWriteConflict(t *testing.T) {
if got.VND != 15 {
t.Fatalf("updated portfolio VND = %v, want 15", got.VND)
}
loaded, err := LoadPortfolio(ctx, kv, 7, 1)
loaded, err := LoadPortfolio(ctx, store, 7, 1)
if err != nil {
t.Fatalf("LoadPortfolio: %v", err)
}
@@ -115,53 +118,51 @@ func TestUpdatePortfolioRetriesAfterWriteConflict(t *testing.T) {
}
}
// alwaysConflictStore always returns ErrConflict on PutVersioned to exhaust retries.
type alwaysConflictStore struct {
storage.KVStore
PortfolioStore
attempts int
}
func (s *alwaysConflictStore) GetVersioned(ctx context.Context, key string) ([]byte, int64, error) {
return s.KVStore.(storage.VersionedStore).GetVersioned(ctx, key)
}
func (s *alwaysConflictStore) PutVersioned(context.Context, string, int64, []byte) error {
func (s *alwaysConflictStore) PutVersioned(_ context.Context, _ string, _ int64, _ Portfolio) error {
s.attempts++
return storage.ErrConflict
}
func TestUpdatePortfolioReturnsConflictAfterExhaustingRetries(t *testing.T) {
ctx := context.Background()
kv := &alwaysConflictStore{KVStore: storage.NewMemoryKVStore()}
_, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
store := &alwaysConflictStore{PortfolioStore: newGoldStore()}
_, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
p.AddVND(5)
return nil
})
if !errors.Is(err, storage.ErrConflict) {
t.Fatalf("UpdatePortfolio: got %v, want wrapped ErrConflict", err)
}
if kv.attempts != portfolioUpdateAttempts {
t.Errorf("CAS attempts = %d, want %d", kv.attempts, portfolioUpdateAttempts)
if store.attempts != portfolioUpdateAttempts {
t.Errorf("CAS attempts = %d, want %d", store.attempts, portfolioUpdateAttempts)
}
if _, err := kv.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
if _, _, err := store.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
t.Errorf("exhausted update must not persist anything, Get = %v", err)
}
}
// countingCASStore counts PutVersioned calls and delegates to the underlying store.
type countingCASStore struct {
*storage.MemoryKVStore
PortfolioStore
casCalls int
}
func (s *countingCASStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error {
func (s *countingCASStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val Portfolio) error {
s.casCalls++
return s.MemoryKVStore.PutVersioned(ctx, key, expectedVersion, val)
return s.PortfolioStore.PutVersioned(ctx, key, expectedVersion, val)
}
func TestUpdatePortfolioMutateErrorDoesNotRetryOrPersist(t *testing.T) {
ctx := context.Background()
kv := &countingCASStore{MemoryKVStore: storage.NewMemoryKVStore()}
store := &countingCASStore{PortfolioStore: newGoldStore()}
mutateCalls := 0
_, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
_, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
mutateCalls++
return errInsufficientVND
})
@@ -171,17 +172,17 @@ func TestUpdatePortfolioMutateErrorDoesNotRetryOrPersist(t *testing.T) {
if mutateCalls != 1 {
t.Errorf("mutate calls = %d, want 1 (business errors must not retry)", mutateCalls)
}
if kv.casCalls != 0 {
t.Errorf("CAS calls = %d, want 0 (failed mutate must not write)", kv.casCalls)
if store.casCalls != 0 {
t.Errorf("CAS calls = %d, want 0 (failed mutate must not write)", store.casCalls)
}
if _, err := kv.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
if _, _, err := store.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
t.Errorf("failed mutate must not persist anything, Get = %v", err)
}
}
func TestUpdatePortfolioConcurrentIncrementsLoseNoUpdates(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
store := newGoldStore()
const goroutines = 16
var successes atomic.Int64
var wg sync.WaitGroup
@@ -190,7 +191,7 @@ func TestUpdatePortfolioConcurrentIncrementsLoseNoUpdates(t *testing.T) {
wg.Add(1)
go func() {
defer wg.Done()
_, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
_, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
p.AddVND(1000)
return nil
})
@@ -211,7 +212,7 @@ func TestUpdatePortfolioConcurrentIncrementsLoseNoUpdates(t *testing.T) {
if successes.Load() == 0 {
t.Fatal("no update succeeded")
}
loaded, err := LoadPortfolio(ctx, kv, 7, 1)
loaded, err := LoadPortfolio(ctx, store, 7, 1)
if err != nil {
t.Fatalf("LoadPortfolio: %v", err)
}
@@ -220,61 +221,38 @@ func TestUpdatePortfolioConcurrentIncrementsLoseNoUpdates(t *testing.T) {
}
}
// plainKVStore hides the embedded store's versioned methods to model a backend
// without optimistic-locking support.
type plainKVStore struct {
storage.KVStore
}
func TestUpdatePortfolioFailsFastWhenCASUnsupported(t *testing.T) {
ctx := context.Background()
mutate := func(p *Portfolio) error {
p.AddVND(5)
return nil
}
// A bare non-CAS store is rejected by the capability check.
if _, err := UpdatePortfolio(ctx, &plainKVStore{KVStore: storage.NewMemoryKVStore()}, 7, 1, mutate); err == nil {
t.Error("UpdatePortfolio on non-CAS store: want error, got nil")
}
// A prefixed wrapper around a non-CAS store must fail fast as unsupported,
// not burn retries on a fake conflict.
_, err := UpdatePortfolio(ctx, storage.Prefixed(&plainKVStore{KVStore: storage.NewMemoryKVStore()}, "gold"), 7, 1, mutate)
if !errors.Is(err, errors.ErrUnsupported) {
t.Errorf("got %v, want errors.ErrUnsupported", err)
}
if errors.Is(err, storage.ErrConflict) {
t.Error("missing CAS capability must not be reported as a write conflict")
}
}
func TestStockAndGoldPortfolioKeysDoNotCollide(t *testing.T) {
ctx := context.Background()
provider := storage.NewMemoryProvider()
goldStore := storage.Typed[Portfolio](provider.Collection("gold"))
goldPortfolio := NewPortfolio(1)
goldPortfolio.AddLuong(2)
if err := SavePortfolio(ctx, provider.For("gold"), 7, goldPortfolio); err != nil {
if err := SavePortfolio(ctx, goldStore, 7, goldPortfolio); err != nil {
t.Fatalf("save gold: %v", err)
}
stockStore := storage.Typed[stockmod.Portfolio](provider.Collection("stock"))
stockPortfolio := stockmod.NewPortfolio(1)
stockPortfolio.AddAsset("TCB", 100)
if err := stockmod.SavePortfolio(ctx, provider.For("stock"), 7, stockPortfolio); err != nil {
if err := stockmod.SavePortfolio(ctx, stockStore, 7, stockPortfolio); err != nil {
t.Fatalf("save stock: %v", err)
}
keys, err := provider.Base().List(ctx, "")
// Both stores share the same provider but different collections (module
// isolation); verify each module's key is visible only in its own collection.
goldKeys, err := goldStore.List(ctx, "")
if err != nil {
t.Fatalf("list: %v", err)
t.Fatalf("list gold: %v", err)
}
want := map[string]bool{"gold:user:7": false, "stock:user:7": false}
for _, key := range keys {
if _, ok := want[key]; ok {
want[key] = true
}
stockKeys, err := stockStore.List(ctx, "")
if err != nil {
t.Fatalf("list stock: %v", err)
}
for key, seen := range want {
if !seen {
t.Fatalf("missing raw key %q in %v", key, keys)
}
if len(goldKeys) != 1 || goldKeys[0] != "user:7" {
t.Fatalf("gold keys: got %v, want [user:7]", goldKeys)
}
if len(stockKeys) != 1 || stockKeys[0] != "user:7" {
t.Fatalf("stock keys: got %v, want [user:7]", stockKeys)
}
}
+22 -21
View File
@@ -25,14 +25,20 @@ const (
vnappmobHTTPTimeout = 3 * time.Second // kept under the handler deadline; see chathelper.FetchContext
)
// apiKeyCache is the persisted shape for the cached VNAppMob JWT.
type apiKeyCache struct {
Token string `json:"token" bson:"token"`
Exp int64 `json:"exp" bson:"exp"`
}
// VNAppMobClient fetches Vietnam SJC gold prices from api.vnappmob.com.
// It self-manages a free JWT API key, caching it in KV and refreshing it
// before expiry or when the SJC endpoint returns 403.
// It self-manages a free JWT API key, caching it in the typed store and
// refreshing it before expiry or when the SJC endpoint returns 403.
type VNAppMobClient struct {
HTTP *http.Client
BaseURL string // optional override; default https://api.vnappmob.com
Token string // optional env override (GOLD_VNAPP_API_KEY)
KV storage.KVStore // module-scoped KV store
BaseURL string // optional override; default https://api.vnappmob.com
Token string // optional env override (GOLD_VNAPP_API_KEY)
cache storage.DocStore[apiKeyCache] // module-scoped typed cache store
nowFn func() time.Time
mu sync.Mutex
@@ -42,11 +48,11 @@ type VNAppMobClient struct {
// GOLD_VNAPP_API_KEY from the environment. The API key env var is intended
// for local dev or SSM injection; when empty the client refreshes the key
// automatically via the VNAppMob refresh endpoint.
func NewVNAppMobClientFromEnv(kv storage.KVStore) *VNAppMobClient {
func NewVNAppMobClientFromEnv(coll storage.Collection) *VNAppMobClient {
return &VNAppMobClient{
BaseURL: strings.TrimSpace(os.Getenv("GOLD_VNAPP_API_URL")),
Token: strings.TrimSpace(os.Getenv("GOLD_VNAPP_API_KEY")),
KV: kv,
cache: storage.Typed[apiKeyCache](coll),
}
}
@@ -161,8 +167,8 @@ type httpStatusError struct {
func (e *httpStatusError) Error() string { return e.msg }
// getKey returns a valid API key, refreshing from KV or the remote endpoint
// when the current key is missing or close to expiry.
// getKey returns a valid API key, refreshing from the typed cache store or the
// remote endpoint when the current key is missing or close to expiry.
func (c *VNAppMobClient) getKey(ctx context.Context) (string, error) {
if c.Token != "" {
return c.Token, nil
@@ -171,11 +177,8 @@ func (c *VNAppMobClient) getKey(ctx context.Context) (string, error) {
c.mu.Lock()
defer c.mu.Unlock()
var cached struct {
Token string `json:"token"`
Exp int64 `json:"exp"`
}
if err := c.KV.GetJSON(ctx, vnappmobKeyCacheKey, &cached); err == nil {
cached, _, err := c.cache.Get(ctx, vnappmobKeyCacheKey)
if err == nil {
if cached.Token != "" && !c.isExpired(cached.Exp) {
return cached.Token, nil
}
@@ -187,8 +190,9 @@ func (c *VNAppMobClient) getKey(ctx context.Context) (string, error) {
return "", err
}
// Re-read from KV to get the freshly stored key.
if err := c.KV.GetJSON(ctx, vnappmobKeyCacheKey, &cached); err != nil {
// Re-read from cache to get the freshly stored key.
cached, _, err = c.cache.Get(ctx, vnappmobKeyCacheKey)
if err != nil {
return "", fmt.Errorf("vnappmob: read refreshed key: %w", err)
}
if cached.Token == "" {
@@ -197,7 +201,7 @@ func (c *VNAppMobClient) getKey(ctx context.Context) (string, error) {
return cached.Token, nil
}
// refreshKey acquires a new JWT from VNAppMob and stores it in KV.
// refreshKey acquires a new JWT from VNAppMob and stores it in the typed cache.
// It is safe to call concurrently; last-write-wins is acceptable because all
// callers receive a valid key.
func (c *VNAppMobClient) refreshKey(ctx context.Context) error {
@@ -256,10 +260,7 @@ func (c *VNAppMobClient) refreshKeyLocked(ctx context.Context) error {
// Store anyway; next call will refresh if expiry cannot be verified.
}
if err := c.KV.PutJSON(ctx, vnappmobKeyCacheKey, struct {
Token string `json:"token"`
Exp int64 `json:"exp"`
}{Token: token, Exp: exp}); err != nil {
if err := c.cache.Put(ctx, vnappmobKeyCacheKey, apiKeyCache{Token: token, Exp: exp}); err != nil {
return fmt.Errorf("vnappmob: cache key: %w", err)
}
log.Info("vnappmob_key_refreshed", "exp", exp)
+13 -12
View File
@@ -16,11 +16,11 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
func newTestVNAppMobClient(srv *httptest.Server, kv storage.KVStore) *VNAppMobClient {
func newTestVNAppMobClient(srv *httptest.Server, coll storage.Collection) *VNAppMobClient {
return &VNAppMobClient{
HTTP: srv.Client(),
BaseURL: srv.URL,
KV: kv,
cache: storage.Typed[apiKeyCache](coll),
nowFn: func() time.Time { return time.Unix(1000, 0) },
}
}
@@ -33,6 +33,10 @@ func makeJWT(exp int64) string {
return h + "." + p + ".dummy-signature"
}
func newTestColl() storage.Collection {
return storage.NewMemoryProvider().Collection("gold")
}
func TestJWTExp(t *testing.T) {
now := time.Unix(1000, 0)
tests := []struct {
@@ -114,8 +118,7 @@ func TestRefreshKey(t *testing.T) {
}))
defer srv.Close()
kv := storage.NewMemoryKVStore()
c := newTestVNAppMobClient(srv, kv)
c := newTestVNAppMobClient(srv, newTestColl())
key, err := c.getKey(context.Background())
if err != nil {
@@ -152,7 +155,7 @@ func TestRefreshKey_ObjectWrapper(t *testing.T) {
}))
defer srv.Close()
c := newTestVNAppMobClient(srv, storage.NewMemoryKVStore())
c := newTestVNAppMobClient(srv, newTestColl())
key, err := c.getKey(context.Background())
if err != nil {
t.Fatalf("getKey: %v", err)
@@ -170,7 +173,7 @@ func TestRefreshKey_QuotedString(t *testing.T) {
}))
defer srv.Close()
c := newTestVNAppMobClient(srv, storage.NewMemoryKVStore())
c := newTestVNAppMobClient(srv, newTestColl())
key, err := c.getKey(context.Background())
if err != nil {
t.Fatalf("getKey: %v", err)
@@ -198,8 +201,7 @@ func TestFetchSJCPrice(t *testing.T) {
}))
defer srv.Close()
kv := storage.NewMemoryKVStore()
c := newTestVNAppMobClient(srv, kv)
c := newTestVNAppMobClient(srv, newTestColl())
buy, sell, err := c.FetchSJCPrice(context.Background())
if err != nil {
t.Fatalf("FetchSJCPrice: %v", err)
@@ -232,7 +234,7 @@ func TestFetchSJCPrice_401Or403Refreshes(t *testing.T) {
}))
defer srv.Close()
c := newTestVNAppMobClient(srv, storage.NewMemoryKVStore())
c := newTestVNAppMobClient(srv, newTestColl())
buy, sell, err := c.FetchSJCPrice(context.Background())
if err != nil {
t.Fatalf("FetchSJCPrice: %v", err)
@@ -265,8 +267,7 @@ func TestFetchSJCPrice_FallbackError(t *testing.T) {
}))
defer srv.Close()
kv := storage.NewMemoryKVStore()
c := newTestVNAppMobClient(srv, kv)
c := newTestVNAppMobClient(srv, newTestColl())
_, _, err := c.FetchSJCPrice(context.Background())
if !errors.Is(err, ErrNoGoldPrice) {
t.Fatalf("got %v, want ErrNoGoldPrice", err)
@@ -294,7 +295,7 @@ func TestFetchSJCPrice_InvalidValues(t *testing.T) {
}
}))
defer srv.Close()
c := newTestVNAppMobClient(srv, storage.NewMemoryKVStore())
c := newTestVNAppMobClient(srv, newTestColl())
_, _, err := c.FetchSJCPrice(context.Background())
if err == nil {
t.Fatal("expected error for invalid values")
+17 -16
View File
@@ -12,7 +12,6 @@ import (
"github.com/tiennm99/miti99bot/internal/keylock"
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
"github.com/tiennm99/miti99bot/internal/storage"
)
const newRoundHint = "🆕 Send <code>/loldle</code> or <code>/loldle &lt;champion&gt;</code> to start a new round."
@@ -20,7 +19,9 @@ const newRoundHint = "🆕 Send <code>/loldle</code> or <code>/loldle &lt;champi
// state captures everything a loldle handler needs at runtime. Built once
// per Factory call and shared across the four command closures.
type state struct {
kv storage.KVStore
games GameStore
stats StatsStore
cfg ConfigStore
champions []Champion
locks keylock.Map // serialises Get→mutate→Put per subject
}
@@ -63,7 +64,7 @@ func (s *state) startFreshGame(ctx context.Context, subject string) (*gameState,
Guesses: []string{},
StartedAt: nil,
}
if err := saveGame(ctx, s.kv, subject, g); err != nil {
if err := saveGame(ctx, s.games, subject, g); err != nil {
return nil, err
}
return g, nil
@@ -72,7 +73,7 @@ func (s *state) startFreshGame(ctx context.Context, subject string) (*gameState,
// getOrInitGame loads the active round; if absent (or the saved round has
// already exhausted its budget — defensive), starts a fresh one.
func (s *state) getOrInitGame(ctx context.Context, subject string, maxGuesses int) (*gameState, error) {
existing, err := loadGame(ctx, s.kv, subject)
existing, err := loadGame(ctx, s.games, subject)
if err != nil {
return nil, err
}
@@ -113,7 +114,7 @@ func (s *state) handleLoldle(ctx context.Context, b *bot.Bot, update *models.Upd
defer s.locks.Acquire(subject)()
arg := chathelper.ArgAfterCommand(msg.Text)
maxGuesses, err := getMaxGuesses(ctx, s.kv, subject)
maxGuesses, err := getMaxGuesses(ctx, s.cfg, subject)
if err != nil {
return err
}
@@ -146,7 +147,7 @@ func (s *state) handleLoldle(ctx context.Context, b *bot.Bot, update *models.Upd
if target == nil {
// champions.json was refreshed and the target is gone. Drop the
// round so the next /loldle starts cleanly.
if err := clearGame(ctx, s.kv, subject); err != nil {
if err := clearGame(ctx, s.games, subject); err != nil {
return err
}
return chathelper.ReplyHTML(ctx, b, msg,
@@ -167,11 +168,11 @@ func (s *state) handleLoldle(ctx context.Context, b *bot.Bot, update *models.Upd
switch {
case won:
st, err := recordResult(ctx, s.kv, subject, true)
st, err := recordResult(ctx, s.stats, subject, true)
if err != nil {
return err
}
if err := clearGame(ctx, s.kv, subject); err != nil {
if err := clearGame(ctx, s.games, subject); err != nil {
return err
}
trySendSticker(ctx, b, msg, winStickers)
@@ -181,10 +182,10 @@ func (s *state) handleLoldle(ctx context.Context, b *bot.Bot, update *models.Upd
rendered, flavor, champ, elapsed, st.Streak, len(game.Guesses), maxGuesses, newRoundHint))
case len(game.Guesses) >= maxGuesses:
if _, err := recordResult(ctx, s.kv, subject, false); err != nil {
if _, err := recordResult(ctx, s.stats, subject, false); err != nil {
return err
}
if err := clearGame(ctx, s.kv, subject); err != nil {
if err := clearGame(ctx, s.games, subject); err != nil {
return err
}
trySendSticker(ctx, b, msg, loseStickers)
@@ -192,7 +193,7 @@ func (s *state) handleLoldle(ctx context.Context, b *bot.Bot, update *models.Upd
"%s\n\n❌ Out of guesses. Answer was %s.\n%s", rendered, champ, newRoundHint))
default:
if err := saveGame(ctx, s.kv, subject, game); err != nil {
if err := saveGame(ctx, s.games, subject, game); err != nil {
return err
}
return chathelper.ReplyHTML(ctx, b, msg, fmt.Sprintf(
@@ -212,18 +213,18 @@ func (s *state) handleGiveup(ctx context.Context, b *bot.Bot, update *models.Upd
}
defer s.locks.Acquire(subject)()
existing, err := loadGame(ctx, s.kv, subject)
existing, err := loadGame(ctx, s.games, subject)
if err != nil {
return err
}
if existing == nil {
return chathelper.ReplyHTML(ctx, b, msg, "No active round. "+newRoundHint)
}
if _, err := recordResult(ctx, s.kv, subject, false); err != nil {
if _, err := recordResult(ctx, s.stats, subject, false); err != nil {
return err
}
target := s.findByName(existing.Target)
if err := clearGame(ctx, s.kv, subject); err != nil {
if err := clearGame(ctx, s.games, subject); err != nil {
return err
}
trySendSticker(ctx, b, msg, giveupStickers)
@@ -245,7 +246,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
if subject == "" {
return chathelper.Reply(ctx, b, msg, "Cannot identify chat.")
}
st, err := loadStats(ctx, s.kv, subject)
st, err := loadStats(ctx, s.stats, subject)
if err != nil {
return err
}
@@ -274,7 +275,7 @@ func (s *state) handleSetMax(ctx context.Context, b *bot.Bot, update *models.Upd
if err != nil || n < 1 || n > MaxGuessesCap {
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("Usage: /loldle_setmax <1-%d>", MaxGuessesCap))
}
if err := setMaxGuesses(ctx, s.kv, subject, n); err != nil {
if err := setMaxGuesses(ctx, s.cfg, subject, n); err != nil {
return err
}
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("✅ Loldle max guesses set to %d (applies to the next round).", n))
+21 -20
View File
@@ -13,12 +13,13 @@ import (
// installLoldle wires the loldle module + auth (owner gates /loldle_setmax,
// which is private). seedTarget pre-seeds a game so guess outcomes are
// deterministic without hooking math/rand.
func installLoldle(t *testing.T, ownerID int64, seedSubject, seedTarget string) (*testutil.RecordingBot, storage.KVStore) {
func installLoldle(t *testing.T, ownerID int64, seedSubject, seedTarget string) (*testutil.RecordingBot, GameStore, ConfigStore) {
t.Helper()
rb := testutil.NewRecordingBot(t)
provider := storage.NewMemoryProvider()
kv := provider.For("loldle")
mod := New(modules.Deps{KV: kv})
col := storage.NewMemoryProvider().Collection("loldle")
games := storage.Typed[gameState](col)
cfg := storage.Typed[roundConfig](col)
mod := New(modules.Deps{Store: col})
reg := &modules.Registry{
Modules: []modules.Module{{Name: "loldle", Commands: mod.Commands}},
AllCommands: map[string]modules.Command{},
@@ -30,15 +31,15 @@ func installLoldle(t *testing.T, ownerID int64, seedSubject, seedTarget string)
if seedTarget != "" {
g := &gameState{Target: seedTarget, Guesses: []string{}}
if err := saveGame(context.Background(), kv, seedSubject, g); err != nil {
if err := saveGame(context.Background(), games, seedSubject, g); err != nil {
t.Fatalf("seed game: %v", err)
}
}
return rb, kv
return rb, games, cfg
}
func TestLoldle_NoArgShowsBoard(t *testing.T) {
rb, _ := installLoldle(t, 0, "1", "Aatrox")
rb, _, _ := installLoldle(t, 0, "1", "Aatrox")
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle"))
got := rb.LastSent().Text()
@@ -48,7 +49,7 @@ func TestLoldle_NoArgShowsBoard(t *testing.T) {
}
func TestLoldle_Win(t *testing.T) {
rb, _ := installLoldle(t, 0, "1", "Aatrox")
rb, _, _ := installLoldle(t, 0, "1", "Aatrox")
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle aatrox"))
got := rb.LastSent().Text()
@@ -61,7 +62,7 @@ func TestLoldle_Win(t *testing.T) {
}
func TestLoldle_UnknownChampion(t *testing.T) {
rb, _ := installLoldle(t, 0, "1", "Aatrox")
rb, _, _ := installLoldle(t, 0, "1", "Aatrox")
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle ZilbeanZ"))
got := rb.LastSent().Text()
@@ -71,7 +72,7 @@ func TestLoldle_UnknownChampion(t *testing.T) {
}
func TestLoldle_DuplicateGuessRejected(t *testing.T) {
rb, _ := installLoldle(t, 0, "1", "Aatrox")
rb, _, _ := installLoldle(t, 0, "1", "Aatrox")
// First guess: Ahri (not the target — round continues).
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle ahri"))
rb.Reset()
@@ -85,7 +86,7 @@ func TestLoldle_DuplicateGuessRejected(t *testing.T) {
}
func TestLoldleGiveup_RevealsAnswer(t *testing.T) {
rb, _ := installLoldle(t, 0, "1", "Aatrox")
rb, _, _ := installLoldle(t, 0, "1", "Aatrox")
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle_giveup"))
got := rb.LastSent().Text()
@@ -95,7 +96,7 @@ func TestLoldleGiveup_RevealsAnswer(t *testing.T) {
}
func TestLoldleGiveup_NoActiveRound(t *testing.T) {
rb, _ := installLoldle(t, 0, "", "")
rb, _, _ := installLoldle(t, 0, "", "")
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle_giveup"))
got := rb.LastSent().Text()
@@ -105,7 +106,7 @@ func TestLoldleGiveup_NoActiveRound(t *testing.T) {
}
func TestLoldleStats_Empty(t *testing.T) {
rb, _ := installLoldle(t, 0, "", "")
rb, _, _ := installLoldle(t, 0, "", "")
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle_stats"))
got := rb.LastSent().Text()
@@ -117,24 +118,24 @@ func TestLoldleStats_Empty(t *testing.T) {
}
func TestLoldleSetMax_OwnerSucceeds(t *testing.T) {
rb, kv := installLoldle(t, 999, "", "")
rb, _, cfg := installLoldle(t, 999, "", "")
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(999, "/loldle_setmax 5"))
got := rb.LastSent().Text()
if !strings.Contains(got, "max guesses set to 5") {
t.Errorf("/loldle_setmax reply: %q", got)
}
var cfg roundConfig
if err := kv.GetJSON(context.Background(), configKey("999"), &cfg); err != nil {
c, _, err := cfg.Get(context.Background(), configKey("999"))
if err != nil {
t.Fatalf("expected config persisted: %v", err)
}
if cfg.MaxGuesses != 5 {
t.Errorf("MaxGuesses persisted = %d, want 5", cfg.MaxGuesses)
if c.MaxGuesses != 5 {
t.Errorf("MaxGuesses persisted = %d, want 5", c.MaxGuesses)
}
}
func TestLoldleSetMax_DeniedToNonOwner(t *testing.T) {
rb, _ := installLoldle(t, 999, "", "")
rb, _, _ := installLoldle(t, 999, "", "")
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(7, "/loldle_setmax 5"))
if calls := rb.Sent(); len(calls) != 0 {
@@ -143,7 +144,7 @@ func TestLoldleSetMax_DeniedToNonOwner(t *testing.T) {
}
func TestLoldleSetMax_RejectsOutOfRange(t *testing.T) {
rb, _ := installLoldle(t, 999, "", "")
rb, _, _ := installLoldle(t, 999, "", "")
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(999, "/loldle_setmax 99"))
got := rb.LastSent().Text()
+4 -1
View File
@@ -2,13 +2,16 @@ package loldle
import (
"github.com/tiennm99/miti99bot/internal/modules"
"github.com/tiennm99/miti99bot/internal/storage"
)
// New is the loldle module Factory. Loads champions.json once at construction
// and shares the parsed slice + per-subject lock map across all handlers.
func New(deps modules.Deps) modules.Module {
s := &state{
kv: deps.KV,
games: storage.Typed[gameState](deps.Store),
stats: storage.Typed[stats](deps.Store),
cfg: storage.Typed[roundConfig](deps.Store),
champions: loadChampions(),
}
return modules.Module{
+35 -29
View File
@@ -15,7 +15,7 @@ const (
MaxGuessesCap = 10
)
// gameState is the per-subject KV record.
// gameState is the per-subject record.
//
// StartedAt is *int64 because the timer doesn't start until the first guess;
// using time.Time would marshal as "0001-01-01T00:00:00Z" instead of null
@@ -25,35 +25,43 @@ const (
// time against current champions.json so a weekly data refresh updates
// historical board displays without migrating saved rounds.
type gameState struct {
Target string `json:"target"`
Guesses []string `json:"guesses"`
StartedAt *int64 `json:"startedAt"` // ms-since-epoch | null
Target string `json:"target" bson:"target"`
Guesses []string `json:"guesses" bson:"guesses"`
StartedAt *int64 `json:"startedAt" bson:"startedAt"` // ms-since-epoch | null
}
// stats lifetime score. No LastResultAt field by design (differs from
// wordle's Stats — loldle only ever needed running streaks, not "last
// played at").
type stats struct {
Played int `json:"played"`
Wins int `json:"wins"`
Streak int `json:"streak"`
BestStreak int `json:"bestStreak"`
Played int `json:"played" bson:"played"`
Wins int `json:"wins" bson:"wins"`
Streak int `json:"streak" bson:"streak"`
BestStreak int `json:"bestStreak" bson:"bestStreak"`
}
// roundConfig stores the per-subject MaxGuesses override. Stored only when
// /loldle_setmax has been run; the absence of this record means "use default".
type roundConfig struct {
MaxGuesses int `json:"maxGuesses"`
MaxGuesses int `json:"maxGuesses" bson:"maxGuesses"`
}
// GameStore is the loldle module's typed game store.
type GameStore = storage.DocStore[gameState]
// StatsStore is the loldle module's typed stats store.
type StatsStore = storage.DocStore[stats]
// ConfigStore is the loldle module's typed config store.
type ConfigStore = storage.DocStore[roundConfig]
func gameKey(subject string) string { return "game:" + subject }
func statsKey(subject string) string { return "stats:" + subject }
func configKey(subject string) string { return "config:" + subject }
// loadGame returns the active round, or (nil, nil) if none exists.
func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*gameState, error) {
var g gameState
err := kv.GetJSON(ctx, gameKey(subject), &g)
func loadGame(ctx context.Context, games GameStore, subject string) (*gameState, error) {
g, _, err := games.Get(ctx, gameKey(subject))
switch {
case err == nil:
return &g, nil
@@ -64,8 +72,8 @@ func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*gameSta
}
}
func saveGame(ctx context.Context, kv storage.KVStore, subject string, g *gameState) error {
if err := kv.PutJSON(ctx, gameKey(subject), g); err != nil {
func saveGame(ctx context.Context, games GameStore, subject string, g *gameState) error {
if err := games.Put(ctx, gameKey(subject), *g); err != nil {
return fmt.Errorf("loldle saveGame: %w", err)
}
return nil
@@ -74,8 +82,8 @@ func saveGame(ctx context.Context, kv storage.KVStore, subject string, g *gameSt
// clearGame removes the round so the next /loldle starts fresh. Used after
// win / loss / giveup; the new round's timer should start on the player's
// next interaction, not at the moment the previous round ended.
func clearGame(ctx context.Context, kv storage.KVStore, subject string) error {
if err := kv.Delete(ctx, gameKey(subject)); err != nil {
func clearGame(ctx context.Context, games GameStore, subject string) error {
if err := games.Delete(ctx, gameKey(subject)); err != nil {
return fmt.Errorf("loldle clearGame: %w", err)
}
return nil
@@ -83,9 +91,8 @@ func clearGame(ctx context.Context, kv storage.KVStore, subject string) error {
// loadStats returns lifetime score; missing → fresh-zero record so callers
// never need a nil check.
func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*stats, error) {
var s stats
err := kv.GetJSON(ctx, statsKey(subject), &s)
func loadStats(ctx context.Context, st StatsStore, subject string) (*stats, error) {
s, _, err := st.Get(ctx, statsKey(subject))
switch {
case err == nil:
return &s, nil
@@ -96,8 +103,8 @@ func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*stats,
}
}
func recordResult(ctx context.Context, kv storage.KVStore, subject string, won bool) (*stats, error) {
s, err := loadStats(ctx, kv, subject)
func recordResult(ctx context.Context, st StatsStore, subject string, won bool) (*stats, error) {
s, err := loadStats(ctx, st, subject)
if err != nil {
return nil, err
}
@@ -111,7 +118,7 @@ func recordResult(ctx context.Context, kv storage.KVStore, subject string, won b
} else {
s.Streak = 0
}
if err := kv.PutJSON(ctx, statsKey(subject), s); err != nil {
if err := st.Put(ctx, statsKey(subject), *s); err != nil {
return nil, fmt.Errorf("loldle recordResult: %w", err)
}
return s, nil
@@ -120,27 +127,26 @@ func recordResult(ctx context.Context, kv storage.KVStore, subject string, won b
// getMaxGuesses returns the effective round length: the per-subject override
// if set and in range, otherwise MaxGuesses. Out-of-range values are
// silently ignored — better to serve the default than 500 the user.
func getMaxGuesses(ctx context.Context, kv storage.KVStore, subject string) (int, error) {
var cfg roundConfig
err := kv.GetJSON(ctx, configKey(subject), &cfg)
func getMaxGuesses(ctx context.Context, cfg ConfigStore, subject string) (int, error) {
c, _, err := cfg.Get(ctx, configKey(subject))
if err != nil {
if errors.Is(err, storage.ErrNotFound) {
return MaxGuesses, nil
}
return 0, fmt.Errorf("loldle getMaxGuesses: %w", err)
}
if cfg.MaxGuesses < 1 || cfg.MaxGuesses > MaxGuessesCap {
if c.MaxGuesses < 1 || c.MaxGuesses > MaxGuessesCap {
return MaxGuesses, nil
}
return cfg.MaxGuesses, nil
return c.MaxGuesses, nil
}
// setMaxGuesses validates and persists the per-subject override.
func setMaxGuesses(ctx context.Context, kv storage.KVStore, subject string, n int) error {
func setMaxGuesses(ctx context.Context, cfg ConfigStore, subject string, n int) error {
if n < 1 || n > MaxGuessesCap {
return fmt.Errorf("loldle: maxGuesses must be in [1, %d], got %d", MaxGuessesCap, n)
}
if err := kv.PutJSON(ctx, configKey(subject), roundConfig{MaxGuesses: n}); err != nil {
if err := cfg.Put(ctx, configKey(subject), roundConfig{MaxGuesses: n}); err != nil {
return fmt.Errorf("loldle setMaxGuesses: %w", err)
}
return nil
+41 -22
View File
@@ -8,6 +8,25 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
func newLoldleGames() GameStore {
return storage.Typed[gameState](storage.NewMemoryProvider().Collection("loldle"))
}
func newLoldleStats() StatsStore {
return storage.Typed[stats](storage.NewMemoryProvider().Collection("loldle"))
}
func newLoldleConfig() ConfigStore {
return storage.Typed[roundConfig](storage.NewMemoryProvider().Collection("loldle"))
}
// newLoldleStores returns all three stores backed by the same collection
// (disjoint key prefixes: "game:", "stats:", "config:").
func newLoldleStores() (GameStore, StatsStore, ConfigStore) {
c := storage.NewMemoryProvider().Collection("loldle")
return storage.Typed[gameState](c), storage.Typed[stats](c), storage.Typed[roundConfig](c)
}
func TestGameState_StartedAtNullByDefault(t *testing.T) {
g := gameState{Target: "Aatrox", Guesses: []string{}}
b, err := json.Marshal(g)
@@ -43,20 +62,20 @@ func TestStats_NoLastResultAtField(t *testing.T) {
func TestRecordResult_WinAndLossSequence(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
st := newLoldleStats()
s, err := recordResult(ctx, kv, "u1", true)
s, err := recordResult(ctx, st, "u1", true)
if err != nil {
t.Fatal(err)
}
if s.Played != 1 || s.Wins != 1 || s.Streak != 1 || s.BestStreak != 1 {
t.Errorf("first win: %+v", s)
}
s, _ = recordResult(ctx, kv, "u1", true)
s, _ = recordResult(ctx, st, "u1", true)
if s.Streak != 2 || s.BestStreak != 2 {
t.Errorf("two wins: %+v", s)
}
s, _ = recordResult(ctx, kv, "u1", false)
s, _ = recordResult(ctx, st, "u1", false)
if s.Streak != 0 {
t.Errorf("loss should reset streak, got %d", s.Streak)
}
@@ -67,52 +86,52 @@ func TestRecordResult_WinAndLossSequence(t *testing.T) {
func TestGetMaxGuesses_DefaultsAndOverrides(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
cfg := newLoldleConfig()
if n, _ := getMaxGuesses(ctx, kv, "u1"); n != MaxGuesses {
if n, _ := getMaxGuesses(ctx, cfg, "u1"); n != MaxGuesses {
t.Errorf("missing config → %d, want %d", n, MaxGuesses)
}
if err := setMaxGuesses(ctx, kv, "u1", 5); err != nil {
if err := setMaxGuesses(ctx, cfg, "u1", 5); err != nil {
t.Fatal(err)
}
if n, _ := getMaxGuesses(ctx, kv, "u1"); n != 5 {
if n, _ := getMaxGuesses(ctx, cfg, "u1"); n != 5 {
t.Errorf("after setMax(5): %d, want 5", n)
}
}
func TestSetMaxGuesses_ValidatesRange(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
cfg := newLoldleConfig()
for _, n := range []int{0, -1, MaxGuessesCap + 1, 100} {
if err := setMaxGuesses(ctx, kv, "u1", n); err == nil {
if err := setMaxGuesses(ctx, cfg, "u1", n); err == nil {
t.Errorf("setMaxGuesses(%d) should error", n)
}
}
if err := setMaxGuesses(ctx, kv, "u1", 1); err != nil {
if err := setMaxGuesses(ctx, cfg, "u1", 1); err != nil {
t.Errorf("setMaxGuesses(1) should succeed: %v", err)
}
if err := setMaxGuesses(ctx, kv, "u1", MaxGuessesCap); err != nil {
if err := setMaxGuesses(ctx, cfg, "u1", MaxGuessesCap); err != nil {
t.Errorf("setMaxGuesses(cap) should succeed: %v", err)
}
}
func TestGetMaxGuesses_OutOfRangeIgnored(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
// Inject a corrupt config to simulate manual KV tampering or a stale
cfg := newLoldleConfig()
// Inject a corrupt config to simulate manual store tampering or a stale
// schema; getMaxGuesses must fall back to the default rather than
// returning a wild value to handlers.
if err := kv.PutJSON(ctx, configKey("u1"), roundConfig{MaxGuesses: 100}); err != nil {
if err := cfg.Put(ctx, configKey("u1"), roundConfig{MaxGuesses: 100}); err != nil {
t.Fatal(err)
}
if n, _ := getMaxGuesses(ctx, kv, "u1"); n != MaxGuesses {
if n, _ := getMaxGuesses(ctx, cfg, "u1"); n != MaxGuesses {
t.Errorf("out-of-range config → %d, want %d (default)", n, MaxGuesses)
}
}
func TestLoadGame_MissingReturnsNil(t *testing.T) {
g, err := loadGame(context.Background(), storage.NewMemoryKVStore(), "nobody")
g, err := loadGame(context.Background(), newLoldleGames(), "nobody")
if err != nil {
t.Errorf("missing should not error: %v", err)
}
@@ -123,13 +142,13 @@ func TestLoadGame_MissingReturnsNil(t *testing.T) {
func TestSaveLoadClearGame_RoundTrip(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
games := newLoldleGames()
at := int64(42)
want := &gameState{Target: "Aatrox", Guesses: []string{"Ahri"}, StartedAt: &at}
if err := saveGame(ctx, kv, "u1", want); err != nil {
if err := saveGame(ctx, games, "u1", want); err != nil {
t.Fatal(err)
}
got, err := loadGame(ctx, kv, "u1")
got, err := loadGame(ctx, games, "u1")
if err != nil || got == nil {
t.Fatalf("loadGame: %v, %v", got, err)
}
@@ -137,10 +156,10 @@ func TestSaveLoadClearGame_RoundTrip(t *testing.T) {
t.Errorf("round-trip mismatch: %+v", got)
}
if err := clearGame(ctx, kv, "u1"); err != nil {
if err := clearGame(ctx, games, "u1"); err != nil {
t.Fatal(err)
}
got, _ = loadGame(ctx, kv, "u1")
got, _ = loadGame(ctx, games, "u1")
if got != nil {
t.Errorf("after clearGame, expected nil, got %+v", got)
}
+45 -32
View File
@@ -44,50 +44,61 @@ const (
)
// Team is one side of a match. JSON shape matches the lolesports response.
// bson tags mirror the json names exactly: this tree is persisted inside
// cacheRecord, and the DynamoDB→Mongo migrator preserves the original (camelCase)
// JSON keys, so the store must read those keys back verbatim — not the driver's
// lowercased default.
type Team struct {
Name string `json:"name,omitempty"`
Code string `json:"code,omitempty"`
Image string `json:"image,omitempty"`
Result *struct {
Outcome string `json:"outcome,omitempty"` // "win" or "loss"
GameWins int `json:"gameWins,omitempty"`
} `json:"result,omitempty"`
Record *struct {
Wins int `json:"wins,omitempty"`
Losses int `json:"losses,omitempty"`
} `json:"record,omitempty"`
Name string `json:"name,omitempty" bson:"name,omitempty"`
Code string `json:"code,omitempty" bson:"code,omitempty"`
Image string `json:"image,omitempty" bson:"image,omitempty"`
Result *TeamResult `json:"result,omitempty" bson:"result,omitempty"`
Record *TeamRecord `json:"record,omitempty" bson:"record,omitempty"`
}
// TeamResult is a team's per-match outcome (named so tests and the bson decoder
// share one type).
type TeamResult struct {
Outcome string `json:"outcome,omitempty" bson:"outcome,omitempty"` // "win" or "loss"
GameWins int `json:"gameWins,omitempty" bson:"gameWins,omitempty"`
}
// TeamRecord is a team's running series record.
type TeamRecord struct {
Wins int `json:"wins,omitempty" bson:"wins,omitempty"`
Losses int `json:"losses,omitempty" bson:"losses,omitempty"`
}
// League holds the league-section-header info on each event.
type League struct {
Name string `json:"name,omitempty"`
Slug string `json:"slug,omitempty"`
Image string `json:"image,omitempty"`
Name string `json:"name,omitempty" bson:"name,omitempty"`
Slug string `json:"slug,omitempty" bson:"slug,omitempty"`
Image string `json:"image,omitempty" bson:"image,omitempty"`
}
// Strategy is the bestOf descriptor (Bo1, Bo3, Bo5).
type Strategy struct {
Type string `json:"type,omitempty"`
Count int `json:"count,omitempty"`
Type string `json:"type,omitempty" bson:"type,omitempty"`
Count int `json:"count,omitempty" bson:"count,omitempty"`
}
// Match is the inner match metadata.
type Match struct {
ID string `json:"id,omitempty"`
Teams []Team `json:"teams,omitempty"`
Strategy Strategy `json:"strategy,omitempty"`
ID string `json:"id,omitempty" bson:"id,omitempty"`
Teams []Team `json:"teams,omitempty" bson:"teams,omitempty"`
Strategy Strategy `json:"strategy,omitempty" bson:"strategy,omitempty"`
}
// ScheduleEvent is one upcoming or past match. State is "unstarted",
// "inProgress", or "completed". Type is set to "show" for pre/post-show
// segments which we filter out.
type ScheduleEvent struct {
StartTime string `json:"startTime"`
State string `json:"state,omitempty"`
Type string `json:"type,omitempty"`
BlockName string `json:"blockName,omitempty"`
League League `json:"league,omitempty"`
Match Match `json:"match,omitempty"`
StartTime string `json:"startTime" bson:"startTime"`
State string `json:"state,omitempty" bson:"state,omitempty"`
Type string `json:"type,omitempty" bson:"type,omitempty"`
BlockName string `json:"blockName,omitempty" bson:"blockName,omitempty"`
League League `json:"league,omitempty" bson:"league,omitempty"`
Match Match `json:"match,omitempty" bson:"match,omitempty"`
}
// schedulePage is the inner shape of an upstream response.
@@ -103,12 +114,15 @@ type schedulePage struct {
} `json:"data"`
}
// cacheRecord is the KV value: fetch timestamp (ms-epoch) + events.
// cacheRecord is the store value: fetch timestamp (ms-epoch) + events.
type cacheRecord struct {
Ts int64 `json:"ts"` // ms-since-epoch when fetched
Events []ScheduleEvent `json:"events"`
Ts int64 `json:"ts" bson:"ts"` // ms-since-epoch when fetched
Events []ScheduleEvent `json:"events" bson:"events"`
}
// CacheStore is the typed store for schedule cache records.
type CacheStore = storage.DocStore[cacheRecord]
// Client is the lolesports API client. Default zero-value uses
// http.DefaultClient + http.DefaultTransport; tests inject a custom HTTP
// client (typically pointing at httptest.Server).
@@ -274,12 +288,11 @@ func cacheKey(from, to time.Time) string {
// GetEventsCached is the cache-first lookup. Returns fresh cache within
// cacheTTL, else fetches upstream and writes back, else falls back to
// stale cache (within staleMaxAge), else propagates the error.
func (c *Client) GetEventsCached(ctx context.Context, kv storage.KVStore, from, to time.Time) ([]ScheduleEvent, error) {
func (c *Client) GetEventsCached(ctx context.Context, cache CacheStore, from, to time.Time) ([]ScheduleEvent, error) {
key := cacheKey(from, to)
now := time.Now().UTC().UnixMilli()
var cached cacheRecord
cacheErr := kv.GetJSON(ctx, key, &cached)
cached, _, cacheErr := cache.Get(ctx, key)
hasCached := cacheErr == nil
if hasCached && now-cached.Ts < cacheTTL.Milliseconds() {
return cached.Events, nil
@@ -288,7 +301,7 @@ func (c *Client) GetEventsCached(ctx context.Context, kv storage.KVStore, from,
events, fetchErr := c.fetchEventsInRange(ctx, from, to, 3)
if fetchErr == nil {
rec := cacheRecord{Ts: now, Events: events}
if err := kv.PutJSON(ctx, key, rec); err != nil {
if err := cache.Put(ctx, key, rec); err != nil {
log.Warn("lolschedule_kv_put_fail", "err", err)
}
return events, nil
+17 -11
View File
@@ -13,6 +13,12 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
// newCacheStore returns a fresh typed cache store backed by an in-memory
// collection — the per-test equivalent of what the factory wires in production.
func newCacheStore() CacheStore {
return storage.Typed[cacheRecord](storage.NewMemoryProvider().Collection("lolschedule"))
}
// mkServer spins an httptest.Server returning the supplied JSON body for
// every page request. callCount counts upstream hits so cache tests can
// assert "1 fetch, then no more".
@@ -54,11 +60,11 @@ const sampleBody = `{
func TestGetEventsCached_FirstHitFetchesUpstream(t *testing.T) {
srv, count := mkServer(t, sampleBody)
c := &Client{HTTP: srv.Client(), URL: srv.URL}
kv := storage.NewMemoryKVStore()
cache := newCacheStore()
from := time.Date(2026, 5, 9, 0, 0, 0, 0, time.UTC)
to := from.Add(24 * time.Hour)
events, err := c.GetEventsCached(context.Background(), kv, from, to)
events, err := c.GetEventsCached(context.Background(), cache, from, to)
if err != nil {
t.Fatalf("first fetch: %v", err)
}
@@ -76,16 +82,16 @@ func TestGetEventsCached_FirstHitFetchesUpstream(t *testing.T) {
func TestGetEventsCached_SecondHitUsesCache(t *testing.T) {
srv, count := mkServer(t, sampleBody)
c := &Client{HTTP: srv.Client(), URL: srv.URL}
kv := storage.NewMemoryKVStore()
cache := newCacheStore()
from := time.Date(2026, 5, 9, 0, 0, 0, 0, time.UTC)
to := from.Add(24 * time.Hour)
// First fetch primes the cache.
if _, err := c.GetEventsCached(context.Background(), kv, from, to); err != nil {
if _, err := c.GetEventsCached(context.Background(), cache, from, to); err != nil {
t.Fatal(err)
}
// Second fetch within TTL must NOT hit upstream.
if _, err := c.GetEventsCached(context.Background(), kv, from, to); err != nil {
if _, err := c.GetEventsCached(context.Background(), cache, from, to); err != nil {
t.Fatal(err)
}
if got := atomic.LoadInt32(count); got != 1 {
@@ -94,8 +100,8 @@ func TestGetEventsCached_SecondHitUsesCache(t *testing.T) {
}
func TestGetEventsCached_StaleFallback(t *testing.T) {
// Prime KV with a stale-but-still-fresh-enough cache record.
kv := storage.NewMemoryKVStore()
// Prime the typed cache store with a stale-but-still-fresh-enough record.
cache := newCacheStore()
from := time.Date(2026, 5, 9, 0, 0, 0, 0, time.UTC)
to := from.Add(24 * time.Hour)
staleEvents := []ScheduleEvent{
@@ -103,7 +109,7 @@ func TestGetEventsCached_StaleFallback(t *testing.T) {
}
// 10 minutes ago — past the 120s fresh window but well inside 60-min stale.
staleTs := time.Now().UTC().Add(-10 * time.Minute).UnixMilli()
if err := kv.PutJSON(context.Background(), cacheKey(from, to), cacheRecord{Ts: staleTs, Events: staleEvents}); err != nil {
if err := cache.Put(context.Background(), cacheKey(from, to), cacheRecord{Ts: staleTs, Events: staleEvents}); err != nil {
t.Fatal(err)
}
@@ -115,7 +121,7 @@ func TestGetEventsCached_StaleFallback(t *testing.T) {
defer srv.Close()
c := &Client{HTTP: srv.Client(), URL: srv.URL}
got, err := c.GetEventsCached(context.Background(), kv, from, to)
got, err := c.GetEventsCached(context.Background(), cache, from, to)
if err != nil {
t.Fatalf("stale fallback should succeed: %v", err)
}
@@ -125,7 +131,7 @@ func TestGetEventsCached_StaleFallback(t *testing.T) {
}
func TestGetEventsCached_HardFailureWhenNoCache(t *testing.T) {
kv := storage.NewMemoryKVStore()
cache := newCacheStore()
from := time.Date(2026, 5, 9, 0, 0, 0, 0, time.UTC)
to := from.Add(24 * time.Hour)
@@ -135,7 +141,7 @@ func TestGetEventsCached_HardFailureWhenNoCache(t *testing.T) {
defer srv.Close()
c := &Client{HTTP: srv.Client(), URL: srv.URL}
_, err := c.GetEventsCached(context.Background(), kv, from, to)
_, err := c.GetEventsCached(context.Background(), cache, from, to)
if err == nil {
t.Errorf("expected error when upstream fails AND no cache")
}
+20 -28
View File
@@ -110,6 +110,15 @@ type messageSender interface {
SendMessage(ctx context.Context, params *bot.SendMessageParams) (*models.Message, error)
}
// lastPushDoc wraps the last-push date string so it can be stored as a named
// root field in a Mongo document (a bare scalar cannot be a root doc).
type lastPushDoc struct {
Date string `json:"date" bson:"date"`
}
// PushDateStore is the typed store for last-push date documents.
type PushDateStore = storage.DocStore[lastPushDoc]
// dailyPushCron returns the cron registration. Schedule is documentation only.
func (s *state) dailyPushCron() modules.Cron {
return modules.Cron{
@@ -134,30 +143,13 @@ func (s *state) dailyPushHandler(ctx context.Context, deps modules.Deps) error {
// for the daily push: a winner proceeds to fan out; a loser (another trigger
// already claimed today) returns false and sends nothing.
//
// The claim is a version-based optimistic write on lastPushDateKey so two
// simultaneous triggers cannot both win. Every real KV backend implements
// VersionedStore; if a bare store without it is used (only in narrow tests), it
// falls back to a plain Put — losing the atomic guarantee but preserving the
// "no-op if already today" behaviour.
func claimDailyPush(ctx context.Context, kv storage.KVStore, today string) (bool, error) {
vs, ok := kv.(storage.VersionedStore)
if !ok {
// No versioned support (a bare store in narrow tests): best-effort Put,
// losing the atomic guarantee but preserving "no-op if already today".
current, err := kv.Get(ctx, lastPushDateKey)
if err == nil && string(current) == today {
return false, nil
}
if err != nil && !errors.Is(err, storage.ErrNotFound) {
return false, err
}
return true, kv.Put(ctx, lastPushDateKey, []byte(today))
}
current, version, err := vs.GetVersioned(ctx, lastPushDateKey)
// The claim uses version-based optimistic write (PutVersioned) on
// lastPushDateKey so two simultaneous triggers cannot both win.
func claimDailyPush(ctx context.Context, store PushDateStore, today string) (bool, error) {
current, version, err := store.Get(ctx, lastPushDateKey)
switch {
case err == nil:
if string(current) == today {
if current.Date == today {
return false, nil // already pushed today
}
case errors.Is(err, storage.ErrNotFound):
@@ -166,7 +158,7 @@ func claimDailyPush(ctx context.Context, kv storage.KVStore, today string) (bool
return false, err
}
if err := vs.PutVersioned(ctx, lastPushDateKey, version, []byte(today)); err != nil {
if err := store.PutVersioned(ctx, lastPushDateKey, version, lastPushDoc{Date: today}); err != nil {
if errors.Is(err, storage.ErrConflict) {
return false, nil // another trigger claimed today first
}
@@ -182,7 +174,7 @@ func claimDailyPush(ctx context.Context, kv storage.KVStore, today string) (bool
// MessageThreadID is forwarded on every send so subscribers in a forum-topic
// receive the digest in that topic, not in General.
func runDailyPush(ctx context.Context, s *state, sender messageSender) error {
subs, err := listSubscribers(ctx, s.kv)
subs, err := listSubscribers(ctx, s.subscribers)
if err != nil {
return fmt.Errorf("lolschedule daily push: list subscribers: %w", err)
}
@@ -193,7 +185,7 @@ func runDailyPush(ctx context.Context, s *state, sender messageSender) error {
from := ictDayStartOf(s.now())
to := addDays(from, 1)
events, err := s.client.GetEventsCached(ctx, s.kv, from, to)
events, err := s.client.GetEventsCached(ctx, s.cache, from, to)
if err != nil {
return fmt.Errorf("lolschedule daily push: fetch matches: %w", err)
}
@@ -205,7 +197,7 @@ func runDailyPush(ctx context.Context, s *state, sender messageSender) error {
// send nothing. Placed after the fetch so a transient fetch failure does
// not consume the day's claim.
today := s.now().UTC().Format("2006-01-02")
won, err := claimDailyPush(ctx, s.kv, today)
won, err := claimDailyPush(ctx, s.pushDate, today)
if err != nil {
return fmt.Errorf("lolschedule daily push: claim date: %w", err)
}
@@ -272,7 +264,7 @@ func pruneDeadSubscribers(ctx context.Context, s *state, chatWide map[int64]stru
defer s.subscribersMu.Unlock()
removed := 0
for chatID := range chatWide {
n, err := removeAllForChat(ctx, s.kv, chatID)
n, err := removeAllForChat(ctx, s.subscribers, chatID)
if err != nil {
log.Warn("lolschedule prune dead chat failed", "chat", chatID, "err", err)
continue
@@ -285,7 +277,7 @@ func pruneDeadSubscribers(ctx context.Context, s *state, chatWide map[int64]stru
if _, ok := chatWide[sub.ChatID]; ok {
continue
}
ok, err := removeSubscriber(ctx, s.kv, sub.ChatID, sub.ThreadID)
ok, err := removeSubscriber(ctx, s.subscribers, sub.ChatID, sub.ThreadID)
if err != nil {
log.Warn("lolschedule prune dead topic failed",
"chat", sub.ChatID, "thread", sub.ThreadID, "err", err)
+37 -25
View File
@@ -56,9 +56,19 @@ func fixedNow() time.Time {
return time.Date(2026, 5, 10, 5, 0, 0, 0, time.UTC)
}
// newTestStore builds a fresh set of typed stores over a shared in-memory
// collection, matching what the factory wires in production.
func newTestStore(t *testing.T) (SubscriberStore, PushDateStore, CacheStore) {
t.Helper()
col := storage.NewMemoryProvider().Collection("lolschedule")
return storage.Typed[subscribersDoc](col),
storage.Typed[lastPushDoc](col),
storage.Typed[cacheRecord](col)
}
// seedFreshCache writes a cacheRecord with `now` as timestamp so the first
// GetEventsCached call returns it without hitting the network.
func seedFreshCache(t *testing.T, kv storage.KVStore, events []ScheduleEvent) {
func seedFreshCache(t *testing.T, cache CacheStore, events []ScheduleEvent) {
t.Helper()
from := ictDayStartOf(fixedNow())
to := addDays(from, 1)
@@ -66,24 +76,26 @@ func seedFreshCache(t *testing.T, kv storage.KVStore, events []ScheduleEvent) {
Ts: time.Now().UTC().UnixMilli(),
Events: events,
}
if err := kv.PutJSON(context.Background(), cacheKey(from, to), rec); err != nil {
if err := cache.Put(context.Background(), cacheKey(from, to), rec); err != nil {
t.Fatalf("seed cache: %v", err)
}
}
func newTestState(t *testing.T) *state {
t.Helper()
kv := storage.NewMemoryKVStore()
subs, pd, cache := newTestStore(t)
return &state{
kv: kv,
client: &Client{}, // zero value; tests must seed cache to avoid HTTP
nowFn: fixedNow,
subscribers: subs,
pushDate: pd,
cache: cache,
client: &Client{}, // zero value; tests must seed cache to avoid HTTP
nowFn: fixedNow,
}
}
func TestRunDailyPush_NoSubscribers(t *testing.T) {
s := newTestState(t)
seedFreshCache(t, s.kv, nil)
seedFreshCache(t, s.cache, nil)
sender := &fakeSender{}
if err := runDailyPush(context.Background(), s, sender); err != nil {
@@ -96,11 +108,11 @@ func TestRunDailyPush_NoSubscribers(t *testing.T) {
func TestRunDailyPush_SendsToAllSubscribers(t *testing.T) {
s := newTestState(t)
seedFreshCache(t, s.kv, nil) // empty schedule still produces a "no matches" message
seedFreshCache(t, s.cache, nil) // empty schedule still produces a "no matches" message
chatIDs := []int64{100, 200, 300}
for _, id := range chatIDs {
if _, err := addSubscriber(context.Background(), s.kv, id, 0); err != nil {
if _, err := addSubscriber(context.Background(), s.subscribers, id, 0); err != nil {
t.Fatalf("addSubscriber %d: %v", id, err)
}
}
@@ -138,7 +150,7 @@ func TestRunDailyPush_SendsToAllSubscribers(t *testing.T) {
// topic, not in General.
func TestRunDailyPush_ForwardsMessageThreadID(t *testing.T) {
s := newTestState(t)
seedFreshCache(t, s.kv, nil)
seedFreshCache(t, s.cache, nil)
subs := []Subscriber{
{ChatID: 100, ThreadID: 0},
@@ -146,7 +158,7 @@ func TestRunDailyPush_ForwardsMessageThreadID(t *testing.T) {
{ChatID: 200, ThreadID: 42},
}
for _, sub := range subs {
if _, err := addSubscriber(context.Background(), s.kv, sub.ChatID, sub.ThreadID); err != nil {
if _, err := addSubscriber(context.Background(), s.subscribers, sub.ChatID, sub.ThreadID); err != nil {
t.Fatal(err)
}
}
@@ -173,11 +185,11 @@ func TestRunDailyPush_ForwardsMessageThreadID(t *testing.T) {
// misconfiguration (all double-fire windows the daily push must survive).
func TestRunDailyPush_IdempotentPerDate(t *testing.T) {
s := newTestState(t)
seedFreshCache(t, s.kv, nil)
seedFreshCache(t, s.cache, nil)
chatIDs := []int64{100, 200, 300}
for _, id := range chatIDs {
if _, err := addSubscriber(context.Background(), s.kv, id, 0); err != nil {
if _, err := addSubscriber(context.Background(), s.subscribers, id, 0); err != nil {
t.Fatalf("addSubscriber %d: %v", id, err)
}
}
@@ -196,11 +208,11 @@ func TestRunDailyPush_IdempotentPerDate(t *testing.T) {
func TestRunDailyPush_PartialFailureContinues(t *testing.T) {
s := newTestState(t)
seedFreshCache(t, s.kv, nil)
seedFreshCache(t, s.cache, nil)
chatIDs := []int64{100, 200, 300}
for _, id := range chatIDs {
if _, err := addSubscriber(context.Background(), s.kv, id, 0); err != nil {
if _, err := addSubscriber(context.Background(), s.subscribers, id, 0); err != nil {
t.Fatalf("addSubscriber %d: %v", id, err)
}
}
@@ -220,7 +232,7 @@ func TestRunDailyPush_PartialFailureContinues(t *testing.T) {
// terminal error removes every topic subscription for that chat.
func TestRunDailyPush_PrunesDeadSubscribers(t *testing.T) {
s := newTestState(t)
seedFreshCache(t, s.kv, nil)
seedFreshCache(t, s.cache, nil)
// Chat 400 has two topic subs; both should be pruned when the chat
// returns a chat-wide terminal error.
@@ -232,7 +244,7 @@ func TestRunDailyPush_PrunesDeadSubscribers(t *testing.T) {
{ChatID: 400, ThreadID: 9},
}
for _, sub := range seedSubs {
if _, err := addSubscriber(context.Background(), s.kv, sub.ChatID, sub.ThreadID); err != nil {
if _, err := addSubscriber(context.Background(), s.subscribers, sub.ChatID, sub.ThreadID); err != nil {
t.Fatalf("addSubscriber %v: %v", sub, err)
}
}
@@ -245,7 +257,7 @@ func TestRunDailyPush_PrunesDeadSubscribers(t *testing.T) {
t.Fatalf("runDailyPush: %v", err)
}
remaining, err := listSubscribers(context.Background(), s.kv)
remaining, err := listSubscribers(context.Background(), s.subscribers)
if err != nil {
t.Fatalf("listSubscribers: %v", err)
}
@@ -265,7 +277,7 @@ func TestRunDailyPush_PrunesDeadSubscribers(t *testing.T) {
// (ChatID, ThreadID) entry — sister topics in the same chat stay subscribed.
func TestRunDailyPush_TopicOnlyTerminalPrunesOneTopic(t *testing.T) {
s := newTestState(t)
seedFreshCache(t, s.kv, nil)
seedFreshCache(t, s.cache, nil)
seedSubs := []Subscriber{
{ChatID: 500, ThreadID: 0},
@@ -273,7 +285,7 @@ func TestRunDailyPush_TopicOnlyTerminalPrunesOneTopic(t *testing.T) {
{ChatID: 500, ThreadID: 22},
}
for _, sub := range seedSubs {
if _, err := addSubscriber(context.Background(), s.kv, sub.ChatID, sub.ThreadID); err != nil {
if _, err := addSubscriber(context.Background(), s.subscribers, sub.ChatID, sub.ThreadID); err != nil {
t.Fatal(err)
}
}
@@ -286,7 +298,7 @@ func TestRunDailyPush_TopicOnlyTerminalPrunesOneTopic(t *testing.T) {
t.Fatalf("runDailyPush: %v", err)
}
remaining, _ := listSubscribers(context.Background(), s.kv)
remaining, _ := listSubscribers(context.Background(), s.subscribers)
if len(remaining) != 0 {
t.Errorf("expected all topic-only entries pruned, got %v", remaining)
}
@@ -296,7 +308,7 @@ func TestRunDailyPush_TopicOnlyTerminalPrunesOneTopic(t *testing.T) {
// chat goes bad; the other topics in the same chat must stay.
func TestRunDailyPush_TopicOnlyTerminalKeepsOtherTopics(t *testing.T) {
s := newTestState(t)
seedFreshCache(t, s.kv, nil)
seedFreshCache(t, s.cache, nil)
// Two distinct chats, each with multiple topic subs. Only chat 600 hits
// the topic-terminal error; chat 700 sends cleanly.
@@ -306,7 +318,7 @@ func TestRunDailyPush_TopicOnlyTerminalKeepsOtherTopics(t *testing.T) {
{ChatID: 700, ThreadID: 3},
}
for _, sub := range seedSubs {
if _, err := addSubscriber(context.Background(), s.kv, sub.ChatID, sub.ThreadID); err != nil {
if _, err := addSubscriber(context.Background(), s.subscribers, sub.ChatID, sub.ThreadID); err != nil {
t.Fatal(err)
}
}
@@ -317,7 +329,7 @@ func TestRunDailyPush_TopicOnlyTerminalKeepsOtherTopics(t *testing.T) {
t.Fatalf("runDailyPush: %v", err)
}
remaining, _ := listSubscribers(context.Background(), s.kv)
remaining, _ := listSubscribers(context.Background(), s.subscribers)
if len(remaining) != 1 || remaining[0] != (Subscriber{ChatID: 700, ThreadID: 3}) {
t.Errorf("after topic-terminal prune of chat 600: got %v, want [{700 3}]", remaining)
}
@@ -362,7 +374,7 @@ func TestClassifyTerminal(t *testing.T) {
func TestDailyPushHandler_NilBot_ReturnsError(t *testing.T) {
s := newTestState(t)
deps := modules.Deps{KV: s.kv} // Bot intentionally nil
deps := modules.Deps{Store: storage.NewMemoryProvider().Collection("lolschedule")}
err := s.dailyPushHandler(context.Background(), deps)
if err == nil {
t.Fatal("expected error when deps.Bot is nil, got nil")
+3 -12
View File
@@ -37,14 +37,8 @@ func TestFormatEventLine_Unstarted(t *testing.T) {
}
func TestFormatEventLine_Completed_BoldsWinner(t *testing.T) {
winResult := &struct {
Outcome string `json:"outcome,omitempty"`
GameWins int `json:"gameWins,omitempty"`
}{Outcome: "win", GameWins: 3}
loseResult := &struct {
Outcome string `json:"outcome,omitempty"`
GameWins int `json:"gameWins,omitempty"`
}{Outcome: "loss", GameWins: 1}
winResult := &TeamResult{Outcome: "win", GameWins: 3}
loseResult := &TeamResult{Outcome: "loss", GameWins: 1}
e := ScheduleEvent{
StartTime: "2026-05-09T05:00:00Z",
State: "completed",
@@ -73,10 +67,7 @@ func TestFormatEventLine_Completed_BoldsWinner(t *testing.T) {
}
func TestFormatEventLine_InProgress(t *testing.T) {
w := &struct {
Outcome string `json:"outcome,omitempty"`
GameWins int `json:"gameWins,omitempty"`
}{GameWins: 1}
w := &TeamResult{GameWins: 1}
e := ScheduleEvent{
StartTime: "2026-05-09T05:00:00Z",
State: "inProgress",
+8 -7
View File
@@ -10,17 +10,18 @@ import (
"github.com/tiennm99/miti99bot/internal/log"
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
"github.com/tiennm99/miti99bot/internal/storage"
)
// state captures everything a lolschedule handler needs at runtime.
type state struct {
kv storage.KVStore
client *Client
subscribers SubscriberStore
pushDate PushDateStore
cache CacheStore
client *Client
// nowFn allows tests to inject a deterministic clock. Production code
// uses time.Now via the default zero-value.
nowFn func() time.Time
// subscribersMu serializes Get→mutate→Put on the single subscribers KV
// subscribersMu serializes Get→mutate→Put on the single subscribers store
// slot. Two concurrent /lolschedule_subscribe calls in the same
// millisecond would otherwise race and drop one append.
subscribersMu sync.Mutex
@@ -72,7 +73,7 @@ func (s *state) handleWeek(ctx context.Context, b *bot.Bot, update *models.Updat
// replyForRange fetches + filters + renders a date window. week=true uses
// RenderWeek; false uses RenderToday.
func (s *state) replyForRange(ctx context.Context, b *bot.Bot, msg *models.Message, from, to time.Time, week bool) error {
events, err := s.client.GetEventsCached(ctx, s.kv, from, to)
events, err := s.client.GetEventsCached(ctx, s.cache, from, to)
if err != nil {
log.Error("lolschedule_fetch_fail", "err", err, "from", from, "to", to)
hint := "Could not fetch matches. Try again later."
@@ -100,7 +101,7 @@ func (s *state) handleSubscribe(ctx context.Context, b *bot.Bot, update *models.
}
s.subscribersMu.Lock()
defer s.subscribersMu.Unlock()
added, err := addSubscriber(ctx, s.kv, msg.Chat.ID, msg.MessageThreadID)
added, err := addSubscriber(ctx, s.subscribers, msg.Chat.ID, msg.MessageThreadID)
if err != nil {
return err
}
@@ -120,7 +121,7 @@ func (s *state) handleUnsubscribe(ctx context.Context, b *bot.Bot, update *model
}
s.subscribersMu.Lock()
defer s.subscribersMu.Unlock()
removed, err := removeSubscriber(ctx, s.kv, msg.Chat.ID, msg.MessageThreadID)
removed, err := removeSubscriber(ctx, s.subscribers, msg.Chat.ID, msg.MessageThreadID)
if err != nil {
return err
}
+21 -18
View File
@@ -16,7 +16,8 @@ import (
// installSchedule wires the lolschedule module to a recording bot, with a
// custom upstream HTTP server returning bodyJSON for every request. nowMs
// fixes the clock so date-based handlers are deterministic.
func installSchedule(t *testing.T, bodyJSON string, nowMs int64) (*testutil.RecordingBot, storage.KVStore) {
// Returns the recording bot and the subscriber store for inspection in tests.
func installSchedule(t *testing.T, bodyJSON string, nowMs int64) (*testutil.RecordingBot, SubscriberStore) {
t.Helper()
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
@@ -25,13 +26,14 @@ func installSchedule(t *testing.T, bodyJSON string, nowMs int64) (*testutil.Reco
t.Cleanup(upstream.Close)
rb := testutil.NewRecordingBot(t)
provider := storage.NewMemoryProvider()
kv := provider.For("lolschedule")
col := storage.NewMemoryProvider().Collection("lolschedule")
s := &state{
kv: kv,
client: &Client{HTTP: upstream.Client(), URL: upstream.URL},
nowFn: func() time.Time { return time.UnixMilli(nowMs).UTC() },
subscribers: storage.Typed[subscribersDoc](col),
pushDate: storage.Typed[lastPushDoc](col),
cache: storage.Typed[cacheRecord](col),
client: &Client{HTTP: upstream.Client(), URL: upstream.URL},
nowFn: func() time.Time { return time.UnixMilli(nowMs).UTC() },
}
mod := modules.Module{
Name: "lolschedule",
@@ -51,7 +53,7 @@ func installSchedule(t *testing.T, bodyJSON string, nowMs int64) (*testutil.Reco
reg.AllCommands[c.Name] = c
}
modules.Install(rb.Bot, reg, modules.Auth{})
return rb, kv
return rb, s.subscribers
}
// 2026-05-09 12:00 UTC = 19:00 ICT (still May 9 ICT day). Used as the fake
@@ -119,7 +121,7 @@ func TestHandleWeek_RendersWeek(t *testing.T) {
}
func TestHandleSubscribe_AddsAndIsIdempotent(t *testing.T) {
rb, kv := installSchedule(t, todayBody, fakeNowMs)
rb, subsStore := installSchedule(t, todayBody, fakeNowMs)
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(7, "/lolschedule_subscribe"))
if got := rb.LastSent().Text(); !strings.HasPrefix(got, "✅") {
t.Errorf("first subscribe should confirm; got %q", got)
@@ -129,7 +131,7 @@ func TestHandleSubscribe_AddsAndIsIdempotent(t *testing.T) {
if got := rb.LastSent().Text(); !strings.Contains(got, "Already subscribed") {
t.Errorf("duplicate subscribe should report Already; got %q", got)
}
ids, _ := listSubscribers(context.Background(), kv)
ids, _ := listSubscribers(context.Background(), subsStore)
if len(ids) != 1 || ids[0] != (Subscriber{ChatID: 7}) {
t.Errorf("subscribers = %v, want [{7 0}]", ids)
}
@@ -140,12 +142,12 @@ func TestHandleSubscribe_AddsAndIsIdempotent(t *testing.T) {
// records that thread on the Subscriber so the daily push lands in the
// originating topic instead of General.
func TestHandleSubscribe_ForumTopic_CapturesThreadID(t *testing.T) {
rb, kv := installSchedule(t, todayBody, fakeNowMs)
rb, subsStore := installSchedule(t, todayBody, fakeNowMs)
upd := testutil.NewSupergroupMessage(555, 999, "/lolschedule_subscribe")
upd.Message.MessageThreadID = 42
rb.Bot.ProcessUpdate(context.Background(), upd)
subs, _ := listSubscribers(context.Background(), kv)
subs, _ := listSubscribers(context.Background(), subsStore)
want := Subscriber{ChatID: 555, ThreadID: 42}
if len(subs) != 1 || subs[0] != want {
t.Errorf("subscribers = %v, want [%v]", subs, want)
@@ -156,7 +158,7 @@ func TestHandleSubscribe_ForumTopic_CapturesThreadID(t *testing.T) {
// unsubscribing from one topic does not affect sister-topic subscriptions
// in the same chat.
func TestHandleUnsubscribe_ForumTopic_RemovesOnlyThatTopic(t *testing.T) {
rb, kv := installSchedule(t, todayBody, fakeNowMs)
rb, subsStore := installSchedule(t, todayBody, fakeNowMs)
// Subscribe in two topics of the same supergroup.
for _, tid := range []int{42, 99} {
@@ -170,7 +172,7 @@ func TestHandleUnsubscribe_ForumTopic_RemovesOnlyThatTopic(t *testing.T) {
upd.Message.MessageThreadID = 42
rb.Bot.ProcessUpdate(context.Background(), upd)
subs, _ := listSubscribers(context.Background(), kv)
subs, _ := listSubscribers(context.Background(), subsStore)
want := Subscriber{ChatID: 555, ThreadID: 99}
if len(subs) != 1 || subs[0] != want {
t.Errorf("subscribers = %v, want [%v]", subs, want)
@@ -199,12 +201,13 @@ func TestHandleSchedule_UpstreamFailureGivesFriendlyError(t *testing.T) {
defer upstream.Close()
rb := testutil.NewRecordingBot(t)
provider := storage.NewMemoryProvider()
kv := provider.For("lolschedule")
col := storage.NewMemoryProvider().Collection("lolschedule")
s := &state{
kv: kv,
client: &Client{HTTP: upstream.Client(), URL: upstream.URL},
nowFn: func() time.Time { return time.UnixMilli(fakeNowMs).UTC() },
subscribers: storage.Typed[subscribersDoc](col),
pushDate: storage.Typed[lastPushDoc](col),
cache: storage.Typed[cacheRecord](col),
client: &Client{HTTP: upstream.Client(), URL: upstream.URL},
nowFn: func() time.Time { return time.UnixMilli(fakeNowMs).UTC() },
}
cmd := modules.Command{Name: "lolschedule_today", Visibility: modules.VisibilityPublic, Description: "x", Handler: s.handleToday}
reg := &modules.Registry{
+7 -1
View File
@@ -2,6 +2,7 @@ package lolschedule
import (
"github.com/tiennm99/miti99bot/internal/modules"
"github.com/tiennm99/miti99bot/internal/storage"
)
// New is the lolschedule module Factory. The 5 user-facing commands plus the
@@ -10,7 +11,12 @@ import (
// time — main.go must wire BuildOptions.Bot for the cron to function;
// without it the handler fails fast with a clear error.
func New(deps modules.Deps) modules.Module {
s := &state{kv: deps.KV, client: &Client{}}
s := &state{
subscribers: storage.Typed[subscribersDoc](deps.Store),
pushDate: storage.Typed[lastPushDoc](deps.Store),
cache: storage.Typed[cacheRecord](deps.Store),
client: &Client{},
}
return modules.Module{
Commands: []modules.Command{
{
+34 -15
View File
@@ -9,7 +9,7 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
// subscribersKey is the KV slot holding the per-module subscriber list.
// subscribersKey is the store slot holding the per-module subscriber list.
const subscribersKey = "subscribers"
// Subscriber is one row in the subscriber list. ThreadID is the Telegram
@@ -21,10 +21,19 @@ const subscribersKey = "subscribers"
// to the General topic, so carrying ThreadID alongside ChatID is what keeps
// the daily push landing in the topic the user subscribed from.
type Subscriber struct {
ChatID int64 `json:"chat_id"`
ThreadID int `json:"thread_id,omitempty"`
ChatID int64 `json:"chat_id" bson:"chat_id"`
ThreadID int `json:"thread_id,omitempty" bson:"thread_id,omitempty"`
}
// subscribersDoc wraps the subscriber list so it can be stored as a named
// root field in a Mongo document (a bare JSON array cannot be a root doc).
type subscribersDoc struct {
Subscribers []Subscriber `json:"subscribers" bson:"subscribers"`
}
// SubscriberStore is the typed store for subscriber documents.
type SubscriberStore = storage.DocStore[subscribersDoc]
// listSubscribers returns the current subscriber list, or an empty slice
// if none have ever subscribed.
//
@@ -36,14 +45,24 @@ type Subscriber struct {
// The decoder handles legacy rows indefinitely; any successful add/remove
// rewrites the slot in the new shape, but a no-op duplicate add leaves the
// legacy bytes in place — that's fine, the fallback path stays correct.
func listSubscribers(ctx context.Context, kv storage.KVStore) ([]Subscriber, error) {
raw, err := kv.Get(ctx, subscribersKey)
func listSubscribers(ctx context.Context, store SubscriberStore) ([]Subscriber, error) {
doc, _, err := store.Get(ctx, subscribersKey)
switch {
case errors.Is(err, storage.ErrNotFound):
return nil, nil
case err != nil:
return nil, fmt.Errorf("lolschedule listSubscribers: %w", err)
}
if doc.Subscribers != nil {
return doc.Subscribers, nil
}
return nil, nil
}
// listSubscribersLegacy decodes a raw JSON byte slice that may be either the
// current [{chat_id,thread_id}] shape or the legacy [int64] shape. Used by
// the legacy-migration path when a key was written by an older version.
func listSubscribersLegacy(raw []byte) ([]Subscriber, error) {
var subs []Subscriber
if err := json.Unmarshal(raw, &subs); err == nil {
return subs, nil
@@ -62,12 +81,12 @@ func listSubscribers(ctx context.Context, kv storage.KVStore) ([]Subscriber, err
// addSubscriber appends (chatID, threadID) if that exact pair is absent.
// Returns true on first-add, false when already subscribed (idempotent).
//
// Concurrency: the list lives in a single KV slot, so a concurrent
// Concurrency: the list lives in a single store slot, so a concurrent
// Get→mutate→Put from two chats subscribing in the same millisecond would
// drop one write. Callers MUST serialize through state.subscribersMu (or an
// equivalent module-scoped lock) before calling this.
func addSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, threadID int) (bool, error) {
subs, err := listSubscribers(ctx, kv)
func addSubscriber(ctx context.Context, store SubscriberStore, chatID int64, threadID int) (bool, error) {
subs, err := listSubscribers(ctx, store)
if err != nil {
return false, err
}
@@ -77,7 +96,7 @@ func addSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, thread
}
}
subs = append(subs, Subscriber{ChatID: chatID, ThreadID: threadID})
if err := kv.PutJSON(ctx, subscribersKey, subs); err != nil {
if err := store.Put(ctx, subscribersKey, subscribersDoc{Subscribers: subs}); err != nil {
return false, fmt.Errorf("lolschedule addSubscriber: %w", err)
}
return true, nil
@@ -88,8 +107,8 @@ func addSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, thread
//
// Concurrency: same single-slot Get→mutate→Put as addSubscriber; callers
// must hold state.subscribersMu.
func removeSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, threadID int) (bool, error) {
subs, err := listSubscribers(ctx, kv)
func removeSubscriber(ctx context.Context, store SubscriberStore, chatID int64, threadID int) (bool, error) {
subs, err := listSubscribers(ctx, store)
if err != nil {
return false, err
}
@@ -105,7 +124,7 @@ func removeSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, thr
if !removed {
return false, nil
}
if err := kv.PutJSON(ctx, subscribersKey, out); err != nil {
if err := store.Put(ctx, subscribersKey, subscribersDoc{Subscribers: out}); err != nil {
return false, fmt.Errorf("lolschedule removeSubscriber: %w", err)
}
return true, nil
@@ -118,8 +137,8 @@ func removeSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, thr
// number of entries actually removed.
//
// Concurrency: callers must hold state.subscribersMu.
func removeAllForChat(ctx context.Context, kv storage.KVStore, chatID int64) (int, error) {
subs, err := listSubscribers(ctx, kv)
func removeAllForChat(ctx context.Context, store SubscriberStore, chatID int64) (int, error) {
subs, err := listSubscribers(ctx, store)
if err != nil {
return 0, err
}
@@ -135,7 +154,7 @@ func removeAllForChat(ctx context.Context, kv storage.KVStore, chatID int64) (in
if removed == 0 {
return 0, nil
}
if err := kv.PutJSON(ctx, subscribersKey, out); err != nil {
if err := store.Put(ctx, subscribersKey, subscribersDoc{Subscribers: out}); err != nil {
return 0, fmt.Errorf("lolschedule removeAllForChat: %w", err)
}
return removed, nil
@@ -2,52 +2,57 @@ package lolschedule
import (
"context"
"encoding/json"
"testing"
"github.com/tiennm99/miti99bot/internal/storage"
)
func newSubscriberStore() SubscriberStore {
return storage.Typed[subscribersDoc](storage.NewMemoryProvider().Collection("lolschedule"))
}
func TestSubscribers_AddRemoveListIdempotent(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
store := newSubscriberStore()
got, _ := listSubscribers(ctx, kv)
got, _ := listSubscribers(ctx, store)
if len(got) != 0 {
t.Errorf("empty list = %v, want []", got)
}
added, err := addSubscriber(ctx, kv, 42, 0)
added, err := addSubscriber(ctx, store, 42, 0)
if err != nil || !added {
t.Fatalf("first add: added=%v err=%v", added, err)
}
added, _ = addSubscriber(ctx, kv, 42, 0)
added, _ = addSubscriber(ctx, store, 42, 0)
if added {
t.Errorf("re-add should be no-op")
}
got, _ = listSubscribers(ctx, kv)
got, _ = listSubscribers(ctx, store)
if len(got) != 1 || got[0] != (Subscriber{ChatID: 42}) {
t.Errorf("after add(42,0): %v, want [{42 0}]", got)
}
if _, err := addSubscriber(ctx, kv, 7, 0); err != nil {
if _, err := addSubscriber(ctx, store, 7, 0); err != nil {
t.Fatal(err)
}
got, _ = listSubscribers(ctx, kv)
got, _ = listSubscribers(ctx, store)
if len(got) != 2 {
t.Errorf("after add(7,0): %v, want len 2", got)
}
removed, err := removeSubscriber(ctx, kv, 42, 0)
removed, err := removeSubscriber(ctx, store, 42, 0)
if err != nil || !removed {
t.Fatalf("remove(42,0): removed=%v err=%v", removed, err)
}
removed, _ = removeSubscriber(ctx, kv, 42, 0)
removed, _ = removeSubscriber(ctx, store, 42, 0)
if removed {
t.Errorf("re-remove should be no-op")
}
got, _ = listSubscribers(ctx, kv)
got, _ = listSubscribers(ctx, store)
if len(got) != 1 || got[0].ChatID != 7 {
t.Errorf("after remove(42,0): %v, want [{7 0}]", got)
}
@@ -58,29 +63,29 @@ func TestSubscribers_AddRemoveListIdempotent(t *testing.T) {
// topic must not affect the others.
func TestSubscribers_TopicsAreDistinct(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
store := newSubscriberStore()
for _, tid := range []int{0, 5, 9} {
added, err := addSubscriber(ctx, kv, 100, tid)
added, err := addSubscriber(ctx, store, 100, tid)
if err != nil || !added {
t.Fatalf("add(100,%d): added=%v err=%v", tid, added, err)
}
}
// Same (chat, thread) is rejected as duplicate.
if added, _ := addSubscriber(ctx, kv, 100, 5); added {
if added, _ := addSubscriber(ctx, store, 100, 5); added {
t.Errorf("duplicate (100,5) should be no-op")
}
got, _ := listSubscribers(ctx, kv)
got, _ := listSubscribers(ctx, store)
if len(got) != 3 {
t.Fatalf("after 3 adds: %v, want len 3", got)
}
// Removing one topic leaves the other two intact.
if removed, _ := removeSubscriber(ctx, kv, 100, 5); !removed {
if removed, _ := removeSubscriber(ctx, store, 100, 5); !removed {
t.Error("remove(100,5) should report removed=true")
}
got, _ = listSubscribers(ctx, kv)
got, _ = listSubscribers(ctx, store)
if len(got) != 2 {
t.Errorf("after remove(100,5): %v, want len 2", got)
}
@@ -95,18 +100,18 @@ func TestSubscribers_TopicsAreDistinct(t *testing.T) {
// terminal error means every topic in that chat is dead too.
func TestSubscribers_RemoveAllForChat(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
store := newSubscriberStore()
for _, tid := range []int{0, 5, 9} {
if _, err := addSubscriber(ctx, kv, 100, tid); err != nil {
if _, err := addSubscriber(ctx, store, 100, tid); err != nil {
t.Fatal(err)
}
}
if _, err := addSubscriber(ctx, kv, 200, 0); err != nil {
if _, err := addSubscriber(ctx, store, 200, 0); err != nil {
t.Fatal(err)
}
n, err := removeAllForChat(ctx, kv, 100)
n, err := removeAllForChat(ctx, store, 100)
if err != nil {
t.Fatalf("removeAllForChat(100): %v", err)
}
@@ -114,48 +119,78 @@ func TestSubscribers_RemoveAllForChat(t *testing.T) {
t.Errorf("removeAllForChat(100) returned %d, want 3", n)
}
got, _ := listSubscribers(ctx, kv)
got, _ := listSubscribers(ctx, store)
if len(got) != 1 || got[0].ChatID != 200 {
t.Errorf("after wipe(100): %v, want only chat 200", got)
}
// Idempotent: second call removes nothing.
if n, _ := removeAllForChat(ctx, kv, 100); n != 0 {
if n, _ := removeAllForChat(ctx, store, 100); n != 0 {
t.Errorf("second removeAllForChat(100) = %d, want 0", n)
}
}
// TestSubscribers_LegacyDecode locks in backward-compat reads of rows
// written before topic-aware subscriptions (raw []int64 JSON).
// written before topic-aware subscriptions (raw []int64 JSON). The legacy
// helper decodes both shapes independently of the store.
func TestSubscribers_LegacyDecode(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
store := newSubscriberStore()
// Write legacy shape directly.
if err := kv.Put(ctx, subscribersKey, []byte(`[11,22,33]`)); err != nil {
// Write the current shape directly via Put so we can confirm round-trip.
currentSubs := []Subscriber{{ChatID: 11}, {ChatID: 22}, {ChatID: 33}}
if err := store.Put(ctx, subscribersKey, subscribersDoc{Subscribers: currentSubs}); err != nil {
t.Fatal(err)
}
got, err := listSubscribers(ctx, kv)
got, err := listSubscribers(ctx, store)
if err != nil {
t.Fatalf("listSubscribers: %v", err)
}
want := []Subscriber{{ChatID: 11}, {ChatID: 22}, {ChatID: 33}}
if len(got) != len(want) {
t.Fatalf("legacy decode: got %v, want %v", got, want)
t.Fatalf("decode: got %v, want %v", got, want)
}
for i, s := range got {
if s != want[i] {
t.Errorf("[%d]: got %v, want %v", i, s, want[i])
}
}
// Verify the legacy JSON helper handles the old []int64 shape.
legacyRaw := []byte(`[11,22,33]`)
legacySubs, err := listSubscribersLegacy(legacyRaw)
if err != nil {
t.Fatalf("listSubscribersLegacy: %v", err)
}
if len(legacySubs) != 3 {
t.Fatalf("legacy decode length = %d, want 3", len(legacySubs))
}
for i, s := range legacySubs {
if s != want[i] {
t.Errorf("legacy[%d]: got %v, want %v", i, s, want[i])
}
}
// Next mutation rewrites in the new shape — verify by reading raw.
if _, err := addSubscriber(ctx, kv, 44, 7); err != nil {
// Verify the legacy helper also handles the current [{chat_id,...}] shape.
currentRaw, _ := json.Marshal(currentSubs)
currentDecoded, err := listSubscribersLegacy(currentRaw)
if err != nil {
t.Fatalf("listSubscribersLegacy (current shape): %v", err)
}
if len(currentDecoded) != 3 {
t.Fatalf("current shape via legacy helper = %d, want 3", len(currentDecoded))
}
// Next mutation rewrites in the new shape — verify subscribers wrap correctly.
if _, err := addSubscriber(ctx, store, 44, 7); err != nil {
t.Fatal(err)
}
raw, _ := kv.Get(ctx, subscribersKey)
if string(raw) == `[11,22,33,44]` {
t.Errorf("expected rewrite to new shape, still legacy: %s", raw)
doc, _, _ := store.Get(ctx, subscribersKey)
if doc.Subscribers == nil {
t.Error("expected non-nil Subscribers field after add")
}
if len(doc.Subscribers) != 4 {
t.Errorf("after add: doc.Subscribers len = %d, want 4", len(doc.Subscribers))
}
}
+13 -12
View File
@@ -14,14 +14,15 @@ import (
)
// installMisc wires the misc module to a recording bot with a fresh
// in-memory KV. Returns the bot, the kv (so tests can pre-seed or read),
// and an Auth that permits Owner + Admin so /mstats /fortytwo dispatch.
func installMisc(t *testing.T, ownerID int64) (*testutil.RecordingBot, storage.KVStore) {
// in-memory store. Returns the bot and the typed store (so tests can pre-seed
// or read), plus an Auth that permits Owner + Admin so /mstats /fortytwo dispatch.
func installMisc(t *testing.T, ownerID int64) (*testutil.RecordingBot, storage.DocStore[lastPing]) {
t.Helper()
rb := testutil.NewRecordingBot(t)
provider := storage.NewMemoryProvider()
kv := provider.For("misc")
mod := New(modules.Deps{KV: kv})
coll := provider.Collection("misc")
mod := New(modules.Deps{Store: coll})
store := storage.Typed[lastPing](coll)
reg := &modules.Registry{
Modules: []modules.Module{{Name: "misc", Commands: mod.Commands}},
@@ -32,19 +33,19 @@ func installMisc(t *testing.T, ownerID int64) (*testutil.RecordingBot, storage.K
}
auth := modules.Auth{BotOwnerID: ownerID}
modules.Install(rb.Bot, reg, auth)
return rb, kv
return rb, store
}
func TestPing_RepliesPongAndWritesKV(t *testing.T) {
rb, kv := installMisc(t, 999)
func TestPing_RepliesPongAndWritesStore(t *testing.T) {
rb, store := installMisc(t, 999)
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(999, "/ping"))
if got := rb.LastSent().Text(); got != "pong" {
t.Errorf("ping reply = %q, want %q", got, "pong")
}
var stored lastPing
if err := kv.GetJSON(context.Background(), lastPingKey, &stored); err != nil {
t.Fatalf("expected lastPing in KV: %v", err)
stored, _, err := store.Get(context.Background(), lastPingKey)
if err != nil {
t.Fatalf("expected lastPing in store: %v", err)
}
if stored.At <= 0 {
t.Errorf("lastPing.At = %d, want positive", stored.At)
@@ -55,7 +56,7 @@ func TestPing_RepliesPongAndWritesKV(t *testing.T) {
}
}
func TestMstats_NeverWhenKVEmpty(t *testing.T) {
func TestMstats_NeverWhenStoreEmpty(t *testing.T) {
rb, _ := installMisc(t, 999)
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(999, "/mstats"))
+15 -15
View File
@@ -20,7 +20,7 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
// lastPingKey is the per-module KV key /ping writes and /mstats reads.
// lastPingKey is the per-module store key /ping writes and /mstats reads.
const lastPingKey = "last_ping"
// defaultTarget is the substituted "investigator" name when /trongtruonghop is
@@ -35,23 +35,24 @@ const trongTruongHopTemplate = "Trong trường hợp nhóm này bị điều tr
// int64 ms-epoch (not time.Time → RFC3339) keeps the on-disk shape compact
// and consistent with every other timestamp field in the bot's KV.
type lastPing struct {
At int64 `json:"at"`
At int64 `json:"at" bson:"at"`
}
// New is the module Factory. Captures the per-module Deps via closure so each
// command handler has direct access to its KV store.
// New is the module Factory. Builds a typed store from deps.Store and captures
// it via closure so each command handler has direct access.
func New(deps modules.Deps) modules.Module {
store := storage.Typed[lastPing](deps.Store)
return modules.Module{
Commands: []modules.Command{
pingCommand(deps),
mstatsCommand(deps),
pingCommand(store),
mstatsCommand(store),
fortytwoCommand(),
trongTruongHopCommand(),
},
}
}
func pingCommand(deps modules.Deps) modules.Command {
func pingCommand(store storage.DocStore[lastPing]) modules.Command {
return modules.Command{
Name: "ping",
Visibility: modules.VisibilityPublic,
@@ -60,17 +61,17 @@ func pingCommand(deps modules.Deps) modules.Command {
if update.Message == nil {
return nil
}
// Best-effort write — if KV is unavailable, still reply.
// Best-effort write — if store is unavailable, still reply.
payload := lastPing{At: chathelper.NowMillis()}
if err := deps.KV.PutJSON(ctx, lastPingKey, payload); err != nil {
log.Error("kv put failed", "module", "misc", "command", "ping", "key", lastPingKey, "err", err)
if err := store.Put(ctx, lastPingKey, payload); err != nil {
log.Error("store put failed", "module", "misc", "command", "ping", "key", lastPingKey, "err", err)
}
return chathelper.Reply(ctx, b, update.Message, "pong")
},
}
}
func mstatsCommand(deps modules.Deps) modules.Command {
func mstatsCommand(store storage.DocStore[lastPing]) modules.Command {
return modules.Command{
Name: "mstats",
Visibility: modules.VisibilityProtected,
@@ -79,18 +80,17 @@ func mstatsCommand(deps modules.Deps) modules.Command {
if update.Message == nil {
return nil
}
var last lastPing
text := "last ping: never"
err := deps.KV.GetJSON(ctx, lastPingKey, &last)
last, _, err := store.Get(ctx, lastPingKey)
switch {
case err == nil && last.At > 0:
text = fmt.Sprintf("last ping: %s",
time.UnixMilli(last.At).UTC().Format(time.RFC3339))
case err != nil && !errors.Is(err, storage.ErrNotFound):
// User-visible reply mirrors how stock/wordle/loldle handle
// transient KV failures — returning the error here would leave
// transient store failures — returning the error here would leave
// the user with no reply at all.
log.Error("kv get failed", "module", "misc", "command", "mstats", "key", lastPingKey, "err", err)
log.Error("store get failed", "module", "misc", "command", "mstats", "key", lastPingKey, "err", err)
text = "Could not load stats. Try again later."
}
return chathelper.Reply(ctx, b, update.Message, text)
+22 -30
View File
@@ -3,18 +3,22 @@ package misc
import (
"context"
"testing"
"time"
"github.com/tiennm99/miti99bot/internal/modules"
"github.com/tiennm99/miti99bot/internal/storage"
)
// We test the per-command KV behaviour directly — the bot/Telegram side is
// We test the per-command store behaviour directly — the bot/Telegram side is
// thin (single SendMessage) and exercising it would require a fake bot HTTP
// server. The KV interaction is the part with logic worth locking down.
// server. The store interaction is the part with logic worth locking down.
// newMiscStore returns a fresh in-memory typed lastPing store for tests.
func newMiscStore() storage.DocStore[lastPing] {
return storage.Typed[lastPing](storage.NewMemoryProvider().Collection("misc"))
}
func TestNew_RegistersExpectedCommands(t *testing.T) {
deps := modules.Deps{KV: storage.NewMemoryKVStore()}
deps := modules.Deps{Store: storage.NewMemoryProvider().Collection("misc")}
mod := New(deps)
want := map[string]modules.Visibility{
@@ -41,43 +45,31 @@ func TestNew_RegistersExpectedCommands(t *testing.T) {
}
}
func TestPing_WritesLastPingKV(t *testing.T) {
func TestPing_WritesLastPingStore(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
store := newMiscStore()
// Drive the KV side directly: lock the wire format as a ms-epoch number
// (not an RFC3339 string), matching every other timestamp in the bot's KV.
if err := kv.PutJSON(ctx, lastPingKey, lastPing{At: time.Now().UTC().UnixMilli()}); err != nil {
t.Fatalf("PutJSON: %v", err)
// Drive the store side directly: lock the wire format as a ms-epoch number
// (not an RFC3339 string), matching every other timestamp in the bot's store.
if err := store.Put(ctx, lastPingKey, lastPing{At: 1700000000000}); err != nil {
t.Fatalf("Put: %v", err)
}
var got lastPing
if err := kv.GetJSON(ctx, lastPingKey, &got); err != nil {
t.Fatalf("GetJSON: %v", err)
}
if got.At <= 0 {
t.Errorf("read-back lastPing.At = %d, want positive ms-epoch", got.At)
}
// Verify a hand-written {"at": <ms-epoch>} document decodes correctly.
if err := kv.Put(ctx, lastPingKey, []byte(`{"at":1700000000000}`)); err != nil {
t.Fatal(err)
}
got = lastPing{}
if err := kv.GetJSON(ctx, lastPingKey, &got); err != nil {
t.Fatalf("GetJSON ms-epoch shape: %v", err)
got, _, err := store.Get(ctx, lastPingKey)
if err != nil {
t.Fatalf("Get: %v", err)
}
if got.At != 1700000000000 {
t.Errorf("ms-epoch round-trip: At = %d, want 1700000000000", got.At)
}
}
func TestMstats_MissingKVReturnsErrNotFound(t *testing.T) {
func TestMstats_MissingKeyReturnsErrNotFound(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
store := newMiscStore()
var dst lastPing
if err := kv.GetJSON(ctx, lastPingKey, &dst); err != storage.ErrNotFound {
t.Errorf("GetJSON missing = %v, want ErrNotFound", err)
_, _, err := store.Get(ctx, lastPingKey)
if err != storage.ErrNotFound {
t.Errorf("Get missing = %v, want ErrNotFound", err)
}
}
+4 -4
View File
@@ -71,10 +71,10 @@ type Module struct {
// Modules that need to introspect commands (e.g. /help) capture this pointer
// in their handler closures.
type Deps struct {
KV storage.KVStore // already prefixed with the module name when passed to a Factory
Registry *Registry // populated by Build; safe to capture but read-only at module use
Chatter ai.Chatter // nil if GEMINI_API_KEY unset; twentyq must check
Bot *bot.Bot // nil-safe: only crons that fan-out (lolschedule daily push) need it
Store storage.Collection // the module's own collection; build typed views with storage.Typed[T]
Registry *Registry // populated by Build; safe to capture but read-only at module use
Chatter ai.Chatter // nil if GEMINI_API_KEY unset; twentyq must check
Bot *bot.Bot // nil-safe: only crons that fan-out (lolschedule daily push) need it
}
// Factory constructs a Module from its Deps. Deps are passed directly (instead
+8 -9
View File
@@ -83,11 +83,10 @@ func (r *Registry) Crons() []Cron {
return out
}
// Build constructs a Registry from the requested module names. The KVProvider
// supplies a per-module-isolated KVStore (MemoryProvider key-prefixes a shared
// store; FirestoreProvider hands out one collection per module). Build validates
// every command/cron and aborts on duplicate command names across the union of
// all visibilities.
// Build constructs a Registry from the requested module names. The Provider
// supplies a per-module-isolated Collection (one MongoDB collection per module;
// MemoryProvider keeps one map per module). Build validates every command/cron
// and aborts on duplicate command names across the union of all visibilities.
//
// Names not present in factories are reported as a single error so a typo in
// MODULES does not silently load a smaller bot than intended. Duplicate names
@@ -100,9 +99,9 @@ type BuildOptions struct {
Bot *bot.Bot
}
func Build(enabled []string, factories map[string]Factory, kv storage.KVProvider, opts BuildOptions) (*Registry, error) {
if kv == nil {
return nil, fmt.Errorf("modules: KVProvider is required")
func Build(enabled []string, factories map[string]Factory, provider storage.Provider, opts BuildOptions) (*Registry, error) {
if provider == nil {
return nil, fmt.Errorf("modules: storage Provider is required")
}
reg := &Registry{
@@ -135,7 +134,7 @@ func Build(enabled []string, factories map[string]Factory, kv storage.KVProvider
}
moduleDeps := Deps{
KV: kv.For(name),
Store: provider.Collection(name),
Registry: reg,
Chatter: opts.Chatter,
Bot: opts.Bot,
+22 -22
View File
@@ -35,7 +35,7 @@ func factory(name string, cmds []Command, crons []Cron) Factory {
}
}
func newProvider() storage.KVProvider { return storage.NewMemoryProvider() }
func newProvider() storage.Provider { return storage.NewMemoryProvider() }
func TestBuild_EmptyModulesBootsCleanly(t *testing.T) {
reg, err := Build(nil, map[string]Factory{}, newProvider(), BuildOptions{})
@@ -165,7 +165,7 @@ func TestDispatchScheduled_UnknownReturnsErrCronNotFound(t *testing.T) {
}
}
func TestDispatchScheduled_PassesPrefixedDeps(t *testing.T) {
func TestDispatchScheduled_PassesScopedDeps(t *testing.T) {
ctx := context.Background()
provider := storage.NewMemoryProvider()
@@ -174,7 +174,7 @@ func TestDispatchScheduled_PassesPrefixedDeps(t *testing.T) {
return Module{Crons: []Cron{{
Name: "tick_a",
Handler: func(ctx context.Context, deps Deps) error {
return deps.KV.Put(ctx, "last", []byte("A"))
return storage.Typed[string](deps.Store).Put(ctx, "last", "A")
},
}}}
},
@@ -182,7 +182,7 @@ func TestDispatchScheduled_PassesPrefixedDeps(t *testing.T) {
return Module{Crons: []Cron{{
Name: "tick_b",
Handler: func(ctx context.Context, deps Deps) error {
return deps.KV.Put(ctx, "last", []byte("B"))
return storage.Typed[string](deps.Store).Put(ctx, "last", "B")
},
}}}
},
@@ -198,14 +198,14 @@ func TestDispatchScheduled_PassesPrefixedDeps(t *testing.T) {
t.Fatalf("tick_b: %v", err)
}
// Underlying base store should hold each module's prefixed key separately.
gotA, err := provider.Base().Get(ctx, "alpha:last")
if err != nil || string(gotA) != "A" {
t.Errorf("alpha:last = %q (err=%v), want A", gotA, err)
// Each module's per-collection store holds its own "last" key separately.
gotA, _, err := storage.Typed[string](provider.Collection("alpha")).Get(ctx, "last")
if err != nil || gotA != "A" {
t.Errorf("alpha/last = %q (err=%v), want A", gotA, err)
}
gotB, err := provider.Base().Get(ctx, "beta:last")
if err != nil || string(gotB) != "B" {
t.Errorf("beta:last = %q (err=%v), want B", gotB, err)
gotB, _, err := storage.Typed[string](provider.Collection("beta")).Get(ctx, "last")
if err != nil || gotB != "B" {
t.Errorf("beta/last = %q (err=%v), want B", gotB, err)
}
}
@@ -280,12 +280,12 @@ func TestBuild_RejectsDuplicateModuleInEnv(t *testing.T) {
}
}
func TestBuild_PerModulePrefixedKV(t *testing.T) {
func TestBuild_PerModuleStoreIsolation(t *testing.T) {
ctx := context.Background()
provider := storage.NewMemoryProvider()
// Each module writes a value to the same key; with per-module prefixing
// they must not collide.
// Each module writes a value to the same key; with one collection per
// module they must not collide.
captured := map[string]Deps{}
factories := map[string]Factory{
"alpha": func(d Deps) Module {
@@ -301,19 +301,19 @@ func TestBuild_PerModulePrefixedKV(t *testing.T) {
t.Fatalf("Build: %v", err)
}
if err := captured["alpha"].KV.Put(ctx, "score", []byte("1")); err != nil {
alpha := storage.Typed[string](captured["alpha"].Store)
beta := storage.Typed[string](captured["beta"].Store)
if err := alpha.Put(ctx, "score", "1"); err != nil {
t.Fatal(err)
}
if err := captured["beta"].KV.Put(ctx, "score", []byte("2")); err != nil {
if err := beta.Put(ctx, "score", "2"); err != nil {
t.Fatal(err)
}
got, _ := captured["alpha"].KV.Get(ctx, "score")
if string(got) != "1" {
t.Errorf("alpha.KV.score = %q, want 1", got)
if got, _, _ := alpha.Get(ctx, "score"); got != "1" {
t.Errorf("alpha/score = %q, want 1", got)
}
got, _ = captured["beta"].KV.Get(ctx, "score")
if string(got) != "2" {
t.Errorf("beta.KV.score = %q, want 2", got)
if got, _, _ := beta.Get(ctx, "score"); got != "2" {
t.Errorf("beta/score = %q, want 2", got)
}
}
+33 -25
View File
@@ -1,5 +1,5 @@
// Package stats tracks per-command and per-user invocation counts persistently
// in KV and exposes /stats subcommands to display them sorted by popularity.
// in DocStore and exposes /stats subcommands to display them sorted by popularity.
package stats
import (
@@ -15,10 +15,11 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
// Sort-key shapes inside the stats module's KV partition:
// count:<cmd> → countEntry — total per command
// user:<userID> → userEntry — cached username + total per user
// pair:<cmd>:<userID> → countEntry — per (command, user) pair
// Sort-key shapes inside the stats module's partition:
//
// count:<cmd> → countEntry — total per command
// user:<userID> → userEntry — cached username + total per user
// pair:<cmd>:<userID> → countEntry — per (command, user) pair
const (
countPrefix = "count:"
userPrefix = "user:"
@@ -27,16 +28,20 @@ const (
)
type countEntry struct {
N int64 `json:"n"`
N int64 `json:"n" bson:"n"`
}
type userEntry struct {
Username string `json:"username"`
N int64 `json:"n"`
Username string `json:"username" bson:"username"`
N int64 `json:"n" bson:"n"`
}
// counter holds two typed views over the same module Collection:
// one for countEntry values and one for userEntry values.
// Keys are disjoint by prefix so both views safely share the collection.
type counter struct {
kv storage.KVStore
counts storage.DocStore[countEntry]
users storage.DocStore[userEntry]
}
func countKey(name string) string { return countPrefix + name }
@@ -87,48 +92,51 @@ func (c *counter) Inc(ctx context.Context, name string, update *models.Update) {
func (c *counter) incCount(ctx context.Context, name string) {
key := countKey(name)
var entry countEntry
if err := c.kv.GetJSON(ctx, key, &entry); err != nil && !errors.Is(err, storage.ErrNotFound) {
log.Error("stats: kv get failed", "key", key, "err", err)
entry, _, err := c.counts.Get(ctx, key)
if err != nil && !errors.Is(err, storage.ErrNotFound) {
log.Error("stats: store get failed", "key", key, "err", err)
return
}
entry.N++
if err := c.kv.PutJSON(ctx, key, entry); err != nil {
log.Error("stats: kv put failed", "key", key, "err", err)
if err := c.counts.Put(ctx, key, entry); err != nil {
log.Error("stats: store put failed", "key", key, "err", err)
}
}
func (c *counter) incUser(ctx context.Context, id int64, username string) {
key := userKey(id)
var entry userEntry
if err := c.kv.GetJSON(ctx, key, &entry); err != nil && !errors.Is(err, storage.ErrNotFound) {
log.Error("stats: kv get failed", "key", key, "err", err)
entry, _, err := c.users.Get(ctx, key)
if err != nil && !errors.Is(err, storage.ErrNotFound) {
log.Error("stats: store get failed", "key", key, "err", err)
return
}
entry.Username = username
entry.N++
if err := c.kv.PutJSON(ctx, key, entry); err != nil {
log.Error("stats: kv put failed", "key", key, "err", err)
if err := c.users.Put(ctx, key, entry); err != nil {
log.Error("stats: store put failed", "key", key, "err", err)
}
}
func (c *counter) incPair(ctx context.Context, name string, id int64) {
key := pairKey(name, id)
var entry countEntry
if err := c.kv.GetJSON(ctx, key, &entry); err != nil && !errors.Is(err, storage.ErrNotFound) {
log.Error("stats: kv get failed", "key", key, "err", err)
entry, _, err := c.counts.Get(ctx, key)
if err != nil && !errors.Is(err, storage.ErrNotFound) {
log.Error("stats: store get failed", "key", key, "err", err)
return
}
entry.N++
if err := c.kv.PutJSON(ctx, key, entry); err != nil {
log.Error("stats: kv put failed", "key", key, "err", err)
if err := c.counts.Put(ctx, key, entry); err != nil {
log.Error("stats: store put failed", "key", key, "err", err)
}
}
// New is the module Factory. Registers a CommandHook that persists counts and
// a /stats command that displays them.
func New(deps modules.Deps) modules.Module {
c := &counter{kv: deps.KV}
c := &counter{
counts: storage.Typed[countEntry](deps.Store),
users: storage.Typed[userEntry](deps.Store),
}
return modules.Module{
CommandHook: c.Inc,
Commands: []modules.Command{
+52 -46
View File
@@ -2,6 +2,7 @@ package stats
import (
"context"
"errors"
"fmt"
"strings"
"testing"
@@ -13,8 +14,17 @@ import (
"github.com/tiennm99/miti99bot/internal/testutil"
)
// newStatsCounter returns a fresh counter backed by an in-memory collection.
func newStatsCounter() *counter {
col := storage.NewMemoryProvider().Collection("stats")
return &counter{
counts: storage.Typed[countEntry](col),
users: storage.Typed[userEntry](col),
}
}
func TestNew_RegistersExpectedCommands(t *testing.T) {
deps := modules.Deps{KV: storage.NewMemoryKVStore()}
deps := modules.Deps{Store: storage.NewMemoryProvider().Collection("stats")}
mod := New(deps)
if len(mod.Commands) != 1 {
@@ -45,58 +55,56 @@ func updateFrom(id int64, username string) *models.Update {
}
}
func TestInc_PersistsCountInKV(t *testing.T) {
func TestInc_PersistsCountInStore(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
c := &counter{kv: kv}
c := newStatsCounter()
c.Inc(ctx, "ping", nil)
c.Inc(ctx, "ping", nil)
c.Inc(ctx, "wordle", nil)
var entry countEntry
if err := kv.GetJSON(ctx, countKey("ping"), &entry); err != nil {
t.Fatalf("GetJSON ping: %v", err)
entry, _, err := c.counts.Get(ctx, countKey("ping"))
if err != nil {
t.Fatalf("Get ping: %v", err)
}
if entry.N != 2 {
t.Errorf("ping count = %d, want 2", entry.N)
}
entry = countEntry{}
if err := kv.GetJSON(ctx, countKey("wordle"), &entry); err != nil {
t.Fatalf("GetJSON wordle: %v", err)
entry2, _, err := c.counts.Get(ctx, countKey("wordle"))
if err != nil {
t.Fatalf("Get wordle: %v", err)
}
if entry.N != 1 {
t.Errorf("wordle count = %d, want 1", entry.N)
if entry2.N != 1 {
t.Errorf("wordle count = %d, want 1", entry2.N)
}
}
func TestInc_WithUsernameWritesAllThreeKeys(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
c := &counter{kv: kv}
c := newStatsCounter()
c.Inc(ctx, "ping", updateFrom(42, "alice"))
c.Inc(ctx, "ping", updateFrom(42, "alice"))
var ce countEntry
if err := kv.GetJSON(ctx, countKey("ping"), &ce); err != nil {
ce, _, err := c.counts.Get(ctx, countKey("ping"))
if err != nil {
t.Fatalf("count:ping: %v", err)
}
if ce.N != 2 {
t.Errorf("count:ping N = %d, want 2", ce.N)
}
var ue userEntry
if err := kv.GetJSON(ctx, userKey(42), &ue); err != nil {
ue, _, err := c.users.Get(ctx, userKey(42))
if err != nil {
t.Fatalf("user:42: %v", err)
}
if ue.N != 2 || ue.Username != "alice" {
t.Errorf("user:42 = {%q, %d}, want {alice, 2}", ue.Username, ue.N)
}
var pe countEntry
if err := kv.GetJSON(ctx, pairKey("ping", 42), &pe); err != nil {
pe, _, err := c.counts.Get(ctx, pairKey("ping", 42))
if err != nil {
t.Fatalf("pair:ping:42: %v", err)
}
if pe.N != 2 {
@@ -106,37 +114,35 @@ func TestInc_WithUsernameWritesAllThreeKeys(t *testing.T) {
func TestInc_EmptyUsernameSkipsUserAndPair(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
c := &counter{kv: kv}
c := newStatsCounter()
c.Inc(ctx, "ping", updateFrom(42, ""))
var ce countEntry
if err := kv.GetJSON(ctx, countKey("ping"), &ce); err != nil {
ce, _, err := c.counts.Get(ctx, countKey("ping"))
if err != nil {
t.Fatalf("count:ping: %v", err)
}
if ce.N != 1 {
t.Errorf("count:ping N = %d, want 1", ce.N)
}
if _, err := kv.Get(ctx, userKey(42)); err != storage.ErrNotFound {
if _, _, err := c.users.Get(ctx, userKey(42)); !errors.Is(err, storage.ErrNotFound) {
t.Errorf("user:42 should be absent, got err=%v", err)
}
if _, err := kv.Get(ctx, pairKey("ping", 42)); err != storage.ErrNotFound {
if _, _, err := c.counts.Get(ctx, pairKey("ping", 42)); !errors.Is(err, storage.ErrNotFound) {
t.Errorf("pair:ping:42 should be absent, got err=%v", err)
}
}
func TestInc_RefreshesUsernameOnRename(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
c := &counter{kv: kv}
c := newStatsCounter()
c.Inc(ctx, "ping", updateFrom(42, "alice"))
c.Inc(ctx, "ping", updateFrom(42, "alice2"))
var ue userEntry
if err := kv.GetJSON(ctx, userKey(42), &ue); err != nil {
ue, _, err := c.users.Get(ctx, userKey(42))
if err != nil {
t.Fatalf("user:42: %v", err)
}
if ue.Username != "alice2" {
@@ -150,8 +156,7 @@ func TestInc_RefreshesUsernameOnRename(t *testing.T) {
func installStats(t *testing.T) (*testutil.RecordingBot, *counter) {
t.Helper()
rb := testutil.NewRecordingBot(t)
kv := storage.NewMemoryKVStore()
c := &counter{kv: kv}
c := newStatsCounter()
mod := modules.Module{
Commands: []modules.Command{statsCommand(c)},
}
@@ -226,10 +231,11 @@ func TestCommandHook_FiredThroughModulesBuild(t *testing.T) {
reg.RunCommandHooks(ctx, "ping", nil)
statsKV := provider.For("stats")
var entry countEntry
if err := statsKV.GetJSON(ctx, countKey("ping"), &entry); err != nil {
t.Fatalf("expected count:ping in KV after hook: %v", err)
// Verify by reading back via typed store (the same collection Build gave to stats).
statsStore := storage.Typed[countEntry](provider.Collection("stats"))
entry, _, err := statsStore.Get(ctx, countKey("ping"))
if err != nil {
t.Fatalf("expected count:ping in store after hook: %v", err)
}
if entry.N != 1 {
t.Errorf("count:ping = %d, want 1", entry.N)
@@ -256,7 +262,7 @@ func seedFixture(t *testing.T, c *counter) {
}
func TestRenderStats_TopCommands(t *testing.T) {
c := &counter{kv: storage.NewMemoryKVStore()}
c := newStatsCounter()
seedFixture(t, c)
got := renderStats(context.Background(), c, "")
@@ -275,7 +281,7 @@ func TestRenderStats_TopCommands(t *testing.T) {
}
func TestRenderStats_Users(t *testing.T) {
c := &counter{kv: storage.NewMemoryKVStore()}
c := newStatsCounter()
seedFixture(t, c)
got := renderStats(context.Background(), c, "users")
@@ -295,7 +301,7 @@ func TestRenderStats_Users(t *testing.T) {
}
func TestRenderStats_UserCommands(t *testing.T) {
c := &counter{kv: storage.NewMemoryKVStore()}
c := newStatsCounter()
seedFixture(t, c)
got := renderStats(context.Background(), c, "user @alice")
@@ -320,7 +326,7 @@ func TestRenderStats_UserCommands(t *testing.T) {
// flagged this as a potential bug; this test proves the absence of the bug.
func TestRenderStats_UserCommands_IDSuffixDoesNotFalseMatch(t *testing.T) {
ctx := context.Background()
c := &counter{kv: storage.NewMemoryKVStore()}
c := newStatsCounter()
// user 2 ("two") calls /ping; user 12 ("twelve") calls /wordle.
c.Inc(ctx, "ping", updateFrom(2, "two"))
c.Inc(ctx, "wordle", updateFrom(12, "twelve"))
@@ -338,7 +344,7 @@ func TestRenderStats_UserCommands_IDSuffixDoesNotFalseMatch(t *testing.T) {
}
func TestRenderStats_UserCommands_BareNameAlsoWorks(t *testing.T) {
c := &counter{kv: storage.NewMemoryKVStore()}
c := newStatsCounter()
seedFixture(t, c)
got := renderStats(context.Background(), c, "user alice")
@@ -348,7 +354,7 @@ func TestRenderStats_UserCommands_BareNameAlsoWorks(t *testing.T) {
}
func TestRenderStats_UserNotFound(t *testing.T) {
c := &counter{kv: storage.NewMemoryKVStore()}
c := newStatsCounter()
seedFixture(t, c)
got := renderStats(context.Background(), c, "user @ghost")
@@ -358,7 +364,7 @@ func TestRenderStats_UserNotFound(t *testing.T) {
}
func TestRenderStats_CmdUsers(t *testing.T) {
c := &counter{kv: storage.NewMemoryKVStore()}
c := newStatsCounter()
seedFixture(t, c)
got := renderStats(context.Background(), c, "cmd wordle")
@@ -380,7 +386,7 @@ func TestRenderStats_CmdUsers(t *testing.T) {
}
func TestRenderStats_CmdNotFound(t *testing.T) {
c := &counter{kv: storage.NewMemoryKVStore()}
c := newStatsCounter()
seedFixture(t, c)
got := renderStats(context.Background(), c, "cmd nonexistent")
@@ -390,7 +396,7 @@ func TestRenderStats_CmdNotFound(t *testing.T) {
}
func TestRenderStats_UnknownSubcommand(t *testing.T) {
c := &counter{kv: storage.NewMemoryKVStore()}
c := newStatsCounter()
got := renderStats(context.Background(), c, "bogus")
if !strings.HasPrefix(got, "Usage:") {
t.Errorf("expected usage string, got %q", got)
@@ -398,7 +404,7 @@ func TestRenderStats_UnknownSubcommand(t *testing.T) {
}
func TestRenderStats_MissingRequiredArg(t *testing.T) {
c := &counter{kv: storage.NewMemoryKVStore()}
c := newStatsCounter()
if got := renderStats(context.Background(), c, "user"); !strings.HasPrefix(got, "Usage:") {
t.Errorf("user (no arg) should return usage, got %q", got)
}
+20 -20
View File
@@ -89,9 +89,9 @@ func parseSubargs(update *models.Update) string {
}
func viewTopCommands(ctx context.Context, c *counter) string {
keys, err := c.kv.List(ctx, countPrefix)
keys, err := c.counts.List(ctx, countPrefix)
if err != nil {
log.Error("stats: kv list failed", "err", err)
log.Error("stats: store list failed", "err", err)
return "Could not load stats. Try again later."
}
if len(keys) == 0 {
@@ -145,9 +145,9 @@ func viewUserCommands(ctx context.Context, c *counter, username string) string {
return fmt.Sprintf("User @%s not found.", username)
}
keys, err := c.kv.List(ctx, pairPrefix)
keys, err := c.counts.List(ctx, pairPrefix)
if err != nil {
log.Error("stats: kv list failed", "err", err)
log.Error("stats: store list failed", "err", err)
return "Could not load stats. Try again later."
}
// Leading colon disambiguates: ":2" does not match a key ending in "12"
@@ -181,9 +181,9 @@ func viewUserCommands(ctx context.Context, c *counter, username string) string {
func viewCmdUsers(ctx context.Context, c *counter, cmd string) string {
prefix := pairPrefix + cmd + ":"
keys, err := c.kv.List(ctx, prefix)
keys, err := c.counts.List(ctx, prefix)
if err != nil {
log.Error("stats: kv list failed", "err", err)
log.Error("stats: store list failed", "err", err)
return "Could not load stats. Try again later."
}
if len(keys) == 0 {
@@ -200,9 +200,9 @@ func viewCmdUsers(ctx context.Context, c *counter, cmd string) string {
wg.Add(1)
go func(i int, k string) {
defer wg.Done()
var ce countEntry
if err := c.kv.GetJSON(ctx, k, &ce); err != nil {
log.Error("stats: kv get failed", "key", k, "err", err)
ce, _, err := c.counts.Get(ctx, k)
if err != nil {
log.Error("stats: store get failed", "key", k, "err", err)
return
}
idStr := strings.TrimPrefix(k, prefix)
@@ -210,8 +210,8 @@ func viewCmdUsers(ctx context.Context, c *counter, cmd string) string {
if err != nil {
return
}
var ue userEntry
if err := c.kv.GetJSON(ctx, userKey(id), &ue); err != nil || ue.Username == "" {
ue, _, err := c.users.Get(ctx, userKey(id))
if err != nil || ue.Username == "" {
return
}
results[i] = result{r: row{display: "@" + ue.Username, n: ce.N}, ok: true}
@@ -232,7 +232,7 @@ func viewCmdUsers(ctx context.Context, c *counter, cmd string) string {
return renderTopN(fmt.Sprintf("Users of /%s:", cmd), rows, topK)
}
// fanOutCountRows runs GetJSON on each key in parallel. displayFor maps the
// fanOutCountRows runs Get on each key in parallel. displayFor maps the
// raw sort key to the human-readable label that gets rendered. Missing /
// errored keys are skipped (logged at error level).
func fanOutCountRows(ctx context.Context, c *counter, keys []string, displayFor func(k string) string) []row {
@@ -246,9 +246,9 @@ func fanOutCountRows(ctx context.Context, c *counter, keys []string, displayFor
wg.Add(1)
go func(i int, k string) {
defer wg.Done()
var entry countEntry
if err := c.kv.GetJSON(ctx, k, &entry); err != nil {
log.Error("stats: kv get failed during render", "key", k, "err", err)
entry, _, err := c.counts.Get(ctx, k)
if err != nil {
log.Error("stats: store get failed during render", "key", k, "err", err)
return
}
results[i] = result{r: row{display: displayFor(k), n: entry.N}, ok: true}
@@ -268,9 +268,9 @@ func fanOutCountRows(ctx context.Context, c *counter, keys []string, displayFor
// loadUserRowsWithID returns userID → userEntry for every user:* row. Used by
// viewTopUsers (renders) and viewUserCommands (resolves a username to its ID).
func loadUserRowsWithID(ctx context.Context, c *counter) map[int64]userEntry {
keys, err := c.kv.List(ctx, userPrefix)
keys, err := c.users.List(ctx, userPrefix)
if err != nil {
log.Error("stats: kv list failed", "err", err)
log.Error("stats: store list failed", "err", err)
return nil
}
type pair struct {
@@ -289,9 +289,9 @@ func loadUserRowsWithID(ctx context.Context, c *counter) map[int64]userEntry {
if err != nil {
return
}
var entry userEntry
if err := c.kv.GetJSON(ctx, k, &entry); err != nil {
log.Error("stats: kv get failed", "key", k, "err", err)
entry, _, err := c.users.Get(ctx, k)
if err != nil {
log.Error("stats: store get failed", "key", k, "err", err)
return
}
results[i] = pair{id: id, entry: entry, ok: true}
+15 -16
View File
@@ -13,14 +13,13 @@ import (
"github.com/tiennm99/miti99bot/internal/keylock"
"github.com/tiennm99/miti99bot/internal/log"
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
"github.com/tiennm99/miti99bot/internal/storage"
)
// state is the per-module runtime. KV is module-scoped (the framework
// state is the per-module runtime. store is module-scoped (the framework
// prefixes/partitions). PriceClient is reused across calls; nowFn allows
// tests to inject a deterministic clock for portfolio CreatedAt.
type state struct {
kv storage.KVStore
store Store
prices *PriceClient
locks keylock.Map
nowFn func() time.Time
@@ -35,9 +34,9 @@ func (s *state) now() time.Time {
}
// newState builds the default state used by the module factory.
func newState(kv storage.KVStore) *state {
func newState(store Store) *state {
return &state{
kv: kv,
store: store,
prices: &PriceClient{},
comingSoonMessage: "Crypto, gold & currency exchange coming soon!",
}
@@ -87,14 +86,14 @@ func (s *state) handleTopup(ctx context.Context, b *bot.Bot, update *models.Upda
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
if err != nil {
log.Error("stock_load_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
}
p.AddCurrency("VND", amount)
p.Meta.Invested += amount
if err := SavePortfolio(ctx, s.kv, userID, p); err != nil {
if err := SavePortfolio(ctx, s.store, userID, p); err != nil {
log.Error("stock_save_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not save portfolio. Try again later.")
}
@@ -138,7 +137,7 @@ func (s *state) handleBuy(ctx context.Context, b *bot.Bot, update *models.Update
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
if err != nil {
log.Error("stock_load_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
@@ -149,7 +148,7 @@ func (s *state) handleBuy(ctx context.Context, b *bot.Bot, update *models.Update
"Insufficient VND. Need "+FormatVND(cost)+", have "+FormatVND(balance)+".")
}
p.AddAsset(symbol, qty)
if err := SavePortfolio(ctx, s.kv, userID, p); err != nil {
if err := SavePortfolio(ctx, s.store, userID, p); err != nil {
log.Error("stock_save_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not save portfolio. Try again later.")
}
@@ -196,7 +195,7 @@ func (s *state) handleSell(ctx context.Context, b *bot.Bot, update *models.Updat
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
if err != nil {
log.Error("stock_load_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
@@ -208,7 +207,7 @@ func (s *state) handleSell(ctx context.Context, b *bot.Bot, update *models.Updat
}
revenue := float64(qty) * price
p.AddCurrency("VND", revenue)
if err := SavePortfolio(ctx, s.kv, userID, p); err != nil {
if err := SavePortfolio(ctx, s.store, userID, p); err != nil {
log.Error("stock_save_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not save portfolio. Try again later.")
}
@@ -245,7 +244,7 @@ func (s *state) handleIncomeStock(ctx context.Context, b *bot.Bot, update *model
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
if err != nil {
log.Error("stock_load_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
@@ -256,7 +255,7 @@ func (s *state) handleIncomeStock(ctx context.Context, b *bot.Bot, update *model
"You don't hold any "+symbol+" to receive a stock dividend.")
}
p.AddAsset(symbol, qty)
if err := SavePortfolio(ctx, s.kv, userID, p); err != nil {
if err := SavePortfolio(ctx, s.store, userID, p); err != nil {
log.Error("stock_save_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not save portfolio. Try again later.")
}
@@ -292,7 +291,7 @@ func (s *state) handleIncomeVND(ctx context.Context, b *bot.Bot, update *models.
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
if err != nil {
log.Error("stock_load_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
@@ -304,7 +303,7 @@ func (s *state) handleIncomeVND(ctx context.Context, b *bot.Bot, update *models.
}
total := amountPerShare * float64(held)
p.AddCurrency("VND", total)
if err := SavePortfolio(ctx, s.kv, userID, p); err != nil {
if err := SavePortfolio(ctx, s.store, userID, p); err != nil {
log.Error("stock_save_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not save portfolio. Try again later.")
}
@@ -330,7 +329,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
return chathelper.Reply(ctx, b, update.Message,
"Cannot identify user — /stock_stats needs a sender.")
}
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
if err != nil {
log.Error("stock_load_portfolio", "user", userID, "err", err)
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
+13 -11
View File
@@ -9,19 +9,22 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
// Store is the stock module's typed portfolio store.
type Store = storage.DocStore[Portfolio]
// Portfolio is the per-user stock state. Currency is a map for forward-
// compat with USD/EUR (currently VND-only). Assets is a flat ticker→qty map.
type Portfolio struct {
Currency map[string]float64 `json:"currency"`
Assets map[string]int64 `json:"assets"`
Meta PortfolioMeta `json:"meta"`
Currency map[string]float64 `json:"currency" bson:"currency"`
Assets map[string]int64 `json:"assets" bson:"assets"`
Meta PortfolioMeta `json:"meta" bson:"meta"`
}
// PortfolioMeta tracks invested cost basis for P&L. CreatedAt is purely
// informational (ms-epoch when the portfolio first existed).
type PortfolioMeta struct {
Invested float64 `json:"invested"`
CreatedAt int64 `json:"createdAt"`
Invested float64 `json:"invested" bson:"invested"`
CreatedAt int64 `json:"createdAt" bson:"createdAt"`
}
// NewPortfolio returns an empty starting state. Currency map seeded with VND=0
@@ -39,11 +42,10 @@ func portfolioKey(userID int64) string {
return "user:" + strconv.FormatInt(userID, 10)
}
// LoadPortfolio reads from KV; returns an empty portfolio on first-time use.
// LoadPortfolio reads from store; returns an empty portfolio on first-time use.
// Defensively initialises nil maps so callers never need a nil check.
func LoadPortfolio(ctx context.Context, kv storage.KVStore, userID int64, now int64) (Portfolio, error) {
var p Portfolio
err := kv.GetJSON(ctx, portfolioKey(userID), &p)
func LoadPortfolio(ctx context.Context, store Store, userID int64, now int64) (Portfolio, error) {
p, _, err := store.Get(ctx, portfolioKey(userID))
switch {
case err == nil:
// Repair any nils from older / partial saves — defence in depth.
@@ -64,8 +66,8 @@ func LoadPortfolio(ctx context.Context, kv storage.KVStore, userID int64, now in
}
// SavePortfolio persists the portfolio.
func SavePortfolio(ctx context.Context, kv storage.KVStore, userID int64, p Portfolio) error {
if err := kv.PutJSON(ctx, portfolioKey(userID), p); err != nil {
func SavePortfolio(ctx context.Context, store Store, userID int64, p Portfolio) error {
if err := store.Put(ctx, portfolioKey(userID), p); err != nil {
return fmt.Errorf("stock: save portfolio %d: %w", userID, err)
}
return nil
+11 -6
View File
@@ -7,9 +7,14 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
// newStockStore returns a fresh in-memory typed portfolio store for tests.
func newStockStore() Store {
return storage.Typed[Portfolio](storage.NewMemoryProvider().Collection("stock"))
}
func TestLoadPortfolio_FirstTimeUser(t *testing.T) {
kv := storage.NewMemoryKVStore()
p, err := LoadPortfolio(context.Background(), kv, 42, 1234567890)
store := newStockStore()
p, err := LoadPortfolio(context.Background(), store, 42, 1234567890)
if err != nil {
t.Fatalf("LoadPortfolio: %v", err)
}
@@ -25,15 +30,15 @@ func TestLoadPortfolio_FirstTimeUser(t *testing.T) {
}
func TestSaveAndLoadRoundTrip(t *testing.T) {
kv := storage.NewMemoryKVStore()
p, _ := LoadPortfolio(context.Background(), kv, 42, 1)
store := newStockStore()
p, _ := LoadPortfolio(context.Background(), store, 42, 1)
p.AddCurrency("VND", 5_000_000)
p.AddAsset("TCB", 100)
p.Meta.Invested = 5_000_000
if err := SavePortfolio(context.Background(), kv, 42, p); err != nil {
if err := SavePortfolio(context.Background(), store, 42, p); err != nil {
t.Fatalf("Save: %v", err)
}
got, err := LoadPortfolio(context.Background(), kv, 42, 999) // CreatedAt should NOT be reset
got, err := LoadPortfolio(context.Background(), store, 42, 999) // CreatedAt should NOT be reset
if err != nil {
t.Fatalf("Load: %v", err)
}
+3 -4
View File
@@ -10,7 +10,6 @@ import (
"testing"
"time"
"github.com/tiennm99/miti99bot/internal/storage"
"github.com/tiennm99/miti99bot/internal/testutil"
)
@@ -37,19 +36,19 @@ func TestHandleStats_UsesSSIBatchPrices(t *testing.T) {
}))
t.Cleanup(priceSrv.Close)
kv := storage.NewMemoryKVStore()
store := newStockStore()
p := NewPortfolio(now.UnixMilli())
p.Currency["VND"] = 2335000
p.Meta.Invested = 1000000000
p.AddAsset("MWG", 1800)
p.AddAsset("TCB", 4200)
p.AddAsset("FPT", 2300)
if err := SavePortfolio(ctx, kv, 7, p); err != nil {
if err := SavePortfolio(ctx, store, 7, p); err != nil {
t.Fatalf("SavePortfolio: %v", err)
}
s := &state{
kv: kv,
store: store,
prices: &PriceClient{HTTP: priceSrv.Client(), URL: priceSrv.URL},
nowFn: func() time.Time { return now },
}
+2 -1
View File
@@ -2,13 +2,14 @@ package stock
import (
"github.com/tiennm99/miti99bot/internal/modules"
"github.com/tiennm99/miti99bot/internal/storage"
)
// New is the stock module Factory. Five user-facing commands; no crons.
// (Original miti99bot only has a SQL retention cron, which our KV-only port
// does not implement — keeping commits paper-ledger-only is acceptable.)
func New(deps modules.Deps) modules.Module {
s := newState(deps.KV)
s := newState(storage.Typed[Portfolio](deps.Store))
return modules.Module{
Commands: []modules.Command{
{
+12 -12
View File
@@ -15,7 +15,6 @@ import (
"github.com/tiennm99/miti99bot/internal/keylock"
"github.com/tiennm99/miti99bot/internal/log"
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
"github.com/tiennm99/miti99bot/internal/storage"
)
const (
@@ -26,7 +25,8 @@ const (
)
type state struct {
kv storage.KVStore
games GameStore
stats StatsStore
chatter ai.Chatter
limiter *ai.PerUserLimiter
@@ -88,7 +88,7 @@ func (s *state) handleTwentyq(ctx context.Context, b *bot.Bot, update *models.Up
arg := chathelper.ArgAfterCommand(msg.Text)
game, err := loadGame(ctx, s.kv, subject)
game, err := loadGame(ctx, s.games, subject)
if err != nil {
return err
}
@@ -99,7 +99,7 @@ func (s *state) handleTwentyq(ctx context.Context, b *bot.Bot, update *models.Up
// branch re-enters every call and the user is stuck. Log but keep
// going — the fresh-round write below will overwrite the slot
// regardless.
if err := clearGame(ctx, s.kv, subject); err != nil {
if err := clearGame(ctx, s.games, subject); err != nil {
log.Warn("twentyq clearGame on solved-relaunch failed", "subject", subject, "err", err)
}
game = nil
@@ -117,7 +117,7 @@ func (s *state) handleTwentyq(ctx context.Context, b *bot.Bot, update *models.Up
log.Warn("twentyq roundstart failed", "err", err)
return chathelper.Reply(ctx, b, msg, upstreamFail)
}
if err := saveGame(ctx, s.kv, subject, fresh); err != nil {
if err := saveGame(ctx, s.games, subject, fresh); err != nil {
return err
}
if arg == "" {
@@ -182,17 +182,17 @@ func (s *state) submitTurn(ctx context.Context, b *bot.Bot, msg *models.Message,
if won {
game.Solved = true
count := len(game.Turns)
if _, err := recordResult(ctx, s.kv, subject, true, count, chathelper.NowMillis()); err != nil {
if _, err := recordResult(ctx, s.stats, subject, true, count, chathelper.NowMillis()); err != nil {
return err
}
if err := clearGame(ctx, s.kv, subject); err != nil {
if err := clearGame(ctx, s.games, subject); err != nil {
return err
}
return chathelper.ReplyHTML(ctx, b, msg,
formatTurnReply(turn, true, game.Target, count))
}
if err := saveGame(ctx, s.kv, subject, game); err != nil {
if err := saveGame(ctx, s.games, subject, game); err != nil {
return err
}
return chathelper.ReplyHTML(ctx, b, msg,
@@ -209,17 +209,17 @@ func (s *state) handleGiveup(ctx context.Context, b *bot.Bot, update *models.Upd
return chathelper.Reply(ctx, b, msg, "Cannot identify chat.")
}
defer s.locks.Acquire(subject)()
game, err := loadGame(ctx, s.kv, subject)
game, err := loadGame(ctx, s.games, subject)
if err != nil {
return err
}
if game == nil {
return chathelper.ReplyHTML(ctx, b, msg, noRound)
}
if _, err := recordResult(ctx, s.kv, subject, false, len(game.Turns), chathelper.NowMillis()); err != nil {
if _, err := recordResult(ctx, s.stats, subject, false, len(game.Turns), chathelper.NowMillis()); err != nil {
return err
}
if err := clearGame(ctx, s.kv, subject); err != nil {
if err := clearGame(ctx, s.games, subject); err != nil {
return err
}
return chathelper.ReplyHTML(ctx, b, msg, formatGiveup(*game))
@@ -234,7 +234,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
if subject == "" {
return chathelper.Reply(ctx, b, msg, "Cannot identify chat.")
}
st, err := loadStats(ctx, s.kv, subject)
st, err := loadStats(ctx, s.stats, subject)
if err != nil {
return err
}
+33 -29
View File
@@ -10,36 +10,41 @@ import (
// Turn is one Q&A entry stored in the game's history.
type Turn struct {
Text string `json:"text"`
IsGuess bool `json:"isGuess"`
Answer string `json:"answer"` // "yes" | "no"
Hint string `json:"hint"`
TS int64 `json:"ts"`
Text string `json:"text" bson:"text"`
IsGuess bool `json:"isGuess" bson:"isGuess"`
Answer string `json:"answer" bson:"answer"` // "yes" | "no"
Hint string `json:"hint" bson:"hint"`
TS int64 `json:"ts" bson:"ts"`
}
type GameState struct {
Category string `json:"category"`
Target string `json:"target"`
InitialHint string `json:"initialHint"`
StartedAt *int64 `json:"startedAt"`
Solved bool `json:"solved"`
Turns []Turn `json:"turns"`
Category string `json:"category" bson:"category"`
Target string `json:"target" bson:"target"`
InitialHint string `json:"initialHint" bson:"initialHint"`
StartedAt *int64 `json:"startedAt" bson:"startedAt"`
Solved bool `json:"solved" bson:"solved"`
Turns []Turn `json:"turns" bson:"turns"`
}
type Stats struct {
Played int `json:"played"`
Solved int `json:"solved"`
TotalTurns int `json:"totalTurns"`
BestTurnCount *int `json:"bestTurnCount"`
LastResultAt *int64 `json:"lastResultAt"`
Played int `json:"played" bson:"played"`
Solved int `json:"solved" bson:"solved"`
TotalTurns int `json:"totalTurns" bson:"totalTurns"`
BestTurnCount *int `json:"bestTurnCount" bson:"bestTurnCount"`
LastResultAt *int64 `json:"lastResultAt" bson:"lastResultAt"`
}
// GameStore is the twentyq module's typed game store.
type GameStore = storage.DocStore[GameState]
// StatsStore is the twentyq module's typed stats store.
type StatsStore = storage.DocStore[Stats]
func gameKey(subject string) string { return "game:" + subject }
func statsKey(subject string) string { return "stats:" + subject }
func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*GameState, error) {
var g GameState
err := kv.GetJSON(ctx, gameKey(subject), &g)
func loadGame(ctx context.Context, games GameStore, subject string) (*GameState, error) {
g, _, err := games.Get(ctx, gameKey(subject))
switch {
case err == nil:
return &g, nil
@@ -50,23 +55,22 @@ func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*GameSta
}
}
func saveGame(ctx context.Context, kv storage.KVStore, subject string, g *GameState) error {
if err := kv.PutJSON(ctx, gameKey(subject), g); err != nil {
func saveGame(ctx context.Context, games GameStore, subject string, g *GameState) error {
if err := games.Put(ctx, gameKey(subject), *g); err != nil {
return fmt.Errorf("twentyq saveGame: %w", err)
}
return nil
}
func clearGame(ctx context.Context, kv storage.KVStore, subject string) error {
if err := kv.Delete(ctx, gameKey(subject)); err != nil && !errors.Is(err, storage.ErrNotFound) {
func clearGame(ctx context.Context, games GameStore, subject string) error {
if err := games.Delete(ctx, gameKey(subject)); err != nil && !errors.Is(err, storage.ErrNotFound) {
return fmt.Errorf("twentyq clearGame: %w", err)
}
return nil
}
func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*Stats, error) {
var s Stats
err := kv.GetJSON(ctx, statsKey(subject), &s)
func loadStats(ctx context.Context, st StatsStore, subject string) (*Stats, error) {
s, _, err := st.Get(ctx, statsKey(subject))
switch {
case err == nil:
return &s, nil
@@ -77,8 +81,8 @@ func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*Stats,
}
}
func recordResult(ctx context.Context, kv storage.KVStore, subject string, solved bool, turnCount int, nowMillis int64) (*Stats, error) {
s, err := loadStats(ctx, kv, subject)
func recordResult(ctx context.Context, st StatsStore, subject string, solved bool, turnCount int, nowMillis int64) (*Stats, error) {
s, err := loadStats(ctx, st, subject)
if err != nil {
return nil, err
}
@@ -93,7 +97,7 @@ func recordResult(ctx context.Context, kv storage.KVStore, subject string, solve
}
now := nowMillis
s.LastResultAt = &now
if err := kv.PutJSON(ctx, statsKey(subject), s); err != nil {
if err := st.Put(ctx, statsKey(subject), *s); err != nil {
return nil, fmt.Errorf("twentyq recordResult: %w", err)
}
return s, nil
+3 -1
View File
@@ -7,6 +7,7 @@ import (
"github.com/tiennm99/miti99bot/internal/ai"
"github.com/tiennm99/miti99bot/internal/modules"
"github.com/tiennm99/miti99bot/internal/storage"
)
// New is the twentyq module Factory. If Deps.Chatter is nil
@@ -14,7 +15,8 @@ import (
// config-error message — keeps the rest of the bot functional.
func New(deps modules.Deps) modules.Module {
s := &state{
kv: deps.KV,
games: storage.Typed[GameState](deps.Store),
stats: storage.Typed[Stats](deps.Store),
chatter: deps.Chatter,
limiter: ai.NewPerUserLimiter(5.0/60.0, 5),
rng: newRNG(),
+5 -3
View File
@@ -14,10 +14,12 @@ const (
ResultWrong = "wrong"
)
// LetterScore is the JSON shape stored in KV per guess.
// LetterScore is the shape stored per guess (nested in GameState). bson tags
// mirror the json names so migrated docs (which keep the original JSON keys)
// read back verbatim.
type LetterScore struct {
Letter string `json:"letter"`
Result string `json:"result"`
Letter string `json:"letter" bson:"letter"`
Result string `json:"result" bson:"result"`
}
// CompareWords scores guess against target letter-by-letter. Both are assumed
+15 -15
View File
@@ -10,13 +10,13 @@ import (
"github.com/tiennm99/miti99bot/internal/keylock"
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
"github.com/tiennm99/miti99bot/internal/storage"
)
// state captures everything a wordle command needs at handler-time. Built
// once in New and shared across all four handlers via closure.
type state struct {
kv storage.KVStore
games GameStore
stats StatsStore
words []string
set map[string]struct{}
locks keylock.Map // per-subject mutex; serialises Get→mutate→Put
@@ -35,7 +35,7 @@ func rejectMessage(reason rejectReason) string {
}
// startFresh writes a brand-new round and returns it. Errors propagate to
// the caller (not swallowed — a KV failure means subsequent ops will lie).
// the caller (not swallowed — a store failure means subsequent ops will lie).
func (s *state) startFresh(ctx context.Context, subject string) (*GameState, error) {
target, err := pickRandom(s.words, nil)
if err != nil {
@@ -48,14 +48,14 @@ func (s *state) startFresh(ctx context.Context, subject string) (*GameState, err
Giveup: false,
StartedAt: chathelper.NowMillis(),
}
if err := saveGame(ctx, s.kv, subject, g); err != nil {
if err := saveGame(ctx, s.games, subject, g); err != nil {
return nil, err
}
return g, nil
}
func (s *state) getOrInit(ctx context.Context, subject string) (*GameState, error) {
g, err := loadGame(ctx, s.kv, subject)
g, err := loadGame(ctx, s.games, subject)
if err != nil {
return nil, err
}
@@ -112,21 +112,21 @@ func (s *state) handleWordle(ctx context.Context, b *bot.Bot, update *models.Upd
if won {
g.Solved = true
}
if err := saveGame(ctx, s.kv, subject, g); err != nil {
if err := saveGame(ctx, s.games, subject, g); err != nil {
return err
}
rendered := renderGuess(v.Word, results)
switch {
case won:
stats, err := recordResult(ctx, s.kv, subject, true, chathelper.NowMillis())
st, err := recordResult(ctx, s.stats, subject, true, chathelper.NowMillis())
if err != nil {
return err
}
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("%s\n\n🎉 Solved in %d/%d! Streak: %d. /wordle_new for another.",
rendered, len(g.Guesses), MaxGuesses, stats.Streak))
rendered, len(g.Guesses), MaxGuesses, st.Streak))
case len(g.Guesses) >= MaxGuesses:
if _, err := recordResult(ctx, s.kv, subject, false, chathelper.NowMillis()); err != nil {
if _, err := recordResult(ctx, s.stats, subject, false, chathelper.NowMillis()); err != nil {
return err
}
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("%s\n\n❌ Out of guesses. Answer was %s. /wordle_new to retry.",
@@ -150,12 +150,12 @@ func (s *state) handleNew(ctx context.Context, b *bot.Bot, update *models.Update
defer s.locks.Acquire(subject)()
prelude := ""
prior, err := loadGame(ctx, s.kv, subject)
prior, err := loadGame(ctx, s.games, subject)
if err != nil {
return err
}
if prior != nil && !isFinished(prior) {
if _, err := recordResult(ctx, s.kv, subject, false, chathelper.NowMillis()); err != nil {
if _, err := recordResult(ctx, s.stats, subject, false, chathelper.NowMillis()); err != nil {
return err
}
prelude = fmt.Sprintf("🏳️ Previous round abandoned (auto-giveup). Answer was %s.\n\n",
@@ -191,10 +191,10 @@ func (s *state) handleGiveup(ctx context.Context, b *bot.Bot, update *models.Upd
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("Already gave up — %s.", strings.ToUpper(g.Target)))
}
g.Giveup = true
if err := saveGame(ctx, s.kv, subject, g); err != nil {
if err := saveGame(ctx, s.games, subject, g); err != nil {
return err
}
if _, err := recordResult(ctx, s.kv, subject, false, chathelper.NowMillis()); err != nil {
if _, err := recordResult(ctx, s.stats, subject, false, chathelper.NowMillis()); err != nil {
return err
}
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("🏳️ Answer was %s. /wordle_new for another.", strings.ToUpper(g.Target)))
@@ -210,7 +210,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
if subject == "" {
return chathelper.Reply(ctx, b, msg, "Cannot identify chat.")
}
stats, err := loadStats(ctx, s.kv, subject)
st, err := loadStats(ctx, s.stats, subject)
if err != nil {
return err
}
@@ -220,6 +220,6 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
}
return chathelper.Reply(ctx, b, msg, fmt.Sprintf(
"📊 Wordle %s stats\nPlayed: %d\nWins: %d (%d%%)\nCurrent streak: %d\nBest streak: %d",
scope, stats.Played, stats.Wins, chathelper.WinRate(stats.Wins, stats.Played), stats.Streak, stats.BestStreak,
scope, st.Played, st.Wins, chathelper.WinRate(st.Wins, st.Played), st.Streak, st.BestStreak,
))
}
+19 -16
View File
@@ -11,15 +11,15 @@ import (
)
// installWordle wires the wordle module to a recording bot with an
// in-memory KV. seedTarget pre-seeds a game with the supplied target
// in-memory store. seedTarget pre-seeds a game with the supplied target
// (skip dictionary randomness for deterministic tests). Empty seedTarget
// leaves the KV blank so handlers spin up a fresh round on first call.
func installWordle(t *testing.T, ownerID int64, seedSubject, seedTarget string) (*testutil.RecordingBot, storage.KVStore) {
// leaves the store blank so handlers spin up a fresh round on first call.
func installWordle(t *testing.T, ownerID int64, seedSubject, seedTarget string) (*testutil.RecordingBot, GameStore) {
t.Helper()
rb := testutil.NewRecordingBot(t)
provider := storage.NewMemoryProvider()
kv := provider.For("wordle")
mod := New(modules.Deps{KV: kv})
col := storage.NewMemoryProvider().Collection("wordle")
games := storage.Typed[GameState](col)
mod := New(modules.Deps{Store: col})
reg := &modules.Registry{
Modules: []modules.Module{{Name: "wordle", Commands: mod.Commands}},
AllCommands: map[string]modules.Command{},
@@ -31,11 +31,11 @@ func installWordle(t *testing.T, ownerID int64, seedSubject, seedTarget string)
if seedTarget != "" {
g := &GameState{Target: seedTarget, Guesses: []GuessRecord{}, StartedAt: 1}
if err := saveGame(context.Background(), kv, seedSubject, g); err != nil {
if err := saveGame(context.Background(), games, seedSubject, g); err != nil {
t.Fatalf("seed game: %v", err)
}
}
return rb, kv
return rb, games
}
func TestWordle_NoArgShowsBoard(t *testing.T) {
@@ -146,21 +146,24 @@ func TestWordleStats_AfterWin(t *testing.T) {
// Group chats key by chat id, so two private users both writing /wordle
// in the same group must mutate the same game.
func TestWordle_GroupSubjectIsChatID(t *testing.T) {
rb, kv := installWordle(t, 0, "-100", "crane")
rb, games := installWordle(t, 0, "-100", "crane")
rb.Bot.ProcessUpdate(context.Background(), testutil.NewGroupMessage(-100, 7, "/wordle crate"))
// After one guess in chat -100, subject "-100" should have a guess
// recorded; subject "7" should have no game.
var gChat GameState
if err := kv.GetJSON(context.Background(), gameKey("-100"), &gChat); err != nil {
gChat, err := loadGame(context.Background(), games, "-100")
if err != nil {
t.Fatalf("expected game at subject=-100: %v", err)
}
if len(gChat.Guesses) != 1 {
t.Errorf("group game has %d guesses, want 1", len(gChat.Guesses))
if gChat == nil || len(gChat.Guesses) != 1 {
t.Errorf("group game guesses = %v, want 1", gChat)
}
var gUser GameState
err := kv.GetJSON(context.Background(), gameKey("7"), &gUser)
if err == nil {
gUser, err := loadGame(context.Background(), games, "7")
if err != nil {
t.Fatalf("loadGame for user subject: %v", err)
}
if gUser != nil {
t.Errorf("user-keyed game leaked despite group context: %+v", gUser)
}
}
+30 -27
View File
@@ -11,7 +11,7 @@ import (
// MaxGuesses is the standard wordle round length.
const MaxGuesses = 6
// Note: the KV store has no native per-document TTL — saved games linger
// Note: the store has no native per-document TTL — saved games linger
// until manually cleaned. Out of scope today; could be added via a sweep
// cron if storage cost ever matters.
@@ -19,43 +19,47 @@ const MaxGuesses = 6
//
// { "word": "crane", "results": [{"letter":"c","result":"correct"}, ...] }
type GuessRecord struct {
Word string `json:"word"`
Results []LetterScore `json:"results"`
Word string `json:"word" bson:"word"`
Results []LetterScore `json:"results" bson:"results"`
}
// GameState is the per-subject KV record for an in-progress (or finished)
// round.
// GameState is the per-subject record for an in-progress (or finished) round.
//
// `giveup` is always emitted (initialized to false on /wordle_new). Do NOT
// add omitempty — the field is part of the stored document's shape, so
// emitting it unconditionally keeps already-saved games self-describing
// when inspected via raw KV dumps.
// when inspected via raw dumps.
type GameState struct {
Target string `json:"target"`
Guesses []GuessRecord `json:"guesses"`
Solved bool `json:"solved"`
Giveup bool `json:"giveup"`
StartedAt int64 `json:"startedAt"` // ms-since-epoch (Date.now())
Target string `json:"target" bson:"target"`
Guesses []GuessRecord `json:"guesses" bson:"guesses"`
Solved bool `json:"solved" bson:"solved"`
Giveup bool `json:"giveup" bson:"giveup"`
StartedAt int64 `json:"startedAt" bson:"startedAt"` // ms-since-epoch (Date.now())
}
// Stats is the lifetime score record. lastResultAt is *int64 so an unplayed
// account marshals as `"lastResultAt": null` — distinguishes "never played"
// from "played at epoch zero".
type Stats struct {
Played int `json:"played"`
Wins int `json:"wins"`
Streak int `json:"streak"`
BestStreak int `json:"bestStreak"`
LastResultAt *int64 `json:"lastResultAt"` // ms-since-epoch | null
Played int `json:"played" bson:"played"`
Wins int `json:"wins" bson:"wins"`
Streak int `json:"streak" bson:"streak"`
BestStreak int `json:"bestStreak" bson:"bestStreak"`
LastResultAt *int64 `json:"lastResultAt" bson:"lastResultAt"` // ms-since-epoch | null
}
// GameStore is the wordle module's typed game store.
type GameStore = storage.DocStore[GameState]
// StatsStore is the wordle module's typed stats store.
type StatsStore = storage.DocStore[Stats]
func gameKey(subject string) string { return "game:" + subject }
func statsKey(subject string) string { return "stats:" + subject }
// loadGame returns the active round, or (nil, nil) if none exists.
func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*GameState, error) {
var g GameState
err := kv.GetJSON(ctx, gameKey(subject), &g)
func loadGame(ctx context.Context, games GameStore, subject string) (*GameState, error) {
g, _, err := games.Get(ctx, gameKey(subject))
switch {
case err == nil:
return &g, nil
@@ -67,8 +71,8 @@ func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*GameSta
}
// saveGame writes the round.
func saveGame(ctx context.Context, kv storage.KVStore, subject string, g *GameState) error {
if err := kv.PutJSON(ctx, gameKey(subject), g); err != nil {
func saveGame(ctx context.Context, games GameStore, subject string, g *GameState) error {
if err := games.Put(ctx, gameKey(subject), *g); err != nil {
return fmt.Errorf("wordle saveGame: %w", err)
}
return nil
@@ -76,9 +80,8 @@ func saveGame(ctx context.Context, kv storage.KVStore, subject string, g *GameSt
// loadStats returns lifetime stats; missing → fresh-zero record (with
// LastResultAt=nil) so callers never need a nil check.
func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*Stats, error) {
var s Stats
err := kv.GetJSON(ctx, statsKey(subject), &s)
func loadStats(ctx context.Context, stats StatsStore, subject string) (*Stats, error) {
s, _, err := stats.Get(ctx, statsKey(subject))
switch {
case err == nil:
return &s, nil
@@ -92,8 +95,8 @@ func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*Stats,
// recordResult bumps stats with the round outcome (won true → win + streak,
// false → reset streak). Returns the updated stats so callers can show the
// new streak in the win message.
func recordResult(ctx context.Context, kv storage.KVStore, subject string, won bool, nowMillis int64) (*Stats, error) {
s, err := loadStats(ctx, kv, subject)
func recordResult(ctx context.Context, stats StatsStore, subject string, won bool, nowMillis int64) (*Stats, error) {
s, err := loadStats(ctx, stats, subject)
if err != nil {
return nil, err
}
@@ -109,7 +112,7 @@ func recordResult(ctx context.Context, kv storage.KVStore, subject string, won b
}
now := nowMillis
s.LastResultAt = &now
if err := kv.PutJSON(ctx, statsKey(subject), s); err != nil {
if err := stats.Put(ctx, statsKey(subject), *s); err != nil {
return nil, fmt.Errorf("wordle recordResult: %w", err)
}
return s, nil
+26 -11
View File
@@ -8,6 +8,21 @@ import (
"github.com/tiennm99/miti99bot/internal/storage"
)
func newWordleGames() GameStore {
return storage.Typed[GameState](storage.NewMemoryProvider().Collection("wordle"))
}
func newWordleStats() StatsStore {
return storage.Typed[Stats](storage.NewMemoryProvider().Collection("wordle"))
}
// newWordleStores returns a games + stats store backed by the same collection
// (disjoint key prefixes: "game:" vs "stats:").
func newWordleStores() (GameStore, StatsStore) {
c := storage.NewMemoryProvider().Collection("wordle")
return storage.Typed[GameState](c), storage.Typed[Stats](c)
}
func TestStats_DefaultLastResultAtIsNull(t *testing.T) {
// Go's *int64 must marshal as null when nil, so unplayed accounts emit
// `"lastResultAt": null` and the field stays distinguishable from
@@ -62,9 +77,9 @@ func TestGameState_JSONShapeIsStable(t *testing.T) {
func TestRecordResult_WinIncrementsStreak(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
stats := newWordleStats()
s, err := recordResult(ctx, kv, "u1", true, 100)
s, err := recordResult(ctx, stats, "u1", true, 100)
if err != nil {
t.Fatalf("recordResult: %v", err)
}
@@ -76,7 +91,7 @@ func TestRecordResult_WinIncrementsStreak(t *testing.T) {
}
// Second win bumps streak; bestStreak follows.
s, _ = recordResult(ctx, kv, "u1", true, 200)
s, _ = recordResult(ctx, stats, "u1", true, 200)
if s.Streak != 2 || s.BestStreak != 2 {
t.Errorf("two wins: streak=%d best=%d", s.Streak, s.BestStreak)
}
@@ -84,11 +99,11 @@ func TestRecordResult_WinIncrementsStreak(t *testing.T) {
func TestRecordResult_LossResetsStreakKeepsBest(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
stats := newWordleStats()
_, _ = recordResult(ctx, kv, "u1", true, 100)
_, _ = recordResult(ctx, kv, "u1", true, 200)
s, _ := recordResult(ctx, kv, "u1", false, 300)
_, _ = recordResult(ctx, stats, "u1", true, 100)
_, _ = recordResult(ctx, stats, "u1", true, 200)
s, _ := recordResult(ctx, stats, "u1", false, 300)
if s.Streak != 0 {
t.Errorf("loss should reset streak, got %d", s.Streak)
}
@@ -101,7 +116,7 @@ func TestRecordResult_LossResetsStreakKeepsBest(t *testing.T) {
}
func TestLoadGame_MissingReturnsNil(t *testing.T) {
g, err := loadGame(context.Background(), storage.NewMemoryKVStore(), "nobody")
g, err := loadGame(context.Background(), newWordleGames(), "nobody")
if err != nil {
t.Errorf("missing game should not error: %v", err)
}
@@ -112,12 +127,12 @@ func TestLoadGame_MissingReturnsNil(t *testing.T) {
func TestSaveGame_RoundTrip(t *testing.T) {
ctx := context.Background()
kv := storage.NewMemoryKVStore()
games := newWordleGames()
want := &GameState{Target: "crane", Guesses: []GuessRecord{}, StartedAt: 42}
if err := saveGame(ctx, kv, "u1", want); err != nil {
if err := saveGame(ctx, games, "u1", want); err != nil {
t.Fatal(err)
}
got, err := loadGame(ctx, kv, "u1")
got, err := loadGame(ctx, games, "u1")
if err != nil || got == nil {
t.Fatalf("loadGame: got=%v err=%v", got, err)
}
+8 -2
View File
@@ -2,13 +2,19 @@ package wordle
import (
"github.com/tiennm99/miti99bot/internal/modules"
"github.com/tiennm99/miti99bot/internal/storage"
)
// New is the wordle module Factory. Loads the embedded dictionary once,
// captures the per-module KV via closure, and registers all four commands.
// captures the per-module typed stores via closure, and registers all four commands.
func New(deps modules.Deps) modules.Module {
words, set := loadWords()
s := &state{kv: deps.KV, words: words, set: set}
s := &state{
games: storage.Typed[GameState](deps.Store),
stats: storage.Typed[Stats](deps.Store),
words: words,
set: set,
}
return modules.Module{
Commands: []modules.Command{
+156
View File
@@ -0,0 +1,156 @@
package storage
import (
"context"
"errors"
"fmt"
"reflect"
"strings"
)
// ErrNotFound is returned by a DocStore when a key has no value.
var ErrNotFound = errors.New("storage: key not found")
// ErrConflict is returned by versioned writes when the stored version changed.
var ErrConflict = errors.New("storage: write conflict")
// ErrInvalidModuleName is returned by every op on a store built from an invalid
// module/collection name — surfacing the configuration bug at first use rather
// than silently reading/writing an attacker-controllable collection name.
var ErrInvalidModuleName = errors.New("storage: invalid module name")
// DocStore is the per-module, per-payload-type storage contract. A value is
// persisted as a native document (its fields hoisted to the document root)
// alongside an optimistic-locking version. Implementations are safe for
// concurrent use and return ErrNotFound for missing keys.
//
// Concurrency control is version-based: each key carries a monotonic version, so
// a writer swaps only if the version it read is unchanged. This decouples the
// concurrency control from the value encoding (values are native documents, not
// exact byte blobs).
//
// Contract:
// - Get returns the value and its current version, or ErrNotFound.
// - Put overwrites unconditionally and bumps the version.
// - PutVersioned writes only if the stored version equals expectedVersion,
// then bumps it. expectedVersion == 0 means "create (or adopt a not-yet-
// versioned key)". A mismatch returns ErrConflict.
type DocStore[T any] interface {
Get(ctx context.Context, id string) (val T, version int64, err error)
Put(ctx context.Context, id string, val T) error
PutVersioned(ctx context.Context, id string, expectedVersion int64, val T) error
Delete(ctx context.Context, id string) error
List(ctx context.Context, prefix string) ([]string, error)
}
// Provider yields a per-module Collection handle. Implementations decide how
// isolation is achieved: MemoryProvider keeps one map per module, MongoProvider
// uses one MongoDB collection per module. Modules never construct stores
// directly — they receive a Collection through their factory's Deps and build
// typed views with Typed.
type Provider interface {
Collection(module string) Collection
}
// Collection is an opaque per-module storage handle. Build a typed store over it
// with Typed[T]. A module may build several typed views over the same
// Collection (one per payload type) as long as their key prefixes are disjoint.
type Collection interface {
// isCollection seals the interface to this package's implementations so
// Typed's type switch is exhaustive.
isCollection()
}
// Typed returns a DocStore[T] over c. Go methods cannot be generic, so this is a
// package-level function rather than a Provider method; it is the single switch
// point that binds a Collection to its backend's typed store.
//
// It panics if T is a struct whose BSON field names collide with a reserved root
// field (_id, version, updatedAt) — a programmer error caught at startup, in the
// same spirit as Prefixed panicking on an empty prefix.
func Typed[T any](c Collection) DocStore[T] {
if err := checkReservedFields[T](); err != nil {
panic(err)
}
switch h := c.(type) {
case mongoCollection:
return &mongoDocStore[T]{coll: h.coll, module: h.module}
case *memoryCollection:
return &memoryDocStore[T]{c: h}
case invalidCollection:
return invalidDocStore[T]{name: h.name}
default:
panic(fmt.Sprintf("storage: unknown collection type %T", c))
}
}
// reservedRootFields are the document-root field names the store owns; a payload
// type must not define BSON tags that collide with them (inline marshalling
// would otherwise produce duplicate keys).
var reservedRootFields = map[string]bool{
mongoIDField: true,
mongoVersionField: true,
mongoUpdatedAtField: true,
}
// checkReservedFields reports an error if struct type T declares a BSON field
// name colliding with a reserved root field. Non-struct T (e.g. memory-only
// test usage) is skipped — only the inline Mongo encoding needs the guard.
func checkReservedFields[T any]() error {
t := reflect.TypeOf((*T)(nil)).Elem()
if t.Kind() != reflect.Struct {
return nil
}
for i := 0; i < t.NumField(); i++ {
f := t.Field(i)
if !f.IsExported() {
continue
}
name := bsonFieldName(f)
if reservedRootFields[name] {
return fmt.Errorf("storage: payload type %s field %q maps to reserved root field %q", t.Name(), f.Name, name)
}
}
return nil
}
// bsonFieldName returns the BSON document field name for a struct field: the
// explicit `bson:"name"` tag if present, else the Go field name (matching the
// driver's default of using the field name verbatim).
func bsonFieldName(f reflect.StructField) string {
tag := f.Tag.Get("bson")
if tag == "" {
return f.Name
}
name := strings.Split(tag, ",")[0]
if name == "" {
return f.Name
}
return name
}
// invalidCollection is returned by a Provider for a module name that fails
// validation. Typed turns it into an invalidDocStore.
type invalidCollection struct{ name string }
func (invalidCollection) isCollection() {}
// invalidDocStore errors on every operation, wrapping ErrInvalidModuleName.
type invalidDocStore[T any] struct{ name string }
func (s invalidDocStore[T]) err(op string) error {
return fmt.Errorf("%w: %q (op=%s)", ErrInvalidModuleName, s.name, op)
}
func (s invalidDocStore[T]) Get(context.Context, string) (T, int64, error) {
var z T
return z, 0, s.err("Get")
}
func (s invalidDocStore[T]) Put(context.Context, string, T) error { return s.err("Put") }
func (s invalidDocStore[T]) PutVersioned(context.Context, string, int64, T) error {
return s.err("PutVersioned")
}
func (s invalidDocStore[T]) Delete(context.Context, string) error { return s.err("Delete") }
func (s invalidDocStore[T]) List(context.Context, string) ([]string, error) {
return nil, s.err("List")
}
-181
View File
@@ -1,181 +0,0 @@
package storage
import (
"context"
"encoding/json"
"fmt"
"strconv"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)
// DynamoDB attribute names. `value` is reserved by DynamoDB expressions, so
// every reference goes through ExpressionAttributeNames (#v).
const (
dynamoPKAttr = "pk"
dynamoSKAttr = "sk"
dynamoValueAttr = "value"
dynamoUpdatedAtAttr = "updatedAt"
)
// DynamoDBKVStore is a KVStore backed by a single DynamoDB table with a
// composite key (pk, sk). FirestoreProvider uses one collection per module;
// DynamoDBProvider uses one partition (pk = moduleName) per module — the
// table itself is shared. The user-supplied key becomes the sort key, so
// List(prefix) maps to a Query with begins_with(sk, prefix).
type DynamoDBKVStore struct {
client *dynamodb.Client
table string
moduleName string
}
// NewDynamoDBKVStore returns a store partition-scoped to moduleName.
// Callers must validate moduleName beforehand (DynamoDBProvider does).
func NewDynamoDBKVStore(client *dynamodb.Client, table, moduleName string) *DynamoDBKVStore {
return &DynamoDBKVStore{client: client, table: table, moduleName: moduleName}
}
// Get returns the raw bytes stored at key, or ErrNotFound. Strong read for
// parity with the Firestore impl's strong-read default.
func (s *DynamoDBKVStore) Get(ctx context.Context, key string) ([]byte, error) {
if err := validateKey(key); err != nil {
return nil, err
}
out, err := s.client.GetItem(ctx, &dynamodb.GetItemInput{
TableName: aws.String(s.table),
ConsistentRead: aws.Bool(true),
Key: map[string]types.AttributeValue{
dynamoPKAttr: &types.AttributeValueMemberS{Value: s.moduleName},
dynamoSKAttr: &types.AttributeValueMemberS{Value: key},
},
ExpressionAttributeNames: map[string]string{"#v": dynamoValueAttr},
ProjectionExpression: aws.String("#v"),
})
if err != nil {
return nil, fmt.Errorf("dynamodb get %s/%s: %w", s.moduleName, key, err)
}
if len(out.Item) == 0 {
return nil, ErrNotFound
}
rawAttr, ok := out.Item[dynamoValueAttr]
if !ok {
return nil, fmt.Errorf("dynamodb get %s/%s: missing %q attribute", s.moduleName, key, dynamoValueAttr)
}
str, ok := rawAttr.(*types.AttributeValueMemberS)
if !ok {
return nil, fmt.Errorf("dynamodb get %s/%s: unexpected attribute type %T", s.moduleName, key, rawAttr)
}
return []byte(str.Value), nil
}
// GetJSON decodes the value at key into dst.
func (s *DynamoDBKVStore) GetJSON(ctx context.Context, key string, dst any) error {
raw, err := s.Get(ctx, key)
if err != nil {
return err
}
if err := json.Unmarshal(raw, dst); err != nil {
return fmt.Errorf("dynamodb get %s/%s: json decode: %w", s.moduleName, key, err)
}
return nil
}
// Put writes raw bytes at key, creating or overwriting. The value attribute
// is stored as a DynamoDB String so it is human-readable in the AWS console.
// Every current caller writes JSON, which is UTF-8 safe; non-UTF-8 callers
// must encode upstream (e.g. base64).
func (s *DynamoDBKVStore) Put(ctx context.Context, key string, val []byte) error {
if err := validateKey(key); err != nil {
return err
}
_, err := s.client.PutItem(ctx, &dynamodb.PutItemInput{
TableName: aws.String(s.table),
Item: map[string]types.AttributeValue{
dynamoPKAttr: &types.AttributeValueMemberS{Value: s.moduleName},
dynamoSKAttr: &types.AttributeValueMemberS{Value: key},
dynamoValueAttr: &types.AttributeValueMemberS{Value: string(val)},
dynamoUpdatedAtAttr: &types.AttributeValueMemberN{Value: strconv.FormatInt(time.Now().UTC().UnixNano(), 10)},
},
})
if err != nil {
return fmt.Errorf("dynamodb put %s/%s: %w", s.moduleName, key, err)
}
return nil
}
// Note: DynamoDB intentionally does NOT implement VersionedStore. Post-self-host
// it is reachable only via the data migrator (Scan + Put), which never needs
// optimistic locking. The live runtime backend is MongoDB; memory covers tests.
// PutJSON marshals val and writes the bytes at key.
func (s *DynamoDBKVStore) PutJSON(ctx context.Context, key string, val any) error {
raw, err := json.Marshal(val)
if err != nil {
return fmt.Errorf("dynamodb put %s/%s: json encode: %w", s.moduleName, key, err)
}
return s.Put(ctx, key, raw)
}
// Delete removes the item at key. Deleting a missing key is not an error.
func (s *DynamoDBKVStore) Delete(ctx context.Context, key string) error {
if err := validateKey(key); err != nil {
return err
}
_, err := s.client.DeleteItem(ctx, &dynamodb.DeleteItemInput{
TableName: aws.String(s.table),
Key: map[string]types.AttributeValue{
dynamoPKAttr: &types.AttributeValueMemberS{Value: s.moduleName},
dynamoSKAttr: &types.AttributeValueMemberS{Value: key},
},
})
if err != nil {
return fmt.Errorf("dynamodb delete %s/%s: %w", s.moduleName, key, err)
}
return nil
}
// List returns all sort keys in the partition that start with prefix. Empty
// prefix scans the entire partition (Query is still cheap because it's
// partition-bounded; no Scan).
func (s *DynamoDBKVStore) List(ctx context.Context, prefix string) ([]string, error) {
if err := validatePrefix(prefix); err != nil {
return nil, err
}
input := &dynamodb.QueryInput{
TableName: aws.String(s.table),
ProjectionExpression: aws.String(dynamoSKAttr),
ExpressionAttributeValues: map[string]types.AttributeValue{
":pk": &types.AttributeValueMemberS{Value: s.moduleName},
},
}
if prefix == "" {
input.KeyConditionExpression = aws.String("pk = :pk")
} else {
input.KeyConditionExpression = aws.String("pk = :pk AND begins_with(sk, :prefix)")
input.ExpressionAttributeValues[":prefix"] = &types.AttributeValueMemberS{Value: prefix}
}
var keys []string
pager := dynamodb.NewQueryPaginator(s.client, input)
for pager.HasMorePages() {
page, err := pager.NextPage(ctx)
if err != nil {
return nil, fmt.Errorf("dynamodb list %s prefix=%q: %w", s.moduleName, prefix, err)
}
for _, item := range page.Items {
skAttr, ok := item[dynamoSKAttr]
if !ok {
continue
}
sk, ok := skAttr.(*types.AttributeValueMemberS)
if !ok {
continue
}
keys = append(keys, sk.Value)
}
}
return keys, nil
}
-166
View File
@@ -1,166 +0,0 @@
package storage
import (
"context"
"errors"
"os"
"testing"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)
// dynamoDBLocalSetup ensures a fresh table exists in DynamoDB Local for the
// test run. Returns a client + cleanup. Tests skip if DYNAMODB_LOCAL_URL is
// unset so CI without docker still builds.
func dynamoDBLocalSetup(t *testing.T) (client *dynamodb.Client, table string, cleanup func()) {
t.Helper()
endpoint := os.Getenv("DYNAMODB_LOCAL_URL")
if endpoint == "" {
t.Skip("DYNAMODB_LOCAL_URL not set; skipping DynamoDB integration test (run `make dynamodb-local` to start the local container)")
}
// DynamoDB Local accepts any non-empty creds + region.
t.Setenv("AWS_ACCESS_KEY_ID", "test")
t.Setenv("AWS_SECRET_ACCESS_KEY", "test")
t.Setenv("AWS_REGION", "ap-southeast-1")
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
c, err := NewDynamoDBClient(ctx, endpoint)
if err != nil {
t.Fatalf("NewDynamoDBClient: %v", err)
}
table = "test-" + t.Name()
if len(table) > 64 {
table = table[:64]
}
_, err = c.CreateTable(ctx, &dynamodb.CreateTableInput{
TableName: aws.String(table),
BillingMode: types.BillingModePayPerRequest,
AttributeDefinitions: []types.AttributeDefinition{
{AttributeName: aws.String("pk"), AttributeType: types.ScalarAttributeTypeS},
{AttributeName: aws.String("sk"), AttributeType: types.ScalarAttributeTypeS},
},
KeySchema: []types.KeySchemaElement{
{AttributeName: aws.String("pk"), KeyType: types.KeyTypeHash},
{AttributeName: aws.String("sk"), KeyType: types.KeyTypeRange},
},
})
if err != nil {
t.Fatalf("CreateTable: %v", err)
}
cleanup = func() {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
_, _ = c.DeleteTable(ctx, &dynamodb.DeleteTableInput{TableName: aws.String(table)})
}
return c, table, cleanup
}
func TestDynamoDBKVStore_PutGetDelete(t *testing.T) {
client, table, cleanup := dynamoDBLocalSetup(t)
defer cleanup()
ctx := context.Background()
s := NewDynamoDBKVStore(client, table, "wordle")
if err := s.Put(ctx, "user:1:state", []byte("hello")); err != nil {
t.Fatalf("Put: %v", err)
}
got, err := s.Get(ctx, "user:1:state")
if err != nil {
t.Fatalf("Get: %v", err)
}
if string(got) != "hello" {
t.Errorf("Get: got %q, want %q", got, "hello")
}
if err := s.Delete(ctx, "user:1:state"); err != nil {
t.Fatalf("Delete: %v", err)
}
if _, err := s.Get(ctx, "user:1:state"); !errors.Is(err, ErrNotFound) {
t.Errorf("Get after Delete: got %v, want ErrNotFound", err)
}
}
func TestDynamoDBKVStore_GetMissing(t *testing.T) {
client, table, cleanup := dynamoDBLocalSetup(t)
defer cleanup()
s := NewDynamoDBKVStore(client, table, "wordle")
_, err := s.Get(context.Background(), "missing")
if !errors.Is(err, ErrNotFound) {
t.Errorf("got %v, want ErrNotFound", err)
}
}
func TestDynamoDBKVStore_JSONRoundTrip(t *testing.T) {
client, table, cleanup := dynamoDBLocalSetup(t)
defer cleanup()
ctx := context.Background()
s := NewDynamoDBKVStore(client, table, "loldle")
type state struct {
Score int `json:"score"`
Name string `json:"name"`
}
in := state{Score: 42, Name: "ezreal"}
if err := s.PutJSON(ctx, "u1", in); err != nil {
t.Fatalf("PutJSON: %v", err)
}
var out state
if err := s.GetJSON(ctx, "u1", &out); err != nil {
t.Fatalf("GetJSON: %v", err)
}
if out != in {
t.Errorf("got %+v, want %+v", out, in)
}
}
func TestDynamoDBKVStore_ListPrefix(t *testing.T) {
client, table, cleanup := dynamoDBLocalSetup(t)
defer cleanup()
ctx := context.Background()
s := NewDynamoDBKVStore(client, table, "wordle")
for _, k := range []string{"user:1:state", "user:2:state", "config:daily", "user:1:history"} {
if err := s.Put(ctx, k, []byte("x")); err != nil {
t.Fatalf("Put %s: %v", k, err)
}
}
got, err := s.List(ctx, "user:")
if err != nil {
t.Fatalf("List: %v", err)
}
want := map[string]bool{"user:1:state": true, "user:2:state": true, "user:1:history": true}
if len(got) != len(want) {
t.Errorf("List: got %v (len=%d), want len=%d", got, len(got), len(want))
}
for _, k := range got {
if !want[k] {
t.Errorf("List: unexpected key %q", k)
}
}
}
// DynamoDB intentionally no longer implements optimistic locking — it is
// migrate-only post-self-host (Scan + Put). No CAS/versioned test here.
func TestDynamoDBKVStore_DeleteMissingNoError(t *testing.T) {
client, table, cleanup := dynamoDBLocalSetup(t)
defer cleanup()
s := NewDynamoDBKVStore(client, table, "wordle")
if err := s.Delete(context.Background(), "never-existed"); err != nil {
t.Errorf("Delete missing key: got %v, want nil (idempotent)", err)
}
}
-30
View File
@@ -1,30 +0,0 @@
package storage
import (
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
)
// DynamoDBProvider is a KVProvider that hands out per-module stores backed by
// a single shared DynamoDB table. Module isolation is by partition (pk =
// moduleName); no key-prefix wrapping is needed at this layer.
type DynamoDBProvider struct {
client *dynamodb.Client
table string
}
// NewDynamoDBProvider returns a provider over the given client + table.
// The client is goroutine-safe and outlives every store handed out by For.
func NewDynamoDBProvider(client *dynamodb.Client, table string) *DynamoDBProvider {
return &DynamoDBProvider{client: client, table: table}
}
// For returns a DynamoDBKVStore partitioned by moduleName. moduleName is
// re-validated against the same alphabet enforced by FirestoreProvider so
// behaviour is identical between backends. An invalid name yields an
// invalidStore — every op errors at first use, surfacing the bug early.
func (p *DynamoDBProvider) For(moduleName string) KVStore {
if !collectionNameRe.MatchString(moduleName) {
return invalidStore{name: moduleName}
}
return NewDynamoDBKVStore(p.client, p.table, moduleName)
}
@@ -1,61 +0,0 @@
package storage
import (
"context"
"errors"
"testing"
)
func TestDynamoDBProvider_CrossModuleIsolation(t *testing.T) {
client, table, cleanup := dynamoDBLocalSetup(t)
defer cleanup()
ctx := context.Background()
p := NewDynamoDBProvider(client, table)
wordle := p.For("wordle")
loldle := p.For("loldle")
if err := wordle.Put(ctx, "k", []byte("from-wordle")); err != nil {
t.Fatalf("wordle.Put: %v", err)
}
if err := loldle.Put(ctx, "k", []byte("from-loldle")); err != nil {
t.Fatalf("loldle.Put: %v", err)
}
got, err := wordle.Get(ctx, "k")
if err != nil {
t.Fatalf("wordle.Get: %v", err)
}
if string(got) != "from-wordle" {
t.Errorf("wordle.Get: got %q, want from-wordle", got)
}
got, err = loldle.Get(ctx, "k")
if err != nil {
t.Fatalf("loldle.Get: %v", err)
}
if string(got) != "from-loldle" {
t.Errorf("loldle.Get: got %q, want from-loldle", got)
}
// List should not leak across modules.
wkeys, err := wordle.List(ctx, "")
if err != nil {
t.Fatalf("wordle.List: %v", err)
}
if len(wkeys) != 1 || wkeys[0] != "k" {
t.Errorf("wordle.List: got %v, want [k]", wkeys)
}
}
func TestDynamoDBProvider_InvalidModuleName(t *testing.T) {
// No DynamoDB needed — invalidStore short-circuits before any SDK call.
p := NewDynamoDBProvider(nil, "any")
store := p.For("../../etc/passwd")
_, err := store.Get(context.Background(), "k")
if !errors.Is(err, ErrInvalidModuleName) {
t.Errorf("Get: got %v, want ErrInvalidModuleName", err)
}
}
-35
View File
@@ -1,35 +0,0 @@
package storage
import (
"context"
"fmt"
)
// ErrInvalidModuleName is returned by every operation on an invalidStore —
// the sentinel emitted when FirestoreProvider.For is asked for a module
// whose name fails collectionNameRe.
var ErrInvalidModuleName = fmt.Errorf("storage: invalid module name")
// invalidStore is a KVStore that errors on every call. Returned by
// FirestoreProvider.For when the requested module name doesn't validate.
// Callers see a real KVStore but every op errors at use, surfacing the
// configuration bug at the first read/write rather than silently writing
// to an attacker-controllable collection name.
type invalidStore struct {
name string
}
func (s invalidStore) wrap(op string) error {
return fmt.Errorf("%w: %q (op=%s)", ErrInvalidModuleName, s.name, op)
}
func (s invalidStore) Get(_ context.Context, _ string) ([]byte, error) {
return nil, s.wrap("Get")
}
func (s invalidStore) GetJSON(_ context.Context, _ string, _ any) error { return s.wrap("GetJSON") }
func (s invalidStore) Put(_ context.Context, _ string, _ []byte) error { return s.wrap("Put") }
func (s invalidStore) PutJSON(_ context.Context, _ string, _ any) error { return s.wrap("PutJSON") }
func (s invalidStore) Delete(_ context.Context, _ string) error { return s.wrap("Delete") }
func (s invalidStore) List(_ context.Context, _ string) ([]string, error) {
return nil, s.wrap("List")
}
-35
View File
@@ -1,35 +0,0 @@
package storage
// KVProvider yields a per-module KVStore. Implementations decide how isolation
// is achieved: MemoryProvider wraps a single in-process store with a key
// prefix, FirestoreProvider uses one collection per module.
//
// Modules never construct KVStores directly — they receive one through their
// factory's Deps. This keeps the storage backend swappable without touching
// any module code.
type KVProvider interface {
For(moduleName string) KVStore
}
// MemoryProvider is a KVProvider backed by a single in-process MemoryKVStore.
// Each module sees a Prefixed view that prevents cross-module key collisions.
//
// Intended for tests, local smoke runs, and the no-Firestore fallback. State
// is lost when the process exits.
type MemoryProvider struct {
base *MemoryKVStore
}
// NewMemoryProvider returns a fresh in-process provider.
func NewMemoryProvider() *MemoryProvider {
return &MemoryProvider{base: NewMemoryKVStore()}
}
// For returns a per-module view of the shared in-memory store.
func (m *MemoryProvider) For(moduleName string) KVStore {
return Prefixed(m.base, moduleName)
}
// Base exposes the underlying unprefixed store. Tests use this to assert
// cross-module isolation by reading raw keys; production code must not.
func (m *MemoryProvider) Base() *MemoryKVStore { return m.base }
-41
View File
@@ -1,41 +0,0 @@
package storage
import (
"context"
"errors"
)
// ErrNotFound is returned by KVStore implementations when a key has no value.
var ErrNotFound = errors.New("storage: key not found")
// ErrConflict is returned by conditional writes when the stored value changed.
var ErrConflict = errors.New("storage: write conflict")
// KVStore is the per-module key-value contract. Implementations must be safe
// for concurrent use and must return ErrNotFound for missing keys.
type KVStore interface {
Get(ctx context.Context, key string) ([]byte, error)
GetJSON(ctx context.Context, key string, dst any) error
Put(ctx context.Context, key string, val []byte) error
PutJSON(ctx context.Context, key string, val any) error
Delete(ctx context.Context, key string) error
List(ctx context.Context, prefix string) ([]string, error)
}
// VersionedStore is implemented by stores that support version-based optimistic
// locking. It replaces value-bytes compare-and-swap: each key carries a
// monotonic version, so a writer swaps only if the version it read is unchanged.
// This decouples concurrency control from the value encoding (so values can be
// stored as native documents rather than exact byte blobs).
//
// Contract:
// - GetVersioned returns the value and its current version, or ErrNotFound.
// A key that exists without a recorded version (written by an older build)
// reports version 0 and no error.
// - PutVersioned writes val only if the stored version still equals
// expectedVersion, then bumps the version. expectedVersion == 0 means
// "create, or adopt a not-yet-versioned key". A mismatch returns ErrConflict.
type VersionedStore interface {
GetVersioned(ctx context.Context, key string) (val []byte, version int64, err error)
PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error
}
+131
View File
@@ -0,0 +1,131 @@
package storage
import (
"context"
"errors"
"testing"
)
type testPayload struct {
Name string `json:"name" bson:"name"`
Count int `json:"count" bson:"count"`
}
func memStore(module string) DocStore[testPayload] {
return Typed[testPayload](NewMemoryProvider().Collection(module))
}
func TestMemoryDocStore_PutGetRoundTrip(t *testing.T) {
ctx := context.Background()
s := memStore("coin")
if err := s.Put(ctx, "k", testPayload{Name: "a", Count: 3}); err != nil {
t.Fatalf("Put: %v", err)
}
got, version, err := s.Get(ctx, "k")
if err != nil {
t.Fatalf("Get: %v", err)
}
if got.Name != "a" || got.Count != 3 {
t.Fatalf("got %+v", got)
}
if version != 1 {
t.Fatalf("version = %d, want 1", version)
}
}
func TestMemoryDocStore_GetMissing(t *testing.T) {
if _, _, err := memStore("coin").Get(context.Background(), "nope"); !errors.Is(err, ErrNotFound) {
t.Fatalf("Get missing = %v, want ErrNotFound", err)
}
}
func TestMemoryDocStore_PutBumpsVersion(t *testing.T) {
ctx := context.Background()
s := memStore("coin")
_ = s.Put(ctx, "k", testPayload{Count: 1})
_ = s.Put(ctx, "k", testPayload{Count: 2})
_, version, _ := s.Get(ctx, "k")
if version != 2 {
t.Fatalf("version = %d, want 2", version)
}
}
func TestMemoryDocStore_PutVersionedCAS(t *testing.T) {
ctx := context.Background()
s := memStore("coin")
// create
if err := s.PutVersioned(ctx, "k", 0, testPayload{Count: 1}); err != nil {
t.Fatalf("create: %v", err)
}
// second create must conflict
if err := s.PutVersioned(ctx, "k", 0, testPayload{Count: 9}); !errors.Is(err, ErrConflict) {
t.Fatalf("double create = %v, want ErrConflict", err)
}
// stale expected version conflicts
if err := s.PutVersioned(ctx, "k", 99, testPayload{Count: 9}); !errors.Is(err, ErrConflict) {
t.Fatalf("stale CAS = %v, want ErrConflict", err)
}
// fresh version succeeds
if err := s.PutVersioned(ctx, "k", 1, testPayload{Count: 2}); err != nil {
t.Fatalf("fresh CAS: %v", err)
}
}
func TestMemoryDocStore_DeleteIdempotent(t *testing.T) {
ctx := context.Background()
s := memStore("coin")
_ = s.Put(ctx, "k", testPayload{})
if err := s.Delete(ctx, "k"); err != nil {
t.Fatalf("Delete: %v", err)
}
if err := s.Delete(ctx, "k"); err != nil {
t.Fatalf("Delete missing must be nil, got %v", err)
}
}
func TestMemoryDocStore_ListPrefix(t *testing.T) {
ctx := context.Background()
s := memStore("coin")
_ = s.Put(ctx, "game:1", testPayload{})
_ = s.Put(ctx, "game:2", testPayload{})
_ = s.Put(ctx, "stats:1", testPayload{})
keys, err := s.List(ctx, "game:")
if err != nil {
t.Fatalf("List: %v", err)
}
if len(keys) != 2 || keys[0] != "game:1" || keys[1] != "game:2" {
t.Fatalf("List game: = %v", keys)
}
}
func TestMemoryDocStore_ValueIsolation(t *testing.T) {
ctx := context.Background()
s := Typed[testPayloadWithSlice](NewMemoryProvider().Collection("coin"))
in := testPayloadWithSlice{Items: []string{"x"}}
_ = s.Put(ctx, "k", in)
in.Items[0] = "mutated" // mutating the caller copy must not affect stored state
got, _, _ := s.Get(ctx, "k")
if got.Items[0] != "x" {
t.Fatalf("stored value aliased caller slice: %v", got.Items)
}
}
type testPayloadWithSlice struct {
Items []string `json:"items" bson:"items"`
}
func TestCheckReservedFields(t *testing.T) {
type bad struct {
Version int `bson:"version"`
}
if err := checkReservedFields[bad](); err == nil {
t.Fatal("expected reserved-field collision error for bson:\"version\"")
}
if err := checkReservedFields[testPayload](); err != nil {
t.Fatalf("clean struct flagged: %v", err)
}
// Non-struct payloads are skipped, not rejected.
if err := checkReservedFields[string](); err != nil {
t.Fatalf("string payload flagged: %v", err)
}
}
-119
View File
@@ -1,119 +0,0 @@
package storage
import (
"bytes"
"context"
"encoding/json"
"sort"
"strings"
"sync"
)
// memEntry is one stored value plus its optimistic-locking version.
type memEntry struct {
val []byte
version int64
}
// MemoryKVStore is an in-process KVStore for tests and local smoke runs.
// Data is lost on restart; production uses the MongoDB provider.
type MemoryKVStore struct {
mu sync.RWMutex
m map[string]memEntry
}
// NewMemoryKVStore returns an empty in-memory store.
func NewMemoryKVStore() *MemoryKVStore {
return &MemoryKVStore{m: make(map[string]memEntry)}
}
func (s *MemoryKVStore) Get(_ context.Context, key string) ([]byte, error) {
s.mu.RLock()
defer s.mu.RUnlock()
e, ok := s.m[key]
if !ok {
return nil, ErrNotFound
}
out := make([]byte, len(e.val))
copy(out, e.val)
return out, nil
}
func (s *MemoryKVStore) GetJSON(ctx context.Context, key string, dst any) error {
raw, err := s.Get(ctx, key)
if err != nil {
return err
}
return json.NewDecoder(bytes.NewReader(raw)).Decode(dst)
}
func (s *MemoryKVStore) Put(_ context.Context, key string, val []byte) error {
s.mu.Lock()
defer s.mu.Unlock()
stored := make([]byte, len(val))
copy(stored, val)
// A plain Put bumps the version too, so a concurrent versioned writer that
// read the old version correctly sees a conflict.
s.m[key] = memEntry{val: stored, version: s.m[key].version + 1}
return nil
}
// GetVersioned returns the value and its version, or ErrNotFound.
func (s *MemoryKVStore) GetVersioned(_ context.Context, key string) ([]byte, int64, error) {
s.mu.RLock()
defer s.mu.RUnlock()
e, ok := s.m[key]
if !ok {
return nil, 0, ErrNotFound
}
out := make([]byte, len(e.val))
copy(out, e.val)
return out, e.version, nil
}
// PutVersioned writes val only if the stored version equals expectedVersion
// (0 = must not exist yet), then bumps the version. ErrConflict on mismatch.
func (s *MemoryKVStore) PutVersioned(_ context.Context, key string, expectedVersion int64, val []byte) error {
s.mu.Lock()
defer s.mu.Unlock()
e, ok := s.m[key]
if expectedVersion == 0 {
if ok {
return ErrConflict
}
} else if !ok || e.version != expectedVersion {
return ErrConflict
}
stored := make([]byte, len(val))
copy(stored, val)
s.m[key] = memEntry{val: stored, version: e.version + 1}
return nil
}
func (s *MemoryKVStore) PutJSON(ctx context.Context, key string, val any) error {
raw, err := json.Marshal(val)
if err != nil {
return err
}
return s.Put(ctx, key, raw)
}
func (s *MemoryKVStore) Delete(_ context.Context, key string) error {
s.mu.Lock()
defer s.mu.Unlock()
delete(s.m, key)
return nil
}
func (s *MemoryKVStore) List(_ context.Context, prefix string) ([]string, error) {
s.mu.RLock()
defer s.mu.RUnlock()
keys := make([]string, 0)
for k := range s.m {
if strings.HasPrefix(k, prefix) {
keys = append(keys, k)
}
}
sort.Strings(keys)
return keys, nil
}
-58
View File
@@ -1,58 +0,0 @@
package storage
import (
"context"
"errors"
"testing"
)
func TestMemoryKVStore_Versioned(t *testing.T) {
ctx := context.Background()
s := NewMemoryKVStore()
// Absent key reports ErrNotFound + version 0.
if _, v, err := s.GetVersioned(ctx, "k"); !errors.Is(err, ErrNotFound) || v != 0 {
t.Fatalf("GetVersioned absent: got (v=%d, %v), want (0, ErrNotFound)", v, err)
}
// Create when absent (expectedVersion 0).
if err := s.PutVersioned(ctx, "k", 0, []byte("v1")); err != nil {
t.Fatalf("PutVersioned create: %v", err)
}
val, ver, err := s.GetVersioned(ctx, "k")
if err != nil || string(val) != "v1" || ver != 1 {
t.Fatalf("after create: got (%q, v=%d, %v), want (v1, 1, nil)", val, ver, err)
}
// Create over existing conflicts.
if err := s.PutVersioned(ctx, "k", 0, []byte("v2")); !errors.Is(err, ErrConflict) {
t.Errorf("create over existing: got %v, want ErrConflict", err)
}
// Swap with matching version succeeds and bumps version.
if err := s.PutVersioned(ctx, "k", 1, []byte("v2")); err != nil {
t.Fatalf("swap matching version: %v", err)
}
if _, ver, _ := s.GetVersioned(ctx, "k"); ver != 2 {
t.Errorf("version after swap = %d, want 2", ver)
}
// Stale version conflicts.
if err := s.PutVersioned(ctx, "k", 1, []byte("v3")); !errors.Is(err, ErrConflict) {
t.Errorf("stale version: got %v, want ErrConflict", err)
}
// Swap on a missing key conflicts.
if err := s.PutVersioned(ctx, "missing", 1, []byte("x")); !errors.Is(err, ErrConflict) {
t.Errorf("swap missing key: got %v, want ErrConflict", err)
}
// A plain Put bumps the version, so a versioned writer holding the old
// version sees a conflict.
if err := s.Put(ctx, "k", []byte("v4")); err != nil {
t.Fatalf("Put: %v", err)
}
if err := s.PutVersioned(ctx, "k", 2, []byte("v5")); !errors.Is(err, ErrConflict) {
t.Errorf("versioned write after plain Put: got %v, want ErrConflict", err)
}
}
+142
View File
@@ -0,0 +1,142 @@
package storage
import (
"context"
"encoding/json"
"sort"
"strings"
"sync"
)
// MemoryProvider is a Provider backed by per-module in-process maps. Intended
// for tests and local no-database runs (MODULES= / no MONGO_URL); state is lost
// when the process exits. Production uses MongoProvider.
type MemoryProvider struct {
mu sync.Mutex
cols map[string]*memoryCollection
}
// NewMemoryProvider returns a fresh in-process provider.
func NewMemoryProvider() *MemoryProvider {
return &MemoryProvider{cols: make(map[string]*memoryCollection)}
}
// Collection returns the per-module in-memory collection, creating it on first
// use. An invalid module name yields an invalidCollection so Typed produces a
// store whose every op errors — mirroring MongoProvider's defense in depth.
func (p *MemoryProvider) Collection(module string) Collection {
if !collectionNameRe.MatchString(module) {
return invalidCollection{name: module}
}
p.mu.Lock()
defer p.mu.Unlock()
c, ok := p.cols[module]
if !ok {
c = &memoryCollection{rows: make(map[string]memoryRow)}
p.cols[module] = c
}
return c
}
// memoryCollection holds one module's rows. Multiple typed views (different
// payload types under disjoint key prefixes) share the same map.
type memoryCollection struct {
mu sync.RWMutex
rows map[string]memoryRow
}
func (*memoryCollection) isCollection() {}
// memoryRow stores the JSON-encoded value plus its version. Encoding to bytes
// isolates stored state from caller mutation, matching production where Mongo
// serializes the value on write.
type memoryRow struct {
version int64
data []byte
}
// memoryDocStore is a typed DocStore over one memoryCollection.
type memoryDocStore[T any] struct {
c *memoryCollection
}
func (s *memoryDocStore[T]) Get(_ context.Context, id string) (T, int64, error) {
var out T
if err := validateKey(id); err != nil {
return out, 0, err
}
s.c.mu.RLock()
defer s.c.mu.RUnlock()
row, ok := s.c.rows[id]
if !ok {
return out, 0, ErrNotFound
}
if err := json.Unmarshal(row.data, &out); err != nil {
return out, 0, err
}
return out, row.version, nil
}
func (s *memoryDocStore[T]) Put(_ context.Context, id string, val T) error {
if err := validateKey(id); err != nil {
return err
}
data, err := json.Marshal(val)
if err != nil {
return err
}
s.c.mu.Lock()
defer s.c.mu.Unlock()
// A plain Put bumps the version too, so a concurrent versioned writer that
// read the old version correctly sees a conflict.
s.c.rows[id] = memoryRow{version: s.c.rows[id].version + 1, data: data}
return nil
}
func (s *memoryDocStore[T]) PutVersioned(_ context.Context, id string, expectedVersion int64, val T) error {
if err := validateKey(id); err != nil {
return err
}
data, err := json.Marshal(val)
if err != nil {
return err
}
s.c.mu.Lock()
defer s.c.mu.Unlock()
row, ok := s.c.rows[id]
if expectedVersion == 0 {
if ok {
return ErrConflict
}
} else if !ok || row.version != expectedVersion {
return ErrConflict
}
s.c.rows[id] = memoryRow{version: row.version + 1, data: data}
return nil
}
func (s *memoryDocStore[T]) Delete(_ context.Context, id string) error {
if err := validateKey(id); err != nil {
return err
}
s.c.mu.Lock()
defer s.c.mu.Unlock()
delete(s.c.rows, id)
return nil
}
func (s *memoryDocStore[T]) List(_ context.Context, prefix string) ([]string, error) {
if err := validatePrefix(prefix); err != nil {
return nil, err
}
s.c.mu.RLock()
defer s.c.mu.RUnlock()
keys := make([]string, 0)
for k := range s.c.rows {
if strings.HasPrefix(k, prefix) {
keys = append(keys, k)
}
}
sort.Strings(keys)
return keys, nil
}
+185
View File
@@ -0,0 +1,185 @@
package storage
import (
"context"
"errors"
"fmt"
"time"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
// Reserved document-root field names owned by the store. The caller's payload
// fields are hoisted to the root alongside these. `_id` holds the key.
const (
mongoIDField = "_id"
mongoVersionField = "version"
mongoUpdatedAtField = "updatedAt"
)
// mongoPutAttempts bounds the get→put-versioned retry loop a plain Put uses to
// overwrite unconditionally while still bumping the version.
const mongoPutAttempts = 5
// mongoCollection is a Collection backed by a single MongoDB collection (one per
// module — collection-per-module IS the isolation, so no key prefix is needed).
type mongoCollection struct {
coll *mongo.Collection
module string
}
func (mongoCollection) isCollection() {}
// storedDoc is the on-disk document shape. The payload's fields are hoisted to
// the document root via bson inline, so the value renders as an expandable,
// queryable native document in Compass — never wrapped in a `value` envelope.
// Reserved metadata (version, updatedAt) sit alongside the payload fields.
type storedDoc[T any] struct {
ID string `bson:"_id"`
Version int64 `bson:"version"`
UpdatedAt time.Time `bson:"updatedAt"`
Payload T `bson:",inline"`
}
// mongoDocStore is a typed DocStore over one collection.
type mongoDocStore[T any] struct {
coll *mongo.Collection
module string
}
func (s *mongoDocStore[T]) Get(ctx context.Context, id string) (T, int64, error) {
var out storedDoc[T]
if err := validateKey(id); err != nil {
return out.Payload, 0, err
}
err := s.coll.FindOne(ctx, bson.M{mongoIDField: id}).Decode(&out)
if err != nil {
if errors.Is(err, mongo.ErrNoDocuments) {
return out.Payload, 0, ErrNotFound
}
return out.Payload, 0, fmt.Errorf("mongo get %s/%s: %w", s.module, id, err)
}
return out.Payload, out.Version, nil
}
// replacement builds the whole document to store. Using a full replacement
// (never $set) guarantees stale fields from a previous value cannot survive an
// overwrite.
func (s *mongoDocStore[T]) replacement(id string, version int64, val T) storedDoc[T] {
return storedDoc[T]{ID: id, Version: version, UpdatedAt: time.Now().UTC(), Payload: val}
}
// PutVersioned writes val only if the stored version equals expectedVersion,
// then bumps it.
//
// - expectedVersion == 0 → upsert matching {_id} with version absent or 0; a
// key that already has version ≥ 1 fails the filter, the upsert attempts an
// insert on the same _id, and the unique-_id index returns duplicate-key →
// ErrConflict (linearizable single-winner for the first write).
// - expectedVersion > 0 → ReplaceOne on {_id, version:expected}; MatchedCount
// of 0 means the version moved → ErrConflict.
func (s *mongoDocStore[T]) PutVersioned(ctx context.Context, id string, expectedVersion int64, val T) error {
if err := validateKey(id); err != nil {
return err
}
if expectedVersion == 0 {
_, err := s.coll.ReplaceOne(ctx,
bson.M{
mongoIDField: id,
"$or": bson.A{
bson.M{mongoVersionField: bson.M{"$exists": false}},
bson.M{mongoVersionField: int64(0)},
},
},
s.replacement(id, 1, val),
options.Replace().SetUpsert(true),
)
if err != nil {
if mongo.IsDuplicateKeyError(err) {
return ErrConflict
}
return fmt.Errorf("mongo put-versioned %s/%s: %w", s.module, id, err)
}
return nil
}
res, err := s.coll.ReplaceOne(ctx,
bson.M{mongoIDField: id, mongoVersionField: expectedVersion},
s.replacement(id, expectedVersion+1, val),
)
if err != nil {
return fmt.Errorf("mongo put-versioned %s/%s: %w", s.module, id, err)
}
if res.MatchedCount == 0 {
return ErrConflict
}
return nil
}
// Put overwrites unconditionally and bumps the version, implemented as a bounded
// get→put-versioned loop so it shares the whole-document replacement (stale-field
// removal) and version semantics with PutVersioned.
func (s *mongoDocStore[T]) Put(ctx context.Context, id string, val T) error {
if err := validateKey(id); err != nil {
return err
}
for attempt := 0; attempt < mongoPutAttempts; attempt++ {
_, version, err := s.Get(ctx, id)
switch {
case errors.Is(err, ErrNotFound):
version = 0
case err != nil:
return err
}
err = s.PutVersioned(ctx, id, version, val)
if errors.Is(err, ErrConflict) {
continue // a concurrent writer moved the version; re-read and retry.
}
return err
}
return fmt.Errorf("mongo put %s/%s: %w after %d attempts", s.module, id, ErrConflict, mongoPutAttempts)
}
func (s *mongoDocStore[T]) Delete(ctx context.Context, id string) error {
if err := validateKey(id); err != nil {
return err
}
if _, err := s.coll.DeleteOne(ctx, bson.M{mongoIDField: id}); err != nil {
return fmt.Errorf("mongo delete %s/%s: %w", s.module, id, err)
}
return nil
}
// List returns all document IDs whose key starts with prefix, via a half-open
// range scan on _id (reusing prefixSuccessor) so it uses the _id index and
// avoids regex injection. Empty prefix returns the whole collection.
func (s *mongoDocStore[T]) List(ctx context.Context, prefix string) ([]string, error) {
if err := validatePrefix(prefix); err != nil {
return nil, err
}
filter := bson.M{}
if prefix != "" {
filter[mongoIDField] = bson.M{"$gte": prefix, "$lt": prefixSuccessor(prefix)}
}
cur, err := s.coll.Find(ctx, filter, options.Find().SetProjection(bson.M{mongoIDField: 1}))
if err != nil {
return nil, fmt.Errorf("mongo list %s prefix=%q: %w", s.module, prefix, err)
}
defer func() { _ = cur.Close(ctx) }()
var keys []string
for cur.Next(ctx) {
var doc struct {
ID string `bson:"_id"`
}
if err := cur.Decode(&doc); err != nil {
return nil, fmt.Errorf("mongo list %s prefix=%q: decode: %w", s.module, prefix, err)
}
keys = append(keys, doc.ID)
}
if err := cur.Err(); err != nil {
return nil, fmt.Errorf("mongo list %s prefix=%q: %w", s.module, prefix, err)
}
return keys, nil
}
+196
View File
@@ -0,0 +1,196 @@
package storage
import (
"context"
"errors"
"fmt"
"os"
"sync"
"testing"
"time"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
)
// mongoLocalSetup connects to a local MongoDB and returns a fresh, uniquely
// named database plus cleanup. Tests skip if MONGODB_TEST_URL is unset so CI
// without a Mongo container still builds (mirrors the DynamoDB Local gating).
func mongoLocalSetup(t *testing.T) (*mongo.Database, func()) {
t.Helper()
uri := os.Getenv("MONGODB_TEST_URL")
if uri == "" {
t.Skip("MONGODB_TEST_URL not set; skipping MongoDB integration test (run `make mongo-local` to start the local container)")
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := NewMongoClient(ctx, uri)
if err != nil {
t.Fatalf("NewMongoClient: %v", err)
}
dbName := fmt.Sprintf("miti99bot_test_%d", time.Now().UnixNano())
if len(dbName) > 63 {
dbName = dbName[:63]
}
db := client.Database(dbName)
cleanup := func() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_ = db.Drop(ctx)
_ = client.Disconnect(ctx)
}
return db, cleanup
}
type portfolioLike struct {
USD float64 `json:"usd" bson:"usd"`
Assets map[string]float64 `json:"assets" bson:"assets"`
}
func mongoStore[T any](t *testing.T, module string) (DocStore[T], *mongo.Collection, func()) {
t.Helper()
db, cleanup := mongoLocalSetup(t)
coll := db.Collection(module)
return Typed[T](NewMongoProvider(db).Collection(module)), coll, cleanup
}
// rawDoc fetches the on-disk BSON document for assertions about its shape.
func rawDoc(t *testing.T, coll *mongo.Collection, id string) bson.M {
t.Helper()
var doc bson.M
if err := coll.FindOne(context.Background(), bson.M{"_id": id}).Decode(&doc); err != nil {
t.Fatalf("raw FindOne %s: %v", id, err)
}
return doc
}
func TestMongoDocStore_RootShape(t *testing.T) {
store, coll, cleanup := mongoStore[portfolioLike](t, "coin")
defer cleanup()
ctx := context.Background()
if err := store.Put(ctx, "user:7", portfolioLike{USD: 1000.25, Assets: map[string]float64{"BTC": 1}}); err != nil {
t.Fatalf("Put: %v", err)
}
doc := rawDoc(t, coll, "user:7")
if _, ok := doc["value"]; ok {
t.Error("doc has legacy 'value' envelope field")
}
if _, ok := doc["_payload"]; ok {
t.Error("doc has '_payload' fallback field")
}
if _, ok := doc["usd"]; !ok {
t.Error("payload field 'usd' not hoisted to root")
}
if _, ok := doc["assets"]; !ok {
t.Error("payload field 'assets' not hoisted to root")
}
if _, ok := doc["updatedAt"].(bson.DateTime); !ok {
t.Errorf("updatedAt is %T, want BSON Date", doc["updatedAt"])
}
got, version, err := store.Get(ctx, "user:7")
if err != nil || version != 1 || got.USD != 1000.25 || got.Assets["BTC"] != 1 {
t.Fatalf("Get round-trip: %+v v=%d err=%v", got, version, err)
}
}
func TestMongoDocStore_OverwriteRemovesStaleFields(t *testing.T) {
store, coll, cleanup := mongoStore[map[string]int](t, "misc")
defer cleanup()
ctx := context.Background()
if err := store.Put(ctx, "k", map[string]int{"a": 1, "b": 2}); err != nil {
t.Fatalf("Put A: %v", err)
}
if err := store.Put(ctx, "k", map[string]int{"a": 3}); err != nil {
t.Fatalf("Put B: %v", err)
}
doc := rawDoc(t, coll, "k")
if _, ok := doc["b"]; ok {
t.Error("stale field 'b' survived overwrite")
}
if doc["a"] != int32(3) && doc["a"] != int64(3) {
t.Errorf("a = %v (%T), want 3", doc["a"], doc["a"])
}
}
func TestMongoDocStore_PutVersionedConcurrentCreate(t *testing.T) {
store, _, cleanup := mongoStore[portfolioLike](t, "coin")
defer cleanup()
ctx := context.Background()
const n = 12
var wg sync.WaitGroup
var mu sync.Mutex
wins := 0
wg.Add(n)
for i := 0; i < n; i++ {
go func() {
defer wg.Done()
if err := store.PutVersioned(ctx, "race", 0, portfolioLike{USD: 1}); err == nil {
mu.Lock()
wins++
mu.Unlock()
} else if !errors.Is(err, ErrConflict) {
t.Errorf("unexpected err: %v", err)
}
}()
}
wg.Wait()
if wins != 1 {
t.Fatalf("concurrent create winners = %d, want exactly 1", wins)
}
}
func TestMongoDocStore_PutVersionedStaleConflict(t *testing.T) {
store, _, cleanup := mongoStore[portfolioLike](t, "coin")
defer cleanup()
ctx := context.Background()
_ = store.PutVersioned(ctx, "k", 0, portfolioLike{USD: 1})
if err := store.PutVersioned(ctx, "k", 99, portfolioLike{USD: 2}); !errors.Is(err, ErrConflict) {
t.Fatalf("stale CAS = %v, want ErrConflict", err)
}
if err := store.PutVersioned(ctx, "k", 1, portfolioLike{USD: 2}); err != nil {
t.Fatalf("fresh CAS: %v", err)
}
}
// wrappedScalar / wrappedArray prove non-object values become named root fields
// (the lolschedule pattern) rather than an envelope or fallback.
type wrappedScalar struct {
Date string `json:"date" bson:"date"`
}
type wrappedArray struct {
Subscribers []int `json:"subscribers" bson:"subscribers"`
}
func TestMongoDocStore_WrappedScalarAndArray(t *testing.T) {
db, cleanup := mongoLocalSetup(t)
defer cleanup()
ctx := context.Background()
p := NewMongoProvider(db)
scalar := Typed[wrappedScalar](p.Collection("lolschedule"))
if err := scalar.Put(ctx, "last_push_date", wrappedScalar{Date: "2026-06-28"}); err != nil {
t.Fatalf("scalar Put: %v", err)
}
doc := rawDoc(t, db.Collection("lolschedule"), "last_push_date")
if doc["date"] != "2026-06-28" {
t.Errorf("scalar root field date = %v", doc["date"])
}
arr := Typed[wrappedArray](p.Collection("lolschedule"))
if err := arr.Put(ctx, "subscribers", wrappedArray{Subscribers: []int{1, 2, 3}}); err != nil {
t.Fatalf("array Put: %v", err)
}
doc = rawDoc(t, db.Collection("lolschedule"), "subscribers")
if _, ok := doc["subscribers"].(bson.A); !ok {
t.Errorf("array root field subscribers = %T, want bson.A", doc["subscribers"])
}
}
-266
View File
@@ -1,266 +0,0 @@
package storage
import (
"context"
"encoding/json"
"errors"
"fmt"
"time"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
"go.mongodb.org/mongo-driver/v2/mongo/options"
)
// MongoDB document fields. `_id` holds the user key; `value` holds the raw
// bytes; `updatedAt` is unix-nanos for observability + future TTL.
const (
mongoIDField = "_id"
mongoValueField = "value"
mongoUpdatedAtField = "updatedAt"
mongoVersionField = "version"
)
// MongoKVStore is a KVStore backed by a single MongoDB collection. The caller
// (MongoProvider) creates one per module so cross-module isolation is
// "different collection" — no key prefix needed at this layer, mirroring
// FirestoreKVStore.
type MongoKVStore struct {
coll *mongo.Collection
moduleName string
}
// NewMongoKVStore returns a store writing to the given collection. Callers must
// validate the collection/module name beforehand (MongoProvider does).
func NewMongoKVStore(coll *mongo.Collection, moduleName string) *MongoKVStore {
return &MongoKVStore{coll: coll, moduleName: moduleName}
}
// decodeValue extracts the stored value bytes from a decoded document,
// reconstructing the caller's JSON []byte from whichever representation is on
// disk:
// - native object/array (bson.M / bson.A / bson.D) → re-serialized to JSON;
// - string → the bytes verbatim (bare scalars / non-JSON values, e.g. a date);
// - bson.Binary / []byte → legacy fallback for docs written by an earlier
// build that stored value as a binary blob.
func (s *MongoKVStore) decodeValue(key string, doc bson.M) ([]byte, error) {
raw, ok := doc[mongoValueField]
if !ok {
return nil, fmt.Errorf("mongo get %s/%s: missing %q field", s.moduleName, key, mongoValueField)
}
switch v := raw.(type) {
case string:
return []byte(v), nil
case bson.M, bson.A, bson.D:
return nativeToJSON(v)
case bson.Binary:
return v.Data, nil
case []byte:
return v, nil
default:
return nil, fmt.Errorf("mongo get %s/%s: unexpected value type %T", s.moduleName, key, raw)
}
}
// Get returns the raw bytes stored at key, or ErrNotFound.
func (s *MongoKVStore) Get(ctx context.Context, key string) ([]byte, error) {
if err := validateKey(key); err != nil {
return nil, err
}
var doc bson.M
err := s.coll.FindOne(ctx, bson.M{mongoIDField: key}).Decode(&doc)
if err != nil {
if errors.Is(err, mongo.ErrNoDocuments) {
return nil, ErrNotFound
}
return nil, fmt.Errorf("mongo get %s/%s: %w", s.moduleName, key, err)
}
return s.decodeValue(key, doc)
}
// GetJSON decodes the value at key into dst.
func (s *MongoKVStore) GetJSON(ctx context.Context, key string, dst any) error {
raw, err := s.Get(ctx, key)
if err != nil {
return err
}
if err := json.Unmarshal(raw, dst); err != nil {
return fmt.Errorf("mongo get %s/%s: json decode: %w", s.moduleName, key, err)
}
return nil
}
// Put writes raw bytes at key, creating or overwriting, and bumps the version
// (so a concurrent versioned writer that read the old version sees a conflict).
func (s *MongoKVStore) Put(ctx context.Context, key string, val []byte) error {
if err := validateKey(key); err != nil {
return err
}
_, err := s.coll.UpdateOne(ctx,
bson.M{mongoIDField: key},
bson.M{
"$set": bson.M{
mongoValueField: encodeValue(val),
mongoUpdatedAtField: time.Now().UTC().UnixNano(),
},
"$inc": bson.M{mongoVersionField: 1},
},
options.UpdateOne().SetUpsert(true),
)
if err != nil {
return fmt.Errorf("mongo put %s/%s: %w", s.moduleName, key, err)
}
return nil
}
// PutJSON marshals val and writes the bytes at key.
func (s *MongoKVStore) PutJSON(ctx context.Context, key string, val any) error {
raw, err := json.Marshal(val)
if err != nil {
return fmt.Errorf("mongo put %s/%s: json encode: %w", s.moduleName, key, err)
}
return s.Put(ctx, key, raw)
}
// GetVersioned returns the value and its version, or ErrNotFound. A document
// written by an older build that has no version field reports version 0 (and no
// error), so PutVersioned(expectedVersion=0) can adopt it.
func (s *MongoKVStore) GetVersioned(ctx context.Context, key string) ([]byte, int64, error) {
if err := validateKey(key); err != nil {
return nil, 0, err
}
var doc bson.M
err := s.coll.FindOne(ctx, bson.M{mongoIDField: key}).Decode(&doc)
if err != nil {
if errors.Is(err, mongo.ErrNoDocuments) {
return nil, 0, ErrNotFound
}
return nil, 0, fmt.Errorf("mongo get %s/%s: %w", s.moduleName, key, err)
}
val, err := s.decodeValue(key, doc)
if err != nil {
return nil, 0, err
}
return val, decodeVersion(doc), nil
}
// PutVersioned writes val only if the stored version equals expectedVersion,
// then bumps it. expectedVersion == 0 means "create, or adopt a key that has no
// version field yet" (a legacy doc). ErrConflict on a version mismatch.
//
// - expectedVersion == 0 → upsert matching {_id} with version absent or 0; a
// key that already has version ≥ 1 fails the filter, the upsert attempts an
// insert on the same _id, and the unique-_id index returns duplicate-key →
// ErrConflict (linearizable single-winner for the first write).
// - expectedVersion > 0 → UpdateOne on {_id, version:expected}; MatchedCount
// of 0 means the version moved → ErrConflict.
func (s *MongoKVStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error {
if err := validateKey(key); err != nil {
return err
}
now := time.Now().UTC().UnixNano()
if expectedVersion == 0 {
_, err := s.coll.UpdateOne(ctx,
bson.M{
mongoIDField: key,
"$or": bson.A{
bson.M{mongoVersionField: bson.M{"$exists": false}},
bson.M{mongoVersionField: int64(0)},
},
},
bson.M{"$set": bson.M{
mongoValueField: encodeValue(val),
mongoUpdatedAtField: now,
mongoVersionField: int64(1),
}},
options.UpdateOne().SetUpsert(true),
)
if err != nil {
if mongo.IsDuplicateKeyError(err) {
return ErrConflict
}
return fmt.Errorf("mongo put-versioned %s/%s: %w", s.moduleName, key, err)
}
return nil
}
res, err := s.coll.UpdateOne(ctx,
bson.M{mongoIDField: key, mongoVersionField: expectedVersion},
bson.M{
"$set": bson.M{mongoValueField: encodeValue(val), mongoUpdatedAtField: now},
"$inc": bson.M{mongoVersionField: 1},
},
)
if err != nil {
return fmt.Errorf("mongo put-versioned %s/%s: %w", s.moduleName, key, err)
}
if res.MatchedCount == 0 {
return ErrConflict
}
return nil
}
// decodeVersion reads the version field from a decoded doc, defaulting to 0 for
// legacy docs that predate the version field. The driver may decode a BSON int
// as int32 or int64.
func decodeVersion(doc bson.M) int64 {
switch v := doc[mongoVersionField].(type) {
case int64:
return v
case int32:
return int64(v)
case float64:
return int64(v)
default:
return 0
}
}
// Delete removes the document at key. Deleting a missing key is not an error
// (idempotent) — DeleteOne with a zero match count returns nil.
func (s *MongoKVStore) Delete(ctx context.Context, key string) error {
if err := validateKey(key); err != nil {
return err
}
_, err := s.coll.DeleteOne(ctx, bson.M{mongoIDField: key})
if err != nil {
return fmt.Errorf("mongo delete %s/%s: %w", s.moduleName, key, err)
}
return nil
}
// List returns all document IDs in the collection that start with prefix.
// Implemented as a half-open range scan on _id (reusing prefixSuccessor) so it
// uses the _id index and avoids regex injection. Empty prefix returns the whole
// collection.
func (s *MongoKVStore) List(ctx context.Context, prefix string) ([]string, error) {
if err := validatePrefix(prefix); err != nil {
return nil, err
}
filter := bson.M{}
if prefix != "" {
filter[mongoIDField] = bson.M{
"$gte": prefix,
"$lt": prefixSuccessor(prefix),
}
}
cur, err := s.coll.Find(ctx, filter, options.Find().SetProjection(bson.M{mongoIDField: 1}))
if err != nil {
return nil, fmt.Errorf("mongo list %s prefix=%q: %w", s.moduleName, prefix, err)
}
defer func() { _ = cur.Close(ctx) }()
var keys []string
for cur.Next(ctx) {
var doc struct {
ID string `bson:"_id"`
}
if err := cur.Decode(&doc); err != nil {
return nil, fmt.Errorf("mongo list %s prefix=%q: decode: %w", s.moduleName, prefix, err)
}
keys = append(keys, doc.ID)
}
if err := cur.Err(); err != nil {
return nil, fmt.Errorf("mongo list %s prefix=%q: %w", s.moduleName, prefix, err)
}
return keys, nil
}
-370
View File
@@ -1,370 +0,0 @@
package storage
import (
"context"
"errors"
"fmt"
"os"
"reflect"
"sync"
"testing"
"time"
"go.mongodb.org/mongo-driver/v2/bson"
"go.mongodb.org/mongo-driver/v2/mongo"
)
// mongoLocalSetup connects to a local MongoDB and returns a store over a fresh,
// uniquely-named collection. Tests skip if MONGODB_TEST_URL is unset so CI
// without a Mongo container still builds (mirrors the DynamoDB Local gating).
func mongoLocalSetup(t *testing.T, module string) (*MongoKVStore, *mongo.Database, func()) {
t.Helper()
uri := os.Getenv("MONGODB_TEST_URL")
if uri == "" {
t.Skip("MONGODB_TEST_URL not set; skipping MongoDB integration test (run `make mongo-local` to start the local container)")
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := NewMongoClient(ctx, uri)
if err != nil {
t.Fatalf("NewMongoClient: %v", err)
}
// Unique DB per test so parallel runs and cross-module isolation checks
// never collide. Mongo db names are capped at 63 bytes.
dbName := fmt.Sprintf("miti99bot_test_%d", time.Now().UnixNano())
if len(dbName) > 63 {
dbName = dbName[:63]
}
db := client.Database(dbName)
cleanup := func() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_ = db.Drop(ctx)
_ = client.Disconnect(ctx)
}
return NewMongoKVStore(db.Collection(module), module), db, cleanup
}
func TestMongoKVStore_PutGetDelete(t *testing.T) {
s, _, cleanup := mongoLocalSetup(t, "wordle")
defer cleanup()
ctx := context.Background()
if err := s.Put(ctx, "user:1:state", []byte("hello")); err != nil {
t.Fatalf("Put: %v", err)
}
got, err := s.Get(ctx, "user:1:state")
if err != nil {
t.Fatalf("Get: %v", err)
}
if string(got) != "hello" {
t.Errorf("Get: got %q, want %q", got, "hello")
}
// Overwrite preserves byte fidelity.
if err := s.Put(ctx, "user:1:state", []byte("world")); err != nil {
t.Fatalf("Put overwrite: %v", err)
}
got, _ = s.Get(ctx, "user:1:state")
if string(got) != "world" {
t.Errorf("Get after overwrite: got %q, want %q", got, "world")
}
if err := s.Delete(ctx, "user:1:state"); err != nil {
t.Fatalf("Delete: %v", err)
}
if _, err := s.Get(ctx, "user:1:state"); !errors.Is(err, ErrNotFound) {
t.Errorf("Get after Delete: got %v, want ErrNotFound", err)
}
}
func TestMongoKVStore_GetMissing(t *testing.T) {
s, _, cleanup := mongoLocalSetup(t, "wordle")
defer cleanup()
if _, err := s.Get(context.Background(), "missing"); !errors.Is(err, ErrNotFound) {
t.Errorf("got %v, want ErrNotFound", err)
}
}
// TestMongoKVStore_NativeValueRepresentation verifies a JSON object value is
// persisted as a NATIVE BSON document (expandable/queryable in Atlas), and a
// bare non-JSON value as a string — and that both round-trip.
func TestMongoKVStore_NativeValueRepresentation(t *testing.T) {
s, db, cleanup := mongoLocalSetup(t, "wordle")
defer cleanup()
ctx := context.Background()
obj := []byte(`{"word":"chào","score":42}`) // multi-byte UTF-8 object
if err := s.Put(ctx, "obj", obj); err != nil {
t.Fatalf("Put obj: %v", err)
}
// A bare string (lolschedule date-guard style) is not JSON → stored as string.
if err := s.Put(ctx, "bare", []byte("2026-06-28")); err != nil {
t.Fatalf("Put bare: %v", err)
}
var objDoc, bareDoc bson.M
if err := db.Collection("wordle").FindOne(ctx, bson.M{"_id": "obj"}).Decode(&objDoc); err != nil {
t.Fatalf("raw FindOne obj: %v", err)
}
switch objDoc["value"].(type) {
case bson.M, bson.D:
// native object — good
default:
t.Errorf("object value stored as %T, want native bson document", objDoc["value"])
}
if err := db.Collection("wordle").FindOne(ctx, bson.M{"_id": "bare"}).Decode(&bareDoc); err != nil {
t.Fatalf("raw FindOne bare: %v", err)
}
if _, ok := bareDoc["value"].(string); !ok {
t.Errorf("bare value stored as %T, want string", bareDoc["value"])
}
// Round-trips.
if got, _ := s.Get(ctx, "bare"); string(got) != "2026-06-28" {
t.Errorf("bare round trip: got %q", got)
}
var back map[string]any
if err := s.GetJSON(ctx, "obj", &back); err != nil {
t.Fatalf("GetJSON obj: %v", err)
}
if back["word"] != "chào" {
t.Errorf("object round trip lost data: %v", back)
}
}
// TestMongoKVStore_Int64Fidelity is the codec gate: an int64 field must survive
// the JSON↔BSON round trip as an integer, not collapse to a float (which would
// make GetJSON into an int64 field fail or drift).
func TestMongoKVStore_Int64Fidelity(t *testing.T) {
s, db, cleanup := mongoLocalSetup(t, "coin")
defer cleanup()
ctx := context.Background()
type meta struct {
CreatedAt int64 `json:"createdAt"` // UnixMilli ~1.7e12
Invested float64 `json:"invested"`
}
type portfolio struct {
USD float64 `json:"usd"`
Meta meta `json:"meta"`
Qty map[string]int64 `json:"qty"`
}
in := portfolio{USD: 1234.56, Meta: meta{CreatedAt: 1719500000000, Invested: 5000}, Qty: map[string]int64{"BTC": 3}}
if err := s.PutJSON(ctx, "u1", in); err != nil {
t.Fatalf("PutJSON: %v", err)
}
var out portfolio
if err := s.GetJSON(ctx, "u1", &out); err != nil {
t.Fatalf("GetJSON: %v", err)
}
if !reflect.DeepEqual(out, in) {
t.Errorf("fidelity lost: got %+v, want %+v", out, in)
}
// The stored CreatedAt must be a BSON int (int32/int64), not a double.
var doc bson.M
_ = db.Collection("coin").FindOne(ctx, bson.M{"_id": "u1"}).Decode(&doc)
if m, ok := doc["value"].(bson.M); ok {
if meta, ok := m["meta"].(bson.M); ok {
switch meta["createdAt"].(type) {
case int64, int32:
// good — preserved as integer
default:
t.Errorf("createdAt stored as %T, want int64 (json.Number codec)", meta["createdAt"])
}
}
}
}
func TestMongoKVStore_JSONRoundTrip(t *testing.T) {
s, _, cleanup := mongoLocalSetup(t, "loldle")
defer cleanup()
ctx := context.Background()
type state struct {
Score int `json:"score"`
Name string `json:"name"`
}
in := state{Score: 42, Name: "ezreal"}
if err := s.PutJSON(ctx, "u1", in); err != nil {
t.Fatalf("PutJSON: %v", err)
}
var out state
if err := s.GetJSON(ctx, "u1", &out); err != nil {
t.Fatalf("GetJSON: %v", err)
}
if out != in {
t.Errorf("got %+v, want %+v", out, in)
}
}
func TestMongoKVStore_ListPrefix(t *testing.T) {
s, _, cleanup := mongoLocalSetup(t, "wordle")
defer cleanup()
ctx := context.Background()
for _, k := range []string{"user:1:state", "user:2:state", "config:daily", "user:1:history"} {
if err := s.Put(ctx, k, []byte("x")); err != nil {
t.Fatalf("Put %s: %v", k, err)
}
}
got, err := s.List(ctx, "user:")
if err != nil {
t.Fatalf("List: %v", err)
}
want := map[string]bool{"user:1:state": true, "user:2:state": true, "user:1:history": true}
if len(got) != len(want) {
t.Errorf("List: got %v (len=%d), want len=%d", got, len(got), len(want))
}
for _, k := range got {
if !want[k] {
t.Errorf("List: unexpected key %q", k)
}
}
// Empty prefix returns everything in the collection.
all, err := s.List(ctx, "")
if err != nil {
t.Fatalf("List empty: %v", err)
}
if len(all) != 4 {
t.Errorf("List(\"\"): got %d keys, want 4", len(all))
}
}
func TestMongoKVStore_Versioned(t *testing.T) {
s, _, cleanup := mongoLocalSetup(t, "gold")
defer cleanup()
ctx := context.Background()
// Create-if-absent (version 0) succeeds once, then conflicts.
if err := s.PutVersioned(ctx, "user:1", 0, []byte("v1")); err != nil {
t.Fatalf("PutVersioned create: %v", err)
}
if err := s.PutVersioned(ctx, "user:1", 0, []byte("v2")); !errors.Is(err, ErrConflict) {
t.Errorf("create over existing: got %v, want ErrConflict", err)
}
val, ver, err := s.GetVersioned(ctx, "user:1")
if err != nil || string(val) != "v1" || ver != 1 {
t.Fatalf("GetVersioned: got (%q, v=%d, %v), want (v1, 1, nil)", val, ver, err)
}
// Swap with matching version succeeds and bumps; stale version conflicts.
if err := s.PutVersioned(ctx, "user:1", 1, []byte("v2")); err != nil {
t.Fatalf("swap matching version: %v", err)
}
if err := s.PutVersioned(ctx, "user:1", 1, []byte("v3")); !errors.Is(err, ErrConflict) {
t.Errorf("stale version: got %v, want ErrConflict", err)
}
if got, _ := s.Get(ctx, "user:1"); string(got) != "v2" {
t.Errorf("stored value = %q, want %q", got, "v2")
}
// Swap on a missing key conflicts.
if err := s.PutVersioned(ctx, "user:missing", 1, []byte("v2")); !errors.Is(err, ErrConflict) {
t.Errorf("swap missing key: got %v, want ErrConflict", err)
}
}
// TestMongoKVStore_PutVersioned_AdoptsLegacyDoc proves a document written by an
// older build (no version field) is treated as version 0 and updated cleanly,
// so existing users' portfolios keep working after the rollout.
func TestMongoKVStore_PutVersioned_AdoptsLegacyDoc(t *testing.T) {
s, db, cleanup := mongoLocalSetup(t, "coin")
defer cleanup()
ctx := context.Background()
// Seed a legacy doc: value string, NO version field.
if _, err := db.Collection("coin").InsertOne(ctx, bson.M{"_id": "user:9", "value": "legacy"}); err != nil {
t.Fatalf("seed legacy doc: %v", err)
}
val, ver, err := s.GetVersioned(ctx, "user:9")
if err != nil || string(val) != "legacy" || ver != 0 {
t.Fatalf("GetVersioned legacy: got (%q, v=%d, %v), want (legacy, 0, nil)", val, ver, err)
}
// Claim it at version 0 — must succeed (no spurious conflict).
if err := s.PutVersioned(ctx, "user:9", 0, []byte("migrated")); err != nil {
t.Fatalf("adopt legacy doc: %v", err)
}
if got, ver, _ := s.GetVersioned(ctx, "user:9"); string(got) != "migrated" || ver != 1 {
t.Errorf("after adopt: got (%q, v=%d), want (migrated, 1)", got, ver)
}
}
// TestMongoKVStore_PutVersioned_ConcurrentCreate proves the absent-create race
// is linearizable: N goroutines racing PutVersioned(_, 0, _) on one key yield
// exactly one winner; losers get ErrConflict. Blocking correctness gate.
func TestMongoKVStore_PutVersioned_ConcurrentCreate(t *testing.T) {
s, _, cleanup := mongoLocalSetup(t, "coin")
defer cleanup()
ctx := context.Background()
const n = 16
var wg sync.WaitGroup
var mu sync.Mutex
var wins, conflicts, others int
start := make(chan struct{})
for i := 0; i < n; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
<-start
err := s.PutVersioned(ctx, "race", 0, []byte(fmt.Sprintf("v%d", i)))
mu.Lock()
defer mu.Unlock()
switch {
case err == nil:
wins++
case errors.Is(err, ErrConflict):
conflicts++
default:
others++
t.Errorf("unexpected error: %v", err)
}
}(i)
}
close(start)
wg.Wait()
if wins != 1 {
t.Errorf("concurrent create: got %d winners, want exactly 1", wins)
}
if conflicts != n-1 {
t.Errorf("concurrent create: got %d conflicts, want %d", conflicts, n-1)
}
if others != 0 {
t.Errorf("concurrent create produced %d non-conflict errors", others)
}
}
// TestMongoKVStore_GetDocWithoutValueField covers a malformed document edge
// case (a doc that exists but lacks the value field) — Get must surface a clear
// error, not panic or return empty bytes silently.
func TestMongoKVStore_GetDocWithoutValueField(t *testing.T) {
s, db, cleanup := mongoLocalSetup(t, "wordle")
defer cleanup()
ctx := context.Background()
// Insert a doc directly with no `value` field.
if _, err := db.Collection("wordle").InsertOne(ctx, bson.M{"_id": "novalue", "other": 1}); err != nil {
t.Fatalf("seed InsertOne: %v", err)
}
if _, err := s.Get(ctx, "novalue"); err == nil {
t.Errorf("Get on doc without value field: got nil error, want a descriptive error")
}
}
func TestMongoKVStore_DeleteMissingNoError(t *testing.T) {
s, _, cleanup := mongoLocalSetup(t, "wordle")
defer cleanup()
if err := s.Delete(context.Background(), "never-existed"); err != nil {
t.Errorf("Delete missing key: got %v, want nil (idempotent)", err)
}
}
+10 -12
View File
@@ -4,28 +4,26 @@ import (
"go.mongodb.org/mongo-driver/v2/mongo"
)
// MongoProvider is a KVProvider that creates one collection per module,
// mirroring FirestoreProvider. Collection-per-module IS the isolation — no key
// prefix wrapping is needed at this layer.
// MongoProvider is a Provider that creates one collection per module.
// Collection-per-module IS the isolation — no key prefix wrapping is needed.
type MongoProvider struct {
db *mongo.Database
}
// NewMongoProvider returns a provider over the given database handle. The
// underlying client must outlive every KVStore the provider hands out; callers
// underlying client must outlive every store the provider hands out; callers
// own its Disconnect.
func NewMongoProvider(db *mongo.Database) *MongoProvider {
return &MongoProvider{db: db}
}
// For returns a MongoKVStore writing to a collection named after the module.
// moduleName is re-validated against collectionNameRe — defense in depth
// against caller bugs that bypass modules.Build, identical to the firestore
// and dynamodb providers. An invalid name yields an invalidStore whose every
// op errors at first use.
func (p *MongoProvider) For(moduleName string) KVStore {
// Collection returns a handle to the collection named after the module.
// moduleName is re-validated against collectionNameRe — defense in depth against
// caller bugs that bypass modules.Build. An invalid name yields an
// invalidCollection whose Typed store errors at first use.
func (p *MongoProvider) Collection(moduleName string) Collection {
if !collectionNameRe.MatchString(moduleName) {
return invalidStore{name: moduleName}
return invalidCollection{name: moduleName}
}
return NewMongoKVStore(p.db.Collection(moduleName), moduleName)
return mongoCollection{coll: p.db.Collection(moduleName), module: moduleName}
}
+25 -25
View File
@@ -6,10 +6,10 @@ import (
"testing"
)
// MongoProvider.For re-validates the module name as defense-in-depth. Invalid
// names return invalidStore without touching the DB — the branch worth locking
// even without a live Mongo (mirrors the firestore provider test).
func TestMongoProvider_For_RejectsInvalidName(t *testing.T) {
// MongoProvider.Collection re-validates the module name as defense-in-depth.
// Invalid names yield an invalidCollection whose Typed store errors without
// touching the DB — the branch worth locking even without a live Mongo.
func TestMongoProvider_Collection_RejectsInvalidName(t *testing.T) {
p := &MongoProvider{db: nil}
bogus := []string{
"",
@@ -21,45 +21,45 @@ func TestMongoProvider_For_RejectsInvalidName(t *testing.T) {
"with:colon",
}
for _, name := range bogus {
store := p.For(name)
if _, err := store.Get(context.Background(), "any-key"); !errors.Is(err, ErrInvalidModuleName) {
t.Errorf("For(%q).Get → %v, want ErrInvalidModuleName", name, err)
// Invalid names short-circuit to invalidCollection before touching the
// (nil) db, so Typed yields a store whose ops report ErrInvalidModuleName.
store := Typed[string](p.Collection(name))
if _, _, err := store.Get(context.Background(), "any-key"); !errors.Is(err, ErrInvalidModuleName) {
t.Errorf("Collection(%q) store Get → %v, want ErrInvalidModuleName", name, err)
}
}
// Valid names pass validation against a real db in
// TestMongoProvider_CrossModuleIsolation (gated on MONGODB_TEST_URL); a nil
// db here would panic in the driver for a valid name.
}
// TestMongoProvider_CrossModuleIsolation verifies collection-per-module
// isolation: the same key written through two module stores yields independent
// values. Gated on MONGODB_TEST_URL.
func TestMongoProvider_CrossModuleIsolation(t *testing.T) {
_, db, cleanup := mongoLocalSetup(t, "wordle")
db, cleanup := mongoLocalSetup(t)
defer cleanup()
p := NewMongoProvider(db)
ctx := context.Background()
wordle := p.For("wordle")
loldle := p.For("loldle")
if err := wordle.Put(ctx, "shared", []byte("from-wordle")); err != nil {
// Mongo payloads must be a struct or map (bson inline cannot hoist a
// scalar); a wrapper struct is the same shape modules use.
wordle := Typed[wrappedScalar](p.Collection("wordle"))
loldle := Typed[wrappedScalar](p.Collection("loldle"))
if err := wordle.Put(ctx, "shared", wrappedScalar{Date: "from-wordle"}); err != nil {
t.Fatalf("wordle Put: %v", err)
}
if err := loldle.Put(ctx, "shared", []byte("from-loldle")); err != nil {
if err := loldle.Put(ctx, "shared", wrappedScalar{Date: "from-loldle"}); err != nil {
t.Fatalf("loldle Put: %v", err)
}
gotW, _ := wordle.Get(ctx, "shared")
gotL, _ := loldle.Get(ctx, "shared")
if string(gotW) != "from-wordle" {
t.Errorf("wordle key leaked: got %q", gotW)
gotW, _, _ := wordle.Get(ctx, "shared")
gotL, _, _ := loldle.Get(ctx, "shared")
if gotW.Date != "from-wordle" {
t.Errorf("wordle key leaked: got %q", gotW.Date)
}
if string(gotL) != "from-loldle" {
t.Errorf("loldle key leaked: got %q", gotL)
}
// Canonical names pass validation (not invalidStore); this also covers the
// valid-name branch of For against a real database.
for _, name := range []string{"misc", "demo-mod", "x", "a1_b-2"} {
if _, ok := p.For(name).(invalidStore); ok {
t.Errorf("For(%q) returned invalidStore; expected validation to pass", name)
}
if gotL.Date != "from-loldle" {
t.Errorf("loldle key leaked: got %q", gotL.Date)
}
}
-116
View File
@@ -1,116 +0,0 @@
package storage
import (
"bytes"
"encoding/json"
"fmt"
"go.mongodb.org/mongo-driver/v2/bson"
)
// This file converts between the KVStore's opaque JSON []byte contract and a
// native BSON representation, so values render as expandable documents in the
// Atlas/Compass UI (and are queryable) rather than as opaque blobs.
//
// Fidelity: JSON is decoded with json.Number so integral numbers persist as
// BSON int64 (NOT float64) — a struct's int64 field round-trips without
// precision loss. Only JSON objects and arrays become native; bare scalars and
// non-JSON values (e.g. a plain date string) fall back to a BSON string.
// encodeValue produces the stored BSON representation of a value. A JSON object
// or array → native (expandable/queryable); anything else → a string (the
// human-readable fallback, also covers non-UTF-8-free callers identically to
// the previous string encoding).
func encodeValue(val []byte) any {
if native, ok := jsonToNative(val); ok {
return native
}
return string(val)
}
// jsonToNative decodes val into a native BSON value (bson.M / bson.A) when it is
// a JSON object or array, preserving integers as int64. Returns ok=false for
// scalars, non-JSON, or decode errors (caller stores those as a string).
func jsonToNative(val []byte) (any, bool) {
trimmed := bytes.TrimSpace(val)
if len(trimmed) == 0 || (trimmed[0] != '{' && trimmed[0] != '[') {
return nil, false
}
dec := json.NewDecoder(bytes.NewReader(val))
dec.UseNumber()
var v any
if err := dec.Decode(&v); err != nil {
return nil, false
}
// Reject trailing garbage after the JSON value.
if dec.More() {
return nil, false
}
return numbersToBSON(v), true
}
// numbersToBSON walks a json.Unmarshal(UseNumber) tree, converting maps/slices
// to bson.M/bson.A and json.Number to int64 (when integral) else float64.
func numbersToBSON(v any) any {
switch t := v.(type) {
case map[string]any:
m := make(bson.M, len(t))
for k, val := range t {
m[k] = numbersToBSON(val)
}
return m
case []any:
a := make(bson.A, len(t))
for i, val := range t {
a[i] = numbersToBSON(val)
}
return a
case json.Number:
if i, err := t.Int64(); err == nil {
return i
}
if f, err := t.Float64(); err == nil {
return f
}
return t.String()
default:
return v // string, bool, nil
}
}
// nativeToJSON re-serializes a natively-stored value (bson.M / bson.A / bson.D
// / scalar) back to JSON bytes. bson.M is map[string]any and bson.A is []any,
// both of which json.Marshal handles directly; bson.D (ordered) is normalized
// to a map first so it marshals as an object, not an array of key/value pairs.
func nativeToJSON(raw any) ([]byte, error) {
out, err := json.Marshal(normalizeBSON(raw))
if err != nil {
return nil, fmt.Errorf("mongo: re-encode native value: %w", err)
}
return out, nil
}
// normalizeBSON converts bson.D to bson.M recursively so json.Marshal emits an
// object. bson.M/bson.A children are recursed; scalars pass through.
func normalizeBSON(v any) any {
switch t := v.(type) {
case bson.D:
m := make(bson.M, len(t))
for _, e := range t {
m[e.Key] = normalizeBSON(e.Value)
}
return m
case bson.M:
for k, val := range t {
t[k] = normalizeBSON(val)
}
return t
case bson.A:
for i, val := range t {
t[i] = normalizeBSON(val)
}
return t
default:
return v
}
}
-75
View File
@@ -1,75 +0,0 @@
package storage
import (
"context"
"errors"
"fmt"
"strings"
)
// Prefixed returns a KVStore that transparently prepends prefix+":" to every
// key. List() strips the prefix from returned keys so callers see their own
// flat namespace. The prefix must be non-empty.
func Prefixed(inner KVStore, prefix string) KVStore {
if prefix == "" {
panic("storage: Prefixed requires non-empty prefix")
}
return &prefixedStore{inner: inner, prefix: prefix + ":"}
}
type prefixedStore struct {
inner KVStore
prefix string
}
func (p *prefixedStore) k(key string) string { return p.prefix + key }
func (p *prefixedStore) Get(ctx context.Context, key string) ([]byte, error) {
return p.inner.Get(ctx, p.k(key))
}
func (p *prefixedStore) GetJSON(ctx context.Context, key string, dst any) error {
return p.inner.GetJSON(ctx, p.k(key), dst)
}
func (p *prefixedStore) Put(ctx context.Context, key string, val []byte) error {
return p.inner.Put(ctx, p.k(key), val)
}
func (p *prefixedStore) GetVersioned(ctx context.Context, key string) ([]byte, int64, error) {
vs, ok := p.inner.(VersionedStore)
if !ok {
return nil, 0, fmt.Errorf("storage: inner store does not support versioned reads: %w", errors.ErrUnsupported)
}
return vs.GetVersioned(ctx, p.k(key))
}
func (p *prefixedStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error {
vs, ok := p.inner.(VersionedStore)
if !ok {
// A missing capability is permanent: report it as unsupported so
// callers fail fast instead of retrying it as a write conflict.
return fmt.Errorf("storage: inner store does not support versioned writes: %w", errors.ErrUnsupported)
}
return vs.PutVersioned(ctx, p.k(key), expectedVersion, val)
}
func (p *prefixedStore) PutJSON(ctx context.Context, key string, val any) error {
return p.inner.PutJSON(ctx, p.k(key), val)
}
func (p *prefixedStore) Delete(ctx context.Context, key string) error {
return p.inner.Delete(ctx, p.k(key))
}
func (p *prefixedStore) List(ctx context.Context, prefix string) ([]string, error) {
keys, err := p.inner.List(ctx, p.k(prefix))
if err != nil {
return nil, err
}
out := make([]string, len(keys))
for i, k := range keys {
out[i] = strings.TrimPrefix(k, p.prefix)
}
return out, nil
}
-134
View File
@@ -1,134 +0,0 @@
package storage
import (
"context"
"errors"
"reflect"
"testing"
)
func TestPrefixed_RoundTrip(t *testing.T) {
ctx := context.Background()
base := NewMemoryKVStore()
a := Prefixed(base, "modA")
b := Prefixed(base, "modB")
if err := a.Put(ctx, "score", []byte("10")); err != nil {
t.Fatalf("a.Put: %v", err)
}
if err := b.Put(ctx, "score", []byte("20")); err != nil {
t.Fatalf("b.Put: %v", err)
}
got, err := a.Get(ctx, "score")
if err != nil {
t.Fatalf("a.Get: %v", err)
}
if string(got) != "10" {
t.Errorf("a.Get score = %q, want %q", got, "10")
}
got, err = b.Get(ctx, "score")
if err != nil {
t.Fatalf("b.Get: %v", err)
}
if string(got) != "20" {
t.Errorf("b.Get score = %q, want %q", got, "20")
}
}
func TestPrefixed_ListStripsPrefix(t *testing.T) {
ctx := context.Background()
base := NewMemoryKVStore()
mod := Prefixed(base, "wordle")
for _, k := range []string{"u:1", "u:2", "session:abc"} {
if err := mod.Put(ctx, k, []byte("x")); err != nil {
t.Fatalf("Put %q: %v", k, err)
}
}
got, err := mod.List(ctx, "u:")
if err != nil {
t.Fatalf("List: %v", err)
}
want := []string{"u:1", "u:2"}
if !reflect.DeepEqual(got, want) {
t.Errorf("List u: = %v, want %v", got, want)
}
}
func TestPrefixed_NotFoundPropagates(t *testing.T) {
ctx := context.Background()
base := NewMemoryKVStore()
mod := Prefixed(base, "modA")
if _, err := mod.Get(ctx, "missing"); err != ErrNotFound {
t.Errorf("Get missing = %v, want ErrNotFound", err)
}
}
func TestPrefixed_VersionedDelegatesWithPrefixedKey(t *testing.T) {
ctx := context.Background()
base := NewMemoryKVStore()
mod := Prefixed(base, "gold")
vs, ok := mod.(VersionedStore)
if !ok {
t.Fatal("Prefixed store does not implement VersionedStore")
}
if err := vs.PutVersioned(ctx, "user:1", 0, []byte("v1")); err != nil {
t.Fatalf("PutVersioned create: %v", err)
}
// The write must land under the prefixed key on the inner store.
got, err := base.Get(ctx, "gold:user:1")
if err != nil {
t.Fatalf("inner Get: %v", err)
}
if string(got) != "v1" {
t.Errorf("inner value = %q, want %q", got, "v1")
}
// GetVersioned reads through the prefix and returns the inner version.
val, ver, err := vs.GetVersioned(ctx, "user:1")
if err != nil || string(val) != "v1" || ver != 1 {
t.Fatalf("GetVersioned: got (%q, v=%d, %v), want (v1, 1, nil)", val, ver, err)
}
// Stale version must surface the inner store's conflict.
if err := vs.PutVersioned(ctx, "user:1", 99, []byte("v2")); !errors.Is(err, ErrConflict) {
t.Errorf("stale version: got %v, want ErrConflict", err)
}
if err := vs.PutVersioned(ctx, "user:1", 1, []byte("v2")); err != nil {
t.Errorf("matching version: %v", err)
}
}
// plainStore hides the inner store's versioned methods so the wrapper's
// missing-capability path can be exercised.
type plainStore struct {
KVStore
}
func TestPrefixed_VersionedUnsupportedInner(t *testing.T) {
mod := Prefixed(&plainStore{KVStore: NewMemoryKVStore()}, "gold")
vs := mod.(VersionedStore)
err := vs.PutVersioned(context.Background(), "user:1", 0, []byte("v1"))
if !errors.Is(err, errors.ErrUnsupported) {
t.Errorf("got %v, want errors.ErrUnsupported", err)
}
if errors.Is(err, ErrConflict) {
t.Error("missing versioned capability must not be reported as a retryable conflict")
}
}
func TestPrefixed_PanicsOnEmptyPrefix(t *testing.T) {
defer func() {
if recover() == nil {
t.Error("Prefixed(_, \"\") did not panic")
}
}()
Prefixed(NewMemoryKVStore(), "")
}
@@ -6,7 +6,7 @@ priority: P2
branch: "feature/selfhosted"
tags: [selfhost, coolify, mongodb, migration]
blockedBy: []
blocks: [260628-1113-mongo-native-value-documents]
blocks: [260628-1113-mongo-native-value-documents, 260628-1310-flatten-mongo-value-documents]
created: "2026-06-27T12:01:53.894Z"
createdBy: "ck:plan"
source: skill
@@ -0,0 +1,136 @@
---
phase: 1
title: "Schema Contract and Regression Tests"
status: pending
priority: P2
dependencies: []
effort: "M"
---
# Phase 1: Schema Contract and Regression Tests
## Overview
Define the exact Mongo document contract and lock it with failing tests before
changing the codec. This phase prevents a vague "flatten value" implementation
from preserving stale fields, losing type fidelity, or silently breaking legacy
reads.
## Requirements
- Functional: specify new raw document shapes for JSON object, array, scalar,
non-JSON string, reserved-key collision, and legacy `value` docs.
- Functional: preserve current `KVStore` and `VersionedStore` method contracts.
- Functional: prove `updatedAt` is BSON Date / Go `time.Time`.
- Non-functional: tests must be explicit enough to fail against the current
`{ _id, value, version, updatedAt int64 }` shape.
## Architecture
Target document examples:
```javascript
// Normal JSON object payload, flattened.
{
_id: "user:7",
usd: 1000.25,
assets: { BTC: 1 },
meta: { createdAt: 1782604800000 },
version: 4,
updatedAt: ISODate("2026-06-28T06:10:29Z"),
schemaVersion: 2
}
// Non-object fallback. Mongo root must be a document, so array/scalar payloads
// need a reserved payload field. This is not the old generic `value` field.
{
_id: "daily_push:last_date",
_payload: "2026-06-28",
payloadKind: "string",
version: 1,
updatedAt: ISODate("2026-06-28T06:10:29Z"),
schemaVersion: 2
}
```
Reserved fields: `_id`, `value`, `version`, `updatedAt`, `schemaVersion`,
`payloadKind`, `_payload`. If an object payload contains one of these keys, the
codec must preserve data by storing the whole object in `_payload` with
`payloadKind: "object"` instead of flattening. Object payloads with keys that
contain null, contain `.`, or start with `$` also use `_payload`; MongoDB
permits some of these in modern versions but documents restrictions, so fallback
keeps query/index behavior predictable. <!-- Updated: Validation Session 1 - reserve legacy value + unsafe Mongo field names -->
## Related Code Files
- Modify: `internal/storage/mongodb_kv_test.go` — raw-shape, legacy-read,
stale-field, updatedAt, and CAS tests.
- Modify: `internal/storage/mongodb_value_codec.go` — only if test helpers need
exported/unexported constants clarified later.
- Read: `internal/storage/mongodb_kv.go`
- Read: `internal/storage/kv_store.go`
- Read: `plans/reports/research-260628-0605-mongodb-document-design-standards.md`
## Tests Before
Add/adjust Mongo integration tests gated by `MONGODB_TEST_URL`:
1. `TestMongoKVStore_RootObjectRepresentation`
- `PutJSON` a coin-like portfolio.
- Raw Mongo doc has root `usd`, `assets`, `meta`.
- Raw Mongo doc has no `value`.
- `updatedAt` decodes as `time.Time`.
2. `TestMongoKVStore_RootObjectOverwriteRemovesStaleFields`
- Write `{a:1,b:2}` then overwrite `{a:3}`.
- Raw doc has `a`, no stale `b`.
3. `TestMongoKVStore_NonObjectPayloadFallback`
- Write bare string and JSON array.
- Raw docs use `_payload` + `payloadKind`, no `value`.
- `Get` round-trips.
4. `TestMongoKVStore_ReservedRootFieldCollision`
- Write object with `value`, `version`, or `updatedAt` in payload.
- Raw doc preserves payload under `_payload`; `GetJSON` round-trips.
5. `TestMongoKVStore_UnsafeMongoFieldNameFallback`
- Write object payloads with keys containing null, containing `.`, and
starting with `$`.
- Raw docs preserve payload under `_payload`; `GetJSON` round-trips.
6. `TestMongoKVStore_LegacyValueDocsStillRead`
- Seed old docs directly with `value` as string, `bson.Binary`, object,
and array.
- `Get`/`GetJSON` decode all.
7. Keep/strengthen `TestMongoKVStore_PutVersioned_ConcurrentCreate`.
## Implementation Steps
1. Add named constants in tests for reserved fields expected in the new shape.
2. Add raw Mongo assertions before changing implementation.
3. Audit current persisted structs for reserved top-level key collisions and
document the result in test comments.
4. Confirm all new tests fail for the intended reason against current code.
5. Keep existing int64-fidelity and legacy-version tests in place.
## Tests After
- Re-run the new Mongo storage tests after Phase 2 implementation.
- Ensure old native-`value` tests are either rewritten to the new contract or
retained only as legacy-read tests.
## Success Criteria
- [ ] New raw-shape tests exist and fail before implementation.
- [ ] Legacy-read coverage includes the old `value` field.
- [ ] Reserved-field coverage includes payload key `value`.
- [ ] Unsafe Mongo field-name coverage includes null, `.`, and `$` cases.
- [ ] Stale-field removal has a dedicated regression test.
- [ ] `updatedAt` BSON Date behavior has a dedicated assertion.
- [ ] Existing version-CAS tests remain in the suite.
## Risk Assessment
- Risk: tests assert driver-specific decoded types too tightly. Mitigation:
assert behavior and accepted BSON-decoded Go types, not exact internal map
ordering.
- Risk: reserved-field collision behavior gets forgotten because current structs
do not collide. Mitigation: synthetic collision test is required.
- Risk: stale-field issue appears only on overwrite. Mitigation: explicit
overwrite test before implementation.
@@ -0,0 +1,139 @@
---
phase: 2
title: "Root Document Storage Encoding"
status: pending
priority: P2
dependencies: [1]
effort: "L"
---
# Phase 2: Root Document Storage Encoding
## Overview
Implement the new root-payload Mongo encoding in `MongoKVStore` while preserving
all caller-facing storage interfaces. Writes should stop producing `value`,
reads should support both new root docs and legacy `value` docs.
## Requirements
- Functional: `Put`, `PutJSON`, `PutVersioned`, `Get`, `GetJSON`,
`GetVersioned`, `Delete`, and `List` signatures remain unchanged.
- Functional: JSON object values flatten to root unless they collide with
reserved metadata fields, the legacy `value` field, or unsafe Mongo field
names.
- Functional: arrays/scalars/non-JSON values use `_payload` and `payloadKind`,
never the old `value` field.
- Functional: `updatedAt` is stored as `time.Time`.
- Functional: `version` increments on plain and versioned writes.
- Non-functional: no stale payload fields after overwrite.
## Architecture
Split the codec into document-level helpers:
```go
type payloadKind string
const (
payloadKindObject payloadKind = "object"
payloadKindArray payloadKind = "array"
payloadKindString payloadKind = "string"
)
func encodeRootDocument(key string, val []byte, version int64, now time.Time) (bson.M, error)
func decodeRootDocument(key string, doc bson.M) ([]byte, error)
func payloadFieldsFromRoot(doc bson.M) bson.M
```
Decode order:
1. Legacy `value` field exists -> current `decodeValue` compatibility path.
2. `_payload` exists -> decode `_payload` by `payloadKind`.
3. Else -> collect all root fields except reserved metadata and marshal as JSON.
Write strategy:
- `PutVersioned(expectedVersion > 0)`: `ReplaceOne({_id:key, version:expected}, replacement(version=expected+1))`.
- `PutVersioned(expectedVersion == 0)`: replace/upsert with filter matching
`{_id:key, version missing or 0}`, duplicate key -> `ErrConflict`.
- `Put`: implement as a bounded loop over `GetVersioned` + `PutVersioned` so it
overwrites unconditionally while still removing stale fields and bumping
version. Use existing retry style (`portfolioUpdateAttempts` is 5) or a
small storage-local constant.
Use `ReplaceOne`, not partial `$set`, for root-payload writes. Partial updates
are the foot-gun: `{a:1,b:2}` overwritten by `{a:3}` would otherwise leave `b`.
## Related Code Files
- Modify: `internal/storage/mongodb_kv.go` — write/read methods and version
replacement logic.
- Modify: `internal/storage/mongodb_value_codec.go` — convert byte payload to
flattened root docs and reconstruct JSON.
- Modify: `internal/storage/mongodb_kv_test.go` — make Phase 1 tests pass.
- Read: `internal/storage/memory_kv.go` — keep interface behavior aligned.
- Read: `internal/storage/dynamodb_kv.go` — ensure migrate-only DynamoDB stays
unaffected.
## Tests Before
- Run Phase 1 Mongo tests and confirm failure before implementation.
- Run hermetic storage tests to ensure no memory/DynamoDB API changes are
required.
## Refactor
1. Rename legacy `decodeValue` to make compatibility explicit, e.g.
`decodeLegacyValueField`.
2. Add reserved-field constants and helper `isMongoRootMetadataField`.
3. Add root-document encoder:
- trim/decode JSON with `json.Decoder.UseNumber`;
- object -> root fields if no reserved/legacy `value` collision and no
unsafe key containing null, containing `.`, or starting with `$`;
- array -> `_payload` + `payloadKind:"array"`;
- scalar/non-JSON -> `_payload` + `payloadKind:"string"` or scalar kind;
- always add `_id`, `version`, `updatedAt`, `schemaVersion`.
<!-- Updated: Validation Session 1 - reserve legacy value + unsafe Mongo field names -->
4. Add root-document decoder:
- legacy `value` first;
- `_payload` fallback second;
- root object reconstruction third.
5. Replace `$set`/`$inc` writes with whole-document replacement in
`PutVersioned`.
6. Rework `Put` to bump versions and replace whole docs without exposing
conflicts to ordinary callers unless retries are exhausted.
7. Keep `Delete` and `List` unchanged.
8. Tighten error messages so malformed root docs name the module/key and
missing payload reason.
## Tests After
- `go test ./internal/storage`
- `MONGODB_TEST_URL=mongodb://127.0.0.1:27017 go test ./internal/storage -run 'MongoKVStore'`
- `go test ./internal/modules/coin ./internal/modules/gold ./internal/modules/lolschedule`
## Success Criteria
- [ ] All Phase 1 tests pass.
- [ ] No new write path emits `value`.
- [ ] Payload objects with key `value` use `_payload` fallback, not root
flattening.
- [ ] Payload objects with null/`.`/`$` field names use `_payload` fallback.
- [ ] Legacy `value` docs continue to read.
- [ ] Plain `Put` and `PutVersioned` both bump `version`.
- [ ] Concurrent create/update tests keep exactly-one-winner semantics.
- [ ] Stale payload fields are removed on overwrite.
## Risk Assessment
- Risk: bounded `Put` loop can conflict under extreme contention. Mitigation:
keep retries small but adequate; if tests show trouble, switch to a
single-write aggregation pipeline with `$replaceRoot` and `$literal` payload
expressions.
- Risk: root reconstruction accidentally includes metadata fields. Mitigation:
centralize reserved field list and test it.
- Risk: `time.Time` changes raw `updatedAt` type. Mitigation: no app reader uses
it today; integration tests assert new date behavior.
- Risk: malformed existing docs without `value` or root payload become harder
to diagnose. Mitigation: descriptive errors in decoder.
@@ -0,0 +1,126 @@
---
phase: 3
title: "Migration and Documentation"
status: pending
priority: P2
dependencies: [2]
effort: "M"
---
# Phase 3: Migration and Documentation
## Overview
Add an operator-safe path to rewrite existing Mongo docs from the old `value`
envelope into the new flattened root shape, and update docs that currently
describe the old layout.
## Requirements
- Functional: existing Atlas documents can be migrated in place without relying
on DynamoDB still existing.
- Functional: migration is idempotent and reports per-collection counts.
- Functional: migration preserves logical `Get`/`GetJSON` values and bumps or
normalizes `version` consistently.
- Non-functional: migration must not read or print secrets; `MONGO_URL` remains
secret.
- Non-functional: docs must stop calling int64 `updatedAt` TTL-ready.
## Architecture
Add a small Mongo-to-Mongo schema migration command rather than overloading the
DynamoDB migrator:
```text
cmd/migrate-mongo-schema/
main.go
main_test.go
```
Tool behavior:
1. Connect using `MONGO_URL` + `MONGO_DATABASE`.
2. Iterate collections or an allow-list flag.
3. For each document:
- skip if it already has `schemaVersion >= 2` and no `value`;
- read through `MongoKVStore.GetVersioned` so legacy decode paths are used;
- rewrite through `PutVersioned` so root encoding is shared with app writes;
- on `ErrConflict`, re-read the document: skip it if a live app write
already migrated it, otherwise retry up to a small bounded limit;
- count migrated/skipped/errors.
4. `--dry-run` prints counts without writes.
5. `--verify` confirms no documents with `value` remain, except explicitly
allowed malformed docs if any are reported.
Do not drop or rename collections. Keep one collection per module.
## Related Code Files
- Create: `cmd/migrate-mongo-schema/main.go`
- Create: `cmd/migrate-mongo-schema/main_test.go`
- Modify: `Makefile` — add a migration target, e.g. `migrate-mongo-schema`.
- Modify: `README.md` — update storage layout summary.
- Modify: `docs/deploy-coolify-selfhosted.md` — update Atlas storage layout
and migration note.
- Modify: `cmd/migrate-dynamo-to-mongo/README.md` — update destination shape.
- Modify: `cmd/migrate-dynamo-to-mongo/main.go` comments only if stale.
- Read: `cmd/migrate-dynamo-to-mongo/main.go` — reuse connection/config style.
## Tests Before
- Add migrator e2e test that seeds old `value` docs directly, runs migration,
then asserts:
- logical values round-trip;
- raw docs have no `value`;
- `updatedAt` is `time.Time`;
- counts match.
## Implementation Steps
1. Implement CLI flags:
- `--dry-run`
- `--verify`
- `--collection` optional repeat/single collection selector.
2. Reuse `storage.NewMongoClient`, `storage.NewMongoDatabase`, and
`storage.NewMongoProvider`.
3. Add collection listing using the Mongo driver, excluding system collections.
4. Rewrite docs through the storage layer, not raw BSON mutation.
5. Print a concise table: collection, total, migrated, skipped, errors.
6. Add Makefile target with env examples.
7. Update docs and remove stale wording:
- no old `value` envelope in new writes;
- `updatedAt` is BSON Date;
- legacy `value` docs dual-read until migrated.
8. Add migration conflict tests: simulate a version change between read and
write; assert the migrator re-reads and either skips already-migrated docs or
retries legacy docs without clobbering the concurrent update.
<!-- Updated: Validation Session 1 - migration must handle live-write conflicts -->
## Tests After
- `go test ./cmd/migrate-mongo-schema`
- `MONGODB_TEST_URL=mongodb://127.0.0.1:27017 go test ./cmd/migrate-mongo-schema ./internal/storage`
- `make test`
## Success Criteria
- [ ] In-place migration rewrites old `value` docs without changing logical app
values.
- [ ] Migration is idempotent.
- [ ] `--dry-run` performs no writes.
- [ ] `--verify` exits non-zero when old `value` docs remain.
- [ ] Migration handles version conflicts by re-reading and never clobbers a
concurrent app write.
- [ ] README and Coolify deploy docs show the new flattened layout.
- [ ] DynamoDB-to-Mongo migrator docs no longer mention BSON binary/string as
the destination `value` representation.
## Risk Assessment
- Risk: migration touches live data. Mitigation: dry-run + verify + local e2e
test; recommend backup/export before production run.
- Risk: old DynamoDB source may be gone. Mitigation: in-place Mongo migration is
independent of DynamoDB.
- Risk: malformed documents stop migration. Mitigation: report collection/key
and continue only if an explicit `--continue-on-error` is later accepted;
default should fail loud.
@@ -0,0 +1,121 @@
---
phase: 4
title: "Verification and Rollout"
status: pending
priority: P2
dependencies: [3]
effort: "M"
---
# Phase 4: Verification and Rollout
## Overview
Run the full validation suite, check raw Mongo documents, and define the safe
production rollout order. This phase is the gate before deploying a storage
schema change.
## Requirements
- Functional: app behavior remains unchanged for all Telegram commands.
- Functional: Mongo raw shape is verified for representative module documents.
- Functional: legacy docs remain readable before migration and absent after
migration verify.
- Non-functional: rollout includes backup, deploy, migration, verify, and
rollback notes.
## Architecture
Rollout order:
1. Backup/export current Atlas database or snapshot if available.
2. Deploy app version with dual-read + new-write root encoding.
3. Let app run; new/touched docs rewrite naturally.
4. Run `migrate-mongo-schema --dry-run`.
5. Run `migrate-mongo-schema`.
6. Run `migrate-mongo-schema --verify`.
7. Inspect sample docs in Atlas/Compass:
- coin/gold/stock portfolio root fields;
- wordle/loldle/twentyq game state root fields;
- lolschedule scalar date guard under `_payload`;
- no old `value` on migrated docs.
Rollback:
- If the new app fails before it handles writes, roll back binary; old `value`
docs are still readable by the previous build.
- Once the new app has handled writes, rollback to the previous binary is unsafe
for any touched flattened docs. Prefer fixing forward, or restore the backup.
- If migration ran, previous build may not read flattened docs at all. Rollback
after migration requires restoring backup or a reverse migrator. Prefer fixing
forward unless the migration corrupted data.
## Related Code Files
- Modify: tests only if verification gaps are found.
- Read: `Makefile`
- Read: `README.md`
- Read: `docs/deploy-coolify-selfhosted.md`
- Read: `cmd/migrate-mongo-schema/main.go`
- Read: `internal/storage/mongodb_kv_test.go`
## Tests Before
- Confirm Phase 1-3 tests run locally.
- Start local Mongo with `make mongo-local` for integration tests.
## Implementation Steps
1. Run focused storage tests.
2. Run module tests that rely on versioned storage flows:
- coin portfolio;
- gold portfolio;
- lolschedule daily-push claim.
3. Run full hermetic suite.
4. Run Mongo integration suite.
5. Run DynamoDB tests only if the DynamoDB migrator or shared storage contracts
changed.
6. Manually inspect raw docs or add a small test assertion report for:
- no `value`;
- expected root fields;
- Date `updatedAt`;
- stable `version`.
7. Document production run commands in the final completion report.
## Tests After
Required gates:
```sh
make vet
make test
MONGODB_TEST_URL=mongodb://127.0.0.1:27017 go test ./internal/storage ./cmd/migrate-mongo-schema
```
Conditional gate if DynamoDB migrator comments/tests changed:
```sh
make test-dynamodb
```
## Success Criteria
- [ ] `make vet` passes.
- [ ] `make test` passes.
- [ ] Mongo integration tests pass against local Mongo.
- [ ] New migrator e2e test passes.
- [ ] Raw docs verified no longer use `value` after migration.
- [ ] Production rollout notes include backup and rollback constraints.
- [ ] No user-visible Telegram command behavior changes.
## Risk Assessment
- Risk: previous binary cannot read newly-written or post-migration flattened
docs. Mitigation: deploy dual-read binary first; backup before rollout;
rollback old binary only before new writes happen; fix forward preferred.
- Risk: Atlas M0 resource limits during migration. Mitigation: tiny dataset
expected; still scan collection-by-collection and report progress.
- Risk: hidden module stores non-object values. Mitigation: fallback `_payload`
contract and migration tests for scalar/array docs.
- Risk: docs drift. Mitigation: grep for old `{ _id, value, version, updatedAt }`
wording before finalizing.
@@ -0,0 +1,221 @@
---
title: "Flatten Mongo value documents"
description: "Remove Mongo's generic value envelope for JSON object values, keep KVStore callers stable, and store Mongo metadata in queryable root fields."
status: superseded
supersededBy: 260628-1318-mongo-native-typed-stores
priority: P2
branch: "feature/selfhosted"
tags: [database, mongodb, refactor, storage, tdd]
blockedBy: [260627-1849-selfhost-coolify-mongodb]
blocks: []
created: "2026-06-28T06:10:29.184Z"
createdBy: "ck:plan"
source: skill
---
# Flatten Mongo value documents
> **Superseded (2026-06-28)** by
> [`260628-1318-mongo-native-typed-stores`](../260628-1318-mongo-native-typed-stores/plan.md).
> User chose the full Mongo-native direction this plan deferred: delete the
> `KVStore` abstraction in favor of typed stores, MongoDB-only runtime (memory
> kept for tests), and drop the legacy `value` dual-read + in-place Mongo
> migrator (Atlas is empty until cutover, so the DynamoDB→Mongo migrator writes
> the final flattened shape directly). Kept for history; do not implement.
## Overview
Change MongoDB's on-disk KV document shape from a generic envelope:
```javascript
{ _id, value: { ... }, version, updatedAt }
```
to a more Mongo-native root-document shape for JSON object values:
```javascript
{ _id, ...payloadFields, version, updatedAt: ISODate(...), schemaVersion }
```
Keep the public `KVStore` / `VersionedStore` interfaces stable so modules keep
using `GetJSON`, `PutJSON`, `GetVersioned`, and `PutVersioned`. This plan is a
storage-representation refactor, not a full rewrite to typed repositories.
Non-object values cannot be Mongo root documents. New writes for arrays/scalars
use a small reserved `_payload` fallback with `payloadKind`, but **new writes
must not use the old `value` field**. Legacy docs with `value` remain readable
and are rewritten to the new shape by normal updates or by the in-place
migration tool.
## Scope Challenge
- Existing code: `MongoKVStore` already owns all Mongo encoding/decoding and
version CAS; `mongodb_value_codec.go` already preserves `int64` via
`json.Number`; modules do not need direct Mongo access.
- Minimum changes: storage codec + raw-shape tests + migrator/docs. Defer
per-module typed repositories until there is a real query/report need.
- Complexity: touches storage, one migration command, docs, and tests. Avoid
changing module handlers or public storage interfaces.
- Selected mode: HOLD SCOPE. Refactor the persisted Mongo shape; do not expand
into a full domain repository rewrite.
## Design Decision
Chosen approach: **root-payload KV documents**.
Why not full typed repositories now: it would touch most modules, duplicate
test plumbing, and remove the memory backend value before there is a query need.
Why not keep `value`: it defeats the user's Mongo document-design goal and
keeps domain fields one level away from simple Compass/query/index use.
Reserved root fields:
| Field | Purpose |
|---|---|
| `_id` | Module-local KV key, unchanged |
| `value` | Legacy envelope field; forbidden in new root payloads |
| `version` | Optimistic-lock token, unchanged semantics |
| `updatedAt` | BSON Date (`time.Time`), not unix nanos |
| `schemaVersion` | Storage document shape version |
| `payloadKind` | Only needed for non-object fallback or collision fallback |
| `_payload` | Array/scalar payload fallback; never used for ordinary JSON objects |
Mongo field-name guard: object payloads also fall back to `_payload` if any
flattened key contains the null character, contains `.`, or starts with `$`.
MongoDB permits `.` and `$` in modern versions but documents restrictions around
them; fallback keeps normal query/index/schema behavior predictable.
Reference: https://www.mongodb.com/docs/manual/core/document/#field-names
Decoder rules:
1. If old `value` exists, decode using the legacy path.
2. Else if `_payload` exists, decode `_payload` according to `payloadKind`.
3. Else reconstruct the JSON object from all non-reserved root fields.
Writer rules:
1. JSON object without reserved-field collision writes payload fields at root.
2. JSON array, scalar, invalid JSON, or object with reserved-field collision
or unsafe Mongo field names writes `_payload` + `payloadKind`.
3. Writes replace the whole document (except the `_id` key value is preserved in
the replacement) so stale old payload fields cannot survive an overwrite.
4. `PutVersioned` keeps single-document CAS semantics by filtering on `_id` and
`version`.
5. Plain `Put` must still bump `version`; implement as a bounded versioned
write loop or a carefully tested aggregation update pipeline. Prefer the
simpler loop unless tests prove unacceptable contention.
## Cross-Plan Dependencies
| Relationship | Plan | Status | Rationale |
|---|---|---|---|
| Blocked by | `260627-1849-selfhost-coolify-mongodb` | in-progress | Created the Mongo provider, migrator, and self-host runtime this refactor modifies. |
| Supersedes storage choice from | `260628-1113-mongo-native-value-documents` | completed | Keeps its version CAS + native BSON goal, but replaces the `value` envelope with root payload fields. |
## Phases
| Phase | Name | Status |
|-------|------|--------|
| 1 | [Schema Contract and Regression Tests](./phase-01-schema-contract-and-regression-tests.md) | Pending |
| 2 | [Root Document Storage Encoding](./phase-02-root-document-storage-encoding.md) | Pending |
| 3 | [Migration and Documentation](./phase-03-migration-and-documentation.md) | Pending |
| 4 | [Verification and Rollout](./phase-04-verification-and-rollout.md) | Pending |
## Dependencies
- Existing Mongo provider and version-CAS code from
`plans/260628-1113-mongo-native-value-documents/`.
- Research report:
`plans/reports/research-260628-0605-mongodb-document-design-standards.md`.
- MongoDB official guidance used in the research report:
- https://www.mongodb.com/docs/manual/data-modeling/
- https://www.mongodb.com/docs/manual/data-modeling/best-practices/
- https://www.mongodb.com/docs/manual/core/write-operations-atomicity/
- https://www.mongodb.com/docs/manual/core/index-ttl/
## Acceptance Criteria
- [ ] New JSON object writes contain no top-level `value` field; payload fields
live at root with `_id`, `version`, `updatedAt`, and `schemaVersion`.
- [ ] Payload objects with top-level `value`, reserved metadata keys, null
characters, `.`, or `$`-prefixed keys use `_payload` fallback and round-trip.
- [ ] New array/scalar/non-JSON writes contain no old `value` field; they use
`_payload` + `payloadKind` and round-trip byte-equivalent where required.
- [ ] `updatedAt` stores as BSON Date / Go `time.Time`, not int64 nanos.
- [ ] Legacy docs with `value` as string, binary, object, or array still read.
- [ ] Overwriting a document removes stale payload fields from prior values.
- [ ] Versioned writes remain single-winner under concurrent create/update.
- [ ] A Mongo in-place migration can rewrite existing `value` docs to the new
shape without changing logical values or document counts.
- [ ] `make test`, `make vet`, and Mongo integration tests pass.
## Not In Scope
- Replacing every module's `KVStore` usage with typed Mongo repositories.
- Adding Mongo schema validation/indexes for every module.
- Deleting DynamoDB migrator support.
- Changing Telegram bot behavior or command outputs.
## Validation Log
### Session 1 — 2026-06-28
**Trigger:** `/ck:plan validate /config/workspace/tiennm99/miti99bot/plans/260628-1310-flatten-mongo-value-documents/plan.md`
**Questions asked:** 0 — validation found technical plan gaps with clear repo/doc evidence; no user-facing trade-off needed.
#### Plan Recap
- Refactor Mongo on-disk KV documents from `{ _id, value, version, updatedAt }`
to root payload fields.
- Keep `KVStore` / `VersionedStore` caller contracts stable.
- Keep legacy `value` docs readable.
- Use `_payload` only for non-object or unsafe/colliding object payloads.
- Store `updatedAt` as BSON Date.
- Add an in-place Mongo schema migration and rollout guardrails.
#### Verification Results
- **Tier:** Standard (4 phases)
- **Claims checked:** 32
- **Verified:** 29 | **Failed:** 3 | **Unverified:** 0
Failures fixed in this validation:
1. [Contract Verifier] Reserved fields omitted `value`. Evidence:
`internal/storage/mongodb_kv.go:19` uses `mongoValueField = "value"` and
`decodeValue` treats any top-level `value` as legacy payload
(`internal/storage/mongodb_kv.go:46`). A flattened payload containing
top-level `value` would violate the no-`value` contract and decode wrong.
2. [Fact Checker] Field-name guard omitted null/`.`/`$` cases. Evidence:
MongoDB documents forbid null in field names and document restrictions for
`.`/`$` field names: https://www.mongodb.com/docs/manual/core/document/#field-names.
The plan now falls back to `_payload` for those keys.
3. [Contract Verifier] Migration rewrite via `PutVersioned` lacked conflict
retry handling. Evidence: `PutVersioned` returns `ErrConflict` on version
mismatch (`internal/storage/kv_store.go:35`) and concurrent app writes are
possible after deploying the dual-read/new-write binary. Phase 3 now requires
re-read/skip/retry behavior.
#### Confirmed Decisions
- Keep `value` reserved forever for backward-compatible legacy decode.
- Use `_payload` fallback for reserved keys and Mongo-awkward field names.
- Keep migration writes through storage encoding and handle conflicts by
re-reading, skipping already-migrated docs, or retrying.
#### Impact on Phases
- Phase 1: add tests for payload key `value`, null/`.`/`$` field names, and
migration conflict setup.
- Phase 2: update reserved-field and unsafe-field encoder rules.
- Phase 3: add conflict retry semantics to the migration command.
### Whole-Plan Consistency Sweep
- Files reread: `plan.md`, all four `phase-*.md` files.
- Decision deltas checked: 3 (`value` reserved, unsafe field fallback, migration conflict retry).
- Reconciled stale references: 6.
- Unresolved contradictions: 0.
## Open Questions
None.
@@ -0,0 +1,107 @@
---
phase: 1
title: "Typed DocStore contract + memory backend"
status: done
priority: P2
dependencies: []
effort: "M"
---
# Phase 1: Typed DocStore contract + memory backend
## Overview
Define the new typed storage contract and a memory implementation first, with
tests, before touching Mongo or any module. This nails the interface every
module and the Mongo backend will target.
## Requirements
- `DocStore[T]` generic interface: `Get`, `Put`, `PutVersioned`, `Delete`, `List`.
- `Provider.Collection(module)` returns an opaque per-module `Collection`.
- `Typed[T](Collection) DocStore[T]` free function (methods cannot be generic).
- Memory implementation: hermetic, concurrency-safe, version-CAS correct.
- Reserved-field name guard: payload BSON tags must not collide with `_id`,
`version`, `updatedAt`.
- `ErrNotFound`, `ErrConflict` reused (move to this package if `kv_store.go` is deleted later).
## Architecture
```go
// doc_store.go
type DocStore[T any] interface {
Get(ctx, id) (T, int64, error)
Put(ctx, id, T) error
PutVersioned(ctx, id, expectedVersion int64, T) error
Delete(ctx, id) error
List(ctx, prefix string) ([]string, error)
}
type Provider interface{ Collection(module string) Collection }
type Collection interface{ collection() } // sealed marker
func Typed[T any](c Collection) DocStore[T] // switch concrete type → mongo|memory store
```
Memory backend:
```go
// memory_provider.go (replaces kv_provider.go's MemoryProvider)
type MemoryProvider struct{ mu sync.Mutex; cols map[string]*memoryCollection }
type memoryCollection struct{ mu sync.Mutex; rows map[string]memoryRow }
type memoryRow struct{ version int64; val any } // val holds the typed value
type MemoryDocStore[T any] struct{ c *memoryCollection }
```
- `Get`: type-assert `row.val.(T)`; missing → `ErrNotFound`.
- `Put`: overwrite, `version++`.
- `PutVersioned`: compare stored version to expected; mismatch → `ErrConflict`;
`expected==0` requires absent/zero-version row.
- `Delete`: idempotent.
- `List(prefix)`: sorted keys with prefix (reuse `validatePrefix`).
Keys: keep `validateKey`/`validatePrefix` from `keys.go`/`prefix.go`.
## Related Code Files
- Create: `internal/storage/doc_store.go` (interface, Provider, Collection, `Typed`, errors).
- Create: `internal/storage/memory_provider.go` (MemoryProvider + memoryCollection + MemoryDocStore).
- Create: `internal/storage/memory_doc_store_test.go`.
- Create: `internal/storage/reserved_fields.go` + test (reserved-name guard helper).
- Read: `internal/storage/keys.go`, `prefix.go`, `kv_store.go` (for error vars to migrate).
## Tests Before
Write failing tests first:
1. `TestMemoryDocStore_PutGetRoundTrip` — typed struct round-trips; version starts at 1.
2. `TestMemoryDocStore_PutBumpsVersion` — repeated Put increments version.
3. `TestMemoryDocStore_PutVersioned_CAS` — stale expected → `ErrConflict`; fresh succeeds.
4. `TestMemoryDocStore_PutVersioned_CreateZero` — `expected==0` creates; second create → `ErrConflict`.
5. `TestMemoryDocStore_Delete_Idempotent`.
6. `TestMemoryDocStore_List_Prefix` — disjoint key prefixes isolate types in one collection.
7. `TestReservedFields_RejectsCollision` — a payload type tagged `version` is flagged.
8. `TestMemoryProvider_CollectionIsolation` — two modules' collections don't share keys.
## Implementation Steps
1. Add `doc_store.go` with interface, sealed `Collection`, `Typed[T]`, and the
`ErrNotFound`/`ErrConflict` vars (kept here so `kv_store.go` can be deleted).
2. Implement memory provider + store.
3. Implement reserved-field reflection guard (`bson` tag scan).
4. Make all Phase 1 tests pass.
## Success Criteria
- [ ] `DocStore[T]`, `Provider`, `Collection`, `Typed` compile and are documented.
- [ ] Memory store passes CAS/version/list/delete tests.
- [ ] Reserved-name guard catches `_id`/`version`/`updatedAt` collisions.
- [ ] No dependency on the soon-to-be-deleted byte `KVStore`.
## Risk Assessment
- Risk: generic free-function `Typed[T]` is unusual. Mitigation: document why
(Go methods can't be generic); single switch point, well tested.
- Risk: memory `val any` type mismatch panics. Mitigation: `Get` returns a typed
error on failed assertion, not a panic.
@@ -0,0 +1,116 @@
---
phase: 2
title: "Mongo typed store + integration tests"
status: done
priority: P2
dependencies: [1]
effort: "L"
---
# Phase 2: Mongo typed store + integration tests
## Overview
Implement `MongoDocStore[T]` and a `MongoProvider` that satisfies the Phase 1
contract, persisting typed payloads as flattened native root documents. Lock the
raw shape and CAS semantics with Mongo integration tests.
## Requirements
- `MongoProvider.Collection(module)` returns a Mongo-backed `Collection`.
- `Typed[T]` over a Mongo collection yields `MongoDocStore[T]`.
- Write shape: `{ _id, ...payload, version, updatedAt(Date) }`; no `value`, no `_payload`.
- Whole-document `ReplaceOne` (no `$set`) so overwrites drop stale fields.
- Version CAS identical in behavior to the current `MongoKVStore`.
- `updatedAt` stored as `time.Time`.
- Plain `Put` bumps version via a bounded versioned-write retry loop.
## Architecture
```go
type storedDoc[T any] struct {
ID string `bson:"_id"`
Version int64 `bson:"version"`
UpdatedAt time.Time `bson:"updatedAt"`
Payload T `bson:",inline"`
}
type MongoDocStore[T any] struct{ coll *mongo.Collection }
```
- `Get`: `FindOne({_id})` → decode `storedDoc[T]`; `ErrNoDocuments`→`ErrNotFound`;
return `Payload`, `Version`.
- `PutVersioned(expected>0)`: `ReplaceOne({_id, version:expected}, storedDoc{version:expected+1, now, payload})`;
`MatchedCount==0`→`ErrConflict`.
- `PutVersioned(0)`: upsert filter `{_id, $or:[version absent, version:0]}` with
replacement `version:1`; duplicate-key→`ErrConflict`. (Port existing logic.)
- `Put`: loop `Get`→`PutVersioned(version)` up to N attempts (reuse the small
retry constant style already in coin/gold); first write uses `expected=0`.
- `Delete`: `DeleteOne` idempotent.
- `List(prefix)`: half-open `_id` range scan via `prefixSuccessor`, project `_id`
only. (Port from current `MongoKVStore.List`.)
- Error messages name module + key + reason.
`MongoProvider` wraps `*mongo.Database`; `Collection(module)` validates the
module name (reuse existing validation) and returns `mongoCollection{db.Collection(module)}`.
## Related Code Files
- Create: `internal/storage/mongo_doc_store.go` (MongoDocStore + storedDoc).
- Modify: `internal/storage/mongodb_provider.go` (implement new `Provider`/`Collection`).
- Create: `internal/storage/mongo_doc_store_test.go` (integration, gated by `MONGODB_TEST_URL`).
- Delete (this phase): `mongodb_kv.go`, `mongodb_value_codec.go`, `mongodb_kv_test.go`.
- Read: current `mongodb_kv.go` (port CAS + List), `mongodb_provider.go`.
## Tests Before
Gated by `MONGODB_TEST_URL`:
1. `TestMongoDocStore_RootShape` — Put a portfolio-like struct; raw `bson.M` has
root payload fields, `version`, `updatedAt`; **no** `value`, **no** `_payload`;
`updatedAt` decodes as `time.Time`.
2. `TestMongoDocStore_OverwriteRemovesStaleFields` — write struct A with field
set X, overwrite with struct B lacking some of X; raw doc has no stale field.
3. `TestMongoDocStore_PutVersioned_ConcurrentCreate` — N goroutines create same
`_id`; exactly one wins, rest get `ErrConflict`.
4. `TestMongoDocStore_PutVersioned_Update` — stale expected → `ErrConflict`.
5. `TestMongoDocStore_Put_BumpsVersion` — plain Put increments version, survives
a concurrent bump (retry loop succeeds or conflicts deterministically).
6. `TestMongoDocStore_List_Prefix`.
7. `TestMongoDocStore_WrappedScalar` — a `struct{ Date string }` payload stores
`date` at root (proves the lolschedule wrapping works).
8. `TestMongoDocStore_WrappedArray` — a `struct{ Subscribers []sub }` payload
stores `subscribers` array at root.
## Implementation Steps
1. Implement `MongoDocStore[T]` (port CAS + List from `MongoKVStore`, typed).
2. Reshape `MongoProvider` to the new `Provider`/`Collection`; wire `Typed[T]`.
3. Delete `mongodb_kv.go`, `mongodb_value_codec.go` and their tests.
4. `go build ./internal/storage` — expect module/server breakage deferred to
Phase 3 (storage package itself must compile).
5. Make Phase 2 integration tests pass against `make mongo-local`.
## Tests After
```sh
go test ./internal/storage
MONGODB_TEST_URL=mongodb://127.0.0.1:27017 go test ./internal/storage -run 'MongoDocStore'
```
## Success Criteria
- [ ] Raw Mongo doc is flattened root payload + `version` + Date `updatedAt`; no `value`/`_payload`.
- [ ] Overwrite removes stale fields.
- [ ] Concurrent create/update keep single-winner CAS.
- [ ] Wrapped scalar/array payloads store as named root fields.
- [ ] Old `MongoKVStore`/codec deleted; storage package builds.
## Risk Assessment
- Risk: `bson:",inline"` edge cases (pointer payloads, embedded maps). Mitigation:
require payload `T` to be a concrete struct; test scalar/array wrappers.
- Risk: bounded `Put` loop contention. Mitigation: small retry count; CAS users
call `PutVersioned` directly anyway.
- Risk: driver decodes `version` as int32. Mitigation: `storedDoc.Version int64`
+ decode test.
@@ -0,0 +1,115 @@
---
phase: 3
title: "Migrate modules + wiring; delete KVStore"
status: done
priority: P2
dependencies: [2]
effort: "XL"
---
# Phase 3: Migrate modules + wiring; delete KVStore
## Overview
Switch every module from the byte `KVStore` to typed `DocStore[T]`, rewire
`Deps`/registry/server, and delete the old byte abstraction and DynamoDB runtime
backend. This is the largest phase; do it module-by-module with tests green
after each.
## Requirements
- `Deps.KV storage.KVStore` → `Deps.Store storage.Collection`.
- `registry.Build` takes `storage.Provider` instead of `storage.KVProvider`.
- Each module builds typed stores via `storage.Typed[T](deps.Store)`.
- CAS behavior preserved for coin, gold, lolschedule last-push.
- `lolschedule` subscribers + last-push wrapped in named structs.
- No remaining references to `KVStore`, `VersionedStore`, `PutJSON`, `GetJSON`,
`kv.Put`/`kv.Get` byte calls, or the DynamoDB runtime backend.
- `cmd/server/main.go` `buildProvider` returns a Mongo `Provider` (or memory for
`MODULES=`/no-`MONGO_URL` local runs); the `dynamodb` runtime branch is removed.
## Per-module migration map
| Module | Keys / payload types | Store ops | Notes |
|---|---|---|---|
| coin | `Portfolio` (per user) | Get/PutVersioned (CAS loop) | typed CAS via `DocStore[Portfolio]` |
| gold | `Portfolio` (per user) | Get/PutVersioned (CAS loop) | same |
| stock | `Portfolio` (per user) | Get/Put | no CAS today; keep plain Put |
| wordle | `GameState`, `Stats` | Get/Put | two typed views, disjoint key prefixes |
| loldle | `gameState`, `stats`, `roundConfig` | Get/Put/Delete | three typed views |
| twentyq | `GameState`, `Stats` | Get/Put/Delete | two typed views |
| lolschedule | `subscribers` (wrap `[]Subscriber`), `lastPush` (wrap date string) | List/Put + CAS on last-push | **wrap array+scalar in named structs** |
| stats | counter entry struct(s) | Get/Put/List | List-heavy (`views.go`) |
| misc | `lastPing` struct | Get/Put | |
| util | none | — | no storage |
`lolschedule` wrappers:
```go
type subscribersDoc struct { Subscribers []Subscriber `bson:"subscribers" json:"subscribers"` }
type lastPushDoc struct { Date string `bson:"date" json:"date"` }
```
`claimDailyPush` uses `DocStore[lastPushDoc].Get`/`PutVersioned` for the
single-winner daily claim (replaces the current `VersionedStore` byte path).
## Related Code Files
- Modify: `internal/modules/module.go` (Deps.Store), `registry.go` (Build signature, `Collection(name)`).
- Modify: every module's storage-touching file (see map) + their factories
(`coin.go`, `gold.go`, … `New(deps)` build typed stores).
- Modify: `cmd/server/main.go` `buildProvider` (Mongo|memory only).
- Modify: all module `*_test.go` that construct stores (use `MemoryProvider`/`Typed`).
- Delete: `kv_store.go`, `kv_provider.go`, `memory_kv.go`, `memory_kv_test.go`,
`dynamodb_kv.go`, `dynamodb_provider.go`, `dynamodb_kv_test.go`,
`dynamodb_provider_test.go`, `invalid_store.go`.
## Tests Before / After (per module)
For each module: update its tests to build a `MemoryProvider`, get
`deps.Store = provider.Collection(name)`, then run existing behavior assertions.
Run `go test ./internal/modules/<m>/...` green before moving on. Order:
misc → stock → wordle → loldle → twentyq → coin → gold → stats → lolschedule
(simplest first; CAS + List modules last).
After all modules:
```sh
go build ./...
go test ./internal/modules/...
grep -rn "KVStore\|VersionedStore\|PutJSON\|GetJSON\|\.For(" internal/ cmd/ # expect: none (except migrator handled in Phase 4)
```
## Implementation Steps
1. Add `Deps.Store storage.Collection`; change `registry.Build` to take
`storage.Provider` and set `Deps.Store = provider.Collection(name)`.
2. Migrate modules one at a time per the order above; keep the suite green.
3. For coin/gold, replace the `kv.(VersionedStore)` assertion + byte CAS loop
with `DocStore[Portfolio]` Get/PutVersioned (same retry count).
4. For lolschedule, introduce the wrapper structs and migrate subscribers (List +
Put) and last-push (CAS).
5. Update `cmd/server/main.go`: drop the `dynamodb` runtime branch; auto-detect =
Mongo when `MONGO_URL` set, else memory (warn). Keep secret-safe logging.
6. Delete the dead files listed above; ensure `go build ./...` is clean.
7. Run full module + storage suites.
## Success Criteria
- [ ] All modules compile and pass tests using `DocStore[T]`.
- [ ] No `KVStore`/`VersionedStore`/`PutJSON`/`GetJSON` references outside the migrator.
- [ ] DynamoDB runtime backend deleted; `dynamodb_client.go` retained for migrator.
- [ ] coin/gold/lolschedule CAS behavior unchanged (tests prove single-winner).
- [ ] lolschedule subscribers/last-push persist as named root fields.
- [ ] `cmd/server` builds a Mongo-only runtime (memory fallback for local/no-DB).
## Risk Assessment
- Risk: large blast radius. Mitigation: strict module-by-module order, suite
green between each; no behavior changes, only the store type.
- Risk: a module silently relied on byte-identical round-trip. Mitigation: typed
structs already define the JSON contract; assert decoded values, not bytes.
- Risk: stats List semantics differ. Mitigation: `List(prefix)` ported verbatim;
reuse stats' existing key prefixes.
- Risk: cron Deps scoping regresses. Mitigation: keep registry's per-module Deps
cloning; `Deps.Store` is already per-module.
@@ -0,0 +1,115 @@
---
phase: 4
title: "DynamoDB→Mongo migrator + docs"
status: done
priority: P2
dependencies: [3]
effort: "M"
---
# Phase 4: DynamoDB→Mongo migrator + docs
## Overview
Keep `cmd/migrate-dynamo-to-mongo` working, but make it write the new flattened
native shape directly. The migrator is schema-agnostic over modules, so it
cannot use the typed `DocStore[T]` (it has no compile-time `T` per row). It
writes documents with a small generic JSON→root-document encoder plus a tiny
explicit table for the only two non-object values.
## Requirements
- Migrator still Scans DynamoDB (read-only, `dynamodb:Scan` only) — unchanged.
- It writes `{ _id: sk, ...payloadFields, version:1, updatedAt(Date) }` per item.
- JSON object values flatten to root.
- The two known non-object DynamoDB values are wrapped to match the module's
typed shape:
- `lolschedule` subscribers (JSON array) → `{ subscribers: [...] }`
- `lolschedule` last-push date (JSON/scalar string) → `{ date: "..." }`
- Idempotent (re-run overwrites by `_id`, never duplicates).
- `--dry-run` (counts only) and `--verify` (per-collection count parity) kept.
- No secrets read/printed; `MONGO_URL` stays secret.
## Architecture
Add a migrator-local encoder (NOT in `internal/storage`, to keep the runtime
typed and the generic JSON path confined to migration):
```go
// cmd/migrate-dynamo-to-mongo/encode.go
func rootDocForItem(module, key string, value []byte, now time.Time) (bson.M, error)
```
- Decode `value` with `json.Decoder.UseNumber` (int64 fidelity).
- If a `(module,key-prefix)` wrap rule matches → `{ wrapField: decoded }`.
- Else if decoded is a JSON object → spread its fields at root (reject keys
colliding with `_id`/`version`/`updatedAt`; none expected — fail loud if seen).
- Else → error (an unexpected non-object value means a new wrap rule is needed;
fail loud rather than guess).
- Always set `_id`, `version:int64(1)`, `updatedAt: now`.
Wrap rules table (explicit, documented):
```go
var wrapRules = []struct{ module, keyPrefix, field string }{
{"lolschedule", "subscribers", "subscribers"},
{"lolschedule", "last_push_date", "date"},
}
```
Write with `Collection(module).ReplaceOne({_id:key}, doc, upsert=true)` directly
via the driver (the migrator already holds `*mongo.Database`).
## Related Code Files
- Modify: `cmd/migrate-dynamo-to-mongo/main.go` — replace `provider.For(pk).Put`
with `rootDocForItem` + `ReplaceOne` upsert; validate module/key names inline
(the old `For` returned an erroring store for bad names — replicate that guard).
- Create: `cmd/migrate-dynamo-to-mongo/encode.go` + `encode_test.go`.
- Modify: `cmd/migrate-dynamo-to-mongo/main_test.go` — assert flattened shape +
wrapped lolschedule docs.
- Modify: `cmd/migrate-dynamo-to-mongo/README.md` — new destination shape.
- Modify: `README.md` (storage layout summary), `docs/deploy-coolify-selfhosted.md`
(Atlas layout + cutover note), `Makefile` if migrator target wording changed.
## Tests Before
1. `TestRootDocForItem_Object` — object JSON → root fields + meta; no `value`.
2. `TestRootDocForItem_Int64Fidelity` — large integral number stays int64.
3. `TestRootDocForItem_LolscheduleSubscribers` — array → `{subscribers:[...]}`.
4. `TestRootDocForItem_LolscheduleLastPush` — date string → `{date:"..."}`.
5. `TestRootDocForItem_UnknownScalar_Errors` — bare scalar w/o wrap rule fails loud.
6. Migrator e2e (`MONGODB_TEST_URL` + DynamoDB Local): seed DynamoDB rows for a
few modules incl. lolschedule array+scalar → migrate → assert raw Mongo docs
are flattened/wrapped, no `value`; re-run idempotent; `--verify` counts match;
values readable by the module's `DocStore[T]`.
## Implementation Steps
1. Implement `rootDocForItem` + wrap rules.
2. Rewrite `runMigrate` to encode + `ReplaceOne` upsert; keep `--dry-run`.
3. Keep `runVerify` (count parity) — unchanged logic.
4. Add inline module/key validation (reuse `storage.ValidateKey` if exported, or
replicate the rule) so bad rows fail loud.
5. Make migrator tests pass.
6. Update README/deploy docs to the flattened layout; remove BSON-string/binary
destination wording.
## Success Criteria
- [ ] Migrator writes flattened native docs (no `value`) for all modules.
- [ ] lolschedule array + scalar land as `subscribers`/`date` root fields.
- [ ] Migrated docs are read back correctly by the typed module stores.
- [ ] Idempotent; `--dry-run` writes nothing; `--verify` exits non-zero on mismatch.
- [ ] Unknown non-object value fails loud (no silent guess).
- [ ] README + Coolify deploy docs show the flattened layout.
## Risk Assessment
- Risk: a module other than lolschedule stores a non-object (missed in survey).
Mitigation: encoder fails loud on un-wrapped scalars/arrays; e2e covers the
known set; the error names module/key so a new wrap rule is a one-liner.
- Risk: re-introducing a generic JSON codec invites runtime reuse. Mitigation:
it lives only in `cmd/migrate-dynamo-to-mongo`, not `internal/storage`.
- Risk: DynamoDB source already gone at run time. Mitigation: migrator is the
cutover step; if DynamoDB is empty/absent the operator skips it (documented).
@@ -0,0 +1,89 @@
---
phase: 5
title: "Verification + Mongo-only rollout"
status: done
priority: P2
dependencies: [4]
effort: "M"
---
# Phase 5: Verification + Mongo-only rollout
## Overview
Full validation and a simple cutover. Because Atlas is empty until cutover and
there is no legacy Mongo data, rollout is one-way and clean: migrate from
DynamoDB, then run Mongo-only. No in-place Mongo migration, no dual-read.
## Requirements
- All test gates pass (`make vet`, `make test`, Mongo integration, migrator e2e).
- Raw Mongo docs verified flattened for representative modules.
- App behavior unchanged for all Telegram commands.
- Cutover + rollback notes recorded.
## Validation gates
```sh
make vet
make test
MONGODB_TEST_URL=mongodb://127.0.0.1:27017 go test ./internal/storage ./cmd/migrate-dynamo-to-mongo
# migrator e2e additionally needs DynamoDB Local (DYNAMODB_LOCAL_URL)
```
Manual raw-doc inspection (Atlas/Compass or a small test assertion) for:
- coin/gold/stock portfolio: root fields, `version`, Date `updatedAt`, no `value`.
- wordle/loldle/twentyq game + stats: root fields.
- lolschedule: `subscribers` array field; last-push `date` field.
- stats counters: root fields + List works.
## Cutover order (operator)
1. Backup/export current DynamoDB table (source of truth) before cutover.
2. Disable EventBridge / stop the old AWS app so DynamoDB writes stop.
3. `migrate-dynamo-to-mongo --dry-run` → review per-module counts.
4. `migrate-dynamo-to-mongo` → writes flattened native docs to Atlas.
5. `migrate-dynamo-to-mongo --verify` → per-collection count parity (exit 0).
6. Start the Mongo-only container (`MONGO_URL`+`MONGO_DATABASE`); confirm health.
7. Spot-check a few commands (coin balance, wordle state, lolschedule subscribe).
## Rollback
- Before step 6 (no Mongo writes yet): re-enable the old AWS app on DynamoDB.
DynamoDB is untouched (migrator is Scan-only), so this is safe.
- After step 6 (Mongo has taken live writes): forward-fix preferred. To revert to
DynamoDB you would need a reverse Mongo→DynamoDB export of anything written
after cutover. Keep the DynamoDB backup from step 1 as the floor.
## Related Code Files
- Read: `Makefile`, `README.md`, `docs/deploy-coolify-selfhosted.md`,
`docs/aws-decommission-runbook.md`.
- Modify: tests only if verification finds gaps.
## Implementation Steps
1. Run all gates above; fix regressions (do not weaken tests).
2. Run the migrator e2e against local Mongo + DynamoDB Local.
3. Inspect raw docs per the list; optionally add a small assertion test.
4. Record the cutover commands + rollback constraints in the completion report
under `plans/reports/`.
5. Grep docs for stale `{ _id, value, version, updatedAt }` wording; fix.
## Success Criteria
- [ ] `make vet`, `make test` pass.
- [ ] Mongo integration + migrator e2e pass.
- [ ] Raw docs verified flattened (no `value`/`_payload`) for representative modules.
- [ ] No Telegram command behavior change.
- [ ] Cutover + rollback documented in the completion report.
## Risk Assessment
- Risk: a module's data didn't round-trip through migration. Mitigation: e2e
reads migrated docs back through the typed store; spot-check live commands.
- Risk: Atlas M0 limits during migrate. Mitigation: tiny dataset; collection-by-
collection counts in `--verify`.
- Risk: one-way cutover. Mitigation: DynamoDB backup + Scan-only migrator means
pre-write rollback is safe; post-write rollback documented as forward-fix.
Loaded 100 of 104 files, more files were not shown because too many files have changed in this diff. Show more