mirror of
https://github.com/tiennm99/tiennm99bot.git
synced 2026-10-17 04:15:25 +00:00
refactor(storage): replace KVStore with generic typed DocStore[T]
Delete the byte-oriented KVStore/VersionedStore abstraction and the DynamoDB/memory KV backends. Add a generic typed store (DocStore[T] with Provider/Collection/Typed) persisting each value as a flattened native Mongo document (storedDoc[T] via bson inline) — no value envelope. - MongoDB is the only runtime backend; memory kept for tests/local. - All modules + deploynotify use typed stores; persisted structs carry bson tags == json names (incl. nested lolschedule/wordle types). - lolschedule wraps its array/scalar values in named structs. - migrate-dynamo-to-mongo writes the flattened shape via Typed[bson.M] with wrap rules; Scan/--dry-run/--verify retained. Verified: go vet/build clean; full go test green hermetically and in-container vs real Mongo 7 + DynamoDB Local (storage integration + migrator e2e).
This commit is contained in:
1 parent
b4910c2289
commit
15a3239ab1
104 files changed
+4036
-2712
No files matched your search
@@ -31,9 +31,11 @@ test: ## Unit tests (no emulator required)
|
||||
# Run DynamoDB integration tests against DynamoDB Local.
|
||||
# Override DDB_PORT if 8001 is taken on your host.
|
||||
DDB_PORT ?= 8001
|
||||
test-dynamodb: dynamodb-local ## Run DynamoDB tests against DynamoDB Local
|
||||
DYNAMODB_LOCAL_URL=http://localhost:$(DDB_PORT) LOG_LEVEL=error \
|
||||
go test -race -count=1 ./internal/storage/...
|
||||
test-dynamodb: dynamodb-local mongo-local ## Run the DynamoDB→Mongo migrator e2e against local emulators
|
||||
DYNAMODB_LOCAL_URL=http://localhost:$(DDB_PORT) \
|
||||
MONGODB_TEST_URL=mongodb://127.0.0.1:$(MONGO_PORT) \
|
||||
MONGO_DATABASE=migrate_test LOG_LEVEL=error \
|
||||
go test -race -count=1 ./cmd/migrate-dynamo-to-mongo/...
|
||||
|
||||
# Run MongoDB integration tests against a local Mongo container.
|
||||
# Override MONGO_PORT if 27017 is taken on your host.
|
||||
|
||||
@@ -28,7 +28,7 @@ internal/server/ HTTP routes (/ health, /cron/{name} manual trigger)
|
||||
internal/telegram/ Telegram long-polling bot wrapper
|
||||
internal/cron/ in-process cron scheduler (replaces EventBridge)
|
||||
internal/modules/ Module framework, registry, dispatchers, modules
|
||||
internal/storage/ KVProvider interface; memory + dynamodb + mongodb (values stored as native BSON documents)
|
||||
internal/storage/ typed DocStore[T] (Provider + Typed); mongodb runtime + memory (tests). Values persist as flattened native BSON root documents
|
||||
internal/ai/ Gemini client (used by twentyq)
|
||||
docker-compose.yml Coolify self-host stack (single bot service)
|
||||
docs/deploy-coolify-selfhosted.md Self-host onboarding + cutover runbook
|
||||
@@ -58,12 +58,11 @@ MONGO_DATABASE=miti99bot_dev \
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
For storage integration tests (each skips when its emulator env var is unset):
|
||||
For integration tests (each skips when its emulator env var is unset):
|
||||
```sh
|
||||
make dynamodb-local # docker run amazon/dynamodb-local on :8001
|
||||
make test-dynamodb # internal/storage tests against DynamoDB Local
|
||||
make mongo-local # docker run mongo:7 on :27017
|
||||
make test-mongo # internal/storage tests against local MongoDB
|
||||
make test-mongo # internal/storage typed-store tests against local MongoDB
|
||||
make test-dynamodb # DynamoDB→Mongo migrator e2e against DynamoDB Local + local Mongo
|
||||
```
|
||||
|
||||
## Test
|
||||
@@ -71,7 +70,8 @@ make test-mongo # internal/storage tests against local MongoDB
|
||||
```sh
|
||||
make vet # go vet
|
||||
make test # full unit suite (no emulator)
|
||||
make test-dynamodb # storage tests against DynamoDB Local (requires Docker)
|
||||
make test-mongo # typed-store integration tests against local Mongo (requires Docker)
|
||||
make test-dynamodb # migrator e2e against DynamoDB Local + Mongo (requires Docker)
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
@@ -7,20 +7,25 @@ app writes, then verifies per-module parity. Idempotent and re-runnable.
|
||||
## What it does
|
||||
|
||||
- Full-table `Scan` of DynamoDB (the table is a small KV) → group by `pk`.
|
||||
- Writes each item through `storage.MongoKVStore.Put`, so the `value` encoding
|
||||
is **byte-identical** to what the running bot writes (BSON binary). This is
|
||||
what keeps re-runs idempotent and future CompareAndSwap correct — a raw
|
||||
`UpdateOne` that stored `value` as a BSON string would silently break both.
|
||||
- `Put` runs `validateKey` on each sort key and upserts by `_id`, so a re-run
|
||||
produces no duplicates and an invalid key fails loud (no key the app's read
|
||||
path can't load is ever written).
|
||||
- Writes each item through the typed Mongo store (`storage.Typed[bson.M]`) as a
|
||||
**flattened native document** — the value's JSON fields are hoisted to the
|
||||
document root alongside `_id`/`version`/`updatedAt`, with no `value` envelope.
|
||||
This is the exact shape the running bot writes, so the app reads migrated docs
|
||||
directly. Integers keep int64 fidelity (decoded with `UseNumber`).
|
||||
- The two non-object values are wrapped into named root fields to match the
|
||||
module's typed shape: lolschedule `subscribers` (a JSON array) → `{subscribers: [...]}`
|
||||
and `daily_push:last_date` (a bare date string) → `{date: "..."}`. Any other
|
||||
non-object value fails loud so a missing wrap rule is obvious (see `encode.go`).
|
||||
- Writing through the store validates the module/collection name and key and
|
||||
upserts by `_id`, so a re-run produces no duplicates and bad input fails loud.
|
||||
|
||||
| DynamoDB | MongoDB |
|
||||
|---|---|
|
||||
| `pk` (module name) | collection name |
|
||||
| `sk` (user key) | document `_id` |
|
||||
| `value` (JSON string) | `value` field (BSON binary, same bytes) |
|
||||
| `updatedAt` | restamped to migration time (write-only field; nothing reads it) |
|
||||
| `value` (JSON object) | payload fields hoisted to the document root (no `value` field) |
|
||||
| `value` (array/scalar, lolschedule) | wrapped in a named root field (`subscribers` / `date`) |
|
||||
| — | `version` = 1, `updatedAt` = migration time (write-only; nothing reads it) |
|
||||
|
||||
## Usage
|
||||
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
)
|
||||
|
||||
// reservedRootFields are owned by the storage layer (see internal/storage); a
|
||||
// flattened object payload must not collide with them.
|
||||
var reservedRootFields = map[string]bool{"_id": true, "version": true, "updatedAt": true}
|
||||
|
||||
// wrapRule names the root field a non-object DynamoDB value must be wrapped in
|
||||
// to match the typed Mongo store's named-struct shape. rawString treats the
|
||||
// DynamoDB value as a bare (non-JSON) string; otherwise it is JSON-decoded.
|
||||
type wrapRule struct {
|
||||
field string
|
||||
rawString bool
|
||||
}
|
||||
|
||||
// wrapRules covers the only KV entries whose value is not a JSON object. They
|
||||
// mirror the named-struct wrappers the lolschedule module persists:
|
||||
// - subscribers: a JSON array → {subscribers: [...]}
|
||||
// - daily_push:last_date: a bare date string → {date: "..."}
|
||||
//
|
||||
// Any other non-object value fails loud in payloadForItem so a missing rule is
|
||||
// obvious rather than silently dropped.
|
||||
var wrapRules = map[[2]string]wrapRule{
|
||||
{"lolschedule", "subscribers"}: {field: "subscribers"},
|
||||
{"lolschedule", "daily_push:last_date"}: {field: "date", rawString: true},
|
||||
}
|
||||
|
||||
// payloadForItem converts one migrated KV row's value into the flattened payload
|
||||
// map the typed Mongo store stores at the document root. The store adds _id,
|
||||
// version, and updatedAt; this returns only the payload fields.
|
||||
func payloadForItem(module, key string, value []byte) (bson.M, error) {
|
||||
if rule, ok := wrapRules[[2]string{module, key}]; ok {
|
||||
if rule.rawString {
|
||||
return bson.M{rule.field: string(value)}, nil
|
||||
}
|
||||
decoded, err := decodeJSONNumber(value)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s/%s: decode value: %w", module, key, err)
|
||||
}
|
||||
return bson.M{rule.field: decoded}, nil
|
||||
}
|
||||
|
||||
decoded, err := decodeJSONNumber(value)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s/%s: decode value: %w", module, key, err)
|
||||
}
|
||||
obj, ok := decoded.(bson.M)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s/%s: value is not a JSON object (type %T) and has no wrap rule — add one to wrapRules", module, key, decoded)
|
||||
}
|
||||
for k := range obj {
|
||||
if reservedRootFields[k] {
|
||||
return nil, fmt.Errorf("%s/%s: payload key %q collides with a reserved root field — add a wrap rule", module, key, k)
|
||||
}
|
||||
}
|
||||
return obj, nil
|
||||
}
|
||||
|
||||
// decodeJSONNumber decodes JSON into a BSON-native value, preserving integral
|
||||
// numbers as int64 (UseNumber) so migrated numbers keep int64 fidelity, matching
|
||||
// the live store's value codec.
|
||||
func decodeJSONNumber(value []byte) (any, error) {
|
||||
dec := json.NewDecoder(bytes.NewReader(value))
|
||||
dec.UseNumber()
|
||||
var v any
|
||||
if err := dec.Decode(&v); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if dec.More() {
|
||||
return nil, fmt.Errorf("trailing data after JSON value")
|
||||
}
|
||||
return numberToBSON(v), nil
|
||||
}
|
||||
|
||||
// numberToBSON walks a json.Unmarshal(UseNumber) tree into bson.M/bson.A,
|
||||
// converting json.Number to int64 when integral, else float64.
|
||||
func numberToBSON(v any) any {
|
||||
switch t := v.(type) {
|
||||
case map[string]any:
|
||||
m := make(bson.M, len(t))
|
||||
for k, val := range t {
|
||||
m[k] = numberToBSON(val)
|
||||
}
|
||||
return m
|
||||
case []any:
|
||||
a := make(bson.A, len(t))
|
||||
for i, val := range t {
|
||||
a[i] = numberToBSON(val)
|
||||
}
|
||||
return a
|
||||
case json.Number:
|
||||
if i, err := t.Int64(); err == nil {
|
||||
return i
|
||||
}
|
||||
if f, err := t.Float64(); err == nil {
|
||||
return f
|
||||
}
|
||||
return t.String()
|
||||
default:
|
||||
return v
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
)
|
||||
|
||||
func TestPayloadForItem_Object(t *testing.T) {
|
||||
got, err := payloadForItem("coin", "user:1", []byte(`{"bal":100,"meta":{"createdAt":5}}`))
|
||||
if err != nil {
|
||||
t.Fatalf("payloadForItem: %v", err)
|
||||
}
|
||||
if got["bal"] != int64(100) {
|
||||
t.Errorf("bal = %v (%T), want int64(100)", got["bal"], got["bal"])
|
||||
}
|
||||
if _, ok := got["value"]; ok {
|
||||
t.Error("payload must not contain a 'value' envelope field")
|
||||
}
|
||||
meta, ok := got["meta"].(bson.M)
|
||||
if !ok || meta["createdAt"] != int64(5) {
|
||||
t.Errorf("nested meta = %v, want {createdAt: int64(5)}", got["meta"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestPayloadForItem_LolscheduleSubscribers(t *testing.T) {
|
||||
got, err := payloadForItem("lolschedule", "subscribers", []byte(`[{"chatId":1},{"chatId":2}]`))
|
||||
if err != nil {
|
||||
t.Fatalf("payloadForItem: %v", err)
|
||||
}
|
||||
arr, ok := got["subscribers"].(bson.A)
|
||||
if !ok || len(arr) != 2 {
|
||||
t.Fatalf("subscribers = %v (%T), want 2-element bson.A", got["subscribers"], got["subscribers"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestPayloadForItem_LolscheduleLastPushRawString(t *testing.T) {
|
||||
// last-push date is stored as a bare (non-JSON) string in DynamoDB.
|
||||
got, err := payloadForItem("lolschedule", "daily_push:last_date", []byte(`2026-06-28`))
|
||||
if err != nil {
|
||||
t.Fatalf("payloadForItem: %v", err)
|
||||
}
|
||||
if got["date"] != "2026-06-28" {
|
||||
t.Errorf("date = %v, want 2026-06-28", got["date"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestPayloadForItem_UnknownNonObjectFailsLoud(t *testing.T) {
|
||||
if _, err := payloadForItem("coin", "user:1", []byte(`"a-bare-string"`)); err == nil {
|
||||
t.Error("non-object value with no wrap rule must fail loud")
|
||||
}
|
||||
if _, err := payloadForItem("coin", "user:1", []byte(`[1,2,3]`)); err == nil {
|
||||
t.Error("array value with no wrap rule must fail loud")
|
||||
}
|
||||
}
|
||||
|
||||
func TestPayloadForItem_ReservedKeyCollision(t *testing.T) {
|
||||
if _, err := payloadForItem("coin", "user:1", []byte(`{"version":7}`)); err == nil {
|
||||
t.Error("payload key colliding with reserved root field must fail loud")
|
||||
}
|
||||
}
|
||||
|
||||
// TestPayloadForItem_CamelCaseFidelity guards the invariant that lets migration
|
||||
// work: the migrator preserves the original (camelCase) JSON keys, so a typed
|
||||
// store struct must declare bson tags matching those names. A struct whose bson
|
||||
// tags drifted to the driver's lowercased default would read these back empty.
|
||||
func TestPayloadForItem_CamelCaseFidelity(t *testing.T) {
|
||||
payload, err := payloadForItem("lolschedule", "events", []byte(`{"startTime":"t","gameWins":3,"blockName":"Week 1"}`))
|
||||
if err != nil {
|
||||
t.Fatalf("payloadForItem: %v", err)
|
||||
}
|
||||
raw, err := bson.Marshal(payload)
|
||||
if err != nil {
|
||||
t.Fatalf("bson.Marshal: %v", err)
|
||||
}
|
||||
var got struct {
|
||||
StartTime string `bson:"startTime"`
|
||||
GameWins int `bson:"gameWins"`
|
||||
BlockName string `bson:"blockName"`
|
||||
}
|
||||
if err := bson.Unmarshal(raw, &got); err != nil {
|
||||
t.Fatalf("bson.Unmarshal: %v", err)
|
||||
}
|
||||
if got.StartTime != "t" || got.GameWins != 3 || got.BlockName != "Week 1" {
|
||||
t.Fatalf("camelCase round-trip lost data: %+v", got)
|
||||
}
|
||||
}
|
||||
@@ -12,16 +12,18 @@
|
||||
// exactly `dynamodb:Scan` on the table ARN (nothing else, no write actions).
|
||||
// For local testing, set DYNAMODB_LOCAL_URL to point at DynamoDB Local.
|
||||
//
|
||||
// - default: scan + write every item through MongoKVStore.Put (same value
|
||||
// encoding as the live app — JSON objects stored as native BSON), then
|
||||
// report per-module counts.
|
||||
// - default: scan + write every item through the typed Mongo store as a
|
||||
// flattened native document (payload fields hoisted to the root, no `value`
|
||||
// envelope) — the exact shape the live app writes — then report per-module
|
||||
// counts.
|
||||
// - --dry-run: scan + report counts, write nothing.
|
||||
// - --verify: tally DynamoDB per pk via Scan vs Mongo CountDocuments per
|
||||
// collection; print a table and exit non-zero on any mismatch.
|
||||
//
|
||||
// Note: writing through Put gives each doc the app's native-BSON value encoding
|
||||
// and a fresh updatedAt/version; nothing in the app reads updatedAt (write-only
|
||||
// today), and --verify compares counts, so this is intentional and harmless.
|
||||
// Note: each migrated doc gets a fresh updatedAt and version=1; nothing in the
|
||||
// app reads updatedAt, and --verify compares counts, so this is intentional and
|
||||
// harmless. The two non-object values (lolschedule subscribers array and
|
||||
// last-push date) are wrapped into named root fields (see encode.go).
|
||||
package main
|
||||
|
||||
import (
|
||||
@@ -142,11 +144,13 @@ func sortedKeys(m map[string]int) []string {
|
||||
}
|
||||
|
||||
// runMigrate scans the table, then (unless dry-run) writes every item through
|
||||
// MongoKVStore.Put so the value uses the same native-BSON encoding the live app
|
||||
// writes — guaranteeing idempotent re-runs and correct version-based CAS. Put
|
||||
// also runs validateKey on each sk and upserts by _id, so a re-run produces no
|
||||
// duplicates and an invalid key fails loud rather than writing data
|
||||
// the app's read path cannot load.
|
||||
// the typed Mongo store as a flattened native document (payload fields hoisted
|
||||
// to the root, no `value` envelope) — the exact shape the live app writes.
|
||||
// payloadForItem wraps the two non-object values (lolschedule subscribers array
|
||||
// and last-push date) into named-struct fields. Writing through
|
||||
// storage.Typed[bson.M] reuses the store's module/key validation and version
|
||||
// semantics, so a re-run produces no duplicates (upsert by _id) and an invalid
|
||||
// module name or key fails loud rather than writing data the app cannot load.
|
||||
func runMigrate(ctx context.Context, ddb *dynamodb.Client, mdb *mongo.Database, table string, dryRun bool) error {
|
||||
items, err := scanTable(ctx, ddb, table)
|
||||
if err != nil {
|
||||
@@ -160,9 +164,11 @@ func runMigrate(ctx context.Context, ddb *dynamodb.Client, mdb *mongo.Database,
|
||||
if dryRun {
|
||||
continue
|
||||
}
|
||||
// For invalid module names For returns a store that errors on Put;
|
||||
// Put validates the key. Either way an error aborts loudly.
|
||||
if err := provider.For(it.pk).Put(ctx, it.sk, it.value); err != nil {
|
||||
payload, err := payloadForItem(it.pk, it.sk, it.value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := storage.Typed[bson.M](provider.Collection(it.pk)).Put(ctx, it.sk, payload); err != nil {
|
||||
return fmt.Errorf("put %s/%s: %w", it.pk, it.sk, err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"github.com/aws/aws-sdk-go-v2/aws"
|
||||
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
|
||||
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
@@ -95,15 +96,16 @@ func TestMigrateAndVerify(t *testing.T) {
|
||||
t.Fatalf("runVerify: %v", err)
|
||||
}
|
||||
|
||||
// Spot-check byte-identical round trip through the app's read path.
|
||||
// Spot-check the migrated value round-trips through the typed store as a
|
||||
// flattened native doc: bal hoisted to the root, preserved as int64.
|
||||
db := mongoDatabase(t, ctx, mongoURL, mongoDB)
|
||||
provider := storage.NewMongoProvider(db)
|
||||
got, err := provider.For("coin").Get(ctx, "user:1")
|
||||
got, _, err := storage.Typed[bson.M](provider.Collection("coin")).Get(ctx, "user:1")
|
||||
if err != nil {
|
||||
t.Fatalf("Get migrated value: %v", err)
|
||||
}
|
||||
if string(got) != `{"bal":100}` {
|
||||
t.Errorf("migrated value = %q, want %q", got, `{"bal":100}`)
|
||||
if got["bal"] != int64(100) {
|
||||
t.Errorf("migrated value bal = %v (%T), want int64(100)", got["bal"], got["bal"])
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+10
-31
@@ -60,10 +60,6 @@ func factories() map[string]modules.Factory {
|
||||
}
|
||||
}
|
||||
|
||||
// dynamodbInitTimeout caps DynamoDB client construction at startup. Lambda
|
||||
// has a 10s init phase; we want to leave headroom for module wiring.
|
||||
const dynamodbInitTimeout = 5 * time.Second
|
||||
|
||||
// mongodbInitTimeout caps MongoDB connect+ping at startup (and Disconnect at
|
||||
// shutdown). Atlas SRV DNS + TLS handshake can take a couple seconds on a cold
|
||||
// container; 10s leaves headroom without hiding a wedged cluster.
|
||||
@@ -153,7 +149,7 @@ func main() {
|
||||
|
||||
deploynotify.Run(rootCtx, deploynotify.Config{
|
||||
Bot: b,
|
||||
KV: provider.For("deploynotify"),
|
||||
Store: deploynotify.NewStore(provider.Collection("deploynotify")),
|
||||
OwnerID: cfg.BotOwnerID,
|
||||
GitSHA: gitSHA,
|
||||
})
|
||||
@@ -210,16 +206,16 @@ func main() {
|
||||
}
|
||||
|
||||
// buildProvider picks the storage backend. Selection order:
|
||||
// 1. Explicit KV_PROVIDER env (memory|dynamodb|mongodb) wins.
|
||||
// 1. Explicit KV_PROVIDER env (memory|mongodb) wins.
|
||||
// 2. Auto-detect: MONGO_URL set → mongodb; otherwise memory.
|
||||
//
|
||||
// The self-host default is mongodb (just set MONGO_URL + MONGO_DATABASE — no
|
||||
// KV_PROVIDER needed). dynamodb remains reachable only via an explicit
|
||||
// KV_PROVIDER, kept for the data migrator and integration tests. The legacy
|
||||
// firestore backend and the AWS_LAMBDA_FUNCTION_NAME auto-detect are removed.
|
||||
// KV_PROVIDER needed). The memory backend is for tests and local no-database
|
||||
// runs (MODULES=). DynamoDB is no longer a runtime backend — it survives only
|
||||
// as the one-off migration source (cmd/migrate-dynamo-to-mongo).
|
||||
//
|
||||
// Returned closer is always non-nil and safe to call exactly once.
|
||||
func buildProvider(ctx context.Context, cfg config) (storage.KVProvider, func(), error) {
|
||||
func buildProvider(ctx context.Context, cfg config) (storage.Provider, func(), error) {
|
||||
backend := strings.ToLower(strings.TrimSpace(cfg.KVProvider))
|
||||
if backend == "" {
|
||||
if cfg.MongoURL != "" {
|
||||
@@ -261,25 +257,10 @@ func buildProvider(ctx context.Context, cfg config) (storage.KVProvider, func(),
|
||||
log.Info("storage backend", "backend", "mongodb", "database", cfg.MongoDatabase)
|
||||
return storage.NewMongoProvider(db), closer, nil
|
||||
|
||||
case "dynamodb":
|
||||
if cfg.DynamoDBTable == "" {
|
||||
return nil, func() {}, errors.New("KV_PROVIDER=dynamodb requires DYNAMODB_TABLE")
|
||||
}
|
||||
initCtx, cancel := context.WithTimeout(ctx, dynamodbInitTimeout)
|
||||
defer cancel()
|
||||
client, err := storage.NewDynamoDBClient(initCtx, storage.DynamoDBEndpointFromEnv())
|
||||
if err != nil {
|
||||
return nil, func() {}, err
|
||||
}
|
||||
log.Info("storage backend",
|
||||
"backend", "dynamodb",
|
||||
"table", cfg.DynamoDBTable,
|
||||
"endpoint_override", storage.DynamoDBEndpointFromEnv() != "")
|
||||
// DynamoDB SDK v2 client has no Close; HTTP client uses the default pool.
|
||||
return storage.NewDynamoDBProvider(client, cfg.DynamoDBTable), func() {}, nil
|
||||
|
||||
default:
|
||||
return nil, func() {}, fmt.Errorf("unknown KV_PROVIDER %q (want memory|dynamodb|mongodb)", backend)
|
||||
// DynamoDB is no longer a runtime backend — it survives only as the
|
||||
// one-off migration source (cmd/migrate-dynamo-to-mongo).
|
||||
return nil, func() {}, fmt.Errorf("unknown KV_PROVIDER %q (want memory|mongodb)", backend)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -298,8 +279,7 @@ type config struct {
|
||||
Modules []string
|
||||
BotOwnerID int64
|
||||
AdminUserIDs map[int64]bool
|
||||
KVProvider string // empty = auto-detect; or "memory"|"dynamodb"|"mongodb"
|
||||
DynamoDBTable string // required when KVProvider=dynamodb
|
||||
KVProvider string // empty = auto-detect; or "memory"|"mongodb"
|
||||
MongoURL string // required when KVProvider=mongodb (Atlas SRV connection string; SECRET — never log)
|
||||
MongoDatabase string // required when KVProvider=mongodb
|
||||
TelegramBotTokenParam string
|
||||
@@ -341,7 +321,6 @@ func loadConfig() config {
|
||||
BotOwnerID: parseInt64(envMap["OWNER_ID"]),
|
||||
AdminUserIDs: parseInt64Set(envMap["ADMIN_IDS"]),
|
||||
KVProvider: envMap["KV_PROVIDER"],
|
||||
DynamoDBTable: envMap["DYNAMODB_TABLE"],
|
||||
MongoURL: envMap["MONGO_URL"],
|
||||
MongoDatabase: envMap["MONGO_DATABASE"],
|
||||
TelegramBotTokenParam: strings.TrimSpace(envMap["TELEGRAM_BOT_TOKEN_PARAMETER_NAME"]),
|
||||
|
||||
@@ -63,13 +63,14 @@ and the `STOCK/COIN/GOLD *_API_URL` overrides (modules use coded defaults).
|
||||
4. Copy the `mongodb+srv://…` connection string into `MONGO_URL` and put the
|
||||
db name in `MONGO_DATABASE`.
|
||||
|
||||
> Storage layout: one collection per module; each document is
|
||||
> `{ _id: <user key>, value: <native BSON document>, version, updatedAt }`.
|
||||
> Values are stored as native BSON (objects/arrays expand and are queryable in
|
||||
> Compass); non-JSON values (e.g. a date guard) are plain strings. Concurrency
|
||||
> uses the `version` field (optimistic lock). If you migrated data with an older
|
||||
> build that stored `value` as a string/blob, re-run the migrator (idempotent)
|
||||
> so values become native.
|
||||
> Storage layout: one collection per module; each document is a flattened native
|
||||
> document — `{ _id: <user key>, ...payload fields, version, updatedAt }` with no
|
||||
> `value` envelope. Payload fields are hoisted to the document root so they
|
||||
> expand and are queryable in Compass. The two non-object values are wrapped in a
|
||||
> named field: lolschedule subscribers under `subscribers` (array) and the daily
|
||||
> push date under `date`. Concurrency uses the `version` field (optimistic lock);
|
||||
> `updatedAt` is a BSON Date. The DynamoDB→Mongo migrator writes this shape
|
||||
> directly, and is idempotent.
|
||||
|
||||
## 2. Coolify
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
// (see Makefile). An empty gitSHA — e.g. `go run` or a build without the
|
||||
// ldflags — is treated as a signal to stay silent.
|
||||
//
|
||||
// Dedup: a single KV record (key=last_notified_sha) holds the most recently
|
||||
// Dedup: a single store record (key=last_notified_sha) holds the most recently
|
||||
// notified SHA. On match we return early; on miss we send first, then write
|
||||
// — so a transient Telegram failure doesn't permanently silence retries.
|
||||
package deploynotify
|
||||
@@ -23,10 +23,10 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// kvKey is the single KV slot this package owns.
|
||||
const kvKey = "last_notified_sha"
|
||||
// storeKey is the single store slot this package owns.
|
||||
const storeKey = "last_notified_sha"
|
||||
|
||||
// defaultTimeout caps the whole Run path (KV read + Telegram send + KV write)
|
||||
// defaultTimeout caps the whole Run path (store read + Telegram send + store write)
|
||||
// so a misbehaving network can never block Lambda init past its 10s budget.
|
||||
const defaultTimeout = 3 * time.Second
|
||||
|
||||
@@ -34,7 +34,7 @@ const defaultTimeout = 3 * time.Second
|
||||
// nil, Run falls back to bot.SendMessage.
|
||||
type Config struct {
|
||||
Bot *bot.Bot
|
||||
KV storage.KVStore
|
||||
Store storage.DocStore[notifyRecord]
|
||||
OwnerID int64
|
||||
GitSHA string
|
||||
Timeout time.Duration
|
||||
@@ -43,11 +43,18 @@ type Config struct {
|
||||
Sender func(ctx context.Context, chatID int64, text string) error
|
||||
}
|
||||
|
||||
// notifyRecord is the KV value shape. At is informational only — useful for
|
||||
// eyeballing in the DynamoDB console; not consulted by the code path.
|
||||
// notifyRecord is the store value shape. At is informational only — useful for
|
||||
// eyeballing in the database console; not consulted by the code path.
|
||||
type notifyRecord struct {
|
||||
SHA string `json:"sha"`
|
||||
At int64 `json:"at"`
|
||||
SHA string `json:"sha" bson:"sha"`
|
||||
At int64 `json:"at" bson:"at"`
|
||||
}
|
||||
|
||||
// NewStore builds the typed store this package needs from a module Collection.
|
||||
// notifyRecord is unexported, so callers cannot name DocStore[notifyRecord]
|
||||
// directly — this is the seam that lets cmd/server wire the store.
|
||||
func NewStore(c storage.Collection) storage.DocStore[notifyRecord] {
|
||||
return storage.Typed[notifyRecord](c)
|
||||
}
|
||||
|
||||
// Run is the entry point. Fire-and-forget — never returns an error and
|
||||
@@ -64,12 +71,12 @@ func Run(ctx context.Context, cfg Config) {
|
||||
ctx, cancel := context.WithTimeout(ctx, timeout)
|
||||
defer cancel()
|
||||
|
||||
notify, err := shouldNotify(ctx, cfg.KV, cfg.GitSHA)
|
||||
notify, err := shouldNotify(ctx, cfg.Store, cfg.GitSHA)
|
||||
if err != nil {
|
||||
// Treat KV errors as "not notified yet": worst case is one extra
|
||||
// Treat store errors as "not notified yet": worst case is one extra
|
||||
// DM on the next cold start, which is far better than going
|
||||
// silent on every deploy because DynamoDB threw a transient.
|
||||
log.Warn("deploynotify kv read failed; will attempt send anyway", "err", err)
|
||||
// silent on every deploy because the store threw a transient error.
|
||||
log.Warn("deploynotify store read failed; will attempt send anyway", "err", err)
|
||||
notify = true
|
||||
}
|
||||
if !notify {
|
||||
@@ -80,23 +87,23 @@ func Run(ctx context.Context, cfg Config) {
|
||||
log.Warn("deploynotify telegram send failed", "err", err, "owner", cfg.OwnerID)
|
||||
return
|
||||
}
|
||||
if err := markNotified(ctx, cfg.KV, cfg.GitSHA); err != nil {
|
||||
log.Warn("deploynotify kv write failed (owner was notified)", "err", err)
|
||||
if err := markNotified(ctx, cfg.Store, cfg.GitSHA); err != nil {
|
||||
log.Warn("deploynotify store write failed (owner was notified)", "err", err)
|
||||
return
|
||||
}
|
||||
log.Info("deploynotify sent", "sha", cfg.GitSHA, "owner", cfg.OwnerID)
|
||||
}
|
||||
|
||||
// skipReason returns a non-empty short string when Run should no-op without
|
||||
// touching KV or Telegram. Empty string ⇒ proceed.
|
||||
// touching the store or Telegram. Empty string ⇒ proceed.
|
||||
func skipReason(cfg Config) string {
|
||||
switch {
|
||||
case cfg.GitSHA == "":
|
||||
return "empty gitSHA (build without -ldflags)"
|
||||
case cfg.OwnerID == 0:
|
||||
return "no OWNER_ID configured"
|
||||
case cfg.KV == nil:
|
||||
return "no KV configured"
|
||||
case cfg.Store == nil:
|
||||
return "no Store configured"
|
||||
case cfg.Bot == nil && cfg.Sender == nil:
|
||||
return "no bot or sender configured"
|
||||
}
|
||||
@@ -105,9 +112,8 @@ func skipReason(cfg Config) string {
|
||||
|
||||
// shouldNotify reports whether sha differs from the last notified value.
|
||||
// A missing record (ErrNotFound) is treated as "yes, notify".
|
||||
func shouldNotify(ctx context.Context, kv storage.KVStore, sha string) (bool, error) {
|
||||
var prev notifyRecord
|
||||
err := kv.GetJSON(ctx, kvKey, &prev)
|
||||
func shouldNotify(ctx context.Context, store storage.DocStore[notifyRecord], sha string) (bool, error) {
|
||||
prev, _, err := store.Get(ctx, storeKey)
|
||||
if errors.Is(err, storage.ErrNotFound) {
|
||||
return true, nil
|
||||
}
|
||||
@@ -117,9 +123,9 @@ func shouldNotify(ctx context.Context, kv storage.KVStore, sha string) (bool, er
|
||||
return prev.SHA != sha, nil
|
||||
}
|
||||
|
||||
// markNotified writes the SHA + current timestamp to KV.
|
||||
func markNotified(ctx context.Context, kv storage.KVStore, sha string) error {
|
||||
return kv.PutJSON(ctx, kvKey, notifyRecord{
|
||||
// markNotified writes the SHA + current timestamp to the store.
|
||||
func markNotified(ctx context.Context, store storage.DocStore[notifyRecord], sha string) error {
|
||||
return store.Put(ctx, storeKey, notifyRecord{
|
||||
SHA: sha,
|
||||
At: time.Now().UTC().UnixMilli(),
|
||||
})
|
||||
|
||||
@@ -9,23 +9,28 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// newNotifyStore returns a fresh in-memory typed notifyRecord store for tests.
|
||||
func newNotifyStore() storage.DocStore[notifyRecord] {
|
||||
return storage.Typed[notifyRecord](storage.NewMemoryProvider().Collection("deploynotify"))
|
||||
}
|
||||
|
||||
func TestShouldNotify_FirstRun(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
got, err := shouldNotify(context.Background(), kv, "abc123")
|
||||
store := newNotifyStore()
|
||||
got, err := shouldNotify(context.Background(), store, "abc123")
|
||||
if err != nil {
|
||||
t.Fatalf("err: %v", err)
|
||||
}
|
||||
if !got {
|
||||
t.Errorf("first run with empty KV → want true, got false")
|
||||
t.Errorf("first run with empty store → want true, got false")
|
||||
}
|
||||
}
|
||||
|
||||
func TestShouldNotify_SameSHA(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
if err := markNotified(context.Background(), kv, "abc123"); err != nil {
|
||||
store := newNotifyStore()
|
||||
if err := markNotified(context.Background(), store, "abc123"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := shouldNotify(context.Background(), kv, "abc123")
|
||||
got, err := shouldNotify(context.Background(), store, "abc123")
|
||||
if err != nil {
|
||||
t.Fatalf("err: %v", err)
|
||||
}
|
||||
@@ -35,11 +40,11 @@ func TestShouldNotify_SameSHA(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestShouldNotify_DifferentSHA(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
if err := markNotified(context.Background(), kv, "old111"); err != nil {
|
||||
store := newNotifyStore()
|
||||
if err := markNotified(context.Background(), store, "old111"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := shouldNotify(context.Background(), kv, "new222")
|
||||
got, err := shouldNotify(context.Background(), store, "new222")
|
||||
if err != nil {
|
||||
t.Fatalf("err: %v", err)
|
||||
}
|
||||
@@ -66,10 +71,10 @@ func (r *recorder) send(_ context.Context, chatID int64, text string) error {
|
||||
}
|
||||
|
||||
func TestRun_SkipsWhenSHAEmpty(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newNotifyStore()
|
||||
rec := &recorder{}
|
||||
Run(context.Background(), Config{
|
||||
KV: kv,
|
||||
Store: store,
|
||||
OwnerID: 42,
|
||||
GitSHA: "",
|
||||
Sender: rec.send,
|
||||
@@ -77,17 +82,17 @@ func TestRun_SkipsWhenSHAEmpty(t *testing.T) {
|
||||
if rec.called {
|
||||
t.Errorf("empty SHA must not send; got call with text=%q", rec.text)
|
||||
}
|
||||
// And no KV write either.
|
||||
if _, err := kv.Get(context.Background(), kvKey); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Errorf("empty SHA must not write KV; got err=%v", err)
|
||||
// And no store write either.
|
||||
if _, _, err := store.Get(context.Background(), storeKey); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Errorf("empty SHA must not write store; got err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRun_SkipsWhenNoOwner(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newNotifyStore()
|
||||
rec := &recorder{}
|
||||
Run(context.Background(), Config{
|
||||
KV: kv,
|
||||
Store: store,
|
||||
OwnerID: 0,
|
||||
GitSHA: "abc123",
|
||||
Sender: rec.send,
|
||||
@@ -95,22 +100,22 @@ func TestRun_SkipsWhenNoOwner(t *testing.T) {
|
||||
if rec.called {
|
||||
t.Errorf("zero owner must not send")
|
||||
}
|
||||
if _, err := kv.Get(context.Background(), kvKey); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Errorf("zero owner must not write KV; got err=%v", err)
|
||||
if _, _, err := store.Get(context.Background(), storeKey); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Errorf("zero owner must not write store; got err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRun_SendsAndPersistsOnFirstRun(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newNotifyStore()
|
||||
rec := &recorder{}
|
||||
Run(context.Background(), Config{
|
||||
KV: kv,
|
||||
Store: store,
|
||||
OwnerID: 42,
|
||||
GitSHA: "abc123",
|
||||
Sender: rec.send,
|
||||
})
|
||||
if !rec.called {
|
||||
t.Fatalf("first run with fresh KV must send")
|
||||
t.Fatalf("first run with fresh store must send")
|
||||
}
|
||||
if rec.chatID != 42 {
|
||||
t.Errorf("chatID = %d, want 42", rec.chatID)
|
||||
@@ -118,10 +123,10 @@ func TestRun_SendsAndPersistsOnFirstRun(t *testing.T) {
|
||||
if !strings.Contains(rec.text, "abc123") {
|
||||
t.Errorf("message %q missing SHA", rec.text)
|
||||
}
|
||||
// KV must now hold the SHA so the next Run is silent.
|
||||
var got notifyRecord
|
||||
if err := kv.GetJSON(context.Background(), kvKey, &got); err != nil {
|
||||
t.Fatalf("post-send KV read: %v", err)
|
||||
// Store must now hold the SHA so the next Run is silent.
|
||||
got, _, err := store.Get(context.Background(), storeKey)
|
||||
if err != nil {
|
||||
t.Fatalf("post-send store read: %v", err)
|
||||
}
|
||||
if got.SHA != "abc123" {
|
||||
t.Errorf("persisted SHA = %q, want abc123", got.SHA)
|
||||
@@ -129,13 +134,13 @@ func TestRun_SendsAndPersistsOnFirstRun(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestRun_SilentOnSecondRunSameSHA(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
if err := markNotified(context.Background(), kv, "abc123"); err != nil {
|
||||
store := newNotifyStore()
|
||||
if err := markNotified(context.Background(), store, "abc123"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rec := &recorder{}
|
||||
Run(context.Background(), Config{
|
||||
KV: kv,
|
||||
Store: store,
|
||||
OwnerID: 42,
|
||||
GitSHA: "abc123",
|
||||
Sender: rec.send,
|
||||
@@ -146,10 +151,10 @@ func TestRun_SilentOnSecondRunSameSHA(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestRun_DoesNotPersistOnSendFailure(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newNotifyStore()
|
||||
rec := &recorder{err: errors.New("telegram is down")}
|
||||
Run(context.Background(), Config{
|
||||
KV: kv,
|
||||
Store: store,
|
||||
OwnerID: 42,
|
||||
GitSHA: "abc123",
|
||||
Sender: rec.send,
|
||||
@@ -158,8 +163,8 @@ func TestRun_DoesNotPersistOnSendFailure(t *testing.T) {
|
||||
t.Fatalf("sender should have been called")
|
||||
}
|
||||
// Send failed → SHA must NOT be persisted, so the next cold start retries.
|
||||
if _, err := kv.Get(context.Background(), kvKey); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Errorf("failed send must not write KV; got err=%v", err)
|
||||
if _, _, err := store.Get(context.Background(), storeKey); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Errorf("failed send must not write store; got err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
// (typically a chat / user / subject identifier) across goroutines.
|
||||
//
|
||||
// Why a separate package: every game module needs a per-subject mutex to
|
||||
// turn KVStore's single-op atomicity into safe Get→mutate→Put. The bot
|
||||
// turn the store's single-op atomicity into safe Get→mutate→Put. The bot
|
||||
// dispatcher runs each Telegram update in its own goroutine, so without
|
||||
// explicit per-subject serialisation two updates to the same game could
|
||||
// race and drop one write.
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
package coin
|
||||
|
||||
import "github.com/tiennm99/miti99bot/internal/modules"
|
||||
import (
|
||||
"github.com/tiennm99/miti99bot/internal/modules"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// New is the coin paper-trading module factory. It is opt-in through MODULES
|
||||
// and keeps its portfolio state separate from stock and gold modules.
|
||||
func New(deps modules.Deps) modules.Module {
|
||||
s := newState(deps.KV)
|
||||
s := newState(storage.Typed[Portfolio](deps.Store))
|
||||
return modules.Module{
|
||||
Commands: []modules.Command{
|
||||
{
|
||||
|
||||
@@ -49,7 +49,7 @@ func (s *state) handleTopup(ctx context.Context, b *bot.Bot, update *models.Upda
|
||||
return chathelper.Reply(ctx, b, update.Message, "Amount must be a positive finite USD number within the supported range.")
|
||||
}
|
||||
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
|
||||
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
p.AddUSD(amount)
|
||||
p.Meta.Invested += amount
|
||||
return nil
|
||||
@@ -94,7 +94,7 @@ func (s *state) handleBuy(ctx context.Context, b *bot.Bot, update *models.Update
|
||||
}
|
||||
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
|
||||
var insufficientBalance *float64
|
||||
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
ok, balance := p.DeductUSD(amount)
|
||||
if !ok {
|
||||
insufficientBalance = &balance
|
||||
@@ -149,7 +149,7 @@ func (s *state) handleSell(ctx context.Context, b *bot.Bot, update *models.Updat
|
||||
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
|
||||
var insufficientHeldQty float64
|
||||
var insufficientHeld bool
|
||||
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
ok, held := p.DeductAsset(coin.Symbol, qty)
|
||||
if !ok {
|
||||
insufficientHeldQty = held
|
||||
|
||||
@@ -19,7 +19,7 @@ func TestHandleBuyAcceptsCoinFirstOrder(t *testing.T) {
|
||||
}
|
||||
|
||||
rb.AssertSentText(t, "Bought 0.0002 BTC")
|
||||
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.USD != 990 || p.Assets["BTC"] != 0.0002 {
|
||||
t.Fatalf("after coin-first buy = %+v", p)
|
||||
}
|
||||
@@ -38,7 +38,7 @@ func TestHandleSellAcceptsCoinFirstOrder(t *testing.T) {
|
||||
}
|
||||
|
||||
rb.AssertSentText(t, "Sold 0.01 BTC")
|
||||
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.USD != 1000 || len(p.Assets) != 0 {
|
||||
t.Fatalf("after coin-first sell = %+v", p)
|
||||
}
|
||||
|
||||
@@ -31,9 +31,14 @@ func (f fakePriceFetcher) FetchUSD(_ context.Context, coin CoinSymbol) (CoinPric
|
||||
return price, nil
|
||||
}
|
||||
|
||||
// newCoinStore returns a fresh in-memory typed portfolio store for tests.
|
||||
func newCoinStore() Store {
|
||||
return storage.Typed[Portfolio](storage.NewMemoryProvider().Collection("coin"))
|
||||
}
|
||||
|
||||
func newTestState(prices map[string]CoinPrice, err error) *state {
|
||||
return &state{
|
||||
kv: storage.NewMemoryKVStore(),
|
||||
store: newCoinStore(),
|
||||
prices: fakePriceFetcher{prices: prices, err: err},
|
||||
nowFn: func() time.Time { return time.UnixMilli(123) },
|
||||
}
|
||||
@@ -100,7 +105,7 @@ func TestHandleTopup(t *testing.T) {
|
||||
t.Fatalf("handleTopup: %v", err)
|
||||
}
|
||||
rb.AssertSentText(t, "Topped up $1,000.00")
|
||||
p, err := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, err := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadPortfolio: %v", err)
|
||||
}
|
||||
@@ -119,7 +124,7 @@ func TestHandleBuyAndSell(t *testing.T) {
|
||||
t.Fatalf("handleBuy: %v", err)
|
||||
}
|
||||
rb.AssertSentText(t, "Bought 0.01 BTC")
|
||||
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.USD != 500 || p.Assets["BTC"] != 0.01 {
|
||||
t.Fatalf("after buy = %+v", p)
|
||||
}
|
||||
@@ -128,7 +133,7 @@ func TestHandleBuyAndSell(t *testing.T) {
|
||||
t.Fatalf("handleSell: %v", err)
|
||||
}
|
||||
rb.AssertSentText(t, "Sold 0.01 BTC")
|
||||
p, _ = LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ = LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.USD != 1000 || len(p.Assets) != 0 {
|
||||
t.Fatalf("after sell = %+v", p)
|
||||
}
|
||||
@@ -205,7 +210,7 @@ func TestHandleSellInsufficientCoinWithHoldings(t *testing.T) {
|
||||
t.Fatalf("insufficient sell message used ambiguous have wording: %q", text)
|
||||
}
|
||||
|
||||
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.USD != 500 || p.Assets["BTC"] != 0.01 {
|
||||
t.Fatalf("portfolio mutated on failed sell = %+v", p)
|
||||
}
|
||||
@@ -224,7 +229,7 @@ func TestHandleSellRejectsDustQuantity(t *testing.T) {
|
||||
}
|
||||
rb.AssertSentText(t, "too small")
|
||||
|
||||
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.USD != 999 || len(p.Assets) != 1 {
|
||||
t.Fatalf("portfolio mutated on dust sell = %+v", p)
|
||||
}
|
||||
@@ -242,7 +247,7 @@ func TestHandleBuyRejectsDustQuantity(t *testing.T) {
|
||||
}
|
||||
rb.AssertSentText(t, "too small")
|
||||
|
||||
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.USD != 1000 || len(p.Assets) != 0 {
|
||||
t.Fatalf("portfolio mutated on dust buy = %+v", p)
|
||||
}
|
||||
@@ -276,7 +281,7 @@ func TestPriceErrorDoesNotMutatePortfolio(t *testing.T) {
|
||||
t.Fatalf("handleBuy: %v", err)
|
||||
}
|
||||
rb.AssertSentText(t, "Could not fetch coin price")
|
||||
p, err := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, err := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadPortfolio: %v", err)
|
||||
}
|
||||
@@ -310,5 +315,5 @@ func TestStatsWithAndWithoutPrice(t *testing.T) {
|
||||
}
|
||||
|
||||
func modDepsForTest() modules.Deps {
|
||||
return modules.Deps{KV: storage.NewMemoryKVStore()}
|
||||
return modules.Deps{Store: storage.NewMemoryProvider().Collection("coin")}
|
||||
}
|
||||
@@ -14,7 +14,6 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/keylock"
|
||||
"github.com/tiennm99/miti99bot/internal/log"
|
||||
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
type priceFetcher interface {
|
||||
@@ -22,14 +21,14 @@ type priceFetcher interface {
|
||||
}
|
||||
|
||||
type state struct {
|
||||
kv storage.KVStore
|
||||
store Store
|
||||
prices priceFetcher
|
||||
locks keylock.Map
|
||||
nowFn func() time.Time
|
||||
}
|
||||
|
||||
func newState(kv storage.KVStore) *state {
|
||||
return &state{kv: kv, prices: NewPriceClientFromEnv()}
|
||||
func newState(store Store) *state {
|
||||
return &state{store: store, prices: NewPriceClientFromEnv()}
|
||||
}
|
||||
|
||||
func (s *state) now() time.Time {
|
||||
|
||||
@@ -2,7 +2,6 @@ package coin
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math"
|
||||
@@ -14,15 +13,18 @@ import (
|
||||
const coinDustEpsilon = 1e-9
|
||||
const portfolioUpdateAttempts = 5
|
||||
|
||||
// Store is the coin module's typed portfolio store.
|
||||
type Store = storage.DocStore[Portfolio]
|
||||
|
||||
type Portfolio struct {
|
||||
USD float64 `json:"usd"`
|
||||
Assets map[string]float64 `json:"assets"`
|
||||
Meta PortfolioMeta `json:"meta"`
|
||||
USD float64 `json:"usd" bson:"usd"`
|
||||
Assets map[string]float64 `json:"assets" bson:"assets"`
|
||||
Meta PortfolioMeta `json:"meta" bson:"meta"`
|
||||
}
|
||||
|
||||
type PortfolioMeta struct {
|
||||
Invested float64 `json:"invested"`
|
||||
CreatedAt int64 `json:"createdAt"`
|
||||
Invested float64 `json:"invested" bson:"invested"`
|
||||
CreatedAt int64 `json:"createdAt" bson:"createdAt"`
|
||||
}
|
||||
|
||||
func NewPortfolio(now int64) Portfolio {
|
||||
@@ -33,42 +35,26 @@ func portfolioKey(userID int64) string {
|
||||
return "user:" + strconv.FormatInt(userID, 10)
|
||||
}
|
||||
|
||||
func LoadPortfolio(ctx context.Context, kv storage.KVStore, userID int64, now int64) (Portfolio, error) {
|
||||
var p Portfolio
|
||||
err := kv.GetJSON(ctx, portfolioKey(userID), &p)
|
||||
switch {
|
||||
case err == nil:
|
||||
p.normalize()
|
||||
if p.Assets == nil {
|
||||
p.Assets = map[string]float64{}
|
||||
}
|
||||
if p.Meta.CreatedAt == 0 {
|
||||
p.Meta.CreatedAt = now
|
||||
}
|
||||
return p, nil
|
||||
case errors.Is(err, storage.ErrNotFound):
|
||||
return NewPortfolio(now), nil
|
||||
default:
|
||||
func LoadPortfolio(ctx context.Context, store Store, userID int64, now int64) (Portfolio, error) {
|
||||
p, _, err := loadPortfolioForUpdate(ctx, store, portfolioKey(userID), now)
|
||||
if err != nil {
|
||||
return Portfolio{}, fmt.Errorf("coin: load portfolio %d: %w", userID, err)
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
|
||||
func SavePortfolio(ctx context.Context, kv storage.KVStore, userID int64, p Portfolio) error {
|
||||
func SavePortfolio(ctx context.Context, store Store, userID int64, p Portfolio) error {
|
||||
p.normalize()
|
||||
if err := kv.PutJSON(ctx, portfolioKey(userID), p); err != nil {
|
||||
if err := store.Put(ctx, portfolioKey(userID), p); err != nil {
|
||||
return fmt.Errorf("coin: save portfolio %d: %w", userID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now int64, mutate func(*Portfolio) error) (Portfolio, error) {
|
||||
vs, ok := kv.(storage.VersionedStore)
|
||||
if !ok {
|
||||
return Portfolio{}, fmt.Errorf("coin: storage does not support versioned portfolio updates")
|
||||
}
|
||||
func UpdatePortfolio(ctx context.Context, store Store, userID int64, now int64, mutate func(*Portfolio) error) (Portfolio, error) {
|
||||
key := portfolioKey(userID)
|
||||
for attempt := 0; attempt < portfolioUpdateAttempts; attempt++ {
|
||||
p, version, err := loadPortfolioForUpdate(ctx, vs, key, now)
|
||||
p, version, err := loadPortfolioForUpdate(ctx, store, key, now)
|
||||
if err != nil {
|
||||
return Portfolio{}, fmt.Errorf("coin: load portfolio %d: %w", userID, err)
|
||||
}
|
||||
@@ -76,11 +62,7 @@ func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now
|
||||
return p, err
|
||||
}
|
||||
p.normalize()
|
||||
next, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
return Portfolio{}, fmt.Errorf("coin: save portfolio %d: json encode: %w", userID, err)
|
||||
}
|
||||
if err := vs.PutVersioned(ctx, key, version, next); err == nil {
|
||||
if err := store.PutVersioned(ctx, key, version, p); err == nil {
|
||||
return p, nil
|
||||
} else if !errors.Is(err, storage.ErrConflict) {
|
||||
return Portfolio{}, fmt.Errorf("coin: save portfolio %d: %w", userID, err)
|
||||
@@ -89,14 +71,10 @@ func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now
|
||||
return Portfolio{}, fmt.Errorf("coin: save portfolio %d: %w", userID, storage.ErrConflict)
|
||||
}
|
||||
|
||||
func loadPortfolioForUpdate(ctx context.Context, vs storage.VersionedStore, key string, now int64) (Portfolio, int64, error) {
|
||||
raw, version, err := vs.GetVersioned(ctx, key)
|
||||
func loadPortfolioForUpdate(ctx context.Context, store Store, key string, now int64) (Portfolio, int64, error) {
|
||||
p, version, err := store.Get(ctx, key)
|
||||
switch {
|
||||
case err == nil:
|
||||
var p Portfolio
|
||||
if err := json.Unmarshal(raw, &p); err != nil {
|
||||
return Portfolio{}, 0, fmt.Errorf("json decode: %w", err)
|
||||
}
|
||||
p.normalize()
|
||||
if p.Assets == nil {
|
||||
p.Assets = map[string]float64{}
|
||||
|
||||
@@ -10,7 +10,7 @@ import (
|
||||
)
|
||||
|
||||
func TestLoadPortfolioFirstTimeUser(t *testing.T) {
|
||||
p, err := LoadPortfolio(context.Background(), storage.NewMemoryKVStore(), 42, 123)
|
||||
p, err := LoadPortfolio(context.Background(), newCoinStore(), 42, 123)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadPortfolio: %v", err)
|
||||
}
|
||||
@@ -67,32 +67,31 @@ func TestNormalizeAmountSpecialValues(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// conflictOnceStore wraps a real typed store and forces exactly one write
|
||||
// conflict (after committing a competing value) before delegating, to exercise
|
||||
// UpdatePortfolio's optimistic-lock retry.
|
||||
type conflictOnceStore struct {
|
||||
storage.KVStore
|
||||
Store
|
||||
conflicted bool
|
||||
}
|
||||
|
||||
func (s *conflictOnceStore) GetVersioned(ctx context.Context, key string) ([]byte, int64, error) {
|
||||
return s.KVStore.(storage.VersionedStore).GetVersioned(ctx, key)
|
||||
}
|
||||
|
||||
func (s *conflictOnceStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error {
|
||||
func (s *conflictOnceStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val Portfolio) error {
|
||||
if !s.conflicted {
|
||||
s.conflicted = true
|
||||
competing := NewPortfolio(1)
|
||||
competing.AddUSD(10)
|
||||
if err := s.PutJSON(ctx, key, competing); err != nil {
|
||||
if err := s.Store.Put(ctx, key, competing); err != nil {
|
||||
return err
|
||||
}
|
||||
return storage.ErrConflict
|
||||
}
|
||||
return s.KVStore.(storage.VersionedStore).PutVersioned(ctx, key, expectedVersion, val)
|
||||
return s.Store.PutVersioned(ctx, key, expectedVersion, val)
|
||||
}
|
||||
|
||||
func TestUpdatePortfolioRetriesAfterWriteConflict(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := &conflictOnceStore{KVStore: storage.NewMemoryKVStore()}
|
||||
got, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
|
||||
store := &conflictOnceStore{Store: newCoinStore()}
|
||||
got, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
|
||||
p.AddUSD(5)
|
||||
return nil
|
||||
})
|
||||
@@ -106,14 +105,14 @@ func TestUpdatePortfolioRetriesAfterWriteConflict(t *testing.T) {
|
||||
|
||||
func TestUpdatePortfolioMutateErrorDoesNotPersist(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
_, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
|
||||
store := newCoinStore()
|
||||
_, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
|
||||
return errInsufficientUSD
|
||||
})
|
||||
if !errors.Is(err, errInsufficientUSD) {
|
||||
t.Fatalf("got %v, want errInsufficientUSD", err)
|
||||
}
|
||||
if _, err := kv.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
|
||||
if _, _, err := store.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Fatalf("failed mutate must not persist, Get = %v", err)
|
||||
}
|
||||
}
|
||||
@@ -17,7 +17,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
|
||||
if !ok {
|
||||
return chathelper.Reply(ctx, b, update.Message, "Cannot identify user - /coin_stats needs a sender.")
|
||||
}
|
||||
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
|
||||
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
|
||||
if err != nil {
|
||||
log.Error("coin_load_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not load coin portfolio. Try again later.")
|
||||
|
||||
@@ -20,9 +20,9 @@ type compositePriceFetcher struct {
|
||||
|
||||
// NewCompositePriceFetcherFromEnv builds the production price fetcher using
|
||||
// only the env-driven VNAppMob client.
|
||||
func NewCompositePriceFetcherFromEnv(kv storage.KVStore) priceFetcher {
|
||||
func NewCompositePriceFetcherFromEnv(coll storage.Collection) priceFetcher {
|
||||
return &compositePriceFetcher{
|
||||
vnappmob: NewVNAppMobClientFromEnv(kv),
|
||||
vnappmob: NewVNAppMobClientFromEnv(coll),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -92,7 +92,7 @@ func TestGoldPriceLines(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestNewCompositePriceFetcherFromEnv(t *testing.T) {
|
||||
f, ok := NewCompositePriceFetcherFromEnv(storage.NewMemoryKVStore()).(*compositePriceFetcher)
|
||||
f, ok := NewCompositePriceFetcherFromEnv(storage.NewMemoryProvider().Collection("gold")).(*compositePriceFetcher)
|
||||
if !ok {
|
||||
t.Fatalf("expected *compositePriceFetcher, got %T", f)
|
||||
}
|
||||
|
||||
@@ -1,11 +1,13 @@
|
||||
package gold
|
||||
|
||||
import "github.com/tiennm99/miti99bot/internal/modules"
|
||||
import (
|
||||
"github.com/tiennm99/miti99bot/internal/modules"
|
||||
)
|
||||
|
||||
// New is the gold paper-trading module factory. It is opt-in through MODULES
|
||||
// and keeps its portfolio state separate from the stock module.
|
||||
func New(deps modules.Deps) modules.Module {
|
||||
s := newState(deps.KV)
|
||||
s := newState(deps.Store)
|
||||
return modules.Module{
|
||||
Commands: []modules.Command{
|
||||
{
|
||||
|
||||
@@ -60,7 +60,7 @@ func (s *state) handleTopup(ctx context.Context, b *bot.Bot, update *models.Upda
|
||||
}
|
||||
|
||||
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
|
||||
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
p.AddVND(amount)
|
||||
p.Meta.Invested += amount
|
||||
return nil
|
||||
@@ -98,7 +98,7 @@ func (s *state) handleBuy(ctx context.Context, b *bot.Bot, update *models.Update
|
||||
|
||||
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
|
||||
var insufficientBalance *float64
|
||||
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
ok, balance := p.DeductVND(cost)
|
||||
if !ok {
|
||||
insufficientBalance = &balance
|
||||
@@ -145,7 +145,7 @@ func (s *state) handleSell(ctx context.Context, b *bot.Bot, update *models.Updat
|
||||
return chathelper.Reply(ctx, b, update.Message, "Trade value is too large.")
|
||||
}
|
||||
var insufficientHeld *float64
|
||||
p, err := UpdatePortfolio(ctx, s.kv, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
p, err := UpdatePortfolio(ctx, s.store, userID, s.now().UnixMilli(), func(p *Portfolio) error {
|
||||
ok, held := p.DeductLuong(qty)
|
||||
if !ok {
|
||||
insufficientHeld = &held
|
||||
@@ -173,7 +173,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
|
||||
return chathelper.Reply(ctx, b, update.Message,
|
||||
"Cannot identify user - /gold_stats needs a sender.")
|
||||
}
|
||||
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
|
||||
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
|
||||
if err != nil {
|
||||
log.Error("gold_load_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not load gold portfolio. Try again later.")
|
||||
|
||||
@@ -38,7 +38,7 @@ func (f fakePriceFetcher) FetchPrice(context.Context) (GoldPrice, error) {
|
||||
|
||||
func newTestState(price float64, err error) *state {
|
||||
return &state{
|
||||
kv: storage.NewMemoryKVStore(),
|
||||
store: newGoldStore(),
|
||||
prices: fakePriceFetcher{price: price, err: err},
|
||||
nowFn: func() time.Time { return time.UnixMilli(123) },
|
||||
}
|
||||
@@ -77,7 +77,7 @@ func TestHandleTopup(t *testing.T) {
|
||||
t.Fatalf("handleTopup: %v", err)
|
||||
}
|
||||
rb.AssertSentText(t, "Topped up 5.000.000 VND")
|
||||
p, err := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, err := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadPortfolio: %v", err)
|
||||
}
|
||||
@@ -98,7 +98,7 @@ func TestHandleBuyAndSell(t *testing.T) {
|
||||
t.Fatalf("buy: %v", err)
|
||||
}
|
||||
rb.AssertSentText(t, "Bought 1.25 luong gold")
|
||||
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.Luong != 1.25 || p.VND != 2_500_000 {
|
||||
t.Fatalf("after buy: %+v", p)
|
||||
}
|
||||
@@ -107,7 +107,7 @@ func TestHandleBuyAndSell(t *testing.T) {
|
||||
t.Fatalf("sell: %v", err)
|
||||
}
|
||||
rb.AssertSentText(t, "Sold 1.25 luong gold")
|
||||
p, _ = LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ = LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.Luong != 0 || p.VND != 5_000_000 {
|
||||
t.Fatalf("after sell: %+v", p)
|
||||
}
|
||||
@@ -139,7 +139,7 @@ func TestPriceErrorDoesNotMutatePortfolio(t *testing.T) {
|
||||
t.Fatalf("buy: %v", err)
|
||||
}
|
||||
rb.AssertSentText(t, "Could not fetch gold price")
|
||||
p, err := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, err := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadPortfolio: %v", err)
|
||||
}
|
||||
@@ -201,7 +201,7 @@ func (f spreadPriceFetcher) FetchPrice(context.Context) (GoldPrice, error) {
|
||||
}
|
||||
|
||||
func modDepsForTest() modules.Deps {
|
||||
return modules.Deps{KV: storage.NewMemoryKVStore()}
|
||||
return modules.Deps{Store: storage.NewMemoryProvider().Collection("gold")}
|
||||
}
|
||||
|
||||
func TestHandlePrice(t *testing.T) {
|
||||
@@ -239,7 +239,7 @@ func TestHandlePriceFetchError(t *testing.T) {
|
||||
func TestHandleBuyUsesSellPrice(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := &state{
|
||||
kv: storage.NewMemoryKVStore(),
|
||||
store: newGoldStore(),
|
||||
prices: spreadPriceFetcher{buy: 90_000_000, sell: 91_000_000},
|
||||
nowFn: func() time.Time { return time.UnixMilli(123) },
|
||||
}
|
||||
@@ -253,7 +253,7 @@ func TestHandleBuyUsesSellPrice(t *testing.T) {
|
||||
if !strings.Contains(text, "91.000.000 VND/luong") {
|
||||
t.Fatalf("expected sell price 91.000.000 in %q", text)
|
||||
}
|
||||
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.VND != 409_000_000 {
|
||||
t.Fatalf("balance after buy at 91M: got %v, want 409000000", p.VND)
|
||||
}
|
||||
@@ -262,7 +262,7 @@ func TestHandleBuyUsesSellPrice(t *testing.T) {
|
||||
func TestHandleSellUsesBuyPrice(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := &state{
|
||||
kv: storage.NewMemoryKVStore(),
|
||||
store: newGoldStore(),
|
||||
prices: spreadPriceFetcher{buy: 90_000_000, sell: 91_000_000},
|
||||
nowFn: func() time.Time { return time.UnixMilli(123) },
|
||||
}
|
||||
@@ -277,7 +277,7 @@ func TestHandleSellUsesBuyPrice(t *testing.T) {
|
||||
if !strings.Contains(text, "90.000.000 VND/luong") {
|
||||
t.Fatalf("expected buy price 90.000.000 in %q", text)
|
||||
}
|
||||
p, _ := LoadPortfolio(ctx, s.kv, 7, 999)
|
||||
p, _ := LoadPortfolio(ctx, s.store, 7, 999)
|
||||
if p.VND != 499_000_000 {
|
||||
t.Fatalf("balance after sell at 90M: got %v, want 499000000", p.VND)
|
||||
}
|
||||
|
||||
@@ -24,14 +24,17 @@ type priceFetcher interface {
|
||||
}
|
||||
|
||||
type state struct {
|
||||
kv storage.KVStore
|
||||
store PortfolioStore
|
||||
prices priceFetcher
|
||||
locks keylock.Map
|
||||
nowFn func() time.Time
|
||||
}
|
||||
|
||||
func newState(kv storage.KVStore) *state {
|
||||
return &state{kv: kv, prices: NewCompositePriceFetcherFromEnv(kv)}
|
||||
func newState(coll storage.Collection) *state {
|
||||
return &state{
|
||||
store: storage.Typed[Portfolio](coll),
|
||||
prices: NewCompositePriceFetcherFromEnv(coll),
|
||||
}
|
||||
}
|
||||
|
||||
func (s *state) now() time.Time {
|
||||
|
||||
@@ -2,7 +2,6 @@ package gold
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math"
|
||||
@@ -14,15 +13,18 @@ import (
|
||||
const goldDustEpsilon = 1e-9
|
||||
const portfolioUpdateAttempts = 5
|
||||
|
||||
// PortfolioStore is the gold module's typed portfolio store.
|
||||
type PortfolioStore = storage.DocStore[Portfolio]
|
||||
|
||||
type Portfolio struct {
|
||||
VND float64 `json:"vnd"`
|
||||
Luong float64 `json:"luong"`
|
||||
Meta PortfolioMeta `json:"meta"`
|
||||
VND float64 `json:"vnd" bson:"vnd"`
|
||||
Luong float64 `json:"luong" bson:"luong"`
|
||||
Meta PortfolioMeta `json:"meta" bson:"meta"`
|
||||
}
|
||||
|
||||
type PortfolioMeta struct {
|
||||
Invested float64 `json:"invested"`
|
||||
CreatedAt int64 `json:"createdAt"`
|
||||
Invested float64 `json:"invested" bson:"invested"`
|
||||
CreatedAt int64 `json:"createdAt" bson:"createdAt"`
|
||||
}
|
||||
|
||||
func NewPortfolio(now int64) Portfolio {
|
||||
@@ -33,39 +35,26 @@ func portfolioKey(userID int64) string {
|
||||
return "user:" + strconv.FormatInt(userID, 10)
|
||||
}
|
||||
|
||||
func LoadPortfolio(ctx context.Context, kv storage.KVStore, userID int64, now int64) (Portfolio, error) {
|
||||
var p Portfolio
|
||||
err := kv.GetJSON(ctx, portfolioKey(userID), &p)
|
||||
switch {
|
||||
case err == nil:
|
||||
p.normalize()
|
||||
if p.Meta.CreatedAt == 0 {
|
||||
p.Meta.CreatedAt = now
|
||||
}
|
||||
return p, nil
|
||||
case errors.Is(err, storage.ErrNotFound):
|
||||
return NewPortfolio(now), nil
|
||||
default:
|
||||
func LoadPortfolio(ctx context.Context, store PortfolioStore, userID int64, now int64) (Portfolio, error) {
|
||||
p, _, err := loadPortfolioForUpdate(ctx, store, portfolioKey(userID), now)
|
||||
if err != nil {
|
||||
return Portfolio{}, fmt.Errorf("gold: load portfolio %d: %w", userID, err)
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
|
||||
func SavePortfolio(ctx context.Context, kv storage.KVStore, userID int64, p Portfolio) error {
|
||||
func SavePortfolio(ctx context.Context, store PortfolioStore, userID int64, p Portfolio) error {
|
||||
p.normalize()
|
||||
if err := kv.PutJSON(ctx, portfolioKey(userID), p); err != nil {
|
||||
if err := store.Put(ctx, portfolioKey(userID), p); err != nil {
|
||||
return fmt.Errorf("gold: save portfolio %d: %w", userID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now int64, mutate func(*Portfolio) error) (Portfolio, error) {
|
||||
vs, ok := kv.(storage.VersionedStore)
|
||||
if !ok {
|
||||
return Portfolio{}, fmt.Errorf("gold: storage does not support versioned portfolio updates")
|
||||
}
|
||||
func UpdatePortfolio(ctx context.Context, store PortfolioStore, userID int64, now int64, mutate func(*Portfolio) error) (Portfolio, error) {
|
||||
key := portfolioKey(userID)
|
||||
for attempt := 0; attempt < portfolioUpdateAttempts; attempt++ {
|
||||
p, version, err := loadPortfolioForUpdate(ctx, vs, key, now)
|
||||
p, version, err := loadPortfolioForUpdate(ctx, store, key, now)
|
||||
if err != nil {
|
||||
return Portfolio{}, fmt.Errorf("gold: load portfolio %d: %w", userID, err)
|
||||
}
|
||||
@@ -73,11 +62,7 @@ func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now
|
||||
return p, err
|
||||
}
|
||||
p.normalize()
|
||||
next, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
return Portfolio{}, fmt.Errorf("gold: save portfolio %d: json encode: %w", userID, err)
|
||||
}
|
||||
if err := vs.PutVersioned(ctx, key, version, next); err == nil {
|
||||
if err := store.PutVersioned(ctx, key, version, p); err == nil {
|
||||
return p, nil
|
||||
} else if !errors.Is(err, storage.ErrConflict) {
|
||||
return Portfolio{}, fmt.Errorf("gold: save portfolio %d: %w", userID, err)
|
||||
@@ -86,14 +71,10 @@ func UpdatePortfolio(ctx context.Context, kv storage.KVStore, userID int64, now
|
||||
return Portfolio{}, fmt.Errorf("gold: save portfolio %d: %w", userID, storage.ErrConflict)
|
||||
}
|
||||
|
||||
func loadPortfolioForUpdate(ctx context.Context, vs storage.VersionedStore, key string, now int64) (Portfolio, int64, error) {
|
||||
raw, version, err := vs.GetVersioned(ctx, key)
|
||||
func loadPortfolioForUpdate(ctx context.Context, store PortfolioStore, key string, now int64) (Portfolio, int64, error) {
|
||||
p, version, err := store.Get(ctx, key)
|
||||
switch {
|
||||
case err == nil:
|
||||
var p Portfolio
|
||||
if err := json.Unmarshal(raw, &p); err != nil {
|
||||
return Portfolio{}, 0, fmt.Errorf("json decode: %w", err)
|
||||
}
|
||||
p.normalize()
|
||||
if p.Meta.CreatedAt == 0 {
|
||||
p.Meta.CreatedAt = now
|
||||
|
||||
@@ -12,9 +12,13 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// newGoldStore returns a fresh in-memory typed portfolio store for tests.
|
||||
func newGoldStore() PortfolioStore {
|
||||
return storage.Typed[Portfolio](storage.NewMemoryProvider().Collection("gold"))
|
||||
}
|
||||
|
||||
func TestLoadPortfolio_FirstTimeUser(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
p, err := LoadPortfolio(context.Background(), kv, 42, 123)
|
||||
p, err := LoadPortfolio(context.Background(), newGoldStore(), 42, 123)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadPortfolio: %v", err)
|
||||
}
|
||||
@@ -24,15 +28,15 @@ func TestLoadPortfolio_FirstTimeUser(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestSaveAndLoadRoundTrip(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newGoldStore()
|
||||
p := NewPortfolio(1)
|
||||
p.AddVND(5_000_000)
|
||||
p.AddLuong(1.25)
|
||||
p.Meta.Invested = 5_000_000
|
||||
if err := SavePortfolio(context.Background(), kv, 42, p); err != nil {
|
||||
if err := SavePortfolio(context.Background(), store, 42, p); err != nil {
|
||||
t.Fatalf("Save: %v", err)
|
||||
}
|
||||
got, err := LoadPortfolio(context.Background(), kv, 42, 999)
|
||||
got, err := LoadPortfolio(context.Background(), store, 42, 999)
|
||||
if err != nil {
|
||||
t.Fatalf("Load: %v", err)
|
||||
}
|
||||
@@ -71,32 +75,31 @@ func TestNormalizeAmountSpecialValues(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// conflictOnceStore wraps a real typed store and forces exactly one write
|
||||
// conflict (after committing a competing value) before delegating, to exercise
|
||||
// UpdatePortfolio's optimistic-lock retry.
|
||||
type conflictOnceStore struct {
|
||||
storage.KVStore
|
||||
PortfolioStore
|
||||
conflicted bool
|
||||
}
|
||||
|
||||
func (s *conflictOnceStore) GetVersioned(ctx context.Context, key string) ([]byte, int64, error) {
|
||||
return s.KVStore.(storage.VersionedStore).GetVersioned(ctx, key)
|
||||
}
|
||||
|
||||
func (s *conflictOnceStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error {
|
||||
func (s *conflictOnceStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val Portfolio) error {
|
||||
if !s.conflicted {
|
||||
s.conflicted = true
|
||||
competing := NewPortfolio(1)
|
||||
competing.AddVND(10)
|
||||
if err := s.PutJSON(ctx, key, competing); err != nil {
|
||||
if err := s.PortfolioStore.Put(ctx, key, competing); err != nil {
|
||||
return err
|
||||
}
|
||||
return storage.ErrConflict
|
||||
}
|
||||
return s.KVStore.(storage.VersionedStore).PutVersioned(ctx, key, expectedVersion, val)
|
||||
return s.PortfolioStore.PutVersioned(ctx, key, expectedVersion, val)
|
||||
}
|
||||
|
||||
func TestUpdatePortfolioRetriesAfterWriteConflict(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := &conflictOnceStore{KVStore: storage.NewMemoryKVStore()}
|
||||
got, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
|
||||
store := &conflictOnceStore{PortfolioStore: newGoldStore()}
|
||||
got, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
|
||||
p.AddVND(5)
|
||||
return nil
|
||||
})
|
||||
@@ -106,7 +109,7 @@ func TestUpdatePortfolioRetriesAfterWriteConflict(t *testing.T) {
|
||||
if got.VND != 15 {
|
||||
t.Fatalf("updated portfolio VND = %v, want 15", got.VND)
|
||||
}
|
||||
loaded, err := LoadPortfolio(ctx, kv, 7, 1)
|
||||
loaded, err := LoadPortfolio(ctx, store, 7, 1)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadPortfolio: %v", err)
|
||||
}
|
||||
@@ -115,53 +118,51 @@ func TestUpdatePortfolioRetriesAfterWriteConflict(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// alwaysConflictStore always returns ErrConflict on PutVersioned to exhaust retries.
|
||||
type alwaysConflictStore struct {
|
||||
storage.KVStore
|
||||
PortfolioStore
|
||||
attempts int
|
||||
}
|
||||
|
||||
func (s *alwaysConflictStore) GetVersioned(ctx context.Context, key string) ([]byte, int64, error) {
|
||||
return s.KVStore.(storage.VersionedStore).GetVersioned(ctx, key)
|
||||
}
|
||||
|
||||
func (s *alwaysConflictStore) PutVersioned(context.Context, string, int64, []byte) error {
|
||||
func (s *alwaysConflictStore) PutVersioned(_ context.Context, _ string, _ int64, _ Portfolio) error {
|
||||
s.attempts++
|
||||
return storage.ErrConflict
|
||||
}
|
||||
|
||||
func TestUpdatePortfolioReturnsConflictAfterExhaustingRetries(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := &alwaysConflictStore{KVStore: storage.NewMemoryKVStore()}
|
||||
_, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
|
||||
store := &alwaysConflictStore{PortfolioStore: newGoldStore()}
|
||||
_, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
|
||||
p.AddVND(5)
|
||||
return nil
|
||||
})
|
||||
if !errors.Is(err, storage.ErrConflict) {
|
||||
t.Fatalf("UpdatePortfolio: got %v, want wrapped ErrConflict", err)
|
||||
}
|
||||
if kv.attempts != portfolioUpdateAttempts {
|
||||
t.Errorf("CAS attempts = %d, want %d", kv.attempts, portfolioUpdateAttempts)
|
||||
if store.attempts != portfolioUpdateAttempts {
|
||||
t.Errorf("CAS attempts = %d, want %d", store.attempts, portfolioUpdateAttempts)
|
||||
}
|
||||
if _, err := kv.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
|
||||
if _, _, err := store.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Errorf("exhausted update must not persist anything, Get = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// countingCASStore counts PutVersioned calls and delegates to the underlying store.
|
||||
type countingCASStore struct {
|
||||
*storage.MemoryKVStore
|
||||
PortfolioStore
|
||||
casCalls int
|
||||
}
|
||||
|
||||
func (s *countingCASStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error {
|
||||
func (s *countingCASStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val Portfolio) error {
|
||||
s.casCalls++
|
||||
return s.MemoryKVStore.PutVersioned(ctx, key, expectedVersion, val)
|
||||
return s.PortfolioStore.PutVersioned(ctx, key, expectedVersion, val)
|
||||
}
|
||||
|
||||
func TestUpdatePortfolioMutateErrorDoesNotRetryOrPersist(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := &countingCASStore{MemoryKVStore: storage.NewMemoryKVStore()}
|
||||
store := &countingCASStore{PortfolioStore: newGoldStore()}
|
||||
mutateCalls := 0
|
||||
_, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
|
||||
_, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
|
||||
mutateCalls++
|
||||
return errInsufficientVND
|
||||
})
|
||||
@@ -171,17 +172,17 @@ func TestUpdatePortfolioMutateErrorDoesNotRetryOrPersist(t *testing.T) {
|
||||
if mutateCalls != 1 {
|
||||
t.Errorf("mutate calls = %d, want 1 (business errors must not retry)", mutateCalls)
|
||||
}
|
||||
if kv.casCalls != 0 {
|
||||
t.Errorf("CAS calls = %d, want 0 (failed mutate must not write)", kv.casCalls)
|
||||
if store.casCalls != 0 {
|
||||
t.Errorf("CAS calls = %d, want 0 (failed mutate must not write)", store.casCalls)
|
||||
}
|
||||
if _, err := kv.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
|
||||
if _, _, err := store.Get(ctx, "user:7"); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Errorf("failed mutate must not persist anything, Get = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpdatePortfolioConcurrentIncrementsLoseNoUpdates(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newGoldStore()
|
||||
const goroutines = 16
|
||||
var successes atomic.Int64
|
||||
var wg sync.WaitGroup
|
||||
@@ -190,7 +191,7 @@ func TestUpdatePortfolioConcurrentIncrementsLoseNoUpdates(t *testing.T) {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
_, err := UpdatePortfolio(ctx, kv, 7, 1, func(p *Portfolio) error {
|
||||
_, err := UpdatePortfolio(ctx, store, 7, 1, func(p *Portfolio) error {
|
||||
p.AddVND(1000)
|
||||
return nil
|
||||
})
|
||||
@@ -211,7 +212,7 @@ func TestUpdatePortfolioConcurrentIncrementsLoseNoUpdates(t *testing.T) {
|
||||
if successes.Load() == 0 {
|
||||
t.Fatal("no update succeeded")
|
||||
}
|
||||
loaded, err := LoadPortfolio(ctx, kv, 7, 1)
|
||||
loaded, err := LoadPortfolio(ctx, store, 7, 1)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadPortfolio: %v", err)
|
||||
}
|
||||
@@ -220,61 +221,38 @@ func TestUpdatePortfolioConcurrentIncrementsLoseNoUpdates(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// plainKVStore hides the embedded store's versioned methods to model a backend
|
||||
// without optimistic-locking support.
|
||||
type plainKVStore struct {
|
||||
storage.KVStore
|
||||
}
|
||||
|
||||
func TestUpdatePortfolioFailsFastWhenCASUnsupported(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
mutate := func(p *Portfolio) error {
|
||||
p.AddVND(5)
|
||||
return nil
|
||||
}
|
||||
|
||||
// A bare non-CAS store is rejected by the capability check.
|
||||
if _, err := UpdatePortfolio(ctx, &plainKVStore{KVStore: storage.NewMemoryKVStore()}, 7, 1, mutate); err == nil {
|
||||
t.Error("UpdatePortfolio on non-CAS store: want error, got nil")
|
||||
}
|
||||
|
||||
// A prefixed wrapper around a non-CAS store must fail fast as unsupported,
|
||||
// not burn retries on a fake conflict.
|
||||
_, err := UpdatePortfolio(ctx, storage.Prefixed(&plainKVStore{KVStore: storage.NewMemoryKVStore()}, "gold"), 7, 1, mutate)
|
||||
if !errors.Is(err, errors.ErrUnsupported) {
|
||||
t.Errorf("got %v, want errors.ErrUnsupported", err)
|
||||
}
|
||||
if errors.Is(err, storage.ErrConflict) {
|
||||
t.Error("missing CAS capability must not be reported as a write conflict")
|
||||
}
|
||||
}
|
||||
|
||||
func TestStockAndGoldPortfolioKeysDoNotCollide(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
provider := storage.NewMemoryProvider()
|
||||
|
||||
goldStore := storage.Typed[Portfolio](provider.Collection("gold"))
|
||||
goldPortfolio := NewPortfolio(1)
|
||||
goldPortfolio.AddLuong(2)
|
||||
if err := SavePortfolio(ctx, provider.For("gold"), 7, goldPortfolio); err != nil {
|
||||
if err := SavePortfolio(ctx, goldStore, 7, goldPortfolio); err != nil {
|
||||
t.Fatalf("save gold: %v", err)
|
||||
}
|
||||
|
||||
stockStore := storage.Typed[stockmod.Portfolio](provider.Collection("stock"))
|
||||
stockPortfolio := stockmod.NewPortfolio(1)
|
||||
stockPortfolio.AddAsset("TCB", 100)
|
||||
if err := stockmod.SavePortfolio(ctx, provider.For("stock"), 7, stockPortfolio); err != nil {
|
||||
if err := stockmod.SavePortfolio(ctx, stockStore, 7, stockPortfolio); err != nil {
|
||||
t.Fatalf("save stock: %v", err)
|
||||
}
|
||||
keys, err := provider.Base().List(ctx, "")
|
||||
|
||||
// Both stores share the same provider but different collections (module
|
||||
// isolation); verify each module's key is visible only in its own collection.
|
||||
goldKeys, err := goldStore.List(ctx, "")
|
||||
if err != nil {
|
||||
t.Fatalf("list: %v", err)
|
||||
t.Fatalf("list gold: %v", err)
|
||||
}
|
||||
want := map[string]bool{"gold:user:7": false, "stock:user:7": false}
|
||||
for _, key := range keys {
|
||||
if _, ok := want[key]; ok {
|
||||
want[key] = true
|
||||
}
|
||||
stockKeys, err := stockStore.List(ctx, "")
|
||||
if err != nil {
|
||||
t.Fatalf("list stock: %v", err)
|
||||
}
|
||||
for key, seen := range want {
|
||||
if !seen {
|
||||
t.Fatalf("missing raw key %q in %v", key, keys)
|
||||
}
|
||||
if len(goldKeys) != 1 || goldKeys[0] != "user:7" {
|
||||
t.Fatalf("gold keys: got %v, want [user:7]", goldKeys)
|
||||
}
|
||||
if len(stockKeys) != 1 || stockKeys[0] != "user:7" {
|
||||
t.Fatalf("stock keys: got %v, want [user:7]", stockKeys)
|
||||
}
|
||||
}
|
||||
@@ -25,14 +25,20 @@ const (
|
||||
vnappmobHTTPTimeout = 3 * time.Second // kept under the handler deadline; see chathelper.FetchContext
|
||||
)
|
||||
|
||||
// apiKeyCache is the persisted shape for the cached VNAppMob JWT.
|
||||
type apiKeyCache struct {
|
||||
Token string `json:"token" bson:"token"`
|
||||
Exp int64 `json:"exp" bson:"exp"`
|
||||
}
|
||||
|
||||
// VNAppMobClient fetches Vietnam SJC gold prices from api.vnappmob.com.
|
||||
// It self-manages a free JWT API key, caching it in KV and refreshing it
|
||||
// before expiry or when the SJC endpoint returns 403.
|
||||
// It self-manages a free JWT API key, caching it in the typed store and
|
||||
// refreshing it before expiry or when the SJC endpoint returns 403.
|
||||
type VNAppMobClient struct {
|
||||
HTTP *http.Client
|
||||
BaseURL string // optional override; default https://api.vnappmob.com
|
||||
Token string // optional env override (GOLD_VNAPP_API_KEY)
|
||||
KV storage.KVStore // module-scoped KV store
|
||||
BaseURL string // optional override; default https://api.vnappmob.com
|
||||
Token string // optional env override (GOLD_VNAPP_API_KEY)
|
||||
cache storage.DocStore[apiKeyCache] // module-scoped typed cache store
|
||||
|
||||
nowFn func() time.Time
|
||||
mu sync.Mutex
|
||||
@@ -42,11 +48,11 @@ type VNAppMobClient struct {
|
||||
// GOLD_VNAPP_API_KEY from the environment. The API key env var is intended
|
||||
// for local dev or SSM injection; when empty the client refreshes the key
|
||||
// automatically via the VNAppMob refresh endpoint.
|
||||
func NewVNAppMobClientFromEnv(kv storage.KVStore) *VNAppMobClient {
|
||||
func NewVNAppMobClientFromEnv(coll storage.Collection) *VNAppMobClient {
|
||||
return &VNAppMobClient{
|
||||
BaseURL: strings.TrimSpace(os.Getenv("GOLD_VNAPP_API_URL")),
|
||||
Token: strings.TrimSpace(os.Getenv("GOLD_VNAPP_API_KEY")),
|
||||
KV: kv,
|
||||
cache: storage.Typed[apiKeyCache](coll),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -161,8 +167,8 @@ type httpStatusError struct {
|
||||
|
||||
func (e *httpStatusError) Error() string { return e.msg }
|
||||
|
||||
// getKey returns a valid API key, refreshing from KV or the remote endpoint
|
||||
// when the current key is missing or close to expiry.
|
||||
// getKey returns a valid API key, refreshing from the typed cache store or the
|
||||
// remote endpoint when the current key is missing or close to expiry.
|
||||
func (c *VNAppMobClient) getKey(ctx context.Context) (string, error) {
|
||||
if c.Token != "" {
|
||||
return c.Token, nil
|
||||
@@ -171,11 +177,8 @@ func (c *VNAppMobClient) getKey(ctx context.Context) (string, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
|
||||
var cached struct {
|
||||
Token string `json:"token"`
|
||||
Exp int64 `json:"exp"`
|
||||
}
|
||||
if err := c.KV.GetJSON(ctx, vnappmobKeyCacheKey, &cached); err == nil {
|
||||
cached, _, err := c.cache.Get(ctx, vnappmobKeyCacheKey)
|
||||
if err == nil {
|
||||
if cached.Token != "" && !c.isExpired(cached.Exp) {
|
||||
return cached.Token, nil
|
||||
}
|
||||
@@ -187,8 +190,9 @@ func (c *VNAppMobClient) getKey(ctx context.Context) (string, error) {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// Re-read from KV to get the freshly stored key.
|
||||
if err := c.KV.GetJSON(ctx, vnappmobKeyCacheKey, &cached); err != nil {
|
||||
// Re-read from cache to get the freshly stored key.
|
||||
cached, _, err = c.cache.Get(ctx, vnappmobKeyCacheKey)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("vnappmob: read refreshed key: %w", err)
|
||||
}
|
||||
if cached.Token == "" {
|
||||
@@ -197,7 +201,7 @@ func (c *VNAppMobClient) getKey(ctx context.Context) (string, error) {
|
||||
return cached.Token, nil
|
||||
}
|
||||
|
||||
// refreshKey acquires a new JWT from VNAppMob and stores it in KV.
|
||||
// refreshKey acquires a new JWT from VNAppMob and stores it in the typed cache.
|
||||
// It is safe to call concurrently; last-write-wins is acceptable because all
|
||||
// callers receive a valid key.
|
||||
func (c *VNAppMobClient) refreshKey(ctx context.Context) error {
|
||||
@@ -256,10 +260,7 @@ func (c *VNAppMobClient) refreshKeyLocked(ctx context.Context) error {
|
||||
// Store anyway; next call will refresh if expiry cannot be verified.
|
||||
}
|
||||
|
||||
if err := c.KV.PutJSON(ctx, vnappmobKeyCacheKey, struct {
|
||||
Token string `json:"token"`
|
||||
Exp int64 `json:"exp"`
|
||||
}{Token: token, Exp: exp}); err != nil {
|
||||
if err := c.cache.Put(ctx, vnappmobKeyCacheKey, apiKeyCache{Token: token, Exp: exp}); err != nil {
|
||||
return fmt.Errorf("vnappmob: cache key: %w", err)
|
||||
}
|
||||
log.Info("vnappmob_key_refreshed", "exp", exp)
|
||||
|
||||
@@ -16,11 +16,11 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
func newTestVNAppMobClient(srv *httptest.Server, kv storage.KVStore) *VNAppMobClient {
|
||||
func newTestVNAppMobClient(srv *httptest.Server, coll storage.Collection) *VNAppMobClient {
|
||||
return &VNAppMobClient{
|
||||
HTTP: srv.Client(),
|
||||
BaseURL: srv.URL,
|
||||
KV: kv,
|
||||
cache: storage.Typed[apiKeyCache](coll),
|
||||
nowFn: func() time.Time { return time.Unix(1000, 0) },
|
||||
}
|
||||
}
|
||||
@@ -33,6 +33,10 @@ func makeJWT(exp int64) string {
|
||||
return h + "." + p + ".dummy-signature"
|
||||
}
|
||||
|
||||
func newTestColl() storage.Collection {
|
||||
return storage.NewMemoryProvider().Collection("gold")
|
||||
}
|
||||
|
||||
func TestJWTExp(t *testing.T) {
|
||||
now := time.Unix(1000, 0)
|
||||
tests := []struct {
|
||||
@@ -114,8 +118,7 @@ func TestRefreshKey(t *testing.T) {
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
kv := storage.NewMemoryKVStore()
|
||||
c := newTestVNAppMobClient(srv, kv)
|
||||
c := newTestVNAppMobClient(srv, newTestColl())
|
||||
|
||||
key, err := c.getKey(context.Background())
|
||||
if err != nil {
|
||||
@@ -152,7 +155,7 @@ func TestRefreshKey_ObjectWrapper(t *testing.T) {
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
c := newTestVNAppMobClient(srv, storage.NewMemoryKVStore())
|
||||
c := newTestVNAppMobClient(srv, newTestColl())
|
||||
key, err := c.getKey(context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("getKey: %v", err)
|
||||
@@ -170,7 +173,7 @@ func TestRefreshKey_QuotedString(t *testing.T) {
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
c := newTestVNAppMobClient(srv, storage.NewMemoryKVStore())
|
||||
c := newTestVNAppMobClient(srv, newTestColl())
|
||||
key, err := c.getKey(context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("getKey: %v", err)
|
||||
@@ -198,8 +201,7 @@ func TestFetchSJCPrice(t *testing.T) {
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
kv := storage.NewMemoryKVStore()
|
||||
c := newTestVNAppMobClient(srv, kv)
|
||||
c := newTestVNAppMobClient(srv, newTestColl())
|
||||
buy, sell, err := c.FetchSJCPrice(context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("FetchSJCPrice: %v", err)
|
||||
@@ -232,7 +234,7 @@ func TestFetchSJCPrice_401Or403Refreshes(t *testing.T) {
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
c := newTestVNAppMobClient(srv, storage.NewMemoryKVStore())
|
||||
c := newTestVNAppMobClient(srv, newTestColl())
|
||||
buy, sell, err := c.FetchSJCPrice(context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("FetchSJCPrice: %v", err)
|
||||
@@ -265,8 +267,7 @@ func TestFetchSJCPrice_FallbackError(t *testing.T) {
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
kv := storage.NewMemoryKVStore()
|
||||
c := newTestVNAppMobClient(srv, kv)
|
||||
c := newTestVNAppMobClient(srv, newTestColl())
|
||||
_, _, err := c.FetchSJCPrice(context.Background())
|
||||
if !errors.Is(err, ErrNoGoldPrice) {
|
||||
t.Fatalf("got %v, want ErrNoGoldPrice", err)
|
||||
@@ -294,7 +295,7 @@ func TestFetchSJCPrice_InvalidValues(t *testing.T) {
|
||||
}
|
||||
}))
|
||||
defer srv.Close()
|
||||
c := newTestVNAppMobClient(srv, storage.NewMemoryKVStore())
|
||||
c := newTestVNAppMobClient(srv, newTestColl())
|
||||
_, _, err := c.FetchSJCPrice(context.Background())
|
||||
if err == nil {
|
||||
t.Fatal("expected error for invalid values")
|
||||
|
||||
@@ -12,7 +12,6 @@ import (
|
||||
|
||||
"github.com/tiennm99/miti99bot/internal/keylock"
|
||||
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
const newRoundHint = "🆕 Send <code>/loldle</code> or <code>/loldle <champion></code> to start a new round."
|
||||
@@ -20,7 +19,9 @@ const newRoundHint = "🆕 Send <code>/loldle</code> or <code>/loldle <champi
|
||||
// state captures everything a loldle handler needs at runtime. Built once
|
||||
// per Factory call and shared across the four command closures.
|
||||
type state struct {
|
||||
kv storage.KVStore
|
||||
games GameStore
|
||||
stats StatsStore
|
||||
cfg ConfigStore
|
||||
champions []Champion
|
||||
locks keylock.Map // serialises Get→mutate→Put per subject
|
||||
}
|
||||
@@ -63,7 +64,7 @@ func (s *state) startFreshGame(ctx context.Context, subject string) (*gameState,
|
||||
Guesses: []string{},
|
||||
StartedAt: nil,
|
||||
}
|
||||
if err := saveGame(ctx, s.kv, subject, g); err != nil {
|
||||
if err := saveGame(ctx, s.games, subject, g); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return g, nil
|
||||
@@ -72,7 +73,7 @@ func (s *state) startFreshGame(ctx context.Context, subject string) (*gameState,
|
||||
// getOrInitGame loads the active round; if absent (or the saved round has
|
||||
// already exhausted its budget — defensive), starts a fresh one.
|
||||
func (s *state) getOrInitGame(ctx context.Context, subject string, maxGuesses int) (*gameState, error) {
|
||||
existing, err := loadGame(ctx, s.kv, subject)
|
||||
existing, err := loadGame(ctx, s.games, subject)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -113,7 +114,7 @@ func (s *state) handleLoldle(ctx context.Context, b *bot.Bot, update *models.Upd
|
||||
defer s.locks.Acquire(subject)()
|
||||
arg := chathelper.ArgAfterCommand(msg.Text)
|
||||
|
||||
maxGuesses, err := getMaxGuesses(ctx, s.kv, subject)
|
||||
maxGuesses, err := getMaxGuesses(ctx, s.cfg, subject)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -146,7 +147,7 @@ func (s *state) handleLoldle(ctx context.Context, b *bot.Bot, update *models.Upd
|
||||
if target == nil {
|
||||
// champions.json was refreshed and the target is gone. Drop the
|
||||
// round so the next /loldle starts cleanly.
|
||||
if err := clearGame(ctx, s.kv, subject); err != nil {
|
||||
if err := clearGame(ctx, s.games, subject); err != nil {
|
||||
return err
|
||||
}
|
||||
return chathelper.ReplyHTML(ctx, b, msg,
|
||||
@@ -167,11 +168,11 @@ func (s *state) handleLoldle(ctx context.Context, b *bot.Bot, update *models.Upd
|
||||
|
||||
switch {
|
||||
case won:
|
||||
st, err := recordResult(ctx, s.kv, subject, true)
|
||||
st, err := recordResult(ctx, s.stats, subject, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := clearGame(ctx, s.kv, subject); err != nil {
|
||||
if err := clearGame(ctx, s.games, subject); err != nil {
|
||||
return err
|
||||
}
|
||||
trySendSticker(ctx, b, msg, winStickers)
|
||||
@@ -181,10 +182,10 @@ func (s *state) handleLoldle(ctx context.Context, b *bot.Bot, update *models.Upd
|
||||
rendered, flavor, champ, elapsed, st.Streak, len(game.Guesses), maxGuesses, newRoundHint))
|
||||
|
||||
case len(game.Guesses) >= maxGuesses:
|
||||
if _, err := recordResult(ctx, s.kv, subject, false); err != nil {
|
||||
if _, err := recordResult(ctx, s.stats, subject, false); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := clearGame(ctx, s.kv, subject); err != nil {
|
||||
if err := clearGame(ctx, s.games, subject); err != nil {
|
||||
return err
|
||||
}
|
||||
trySendSticker(ctx, b, msg, loseStickers)
|
||||
@@ -192,7 +193,7 @@ func (s *state) handleLoldle(ctx context.Context, b *bot.Bot, update *models.Upd
|
||||
"%s\n\n❌ Out of guesses. Answer was %s.\n%s", rendered, champ, newRoundHint))
|
||||
|
||||
default:
|
||||
if err := saveGame(ctx, s.kv, subject, game); err != nil {
|
||||
if err := saveGame(ctx, s.games, subject, game); err != nil {
|
||||
return err
|
||||
}
|
||||
return chathelper.ReplyHTML(ctx, b, msg, fmt.Sprintf(
|
||||
@@ -212,18 +213,18 @@ func (s *state) handleGiveup(ctx context.Context, b *bot.Bot, update *models.Upd
|
||||
}
|
||||
defer s.locks.Acquire(subject)()
|
||||
|
||||
existing, err := loadGame(ctx, s.kv, subject)
|
||||
existing, err := loadGame(ctx, s.games, subject)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if existing == nil {
|
||||
return chathelper.ReplyHTML(ctx, b, msg, "No active round. "+newRoundHint)
|
||||
}
|
||||
if _, err := recordResult(ctx, s.kv, subject, false); err != nil {
|
||||
if _, err := recordResult(ctx, s.stats, subject, false); err != nil {
|
||||
return err
|
||||
}
|
||||
target := s.findByName(existing.Target)
|
||||
if err := clearGame(ctx, s.kv, subject); err != nil {
|
||||
if err := clearGame(ctx, s.games, subject); err != nil {
|
||||
return err
|
||||
}
|
||||
trySendSticker(ctx, b, msg, giveupStickers)
|
||||
@@ -245,7 +246,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
|
||||
if subject == "" {
|
||||
return chathelper.Reply(ctx, b, msg, "Cannot identify chat.")
|
||||
}
|
||||
st, err := loadStats(ctx, s.kv, subject)
|
||||
st, err := loadStats(ctx, s.stats, subject)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -274,7 +275,7 @@ func (s *state) handleSetMax(ctx context.Context, b *bot.Bot, update *models.Upd
|
||||
if err != nil || n < 1 || n > MaxGuessesCap {
|
||||
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("Usage: /loldle_setmax <1-%d>", MaxGuessesCap))
|
||||
}
|
||||
if err := setMaxGuesses(ctx, s.kv, subject, n); err != nil {
|
||||
if err := setMaxGuesses(ctx, s.cfg, subject, n); err != nil {
|
||||
return err
|
||||
}
|
||||
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("✅ Loldle max guesses set to %d (applies to the next round).", n))
|
||||
|
||||
@@ -13,12 +13,13 @@ import (
|
||||
// installLoldle wires the loldle module + auth (owner gates /loldle_setmax,
|
||||
// which is private). seedTarget pre-seeds a game so guess outcomes are
|
||||
// deterministic without hooking math/rand.
|
||||
func installLoldle(t *testing.T, ownerID int64, seedSubject, seedTarget string) (*testutil.RecordingBot, storage.KVStore) {
|
||||
func installLoldle(t *testing.T, ownerID int64, seedSubject, seedTarget string) (*testutil.RecordingBot, GameStore, ConfigStore) {
|
||||
t.Helper()
|
||||
rb := testutil.NewRecordingBot(t)
|
||||
provider := storage.NewMemoryProvider()
|
||||
kv := provider.For("loldle")
|
||||
mod := New(modules.Deps{KV: kv})
|
||||
col := storage.NewMemoryProvider().Collection("loldle")
|
||||
games := storage.Typed[gameState](col)
|
||||
cfg := storage.Typed[roundConfig](col)
|
||||
mod := New(modules.Deps{Store: col})
|
||||
reg := &modules.Registry{
|
||||
Modules: []modules.Module{{Name: "loldle", Commands: mod.Commands}},
|
||||
AllCommands: map[string]modules.Command{},
|
||||
@@ -30,15 +31,15 @@ func installLoldle(t *testing.T, ownerID int64, seedSubject, seedTarget string)
|
||||
|
||||
if seedTarget != "" {
|
||||
g := &gameState{Target: seedTarget, Guesses: []string{}}
|
||||
if err := saveGame(context.Background(), kv, seedSubject, g); err != nil {
|
||||
if err := saveGame(context.Background(), games, seedSubject, g); err != nil {
|
||||
t.Fatalf("seed game: %v", err)
|
||||
}
|
||||
}
|
||||
return rb, kv
|
||||
return rb, games, cfg
|
||||
}
|
||||
|
||||
func TestLoldle_NoArgShowsBoard(t *testing.T) {
|
||||
rb, _ := installLoldle(t, 0, "1", "Aatrox")
|
||||
rb, _, _ := installLoldle(t, 0, "1", "Aatrox")
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle"))
|
||||
|
||||
got := rb.LastSent().Text()
|
||||
@@ -48,7 +49,7 @@ func TestLoldle_NoArgShowsBoard(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestLoldle_Win(t *testing.T) {
|
||||
rb, _ := installLoldle(t, 0, "1", "Aatrox")
|
||||
rb, _, _ := installLoldle(t, 0, "1", "Aatrox")
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle aatrox"))
|
||||
|
||||
got := rb.LastSent().Text()
|
||||
@@ -61,7 +62,7 @@ func TestLoldle_Win(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestLoldle_UnknownChampion(t *testing.T) {
|
||||
rb, _ := installLoldle(t, 0, "1", "Aatrox")
|
||||
rb, _, _ := installLoldle(t, 0, "1", "Aatrox")
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle ZilbeanZ"))
|
||||
|
||||
got := rb.LastSent().Text()
|
||||
@@ -71,7 +72,7 @@ func TestLoldle_UnknownChampion(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestLoldle_DuplicateGuessRejected(t *testing.T) {
|
||||
rb, _ := installLoldle(t, 0, "1", "Aatrox")
|
||||
rb, _, _ := installLoldle(t, 0, "1", "Aatrox")
|
||||
// First guess: Ahri (not the target — round continues).
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle ahri"))
|
||||
rb.Reset()
|
||||
@@ -85,7 +86,7 @@ func TestLoldle_DuplicateGuessRejected(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestLoldleGiveup_RevealsAnswer(t *testing.T) {
|
||||
rb, _ := installLoldle(t, 0, "1", "Aatrox")
|
||||
rb, _, _ := installLoldle(t, 0, "1", "Aatrox")
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle_giveup"))
|
||||
|
||||
got := rb.LastSent().Text()
|
||||
@@ -95,7 +96,7 @@ func TestLoldleGiveup_RevealsAnswer(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestLoldleGiveup_NoActiveRound(t *testing.T) {
|
||||
rb, _ := installLoldle(t, 0, "", "")
|
||||
rb, _, _ := installLoldle(t, 0, "", "")
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle_giveup"))
|
||||
|
||||
got := rb.LastSent().Text()
|
||||
@@ -105,7 +106,7 @@ func TestLoldleGiveup_NoActiveRound(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestLoldleStats_Empty(t *testing.T) {
|
||||
rb, _ := installLoldle(t, 0, "", "")
|
||||
rb, _, _ := installLoldle(t, 0, "", "")
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(1, "/loldle_stats"))
|
||||
|
||||
got := rb.LastSent().Text()
|
||||
@@ -117,24 +118,24 @@ func TestLoldleStats_Empty(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestLoldleSetMax_OwnerSucceeds(t *testing.T) {
|
||||
rb, kv := installLoldle(t, 999, "", "")
|
||||
rb, _, cfg := installLoldle(t, 999, "", "")
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(999, "/loldle_setmax 5"))
|
||||
|
||||
got := rb.LastSent().Text()
|
||||
if !strings.Contains(got, "max guesses set to 5") {
|
||||
t.Errorf("/loldle_setmax reply: %q", got)
|
||||
}
|
||||
var cfg roundConfig
|
||||
if err := kv.GetJSON(context.Background(), configKey("999"), &cfg); err != nil {
|
||||
c, _, err := cfg.Get(context.Background(), configKey("999"))
|
||||
if err != nil {
|
||||
t.Fatalf("expected config persisted: %v", err)
|
||||
}
|
||||
if cfg.MaxGuesses != 5 {
|
||||
t.Errorf("MaxGuesses persisted = %d, want 5", cfg.MaxGuesses)
|
||||
if c.MaxGuesses != 5 {
|
||||
t.Errorf("MaxGuesses persisted = %d, want 5", c.MaxGuesses)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoldleSetMax_DeniedToNonOwner(t *testing.T) {
|
||||
rb, _ := installLoldle(t, 999, "", "")
|
||||
rb, _, _ := installLoldle(t, 999, "", "")
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(7, "/loldle_setmax 5"))
|
||||
|
||||
if calls := rb.Sent(); len(calls) != 0 {
|
||||
@@ -143,7 +144,7 @@ func TestLoldleSetMax_DeniedToNonOwner(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestLoldleSetMax_RejectsOutOfRange(t *testing.T) {
|
||||
rb, _ := installLoldle(t, 999, "", "")
|
||||
rb, _, _ := installLoldle(t, 999, "", "")
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(999, "/loldle_setmax 99"))
|
||||
|
||||
got := rb.LastSent().Text()
|
||||
|
||||
@@ -2,13 +2,16 @@ package loldle
|
||||
|
||||
import (
|
||||
"github.com/tiennm99/miti99bot/internal/modules"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// New is the loldle module Factory. Loads champions.json once at construction
|
||||
// and shares the parsed slice + per-subject lock map across all handlers.
|
||||
func New(deps modules.Deps) modules.Module {
|
||||
s := &state{
|
||||
kv: deps.KV,
|
||||
games: storage.Typed[gameState](deps.Store),
|
||||
stats: storage.Typed[stats](deps.Store),
|
||||
cfg: storage.Typed[roundConfig](deps.Store),
|
||||
champions: loadChampions(),
|
||||
}
|
||||
return modules.Module{
|
||||
|
||||
@@ -15,7 +15,7 @@ const (
|
||||
MaxGuessesCap = 10
|
||||
)
|
||||
|
||||
// gameState is the per-subject KV record.
|
||||
// gameState is the per-subject record.
|
||||
//
|
||||
// StartedAt is *int64 because the timer doesn't start until the first guess;
|
||||
// using time.Time would marshal as "0001-01-01T00:00:00Z" instead of null
|
||||
@@ -25,35 +25,43 @@ const (
|
||||
// time against current champions.json so a weekly data refresh updates
|
||||
// historical board displays without migrating saved rounds.
|
||||
type gameState struct {
|
||||
Target string `json:"target"`
|
||||
Guesses []string `json:"guesses"`
|
||||
StartedAt *int64 `json:"startedAt"` // ms-since-epoch | null
|
||||
Target string `json:"target" bson:"target"`
|
||||
Guesses []string `json:"guesses" bson:"guesses"`
|
||||
StartedAt *int64 `json:"startedAt" bson:"startedAt"` // ms-since-epoch | null
|
||||
}
|
||||
|
||||
// stats lifetime score. No LastResultAt field by design (differs from
|
||||
// wordle's Stats — loldle only ever needed running streaks, not "last
|
||||
// played at").
|
||||
type stats struct {
|
||||
Played int `json:"played"`
|
||||
Wins int `json:"wins"`
|
||||
Streak int `json:"streak"`
|
||||
BestStreak int `json:"bestStreak"`
|
||||
Played int `json:"played" bson:"played"`
|
||||
Wins int `json:"wins" bson:"wins"`
|
||||
Streak int `json:"streak" bson:"streak"`
|
||||
BestStreak int `json:"bestStreak" bson:"bestStreak"`
|
||||
}
|
||||
|
||||
// roundConfig stores the per-subject MaxGuesses override. Stored only when
|
||||
// /loldle_setmax has been run; the absence of this record means "use default".
|
||||
type roundConfig struct {
|
||||
MaxGuesses int `json:"maxGuesses"`
|
||||
MaxGuesses int `json:"maxGuesses" bson:"maxGuesses"`
|
||||
}
|
||||
|
||||
// GameStore is the loldle module's typed game store.
|
||||
type GameStore = storage.DocStore[gameState]
|
||||
|
||||
// StatsStore is the loldle module's typed stats store.
|
||||
type StatsStore = storage.DocStore[stats]
|
||||
|
||||
// ConfigStore is the loldle module's typed config store.
|
||||
type ConfigStore = storage.DocStore[roundConfig]
|
||||
|
||||
func gameKey(subject string) string { return "game:" + subject }
|
||||
func statsKey(subject string) string { return "stats:" + subject }
|
||||
func configKey(subject string) string { return "config:" + subject }
|
||||
|
||||
// loadGame returns the active round, or (nil, nil) if none exists.
|
||||
func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*gameState, error) {
|
||||
var g gameState
|
||||
err := kv.GetJSON(ctx, gameKey(subject), &g)
|
||||
func loadGame(ctx context.Context, games GameStore, subject string) (*gameState, error) {
|
||||
g, _, err := games.Get(ctx, gameKey(subject))
|
||||
switch {
|
||||
case err == nil:
|
||||
return &g, nil
|
||||
@@ -64,8 +72,8 @@ func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*gameSta
|
||||
}
|
||||
}
|
||||
|
||||
func saveGame(ctx context.Context, kv storage.KVStore, subject string, g *gameState) error {
|
||||
if err := kv.PutJSON(ctx, gameKey(subject), g); err != nil {
|
||||
func saveGame(ctx context.Context, games GameStore, subject string, g *gameState) error {
|
||||
if err := games.Put(ctx, gameKey(subject), *g); err != nil {
|
||||
return fmt.Errorf("loldle saveGame: %w", err)
|
||||
}
|
||||
return nil
|
||||
@@ -74,8 +82,8 @@ func saveGame(ctx context.Context, kv storage.KVStore, subject string, g *gameSt
|
||||
// clearGame removes the round so the next /loldle starts fresh. Used after
|
||||
// win / loss / giveup; the new round's timer should start on the player's
|
||||
// next interaction, not at the moment the previous round ended.
|
||||
func clearGame(ctx context.Context, kv storage.KVStore, subject string) error {
|
||||
if err := kv.Delete(ctx, gameKey(subject)); err != nil {
|
||||
func clearGame(ctx context.Context, games GameStore, subject string) error {
|
||||
if err := games.Delete(ctx, gameKey(subject)); err != nil {
|
||||
return fmt.Errorf("loldle clearGame: %w", err)
|
||||
}
|
||||
return nil
|
||||
@@ -83,9 +91,8 @@ func clearGame(ctx context.Context, kv storage.KVStore, subject string) error {
|
||||
|
||||
// loadStats returns lifetime score; missing → fresh-zero record so callers
|
||||
// never need a nil check.
|
||||
func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*stats, error) {
|
||||
var s stats
|
||||
err := kv.GetJSON(ctx, statsKey(subject), &s)
|
||||
func loadStats(ctx context.Context, st StatsStore, subject string) (*stats, error) {
|
||||
s, _, err := st.Get(ctx, statsKey(subject))
|
||||
switch {
|
||||
case err == nil:
|
||||
return &s, nil
|
||||
@@ -96,8 +103,8 @@ func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*stats,
|
||||
}
|
||||
}
|
||||
|
||||
func recordResult(ctx context.Context, kv storage.KVStore, subject string, won bool) (*stats, error) {
|
||||
s, err := loadStats(ctx, kv, subject)
|
||||
func recordResult(ctx context.Context, st StatsStore, subject string, won bool) (*stats, error) {
|
||||
s, err := loadStats(ctx, st, subject)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -111,7 +118,7 @@ func recordResult(ctx context.Context, kv storage.KVStore, subject string, won b
|
||||
} else {
|
||||
s.Streak = 0
|
||||
}
|
||||
if err := kv.PutJSON(ctx, statsKey(subject), s); err != nil {
|
||||
if err := st.Put(ctx, statsKey(subject), *s); err != nil {
|
||||
return nil, fmt.Errorf("loldle recordResult: %w", err)
|
||||
}
|
||||
return s, nil
|
||||
@@ -120,27 +127,26 @@ func recordResult(ctx context.Context, kv storage.KVStore, subject string, won b
|
||||
// getMaxGuesses returns the effective round length: the per-subject override
|
||||
// if set and in range, otherwise MaxGuesses. Out-of-range values are
|
||||
// silently ignored — better to serve the default than 500 the user.
|
||||
func getMaxGuesses(ctx context.Context, kv storage.KVStore, subject string) (int, error) {
|
||||
var cfg roundConfig
|
||||
err := kv.GetJSON(ctx, configKey(subject), &cfg)
|
||||
func getMaxGuesses(ctx context.Context, cfg ConfigStore, subject string) (int, error) {
|
||||
c, _, err := cfg.Get(ctx, configKey(subject))
|
||||
if err != nil {
|
||||
if errors.Is(err, storage.ErrNotFound) {
|
||||
return MaxGuesses, nil
|
||||
}
|
||||
return 0, fmt.Errorf("loldle getMaxGuesses: %w", err)
|
||||
}
|
||||
if cfg.MaxGuesses < 1 || cfg.MaxGuesses > MaxGuessesCap {
|
||||
if c.MaxGuesses < 1 || c.MaxGuesses > MaxGuessesCap {
|
||||
return MaxGuesses, nil
|
||||
}
|
||||
return cfg.MaxGuesses, nil
|
||||
return c.MaxGuesses, nil
|
||||
}
|
||||
|
||||
// setMaxGuesses validates and persists the per-subject override.
|
||||
func setMaxGuesses(ctx context.Context, kv storage.KVStore, subject string, n int) error {
|
||||
func setMaxGuesses(ctx context.Context, cfg ConfigStore, subject string, n int) error {
|
||||
if n < 1 || n > MaxGuessesCap {
|
||||
return fmt.Errorf("loldle: maxGuesses must be in [1, %d], got %d", MaxGuessesCap, n)
|
||||
}
|
||||
if err := kv.PutJSON(ctx, configKey(subject), roundConfig{MaxGuesses: n}); err != nil {
|
||||
if err := cfg.Put(ctx, configKey(subject), roundConfig{MaxGuesses: n}); err != nil {
|
||||
return fmt.Errorf("loldle setMaxGuesses: %w", err)
|
||||
}
|
||||
return nil
|
||||
|
||||
@@ -8,6 +8,25 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
func newLoldleGames() GameStore {
|
||||
return storage.Typed[gameState](storage.NewMemoryProvider().Collection("loldle"))
|
||||
}
|
||||
|
||||
func newLoldleStats() StatsStore {
|
||||
return storage.Typed[stats](storage.NewMemoryProvider().Collection("loldle"))
|
||||
}
|
||||
|
||||
func newLoldleConfig() ConfigStore {
|
||||
return storage.Typed[roundConfig](storage.NewMemoryProvider().Collection("loldle"))
|
||||
}
|
||||
|
||||
// newLoldleStores returns all three stores backed by the same collection
|
||||
// (disjoint key prefixes: "game:", "stats:", "config:").
|
||||
func newLoldleStores() (GameStore, StatsStore, ConfigStore) {
|
||||
c := storage.NewMemoryProvider().Collection("loldle")
|
||||
return storage.Typed[gameState](c), storage.Typed[stats](c), storage.Typed[roundConfig](c)
|
||||
}
|
||||
|
||||
func TestGameState_StartedAtNullByDefault(t *testing.T) {
|
||||
g := gameState{Target: "Aatrox", Guesses: []string{}}
|
||||
b, err := json.Marshal(g)
|
||||
@@ -43,20 +62,20 @@ func TestStats_NoLastResultAtField(t *testing.T) {
|
||||
|
||||
func TestRecordResult_WinAndLossSequence(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
st := newLoldleStats()
|
||||
|
||||
s, err := recordResult(ctx, kv, "u1", true)
|
||||
s, err := recordResult(ctx, st, "u1", true)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if s.Played != 1 || s.Wins != 1 || s.Streak != 1 || s.BestStreak != 1 {
|
||||
t.Errorf("first win: %+v", s)
|
||||
}
|
||||
s, _ = recordResult(ctx, kv, "u1", true)
|
||||
s, _ = recordResult(ctx, st, "u1", true)
|
||||
if s.Streak != 2 || s.BestStreak != 2 {
|
||||
t.Errorf("two wins: %+v", s)
|
||||
}
|
||||
s, _ = recordResult(ctx, kv, "u1", false)
|
||||
s, _ = recordResult(ctx, st, "u1", false)
|
||||
if s.Streak != 0 {
|
||||
t.Errorf("loss should reset streak, got %d", s.Streak)
|
||||
}
|
||||
@@ -67,52 +86,52 @@ func TestRecordResult_WinAndLossSequence(t *testing.T) {
|
||||
|
||||
func TestGetMaxGuesses_DefaultsAndOverrides(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
cfg := newLoldleConfig()
|
||||
|
||||
if n, _ := getMaxGuesses(ctx, kv, "u1"); n != MaxGuesses {
|
||||
if n, _ := getMaxGuesses(ctx, cfg, "u1"); n != MaxGuesses {
|
||||
t.Errorf("missing config → %d, want %d", n, MaxGuesses)
|
||||
}
|
||||
|
||||
if err := setMaxGuesses(ctx, kv, "u1", 5); err != nil {
|
||||
if err := setMaxGuesses(ctx, cfg, "u1", 5); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n, _ := getMaxGuesses(ctx, kv, "u1"); n != 5 {
|
||||
if n, _ := getMaxGuesses(ctx, cfg, "u1"); n != 5 {
|
||||
t.Errorf("after setMax(5): %d, want 5", n)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetMaxGuesses_ValidatesRange(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
cfg := newLoldleConfig()
|
||||
for _, n := range []int{0, -1, MaxGuessesCap + 1, 100} {
|
||||
if err := setMaxGuesses(ctx, kv, "u1", n); err == nil {
|
||||
if err := setMaxGuesses(ctx, cfg, "u1", n); err == nil {
|
||||
t.Errorf("setMaxGuesses(%d) should error", n)
|
||||
}
|
||||
}
|
||||
if err := setMaxGuesses(ctx, kv, "u1", 1); err != nil {
|
||||
if err := setMaxGuesses(ctx, cfg, "u1", 1); err != nil {
|
||||
t.Errorf("setMaxGuesses(1) should succeed: %v", err)
|
||||
}
|
||||
if err := setMaxGuesses(ctx, kv, "u1", MaxGuessesCap); err != nil {
|
||||
if err := setMaxGuesses(ctx, cfg, "u1", MaxGuessesCap); err != nil {
|
||||
t.Errorf("setMaxGuesses(cap) should succeed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGetMaxGuesses_OutOfRangeIgnored(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
// Inject a corrupt config to simulate manual KV tampering or a stale
|
||||
cfg := newLoldleConfig()
|
||||
// Inject a corrupt config to simulate manual store tampering or a stale
|
||||
// schema; getMaxGuesses must fall back to the default rather than
|
||||
// returning a wild value to handlers.
|
||||
if err := kv.PutJSON(ctx, configKey("u1"), roundConfig{MaxGuesses: 100}); err != nil {
|
||||
if err := cfg.Put(ctx, configKey("u1"), roundConfig{MaxGuesses: 100}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n, _ := getMaxGuesses(ctx, kv, "u1"); n != MaxGuesses {
|
||||
if n, _ := getMaxGuesses(ctx, cfg, "u1"); n != MaxGuesses {
|
||||
t.Errorf("out-of-range config → %d, want %d (default)", n, MaxGuesses)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadGame_MissingReturnsNil(t *testing.T) {
|
||||
g, err := loadGame(context.Background(), storage.NewMemoryKVStore(), "nobody")
|
||||
g, err := loadGame(context.Background(), newLoldleGames(), "nobody")
|
||||
if err != nil {
|
||||
t.Errorf("missing should not error: %v", err)
|
||||
}
|
||||
@@ -123,13 +142,13 @@ func TestLoadGame_MissingReturnsNil(t *testing.T) {
|
||||
|
||||
func TestSaveLoadClearGame_RoundTrip(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
games := newLoldleGames()
|
||||
at := int64(42)
|
||||
want := &gameState{Target: "Aatrox", Guesses: []string{"Ahri"}, StartedAt: &at}
|
||||
if err := saveGame(ctx, kv, "u1", want); err != nil {
|
||||
if err := saveGame(ctx, games, "u1", want); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := loadGame(ctx, kv, "u1")
|
||||
got, err := loadGame(ctx, games, "u1")
|
||||
if err != nil || got == nil {
|
||||
t.Fatalf("loadGame: %v, %v", got, err)
|
||||
}
|
||||
@@ -137,10 +156,10 @@ func TestSaveLoadClearGame_RoundTrip(t *testing.T) {
|
||||
t.Errorf("round-trip mismatch: %+v", got)
|
||||
}
|
||||
|
||||
if err := clearGame(ctx, kv, "u1"); err != nil {
|
||||
if err := clearGame(ctx, games, "u1"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, _ = loadGame(ctx, kv, "u1")
|
||||
got, _ = loadGame(ctx, games, "u1")
|
||||
if got != nil {
|
||||
t.Errorf("after clearGame, expected nil, got %+v", got)
|
||||
}
|
||||
|
||||
@@ -44,50 +44,61 @@ const (
|
||||
)
|
||||
|
||||
// Team is one side of a match. JSON shape matches the lolesports response.
|
||||
// bson tags mirror the json names exactly: this tree is persisted inside
|
||||
// cacheRecord, and the DynamoDB→Mongo migrator preserves the original (camelCase)
|
||||
// JSON keys, so the store must read those keys back verbatim — not the driver's
|
||||
// lowercased default.
|
||||
type Team struct {
|
||||
Name string `json:"name,omitempty"`
|
||||
Code string `json:"code,omitempty"`
|
||||
Image string `json:"image,omitempty"`
|
||||
Result *struct {
|
||||
Outcome string `json:"outcome,omitempty"` // "win" or "loss"
|
||||
GameWins int `json:"gameWins,omitempty"`
|
||||
} `json:"result,omitempty"`
|
||||
Record *struct {
|
||||
Wins int `json:"wins,omitempty"`
|
||||
Losses int `json:"losses,omitempty"`
|
||||
} `json:"record,omitempty"`
|
||||
Name string `json:"name,omitempty" bson:"name,omitempty"`
|
||||
Code string `json:"code,omitempty" bson:"code,omitempty"`
|
||||
Image string `json:"image,omitempty" bson:"image,omitempty"`
|
||||
Result *TeamResult `json:"result,omitempty" bson:"result,omitempty"`
|
||||
Record *TeamRecord `json:"record,omitempty" bson:"record,omitempty"`
|
||||
}
|
||||
|
||||
// TeamResult is a team's per-match outcome (named so tests and the bson decoder
|
||||
// share one type).
|
||||
type TeamResult struct {
|
||||
Outcome string `json:"outcome,omitempty" bson:"outcome,omitempty"` // "win" or "loss"
|
||||
GameWins int `json:"gameWins,omitempty" bson:"gameWins,omitempty"`
|
||||
}
|
||||
|
||||
// TeamRecord is a team's running series record.
|
||||
type TeamRecord struct {
|
||||
Wins int `json:"wins,omitempty" bson:"wins,omitempty"`
|
||||
Losses int `json:"losses,omitempty" bson:"losses,omitempty"`
|
||||
}
|
||||
|
||||
// League holds the league-section-header info on each event.
|
||||
type League struct {
|
||||
Name string `json:"name,omitempty"`
|
||||
Slug string `json:"slug,omitempty"`
|
||||
Image string `json:"image,omitempty"`
|
||||
Name string `json:"name,omitempty" bson:"name,omitempty"`
|
||||
Slug string `json:"slug,omitempty" bson:"slug,omitempty"`
|
||||
Image string `json:"image,omitempty" bson:"image,omitempty"`
|
||||
}
|
||||
|
||||
// Strategy is the bestOf descriptor (Bo1, Bo3, Bo5).
|
||||
type Strategy struct {
|
||||
Type string `json:"type,omitempty"`
|
||||
Count int `json:"count,omitempty"`
|
||||
Type string `json:"type,omitempty" bson:"type,omitempty"`
|
||||
Count int `json:"count,omitempty" bson:"count,omitempty"`
|
||||
}
|
||||
|
||||
// Match is the inner match metadata.
|
||||
type Match struct {
|
||||
ID string `json:"id,omitempty"`
|
||||
Teams []Team `json:"teams,omitempty"`
|
||||
Strategy Strategy `json:"strategy,omitempty"`
|
||||
ID string `json:"id,omitempty" bson:"id,omitempty"`
|
||||
Teams []Team `json:"teams,omitempty" bson:"teams,omitempty"`
|
||||
Strategy Strategy `json:"strategy,omitempty" bson:"strategy,omitempty"`
|
||||
}
|
||||
|
||||
// ScheduleEvent is one upcoming or past match. State is "unstarted",
|
||||
// "inProgress", or "completed". Type is set to "show" for pre/post-show
|
||||
// segments which we filter out.
|
||||
type ScheduleEvent struct {
|
||||
StartTime string `json:"startTime"`
|
||||
State string `json:"state,omitempty"`
|
||||
Type string `json:"type,omitempty"`
|
||||
BlockName string `json:"blockName,omitempty"`
|
||||
League League `json:"league,omitempty"`
|
||||
Match Match `json:"match,omitempty"`
|
||||
StartTime string `json:"startTime" bson:"startTime"`
|
||||
State string `json:"state,omitempty" bson:"state,omitempty"`
|
||||
Type string `json:"type,omitempty" bson:"type,omitempty"`
|
||||
BlockName string `json:"blockName,omitempty" bson:"blockName,omitempty"`
|
||||
League League `json:"league,omitempty" bson:"league,omitempty"`
|
||||
Match Match `json:"match,omitempty" bson:"match,omitempty"`
|
||||
}
|
||||
|
||||
// schedulePage is the inner shape of an upstream response.
|
||||
@@ -103,12 +114,15 @@ type schedulePage struct {
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
// cacheRecord is the KV value: fetch timestamp (ms-epoch) + events.
|
||||
// cacheRecord is the store value: fetch timestamp (ms-epoch) + events.
|
||||
type cacheRecord struct {
|
||||
Ts int64 `json:"ts"` // ms-since-epoch when fetched
|
||||
Events []ScheduleEvent `json:"events"`
|
||||
Ts int64 `json:"ts" bson:"ts"` // ms-since-epoch when fetched
|
||||
Events []ScheduleEvent `json:"events" bson:"events"`
|
||||
}
|
||||
|
||||
// CacheStore is the typed store for schedule cache records.
|
||||
type CacheStore = storage.DocStore[cacheRecord]
|
||||
|
||||
// Client is the lolesports API client. Default zero-value uses
|
||||
// http.DefaultClient + http.DefaultTransport; tests inject a custom HTTP
|
||||
// client (typically pointing at httptest.Server).
|
||||
@@ -274,12 +288,11 @@ func cacheKey(from, to time.Time) string {
|
||||
// GetEventsCached is the cache-first lookup. Returns fresh cache within
|
||||
// cacheTTL, else fetches upstream and writes back, else falls back to
|
||||
// stale cache (within staleMaxAge), else propagates the error.
|
||||
func (c *Client) GetEventsCached(ctx context.Context, kv storage.KVStore, from, to time.Time) ([]ScheduleEvent, error) {
|
||||
func (c *Client) GetEventsCached(ctx context.Context, cache CacheStore, from, to time.Time) ([]ScheduleEvent, error) {
|
||||
key := cacheKey(from, to)
|
||||
now := time.Now().UTC().UnixMilli()
|
||||
|
||||
var cached cacheRecord
|
||||
cacheErr := kv.GetJSON(ctx, key, &cached)
|
||||
cached, _, cacheErr := cache.Get(ctx, key)
|
||||
hasCached := cacheErr == nil
|
||||
if hasCached && now-cached.Ts < cacheTTL.Milliseconds() {
|
||||
return cached.Events, nil
|
||||
@@ -288,7 +301,7 @@ func (c *Client) GetEventsCached(ctx context.Context, kv storage.KVStore, from,
|
||||
events, fetchErr := c.fetchEventsInRange(ctx, from, to, 3)
|
||||
if fetchErr == nil {
|
||||
rec := cacheRecord{Ts: now, Events: events}
|
||||
if err := kv.PutJSON(ctx, key, rec); err != nil {
|
||||
if err := cache.Put(ctx, key, rec); err != nil {
|
||||
log.Warn("lolschedule_kv_put_fail", "err", err)
|
||||
}
|
||||
return events, nil
|
||||
|
||||
@@ -13,6 +13,12 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// newCacheStore returns a fresh typed cache store backed by an in-memory
|
||||
// collection — the per-test equivalent of what the factory wires in production.
|
||||
func newCacheStore() CacheStore {
|
||||
return storage.Typed[cacheRecord](storage.NewMemoryProvider().Collection("lolschedule"))
|
||||
}
|
||||
|
||||
// mkServer spins an httptest.Server returning the supplied JSON body for
|
||||
// every page request. callCount counts upstream hits so cache tests can
|
||||
// assert "1 fetch, then no more".
|
||||
@@ -54,11 +60,11 @@ const sampleBody = `{
|
||||
func TestGetEventsCached_FirstHitFetchesUpstream(t *testing.T) {
|
||||
srv, count := mkServer(t, sampleBody)
|
||||
c := &Client{HTTP: srv.Client(), URL: srv.URL}
|
||||
kv := storage.NewMemoryKVStore()
|
||||
cache := newCacheStore()
|
||||
from := time.Date(2026, 5, 9, 0, 0, 0, 0, time.UTC)
|
||||
to := from.Add(24 * time.Hour)
|
||||
|
||||
events, err := c.GetEventsCached(context.Background(), kv, from, to)
|
||||
events, err := c.GetEventsCached(context.Background(), cache, from, to)
|
||||
if err != nil {
|
||||
t.Fatalf("first fetch: %v", err)
|
||||
}
|
||||
@@ -76,16 +82,16 @@ func TestGetEventsCached_FirstHitFetchesUpstream(t *testing.T) {
|
||||
func TestGetEventsCached_SecondHitUsesCache(t *testing.T) {
|
||||
srv, count := mkServer(t, sampleBody)
|
||||
c := &Client{HTTP: srv.Client(), URL: srv.URL}
|
||||
kv := storage.NewMemoryKVStore()
|
||||
cache := newCacheStore()
|
||||
from := time.Date(2026, 5, 9, 0, 0, 0, 0, time.UTC)
|
||||
to := from.Add(24 * time.Hour)
|
||||
|
||||
// First fetch primes the cache.
|
||||
if _, err := c.GetEventsCached(context.Background(), kv, from, to); err != nil {
|
||||
if _, err := c.GetEventsCached(context.Background(), cache, from, to); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Second fetch within TTL must NOT hit upstream.
|
||||
if _, err := c.GetEventsCached(context.Background(), kv, from, to); err != nil {
|
||||
if _, err := c.GetEventsCached(context.Background(), cache, from, to); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := atomic.LoadInt32(count); got != 1 {
|
||||
@@ -94,8 +100,8 @@ func TestGetEventsCached_SecondHitUsesCache(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestGetEventsCached_StaleFallback(t *testing.T) {
|
||||
// Prime KV with a stale-but-still-fresh-enough cache record.
|
||||
kv := storage.NewMemoryKVStore()
|
||||
// Prime the typed cache store with a stale-but-still-fresh-enough record.
|
||||
cache := newCacheStore()
|
||||
from := time.Date(2026, 5, 9, 0, 0, 0, 0, time.UTC)
|
||||
to := from.Add(24 * time.Hour)
|
||||
staleEvents := []ScheduleEvent{
|
||||
@@ -103,7 +109,7 @@ func TestGetEventsCached_StaleFallback(t *testing.T) {
|
||||
}
|
||||
// 10 minutes ago — past the 120s fresh window but well inside 60-min stale.
|
||||
staleTs := time.Now().UTC().Add(-10 * time.Minute).UnixMilli()
|
||||
if err := kv.PutJSON(context.Background(), cacheKey(from, to), cacheRecord{Ts: staleTs, Events: staleEvents}); err != nil {
|
||||
if err := cache.Put(context.Background(), cacheKey(from, to), cacheRecord{Ts: staleTs, Events: staleEvents}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -115,7 +121,7 @@ func TestGetEventsCached_StaleFallback(t *testing.T) {
|
||||
defer srv.Close()
|
||||
c := &Client{HTTP: srv.Client(), URL: srv.URL}
|
||||
|
||||
got, err := c.GetEventsCached(context.Background(), kv, from, to)
|
||||
got, err := c.GetEventsCached(context.Background(), cache, from, to)
|
||||
if err != nil {
|
||||
t.Fatalf("stale fallback should succeed: %v", err)
|
||||
}
|
||||
@@ -125,7 +131,7 @@ func TestGetEventsCached_StaleFallback(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestGetEventsCached_HardFailureWhenNoCache(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
cache := newCacheStore()
|
||||
from := time.Date(2026, 5, 9, 0, 0, 0, 0, time.UTC)
|
||||
to := from.Add(24 * time.Hour)
|
||||
|
||||
@@ -135,7 +141,7 @@ func TestGetEventsCached_HardFailureWhenNoCache(t *testing.T) {
|
||||
defer srv.Close()
|
||||
c := &Client{HTTP: srv.Client(), URL: srv.URL}
|
||||
|
||||
_, err := c.GetEventsCached(context.Background(), kv, from, to)
|
||||
_, err := c.GetEventsCached(context.Background(), cache, from, to)
|
||||
if err == nil {
|
||||
t.Errorf("expected error when upstream fails AND no cache")
|
||||
}
|
||||
|
||||
@@ -110,6 +110,15 @@ type messageSender interface {
|
||||
SendMessage(ctx context.Context, params *bot.SendMessageParams) (*models.Message, error)
|
||||
}
|
||||
|
||||
// lastPushDoc wraps the last-push date string so it can be stored as a named
|
||||
// root field in a Mongo document (a bare scalar cannot be a root doc).
|
||||
type lastPushDoc struct {
|
||||
Date string `json:"date" bson:"date"`
|
||||
}
|
||||
|
||||
// PushDateStore is the typed store for last-push date documents.
|
||||
type PushDateStore = storage.DocStore[lastPushDoc]
|
||||
|
||||
// dailyPushCron returns the cron registration. Schedule is documentation only.
|
||||
func (s *state) dailyPushCron() modules.Cron {
|
||||
return modules.Cron{
|
||||
@@ -134,30 +143,13 @@ func (s *state) dailyPushHandler(ctx context.Context, deps modules.Deps) error {
|
||||
// for the daily push: a winner proceeds to fan out; a loser (another trigger
|
||||
// already claimed today) returns false and sends nothing.
|
||||
//
|
||||
// The claim is a version-based optimistic write on lastPushDateKey so two
|
||||
// simultaneous triggers cannot both win. Every real KV backend implements
|
||||
// VersionedStore; if a bare store without it is used (only in narrow tests), it
|
||||
// falls back to a plain Put — losing the atomic guarantee but preserving the
|
||||
// "no-op if already today" behaviour.
|
||||
func claimDailyPush(ctx context.Context, kv storage.KVStore, today string) (bool, error) {
|
||||
vs, ok := kv.(storage.VersionedStore)
|
||||
if !ok {
|
||||
// No versioned support (a bare store in narrow tests): best-effort Put,
|
||||
// losing the atomic guarantee but preserving "no-op if already today".
|
||||
current, err := kv.Get(ctx, lastPushDateKey)
|
||||
if err == nil && string(current) == today {
|
||||
return false, nil
|
||||
}
|
||||
if err != nil && !errors.Is(err, storage.ErrNotFound) {
|
||||
return false, err
|
||||
}
|
||||
return true, kv.Put(ctx, lastPushDateKey, []byte(today))
|
||||
}
|
||||
|
||||
current, version, err := vs.GetVersioned(ctx, lastPushDateKey)
|
||||
// The claim uses version-based optimistic write (PutVersioned) on
|
||||
// lastPushDateKey so two simultaneous triggers cannot both win.
|
||||
func claimDailyPush(ctx context.Context, store PushDateStore, today string) (bool, error) {
|
||||
current, version, err := store.Get(ctx, lastPushDateKey)
|
||||
switch {
|
||||
case err == nil:
|
||||
if string(current) == today {
|
||||
if current.Date == today {
|
||||
return false, nil // already pushed today
|
||||
}
|
||||
case errors.Is(err, storage.ErrNotFound):
|
||||
@@ -166,7 +158,7 @@ func claimDailyPush(ctx context.Context, kv storage.KVStore, today string) (bool
|
||||
return false, err
|
||||
}
|
||||
|
||||
if err := vs.PutVersioned(ctx, lastPushDateKey, version, []byte(today)); err != nil {
|
||||
if err := store.PutVersioned(ctx, lastPushDateKey, version, lastPushDoc{Date: today}); err != nil {
|
||||
if errors.Is(err, storage.ErrConflict) {
|
||||
return false, nil // another trigger claimed today first
|
||||
}
|
||||
@@ -182,7 +174,7 @@ func claimDailyPush(ctx context.Context, kv storage.KVStore, today string) (bool
|
||||
// MessageThreadID is forwarded on every send so subscribers in a forum-topic
|
||||
// receive the digest in that topic, not in General.
|
||||
func runDailyPush(ctx context.Context, s *state, sender messageSender) error {
|
||||
subs, err := listSubscribers(ctx, s.kv)
|
||||
subs, err := listSubscribers(ctx, s.subscribers)
|
||||
if err != nil {
|
||||
return fmt.Errorf("lolschedule daily push: list subscribers: %w", err)
|
||||
}
|
||||
@@ -193,7 +185,7 @@ func runDailyPush(ctx context.Context, s *state, sender messageSender) error {
|
||||
|
||||
from := ictDayStartOf(s.now())
|
||||
to := addDays(from, 1)
|
||||
events, err := s.client.GetEventsCached(ctx, s.kv, from, to)
|
||||
events, err := s.client.GetEventsCached(ctx, s.cache, from, to)
|
||||
if err != nil {
|
||||
return fmt.Errorf("lolschedule daily push: fetch matches: %w", err)
|
||||
}
|
||||
@@ -205,7 +197,7 @@ func runDailyPush(ctx context.Context, s *state, sender messageSender) error {
|
||||
// send nothing. Placed after the fetch so a transient fetch failure does
|
||||
// not consume the day's claim.
|
||||
today := s.now().UTC().Format("2006-01-02")
|
||||
won, err := claimDailyPush(ctx, s.kv, today)
|
||||
won, err := claimDailyPush(ctx, s.pushDate, today)
|
||||
if err != nil {
|
||||
return fmt.Errorf("lolschedule daily push: claim date: %w", err)
|
||||
}
|
||||
@@ -272,7 +264,7 @@ func pruneDeadSubscribers(ctx context.Context, s *state, chatWide map[int64]stru
|
||||
defer s.subscribersMu.Unlock()
|
||||
removed := 0
|
||||
for chatID := range chatWide {
|
||||
n, err := removeAllForChat(ctx, s.kv, chatID)
|
||||
n, err := removeAllForChat(ctx, s.subscribers, chatID)
|
||||
if err != nil {
|
||||
log.Warn("lolschedule prune dead chat failed", "chat", chatID, "err", err)
|
||||
continue
|
||||
@@ -285,7 +277,7 @@ func pruneDeadSubscribers(ctx context.Context, s *state, chatWide map[int64]stru
|
||||
if _, ok := chatWide[sub.ChatID]; ok {
|
||||
continue
|
||||
}
|
||||
ok, err := removeSubscriber(ctx, s.kv, sub.ChatID, sub.ThreadID)
|
||||
ok, err := removeSubscriber(ctx, s.subscribers, sub.ChatID, sub.ThreadID)
|
||||
if err != nil {
|
||||
log.Warn("lolschedule prune dead topic failed",
|
||||
"chat", sub.ChatID, "thread", sub.ThreadID, "err", err)
|
||||
|
||||
@@ -56,9 +56,19 @@ func fixedNow() time.Time {
|
||||
return time.Date(2026, 5, 10, 5, 0, 0, 0, time.UTC)
|
||||
}
|
||||
|
||||
// newTestStore builds a fresh set of typed stores over a shared in-memory
|
||||
// collection, matching what the factory wires in production.
|
||||
func newTestStore(t *testing.T) (SubscriberStore, PushDateStore, CacheStore) {
|
||||
t.Helper()
|
||||
col := storage.NewMemoryProvider().Collection("lolschedule")
|
||||
return storage.Typed[subscribersDoc](col),
|
||||
storage.Typed[lastPushDoc](col),
|
||||
storage.Typed[cacheRecord](col)
|
||||
}
|
||||
|
||||
// seedFreshCache writes a cacheRecord with `now` as timestamp so the first
|
||||
// GetEventsCached call returns it without hitting the network.
|
||||
func seedFreshCache(t *testing.T, kv storage.KVStore, events []ScheduleEvent) {
|
||||
func seedFreshCache(t *testing.T, cache CacheStore, events []ScheduleEvent) {
|
||||
t.Helper()
|
||||
from := ictDayStartOf(fixedNow())
|
||||
to := addDays(from, 1)
|
||||
@@ -66,24 +76,26 @@ func seedFreshCache(t *testing.T, kv storage.KVStore, events []ScheduleEvent) {
|
||||
Ts: time.Now().UTC().UnixMilli(),
|
||||
Events: events,
|
||||
}
|
||||
if err := kv.PutJSON(context.Background(), cacheKey(from, to), rec); err != nil {
|
||||
if err := cache.Put(context.Background(), cacheKey(from, to), rec); err != nil {
|
||||
t.Fatalf("seed cache: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func newTestState(t *testing.T) *state {
|
||||
t.Helper()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
subs, pd, cache := newTestStore(t)
|
||||
return &state{
|
||||
kv: kv,
|
||||
client: &Client{}, // zero value; tests must seed cache to avoid HTTP
|
||||
nowFn: fixedNow,
|
||||
subscribers: subs,
|
||||
pushDate: pd,
|
||||
cache: cache,
|
||||
client: &Client{}, // zero value; tests must seed cache to avoid HTTP
|
||||
nowFn: fixedNow,
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunDailyPush_NoSubscribers(t *testing.T) {
|
||||
s := newTestState(t)
|
||||
seedFreshCache(t, s.kv, nil)
|
||||
seedFreshCache(t, s.cache, nil)
|
||||
|
||||
sender := &fakeSender{}
|
||||
if err := runDailyPush(context.Background(), s, sender); err != nil {
|
||||
@@ -96,11 +108,11 @@ func TestRunDailyPush_NoSubscribers(t *testing.T) {
|
||||
|
||||
func TestRunDailyPush_SendsToAllSubscribers(t *testing.T) {
|
||||
s := newTestState(t)
|
||||
seedFreshCache(t, s.kv, nil) // empty schedule still produces a "no matches" message
|
||||
seedFreshCache(t, s.cache, nil) // empty schedule still produces a "no matches" message
|
||||
|
||||
chatIDs := []int64{100, 200, 300}
|
||||
for _, id := range chatIDs {
|
||||
if _, err := addSubscriber(context.Background(), s.kv, id, 0); err != nil {
|
||||
if _, err := addSubscriber(context.Background(), s.subscribers, id, 0); err != nil {
|
||||
t.Fatalf("addSubscriber %d: %v", id, err)
|
||||
}
|
||||
}
|
||||
@@ -138,7 +150,7 @@ func TestRunDailyPush_SendsToAllSubscribers(t *testing.T) {
|
||||
// topic, not in General.
|
||||
func TestRunDailyPush_ForwardsMessageThreadID(t *testing.T) {
|
||||
s := newTestState(t)
|
||||
seedFreshCache(t, s.kv, nil)
|
||||
seedFreshCache(t, s.cache, nil)
|
||||
|
||||
subs := []Subscriber{
|
||||
{ChatID: 100, ThreadID: 0},
|
||||
@@ -146,7 +158,7 @@ func TestRunDailyPush_ForwardsMessageThreadID(t *testing.T) {
|
||||
{ChatID: 200, ThreadID: 42},
|
||||
}
|
||||
for _, sub := range subs {
|
||||
if _, err := addSubscriber(context.Background(), s.kv, sub.ChatID, sub.ThreadID); err != nil {
|
||||
if _, err := addSubscriber(context.Background(), s.subscribers, sub.ChatID, sub.ThreadID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -173,11 +185,11 @@ func TestRunDailyPush_ForwardsMessageThreadID(t *testing.T) {
|
||||
// misconfiguration (all double-fire windows the daily push must survive).
|
||||
func TestRunDailyPush_IdempotentPerDate(t *testing.T) {
|
||||
s := newTestState(t)
|
||||
seedFreshCache(t, s.kv, nil)
|
||||
seedFreshCache(t, s.cache, nil)
|
||||
|
||||
chatIDs := []int64{100, 200, 300}
|
||||
for _, id := range chatIDs {
|
||||
if _, err := addSubscriber(context.Background(), s.kv, id, 0); err != nil {
|
||||
if _, err := addSubscriber(context.Background(), s.subscribers, id, 0); err != nil {
|
||||
t.Fatalf("addSubscriber %d: %v", id, err)
|
||||
}
|
||||
}
|
||||
@@ -196,11 +208,11 @@ func TestRunDailyPush_IdempotentPerDate(t *testing.T) {
|
||||
|
||||
func TestRunDailyPush_PartialFailureContinues(t *testing.T) {
|
||||
s := newTestState(t)
|
||||
seedFreshCache(t, s.kv, nil)
|
||||
seedFreshCache(t, s.cache, nil)
|
||||
|
||||
chatIDs := []int64{100, 200, 300}
|
||||
for _, id := range chatIDs {
|
||||
if _, err := addSubscriber(context.Background(), s.kv, id, 0); err != nil {
|
||||
if _, err := addSubscriber(context.Background(), s.subscribers, id, 0); err != nil {
|
||||
t.Fatalf("addSubscriber %d: %v", id, err)
|
||||
}
|
||||
}
|
||||
@@ -220,7 +232,7 @@ func TestRunDailyPush_PartialFailureContinues(t *testing.T) {
|
||||
// terminal error removes every topic subscription for that chat.
|
||||
func TestRunDailyPush_PrunesDeadSubscribers(t *testing.T) {
|
||||
s := newTestState(t)
|
||||
seedFreshCache(t, s.kv, nil)
|
||||
seedFreshCache(t, s.cache, nil)
|
||||
|
||||
// Chat 400 has two topic subs; both should be pruned when the chat
|
||||
// returns a chat-wide terminal error.
|
||||
@@ -232,7 +244,7 @@ func TestRunDailyPush_PrunesDeadSubscribers(t *testing.T) {
|
||||
{ChatID: 400, ThreadID: 9},
|
||||
}
|
||||
for _, sub := range seedSubs {
|
||||
if _, err := addSubscriber(context.Background(), s.kv, sub.ChatID, sub.ThreadID); err != nil {
|
||||
if _, err := addSubscriber(context.Background(), s.subscribers, sub.ChatID, sub.ThreadID); err != nil {
|
||||
t.Fatalf("addSubscriber %v: %v", sub, err)
|
||||
}
|
||||
}
|
||||
@@ -245,7 +257,7 @@ func TestRunDailyPush_PrunesDeadSubscribers(t *testing.T) {
|
||||
t.Fatalf("runDailyPush: %v", err)
|
||||
}
|
||||
|
||||
remaining, err := listSubscribers(context.Background(), s.kv)
|
||||
remaining, err := listSubscribers(context.Background(), s.subscribers)
|
||||
if err != nil {
|
||||
t.Fatalf("listSubscribers: %v", err)
|
||||
}
|
||||
@@ -265,7 +277,7 @@ func TestRunDailyPush_PrunesDeadSubscribers(t *testing.T) {
|
||||
// (ChatID, ThreadID) entry — sister topics in the same chat stay subscribed.
|
||||
func TestRunDailyPush_TopicOnlyTerminalPrunesOneTopic(t *testing.T) {
|
||||
s := newTestState(t)
|
||||
seedFreshCache(t, s.kv, nil)
|
||||
seedFreshCache(t, s.cache, nil)
|
||||
|
||||
seedSubs := []Subscriber{
|
||||
{ChatID: 500, ThreadID: 0},
|
||||
@@ -273,7 +285,7 @@ func TestRunDailyPush_TopicOnlyTerminalPrunesOneTopic(t *testing.T) {
|
||||
{ChatID: 500, ThreadID: 22},
|
||||
}
|
||||
for _, sub := range seedSubs {
|
||||
if _, err := addSubscriber(context.Background(), s.kv, sub.ChatID, sub.ThreadID); err != nil {
|
||||
if _, err := addSubscriber(context.Background(), s.subscribers, sub.ChatID, sub.ThreadID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -286,7 +298,7 @@ func TestRunDailyPush_TopicOnlyTerminalPrunesOneTopic(t *testing.T) {
|
||||
t.Fatalf("runDailyPush: %v", err)
|
||||
}
|
||||
|
||||
remaining, _ := listSubscribers(context.Background(), s.kv)
|
||||
remaining, _ := listSubscribers(context.Background(), s.subscribers)
|
||||
if len(remaining) != 0 {
|
||||
t.Errorf("expected all topic-only entries pruned, got %v", remaining)
|
||||
}
|
||||
@@ -296,7 +308,7 @@ func TestRunDailyPush_TopicOnlyTerminalPrunesOneTopic(t *testing.T) {
|
||||
// chat goes bad; the other topics in the same chat must stay.
|
||||
func TestRunDailyPush_TopicOnlyTerminalKeepsOtherTopics(t *testing.T) {
|
||||
s := newTestState(t)
|
||||
seedFreshCache(t, s.kv, nil)
|
||||
seedFreshCache(t, s.cache, nil)
|
||||
|
||||
// Two distinct chats, each with multiple topic subs. Only chat 600 hits
|
||||
// the topic-terminal error; chat 700 sends cleanly.
|
||||
@@ -306,7 +318,7 @@ func TestRunDailyPush_TopicOnlyTerminalKeepsOtherTopics(t *testing.T) {
|
||||
{ChatID: 700, ThreadID: 3},
|
||||
}
|
||||
for _, sub := range seedSubs {
|
||||
if _, err := addSubscriber(context.Background(), s.kv, sub.ChatID, sub.ThreadID); err != nil {
|
||||
if _, err := addSubscriber(context.Background(), s.subscribers, sub.ChatID, sub.ThreadID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -317,7 +329,7 @@ func TestRunDailyPush_TopicOnlyTerminalKeepsOtherTopics(t *testing.T) {
|
||||
t.Fatalf("runDailyPush: %v", err)
|
||||
}
|
||||
|
||||
remaining, _ := listSubscribers(context.Background(), s.kv)
|
||||
remaining, _ := listSubscribers(context.Background(), s.subscribers)
|
||||
if len(remaining) != 1 || remaining[0] != (Subscriber{ChatID: 700, ThreadID: 3}) {
|
||||
t.Errorf("after topic-terminal prune of chat 600: got %v, want [{700 3}]", remaining)
|
||||
}
|
||||
@@ -362,7 +374,7 @@ func TestClassifyTerminal(t *testing.T) {
|
||||
|
||||
func TestDailyPushHandler_NilBot_ReturnsError(t *testing.T) {
|
||||
s := newTestState(t)
|
||||
deps := modules.Deps{KV: s.kv} // Bot intentionally nil
|
||||
deps := modules.Deps{Store: storage.NewMemoryProvider().Collection("lolschedule")}
|
||||
err := s.dailyPushHandler(context.Background(), deps)
|
||||
if err == nil {
|
||||
t.Fatal("expected error when deps.Bot is nil, got nil")
|
||||
|
||||
@@ -37,14 +37,8 @@ func TestFormatEventLine_Unstarted(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestFormatEventLine_Completed_BoldsWinner(t *testing.T) {
|
||||
winResult := &struct {
|
||||
Outcome string `json:"outcome,omitempty"`
|
||||
GameWins int `json:"gameWins,omitempty"`
|
||||
}{Outcome: "win", GameWins: 3}
|
||||
loseResult := &struct {
|
||||
Outcome string `json:"outcome,omitempty"`
|
||||
GameWins int `json:"gameWins,omitempty"`
|
||||
}{Outcome: "loss", GameWins: 1}
|
||||
winResult := &TeamResult{Outcome: "win", GameWins: 3}
|
||||
loseResult := &TeamResult{Outcome: "loss", GameWins: 1}
|
||||
e := ScheduleEvent{
|
||||
StartTime: "2026-05-09T05:00:00Z",
|
||||
State: "completed",
|
||||
@@ -73,10 +67,7 @@ func TestFormatEventLine_Completed_BoldsWinner(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestFormatEventLine_InProgress(t *testing.T) {
|
||||
w := &struct {
|
||||
Outcome string `json:"outcome,omitempty"`
|
||||
GameWins int `json:"gameWins,omitempty"`
|
||||
}{GameWins: 1}
|
||||
w := &TeamResult{GameWins: 1}
|
||||
e := ScheduleEvent{
|
||||
StartTime: "2026-05-09T05:00:00Z",
|
||||
State: "inProgress",
|
||||
|
||||
@@ -10,17 +10,18 @@ import (
|
||||
|
||||
"github.com/tiennm99/miti99bot/internal/log"
|
||||
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// state captures everything a lolschedule handler needs at runtime.
|
||||
type state struct {
|
||||
kv storage.KVStore
|
||||
client *Client
|
||||
subscribers SubscriberStore
|
||||
pushDate PushDateStore
|
||||
cache CacheStore
|
||||
client *Client
|
||||
// nowFn allows tests to inject a deterministic clock. Production code
|
||||
// uses time.Now via the default zero-value.
|
||||
nowFn func() time.Time
|
||||
// subscribersMu serializes Get→mutate→Put on the single subscribers KV
|
||||
// subscribersMu serializes Get→mutate→Put on the single subscribers store
|
||||
// slot. Two concurrent /lolschedule_subscribe calls in the same
|
||||
// millisecond would otherwise race and drop one append.
|
||||
subscribersMu sync.Mutex
|
||||
@@ -72,7 +73,7 @@ func (s *state) handleWeek(ctx context.Context, b *bot.Bot, update *models.Updat
|
||||
// replyForRange fetches + filters + renders a date window. week=true uses
|
||||
// RenderWeek; false uses RenderToday.
|
||||
func (s *state) replyForRange(ctx context.Context, b *bot.Bot, msg *models.Message, from, to time.Time, week bool) error {
|
||||
events, err := s.client.GetEventsCached(ctx, s.kv, from, to)
|
||||
events, err := s.client.GetEventsCached(ctx, s.cache, from, to)
|
||||
if err != nil {
|
||||
log.Error("lolschedule_fetch_fail", "err", err, "from", from, "to", to)
|
||||
hint := "Could not fetch matches. Try again later."
|
||||
@@ -100,7 +101,7 @@ func (s *state) handleSubscribe(ctx context.Context, b *bot.Bot, update *models.
|
||||
}
|
||||
s.subscribersMu.Lock()
|
||||
defer s.subscribersMu.Unlock()
|
||||
added, err := addSubscriber(ctx, s.kv, msg.Chat.ID, msg.MessageThreadID)
|
||||
added, err := addSubscriber(ctx, s.subscribers, msg.Chat.ID, msg.MessageThreadID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -120,7 +121,7 @@ func (s *state) handleUnsubscribe(ctx context.Context, b *bot.Bot, update *model
|
||||
}
|
||||
s.subscribersMu.Lock()
|
||||
defer s.subscribersMu.Unlock()
|
||||
removed, err := removeSubscriber(ctx, s.kv, msg.Chat.ID, msg.MessageThreadID)
|
||||
removed, err := removeSubscriber(ctx, s.subscribers, msg.Chat.ID, msg.MessageThreadID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -16,7 +16,8 @@ import (
|
||||
// installSchedule wires the lolschedule module to a recording bot, with a
|
||||
// custom upstream HTTP server returning bodyJSON for every request. nowMs
|
||||
// fixes the clock so date-based handlers are deterministic.
|
||||
func installSchedule(t *testing.T, bodyJSON string, nowMs int64) (*testutil.RecordingBot, storage.KVStore) {
|
||||
// Returns the recording bot and the subscriber store for inspection in tests.
|
||||
func installSchedule(t *testing.T, bodyJSON string, nowMs int64) (*testutil.RecordingBot, SubscriberStore) {
|
||||
t.Helper()
|
||||
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
@@ -25,13 +26,14 @@ func installSchedule(t *testing.T, bodyJSON string, nowMs int64) (*testutil.Reco
|
||||
t.Cleanup(upstream.Close)
|
||||
|
||||
rb := testutil.NewRecordingBot(t)
|
||||
provider := storage.NewMemoryProvider()
|
||||
kv := provider.For("lolschedule")
|
||||
col := storage.NewMemoryProvider().Collection("lolschedule")
|
||||
|
||||
s := &state{
|
||||
kv: kv,
|
||||
client: &Client{HTTP: upstream.Client(), URL: upstream.URL},
|
||||
nowFn: func() time.Time { return time.UnixMilli(nowMs).UTC() },
|
||||
subscribers: storage.Typed[subscribersDoc](col),
|
||||
pushDate: storage.Typed[lastPushDoc](col),
|
||||
cache: storage.Typed[cacheRecord](col),
|
||||
client: &Client{HTTP: upstream.Client(), URL: upstream.URL},
|
||||
nowFn: func() time.Time { return time.UnixMilli(nowMs).UTC() },
|
||||
}
|
||||
mod := modules.Module{
|
||||
Name: "lolschedule",
|
||||
@@ -51,7 +53,7 @@ func installSchedule(t *testing.T, bodyJSON string, nowMs int64) (*testutil.Reco
|
||||
reg.AllCommands[c.Name] = c
|
||||
}
|
||||
modules.Install(rb.Bot, reg, modules.Auth{})
|
||||
return rb, kv
|
||||
return rb, s.subscribers
|
||||
}
|
||||
|
||||
// 2026-05-09 12:00 UTC = 19:00 ICT (still May 9 ICT day). Used as the fake
|
||||
@@ -119,7 +121,7 @@ func TestHandleWeek_RendersWeek(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestHandleSubscribe_AddsAndIsIdempotent(t *testing.T) {
|
||||
rb, kv := installSchedule(t, todayBody, fakeNowMs)
|
||||
rb, subsStore := installSchedule(t, todayBody, fakeNowMs)
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(7, "/lolschedule_subscribe"))
|
||||
if got := rb.LastSent().Text(); !strings.HasPrefix(got, "✅") {
|
||||
t.Errorf("first subscribe should confirm; got %q", got)
|
||||
@@ -129,7 +131,7 @@ func TestHandleSubscribe_AddsAndIsIdempotent(t *testing.T) {
|
||||
if got := rb.LastSent().Text(); !strings.Contains(got, "Already subscribed") {
|
||||
t.Errorf("duplicate subscribe should report Already; got %q", got)
|
||||
}
|
||||
ids, _ := listSubscribers(context.Background(), kv)
|
||||
ids, _ := listSubscribers(context.Background(), subsStore)
|
||||
if len(ids) != 1 || ids[0] != (Subscriber{ChatID: 7}) {
|
||||
t.Errorf("subscribers = %v, want [{7 0}]", ids)
|
||||
}
|
||||
@@ -140,12 +142,12 @@ func TestHandleSubscribe_AddsAndIsIdempotent(t *testing.T) {
|
||||
// records that thread on the Subscriber so the daily push lands in the
|
||||
// originating topic instead of General.
|
||||
func TestHandleSubscribe_ForumTopic_CapturesThreadID(t *testing.T) {
|
||||
rb, kv := installSchedule(t, todayBody, fakeNowMs)
|
||||
rb, subsStore := installSchedule(t, todayBody, fakeNowMs)
|
||||
upd := testutil.NewSupergroupMessage(555, 999, "/lolschedule_subscribe")
|
||||
upd.Message.MessageThreadID = 42
|
||||
rb.Bot.ProcessUpdate(context.Background(), upd)
|
||||
|
||||
subs, _ := listSubscribers(context.Background(), kv)
|
||||
subs, _ := listSubscribers(context.Background(), subsStore)
|
||||
want := Subscriber{ChatID: 555, ThreadID: 42}
|
||||
if len(subs) != 1 || subs[0] != want {
|
||||
t.Errorf("subscribers = %v, want [%v]", subs, want)
|
||||
@@ -156,7 +158,7 @@ func TestHandleSubscribe_ForumTopic_CapturesThreadID(t *testing.T) {
|
||||
// unsubscribing from one topic does not affect sister-topic subscriptions
|
||||
// in the same chat.
|
||||
func TestHandleUnsubscribe_ForumTopic_RemovesOnlyThatTopic(t *testing.T) {
|
||||
rb, kv := installSchedule(t, todayBody, fakeNowMs)
|
||||
rb, subsStore := installSchedule(t, todayBody, fakeNowMs)
|
||||
|
||||
// Subscribe in two topics of the same supergroup.
|
||||
for _, tid := range []int{42, 99} {
|
||||
@@ -170,7 +172,7 @@ func TestHandleUnsubscribe_ForumTopic_RemovesOnlyThatTopic(t *testing.T) {
|
||||
upd.Message.MessageThreadID = 42
|
||||
rb.Bot.ProcessUpdate(context.Background(), upd)
|
||||
|
||||
subs, _ := listSubscribers(context.Background(), kv)
|
||||
subs, _ := listSubscribers(context.Background(), subsStore)
|
||||
want := Subscriber{ChatID: 555, ThreadID: 99}
|
||||
if len(subs) != 1 || subs[0] != want {
|
||||
t.Errorf("subscribers = %v, want [%v]", subs, want)
|
||||
@@ -199,12 +201,13 @@ func TestHandleSchedule_UpstreamFailureGivesFriendlyError(t *testing.T) {
|
||||
defer upstream.Close()
|
||||
|
||||
rb := testutil.NewRecordingBot(t)
|
||||
provider := storage.NewMemoryProvider()
|
||||
kv := provider.For("lolschedule")
|
||||
col := storage.NewMemoryProvider().Collection("lolschedule")
|
||||
s := &state{
|
||||
kv: kv,
|
||||
client: &Client{HTTP: upstream.Client(), URL: upstream.URL},
|
||||
nowFn: func() time.Time { return time.UnixMilli(fakeNowMs).UTC() },
|
||||
subscribers: storage.Typed[subscribersDoc](col),
|
||||
pushDate: storage.Typed[lastPushDoc](col),
|
||||
cache: storage.Typed[cacheRecord](col),
|
||||
client: &Client{HTTP: upstream.Client(), URL: upstream.URL},
|
||||
nowFn: func() time.Time { return time.UnixMilli(fakeNowMs).UTC() },
|
||||
}
|
||||
cmd := modules.Command{Name: "lolschedule_today", Visibility: modules.VisibilityPublic, Description: "x", Handler: s.handleToday}
|
||||
reg := &modules.Registry{
|
||||
|
||||
@@ -2,6 +2,7 @@ package lolschedule
|
||||
|
||||
import (
|
||||
"github.com/tiennm99/miti99bot/internal/modules"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// New is the lolschedule module Factory. The 5 user-facing commands plus the
|
||||
@@ -10,7 +11,12 @@ import (
|
||||
// time — main.go must wire BuildOptions.Bot for the cron to function;
|
||||
// without it the handler fails fast with a clear error.
|
||||
func New(deps modules.Deps) modules.Module {
|
||||
s := &state{kv: deps.KV, client: &Client{}}
|
||||
s := &state{
|
||||
subscribers: storage.Typed[subscribersDoc](deps.Store),
|
||||
pushDate: storage.Typed[lastPushDoc](deps.Store),
|
||||
cache: storage.Typed[cacheRecord](deps.Store),
|
||||
client: &Client{},
|
||||
}
|
||||
return modules.Module{
|
||||
Commands: []modules.Command{
|
||||
{
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// subscribersKey is the KV slot holding the per-module subscriber list.
|
||||
// subscribersKey is the store slot holding the per-module subscriber list.
|
||||
const subscribersKey = "subscribers"
|
||||
|
||||
// Subscriber is one row in the subscriber list. ThreadID is the Telegram
|
||||
@@ -21,10 +21,19 @@ const subscribersKey = "subscribers"
|
||||
// to the General topic, so carrying ThreadID alongside ChatID is what keeps
|
||||
// the daily push landing in the topic the user subscribed from.
|
||||
type Subscriber struct {
|
||||
ChatID int64 `json:"chat_id"`
|
||||
ThreadID int `json:"thread_id,omitempty"`
|
||||
ChatID int64 `json:"chat_id" bson:"chat_id"`
|
||||
ThreadID int `json:"thread_id,omitempty" bson:"thread_id,omitempty"`
|
||||
}
|
||||
|
||||
// subscribersDoc wraps the subscriber list so it can be stored as a named
|
||||
// root field in a Mongo document (a bare JSON array cannot be a root doc).
|
||||
type subscribersDoc struct {
|
||||
Subscribers []Subscriber `json:"subscribers" bson:"subscribers"`
|
||||
}
|
||||
|
||||
// SubscriberStore is the typed store for subscriber documents.
|
||||
type SubscriberStore = storage.DocStore[subscribersDoc]
|
||||
|
||||
// listSubscribers returns the current subscriber list, or an empty slice
|
||||
// if none have ever subscribed.
|
||||
//
|
||||
@@ -36,14 +45,24 @@ type Subscriber struct {
|
||||
// The decoder handles legacy rows indefinitely; any successful add/remove
|
||||
// rewrites the slot in the new shape, but a no-op duplicate add leaves the
|
||||
// legacy bytes in place — that's fine, the fallback path stays correct.
|
||||
func listSubscribers(ctx context.Context, kv storage.KVStore) ([]Subscriber, error) {
|
||||
raw, err := kv.Get(ctx, subscribersKey)
|
||||
func listSubscribers(ctx context.Context, store SubscriberStore) ([]Subscriber, error) {
|
||||
doc, _, err := store.Get(ctx, subscribersKey)
|
||||
switch {
|
||||
case errors.Is(err, storage.ErrNotFound):
|
||||
return nil, nil
|
||||
case err != nil:
|
||||
return nil, fmt.Errorf("lolschedule listSubscribers: %w", err)
|
||||
}
|
||||
if doc.Subscribers != nil {
|
||||
return doc.Subscribers, nil
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// listSubscribersLegacy decodes a raw JSON byte slice that may be either the
|
||||
// current [{chat_id,thread_id}] shape or the legacy [int64] shape. Used by
|
||||
// the legacy-migration path when a key was written by an older version.
|
||||
func listSubscribersLegacy(raw []byte) ([]Subscriber, error) {
|
||||
var subs []Subscriber
|
||||
if err := json.Unmarshal(raw, &subs); err == nil {
|
||||
return subs, nil
|
||||
@@ -62,12 +81,12 @@ func listSubscribers(ctx context.Context, kv storage.KVStore) ([]Subscriber, err
|
||||
// addSubscriber appends (chatID, threadID) if that exact pair is absent.
|
||||
// Returns true on first-add, false when already subscribed (idempotent).
|
||||
//
|
||||
// Concurrency: the list lives in a single KV slot, so a concurrent
|
||||
// Concurrency: the list lives in a single store slot, so a concurrent
|
||||
// Get→mutate→Put from two chats subscribing in the same millisecond would
|
||||
// drop one write. Callers MUST serialize through state.subscribersMu (or an
|
||||
// equivalent module-scoped lock) before calling this.
|
||||
func addSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, threadID int) (bool, error) {
|
||||
subs, err := listSubscribers(ctx, kv)
|
||||
func addSubscriber(ctx context.Context, store SubscriberStore, chatID int64, threadID int) (bool, error) {
|
||||
subs, err := listSubscribers(ctx, store)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
@@ -77,7 +96,7 @@ func addSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, thread
|
||||
}
|
||||
}
|
||||
subs = append(subs, Subscriber{ChatID: chatID, ThreadID: threadID})
|
||||
if err := kv.PutJSON(ctx, subscribersKey, subs); err != nil {
|
||||
if err := store.Put(ctx, subscribersKey, subscribersDoc{Subscribers: subs}); err != nil {
|
||||
return false, fmt.Errorf("lolschedule addSubscriber: %w", err)
|
||||
}
|
||||
return true, nil
|
||||
@@ -88,8 +107,8 @@ func addSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, thread
|
||||
//
|
||||
// Concurrency: same single-slot Get→mutate→Put as addSubscriber; callers
|
||||
// must hold state.subscribersMu.
|
||||
func removeSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, threadID int) (bool, error) {
|
||||
subs, err := listSubscribers(ctx, kv)
|
||||
func removeSubscriber(ctx context.Context, store SubscriberStore, chatID int64, threadID int) (bool, error) {
|
||||
subs, err := listSubscribers(ctx, store)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
@@ -105,7 +124,7 @@ func removeSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, thr
|
||||
if !removed {
|
||||
return false, nil
|
||||
}
|
||||
if err := kv.PutJSON(ctx, subscribersKey, out); err != nil {
|
||||
if err := store.Put(ctx, subscribersKey, subscribersDoc{Subscribers: out}); err != nil {
|
||||
return false, fmt.Errorf("lolschedule removeSubscriber: %w", err)
|
||||
}
|
||||
return true, nil
|
||||
@@ -118,8 +137,8 @@ func removeSubscriber(ctx context.Context, kv storage.KVStore, chatID int64, thr
|
||||
// number of entries actually removed.
|
||||
//
|
||||
// Concurrency: callers must hold state.subscribersMu.
|
||||
func removeAllForChat(ctx context.Context, kv storage.KVStore, chatID int64) (int, error) {
|
||||
subs, err := listSubscribers(ctx, kv)
|
||||
func removeAllForChat(ctx context.Context, store SubscriberStore, chatID int64) (int, error) {
|
||||
subs, err := listSubscribers(ctx, store)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
@@ -135,7 +154,7 @@ func removeAllForChat(ctx context.Context, kv storage.KVStore, chatID int64) (in
|
||||
if removed == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
if err := kv.PutJSON(ctx, subscribersKey, out); err != nil {
|
||||
if err := store.Put(ctx, subscribersKey, subscribersDoc{Subscribers: out}); err != nil {
|
||||
return 0, fmt.Errorf("lolschedule removeAllForChat: %w", err)
|
||||
}
|
||||
return removed, nil
|
||||
|
||||
@@ -2,52 +2,57 @@ package lolschedule
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
func newSubscriberStore() SubscriberStore {
|
||||
return storage.Typed[subscribersDoc](storage.NewMemoryProvider().Collection("lolschedule"))
|
||||
}
|
||||
|
||||
func TestSubscribers_AddRemoveListIdempotent(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newSubscriberStore()
|
||||
|
||||
got, _ := listSubscribers(ctx, kv)
|
||||
got, _ := listSubscribers(ctx, store)
|
||||
if len(got) != 0 {
|
||||
t.Errorf("empty list = %v, want []", got)
|
||||
}
|
||||
|
||||
added, err := addSubscriber(ctx, kv, 42, 0)
|
||||
added, err := addSubscriber(ctx, store, 42, 0)
|
||||
if err != nil || !added {
|
||||
t.Fatalf("first add: added=%v err=%v", added, err)
|
||||
}
|
||||
added, _ = addSubscriber(ctx, kv, 42, 0)
|
||||
added, _ = addSubscriber(ctx, store, 42, 0)
|
||||
if added {
|
||||
t.Errorf("re-add should be no-op")
|
||||
}
|
||||
|
||||
got, _ = listSubscribers(ctx, kv)
|
||||
got, _ = listSubscribers(ctx, store)
|
||||
if len(got) != 1 || got[0] != (Subscriber{ChatID: 42}) {
|
||||
t.Errorf("after add(42,0): %v, want [{42 0}]", got)
|
||||
}
|
||||
|
||||
if _, err := addSubscriber(ctx, kv, 7, 0); err != nil {
|
||||
if _, err := addSubscriber(ctx, store, 7, 0); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, _ = listSubscribers(ctx, kv)
|
||||
got, _ = listSubscribers(ctx, store)
|
||||
if len(got) != 2 {
|
||||
t.Errorf("after add(7,0): %v, want len 2", got)
|
||||
}
|
||||
|
||||
removed, err := removeSubscriber(ctx, kv, 42, 0)
|
||||
removed, err := removeSubscriber(ctx, store, 42, 0)
|
||||
if err != nil || !removed {
|
||||
t.Fatalf("remove(42,0): removed=%v err=%v", removed, err)
|
||||
}
|
||||
removed, _ = removeSubscriber(ctx, kv, 42, 0)
|
||||
removed, _ = removeSubscriber(ctx, store, 42, 0)
|
||||
if removed {
|
||||
t.Errorf("re-remove should be no-op")
|
||||
}
|
||||
|
||||
got, _ = listSubscribers(ctx, kv)
|
||||
got, _ = listSubscribers(ctx, store)
|
||||
if len(got) != 1 || got[0].ChatID != 7 {
|
||||
t.Errorf("after remove(42,0): %v, want [{7 0}]", got)
|
||||
}
|
||||
@@ -58,29 +63,29 @@ func TestSubscribers_AddRemoveListIdempotent(t *testing.T) {
|
||||
// topic must not affect the others.
|
||||
func TestSubscribers_TopicsAreDistinct(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newSubscriberStore()
|
||||
|
||||
for _, tid := range []int{0, 5, 9} {
|
||||
added, err := addSubscriber(ctx, kv, 100, tid)
|
||||
added, err := addSubscriber(ctx, store, 100, tid)
|
||||
if err != nil || !added {
|
||||
t.Fatalf("add(100,%d): added=%v err=%v", tid, added, err)
|
||||
}
|
||||
}
|
||||
// Same (chat, thread) is rejected as duplicate.
|
||||
if added, _ := addSubscriber(ctx, kv, 100, 5); added {
|
||||
if added, _ := addSubscriber(ctx, store, 100, 5); added {
|
||||
t.Errorf("duplicate (100,5) should be no-op")
|
||||
}
|
||||
|
||||
got, _ := listSubscribers(ctx, kv)
|
||||
got, _ := listSubscribers(ctx, store)
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("after 3 adds: %v, want len 3", got)
|
||||
}
|
||||
|
||||
// Removing one topic leaves the other two intact.
|
||||
if removed, _ := removeSubscriber(ctx, kv, 100, 5); !removed {
|
||||
if removed, _ := removeSubscriber(ctx, store, 100, 5); !removed {
|
||||
t.Error("remove(100,5) should report removed=true")
|
||||
}
|
||||
got, _ = listSubscribers(ctx, kv)
|
||||
got, _ = listSubscribers(ctx, store)
|
||||
if len(got) != 2 {
|
||||
t.Errorf("after remove(100,5): %v, want len 2", got)
|
||||
}
|
||||
@@ -95,18 +100,18 @@ func TestSubscribers_TopicsAreDistinct(t *testing.T) {
|
||||
// terminal error means every topic in that chat is dead too.
|
||||
func TestSubscribers_RemoveAllForChat(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newSubscriberStore()
|
||||
|
||||
for _, tid := range []int{0, 5, 9} {
|
||||
if _, err := addSubscriber(ctx, kv, 100, tid); err != nil {
|
||||
if _, err := addSubscriber(ctx, store, 100, tid); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if _, err := addSubscriber(ctx, kv, 200, 0); err != nil {
|
||||
if _, err := addSubscriber(ctx, store, 200, 0); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
n, err := removeAllForChat(ctx, kv, 100)
|
||||
n, err := removeAllForChat(ctx, store, 100)
|
||||
if err != nil {
|
||||
t.Fatalf("removeAllForChat(100): %v", err)
|
||||
}
|
||||
@@ -114,48 +119,78 @@ func TestSubscribers_RemoveAllForChat(t *testing.T) {
|
||||
t.Errorf("removeAllForChat(100) returned %d, want 3", n)
|
||||
}
|
||||
|
||||
got, _ := listSubscribers(ctx, kv)
|
||||
got, _ := listSubscribers(ctx, store)
|
||||
if len(got) != 1 || got[0].ChatID != 200 {
|
||||
t.Errorf("after wipe(100): %v, want only chat 200", got)
|
||||
}
|
||||
|
||||
// Idempotent: second call removes nothing.
|
||||
if n, _ := removeAllForChat(ctx, kv, 100); n != 0 {
|
||||
if n, _ := removeAllForChat(ctx, store, 100); n != 0 {
|
||||
t.Errorf("second removeAllForChat(100) = %d, want 0", n)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSubscribers_LegacyDecode locks in backward-compat reads of rows
|
||||
// written before topic-aware subscriptions (raw []int64 JSON).
|
||||
// written before topic-aware subscriptions (raw []int64 JSON). The legacy
|
||||
// helper decodes both shapes independently of the store.
|
||||
func TestSubscribers_LegacyDecode(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newSubscriberStore()
|
||||
|
||||
// Write legacy shape directly.
|
||||
if err := kv.Put(ctx, subscribersKey, []byte(`[11,22,33]`)); err != nil {
|
||||
// Write the current shape directly via Put so we can confirm round-trip.
|
||||
currentSubs := []Subscriber{{ChatID: 11}, {ChatID: 22}, {ChatID: 33}}
|
||||
if err := store.Put(ctx, subscribersKey, subscribersDoc{Subscribers: currentSubs}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
got, err := listSubscribers(ctx, kv)
|
||||
got, err := listSubscribers(ctx, store)
|
||||
if err != nil {
|
||||
t.Fatalf("listSubscribers: %v", err)
|
||||
}
|
||||
want := []Subscriber{{ChatID: 11}, {ChatID: 22}, {ChatID: 33}}
|
||||
if len(got) != len(want) {
|
||||
t.Fatalf("legacy decode: got %v, want %v", got, want)
|
||||
t.Fatalf("decode: got %v, want %v", got, want)
|
||||
}
|
||||
for i, s := range got {
|
||||
if s != want[i] {
|
||||
t.Errorf("[%d]: got %v, want %v", i, s, want[i])
|
||||
}
|
||||
}
|
||||
|
||||
// Verify the legacy JSON helper handles the old []int64 shape.
|
||||
legacyRaw := []byte(`[11,22,33]`)
|
||||
legacySubs, err := listSubscribersLegacy(legacyRaw)
|
||||
if err != nil {
|
||||
t.Fatalf("listSubscribersLegacy: %v", err)
|
||||
}
|
||||
if len(legacySubs) != 3 {
|
||||
t.Fatalf("legacy decode length = %d, want 3", len(legacySubs))
|
||||
}
|
||||
for i, s := range legacySubs {
|
||||
if s != want[i] {
|
||||
t.Errorf("legacy[%d]: got %v, want %v", i, s, want[i])
|
||||
}
|
||||
}
|
||||
|
||||
// Next mutation rewrites in the new shape — verify by reading raw.
|
||||
if _, err := addSubscriber(ctx, kv, 44, 7); err != nil {
|
||||
// Verify the legacy helper also handles the current [{chat_id,...}] shape.
|
||||
currentRaw, _ := json.Marshal(currentSubs)
|
||||
currentDecoded, err := listSubscribersLegacy(currentRaw)
|
||||
if err != nil {
|
||||
t.Fatalf("listSubscribersLegacy (current shape): %v", err)
|
||||
}
|
||||
if len(currentDecoded) != 3 {
|
||||
t.Fatalf("current shape via legacy helper = %d, want 3", len(currentDecoded))
|
||||
}
|
||||
|
||||
// Next mutation rewrites in the new shape — verify subscribers wrap correctly.
|
||||
if _, err := addSubscriber(ctx, store, 44, 7); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw, _ := kv.Get(ctx, subscribersKey)
|
||||
if string(raw) == `[11,22,33,44]` {
|
||||
t.Errorf("expected rewrite to new shape, still legacy: %s", raw)
|
||||
doc, _, _ := store.Get(ctx, subscribersKey)
|
||||
if doc.Subscribers == nil {
|
||||
t.Error("expected non-nil Subscribers field after add")
|
||||
}
|
||||
if len(doc.Subscribers) != 4 {
|
||||
t.Errorf("after add: doc.Subscribers len = %d, want 4", len(doc.Subscribers))
|
||||
}
|
||||
}
|
||||
@@ -14,14 +14,15 @@ import (
|
||||
)
|
||||
|
||||
// installMisc wires the misc module to a recording bot with a fresh
|
||||
// in-memory KV. Returns the bot, the kv (so tests can pre-seed or read),
|
||||
// and an Auth that permits Owner + Admin so /mstats /fortytwo dispatch.
|
||||
func installMisc(t *testing.T, ownerID int64) (*testutil.RecordingBot, storage.KVStore) {
|
||||
// in-memory store. Returns the bot and the typed store (so tests can pre-seed
|
||||
// or read), plus an Auth that permits Owner + Admin so /mstats /fortytwo dispatch.
|
||||
func installMisc(t *testing.T, ownerID int64) (*testutil.RecordingBot, storage.DocStore[lastPing]) {
|
||||
t.Helper()
|
||||
rb := testutil.NewRecordingBot(t)
|
||||
provider := storage.NewMemoryProvider()
|
||||
kv := provider.For("misc")
|
||||
mod := New(modules.Deps{KV: kv})
|
||||
coll := provider.Collection("misc")
|
||||
mod := New(modules.Deps{Store: coll})
|
||||
store := storage.Typed[lastPing](coll)
|
||||
|
||||
reg := &modules.Registry{
|
||||
Modules: []modules.Module{{Name: "misc", Commands: mod.Commands}},
|
||||
@@ -32,19 +33,19 @@ func installMisc(t *testing.T, ownerID int64) (*testutil.RecordingBot, storage.K
|
||||
}
|
||||
auth := modules.Auth{BotOwnerID: ownerID}
|
||||
modules.Install(rb.Bot, reg, auth)
|
||||
return rb, kv
|
||||
return rb, store
|
||||
}
|
||||
|
||||
func TestPing_RepliesPongAndWritesKV(t *testing.T) {
|
||||
rb, kv := installMisc(t, 999)
|
||||
func TestPing_RepliesPongAndWritesStore(t *testing.T) {
|
||||
rb, store := installMisc(t, 999)
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(999, "/ping"))
|
||||
|
||||
if got := rb.LastSent().Text(); got != "pong" {
|
||||
t.Errorf("ping reply = %q, want %q", got, "pong")
|
||||
}
|
||||
var stored lastPing
|
||||
if err := kv.GetJSON(context.Background(), lastPingKey, &stored); err != nil {
|
||||
t.Fatalf("expected lastPing in KV: %v", err)
|
||||
stored, _, err := store.Get(context.Background(), lastPingKey)
|
||||
if err != nil {
|
||||
t.Fatalf("expected lastPing in store: %v", err)
|
||||
}
|
||||
if stored.At <= 0 {
|
||||
t.Errorf("lastPing.At = %d, want positive", stored.At)
|
||||
@@ -55,7 +56,7 @@ func TestPing_RepliesPongAndWritesKV(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestMstats_NeverWhenKVEmpty(t *testing.T) {
|
||||
func TestMstats_NeverWhenStoreEmpty(t *testing.T) {
|
||||
rb, _ := installMisc(t, 999)
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewPrivateMessage(999, "/mstats"))
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// lastPingKey is the per-module KV key /ping writes and /mstats reads.
|
||||
// lastPingKey is the per-module store key /ping writes and /mstats reads.
|
||||
const lastPingKey = "last_ping"
|
||||
|
||||
// defaultTarget is the substituted "investigator" name when /trongtruonghop is
|
||||
@@ -35,23 +35,24 @@ const trongTruongHopTemplate = "Trong trường hợp nhóm này bị điều tr
|
||||
// int64 ms-epoch (not time.Time → RFC3339) keeps the on-disk shape compact
|
||||
// and consistent with every other timestamp field in the bot's KV.
|
||||
type lastPing struct {
|
||||
At int64 `json:"at"`
|
||||
At int64 `json:"at" bson:"at"`
|
||||
}
|
||||
|
||||
// New is the module Factory. Captures the per-module Deps via closure so each
|
||||
// command handler has direct access to its KV store.
|
||||
// New is the module Factory. Builds a typed store from deps.Store and captures
|
||||
// it via closure so each command handler has direct access.
|
||||
func New(deps modules.Deps) modules.Module {
|
||||
store := storage.Typed[lastPing](deps.Store)
|
||||
return modules.Module{
|
||||
Commands: []modules.Command{
|
||||
pingCommand(deps),
|
||||
mstatsCommand(deps),
|
||||
pingCommand(store),
|
||||
mstatsCommand(store),
|
||||
fortytwoCommand(),
|
||||
trongTruongHopCommand(),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func pingCommand(deps modules.Deps) modules.Command {
|
||||
func pingCommand(store storage.DocStore[lastPing]) modules.Command {
|
||||
return modules.Command{
|
||||
Name: "ping",
|
||||
Visibility: modules.VisibilityPublic,
|
||||
@@ -60,17 +61,17 @@ func pingCommand(deps modules.Deps) modules.Command {
|
||||
if update.Message == nil {
|
||||
return nil
|
||||
}
|
||||
// Best-effort write — if KV is unavailable, still reply.
|
||||
// Best-effort write — if store is unavailable, still reply.
|
||||
payload := lastPing{At: chathelper.NowMillis()}
|
||||
if err := deps.KV.PutJSON(ctx, lastPingKey, payload); err != nil {
|
||||
log.Error("kv put failed", "module", "misc", "command", "ping", "key", lastPingKey, "err", err)
|
||||
if err := store.Put(ctx, lastPingKey, payload); err != nil {
|
||||
log.Error("store put failed", "module", "misc", "command", "ping", "key", lastPingKey, "err", err)
|
||||
}
|
||||
return chathelper.Reply(ctx, b, update.Message, "pong")
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func mstatsCommand(deps modules.Deps) modules.Command {
|
||||
func mstatsCommand(store storage.DocStore[lastPing]) modules.Command {
|
||||
return modules.Command{
|
||||
Name: "mstats",
|
||||
Visibility: modules.VisibilityProtected,
|
||||
@@ -79,18 +80,17 @@ func mstatsCommand(deps modules.Deps) modules.Command {
|
||||
if update.Message == nil {
|
||||
return nil
|
||||
}
|
||||
var last lastPing
|
||||
text := "last ping: never"
|
||||
err := deps.KV.GetJSON(ctx, lastPingKey, &last)
|
||||
last, _, err := store.Get(ctx, lastPingKey)
|
||||
switch {
|
||||
case err == nil && last.At > 0:
|
||||
text = fmt.Sprintf("last ping: %s",
|
||||
time.UnixMilli(last.At).UTC().Format(time.RFC3339))
|
||||
case err != nil && !errors.Is(err, storage.ErrNotFound):
|
||||
// User-visible reply mirrors how stock/wordle/loldle handle
|
||||
// transient KV failures — returning the error here would leave
|
||||
// transient store failures — returning the error here would leave
|
||||
// the user with no reply at all.
|
||||
log.Error("kv get failed", "module", "misc", "command", "mstats", "key", lastPingKey, "err", err)
|
||||
log.Error("store get failed", "module", "misc", "command", "mstats", "key", lastPingKey, "err", err)
|
||||
text = "Could not load stats. Try again later."
|
||||
}
|
||||
return chathelper.Reply(ctx, b, update.Message, text)
|
||||
|
||||
@@ -3,18 +3,22 @@ package misc
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/tiennm99/miti99bot/internal/modules"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// We test the per-command KV behaviour directly — the bot/Telegram side is
|
||||
// We test the per-command store behaviour directly — the bot/Telegram side is
|
||||
// thin (single SendMessage) and exercising it would require a fake bot HTTP
|
||||
// server. The KV interaction is the part with logic worth locking down.
|
||||
// server. The store interaction is the part with logic worth locking down.
|
||||
|
||||
// newMiscStore returns a fresh in-memory typed lastPing store for tests.
|
||||
func newMiscStore() storage.DocStore[lastPing] {
|
||||
return storage.Typed[lastPing](storage.NewMemoryProvider().Collection("misc"))
|
||||
}
|
||||
|
||||
func TestNew_RegistersExpectedCommands(t *testing.T) {
|
||||
deps := modules.Deps{KV: storage.NewMemoryKVStore()}
|
||||
deps := modules.Deps{Store: storage.NewMemoryProvider().Collection("misc")}
|
||||
mod := New(deps)
|
||||
|
||||
want := map[string]modules.Visibility{
|
||||
@@ -41,43 +45,31 @@ func TestNew_RegistersExpectedCommands(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestPing_WritesLastPingKV(t *testing.T) {
|
||||
func TestPing_WritesLastPingStore(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newMiscStore()
|
||||
|
||||
// Drive the KV side directly: lock the wire format as a ms-epoch number
|
||||
// (not an RFC3339 string), matching every other timestamp in the bot's KV.
|
||||
if err := kv.PutJSON(ctx, lastPingKey, lastPing{At: time.Now().UTC().UnixMilli()}); err != nil {
|
||||
t.Fatalf("PutJSON: %v", err)
|
||||
// Drive the store side directly: lock the wire format as a ms-epoch number
|
||||
// (not an RFC3339 string), matching every other timestamp in the bot's store.
|
||||
if err := store.Put(ctx, lastPingKey, lastPing{At: 1700000000000}); err != nil {
|
||||
t.Fatalf("Put: %v", err)
|
||||
}
|
||||
|
||||
var got lastPing
|
||||
if err := kv.GetJSON(ctx, lastPingKey, &got); err != nil {
|
||||
t.Fatalf("GetJSON: %v", err)
|
||||
}
|
||||
if got.At <= 0 {
|
||||
t.Errorf("read-back lastPing.At = %d, want positive ms-epoch", got.At)
|
||||
}
|
||||
|
||||
// Verify a hand-written {"at": <ms-epoch>} document decodes correctly.
|
||||
if err := kv.Put(ctx, lastPingKey, []byte(`{"at":1700000000000}`)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got = lastPing{}
|
||||
if err := kv.GetJSON(ctx, lastPingKey, &got); err != nil {
|
||||
t.Fatalf("GetJSON ms-epoch shape: %v", err)
|
||||
got, _, err := store.Get(ctx, lastPingKey)
|
||||
if err != nil {
|
||||
t.Fatalf("Get: %v", err)
|
||||
}
|
||||
if got.At != 1700000000000 {
|
||||
t.Errorf("ms-epoch round-trip: At = %d, want 1700000000000", got.At)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMstats_MissingKVReturnsErrNotFound(t *testing.T) {
|
||||
func TestMstats_MissingKeyReturnsErrNotFound(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newMiscStore()
|
||||
|
||||
var dst lastPing
|
||||
if err := kv.GetJSON(ctx, lastPingKey, &dst); err != storage.ErrNotFound {
|
||||
t.Errorf("GetJSON missing = %v, want ErrNotFound", err)
|
||||
_, _, err := store.Get(ctx, lastPingKey)
|
||||
if err != storage.ErrNotFound {
|
||||
t.Errorf("Get missing = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
@@ -71,10 +71,10 @@ type Module struct {
|
||||
// Modules that need to introspect commands (e.g. /help) capture this pointer
|
||||
// in their handler closures.
|
||||
type Deps struct {
|
||||
KV storage.KVStore // already prefixed with the module name when passed to a Factory
|
||||
Registry *Registry // populated by Build; safe to capture but read-only at module use
|
||||
Chatter ai.Chatter // nil if GEMINI_API_KEY unset; twentyq must check
|
||||
Bot *bot.Bot // nil-safe: only crons that fan-out (lolschedule daily push) need it
|
||||
Store storage.Collection // the module's own collection; build typed views with storage.Typed[T]
|
||||
Registry *Registry // populated by Build; safe to capture but read-only at module use
|
||||
Chatter ai.Chatter // nil if GEMINI_API_KEY unset; twentyq must check
|
||||
Bot *bot.Bot // nil-safe: only crons that fan-out (lolschedule daily push) need it
|
||||
}
|
||||
|
||||
// Factory constructs a Module from its Deps. Deps are passed directly (instead
|
||||
|
||||
@@ -83,11 +83,10 @@ func (r *Registry) Crons() []Cron {
|
||||
return out
|
||||
}
|
||||
|
||||
// Build constructs a Registry from the requested module names. The KVProvider
|
||||
// supplies a per-module-isolated KVStore (MemoryProvider key-prefixes a shared
|
||||
// store; FirestoreProvider hands out one collection per module). Build validates
|
||||
// every command/cron and aborts on duplicate command names across the union of
|
||||
// all visibilities.
|
||||
// Build constructs a Registry from the requested module names. The Provider
|
||||
// supplies a per-module-isolated Collection (one MongoDB collection per module;
|
||||
// MemoryProvider keeps one map per module). Build validates every command/cron
|
||||
// and aborts on duplicate command names across the union of all visibilities.
|
||||
//
|
||||
// Names not present in factories are reported as a single error so a typo in
|
||||
// MODULES does not silently load a smaller bot than intended. Duplicate names
|
||||
@@ -100,9 +99,9 @@ type BuildOptions struct {
|
||||
Bot *bot.Bot
|
||||
}
|
||||
|
||||
func Build(enabled []string, factories map[string]Factory, kv storage.KVProvider, opts BuildOptions) (*Registry, error) {
|
||||
if kv == nil {
|
||||
return nil, fmt.Errorf("modules: KVProvider is required")
|
||||
func Build(enabled []string, factories map[string]Factory, provider storage.Provider, opts BuildOptions) (*Registry, error) {
|
||||
if provider == nil {
|
||||
return nil, fmt.Errorf("modules: storage Provider is required")
|
||||
}
|
||||
|
||||
reg := &Registry{
|
||||
@@ -135,7 +134,7 @@ func Build(enabled []string, factories map[string]Factory, kv storage.KVProvider
|
||||
}
|
||||
|
||||
moduleDeps := Deps{
|
||||
KV: kv.For(name),
|
||||
Store: provider.Collection(name),
|
||||
Registry: reg,
|
||||
Chatter: opts.Chatter,
|
||||
Bot: opts.Bot,
|
||||
|
||||
@@ -35,7 +35,7 @@ func factory(name string, cmds []Command, crons []Cron) Factory {
|
||||
}
|
||||
}
|
||||
|
||||
func newProvider() storage.KVProvider { return storage.NewMemoryProvider() }
|
||||
func newProvider() storage.Provider { return storage.NewMemoryProvider() }
|
||||
|
||||
func TestBuild_EmptyModulesBootsCleanly(t *testing.T) {
|
||||
reg, err := Build(nil, map[string]Factory{}, newProvider(), BuildOptions{})
|
||||
@@ -165,7 +165,7 @@ func TestDispatchScheduled_UnknownReturnsErrCronNotFound(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestDispatchScheduled_PassesPrefixedDeps(t *testing.T) {
|
||||
func TestDispatchScheduled_PassesScopedDeps(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
provider := storage.NewMemoryProvider()
|
||||
|
||||
@@ -174,7 +174,7 @@ func TestDispatchScheduled_PassesPrefixedDeps(t *testing.T) {
|
||||
return Module{Crons: []Cron{{
|
||||
Name: "tick_a",
|
||||
Handler: func(ctx context.Context, deps Deps) error {
|
||||
return deps.KV.Put(ctx, "last", []byte("A"))
|
||||
return storage.Typed[string](deps.Store).Put(ctx, "last", "A")
|
||||
},
|
||||
}}}
|
||||
},
|
||||
@@ -182,7 +182,7 @@ func TestDispatchScheduled_PassesPrefixedDeps(t *testing.T) {
|
||||
return Module{Crons: []Cron{{
|
||||
Name: "tick_b",
|
||||
Handler: func(ctx context.Context, deps Deps) error {
|
||||
return deps.KV.Put(ctx, "last", []byte("B"))
|
||||
return storage.Typed[string](deps.Store).Put(ctx, "last", "B")
|
||||
},
|
||||
}}}
|
||||
},
|
||||
@@ -198,14 +198,14 @@ func TestDispatchScheduled_PassesPrefixedDeps(t *testing.T) {
|
||||
t.Fatalf("tick_b: %v", err)
|
||||
}
|
||||
|
||||
// Underlying base store should hold each module's prefixed key separately.
|
||||
gotA, err := provider.Base().Get(ctx, "alpha:last")
|
||||
if err != nil || string(gotA) != "A" {
|
||||
t.Errorf("alpha:last = %q (err=%v), want A", gotA, err)
|
||||
// Each module's per-collection store holds its own "last" key separately.
|
||||
gotA, _, err := storage.Typed[string](provider.Collection("alpha")).Get(ctx, "last")
|
||||
if err != nil || gotA != "A" {
|
||||
t.Errorf("alpha/last = %q (err=%v), want A", gotA, err)
|
||||
}
|
||||
gotB, err := provider.Base().Get(ctx, "beta:last")
|
||||
if err != nil || string(gotB) != "B" {
|
||||
t.Errorf("beta:last = %q (err=%v), want B", gotB, err)
|
||||
gotB, _, err := storage.Typed[string](provider.Collection("beta")).Get(ctx, "last")
|
||||
if err != nil || gotB != "B" {
|
||||
t.Errorf("beta/last = %q (err=%v), want B", gotB, err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -280,12 +280,12 @@ func TestBuild_RejectsDuplicateModuleInEnv(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuild_PerModulePrefixedKV(t *testing.T) {
|
||||
func TestBuild_PerModuleStoreIsolation(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
provider := storage.NewMemoryProvider()
|
||||
|
||||
// Each module writes a value to the same key; with per-module prefixing
|
||||
// they must not collide.
|
||||
// Each module writes a value to the same key; with one collection per
|
||||
// module they must not collide.
|
||||
captured := map[string]Deps{}
|
||||
factories := map[string]Factory{
|
||||
"alpha": func(d Deps) Module {
|
||||
@@ -301,19 +301,19 @@ func TestBuild_PerModulePrefixedKV(t *testing.T) {
|
||||
t.Fatalf("Build: %v", err)
|
||||
}
|
||||
|
||||
if err := captured["alpha"].KV.Put(ctx, "score", []byte("1")); err != nil {
|
||||
alpha := storage.Typed[string](captured["alpha"].Store)
|
||||
beta := storage.Typed[string](captured["beta"].Store)
|
||||
if err := alpha.Put(ctx, "score", "1"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := captured["beta"].KV.Put(ctx, "score", []byte("2")); err != nil {
|
||||
if err := beta.Put(ctx, "score", "2"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
got, _ := captured["alpha"].KV.Get(ctx, "score")
|
||||
if string(got) != "1" {
|
||||
t.Errorf("alpha.KV.score = %q, want 1", got)
|
||||
if got, _, _ := alpha.Get(ctx, "score"); got != "1" {
|
||||
t.Errorf("alpha/score = %q, want 1", got)
|
||||
}
|
||||
got, _ = captured["beta"].KV.Get(ctx, "score")
|
||||
if string(got) != "2" {
|
||||
t.Errorf("beta.KV.score = %q, want 2", got)
|
||||
if got, _, _ := beta.Get(ctx, "score"); got != "2" {
|
||||
t.Errorf("beta/score = %q, want 2", got)
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
// Package stats tracks per-command and per-user invocation counts persistently
|
||||
// in KV and exposes /stats subcommands to display them sorted by popularity.
|
||||
// in DocStore and exposes /stats subcommands to display them sorted by popularity.
|
||||
package stats
|
||||
|
||||
import (
|
||||
@@ -15,10 +15,11 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// Sort-key shapes inside the stats module's KV partition:
|
||||
// count:<cmd> → countEntry — total per command
|
||||
// user:<userID> → userEntry — cached username + total per user
|
||||
// pair:<cmd>:<userID> → countEntry — per (command, user) pair
|
||||
// Sort-key shapes inside the stats module's partition:
|
||||
//
|
||||
// count:<cmd> → countEntry — total per command
|
||||
// user:<userID> → userEntry — cached username + total per user
|
||||
// pair:<cmd>:<userID> → countEntry — per (command, user) pair
|
||||
const (
|
||||
countPrefix = "count:"
|
||||
userPrefix = "user:"
|
||||
@@ -27,16 +28,20 @@ const (
|
||||
)
|
||||
|
||||
type countEntry struct {
|
||||
N int64 `json:"n"`
|
||||
N int64 `json:"n" bson:"n"`
|
||||
}
|
||||
|
||||
type userEntry struct {
|
||||
Username string `json:"username"`
|
||||
N int64 `json:"n"`
|
||||
Username string `json:"username" bson:"username"`
|
||||
N int64 `json:"n" bson:"n"`
|
||||
}
|
||||
|
||||
// counter holds two typed views over the same module Collection:
|
||||
// one for countEntry values and one for userEntry values.
|
||||
// Keys are disjoint by prefix so both views safely share the collection.
|
||||
type counter struct {
|
||||
kv storage.KVStore
|
||||
counts storage.DocStore[countEntry]
|
||||
users storage.DocStore[userEntry]
|
||||
}
|
||||
|
||||
func countKey(name string) string { return countPrefix + name }
|
||||
@@ -87,48 +92,51 @@ func (c *counter) Inc(ctx context.Context, name string, update *models.Update) {
|
||||
|
||||
func (c *counter) incCount(ctx context.Context, name string) {
|
||||
key := countKey(name)
|
||||
var entry countEntry
|
||||
if err := c.kv.GetJSON(ctx, key, &entry); err != nil && !errors.Is(err, storage.ErrNotFound) {
|
||||
log.Error("stats: kv get failed", "key", key, "err", err)
|
||||
entry, _, err := c.counts.Get(ctx, key)
|
||||
if err != nil && !errors.Is(err, storage.ErrNotFound) {
|
||||
log.Error("stats: store get failed", "key", key, "err", err)
|
||||
return
|
||||
}
|
||||
entry.N++
|
||||
if err := c.kv.PutJSON(ctx, key, entry); err != nil {
|
||||
log.Error("stats: kv put failed", "key", key, "err", err)
|
||||
if err := c.counts.Put(ctx, key, entry); err != nil {
|
||||
log.Error("stats: store put failed", "key", key, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *counter) incUser(ctx context.Context, id int64, username string) {
|
||||
key := userKey(id)
|
||||
var entry userEntry
|
||||
if err := c.kv.GetJSON(ctx, key, &entry); err != nil && !errors.Is(err, storage.ErrNotFound) {
|
||||
log.Error("stats: kv get failed", "key", key, "err", err)
|
||||
entry, _, err := c.users.Get(ctx, key)
|
||||
if err != nil && !errors.Is(err, storage.ErrNotFound) {
|
||||
log.Error("stats: store get failed", "key", key, "err", err)
|
||||
return
|
||||
}
|
||||
entry.Username = username
|
||||
entry.N++
|
||||
if err := c.kv.PutJSON(ctx, key, entry); err != nil {
|
||||
log.Error("stats: kv put failed", "key", key, "err", err)
|
||||
if err := c.users.Put(ctx, key, entry); err != nil {
|
||||
log.Error("stats: store put failed", "key", key, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *counter) incPair(ctx context.Context, name string, id int64) {
|
||||
key := pairKey(name, id)
|
||||
var entry countEntry
|
||||
if err := c.kv.GetJSON(ctx, key, &entry); err != nil && !errors.Is(err, storage.ErrNotFound) {
|
||||
log.Error("stats: kv get failed", "key", key, "err", err)
|
||||
entry, _, err := c.counts.Get(ctx, key)
|
||||
if err != nil && !errors.Is(err, storage.ErrNotFound) {
|
||||
log.Error("stats: store get failed", "key", key, "err", err)
|
||||
return
|
||||
}
|
||||
entry.N++
|
||||
if err := c.kv.PutJSON(ctx, key, entry); err != nil {
|
||||
log.Error("stats: kv put failed", "key", key, "err", err)
|
||||
if err := c.counts.Put(ctx, key, entry); err != nil {
|
||||
log.Error("stats: store put failed", "key", key, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
// New is the module Factory. Registers a CommandHook that persists counts and
|
||||
// a /stats command that displays them.
|
||||
func New(deps modules.Deps) modules.Module {
|
||||
c := &counter{kv: deps.KV}
|
||||
c := &counter{
|
||||
counts: storage.Typed[countEntry](deps.Store),
|
||||
users: storage.Typed[userEntry](deps.Store),
|
||||
}
|
||||
return modules.Module{
|
||||
CommandHook: c.Inc,
|
||||
Commands: []modules.Command{
|
||||
|
||||
@@ -2,6 +2,7 @@ package stats
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
@@ -13,8 +14,17 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/testutil"
|
||||
)
|
||||
|
||||
// newStatsCounter returns a fresh counter backed by an in-memory collection.
|
||||
func newStatsCounter() *counter {
|
||||
col := storage.NewMemoryProvider().Collection("stats")
|
||||
return &counter{
|
||||
counts: storage.Typed[countEntry](col),
|
||||
users: storage.Typed[userEntry](col),
|
||||
}
|
||||
}
|
||||
|
||||
func TestNew_RegistersExpectedCommands(t *testing.T) {
|
||||
deps := modules.Deps{KV: storage.NewMemoryKVStore()}
|
||||
deps := modules.Deps{Store: storage.NewMemoryProvider().Collection("stats")}
|
||||
mod := New(deps)
|
||||
|
||||
if len(mod.Commands) != 1 {
|
||||
@@ -45,58 +55,56 @@ func updateFrom(id int64, username string) *models.Update {
|
||||
}
|
||||
}
|
||||
|
||||
func TestInc_PersistsCountInKV(t *testing.T) {
|
||||
func TestInc_PersistsCountInStore(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
c := &counter{kv: kv}
|
||||
c := newStatsCounter()
|
||||
|
||||
c.Inc(ctx, "ping", nil)
|
||||
c.Inc(ctx, "ping", nil)
|
||||
c.Inc(ctx, "wordle", nil)
|
||||
|
||||
var entry countEntry
|
||||
if err := kv.GetJSON(ctx, countKey("ping"), &entry); err != nil {
|
||||
t.Fatalf("GetJSON ping: %v", err)
|
||||
entry, _, err := c.counts.Get(ctx, countKey("ping"))
|
||||
if err != nil {
|
||||
t.Fatalf("Get ping: %v", err)
|
||||
}
|
||||
if entry.N != 2 {
|
||||
t.Errorf("ping count = %d, want 2", entry.N)
|
||||
}
|
||||
|
||||
entry = countEntry{}
|
||||
if err := kv.GetJSON(ctx, countKey("wordle"), &entry); err != nil {
|
||||
t.Fatalf("GetJSON wordle: %v", err)
|
||||
entry2, _, err := c.counts.Get(ctx, countKey("wordle"))
|
||||
if err != nil {
|
||||
t.Fatalf("Get wordle: %v", err)
|
||||
}
|
||||
if entry.N != 1 {
|
||||
t.Errorf("wordle count = %d, want 1", entry.N)
|
||||
if entry2.N != 1 {
|
||||
t.Errorf("wordle count = %d, want 1", entry2.N)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInc_WithUsernameWritesAllThreeKeys(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
c := &counter{kv: kv}
|
||||
c := newStatsCounter()
|
||||
|
||||
c.Inc(ctx, "ping", updateFrom(42, "alice"))
|
||||
c.Inc(ctx, "ping", updateFrom(42, "alice"))
|
||||
|
||||
var ce countEntry
|
||||
if err := kv.GetJSON(ctx, countKey("ping"), &ce); err != nil {
|
||||
ce, _, err := c.counts.Get(ctx, countKey("ping"))
|
||||
if err != nil {
|
||||
t.Fatalf("count:ping: %v", err)
|
||||
}
|
||||
if ce.N != 2 {
|
||||
t.Errorf("count:ping N = %d, want 2", ce.N)
|
||||
}
|
||||
|
||||
var ue userEntry
|
||||
if err := kv.GetJSON(ctx, userKey(42), &ue); err != nil {
|
||||
ue, _, err := c.users.Get(ctx, userKey(42))
|
||||
if err != nil {
|
||||
t.Fatalf("user:42: %v", err)
|
||||
}
|
||||
if ue.N != 2 || ue.Username != "alice" {
|
||||
t.Errorf("user:42 = {%q, %d}, want {alice, 2}", ue.Username, ue.N)
|
||||
}
|
||||
|
||||
var pe countEntry
|
||||
if err := kv.GetJSON(ctx, pairKey("ping", 42), &pe); err != nil {
|
||||
pe, _, err := c.counts.Get(ctx, pairKey("ping", 42))
|
||||
if err != nil {
|
||||
t.Fatalf("pair:ping:42: %v", err)
|
||||
}
|
||||
if pe.N != 2 {
|
||||
@@ -106,37 +114,35 @@ func TestInc_WithUsernameWritesAllThreeKeys(t *testing.T) {
|
||||
|
||||
func TestInc_EmptyUsernameSkipsUserAndPair(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
c := &counter{kv: kv}
|
||||
c := newStatsCounter()
|
||||
|
||||
c.Inc(ctx, "ping", updateFrom(42, ""))
|
||||
|
||||
var ce countEntry
|
||||
if err := kv.GetJSON(ctx, countKey("ping"), &ce); err != nil {
|
||||
ce, _, err := c.counts.Get(ctx, countKey("ping"))
|
||||
if err != nil {
|
||||
t.Fatalf("count:ping: %v", err)
|
||||
}
|
||||
if ce.N != 1 {
|
||||
t.Errorf("count:ping N = %d, want 1", ce.N)
|
||||
}
|
||||
|
||||
if _, err := kv.Get(ctx, userKey(42)); err != storage.ErrNotFound {
|
||||
if _, _, err := c.users.Get(ctx, userKey(42)); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Errorf("user:42 should be absent, got err=%v", err)
|
||||
}
|
||||
if _, err := kv.Get(ctx, pairKey("ping", 42)); err != storage.ErrNotFound {
|
||||
if _, _, err := c.counts.Get(ctx, pairKey("ping", 42)); !errors.Is(err, storage.ErrNotFound) {
|
||||
t.Errorf("pair:ping:42 should be absent, got err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInc_RefreshesUsernameOnRename(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
c := &counter{kv: kv}
|
||||
c := newStatsCounter()
|
||||
|
||||
c.Inc(ctx, "ping", updateFrom(42, "alice"))
|
||||
c.Inc(ctx, "ping", updateFrom(42, "alice2"))
|
||||
|
||||
var ue userEntry
|
||||
if err := kv.GetJSON(ctx, userKey(42), &ue); err != nil {
|
||||
ue, _, err := c.users.Get(ctx, userKey(42))
|
||||
if err != nil {
|
||||
t.Fatalf("user:42: %v", err)
|
||||
}
|
||||
if ue.Username != "alice2" {
|
||||
@@ -150,8 +156,7 @@ func TestInc_RefreshesUsernameOnRename(t *testing.T) {
|
||||
func installStats(t *testing.T) (*testutil.RecordingBot, *counter) {
|
||||
t.Helper()
|
||||
rb := testutil.NewRecordingBot(t)
|
||||
kv := storage.NewMemoryKVStore()
|
||||
c := &counter{kv: kv}
|
||||
c := newStatsCounter()
|
||||
mod := modules.Module{
|
||||
Commands: []modules.Command{statsCommand(c)},
|
||||
}
|
||||
@@ -226,10 +231,11 @@ func TestCommandHook_FiredThroughModulesBuild(t *testing.T) {
|
||||
|
||||
reg.RunCommandHooks(ctx, "ping", nil)
|
||||
|
||||
statsKV := provider.For("stats")
|
||||
var entry countEntry
|
||||
if err := statsKV.GetJSON(ctx, countKey("ping"), &entry); err != nil {
|
||||
t.Fatalf("expected count:ping in KV after hook: %v", err)
|
||||
// Verify by reading back via typed store (the same collection Build gave to stats).
|
||||
statsStore := storage.Typed[countEntry](provider.Collection("stats"))
|
||||
entry, _, err := statsStore.Get(ctx, countKey("ping"))
|
||||
if err != nil {
|
||||
t.Fatalf("expected count:ping in store after hook: %v", err)
|
||||
}
|
||||
if entry.N != 1 {
|
||||
t.Errorf("count:ping = %d, want 1", entry.N)
|
||||
@@ -256,7 +262,7 @@ func seedFixture(t *testing.T, c *counter) {
|
||||
}
|
||||
|
||||
func TestRenderStats_TopCommands(t *testing.T) {
|
||||
c := &counter{kv: storage.NewMemoryKVStore()}
|
||||
c := newStatsCounter()
|
||||
seedFixture(t, c)
|
||||
|
||||
got := renderStats(context.Background(), c, "")
|
||||
@@ -275,7 +281,7 @@ func TestRenderStats_TopCommands(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestRenderStats_Users(t *testing.T) {
|
||||
c := &counter{kv: storage.NewMemoryKVStore()}
|
||||
c := newStatsCounter()
|
||||
seedFixture(t, c)
|
||||
|
||||
got := renderStats(context.Background(), c, "users")
|
||||
@@ -295,7 +301,7 @@ func TestRenderStats_Users(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestRenderStats_UserCommands(t *testing.T) {
|
||||
c := &counter{kv: storage.NewMemoryKVStore()}
|
||||
c := newStatsCounter()
|
||||
seedFixture(t, c)
|
||||
|
||||
got := renderStats(context.Background(), c, "user @alice")
|
||||
@@ -320,7 +326,7 @@ func TestRenderStats_UserCommands(t *testing.T) {
|
||||
// flagged this as a potential bug; this test proves the absence of the bug.
|
||||
func TestRenderStats_UserCommands_IDSuffixDoesNotFalseMatch(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
c := &counter{kv: storage.NewMemoryKVStore()}
|
||||
c := newStatsCounter()
|
||||
// user 2 ("two") calls /ping; user 12 ("twelve") calls /wordle.
|
||||
c.Inc(ctx, "ping", updateFrom(2, "two"))
|
||||
c.Inc(ctx, "wordle", updateFrom(12, "twelve"))
|
||||
@@ -338,7 +344,7 @@ func TestRenderStats_UserCommands_IDSuffixDoesNotFalseMatch(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestRenderStats_UserCommands_BareNameAlsoWorks(t *testing.T) {
|
||||
c := &counter{kv: storage.NewMemoryKVStore()}
|
||||
c := newStatsCounter()
|
||||
seedFixture(t, c)
|
||||
|
||||
got := renderStats(context.Background(), c, "user alice")
|
||||
@@ -348,7 +354,7 @@ func TestRenderStats_UserCommands_BareNameAlsoWorks(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestRenderStats_UserNotFound(t *testing.T) {
|
||||
c := &counter{kv: storage.NewMemoryKVStore()}
|
||||
c := newStatsCounter()
|
||||
seedFixture(t, c)
|
||||
|
||||
got := renderStats(context.Background(), c, "user @ghost")
|
||||
@@ -358,7 +364,7 @@ func TestRenderStats_UserNotFound(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestRenderStats_CmdUsers(t *testing.T) {
|
||||
c := &counter{kv: storage.NewMemoryKVStore()}
|
||||
c := newStatsCounter()
|
||||
seedFixture(t, c)
|
||||
|
||||
got := renderStats(context.Background(), c, "cmd wordle")
|
||||
@@ -380,7 +386,7 @@ func TestRenderStats_CmdUsers(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestRenderStats_CmdNotFound(t *testing.T) {
|
||||
c := &counter{kv: storage.NewMemoryKVStore()}
|
||||
c := newStatsCounter()
|
||||
seedFixture(t, c)
|
||||
|
||||
got := renderStats(context.Background(), c, "cmd nonexistent")
|
||||
@@ -390,7 +396,7 @@ func TestRenderStats_CmdNotFound(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestRenderStats_UnknownSubcommand(t *testing.T) {
|
||||
c := &counter{kv: storage.NewMemoryKVStore()}
|
||||
c := newStatsCounter()
|
||||
got := renderStats(context.Background(), c, "bogus")
|
||||
if !strings.HasPrefix(got, "Usage:") {
|
||||
t.Errorf("expected usage string, got %q", got)
|
||||
@@ -398,7 +404,7 @@ func TestRenderStats_UnknownSubcommand(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestRenderStats_MissingRequiredArg(t *testing.T) {
|
||||
c := &counter{kv: storage.NewMemoryKVStore()}
|
||||
c := newStatsCounter()
|
||||
if got := renderStats(context.Background(), c, "user"); !strings.HasPrefix(got, "Usage:") {
|
||||
t.Errorf("user (no arg) should return usage, got %q", got)
|
||||
}
|
||||
|
||||
@@ -89,9 +89,9 @@ func parseSubargs(update *models.Update) string {
|
||||
}
|
||||
|
||||
func viewTopCommands(ctx context.Context, c *counter) string {
|
||||
keys, err := c.kv.List(ctx, countPrefix)
|
||||
keys, err := c.counts.List(ctx, countPrefix)
|
||||
if err != nil {
|
||||
log.Error("stats: kv list failed", "err", err)
|
||||
log.Error("stats: store list failed", "err", err)
|
||||
return "Could not load stats. Try again later."
|
||||
}
|
||||
if len(keys) == 0 {
|
||||
@@ -145,9 +145,9 @@ func viewUserCommands(ctx context.Context, c *counter, username string) string {
|
||||
return fmt.Sprintf("User @%s not found.", username)
|
||||
}
|
||||
|
||||
keys, err := c.kv.List(ctx, pairPrefix)
|
||||
keys, err := c.counts.List(ctx, pairPrefix)
|
||||
if err != nil {
|
||||
log.Error("stats: kv list failed", "err", err)
|
||||
log.Error("stats: store list failed", "err", err)
|
||||
return "Could not load stats. Try again later."
|
||||
}
|
||||
// Leading colon disambiguates: ":2" does not match a key ending in "12"
|
||||
@@ -181,9 +181,9 @@ func viewUserCommands(ctx context.Context, c *counter, username string) string {
|
||||
|
||||
func viewCmdUsers(ctx context.Context, c *counter, cmd string) string {
|
||||
prefix := pairPrefix + cmd + ":"
|
||||
keys, err := c.kv.List(ctx, prefix)
|
||||
keys, err := c.counts.List(ctx, prefix)
|
||||
if err != nil {
|
||||
log.Error("stats: kv list failed", "err", err)
|
||||
log.Error("stats: store list failed", "err", err)
|
||||
return "Could not load stats. Try again later."
|
||||
}
|
||||
if len(keys) == 0 {
|
||||
@@ -200,9 +200,9 @@ func viewCmdUsers(ctx context.Context, c *counter, cmd string) string {
|
||||
wg.Add(1)
|
||||
go func(i int, k string) {
|
||||
defer wg.Done()
|
||||
var ce countEntry
|
||||
if err := c.kv.GetJSON(ctx, k, &ce); err != nil {
|
||||
log.Error("stats: kv get failed", "key", k, "err", err)
|
||||
ce, _, err := c.counts.Get(ctx, k)
|
||||
if err != nil {
|
||||
log.Error("stats: store get failed", "key", k, "err", err)
|
||||
return
|
||||
}
|
||||
idStr := strings.TrimPrefix(k, prefix)
|
||||
@@ -210,8 +210,8 @@ func viewCmdUsers(ctx context.Context, c *counter, cmd string) string {
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
var ue userEntry
|
||||
if err := c.kv.GetJSON(ctx, userKey(id), &ue); err != nil || ue.Username == "" {
|
||||
ue, _, err := c.users.Get(ctx, userKey(id))
|
||||
if err != nil || ue.Username == "" {
|
||||
return
|
||||
}
|
||||
results[i] = result{r: row{display: "@" + ue.Username, n: ce.N}, ok: true}
|
||||
@@ -232,7 +232,7 @@ func viewCmdUsers(ctx context.Context, c *counter, cmd string) string {
|
||||
return renderTopN(fmt.Sprintf("Users of /%s:", cmd), rows, topK)
|
||||
}
|
||||
|
||||
// fanOutCountRows runs GetJSON on each key in parallel. displayFor maps the
|
||||
// fanOutCountRows runs Get on each key in parallel. displayFor maps the
|
||||
// raw sort key to the human-readable label that gets rendered. Missing /
|
||||
// errored keys are skipped (logged at error level).
|
||||
func fanOutCountRows(ctx context.Context, c *counter, keys []string, displayFor func(k string) string) []row {
|
||||
@@ -246,9 +246,9 @@ func fanOutCountRows(ctx context.Context, c *counter, keys []string, displayFor
|
||||
wg.Add(1)
|
||||
go func(i int, k string) {
|
||||
defer wg.Done()
|
||||
var entry countEntry
|
||||
if err := c.kv.GetJSON(ctx, k, &entry); err != nil {
|
||||
log.Error("stats: kv get failed during render", "key", k, "err", err)
|
||||
entry, _, err := c.counts.Get(ctx, k)
|
||||
if err != nil {
|
||||
log.Error("stats: store get failed during render", "key", k, "err", err)
|
||||
return
|
||||
}
|
||||
results[i] = result{r: row{display: displayFor(k), n: entry.N}, ok: true}
|
||||
@@ -268,9 +268,9 @@ func fanOutCountRows(ctx context.Context, c *counter, keys []string, displayFor
|
||||
// loadUserRowsWithID returns userID → userEntry for every user:* row. Used by
|
||||
// viewTopUsers (renders) and viewUserCommands (resolves a username to its ID).
|
||||
func loadUserRowsWithID(ctx context.Context, c *counter) map[int64]userEntry {
|
||||
keys, err := c.kv.List(ctx, userPrefix)
|
||||
keys, err := c.users.List(ctx, userPrefix)
|
||||
if err != nil {
|
||||
log.Error("stats: kv list failed", "err", err)
|
||||
log.Error("stats: store list failed", "err", err)
|
||||
return nil
|
||||
}
|
||||
type pair struct {
|
||||
@@ -289,9 +289,9 @@ func loadUserRowsWithID(ctx context.Context, c *counter) map[int64]userEntry {
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
var entry userEntry
|
||||
if err := c.kv.GetJSON(ctx, k, &entry); err != nil {
|
||||
log.Error("stats: kv get failed", "key", k, "err", err)
|
||||
entry, _, err := c.users.Get(ctx, k)
|
||||
if err != nil {
|
||||
log.Error("stats: store get failed", "key", k, "err", err)
|
||||
return
|
||||
}
|
||||
results[i] = pair{id: id, entry: entry, ok: true}
|
||||
|
||||
@@ -13,14 +13,13 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/keylock"
|
||||
"github.com/tiennm99/miti99bot/internal/log"
|
||||
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// state is the per-module runtime. KV is module-scoped (the framework
|
||||
// state is the per-module runtime. store is module-scoped (the framework
|
||||
// prefixes/partitions). PriceClient is reused across calls; nowFn allows
|
||||
// tests to inject a deterministic clock for portfolio CreatedAt.
|
||||
type state struct {
|
||||
kv storage.KVStore
|
||||
store Store
|
||||
prices *PriceClient
|
||||
locks keylock.Map
|
||||
nowFn func() time.Time
|
||||
@@ -35,9 +34,9 @@ func (s *state) now() time.Time {
|
||||
}
|
||||
|
||||
// newState builds the default state used by the module factory.
|
||||
func newState(kv storage.KVStore) *state {
|
||||
func newState(store Store) *state {
|
||||
return &state{
|
||||
kv: kv,
|
||||
store: store,
|
||||
prices: &PriceClient{},
|
||||
comingSoonMessage: "Crypto, gold & currency exchange coming soon!",
|
||||
}
|
||||
@@ -87,14 +86,14 @@ func (s *state) handleTopup(ctx context.Context, b *bot.Bot, update *models.Upda
|
||||
|
||||
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
|
||||
|
||||
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
|
||||
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
|
||||
if err != nil {
|
||||
log.Error("stock_load_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
|
||||
}
|
||||
p.AddCurrency("VND", amount)
|
||||
p.Meta.Invested += amount
|
||||
if err := SavePortfolio(ctx, s.kv, userID, p); err != nil {
|
||||
if err := SavePortfolio(ctx, s.store, userID, p); err != nil {
|
||||
log.Error("stock_save_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not save portfolio. Try again later.")
|
||||
}
|
||||
@@ -138,7 +137,7 @@ func (s *state) handleBuy(ctx context.Context, b *bot.Bot, update *models.Update
|
||||
|
||||
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
|
||||
|
||||
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
|
||||
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
|
||||
if err != nil {
|
||||
log.Error("stock_load_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
|
||||
@@ -149,7 +148,7 @@ func (s *state) handleBuy(ctx context.Context, b *bot.Bot, update *models.Update
|
||||
"Insufficient VND. Need "+FormatVND(cost)+", have "+FormatVND(balance)+".")
|
||||
}
|
||||
p.AddAsset(symbol, qty)
|
||||
if err := SavePortfolio(ctx, s.kv, userID, p); err != nil {
|
||||
if err := SavePortfolio(ctx, s.store, userID, p); err != nil {
|
||||
log.Error("stock_save_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not save portfolio. Try again later.")
|
||||
}
|
||||
@@ -196,7 +195,7 @@ func (s *state) handleSell(ctx context.Context, b *bot.Bot, update *models.Updat
|
||||
|
||||
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
|
||||
|
||||
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
|
||||
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
|
||||
if err != nil {
|
||||
log.Error("stock_load_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
|
||||
@@ -208,7 +207,7 @@ func (s *state) handleSell(ctx context.Context, b *bot.Bot, update *models.Updat
|
||||
}
|
||||
revenue := float64(qty) * price
|
||||
p.AddCurrency("VND", revenue)
|
||||
if err := SavePortfolio(ctx, s.kv, userID, p); err != nil {
|
||||
if err := SavePortfolio(ctx, s.store, userID, p); err != nil {
|
||||
log.Error("stock_save_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not save portfolio. Try again later.")
|
||||
}
|
||||
@@ -245,7 +244,7 @@ func (s *state) handleIncomeStock(ctx context.Context, b *bot.Bot, update *model
|
||||
|
||||
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
|
||||
|
||||
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
|
||||
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
|
||||
if err != nil {
|
||||
log.Error("stock_load_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
|
||||
@@ -256,7 +255,7 @@ func (s *state) handleIncomeStock(ctx context.Context, b *bot.Bot, update *model
|
||||
"You don't hold any "+symbol+" to receive a stock dividend.")
|
||||
}
|
||||
p.AddAsset(symbol, qty)
|
||||
if err := SavePortfolio(ctx, s.kv, userID, p); err != nil {
|
||||
if err := SavePortfolio(ctx, s.store, userID, p); err != nil {
|
||||
log.Error("stock_save_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not save portfolio. Try again later.")
|
||||
}
|
||||
@@ -292,7 +291,7 @@ func (s *state) handleIncomeVND(ctx context.Context, b *bot.Bot, update *models.
|
||||
|
||||
defer s.locks.Acquire(strconv.FormatInt(userID, 10))()
|
||||
|
||||
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
|
||||
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
|
||||
if err != nil {
|
||||
log.Error("stock_load_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
|
||||
@@ -304,7 +303,7 @@ func (s *state) handleIncomeVND(ctx context.Context, b *bot.Bot, update *models.
|
||||
}
|
||||
total := amountPerShare * float64(held)
|
||||
p.AddCurrency("VND", total)
|
||||
if err := SavePortfolio(ctx, s.kv, userID, p); err != nil {
|
||||
if err := SavePortfolio(ctx, s.store, userID, p); err != nil {
|
||||
log.Error("stock_save_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not save portfolio. Try again later.")
|
||||
}
|
||||
@@ -330,7 +329,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
|
||||
return chathelper.Reply(ctx, b, update.Message,
|
||||
"Cannot identify user — /stock_stats needs a sender.")
|
||||
}
|
||||
p, err := LoadPortfolio(ctx, s.kv, userID, s.now().UnixMilli())
|
||||
p, err := LoadPortfolio(ctx, s.store, userID, s.now().UnixMilli())
|
||||
if err != nil {
|
||||
log.Error("stock_load_portfolio", "user", userID, "err", err)
|
||||
return chathelper.Reply(ctx, b, update.Message, "Could not load portfolio. Try again later.")
|
||||
|
||||
@@ -9,19 +9,22 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// Store is the stock module's typed portfolio store.
|
||||
type Store = storage.DocStore[Portfolio]
|
||||
|
||||
// Portfolio is the per-user stock state. Currency is a map for forward-
|
||||
// compat with USD/EUR (currently VND-only). Assets is a flat ticker→qty map.
|
||||
type Portfolio struct {
|
||||
Currency map[string]float64 `json:"currency"`
|
||||
Assets map[string]int64 `json:"assets"`
|
||||
Meta PortfolioMeta `json:"meta"`
|
||||
Currency map[string]float64 `json:"currency" bson:"currency"`
|
||||
Assets map[string]int64 `json:"assets" bson:"assets"`
|
||||
Meta PortfolioMeta `json:"meta" bson:"meta"`
|
||||
}
|
||||
|
||||
// PortfolioMeta tracks invested cost basis for P&L. CreatedAt is purely
|
||||
// informational (ms-epoch when the portfolio first existed).
|
||||
type PortfolioMeta struct {
|
||||
Invested float64 `json:"invested"`
|
||||
CreatedAt int64 `json:"createdAt"`
|
||||
Invested float64 `json:"invested" bson:"invested"`
|
||||
CreatedAt int64 `json:"createdAt" bson:"createdAt"`
|
||||
}
|
||||
|
||||
// NewPortfolio returns an empty starting state. Currency map seeded with VND=0
|
||||
@@ -39,11 +42,10 @@ func portfolioKey(userID int64) string {
|
||||
return "user:" + strconv.FormatInt(userID, 10)
|
||||
}
|
||||
|
||||
// LoadPortfolio reads from KV; returns an empty portfolio on first-time use.
|
||||
// LoadPortfolio reads from store; returns an empty portfolio on first-time use.
|
||||
// Defensively initialises nil maps so callers never need a nil check.
|
||||
func LoadPortfolio(ctx context.Context, kv storage.KVStore, userID int64, now int64) (Portfolio, error) {
|
||||
var p Portfolio
|
||||
err := kv.GetJSON(ctx, portfolioKey(userID), &p)
|
||||
func LoadPortfolio(ctx context.Context, store Store, userID int64, now int64) (Portfolio, error) {
|
||||
p, _, err := store.Get(ctx, portfolioKey(userID))
|
||||
switch {
|
||||
case err == nil:
|
||||
// Repair any nils from older / partial saves — defence in depth.
|
||||
@@ -64,8 +66,8 @@ func LoadPortfolio(ctx context.Context, kv storage.KVStore, userID int64, now in
|
||||
}
|
||||
|
||||
// SavePortfolio persists the portfolio.
|
||||
func SavePortfolio(ctx context.Context, kv storage.KVStore, userID int64, p Portfolio) error {
|
||||
if err := kv.PutJSON(ctx, portfolioKey(userID), p); err != nil {
|
||||
func SavePortfolio(ctx context.Context, store Store, userID int64, p Portfolio) error {
|
||||
if err := store.Put(ctx, portfolioKey(userID), p); err != nil {
|
||||
return fmt.Errorf("stock: save portfolio %d: %w", userID, err)
|
||||
}
|
||||
return nil
|
||||
|
||||
@@ -7,9 +7,14 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// newStockStore returns a fresh in-memory typed portfolio store for tests.
|
||||
func newStockStore() Store {
|
||||
return storage.Typed[Portfolio](storage.NewMemoryProvider().Collection("stock"))
|
||||
}
|
||||
|
||||
func TestLoadPortfolio_FirstTimeUser(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
p, err := LoadPortfolio(context.Background(), kv, 42, 1234567890)
|
||||
store := newStockStore()
|
||||
p, err := LoadPortfolio(context.Background(), store, 42, 1234567890)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadPortfolio: %v", err)
|
||||
}
|
||||
@@ -25,15 +30,15 @@ func TestLoadPortfolio_FirstTimeUser(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestSaveAndLoadRoundTrip(t *testing.T) {
|
||||
kv := storage.NewMemoryKVStore()
|
||||
p, _ := LoadPortfolio(context.Background(), kv, 42, 1)
|
||||
store := newStockStore()
|
||||
p, _ := LoadPortfolio(context.Background(), store, 42, 1)
|
||||
p.AddCurrency("VND", 5_000_000)
|
||||
p.AddAsset("TCB", 100)
|
||||
p.Meta.Invested = 5_000_000
|
||||
if err := SavePortfolio(context.Background(), kv, 42, p); err != nil {
|
||||
if err := SavePortfolio(context.Background(), store, 42, p); err != nil {
|
||||
t.Fatalf("Save: %v", err)
|
||||
}
|
||||
got, err := LoadPortfolio(context.Background(), kv, 42, 999) // CreatedAt should NOT be reset
|
||||
got, err := LoadPortfolio(context.Background(), store, 42, 999) // CreatedAt should NOT be reset
|
||||
if err != nil {
|
||||
t.Fatalf("Load: %v", err)
|
||||
}
|
||||
|
||||
@@ -10,7 +10,6 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
"github.com/tiennm99/miti99bot/internal/testutil"
|
||||
)
|
||||
|
||||
@@ -37,19 +36,19 @@ func TestHandleStats_UsesSSIBatchPrices(t *testing.T) {
|
||||
}))
|
||||
t.Cleanup(priceSrv.Close)
|
||||
|
||||
kv := storage.NewMemoryKVStore()
|
||||
store := newStockStore()
|
||||
p := NewPortfolio(now.UnixMilli())
|
||||
p.Currency["VND"] = 2335000
|
||||
p.Meta.Invested = 1000000000
|
||||
p.AddAsset("MWG", 1800)
|
||||
p.AddAsset("TCB", 4200)
|
||||
p.AddAsset("FPT", 2300)
|
||||
if err := SavePortfolio(ctx, kv, 7, p); err != nil {
|
||||
if err := SavePortfolio(ctx, store, 7, p); err != nil {
|
||||
t.Fatalf("SavePortfolio: %v", err)
|
||||
}
|
||||
|
||||
s := &state{
|
||||
kv: kv,
|
||||
store: store,
|
||||
prices: &PriceClient{HTTP: priceSrv.Client(), URL: priceSrv.URL},
|
||||
nowFn: func() time.Time { return now },
|
||||
}
|
||||
|
||||
@@ -2,13 +2,14 @@ package stock
|
||||
|
||||
import (
|
||||
"github.com/tiennm99/miti99bot/internal/modules"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// New is the stock module Factory. Five user-facing commands; no crons.
|
||||
// (Original miti99bot only has a SQL retention cron, which our KV-only port
|
||||
// does not implement — keeping commits paper-ledger-only is acceptable.)
|
||||
func New(deps modules.Deps) modules.Module {
|
||||
s := newState(deps.KV)
|
||||
s := newState(storage.Typed[Portfolio](deps.Store))
|
||||
return modules.Module{
|
||||
Commands: []modules.Command{
|
||||
{
|
||||
|
||||
@@ -15,7 +15,6 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/keylock"
|
||||
"github.com/tiennm99/miti99bot/internal/log"
|
||||
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
const (
|
||||
@@ -26,7 +25,8 @@ const (
|
||||
)
|
||||
|
||||
type state struct {
|
||||
kv storage.KVStore
|
||||
games GameStore
|
||||
stats StatsStore
|
||||
chatter ai.Chatter
|
||||
limiter *ai.PerUserLimiter
|
||||
|
||||
@@ -88,7 +88,7 @@ func (s *state) handleTwentyq(ctx context.Context, b *bot.Bot, update *models.Up
|
||||
|
||||
arg := chathelper.ArgAfterCommand(msg.Text)
|
||||
|
||||
game, err := loadGame(ctx, s.kv, subject)
|
||||
game, err := loadGame(ctx, s.games, subject)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -99,7 +99,7 @@ func (s *state) handleTwentyq(ctx context.Context, b *bot.Bot, update *models.Up
|
||||
// branch re-enters every call and the user is stuck. Log but keep
|
||||
// going — the fresh-round write below will overwrite the slot
|
||||
// regardless.
|
||||
if err := clearGame(ctx, s.kv, subject); err != nil {
|
||||
if err := clearGame(ctx, s.games, subject); err != nil {
|
||||
log.Warn("twentyq clearGame on solved-relaunch failed", "subject", subject, "err", err)
|
||||
}
|
||||
game = nil
|
||||
@@ -117,7 +117,7 @@ func (s *state) handleTwentyq(ctx context.Context, b *bot.Bot, update *models.Up
|
||||
log.Warn("twentyq roundstart failed", "err", err)
|
||||
return chathelper.Reply(ctx, b, msg, upstreamFail)
|
||||
}
|
||||
if err := saveGame(ctx, s.kv, subject, fresh); err != nil {
|
||||
if err := saveGame(ctx, s.games, subject, fresh); err != nil {
|
||||
return err
|
||||
}
|
||||
if arg == "" {
|
||||
@@ -182,17 +182,17 @@ func (s *state) submitTurn(ctx context.Context, b *bot.Bot, msg *models.Message,
|
||||
if won {
|
||||
game.Solved = true
|
||||
count := len(game.Turns)
|
||||
if _, err := recordResult(ctx, s.kv, subject, true, count, chathelper.NowMillis()); err != nil {
|
||||
if _, err := recordResult(ctx, s.stats, subject, true, count, chathelper.NowMillis()); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := clearGame(ctx, s.kv, subject); err != nil {
|
||||
if err := clearGame(ctx, s.games, subject); err != nil {
|
||||
return err
|
||||
}
|
||||
return chathelper.ReplyHTML(ctx, b, msg,
|
||||
formatTurnReply(turn, true, game.Target, count))
|
||||
}
|
||||
|
||||
if err := saveGame(ctx, s.kv, subject, game); err != nil {
|
||||
if err := saveGame(ctx, s.games, subject, game); err != nil {
|
||||
return err
|
||||
}
|
||||
return chathelper.ReplyHTML(ctx, b, msg,
|
||||
@@ -209,17 +209,17 @@ func (s *state) handleGiveup(ctx context.Context, b *bot.Bot, update *models.Upd
|
||||
return chathelper.Reply(ctx, b, msg, "Cannot identify chat.")
|
||||
}
|
||||
defer s.locks.Acquire(subject)()
|
||||
game, err := loadGame(ctx, s.kv, subject)
|
||||
game, err := loadGame(ctx, s.games, subject)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if game == nil {
|
||||
return chathelper.ReplyHTML(ctx, b, msg, noRound)
|
||||
}
|
||||
if _, err := recordResult(ctx, s.kv, subject, false, len(game.Turns), chathelper.NowMillis()); err != nil {
|
||||
if _, err := recordResult(ctx, s.stats, subject, false, len(game.Turns), chathelper.NowMillis()); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := clearGame(ctx, s.kv, subject); err != nil {
|
||||
if err := clearGame(ctx, s.games, subject); err != nil {
|
||||
return err
|
||||
}
|
||||
return chathelper.ReplyHTML(ctx, b, msg, formatGiveup(*game))
|
||||
@@ -234,7 +234,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
|
||||
if subject == "" {
|
||||
return chathelper.Reply(ctx, b, msg, "Cannot identify chat.")
|
||||
}
|
||||
st, err := loadStats(ctx, s.kv, subject)
|
||||
st, err := loadStats(ctx, s.stats, subject)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -10,36 +10,41 @@ import (
|
||||
|
||||
// Turn is one Q&A entry stored in the game's history.
|
||||
type Turn struct {
|
||||
Text string `json:"text"`
|
||||
IsGuess bool `json:"isGuess"`
|
||||
Answer string `json:"answer"` // "yes" | "no"
|
||||
Hint string `json:"hint"`
|
||||
TS int64 `json:"ts"`
|
||||
Text string `json:"text" bson:"text"`
|
||||
IsGuess bool `json:"isGuess" bson:"isGuess"`
|
||||
Answer string `json:"answer" bson:"answer"` // "yes" | "no"
|
||||
Hint string `json:"hint" bson:"hint"`
|
||||
TS int64 `json:"ts" bson:"ts"`
|
||||
}
|
||||
|
||||
type GameState struct {
|
||||
Category string `json:"category"`
|
||||
Target string `json:"target"`
|
||||
InitialHint string `json:"initialHint"`
|
||||
StartedAt *int64 `json:"startedAt"`
|
||||
Solved bool `json:"solved"`
|
||||
Turns []Turn `json:"turns"`
|
||||
Category string `json:"category" bson:"category"`
|
||||
Target string `json:"target" bson:"target"`
|
||||
InitialHint string `json:"initialHint" bson:"initialHint"`
|
||||
StartedAt *int64 `json:"startedAt" bson:"startedAt"`
|
||||
Solved bool `json:"solved" bson:"solved"`
|
||||
Turns []Turn `json:"turns" bson:"turns"`
|
||||
}
|
||||
|
||||
type Stats struct {
|
||||
Played int `json:"played"`
|
||||
Solved int `json:"solved"`
|
||||
TotalTurns int `json:"totalTurns"`
|
||||
BestTurnCount *int `json:"bestTurnCount"`
|
||||
LastResultAt *int64 `json:"lastResultAt"`
|
||||
Played int `json:"played" bson:"played"`
|
||||
Solved int `json:"solved" bson:"solved"`
|
||||
TotalTurns int `json:"totalTurns" bson:"totalTurns"`
|
||||
BestTurnCount *int `json:"bestTurnCount" bson:"bestTurnCount"`
|
||||
LastResultAt *int64 `json:"lastResultAt" bson:"lastResultAt"`
|
||||
}
|
||||
|
||||
// GameStore is the twentyq module's typed game store.
|
||||
type GameStore = storage.DocStore[GameState]
|
||||
|
||||
// StatsStore is the twentyq module's typed stats store.
|
||||
type StatsStore = storage.DocStore[Stats]
|
||||
|
||||
func gameKey(subject string) string { return "game:" + subject }
|
||||
func statsKey(subject string) string { return "stats:" + subject }
|
||||
|
||||
func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*GameState, error) {
|
||||
var g GameState
|
||||
err := kv.GetJSON(ctx, gameKey(subject), &g)
|
||||
func loadGame(ctx context.Context, games GameStore, subject string) (*GameState, error) {
|
||||
g, _, err := games.Get(ctx, gameKey(subject))
|
||||
switch {
|
||||
case err == nil:
|
||||
return &g, nil
|
||||
@@ -50,23 +55,22 @@ func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*GameSta
|
||||
}
|
||||
}
|
||||
|
||||
func saveGame(ctx context.Context, kv storage.KVStore, subject string, g *GameState) error {
|
||||
if err := kv.PutJSON(ctx, gameKey(subject), g); err != nil {
|
||||
func saveGame(ctx context.Context, games GameStore, subject string, g *GameState) error {
|
||||
if err := games.Put(ctx, gameKey(subject), *g); err != nil {
|
||||
return fmt.Errorf("twentyq saveGame: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func clearGame(ctx context.Context, kv storage.KVStore, subject string) error {
|
||||
if err := kv.Delete(ctx, gameKey(subject)); err != nil && !errors.Is(err, storage.ErrNotFound) {
|
||||
func clearGame(ctx context.Context, games GameStore, subject string) error {
|
||||
if err := games.Delete(ctx, gameKey(subject)); err != nil && !errors.Is(err, storage.ErrNotFound) {
|
||||
return fmt.Errorf("twentyq clearGame: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*Stats, error) {
|
||||
var s Stats
|
||||
err := kv.GetJSON(ctx, statsKey(subject), &s)
|
||||
func loadStats(ctx context.Context, st StatsStore, subject string) (*Stats, error) {
|
||||
s, _, err := st.Get(ctx, statsKey(subject))
|
||||
switch {
|
||||
case err == nil:
|
||||
return &s, nil
|
||||
@@ -77,8 +81,8 @@ func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*Stats,
|
||||
}
|
||||
}
|
||||
|
||||
func recordResult(ctx context.Context, kv storage.KVStore, subject string, solved bool, turnCount int, nowMillis int64) (*Stats, error) {
|
||||
s, err := loadStats(ctx, kv, subject)
|
||||
func recordResult(ctx context.Context, st StatsStore, subject string, solved bool, turnCount int, nowMillis int64) (*Stats, error) {
|
||||
s, err := loadStats(ctx, st, subject)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -93,7 +97,7 @@ func recordResult(ctx context.Context, kv storage.KVStore, subject string, solve
|
||||
}
|
||||
now := nowMillis
|
||||
s.LastResultAt = &now
|
||||
if err := kv.PutJSON(ctx, statsKey(subject), s); err != nil {
|
||||
if err := st.Put(ctx, statsKey(subject), *s); err != nil {
|
||||
return nil, fmt.Errorf("twentyq recordResult: %w", err)
|
||||
}
|
||||
return s, nil
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
|
||||
"github.com/tiennm99/miti99bot/internal/ai"
|
||||
"github.com/tiennm99/miti99bot/internal/modules"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// New is the twentyq module Factory. If Deps.Chatter is nil
|
||||
@@ -14,7 +15,8 @@ import (
|
||||
// config-error message — keeps the rest of the bot functional.
|
||||
func New(deps modules.Deps) modules.Module {
|
||||
s := &state{
|
||||
kv: deps.KV,
|
||||
games: storage.Typed[GameState](deps.Store),
|
||||
stats: storage.Typed[Stats](deps.Store),
|
||||
chatter: deps.Chatter,
|
||||
limiter: ai.NewPerUserLimiter(5.0/60.0, 5),
|
||||
rng: newRNG(),
|
||||
|
||||
@@ -14,10 +14,12 @@ const (
|
||||
ResultWrong = "wrong"
|
||||
)
|
||||
|
||||
// LetterScore is the JSON shape stored in KV per guess.
|
||||
// LetterScore is the shape stored per guess (nested in GameState). bson tags
|
||||
// mirror the json names so migrated docs (which keep the original JSON keys)
|
||||
// read back verbatim.
|
||||
type LetterScore struct {
|
||||
Letter string `json:"letter"`
|
||||
Result string `json:"result"`
|
||||
Letter string `json:"letter" bson:"letter"`
|
||||
Result string `json:"result" bson:"result"`
|
||||
}
|
||||
|
||||
// CompareWords scores guess against target letter-by-letter. Both are assumed
|
||||
|
||||
@@ -10,13 +10,13 @@ import (
|
||||
|
||||
"github.com/tiennm99/miti99bot/internal/keylock"
|
||||
"github.com/tiennm99/miti99bot/internal/modules/util/chathelper"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// state captures everything a wordle command needs at handler-time. Built
|
||||
// once in New and shared across all four handlers via closure.
|
||||
type state struct {
|
||||
kv storage.KVStore
|
||||
games GameStore
|
||||
stats StatsStore
|
||||
words []string
|
||||
set map[string]struct{}
|
||||
locks keylock.Map // per-subject mutex; serialises Get→mutate→Put
|
||||
@@ -35,7 +35,7 @@ func rejectMessage(reason rejectReason) string {
|
||||
}
|
||||
|
||||
// startFresh writes a brand-new round and returns it. Errors propagate to
|
||||
// the caller (not swallowed — a KV failure means subsequent ops will lie).
|
||||
// the caller (not swallowed — a store failure means subsequent ops will lie).
|
||||
func (s *state) startFresh(ctx context.Context, subject string) (*GameState, error) {
|
||||
target, err := pickRandom(s.words, nil)
|
||||
if err != nil {
|
||||
@@ -48,14 +48,14 @@ func (s *state) startFresh(ctx context.Context, subject string) (*GameState, err
|
||||
Giveup: false,
|
||||
StartedAt: chathelper.NowMillis(),
|
||||
}
|
||||
if err := saveGame(ctx, s.kv, subject, g); err != nil {
|
||||
if err := saveGame(ctx, s.games, subject, g); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return g, nil
|
||||
}
|
||||
|
||||
func (s *state) getOrInit(ctx context.Context, subject string) (*GameState, error) {
|
||||
g, err := loadGame(ctx, s.kv, subject)
|
||||
g, err := loadGame(ctx, s.games, subject)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -112,21 +112,21 @@ func (s *state) handleWordle(ctx context.Context, b *bot.Bot, update *models.Upd
|
||||
if won {
|
||||
g.Solved = true
|
||||
}
|
||||
if err := saveGame(ctx, s.kv, subject, g); err != nil {
|
||||
if err := saveGame(ctx, s.games, subject, g); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
rendered := renderGuess(v.Word, results)
|
||||
switch {
|
||||
case won:
|
||||
stats, err := recordResult(ctx, s.kv, subject, true, chathelper.NowMillis())
|
||||
st, err := recordResult(ctx, s.stats, subject, true, chathelper.NowMillis())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("%s\n\n🎉 Solved in %d/%d! Streak: %d. /wordle_new for another.",
|
||||
rendered, len(g.Guesses), MaxGuesses, stats.Streak))
|
||||
rendered, len(g.Guesses), MaxGuesses, st.Streak))
|
||||
case len(g.Guesses) >= MaxGuesses:
|
||||
if _, err := recordResult(ctx, s.kv, subject, false, chathelper.NowMillis()); err != nil {
|
||||
if _, err := recordResult(ctx, s.stats, subject, false, chathelper.NowMillis()); err != nil {
|
||||
return err
|
||||
}
|
||||
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("%s\n\n❌ Out of guesses. Answer was %s. /wordle_new to retry.",
|
||||
@@ -150,12 +150,12 @@ func (s *state) handleNew(ctx context.Context, b *bot.Bot, update *models.Update
|
||||
defer s.locks.Acquire(subject)()
|
||||
|
||||
prelude := ""
|
||||
prior, err := loadGame(ctx, s.kv, subject)
|
||||
prior, err := loadGame(ctx, s.games, subject)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if prior != nil && !isFinished(prior) {
|
||||
if _, err := recordResult(ctx, s.kv, subject, false, chathelper.NowMillis()); err != nil {
|
||||
if _, err := recordResult(ctx, s.stats, subject, false, chathelper.NowMillis()); err != nil {
|
||||
return err
|
||||
}
|
||||
prelude = fmt.Sprintf("🏳️ Previous round abandoned (auto-giveup). Answer was %s.\n\n",
|
||||
@@ -191,10 +191,10 @@ func (s *state) handleGiveup(ctx context.Context, b *bot.Bot, update *models.Upd
|
||||
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("Already gave up — %s.", strings.ToUpper(g.Target)))
|
||||
}
|
||||
g.Giveup = true
|
||||
if err := saveGame(ctx, s.kv, subject, g); err != nil {
|
||||
if err := saveGame(ctx, s.games, subject, g); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := recordResult(ctx, s.kv, subject, false, chathelper.NowMillis()); err != nil {
|
||||
if _, err := recordResult(ctx, s.stats, subject, false, chathelper.NowMillis()); err != nil {
|
||||
return err
|
||||
}
|
||||
return chathelper.Reply(ctx, b, msg, fmt.Sprintf("🏳️ Answer was %s. /wordle_new for another.", strings.ToUpper(g.Target)))
|
||||
@@ -210,7 +210,7 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
|
||||
if subject == "" {
|
||||
return chathelper.Reply(ctx, b, msg, "Cannot identify chat.")
|
||||
}
|
||||
stats, err := loadStats(ctx, s.kv, subject)
|
||||
st, err := loadStats(ctx, s.stats, subject)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -220,6 +220,6 @@ func (s *state) handleStats(ctx context.Context, b *bot.Bot, update *models.Upda
|
||||
}
|
||||
return chathelper.Reply(ctx, b, msg, fmt.Sprintf(
|
||||
"📊 Wordle %s stats\nPlayed: %d\nWins: %d (%d%%)\nCurrent streak: %d\nBest streak: %d",
|
||||
scope, stats.Played, stats.Wins, chathelper.WinRate(stats.Wins, stats.Played), stats.Streak, stats.BestStreak,
|
||||
scope, st.Played, st.Wins, chathelper.WinRate(st.Wins, st.Played), st.Streak, st.BestStreak,
|
||||
))
|
||||
}
|
||||
@@ -11,15 +11,15 @@ import (
|
||||
)
|
||||
|
||||
// installWordle wires the wordle module to a recording bot with an
|
||||
// in-memory KV. seedTarget pre-seeds a game with the supplied target
|
||||
// in-memory store. seedTarget pre-seeds a game with the supplied target
|
||||
// (skip dictionary randomness for deterministic tests). Empty seedTarget
|
||||
// leaves the KV blank so handlers spin up a fresh round on first call.
|
||||
func installWordle(t *testing.T, ownerID int64, seedSubject, seedTarget string) (*testutil.RecordingBot, storage.KVStore) {
|
||||
// leaves the store blank so handlers spin up a fresh round on first call.
|
||||
func installWordle(t *testing.T, ownerID int64, seedSubject, seedTarget string) (*testutil.RecordingBot, GameStore) {
|
||||
t.Helper()
|
||||
rb := testutil.NewRecordingBot(t)
|
||||
provider := storage.NewMemoryProvider()
|
||||
kv := provider.For("wordle")
|
||||
mod := New(modules.Deps{KV: kv})
|
||||
col := storage.NewMemoryProvider().Collection("wordle")
|
||||
games := storage.Typed[GameState](col)
|
||||
mod := New(modules.Deps{Store: col})
|
||||
reg := &modules.Registry{
|
||||
Modules: []modules.Module{{Name: "wordle", Commands: mod.Commands}},
|
||||
AllCommands: map[string]modules.Command{},
|
||||
@@ -31,11 +31,11 @@ func installWordle(t *testing.T, ownerID int64, seedSubject, seedTarget string)
|
||||
|
||||
if seedTarget != "" {
|
||||
g := &GameState{Target: seedTarget, Guesses: []GuessRecord{}, StartedAt: 1}
|
||||
if err := saveGame(context.Background(), kv, seedSubject, g); err != nil {
|
||||
if err := saveGame(context.Background(), games, seedSubject, g); err != nil {
|
||||
t.Fatalf("seed game: %v", err)
|
||||
}
|
||||
}
|
||||
return rb, kv
|
||||
return rb, games
|
||||
}
|
||||
|
||||
func TestWordle_NoArgShowsBoard(t *testing.T) {
|
||||
@@ -146,21 +146,24 @@ func TestWordleStats_AfterWin(t *testing.T) {
|
||||
// Group chats key by chat id, so two private users both writing /wordle
|
||||
// in the same group must mutate the same game.
|
||||
func TestWordle_GroupSubjectIsChatID(t *testing.T) {
|
||||
rb, kv := installWordle(t, 0, "-100", "crane")
|
||||
rb, games := installWordle(t, 0, "-100", "crane")
|
||||
rb.Bot.ProcessUpdate(context.Background(), testutil.NewGroupMessage(-100, 7, "/wordle crate"))
|
||||
|
||||
// After one guess in chat -100, subject "-100" should have a guess
|
||||
// recorded; subject "7" should have no game.
|
||||
var gChat GameState
|
||||
if err := kv.GetJSON(context.Background(), gameKey("-100"), &gChat); err != nil {
|
||||
gChat, err := loadGame(context.Background(), games, "-100")
|
||||
if err != nil {
|
||||
t.Fatalf("expected game at subject=-100: %v", err)
|
||||
}
|
||||
if len(gChat.Guesses) != 1 {
|
||||
t.Errorf("group game has %d guesses, want 1", len(gChat.Guesses))
|
||||
if gChat == nil || len(gChat.Guesses) != 1 {
|
||||
t.Errorf("group game guesses = %v, want 1", gChat)
|
||||
}
|
||||
var gUser GameState
|
||||
err := kv.GetJSON(context.Background(), gameKey("7"), &gUser)
|
||||
if err == nil {
|
||||
|
||||
gUser, err := loadGame(context.Background(), games, "7")
|
||||
if err != nil {
|
||||
t.Fatalf("loadGame for user subject: %v", err)
|
||||
}
|
||||
if gUser != nil {
|
||||
t.Errorf("user-keyed game leaked despite group context: %+v", gUser)
|
||||
}
|
||||
}
|
||||
@@ -11,7 +11,7 @@ import (
|
||||
// MaxGuesses is the standard wordle round length.
|
||||
const MaxGuesses = 6
|
||||
|
||||
// Note: the KV store has no native per-document TTL — saved games linger
|
||||
// Note: the store has no native per-document TTL — saved games linger
|
||||
// until manually cleaned. Out of scope today; could be added via a sweep
|
||||
// cron if storage cost ever matters.
|
||||
|
||||
@@ -19,43 +19,47 @@ const MaxGuesses = 6
|
||||
//
|
||||
// { "word": "crane", "results": [{"letter":"c","result":"correct"}, ...] }
|
||||
type GuessRecord struct {
|
||||
Word string `json:"word"`
|
||||
Results []LetterScore `json:"results"`
|
||||
Word string `json:"word" bson:"word"`
|
||||
Results []LetterScore `json:"results" bson:"results"`
|
||||
}
|
||||
|
||||
// GameState is the per-subject KV record for an in-progress (or finished)
|
||||
// round.
|
||||
// GameState is the per-subject record for an in-progress (or finished) round.
|
||||
//
|
||||
// `giveup` is always emitted (initialized to false on /wordle_new). Do NOT
|
||||
// add omitempty — the field is part of the stored document's shape, so
|
||||
// emitting it unconditionally keeps already-saved games self-describing
|
||||
// when inspected via raw KV dumps.
|
||||
// when inspected via raw dumps.
|
||||
type GameState struct {
|
||||
Target string `json:"target"`
|
||||
Guesses []GuessRecord `json:"guesses"`
|
||||
Solved bool `json:"solved"`
|
||||
Giveup bool `json:"giveup"`
|
||||
StartedAt int64 `json:"startedAt"` // ms-since-epoch (Date.now())
|
||||
Target string `json:"target" bson:"target"`
|
||||
Guesses []GuessRecord `json:"guesses" bson:"guesses"`
|
||||
Solved bool `json:"solved" bson:"solved"`
|
||||
Giveup bool `json:"giveup" bson:"giveup"`
|
||||
StartedAt int64 `json:"startedAt" bson:"startedAt"` // ms-since-epoch (Date.now())
|
||||
}
|
||||
|
||||
// Stats is the lifetime score record. lastResultAt is *int64 so an unplayed
|
||||
// account marshals as `"lastResultAt": null` — distinguishes "never played"
|
||||
// from "played at epoch zero".
|
||||
type Stats struct {
|
||||
Played int `json:"played"`
|
||||
Wins int `json:"wins"`
|
||||
Streak int `json:"streak"`
|
||||
BestStreak int `json:"bestStreak"`
|
||||
LastResultAt *int64 `json:"lastResultAt"` // ms-since-epoch | null
|
||||
Played int `json:"played" bson:"played"`
|
||||
Wins int `json:"wins" bson:"wins"`
|
||||
Streak int `json:"streak" bson:"streak"`
|
||||
BestStreak int `json:"bestStreak" bson:"bestStreak"`
|
||||
LastResultAt *int64 `json:"lastResultAt" bson:"lastResultAt"` // ms-since-epoch | null
|
||||
}
|
||||
|
||||
// GameStore is the wordle module's typed game store.
|
||||
type GameStore = storage.DocStore[GameState]
|
||||
|
||||
// StatsStore is the wordle module's typed stats store.
|
||||
type StatsStore = storage.DocStore[Stats]
|
||||
|
||||
func gameKey(subject string) string { return "game:" + subject }
|
||||
func statsKey(subject string) string { return "stats:" + subject }
|
||||
|
||||
// loadGame returns the active round, or (nil, nil) if none exists.
|
||||
func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*GameState, error) {
|
||||
var g GameState
|
||||
err := kv.GetJSON(ctx, gameKey(subject), &g)
|
||||
func loadGame(ctx context.Context, games GameStore, subject string) (*GameState, error) {
|
||||
g, _, err := games.Get(ctx, gameKey(subject))
|
||||
switch {
|
||||
case err == nil:
|
||||
return &g, nil
|
||||
@@ -67,8 +71,8 @@ func loadGame(ctx context.Context, kv storage.KVStore, subject string) (*GameSta
|
||||
}
|
||||
|
||||
// saveGame writes the round.
|
||||
func saveGame(ctx context.Context, kv storage.KVStore, subject string, g *GameState) error {
|
||||
if err := kv.PutJSON(ctx, gameKey(subject), g); err != nil {
|
||||
func saveGame(ctx context.Context, games GameStore, subject string, g *GameState) error {
|
||||
if err := games.Put(ctx, gameKey(subject), *g); err != nil {
|
||||
return fmt.Errorf("wordle saveGame: %w", err)
|
||||
}
|
||||
return nil
|
||||
@@ -76,9 +80,8 @@ func saveGame(ctx context.Context, kv storage.KVStore, subject string, g *GameSt
|
||||
|
||||
// loadStats returns lifetime stats; missing → fresh-zero record (with
|
||||
// LastResultAt=nil) so callers never need a nil check.
|
||||
func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*Stats, error) {
|
||||
var s Stats
|
||||
err := kv.GetJSON(ctx, statsKey(subject), &s)
|
||||
func loadStats(ctx context.Context, stats StatsStore, subject string) (*Stats, error) {
|
||||
s, _, err := stats.Get(ctx, statsKey(subject))
|
||||
switch {
|
||||
case err == nil:
|
||||
return &s, nil
|
||||
@@ -92,8 +95,8 @@ func loadStats(ctx context.Context, kv storage.KVStore, subject string) (*Stats,
|
||||
// recordResult bumps stats with the round outcome (won true → win + streak,
|
||||
// false → reset streak). Returns the updated stats so callers can show the
|
||||
// new streak in the win message.
|
||||
func recordResult(ctx context.Context, kv storage.KVStore, subject string, won bool, nowMillis int64) (*Stats, error) {
|
||||
s, err := loadStats(ctx, kv, subject)
|
||||
func recordResult(ctx context.Context, stats StatsStore, subject string, won bool, nowMillis int64) (*Stats, error) {
|
||||
s, err := loadStats(ctx, stats, subject)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -109,7 +112,7 @@ func recordResult(ctx context.Context, kv storage.KVStore, subject string, won b
|
||||
}
|
||||
now := nowMillis
|
||||
s.LastResultAt = &now
|
||||
if err := kv.PutJSON(ctx, statsKey(subject), s); err != nil {
|
||||
if err := stats.Put(ctx, statsKey(subject), *s); err != nil {
|
||||
return nil, fmt.Errorf("wordle recordResult: %w", err)
|
||||
}
|
||||
return s, nil
|
||||
|
||||
@@ -8,6 +8,21 @@ import (
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
func newWordleGames() GameStore {
|
||||
return storage.Typed[GameState](storage.NewMemoryProvider().Collection("wordle"))
|
||||
}
|
||||
|
||||
func newWordleStats() StatsStore {
|
||||
return storage.Typed[Stats](storage.NewMemoryProvider().Collection("wordle"))
|
||||
}
|
||||
|
||||
// newWordleStores returns a games + stats store backed by the same collection
|
||||
// (disjoint key prefixes: "game:" vs "stats:").
|
||||
func newWordleStores() (GameStore, StatsStore) {
|
||||
c := storage.NewMemoryProvider().Collection("wordle")
|
||||
return storage.Typed[GameState](c), storage.Typed[Stats](c)
|
||||
}
|
||||
|
||||
func TestStats_DefaultLastResultAtIsNull(t *testing.T) {
|
||||
// Go's *int64 must marshal as null when nil, so unplayed accounts emit
|
||||
// `"lastResultAt": null` and the field stays distinguishable from
|
||||
@@ -62,9 +77,9 @@ func TestGameState_JSONShapeIsStable(t *testing.T) {
|
||||
|
||||
func TestRecordResult_WinIncrementsStreak(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
stats := newWordleStats()
|
||||
|
||||
s, err := recordResult(ctx, kv, "u1", true, 100)
|
||||
s, err := recordResult(ctx, stats, "u1", true, 100)
|
||||
if err != nil {
|
||||
t.Fatalf("recordResult: %v", err)
|
||||
}
|
||||
@@ -76,7 +91,7 @@ func TestRecordResult_WinIncrementsStreak(t *testing.T) {
|
||||
}
|
||||
|
||||
// Second win bumps streak; bestStreak follows.
|
||||
s, _ = recordResult(ctx, kv, "u1", true, 200)
|
||||
s, _ = recordResult(ctx, stats, "u1", true, 200)
|
||||
if s.Streak != 2 || s.BestStreak != 2 {
|
||||
t.Errorf("two wins: streak=%d best=%d", s.Streak, s.BestStreak)
|
||||
}
|
||||
@@ -84,11 +99,11 @@ func TestRecordResult_WinIncrementsStreak(t *testing.T) {
|
||||
|
||||
func TestRecordResult_LossResetsStreakKeepsBest(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
stats := newWordleStats()
|
||||
|
||||
_, _ = recordResult(ctx, kv, "u1", true, 100)
|
||||
_, _ = recordResult(ctx, kv, "u1", true, 200)
|
||||
s, _ := recordResult(ctx, kv, "u1", false, 300)
|
||||
_, _ = recordResult(ctx, stats, "u1", true, 100)
|
||||
_, _ = recordResult(ctx, stats, "u1", true, 200)
|
||||
s, _ := recordResult(ctx, stats, "u1", false, 300)
|
||||
if s.Streak != 0 {
|
||||
t.Errorf("loss should reset streak, got %d", s.Streak)
|
||||
}
|
||||
@@ -101,7 +116,7 @@ func TestRecordResult_LossResetsStreakKeepsBest(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestLoadGame_MissingReturnsNil(t *testing.T) {
|
||||
g, err := loadGame(context.Background(), storage.NewMemoryKVStore(), "nobody")
|
||||
g, err := loadGame(context.Background(), newWordleGames(), "nobody")
|
||||
if err != nil {
|
||||
t.Errorf("missing game should not error: %v", err)
|
||||
}
|
||||
@@ -112,12 +127,12 @@ func TestLoadGame_MissingReturnsNil(t *testing.T) {
|
||||
|
||||
func TestSaveGame_RoundTrip(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
kv := storage.NewMemoryKVStore()
|
||||
games := newWordleGames()
|
||||
want := &GameState{Target: "crane", Guesses: []GuessRecord{}, StartedAt: 42}
|
||||
if err := saveGame(ctx, kv, "u1", want); err != nil {
|
||||
if err := saveGame(ctx, games, "u1", want); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := loadGame(ctx, kv, "u1")
|
||||
got, err := loadGame(ctx, games, "u1")
|
||||
if err != nil || got == nil {
|
||||
t.Fatalf("loadGame: got=%v err=%v", got, err)
|
||||
}
|
||||
|
||||
@@ -2,13 +2,19 @@ package wordle
|
||||
|
||||
import (
|
||||
"github.com/tiennm99/miti99bot/internal/modules"
|
||||
"github.com/tiennm99/miti99bot/internal/storage"
|
||||
)
|
||||
|
||||
// New is the wordle module Factory. Loads the embedded dictionary once,
|
||||
// captures the per-module KV via closure, and registers all four commands.
|
||||
// captures the per-module typed stores via closure, and registers all four commands.
|
||||
func New(deps modules.Deps) modules.Module {
|
||||
words, set := loadWords()
|
||||
s := &state{kv: deps.KV, words: words, set: set}
|
||||
s := &state{
|
||||
games: storage.Typed[GameState](deps.Store),
|
||||
stats: storage.Typed[Stats](deps.Store),
|
||||
words: words,
|
||||
set: set,
|
||||
}
|
||||
|
||||
return modules.Module{
|
||||
Commands: []modules.Command{
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ErrNotFound is returned by a DocStore when a key has no value.
|
||||
var ErrNotFound = errors.New("storage: key not found")
|
||||
|
||||
// ErrConflict is returned by versioned writes when the stored version changed.
|
||||
var ErrConflict = errors.New("storage: write conflict")
|
||||
|
||||
// ErrInvalidModuleName is returned by every op on a store built from an invalid
|
||||
// module/collection name — surfacing the configuration bug at first use rather
|
||||
// than silently reading/writing an attacker-controllable collection name.
|
||||
var ErrInvalidModuleName = errors.New("storage: invalid module name")
|
||||
|
||||
// DocStore is the per-module, per-payload-type storage contract. A value is
|
||||
// persisted as a native document (its fields hoisted to the document root)
|
||||
// alongside an optimistic-locking version. Implementations are safe for
|
||||
// concurrent use and return ErrNotFound for missing keys.
|
||||
//
|
||||
// Concurrency control is version-based: each key carries a monotonic version, so
|
||||
// a writer swaps only if the version it read is unchanged. This decouples the
|
||||
// concurrency control from the value encoding (values are native documents, not
|
||||
// exact byte blobs).
|
||||
//
|
||||
// Contract:
|
||||
// - Get returns the value and its current version, or ErrNotFound.
|
||||
// - Put overwrites unconditionally and bumps the version.
|
||||
// - PutVersioned writes only if the stored version equals expectedVersion,
|
||||
// then bumps it. expectedVersion == 0 means "create (or adopt a not-yet-
|
||||
// versioned key)". A mismatch returns ErrConflict.
|
||||
type DocStore[T any] interface {
|
||||
Get(ctx context.Context, id string) (val T, version int64, err error)
|
||||
Put(ctx context.Context, id string, val T) error
|
||||
PutVersioned(ctx context.Context, id string, expectedVersion int64, val T) error
|
||||
Delete(ctx context.Context, id string) error
|
||||
List(ctx context.Context, prefix string) ([]string, error)
|
||||
}
|
||||
|
||||
// Provider yields a per-module Collection handle. Implementations decide how
|
||||
// isolation is achieved: MemoryProvider keeps one map per module, MongoProvider
|
||||
// uses one MongoDB collection per module. Modules never construct stores
|
||||
// directly — they receive a Collection through their factory's Deps and build
|
||||
// typed views with Typed.
|
||||
type Provider interface {
|
||||
Collection(module string) Collection
|
||||
}
|
||||
|
||||
// Collection is an opaque per-module storage handle. Build a typed store over it
|
||||
// with Typed[T]. A module may build several typed views over the same
|
||||
// Collection (one per payload type) as long as their key prefixes are disjoint.
|
||||
type Collection interface {
|
||||
// isCollection seals the interface to this package's implementations so
|
||||
// Typed's type switch is exhaustive.
|
||||
isCollection()
|
||||
}
|
||||
|
||||
// Typed returns a DocStore[T] over c. Go methods cannot be generic, so this is a
|
||||
// package-level function rather than a Provider method; it is the single switch
|
||||
// point that binds a Collection to its backend's typed store.
|
||||
//
|
||||
// It panics if T is a struct whose BSON field names collide with a reserved root
|
||||
// field (_id, version, updatedAt) — a programmer error caught at startup, in the
|
||||
// same spirit as Prefixed panicking on an empty prefix.
|
||||
func Typed[T any](c Collection) DocStore[T] {
|
||||
if err := checkReservedFields[T](); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
switch h := c.(type) {
|
||||
case mongoCollection:
|
||||
return &mongoDocStore[T]{coll: h.coll, module: h.module}
|
||||
case *memoryCollection:
|
||||
return &memoryDocStore[T]{c: h}
|
||||
case invalidCollection:
|
||||
return invalidDocStore[T]{name: h.name}
|
||||
default:
|
||||
panic(fmt.Sprintf("storage: unknown collection type %T", c))
|
||||
}
|
||||
}
|
||||
|
||||
// reservedRootFields are the document-root field names the store owns; a payload
|
||||
// type must not define BSON tags that collide with them (inline marshalling
|
||||
// would otherwise produce duplicate keys).
|
||||
var reservedRootFields = map[string]bool{
|
||||
mongoIDField: true,
|
||||
mongoVersionField: true,
|
||||
mongoUpdatedAtField: true,
|
||||
}
|
||||
|
||||
// checkReservedFields reports an error if struct type T declares a BSON field
|
||||
// name colliding with a reserved root field. Non-struct T (e.g. memory-only
|
||||
// test usage) is skipped — only the inline Mongo encoding needs the guard.
|
||||
func checkReservedFields[T any]() error {
|
||||
t := reflect.TypeOf((*T)(nil)).Elem()
|
||||
if t.Kind() != reflect.Struct {
|
||||
return nil
|
||||
}
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
f := t.Field(i)
|
||||
if !f.IsExported() {
|
||||
continue
|
||||
}
|
||||
name := bsonFieldName(f)
|
||||
if reservedRootFields[name] {
|
||||
return fmt.Errorf("storage: payload type %s field %q maps to reserved root field %q", t.Name(), f.Name, name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// bsonFieldName returns the BSON document field name for a struct field: the
|
||||
// explicit `bson:"name"` tag if present, else the Go field name (matching the
|
||||
// driver's default of using the field name verbatim).
|
||||
func bsonFieldName(f reflect.StructField) string {
|
||||
tag := f.Tag.Get("bson")
|
||||
if tag == "" {
|
||||
return f.Name
|
||||
}
|
||||
name := strings.Split(tag, ",")[0]
|
||||
if name == "" {
|
||||
return f.Name
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
// invalidCollection is returned by a Provider for a module name that fails
|
||||
// validation. Typed turns it into an invalidDocStore.
|
||||
type invalidCollection struct{ name string }
|
||||
|
||||
func (invalidCollection) isCollection() {}
|
||||
|
||||
// invalidDocStore errors on every operation, wrapping ErrInvalidModuleName.
|
||||
type invalidDocStore[T any] struct{ name string }
|
||||
|
||||
func (s invalidDocStore[T]) err(op string) error {
|
||||
return fmt.Errorf("%w: %q (op=%s)", ErrInvalidModuleName, s.name, op)
|
||||
}
|
||||
|
||||
func (s invalidDocStore[T]) Get(context.Context, string) (T, int64, error) {
|
||||
var z T
|
||||
return z, 0, s.err("Get")
|
||||
}
|
||||
func (s invalidDocStore[T]) Put(context.Context, string, T) error { return s.err("Put") }
|
||||
func (s invalidDocStore[T]) PutVersioned(context.Context, string, int64, T) error {
|
||||
return s.err("PutVersioned")
|
||||
}
|
||||
func (s invalidDocStore[T]) Delete(context.Context, string) error { return s.err("Delete") }
|
||||
func (s invalidDocStore[T]) List(context.Context, string) ([]string, error) {
|
||||
return nil, s.err("List")
|
||||
}
|
||||
@@ -1,181 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/aws/aws-sdk-go-v2/aws"
|
||||
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
|
||||
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
|
||||
)
|
||||
|
||||
// DynamoDB attribute names. `value` is reserved by DynamoDB expressions, so
|
||||
// every reference goes through ExpressionAttributeNames (#v).
|
||||
const (
|
||||
dynamoPKAttr = "pk"
|
||||
dynamoSKAttr = "sk"
|
||||
dynamoValueAttr = "value"
|
||||
dynamoUpdatedAtAttr = "updatedAt"
|
||||
)
|
||||
|
||||
// DynamoDBKVStore is a KVStore backed by a single DynamoDB table with a
|
||||
// composite key (pk, sk). FirestoreProvider uses one collection per module;
|
||||
// DynamoDBProvider uses one partition (pk = moduleName) per module — the
|
||||
// table itself is shared. The user-supplied key becomes the sort key, so
|
||||
// List(prefix) maps to a Query with begins_with(sk, prefix).
|
||||
type DynamoDBKVStore struct {
|
||||
client *dynamodb.Client
|
||||
table string
|
||||
moduleName string
|
||||
}
|
||||
|
||||
// NewDynamoDBKVStore returns a store partition-scoped to moduleName.
|
||||
// Callers must validate moduleName beforehand (DynamoDBProvider does).
|
||||
func NewDynamoDBKVStore(client *dynamodb.Client, table, moduleName string) *DynamoDBKVStore {
|
||||
return &DynamoDBKVStore{client: client, table: table, moduleName: moduleName}
|
||||
}
|
||||
|
||||
// Get returns the raw bytes stored at key, or ErrNotFound. Strong read for
|
||||
// parity with the Firestore impl's strong-read default.
|
||||
func (s *DynamoDBKVStore) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
if err := validateKey(key); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out, err := s.client.GetItem(ctx, &dynamodb.GetItemInput{
|
||||
TableName: aws.String(s.table),
|
||||
ConsistentRead: aws.Bool(true),
|
||||
Key: map[string]types.AttributeValue{
|
||||
dynamoPKAttr: &types.AttributeValueMemberS{Value: s.moduleName},
|
||||
dynamoSKAttr: &types.AttributeValueMemberS{Value: key},
|
||||
},
|
||||
ExpressionAttributeNames: map[string]string{"#v": dynamoValueAttr},
|
||||
ProjectionExpression: aws.String("#v"),
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("dynamodb get %s/%s: %w", s.moduleName, key, err)
|
||||
}
|
||||
if len(out.Item) == 0 {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
rawAttr, ok := out.Item[dynamoValueAttr]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("dynamodb get %s/%s: missing %q attribute", s.moduleName, key, dynamoValueAttr)
|
||||
}
|
||||
str, ok := rawAttr.(*types.AttributeValueMemberS)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("dynamodb get %s/%s: unexpected attribute type %T", s.moduleName, key, rawAttr)
|
||||
}
|
||||
return []byte(str.Value), nil
|
||||
}
|
||||
|
||||
// GetJSON decodes the value at key into dst.
|
||||
func (s *DynamoDBKVStore) GetJSON(ctx context.Context, key string, dst any) error {
|
||||
raw, err := s.Get(ctx, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.Unmarshal(raw, dst); err != nil {
|
||||
return fmt.Errorf("dynamodb get %s/%s: json decode: %w", s.moduleName, key, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Put writes raw bytes at key, creating or overwriting. The value attribute
|
||||
// is stored as a DynamoDB String so it is human-readable in the AWS console.
|
||||
// Every current caller writes JSON, which is UTF-8 safe; non-UTF-8 callers
|
||||
// must encode upstream (e.g. base64).
|
||||
func (s *DynamoDBKVStore) Put(ctx context.Context, key string, val []byte) error {
|
||||
if err := validateKey(key); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err := s.client.PutItem(ctx, &dynamodb.PutItemInput{
|
||||
TableName: aws.String(s.table),
|
||||
Item: map[string]types.AttributeValue{
|
||||
dynamoPKAttr: &types.AttributeValueMemberS{Value: s.moduleName},
|
||||
dynamoSKAttr: &types.AttributeValueMemberS{Value: key},
|
||||
dynamoValueAttr: &types.AttributeValueMemberS{Value: string(val)},
|
||||
dynamoUpdatedAtAttr: &types.AttributeValueMemberN{Value: strconv.FormatInt(time.Now().UTC().UnixNano(), 10)},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("dynamodb put %s/%s: %w", s.moduleName, key, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Note: DynamoDB intentionally does NOT implement VersionedStore. Post-self-host
|
||||
// it is reachable only via the data migrator (Scan + Put), which never needs
|
||||
// optimistic locking. The live runtime backend is MongoDB; memory covers tests.
|
||||
|
||||
// PutJSON marshals val and writes the bytes at key.
|
||||
func (s *DynamoDBKVStore) PutJSON(ctx context.Context, key string, val any) error {
|
||||
raw, err := json.Marshal(val)
|
||||
if err != nil {
|
||||
return fmt.Errorf("dynamodb put %s/%s: json encode: %w", s.moduleName, key, err)
|
||||
}
|
||||
return s.Put(ctx, key, raw)
|
||||
}
|
||||
|
||||
// Delete removes the item at key. Deleting a missing key is not an error.
|
||||
func (s *DynamoDBKVStore) Delete(ctx context.Context, key string) error {
|
||||
if err := validateKey(key); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err := s.client.DeleteItem(ctx, &dynamodb.DeleteItemInput{
|
||||
TableName: aws.String(s.table),
|
||||
Key: map[string]types.AttributeValue{
|
||||
dynamoPKAttr: &types.AttributeValueMemberS{Value: s.moduleName},
|
||||
dynamoSKAttr: &types.AttributeValueMemberS{Value: key},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("dynamodb delete %s/%s: %w", s.moduleName, key, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// List returns all sort keys in the partition that start with prefix. Empty
|
||||
// prefix scans the entire partition (Query is still cheap because it's
|
||||
// partition-bounded; no Scan).
|
||||
func (s *DynamoDBKVStore) List(ctx context.Context, prefix string) ([]string, error) {
|
||||
if err := validatePrefix(prefix); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
input := &dynamodb.QueryInput{
|
||||
TableName: aws.String(s.table),
|
||||
ProjectionExpression: aws.String(dynamoSKAttr),
|
||||
ExpressionAttributeValues: map[string]types.AttributeValue{
|
||||
":pk": &types.AttributeValueMemberS{Value: s.moduleName},
|
||||
},
|
||||
}
|
||||
if prefix == "" {
|
||||
input.KeyConditionExpression = aws.String("pk = :pk")
|
||||
} else {
|
||||
input.KeyConditionExpression = aws.String("pk = :pk AND begins_with(sk, :prefix)")
|
||||
input.ExpressionAttributeValues[":prefix"] = &types.AttributeValueMemberS{Value: prefix}
|
||||
}
|
||||
|
||||
var keys []string
|
||||
pager := dynamodb.NewQueryPaginator(s.client, input)
|
||||
for pager.HasMorePages() {
|
||||
page, err := pager.NextPage(ctx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("dynamodb list %s prefix=%q: %w", s.moduleName, prefix, err)
|
||||
}
|
||||
for _, item := range page.Items {
|
||||
skAttr, ok := item[dynamoSKAttr]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
sk, ok := skAttr.(*types.AttributeValueMemberS)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
keys = append(keys, sk.Value)
|
||||
}
|
||||
}
|
||||
return keys, nil
|
||||
}
|
||||
@@ -1,166 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/aws/aws-sdk-go-v2/aws"
|
||||
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
|
||||
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
|
||||
)
|
||||
|
||||
// dynamoDBLocalSetup ensures a fresh table exists in DynamoDB Local for the
|
||||
// test run. Returns a client + cleanup. Tests skip if DYNAMODB_LOCAL_URL is
|
||||
// unset so CI without docker still builds.
|
||||
func dynamoDBLocalSetup(t *testing.T) (client *dynamodb.Client, table string, cleanup func()) {
|
||||
t.Helper()
|
||||
endpoint := os.Getenv("DYNAMODB_LOCAL_URL")
|
||||
if endpoint == "" {
|
||||
t.Skip("DYNAMODB_LOCAL_URL not set; skipping DynamoDB integration test (run `make dynamodb-local` to start the local container)")
|
||||
}
|
||||
// DynamoDB Local accepts any non-empty creds + region.
|
||||
t.Setenv("AWS_ACCESS_KEY_ID", "test")
|
||||
t.Setenv("AWS_SECRET_ACCESS_KEY", "test")
|
||||
t.Setenv("AWS_REGION", "ap-southeast-1")
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
c, err := NewDynamoDBClient(ctx, endpoint)
|
||||
if err != nil {
|
||||
t.Fatalf("NewDynamoDBClient: %v", err)
|
||||
}
|
||||
|
||||
table = "test-" + t.Name()
|
||||
if len(table) > 64 {
|
||||
table = table[:64]
|
||||
}
|
||||
|
||||
_, err = c.CreateTable(ctx, &dynamodb.CreateTableInput{
|
||||
TableName: aws.String(table),
|
||||
BillingMode: types.BillingModePayPerRequest,
|
||||
AttributeDefinitions: []types.AttributeDefinition{
|
||||
{AttributeName: aws.String("pk"), AttributeType: types.ScalarAttributeTypeS},
|
||||
{AttributeName: aws.String("sk"), AttributeType: types.ScalarAttributeTypeS},
|
||||
},
|
||||
KeySchema: []types.KeySchemaElement{
|
||||
{AttributeName: aws.String("pk"), KeyType: types.KeyTypeHash},
|
||||
{AttributeName: aws.String("sk"), KeyType: types.KeyTypeRange},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateTable: %v", err)
|
||||
}
|
||||
|
||||
cleanup = func() {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
_, _ = c.DeleteTable(ctx, &dynamodb.DeleteTableInput{TableName: aws.String(table)})
|
||||
}
|
||||
return c, table, cleanup
|
||||
}
|
||||
|
||||
func TestDynamoDBKVStore_PutGetDelete(t *testing.T) {
|
||||
client, table, cleanup := dynamoDBLocalSetup(t)
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
s := NewDynamoDBKVStore(client, table, "wordle")
|
||||
|
||||
if err := s.Put(ctx, "user:1:state", []byte("hello")); err != nil {
|
||||
t.Fatalf("Put: %v", err)
|
||||
}
|
||||
got, err := s.Get(ctx, "user:1:state")
|
||||
if err != nil {
|
||||
t.Fatalf("Get: %v", err)
|
||||
}
|
||||
if string(got) != "hello" {
|
||||
t.Errorf("Get: got %q, want %q", got, "hello")
|
||||
}
|
||||
|
||||
if err := s.Delete(ctx, "user:1:state"); err != nil {
|
||||
t.Fatalf("Delete: %v", err)
|
||||
}
|
||||
if _, err := s.Get(ctx, "user:1:state"); !errors.Is(err, ErrNotFound) {
|
||||
t.Errorf("Get after Delete: got %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDynamoDBKVStore_GetMissing(t *testing.T) {
|
||||
client, table, cleanup := dynamoDBLocalSetup(t)
|
||||
defer cleanup()
|
||||
|
||||
s := NewDynamoDBKVStore(client, table, "wordle")
|
||||
_, err := s.Get(context.Background(), "missing")
|
||||
if !errors.Is(err, ErrNotFound) {
|
||||
t.Errorf("got %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDynamoDBKVStore_JSONRoundTrip(t *testing.T) {
|
||||
client, table, cleanup := dynamoDBLocalSetup(t)
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
s := NewDynamoDBKVStore(client, table, "loldle")
|
||||
|
||||
type state struct {
|
||||
Score int `json:"score"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
in := state{Score: 42, Name: "ezreal"}
|
||||
if err := s.PutJSON(ctx, "u1", in); err != nil {
|
||||
t.Fatalf("PutJSON: %v", err)
|
||||
}
|
||||
var out state
|
||||
if err := s.GetJSON(ctx, "u1", &out); err != nil {
|
||||
t.Fatalf("GetJSON: %v", err)
|
||||
}
|
||||
if out != in {
|
||||
t.Errorf("got %+v, want %+v", out, in)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDynamoDBKVStore_ListPrefix(t *testing.T) {
|
||||
client, table, cleanup := dynamoDBLocalSetup(t)
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
s := NewDynamoDBKVStore(client, table, "wordle")
|
||||
|
||||
for _, k := range []string{"user:1:state", "user:2:state", "config:daily", "user:1:history"} {
|
||||
if err := s.Put(ctx, k, []byte("x")); err != nil {
|
||||
t.Fatalf("Put %s: %v", k, err)
|
||||
}
|
||||
}
|
||||
|
||||
got, err := s.List(ctx, "user:")
|
||||
if err != nil {
|
||||
t.Fatalf("List: %v", err)
|
||||
}
|
||||
want := map[string]bool{"user:1:state": true, "user:2:state": true, "user:1:history": true}
|
||||
if len(got) != len(want) {
|
||||
t.Errorf("List: got %v (len=%d), want len=%d", got, len(got), len(want))
|
||||
}
|
||||
for _, k := range got {
|
||||
if !want[k] {
|
||||
t.Errorf("List: unexpected key %q", k)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// DynamoDB intentionally no longer implements optimistic locking — it is
|
||||
// migrate-only post-self-host (Scan + Put). No CAS/versioned test here.
|
||||
|
||||
func TestDynamoDBKVStore_DeleteMissingNoError(t *testing.T) {
|
||||
client, table, cleanup := dynamoDBLocalSetup(t)
|
||||
defer cleanup()
|
||||
|
||||
s := NewDynamoDBKVStore(client, table, "wordle")
|
||||
if err := s.Delete(context.Background(), "never-existed"); err != nil {
|
||||
t.Errorf("Delete missing key: got %v, want nil (idempotent)", err)
|
||||
}
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
|
||||
)
|
||||
|
||||
// DynamoDBProvider is a KVProvider that hands out per-module stores backed by
|
||||
// a single shared DynamoDB table. Module isolation is by partition (pk =
|
||||
// moduleName); no key-prefix wrapping is needed at this layer.
|
||||
type DynamoDBProvider struct {
|
||||
client *dynamodb.Client
|
||||
table string
|
||||
}
|
||||
|
||||
// NewDynamoDBProvider returns a provider over the given client + table.
|
||||
// The client is goroutine-safe and outlives every store handed out by For.
|
||||
func NewDynamoDBProvider(client *dynamodb.Client, table string) *DynamoDBProvider {
|
||||
return &DynamoDBProvider{client: client, table: table}
|
||||
}
|
||||
|
||||
// For returns a DynamoDBKVStore partitioned by moduleName. moduleName is
|
||||
// re-validated against the same alphabet enforced by FirestoreProvider so
|
||||
// behaviour is identical between backends. An invalid name yields an
|
||||
// invalidStore — every op errors at first use, surfacing the bug early.
|
||||
func (p *DynamoDBProvider) For(moduleName string) KVStore {
|
||||
if !collectionNameRe.MatchString(moduleName) {
|
||||
return invalidStore{name: moduleName}
|
||||
}
|
||||
return NewDynamoDBKVStore(p.client, p.table, moduleName)
|
||||
}
|
||||
@@ -1,61 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestDynamoDBProvider_CrossModuleIsolation(t *testing.T) {
|
||||
client, table, cleanup := dynamoDBLocalSetup(t)
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
p := NewDynamoDBProvider(client, table)
|
||||
|
||||
wordle := p.For("wordle")
|
||||
loldle := p.For("loldle")
|
||||
|
||||
if err := wordle.Put(ctx, "k", []byte("from-wordle")); err != nil {
|
||||
t.Fatalf("wordle.Put: %v", err)
|
||||
}
|
||||
if err := loldle.Put(ctx, "k", []byte("from-loldle")); err != nil {
|
||||
t.Fatalf("loldle.Put: %v", err)
|
||||
}
|
||||
|
||||
got, err := wordle.Get(ctx, "k")
|
||||
if err != nil {
|
||||
t.Fatalf("wordle.Get: %v", err)
|
||||
}
|
||||
if string(got) != "from-wordle" {
|
||||
t.Errorf("wordle.Get: got %q, want from-wordle", got)
|
||||
}
|
||||
|
||||
got, err = loldle.Get(ctx, "k")
|
||||
if err != nil {
|
||||
t.Fatalf("loldle.Get: %v", err)
|
||||
}
|
||||
if string(got) != "from-loldle" {
|
||||
t.Errorf("loldle.Get: got %q, want from-loldle", got)
|
||||
}
|
||||
|
||||
// List should not leak across modules.
|
||||
wkeys, err := wordle.List(ctx, "")
|
||||
if err != nil {
|
||||
t.Fatalf("wordle.List: %v", err)
|
||||
}
|
||||
if len(wkeys) != 1 || wkeys[0] != "k" {
|
||||
t.Errorf("wordle.List: got %v, want [k]", wkeys)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDynamoDBProvider_InvalidModuleName(t *testing.T) {
|
||||
// No DynamoDB needed — invalidStore short-circuits before any SDK call.
|
||||
p := NewDynamoDBProvider(nil, "any")
|
||||
store := p.For("../../etc/passwd")
|
||||
|
||||
_, err := store.Get(context.Background(), "k")
|
||||
if !errors.Is(err, ErrInvalidModuleName) {
|
||||
t.Errorf("Get: got %v, want ErrInvalidModuleName", err)
|
||||
}
|
||||
}
|
||||
@@ -1,35 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// ErrInvalidModuleName is returned by every operation on an invalidStore —
|
||||
// the sentinel emitted when FirestoreProvider.For is asked for a module
|
||||
// whose name fails collectionNameRe.
|
||||
var ErrInvalidModuleName = fmt.Errorf("storage: invalid module name")
|
||||
|
||||
// invalidStore is a KVStore that errors on every call. Returned by
|
||||
// FirestoreProvider.For when the requested module name doesn't validate.
|
||||
// Callers see a real KVStore but every op errors at use, surfacing the
|
||||
// configuration bug at the first read/write rather than silently writing
|
||||
// to an attacker-controllable collection name.
|
||||
type invalidStore struct {
|
||||
name string
|
||||
}
|
||||
|
||||
func (s invalidStore) wrap(op string) error {
|
||||
return fmt.Errorf("%w: %q (op=%s)", ErrInvalidModuleName, s.name, op)
|
||||
}
|
||||
|
||||
func (s invalidStore) Get(_ context.Context, _ string) ([]byte, error) {
|
||||
return nil, s.wrap("Get")
|
||||
}
|
||||
func (s invalidStore) GetJSON(_ context.Context, _ string, _ any) error { return s.wrap("GetJSON") }
|
||||
func (s invalidStore) Put(_ context.Context, _ string, _ []byte) error { return s.wrap("Put") }
|
||||
func (s invalidStore) PutJSON(_ context.Context, _ string, _ any) error { return s.wrap("PutJSON") }
|
||||
func (s invalidStore) Delete(_ context.Context, _ string) error { return s.wrap("Delete") }
|
||||
func (s invalidStore) List(_ context.Context, _ string) ([]string, error) {
|
||||
return nil, s.wrap("List")
|
||||
}
|
||||
@@ -1,35 +0,0 @@
|
||||
package storage
|
||||
|
||||
// KVProvider yields a per-module KVStore. Implementations decide how isolation
|
||||
// is achieved: MemoryProvider wraps a single in-process store with a key
|
||||
// prefix, FirestoreProvider uses one collection per module.
|
||||
//
|
||||
// Modules never construct KVStores directly — they receive one through their
|
||||
// factory's Deps. This keeps the storage backend swappable without touching
|
||||
// any module code.
|
||||
type KVProvider interface {
|
||||
For(moduleName string) KVStore
|
||||
}
|
||||
|
||||
// MemoryProvider is a KVProvider backed by a single in-process MemoryKVStore.
|
||||
// Each module sees a Prefixed view that prevents cross-module key collisions.
|
||||
//
|
||||
// Intended for tests, local smoke runs, and the no-Firestore fallback. State
|
||||
// is lost when the process exits.
|
||||
type MemoryProvider struct {
|
||||
base *MemoryKVStore
|
||||
}
|
||||
|
||||
// NewMemoryProvider returns a fresh in-process provider.
|
||||
func NewMemoryProvider() *MemoryProvider {
|
||||
return &MemoryProvider{base: NewMemoryKVStore()}
|
||||
}
|
||||
|
||||
// For returns a per-module view of the shared in-memory store.
|
||||
func (m *MemoryProvider) For(moduleName string) KVStore {
|
||||
return Prefixed(m.base, moduleName)
|
||||
}
|
||||
|
||||
// Base exposes the underlying unprefixed store. Tests use this to assert
|
||||
// cross-module isolation by reading raw keys; production code must not.
|
||||
func (m *MemoryProvider) Base() *MemoryKVStore { return m.base }
|
||||
@@ -1,41 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
)
|
||||
|
||||
// ErrNotFound is returned by KVStore implementations when a key has no value.
|
||||
var ErrNotFound = errors.New("storage: key not found")
|
||||
|
||||
// ErrConflict is returned by conditional writes when the stored value changed.
|
||||
var ErrConflict = errors.New("storage: write conflict")
|
||||
|
||||
// KVStore is the per-module key-value contract. Implementations must be safe
|
||||
// for concurrent use and must return ErrNotFound for missing keys.
|
||||
type KVStore interface {
|
||||
Get(ctx context.Context, key string) ([]byte, error)
|
||||
GetJSON(ctx context.Context, key string, dst any) error
|
||||
Put(ctx context.Context, key string, val []byte) error
|
||||
PutJSON(ctx context.Context, key string, val any) error
|
||||
Delete(ctx context.Context, key string) error
|
||||
List(ctx context.Context, prefix string) ([]string, error)
|
||||
}
|
||||
|
||||
// VersionedStore is implemented by stores that support version-based optimistic
|
||||
// locking. It replaces value-bytes compare-and-swap: each key carries a
|
||||
// monotonic version, so a writer swaps only if the version it read is unchanged.
|
||||
// This decouples concurrency control from the value encoding (so values can be
|
||||
// stored as native documents rather than exact byte blobs).
|
||||
//
|
||||
// Contract:
|
||||
// - GetVersioned returns the value and its current version, or ErrNotFound.
|
||||
// A key that exists without a recorded version (written by an older build)
|
||||
// reports version 0 and no error.
|
||||
// - PutVersioned writes val only if the stored version still equals
|
||||
// expectedVersion, then bumps the version. expectedVersion == 0 means
|
||||
// "create, or adopt a not-yet-versioned key". A mismatch returns ErrConflict.
|
||||
type VersionedStore interface {
|
||||
GetVersioned(ctx context.Context, key string) (val []byte, version int64, err error)
|
||||
PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type testPayload struct {
|
||||
Name string `json:"name" bson:"name"`
|
||||
Count int `json:"count" bson:"count"`
|
||||
}
|
||||
|
||||
func memStore(module string) DocStore[testPayload] {
|
||||
return Typed[testPayload](NewMemoryProvider().Collection(module))
|
||||
}
|
||||
|
||||
func TestMemoryDocStore_PutGetRoundTrip(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := memStore("coin")
|
||||
if err := s.Put(ctx, "k", testPayload{Name: "a", Count: 3}); err != nil {
|
||||
t.Fatalf("Put: %v", err)
|
||||
}
|
||||
got, version, err := s.Get(ctx, "k")
|
||||
if err != nil {
|
||||
t.Fatalf("Get: %v", err)
|
||||
}
|
||||
if got.Name != "a" || got.Count != 3 {
|
||||
t.Fatalf("got %+v", got)
|
||||
}
|
||||
if version != 1 {
|
||||
t.Fatalf("version = %d, want 1", version)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryDocStore_GetMissing(t *testing.T) {
|
||||
if _, _, err := memStore("coin").Get(context.Background(), "nope"); !errors.Is(err, ErrNotFound) {
|
||||
t.Fatalf("Get missing = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryDocStore_PutBumpsVersion(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := memStore("coin")
|
||||
_ = s.Put(ctx, "k", testPayload{Count: 1})
|
||||
_ = s.Put(ctx, "k", testPayload{Count: 2})
|
||||
_, version, _ := s.Get(ctx, "k")
|
||||
if version != 2 {
|
||||
t.Fatalf("version = %d, want 2", version)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryDocStore_PutVersionedCAS(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := memStore("coin")
|
||||
// create
|
||||
if err := s.PutVersioned(ctx, "k", 0, testPayload{Count: 1}); err != nil {
|
||||
t.Fatalf("create: %v", err)
|
||||
}
|
||||
// second create must conflict
|
||||
if err := s.PutVersioned(ctx, "k", 0, testPayload{Count: 9}); !errors.Is(err, ErrConflict) {
|
||||
t.Fatalf("double create = %v, want ErrConflict", err)
|
||||
}
|
||||
// stale expected version conflicts
|
||||
if err := s.PutVersioned(ctx, "k", 99, testPayload{Count: 9}); !errors.Is(err, ErrConflict) {
|
||||
t.Fatalf("stale CAS = %v, want ErrConflict", err)
|
||||
}
|
||||
// fresh version succeeds
|
||||
if err := s.PutVersioned(ctx, "k", 1, testPayload{Count: 2}); err != nil {
|
||||
t.Fatalf("fresh CAS: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryDocStore_DeleteIdempotent(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := memStore("coin")
|
||||
_ = s.Put(ctx, "k", testPayload{})
|
||||
if err := s.Delete(ctx, "k"); err != nil {
|
||||
t.Fatalf("Delete: %v", err)
|
||||
}
|
||||
if err := s.Delete(ctx, "k"); err != nil {
|
||||
t.Fatalf("Delete missing must be nil, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryDocStore_ListPrefix(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := memStore("coin")
|
||||
_ = s.Put(ctx, "game:1", testPayload{})
|
||||
_ = s.Put(ctx, "game:2", testPayload{})
|
||||
_ = s.Put(ctx, "stats:1", testPayload{})
|
||||
keys, err := s.List(ctx, "game:")
|
||||
if err != nil {
|
||||
t.Fatalf("List: %v", err)
|
||||
}
|
||||
if len(keys) != 2 || keys[0] != "game:1" || keys[1] != "game:2" {
|
||||
t.Fatalf("List game: = %v", keys)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemoryDocStore_ValueIsolation(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := Typed[testPayloadWithSlice](NewMemoryProvider().Collection("coin"))
|
||||
in := testPayloadWithSlice{Items: []string{"x"}}
|
||||
_ = s.Put(ctx, "k", in)
|
||||
in.Items[0] = "mutated" // mutating the caller copy must not affect stored state
|
||||
got, _, _ := s.Get(ctx, "k")
|
||||
if got.Items[0] != "x" {
|
||||
t.Fatalf("stored value aliased caller slice: %v", got.Items)
|
||||
}
|
||||
}
|
||||
|
||||
type testPayloadWithSlice struct {
|
||||
Items []string `json:"items" bson:"items"`
|
||||
}
|
||||
|
||||
func TestCheckReservedFields(t *testing.T) {
|
||||
type bad struct {
|
||||
Version int `bson:"version"`
|
||||
}
|
||||
if err := checkReservedFields[bad](); err == nil {
|
||||
t.Fatal("expected reserved-field collision error for bson:\"version\"")
|
||||
}
|
||||
if err := checkReservedFields[testPayload](); err != nil {
|
||||
t.Fatalf("clean struct flagged: %v", err)
|
||||
}
|
||||
// Non-struct payloads are skipped, not rejected.
|
||||
if err := checkReservedFields[string](); err != nil {
|
||||
t.Fatalf("string payload flagged: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -1,119 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// memEntry is one stored value plus its optimistic-locking version.
|
||||
type memEntry struct {
|
||||
val []byte
|
||||
version int64
|
||||
}
|
||||
|
||||
// MemoryKVStore is an in-process KVStore for tests and local smoke runs.
|
||||
// Data is lost on restart; production uses the MongoDB provider.
|
||||
type MemoryKVStore struct {
|
||||
mu sync.RWMutex
|
||||
m map[string]memEntry
|
||||
}
|
||||
|
||||
// NewMemoryKVStore returns an empty in-memory store.
|
||||
func NewMemoryKVStore() *MemoryKVStore {
|
||||
return &MemoryKVStore{m: make(map[string]memEntry)}
|
||||
}
|
||||
|
||||
func (s *MemoryKVStore) Get(_ context.Context, key string) ([]byte, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
e, ok := s.m[key]
|
||||
if !ok {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
out := make([]byte, len(e.val))
|
||||
copy(out, e.val)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (s *MemoryKVStore) GetJSON(ctx context.Context, key string, dst any) error {
|
||||
raw, err := s.Get(ctx, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return json.NewDecoder(bytes.NewReader(raw)).Decode(dst)
|
||||
}
|
||||
|
||||
func (s *MemoryKVStore) Put(_ context.Context, key string, val []byte) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
stored := make([]byte, len(val))
|
||||
copy(stored, val)
|
||||
// A plain Put bumps the version too, so a concurrent versioned writer that
|
||||
// read the old version correctly sees a conflict.
|
||||
s.m[key] = memEntry{val: stored, version: s.m[key].version + 1}
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetVersioned returns the value and its version, or ErrNotFound.
|
||||
func (s *MemoryKVStore) GetVersioned(_ context.Context, key string) ([]byte, int64, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
e, ok := s.m[key]
|
||||
if !ok {
|
||||
return nil, 0, ErrNotFound
|
||||
}
|
||||
out := make([]byte, len(e.val))
|
||||
copy(out, e.val)
|
||||
return out, e.version, nil
|
||||
}
|
||||
|
||||
// PutVersioned writes val only if the stored version equals expectedVersion
|
||||
// (0 = must not exist yet), then bumps the version. ErrConflict on mismatch.
|
||||
func (s *MemoryKVStore) PutVersioned(_ context.Context, key string, expectedVersion int64, val []byte) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
e, ok := s.m[key]
|
||||
if expectedVersion == 0 {
|
||||
if ok {
|
||||
return ErrConflict
|
||||
}
|
||||
} else if !ok || e.version != expectedVersion {
|
||||
return ErrConflict
|
||||
}
|
||||
stored := make([]byte, len(val))
|
||||
copy(stored, val)
|
||||
s.m[key] = memEntry{val: stored, version: e.version + 1}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *MemoryKVStore) PutJSON(ctx context.Context, key string, val any) error {
|
||||
raw, err := json.Marshal(val)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.Put(ctx, key, raw)
|
||||
}
|
||||
|
||||
func (s *MemoryKVStore) Delete(_ context.Context, key string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
delete(s.m, key)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *MemoryKVStore) List(_ context.Context, prefix string) ([]string, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
keys := make([]string, 0)
|
||||
for k := range s.m {
|
||||
if strings.HasPrefix(k, prefix) {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
}
|
||||
sort.Strings(keys)
|
||||
return keys, nil
|
||||
}
|
||||
@@ -1,58 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestMemoryKVStore_Versioned(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
s := NewMemoryKVStore()
|
||||
|
||||
// Absent key reports ErrNotFound + version 0.
|
||||
if _, v, err := s.GetVersioned(ctx, "k"); !errors.Is(err, ErrNotFound) || v != 0 {
|
||||
t.Fatalf("GetVersioned absent: got (v=%d, %v), want (0, ErrNotFound)", v, err)
|
||||
}
|
||||
|
||||
// Create when absent (expectedVersion 0).
|
||||
if err := s.PutVersioned(ctx, "k", 0, []byte("v1")); err != nil {
|
||||
t.Fatalf("PutVersioned create: %v", err)
|
||||
}
|
||||
val, ver, err := s.GetVersioned(ctx, "k")
|
||||
if err != nil || string(val) != "v1" || ver != 1 {
|
||||
t.Fatalf("after create: got (%q, v=%d, %v), want (v1, 1, nil)", val, ver, err)
|
||||
}
|
||||
|
||||
// Create over existing conflicts.
|
||||
if err := s.PutVersioned(ctx, "k", 0, []byte("v2")); !errors.Is(err, ErrConflict) {
|
||||
t.Errorf("create over existing: got %v, want ErrConflict", err)
|
||||
}
|
||||
|
||||
// Swap with matching version succeeds and bumps version.
|
||||
if err := s.PutVersioned(ctx, "k", 1, []byte("v2")); err != nil {
|
||||
t.Fatalf("swap matching version: %v", err)
|
||||
}
|
||||
if _, ver, _ := s.GetVersioned(ctx, "k"); ver != 2 {
|
||||
t.Errorf("version after swap = %d, want 2", ver)
|
||||
}
|
||||
|
||||
// Stale version conflicts.
|
||||
if err := s.PutVersioned(ctx, "k", 1, []byte("v3")); !errors.Is(err, ErrConflict) {
|
||||
t.Errorf("stale version: got %v, want ErrConflict", err)
|
||||
}
|
||||
|
||||
// Swap on a missing key conflicts.
|
||||
if err := s.PutVersioned(ctx, "missing", 1, []byte("x")); !errors.Is(err, ErrConflict) {
|
||||
t.Errorf("swap missing key: got %v, want ErrConflict", err)
|
||||
}
|
||||
|
||||
// A plain Put bumps the version, so a versioned writer holding the old
|
||||
// version sees a conflict.
|
||||
if err := s.Put(ctx, "k", []byte("v4")); err != nil {
|
||||
t.Fatalf("Put: %v", err)
|
||||
}
|
||||
if err := s.PutVersioned(ctx, "k", 2, []byte("v5")); !errors.Is(err, ErrConflict) {
|
||||
t.Errorf("versioned write after plain Put: got %v, want ErrConflict", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// MemoryProvider is a Provider backed by per-module in-process maps. Intended
|
||||
// for tests and local no-database runs (MODULES= / no MONGO_URL); state is lost
|
||||
// when the process exits. Production uses MongoProvider.
|
||||
type MemoryProvider struct {
|
||||
mu sync.Mutex
|
||||
cols map[string]*memoryCollection
|
||||
}
|
||||
|
||||
// NewMemoryProvider returns a fresh in-process provider.
|
||||
func NewMemoryProvider() *MemoryProvider {
|
||||
return &MemoryProvider{cols: make(map[string]*memoryCollection)}
|
||||
}
|
||||
|
||||
// Collection returns the per-module in-memory collection, creating it on first
|
||||
// use. An invalid module name yields an invalidCollection so Typed produces a
|
||||
// store whose every op errors — mirroring MongoProvider's defense in depth.
|
||||
func (p *MemoryProvider) Collection(module string) Collection {
|
||||
if !collectionNameRe.MatchString(module) {
|
||||
return invalidCollection{name: module}
|
||||
}
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
c, ok := p.cols[module]
|
||||
if !ok {
|
||||
c = &memoryCollection{rows: make(map[string]memoryRow)}
|
||||
p.cols[module] = c
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// memoryCollection holds one module's rows. Multiple typed views (different
|
||||
// payload types under disjoint key prefixes) share the same map.
|
||||
type memoryCollection struct {
|
||||
mu sync.RWMutex
|
||||
rows map[string]memoryRow
|
||||
}
|
||||
|
||||
func (*memoryCollection) isCollection() {}
|
||||
|
||||
// memoryRow stores the JSON-encoded value plus its version. Encoding to bytes
|
||||
// isolates stored state from caller mutation, matching production where Mongo
|
||||
// serializes the value on write.
|
||||
type memoryRow struct {
|
||||
version int64
|
||||
data []byte
|
||||
}
|
||||
|
||||
// memoryDocStore is a typed DocStore over one memoryCollection.
|
||||
type memoryDocStore[T any] struct {
|
||||
c *memoryCollection
|
||||
}
|
||||
|
||||
func (s *memoryDocStore[T]) Get(_ context.Context, id string) (T, int64, error) {
|
||||
var out T
|
||||
if err := validateKey(id); err != nil {
|
||||
return out, 0, err
|
||||
}
|
||||
s.c.mu.RLock()
|
||||
defer s.c.mu.RUnlock()
|
||||
row, ok := s.c.rows[id]
|
||||
if !ok {
|
||||
return out, 0, ErrNotFound
|
||||
}
|
||||
if err := json.Unmarshal(row.data, &out); err != nil {
|
||||
return out, 0, err
|
||||
}
|
||||
return out, row.version, nil
|
||||
}
|
||||
|
||||
func (s *memoryDocStore[T]) Put(_ context.Context, id string, val T) error {
|
||||
if err := validateKey(id); err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := json.Marshal(val)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
s.c.mu.Lock()
|
||||
defer s.c.mu.Unlock()
|
||||
// A plain Put bumps the version too, so a concurrent versioned writer that
|
||||
// read the old version correctly sees a conflict.
|
||||
s.c.rows[id] = memoryRow{version: s.c.rows[id].version + 1, data: data}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *memoryDocStore[T]) PutVersioned(_ context.Context, id string, expectedVersion int64, val T) error {
|
||||
if err := validateKey(id); err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := json.Marshal(val)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
s.c.mu.Lock()
|
||||
defer s.c.mu.Unlock()
|
||||
row, ok := s.c.rows[id]
|
||||
if expectedVersion == 0 {
|
||||
if ok {
|
||||
return ErrConflict
|
||||
}
|
||||
} else if !ok || row.version != expectedVersion {
|
||||
return ErrConflict
|
||||
}
|
||||
s.c.rows[id] = memoryRow{version: row.version + 1, data: data}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *memoryDocStore[T]) Delete(_ context.Context, id string) error {
|
||||
if err := validateKey(id); err != nil {
|
||||
return err
|
||||
}
|
||||
s.c.mu.Lock()
|
||||
defer s.c.mu.Unlock()
|
||||
delete(s.c.rows, id)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *memoryDocStore[T]) List(_ context.Context, prefix string) ([]string, error) {
|
||||
if err := validatePrefix(prefix); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.c.mu.RLock()
|
||||
defer s.c.mu.RUnlock()
|
||||
keys := make([]string, 0)
|
||||
for k := range s.c.rows {
|
||||
if strings.HasPrefix(k, prefix) {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
}
|
||||
sort.Strings(keys)
|
||||
return keys, nil
|
||||
}
|
||||
@@ -0,0 +1,185 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo/options"
|
||||
)
|
||||
|
||||
// Reserved document-root field names owned by the store. The caller's payload
|
||||
// fields are hoisted to the root alongside these. `_id` holds the key.
|
||||
const (
|
||||
mongoIDField = "_id"
|
||||
mongoVersionField = "version"
|
||||
mongoUpdatedAtField = "updatedAt"
|
||||
)
|
||||
|
||||
// mongoPutAttempts bounds the get→put-versioned retry loop a plain Put uses to
|
||||
// overwrite unconditionally while still bumping the version.
|
||||
const mongoPutAttempts = 5
|
||||
|
||||
// mongoCollection is a Collection backed by a single MongoDB collection (one per
|
||||
// module — collection-per-module IS the isolation, so no key prefix is needed).
|
||||
type mongoCollection struct {
|
||||
coll *mongo.Collection
|
||||
module string
|
||||
}
|
||||
|
||||
func (mongoCollection) isCollection() {}
|
||||
|
||||
// storedDoc is the on-disk document shape. The payload's fields are hoisted to
|
||||
// the document root via bson inline, so the value renders as an expandable,
|
||||
// queryable native document in Compass — never wrapped in a `value` envelope.
|
||||
// Reserved metadata (version, updatedAt) sit alongside the payload fields.
|
||||
type storedDoc[T any] struct {
|
||||
ID string `bson:"_id"`
|
||||
Version int64 `bson:"version"`
|
||||
UpdatedAt time.Time `bson:"updatedAt"`
|
||||
Payload T `bson:",inline"`
|
||||
}
|
||||
|
||||
// mongoDocStore is a typed DocStore over one collection.
|
||||
type mongoDocStore[T any] struct {
|
||||
coll *mongo.Collection
|
||||
module string
|
||||
}
|
||||
|
||||
func (s *mongoDocStore[T]) Get(ctx context.Context, id string) (T, int64, error) {
|
||||
var out storedDoc[T]
|
||||
if err := validateKey(id); err != nil {
|
||||
return out.Payload, 0, err
|
||||
}
|
||||
err := s.coll.FindOne(ctx, bson.M{mongoIDField: id}).Decode(&out)
|
||||
if err != nil {
|
||||
if errors.Is(err, mongo.ErrNoDocuments) {
|
||||
return out.Payload, 0, ErrNotFound
|
||||
}
|
||||
return out.Payload, 0, fmt.Errorf("mongo get %s/%s: %w", s.module, id, err)
|
||||
}
|
||||
return out.Payload, out.Version, nil
|
||||
}
|
||||
|
||||
// replacement builds the whole document to store. Using a full replacement
|
||||
// (never $set) guarantees stale fields from a previous value cannot survive an
|
||||
// overwrite.
|
||||
func (s *mongoDocStore[T]) replacement(id string, version int64, val T) storedDoc[T] {
|
||||
return storedDoc[T]{ID: id, Version: version, UpdatedAt: time.Now().UTC(), Payload: val}
|
||||
}
|
||||
|
||||
// PutVersioned writes val only if the stored version equals expectedVersion,
|
||||
// then bumps it.
|
||||
//
|
||||
// - expectedVersion == 0 → upsert matching {_id} with version absent or 0; a
|
||||
// key that already has version ≥ 1 fails the filter, the upsert attempts an
|
||||
// insert on the same _id, and the unique-_id index returns duplicate-key →
|
||||
// ErrConflict (linearizable single-winner for the first write).
|
||||
// - expectedVersion > 0 → ReplaceOne on {_id, version:expected}; MatchedCount
|
||||
// of 0 means the version moved → ErrConflict.
|
||||
func (s *mongoDocStore[T]) PutVersioned(ctx context.Context, id string, expectedVersion int64, val T) error {
|
||||
if err := validateKey(id); err != nil {
|
||||
return err
|
||||
}
|
||||
if expectedVersion == 0 {
|
||||
_, err := s.coll.ReplaceOne(ctx,
|
||||
bson.M{
|
||||
mongoIDField: id,
|
||||
"$or": bson.A{
|
||||
bson.M{mongoVersionField: bson.M{"$exists": false}},
|
||||
bson.M{mongoVersionField: int64(0)},
|
||||
},
|
||||
},
|
||||
s.replacement(id, 1, val),
|
||||
options.Replace().SetUpsert(true),
|
||||
)
|
||||
if err != nil {
|
||||
if mongo.IsDuplicateKeyError(err) {
|
||||
return ErrConflict
|
||||
}
|
||||
return fmt.Errorf("mongo put-versioned %s/%s: %w", s.module, id, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
res, err := s.coll.ReplaceOne(ctx,
|
||||
bson.M{mongoIDField: id, mongoVersionField: expectedVersion},
|
||||
s.replacement(id, expectedVersion+1, val),
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("mongo put-versioned %s/%s: %w", s.module, id, err)
|
||||
}
|
||||
if res.MatchedCount == 0 {
|
||||
return ErrConflict
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Put overwrites unconditionally and bumps the version, implemented as a bounded
|
||||
// get→put-versioned loop so it shares the whole-document replacement (stale-field
|
||||
// removal) and version semantics with PutVersioned.
|
||||
func (s *mongoDocStore[T]) Put(ctx context.Context, id string, val T) error {
|
||||
if err := validateKey(id); err != nil {
|
||||
return err
|
||||
}
|
||||
for attempt := 0; attempt < mongoPutAttempts; attempt++ {
|
||||
_, version, err := s.Get(ctx, id)
|
||||
switch {
|
||||
case errors.Is(err, ErrNotFound):
|
||||
version = 0
|
||||
case err != nil:
|
||||
return err
|
||||
}
|
||||
err = s.PutVersioned(ctx, id, version, val)
|
||||
if errors.Is(err, ErrConflict) {
|
||||
continue // a concurrent writer moved the version; re-read and retry.
|
||||
}
|
||||
return err
|
||||
}
|
||||
return fmt.Errorf("mongo put %s/%s: %w after %d attempts", s.module, id, ErrConflict, mongoPutAttempts)
|
||||
}
|
||||
|
||||
func (s *mongoDocStore[T]) Delete(ctx context.Context, id string) error {
|
||||
if err := validateKey(id); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := s.coll.DeleteOne(ctx, bson.M{mongoIDField: id}); err != nil {
|
||||
return fmt.Errorf("mongo delete %s/%s: %w", s.module, id, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// List returns all document IDs whose key starts with prefix, via a half-open
|
||||
// range scan on _id (reusing prefixSuccessor) so it uses the _id index and
|
||||
// avoids regex injection. Empty prefix returns the whole collection.
|
||||
func (s *mongoDocStore[T]) List(ctx context.Context, prefix string) ([]string, error) {
|
||||
if err := validatePrefix(prefix); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
filter := bson.M{}
|
||||
if prefix != "" {
|
||||
filter[mongoIDField] = bson.M{"$gte": prefix, "$lt": prefixSuccessor(prefix)}
|
||||
}
|
||||
cur, err := s.coll.Find(ctx, filter, options.Find().SetProjection(bson.M{mongoIDField: 1}))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("mongo list %s prefix=%q: %w", s.module, prefix, err)
|
||||
}
|
||||
defer func() { _ = cur.Close(ctx) }()
|
||||
|
||||
var keys []string
|
||||
for cur.Next(ctx) {
|
||||
var doc struct {
|
||||
ID string `bson:"_id"`
|
||||
}
|
||||
if err := cur.Decode(&doc); err != nil {
|
||||
return nil, fmt.Errorf("mongo list %s prefix=%q: decode: %w", s.module, prefix, err)
|
||||
}
|
||||
keys = append(keys, doc.ID)
|
||||
}
|
||||
if err := cur.Err(); err != nil {
|
||||
return nil, fmt.Errorf("mongo list %s prefix=%q: %w", s.module, prefix, err)
|
||||
}
|
||||
return keys, nil
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
)
|
||||
|
||||
// mongoLocalSetup connects to a local MongoDB and returns a fresh, uniquely
|
||||
// named database plus cleanup. Tests skip if MONGODB_TEST_URL is unset so CI
|
||||
// without a Mongo container still builds (mirrors the DynamoDB Local gating).
|
||||
func mongoLocalSetup(t *testing.T) (*mongo.Database, func()) {
|
||||
t.Helper()
|
||||
uri := os.Getenv("MONGODB_TEST_URL")
|
||||
if uri == "" {
|
||||
t.Skip("MONGODB_TEST_URL not set; skipping MongoDB integration test (run `make mongo-local` to start the local container)")
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
client, err := NewMongoClient(ctx, uri)
|
||||
if err != nil {
|
||||
t.Fatalf("NewMongoClient: %v", err)
|
||||
}
|
||||
dbName := fmt.Sprintf("miti99bot_test_%d", time.Now().UnixNano())
|
||||
if len(dbName) > 63 {
|
||||
dbName = dbName[:63]
|
||||
}
|
||||
db := client.Database(dbName)
|
||||
|
||||
cleanup := func() {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
_ = db.Drop(ctx)
|
||||
_ = client.Disconnect(ctx)
|
||||
}
|
||||
return db, cleanup
|
||||
}
|
||||
|
||||
type portfolioLike struct {
|
||||
USD float64 `json:"usd" bson:"usd"`
|
||||
Assets map[string]float64 `json:"assets" bson:"assets"`
|
||||
}
|
||||
|
||||
func mongoStore[T any](t *testing.T, module string) (DocStore[T], *mongo.Collection, func()) {
|
||||
t.Helper()
|
||||
db, cleanup := mongoLocalSetup(t)
|
||||
coll := db.Collection(module)
|
||||
return Typed[T](NewMongoProvider(db).Collection(module)), coll, cleanup
|
||||
}
|
||||
|
||||
// rawDoc fetches the on-disk BSON document for assertions about its shape.
|
||||
func rawDoc(t *testing.T, coll *mongo.Collection, id string) bson.M {
|
||||
t.Helper()
|
||||
var doc bson.M
|
||||
if err := coll.FindOne(context.Background(), bson.M{"_id": id}).Decode(&doc); err != nil {
|
||||
t.Fatalf("raw FindOne %s: %v", id, err)
|
||||
}
|
||||
return doc
|
||||
}
|
||||
|
||||
func TestMongoDocStore_RootShape(t *testing.T) {
|
||||
store, coll, cleanup := mongoStore[portfolioLike](t, "coin")
|
||||
defer cleanup()
|
||||
ctx := context.Background()
|
||||
|
||||
if err := store.Put(ctx, "user:7", portfolioLike{USD: 1000.25, Assets: map[string]float64{"BTC": 1}}); err != nil {
|
||||
t.Fatalf("Put: %v", err)
|
||||
}
|
||||
doc := rawDoc(t, coll, "user:7")
|
||||
|
||||
if _, ok := doc["value"]; ok {
|
||||
t.Error("doc has legacy 'value' envelope field")
|
||||
}
|
||||
if _, ok := doc["_payload"]; ok {
|
||||
t.Error("doc has '_payload' fallback field")
|
||||
}
|
||||
if _, ok := doc["usd"]; !ok {
|
||||
t.Error("payload field 'usd' not hoisted to root")
|
||||
}
|
||||
if _, ok := doc["assets"]; !ok {
|
||||
t.Error("payload field 'assets' not hoisted to root")
|
||||
}
|
||||
if _, ok := doc["updatedAt"].(bson.DateTime); !ok {
|
||||
t.Errorf("updatedAt is %T, want BSON Date", doc["updatedAt"])
|
||||
}
|
||||
|
||||
got, version, err := store.Get(ctx, "user:7")
|
||||
if err != nil || version != 1 || got.USD != 1000.25 || got.Assets["BTC"] != 1 {
|
||||
t.Fatalf("Get round-trip: %+v v=%d err=%v", got, version, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMongoDocStore_OverwriteRemovesStaleFields(t *testing.T) {
|
||||
store, coll, cleanup := mongoStore[map[string]int](t, "misc")
|
||||
defer cleanup()
|
||||
ctx := context.Background()
|
||||
|
||||
if err := store.Put(ctx, "k", map[string]int{"a": 1, "b": 2}); err != nil {
|
||||
t.Fatalf("Put A: %v", err)
|
||||
}
|
||||
if err := store.Put(ctx, "k", map[string]int{"a": 3}); err != nil {
|
||||
t.Fatalf("Put B: %v", err)
|
||||
}
|
||||
doc := rawDoc(t, coll, "k")
|
||||
if _, ok := doc["b"]; ok {
|
||||
t.Error("stale field 'b' survived overwrite")
|
||||
}
|
||||
if doc["a"] != int32(3) && doc["a"] != int64(3) {
|
||||
t.Errorf("a = %v (%T), want 3", doc["a"], doc["a"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestMongoDocStore_PutVersionedConcurrentCreate(t *testing.T) {
|
||||
store, _, cleanup := mongoStore[portfolioLike](t, "coin")
|
||||
defer cleanup()
|
||||
ctx := context.Background()
|
||||
|
||||
const n = 12
|
||||
var wg sync.WaitGroup
|
||||
var mu sync.Mutex
|
||||
wins := 0
|
||||
wg.Add(n)
|
||||
for i := 0; i < n; i++ {
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := store.PutVersioned(ctx, "race", 0, portfolioLike{USD: 1}); err == nil {
|
||||
mu.Lock()
|
||||
wins++
|
||||
mu.Unlock()
|
||||
} else if !errors.Is(err, ErrConflict) {
|
||||
t.Errorf("unexpected err: %v", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if wins != 1 {
|
||||
t.Fatalf("concurrent create winners = %d, want exactly 1", wins)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMongoDocStore_PutVersionedStaleConflict(t *testing.T) {
|
||||
store, _, cleanup := mongoStore[portfolioLike](t, "coin")
|
||||
defer cleanup()
|
||||
ctx := context.Background()
|
||||
|
||||
_ = store.PutVersioned(ctx, "k", 0, portfolioLike{USD: 1})
|
||||
if err := store.PutVersioned(ctx, "k", 99, portfolioLike{USD: 2}); !errors.Is(err, ErrConflict) {
|
||||
t.Fatalf("stale CAS = %v, want ErrConflict", err)
|
||||
}
|
||||
if err := store.PutVersioned(ctx, "k", 1, portfolioLike{USD: 2}); err != nil {
|
||||
t.Fatalf("fresh CAS: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// wrappedScalar / wrappedArray prove non-object values become named root fields
|
||||
// (the lolschedule pattern) rather than an envelope or fallback.
|
||||
type wrappedScalar struct {
|
||||
Date string `json:"date" bson:"date"`
|
||||
}
|
||||
type wrappedArray struct {
|
||||
Subscribers []int `json:"subscribers" bson:"subscribers"`
|
||||
}
|
||||
|
||||
func TestMongoDocStore_WrappedScalarAndArray(t *testing.T) {
|
||||
db, cleanup := mongoLocalSetup(t)
|
||||
defer cleanup()
|
||||
ctx := context.Background()
|
||||
p := NewMongoProvider(db)
|
||||
|
||||
scalar := Typed[wrappedScalar](p.Collection("lolschedule"))
|
||||
if err := scalar.Put(ctx, "last_push_date", wrappedScalar{Date: "2026-06-28"}); err != nil {
|
||||
t.Fatalf("scalar Put: %v", err)
|
||||
}
|
||||
doc := rawDoc(t, db.Collection("lolschedule"), "last_push_date")
|
||||
if doc["date"] != "2026-06-28" {
|
||||
t.Errorf("scalar root field date = %v", doc["date"])
|
||||
}
|
||||
|
||||
arr := Typed[wrappedArray](p.Collection("lolschedule"))
|
||||
if err := arr.Put(ctx, "subscribers", wrappedArray{Subscribers: []int{1, 2, 3}}); err != nil {
|
||||
t.Fatalf("array Put: %v", err)
|
||||
}
|
||||
doc = rawDoc(t, db.Collection("lolschedule"), "subscribers")
|
||||
if _, ok := doc["subscribers"].(bson.A); !ok {
|
||||
t.Errorf("array root field subscribers = %T, want bson.A", doc["subscribers"])
|
||||
}
|
||||
}
|
||||
@@ -1,266 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo/options"
|
||||
)
|
||||
|
||||
// MongoDB document fields. `_id` holds the user key; `value` holds the raw
|
||||
// bytes; `updatedAt` is unix-nanos for observability + future TTL.
|
||||
const (
|
||||
mongoIDField = "_id"
|
||||
mongoValueField = "value"
|
||||
mongoUpdatedAtField = "updatedAt"
|
||||
mongoVersionField = "version"
|
||||
)
|
||||
|
||||
// MongoKVStore is a KVStore backed by a single MongoDB collection. The caller
|
||||
// (MongoProvider) creates one per module so cross-module isolation is
|
||||
// "different collection" — no key prefix needed at this layer, mirroring
|
||||
// FirestoreKVStore.
|
||||
type MongoKVStore struct {
|
||||
coll *mongo.Collection
|
||||
moduleName string
|
||||
}
|
||||
|
||||
// NewMongoKVStore returns a store writing to the given collection. Callers must
|
||||
// validate the collection/module name beforehand (MongoProvider does).
|
||||
func NewMongoKVStore(coll *mongo.Collection, moduleName string) *MongoKVStore {
|
||||
return &MongoKVStore{coll: coll, moduleName: moduleName}
|
||||
}
|
||||
|
||||
// decodeValue extracts the stored value bytes from a decoded document,
|
||||
// reconstructing the caller's JSON []byte from whichever representation is on
|
||||
// disk:
|
||||
// - native object/array (bson.M / bson.A / bson.D) → re-serialized to JSON;
|
||||
// - string → the bytes verbatim (bare scalars / non-JSON values, e.g. a date);
|
||||
// - bson.Binary / []byte → legacy fallback for docs written by an earlier
|
||||
// build that stored value as a binary blob.
|
||||
func (s *MongoKVStore) decodeValue(key string, doc bson.M) ([]byte, error) {
|
||||
raw, ok := doc[mongoValueField]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("mongo get %s/%s: missing %q field", s.moduleName, key, mongoValueField)
|
||||
}
|
||||
switch v := raw.(type) {
|
||||
case string:
|
||||
return []byte(v), nil
|
||||
case bson.M, bson.A, bson.D:
|
||||
return nativeToJSON(v)
|
||||
case bson.Binary:
|
||||
return v.Data, nil
|
||||
case []byte:
|
||||
return v, nil
|
||||
default:
|
||||
return nil, fmt.Errorf("mongo get %s/%s: unexpected value type %T", s.moduleName, key, raw)
|
||||
}
|
||||
}
|
||||
|
||||
// Get returns the raw bytes stored at key, or ErrNotFound.
|
||||
func (s *MongoKVStore) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
if err := validateKey(key); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var doc bson.M
|
||||
err := s.coll.FindOne(ctx, bson.M{mongoIDField: key}).Decode(&doc)
|
||||
if err != nil {
|
||||
if errors.Is(err, mongo.ErrNoDocuments) {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
return nil, fmt.Errorf("mongo get %s/%s: %w", s.moduleName, key, err)
|
||||
}
|
||||
return s.decodeValue(key, doc)
|
||||
}
|
||||
|
||||
// GetJSON decodes the value at key into dst.
|
||||
func (s *MongoKVStore) GetJSON(ctx context.Context, key string, dst any) error {
|
||||
raw, err := s.Get(ctx, key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.Unmarshal(raw, dst); err != nil {
|
||||
return fmt.Errorf("mongo get %s/%s: json decode: %w", s.moduleName, key, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Put writes raw bytes at key, creating or overwriting, and bumps the version
|
||||
// (so a concurrent versioned writer that read the old version sees a conflict).
|
||||
func (s *MongoKVStore) Put(ctx context.Context, key string, val []byte) error {
|
||||
if err := validateKey(key); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err := s.coll.UpdateOne(ctx,
|
||||
bson.M{mongoIDField: key},
|
||||
bson.M{
|
||||
"$set": bson.M{
|
||||
mongoValueField: encodeValue(val),
|
||||
mongoUpdatedAtField: time.Now().UTC().UnixNano(),
|
||||
},
|
||||
"$inc": bson.M{mongoVersionField: 1},
|
||||
},
|
||||
options.UpdateOne().SetUpsert(true),
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("mongo put %s/%s: %w", s.moduleName, key, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// PutJSON marshals val and writes the bytes at key.
|
||||
func (s *MongoKVStore) PutJSON(ctx context.Context, key string, val any) error {
|
||||
raw, err := json.Marshal(val)
|
||||
if err != nil {
|
||||
return fmt.Errorf("mongo put %s/%s: json encode: %w", s.moduleName, key, err)
|
||||
}
|
||||
return s.Put(ctx, key, raw)
|
||||
}
|
||||
|
||||
// GetVersioned returns the value and its version, or ErrNotFound. A document
|
||||
// written by an older build that has no version field reports version 0 (and no
|
||||
// error), so PutVersioned(expectedVersion=0) can adopt it.
|
||||
func (s *MongoKVStore) GetVersioned(ctx context.Context, key string) ([]byte, int64, error) {
|
||||
if err := validateKey(key); err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
var doc bson.M
|
||||
err := s.coll.FindOne(ctx, bson.M{mongoIDField: key}).Decode(&doc)
|
||||
if err != nil {
|
||||
if errors.Is(err, mongo.ErrNoDocuments) {
|
||||
return nil, 0, ErrNotFound
|
||||
}
|
||||
return nil, 0, fmt.Errorf("mongo get %s/%s: %w", s.moduleName, key, err)
|
||||
}
|
||||
val, err := s.decodeValue(key, doc)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
return val, decodeVersion(doc), nil
|
||||
}
|
||||
|
||||
// PutVersioned writes val only if the stored version equals expectedVersion,
|
||||
// then bumps it. expectedVersion == 0 means "create, or adopt a key that has no
|
||||
// version field yet" (a legacy doc). ErrConflict on a version mismatch.
|
||||
//
|
||||
// - expectedVersion == 0 → upsert matching {_id} with version absent or 0; a
|
||||
// key that already has version ≥ 1 fails the filter, the upsert attempts an
|
||||
// insert on the same _id, and the unique-_id index returns duplicate-key →
|
||||
// ErrConflict (linearizable single-winner for the first write).
|
||||
// - expectedVersion > 0 → UpdateOne on {_id, version:expected}; MatchedCount
|
||||
// of 0 means the version moved → ErrConflict.
|
||||
func (s *MongoKVStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error {
|
||||
if err := validateKey(key); err != nil {
|
||||
return err
|
||||
}
|
||||
now := time.Now().UTC().UnixNano()
|
||||
if expectedVersion == 0 {
|
||||
_, err := s.coll.UpdateOne(ctx,
|
||||
bson.M{
|
||||
mongoIDField: key,
|
||||
"$or": bson.A{
|
||||
bson.M{mongoVersionField: bson.M{"$exists": false}},
|
||||
bson.M{mongoVersionField: int64(0)},
|
||||
},
|
||||
},
|
||||
bson.M{"$set": bson.M{
|
||||
mongoValueField: encodeValue(val),
|
||||
mongoUpdatedAtField: now,
|
||||
mongoVersionField: int64(1),
|
||||
}},
|
||||
options.UpdateOne().SetUpsert(true),
|
||||
)
|
||||
if err != nil {
|
||||
if mongo.IsDuplicateKeyError(err) {
|
||||
return ErrConflict
|
||||
}
|
||||
return fmt.Errorf("mongo put-versioned %s/%s: %w", s.moduleName, key, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
res, err := s.coll.UpdateOne(ctx,
|
||||
bson.M{mongoIDField: key, mongoVersionField: expectedVersion},
|
||||
bson.M{
|
||||
"$set": bson.M{mongoValueField: encodeValue(val), mongoUpdatedAtField: now},
|
||||
"$inc": bson.M{mongoVersionField: 1},
|
||||
},
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("mongo put-versioned %s/%s: %w", s.moduleName, key, err)
|
||||
}
|
||||
if res.MatchedCount == 0 {
|
||||
return ErrConflict
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// decodeVersion reads the version field from a decoded doc, defaulting to 0 for
|
||||
// legacy docs that predate the version field. The driver may decode a BSON int
|
||||
// as int32 or int64.
|
||||
func decodeVersion(doc bson.M) int64 {
|
||||
switch v := doc[mongoVersionField].(type) {
|
||||
case int64:
|
||||
return v
|
||||
case int32:
|
||||
return int64(v)
|
||||
case float64:
|
||||
return int64(v)
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
// Delete removes the document at key. Deleting a missing key is not an error
|
||||
// (idempotent) — DeleteOne with a zero match count returns nil.
|
||||
func (s *MongoKVStore) Delete(ctx context.Context, key string) error {
|
||||
if err := validateKey(key); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err := s.coll.DeleteOne(ctx, bson.M{mongoIDField: key})
|
||||
if err != nil {
|
||||
return fmt.Errorf("mongo delete %s/%s: %w", s.moduleName, key, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// List returns all document IDs in the collection that start with prefix.
|
||||
// Implemented as a half-open range scan on _id (reusing prefixSuccessor) so it
|
||||
// uses the _id index and avoids regex injection. Empty prefix returns the whole
|
||||
// collection.
|
||||
func (s *MongoKVStore) List(ctx context.Context, prefix string) ([]string, error) {
|
||||
if err := validatePrefix(prefix); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
filter := bson.M{}
|
||||
if prefix != "" {
|
||||
filter[mongoIDField] = bson.M{
|
||||
"$gte": prefix,
|
||||
"$lt": prefixSuccessor(prefix),
|
||||
}
|
||||
}
|
||||
cur, err := s.coll.Find(ctx, filter, options.Find().SetProjection(bson.M{mongoIDField: 1}))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("mongo list %s prefix=%q: %w", s.moduleName, prefix, err)
|
||||
}
|
||||
defer func() { _ = cur.Close(ctx) }()
|
||||
|
||||
var keys []string
|
||||
for cur.Next(ctx) {
|
||||
var doc struct {
|
||||
ID string `bson:"_id"`
|
||||
}
|
||||
if err := cur.Decode(&doc); err != nil {
|
||||
return nil, fmt.Errorf("mongo list %s prefix=%q: decode: %w", s.moduleName, prefix, err)
|
||||
}
|
||||
keys = append(keys, doc.ID)
|
||||
}
|
||||
if err := cur.Err(); err != nil {
|
||||
return nil, fmt.Errorf("mongo list %s prefix=%q: %w", s.moduleName, prefix, err)
|
||||
}
|
||||
return keys, nil
|
||||
}
|
||||
@@ -1,370 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"reflect"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
)
|
||||
|
||||
// mongoLocalSetup connects to a local MongoDB and returns a store over a fresh,
|
||||
// uniquely-named collection. Tests skip if MONGODB_TEST_URL is unset so CI
|
||||
// without a Mongo container still builds (mirrors the DynamoDB Local gating).
|
||||
func mongoLocalSetup(t *testing.T, module string) (*MongoKVStore, *mongo.Database, func()) {
|
||||
t.Helper()
|
||||
uri := os.Getenv("MONGODB_TEST_URL")
|
||||
if uri == "" {
|
||||
t.Skip("MONGODB_TEST_URL not set; skipping MongoDB integration test (run `make mongo-local` to start the local container)")
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
client, err := NewMongoClient(ctx, uri)
|
||||
if err != nil {
|
||||
t.Fatalf("NewMongoClient: %v", err)
|
||||
}
|
||||
// Unique DB per test so parallel runs and cross-module isolation checks
|
||||
// never collide. Mongo db names are capped at 63 bytes.
|
||||
dbName := fmt.Sprintf("miti99bot_test_%d", time.Now().UnixNano())
|
||||
if len(dbName) > 63 {
|
||||
dbName = dbName[:63]
|
||||
}
|
||||
db := client.Database(dbName)
|
||||
|
||||
cleanup := func() {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
_ = db.Drop(ctx)
|
||||
_ = client.Disconnect(ctx)
|
||||
}
|
||||
return NewMongoKVStore(db.Collection(module), module), db, cleanup
|
||||
}
|
||||
|
||||
func TestMongoKVStore_PutGetDelete(t *testing.T) {
|
||||
s, _, cleanup := mongoLocalSetup(t, "wordle")
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
if err := s.Put(ctx, "user:1:state", []byte("hello")); err != nil {
|
||||
t.Fatalf("Put: %v", err)
|
||||
}
|
||||
got, err := s.Get(ctx, "user:1:state")
|
||||
if err != nil {
|
||||
t.Fatalf("Get: %v", err)
|
||||
}
|
||||
if string(got) != "hello" {
|
||||
t.Errorf("Get: got %q, want %q", got, "hello")
|
||||
}
|
||||
|
||||
// Overwrite preserves byte fidelity.
|
||||
if err := s.Put(ctx, "user:1:state", []byte("world")); err != nil {
|
||||
t.Fatalf("Put overwrite: %v", err)
|
||||
}
|
||||
got, _ = s.Get(ctx, "user:1:state")
|
||||
if string(got) != "world" {
|
||||
t.Errorf("Get after overwrite: got %q, want %q", got, "world")
|
||||
}
|
||||
|
||||
if err := s.Delete(ctx, "user:1:state"); err != nil {
|
||||
t.Fatalf("Delete: %v", err)
|
||||
}
|
||||
if _, err := s.Get(ctx, "user:1:state"); !errors.Is(err, ErrNotFound) {
|
||||
t.Errorf("Get after Delete: got %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMongoKVStore_GetMissing(t *testing.T) {
|
||||
s, _, cleanup := mongoLocalSetup(t, "wordle")
|
||||
defer cleanup()
|
||||
|
||||
if _, err := s.Get(context.Background(), "missing"); !errors.Is(err, ErrNotFound) {
|
||||
t.Errorf("got %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMongoKVStore_NativeValueRepresentation verifies a JSON object value is
|
||||
// persisted as a NATIVE BSON document (expandable/queryable in Atlas), and a
|
||||
// bare non-JSON value as a string — and that both round-trip.
|
||||
func TestMongoKVStore_NativeValueRepresentation(t *testing.T) {
|
||||
s, db, cleanup := mongoLocalSetup(t, "wordle")
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
obj := []byte(`{"word":"chào","score":42}`) // multi-byte UTF-8 object
|
||||
if err := s.Put(ctx, "obj", obj); err != nil {
|
||||
t.Fatalf("Put obj: %v", err)
|
||||
}
|
||||
// A bare string (lolschedule date-guard style) is not JSON → stored as string.
|
||||
if err := s.Put(ctx, "bare", []byte("2026-06-28")); err != nil {
|
||||
t.Fatalf("Put bare: %v", err)
|
||||
}
|
||||
|
||||
var objDoc, bareDoc bson.M
|
||||
if err := db.Collection("wordle").FindOne(ctx, bson.M{"_id": "obj"}).Decode(&objDoc); err != nil {
|
||||
t.Fatalf("raw FindOne obj: %v", err)
|
||||
}
|
||||
switch objDoc["value"].(type) {
|
||||
case bson.M, bson.D:
|
||||
// native object — good
|
||||
default:
|
||||
t.Errorf("object value stored as %T, want native bson document", objDoc["value"])
|
||||
}
|
||||
if err := db.Collection("wordle").FindOne(ctx, bson.M{"_id": "bare"}).Decode(&bareDoc); err != nil {
|
||||
t.Fatalf("raw FindOne bare: %v", err)
|
||||
}
|
||||
if _, ok := bareDoc["value"].(string); !ok {
|
||||
t.Errorf("bare value stored as %T, want string", bareDoc["value"])
|
||||
}
|
||||
|
||||
// Round-trips.
|
||||
if got, _ := s.Get(ctx, "bare"); string(got) != "2026-06-28" {
|
||||
t.Errorf("bare round trip: got %q", got)
|
||||
}
|
||||
var back map[string]any
|
||||
if err := s.GetJSON(ctx, "obj", &back); err != nil {
|
||||
t.Fatalf("GetJSON obj: %v", err)
|
||||
}
|
||||
if back["word"] != "chào" {
|
||||
t.Errorf("object round trip lost data: %v", back)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMongoKVStore_Int64Fidelity is the codec gate: an int64 field must survive
|
||||
// the JSON↔BSON round trip as an integer, not collapse to a float (which would
|
||||
// make GetJSON into an int64 field fail or drift).
|
||||
func TestMongoKVStore_Int64Fidelity(t *testing.T) {
|
||||
s, db, cleanup := mongoLocalSetup(t, "coin")
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
type meta struct {
|
||||
CreatedAt int64 `json:"createdAt"` // UnixMilli ~1.7e12
|
||||
Invested float64 `json:"invested"`
|
||||
}
|
||||
type portfolio struct {
|
||||
USD float64 `json:"usd"`
|
||||
Meta meta `json:"meta"`
|
||||
Qty map[string]int64 `json:"qty"`
|
||||
}
|
||||
in := portfolio{USD: 1234.56, Meta: meta{CreatedAt: 1719500000000, Invested: 5000}, Qty: map[string]int64{"BTC": 3}}
|
||||
if err := s.PutJSON(ctx, "u1", in); err != nil {
|
||||
t.Fatalf("PutJSON: %v", err)
|
||||
}
|
||||
var out portfolio
|
||||
if err := s.GetJSON(ctx, "u1", &out); err != nil {
|
||||
t.Fatalf("GetJSON: %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(out, in) {
|
||||
t.Errorf("fidelity lost: got %+v, want %+v", out, in)
|
||||
}
|
||||
// The stored CreatedAt must be a BSON int (int32/int64), not a double.
|
||||
var doc bson.M
|
||||
_ = db.Collection("coin").FindOne(ctx, bson.M{"_id": "u1"}).Decode(&doc)
|
||||
if m, ok := doc["value"].(bson.M); ok {
|
||||
if meta, ok := m["meta"].(bson.M); ok {
|
||||
switch meta["createdAt"].(type) {
|
||||
case int64, int32:
|
||||
// good — preserved as integer
|
||||
default:
|
||||
t.Errorf("createdAt stored as %T, want int64 (json.Number codec)", meta["createdAt"])
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMongoKVStore_JSONRoundTrip(t *testing.T) {
|
||||
s, _, cleanup := mongoLocalSetup(t, "loldle")
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
type state struct {
|
||||
Score int `json:"score"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
in := state{Score: 42, Name: "ezreal"}
|
||||
if err := s.PutJSON(ctx, "u1", in); err != nil {
|
||||
t.Fatalf("PutJSON: %v", err)
|
||||
}
|
||||
var out state
|
||||
if err := s.GetJSON(ctx, "u1", &out); err != nil {
|
||||
t.Fatalf("GetJSON: %v", err)
|
||||
}
|
||||
if out != in {
|
||||
t.Errorf("got %+v, want %+v", out, in)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMongoKVStore_ListPrefix(t *testing.T) {
|
||||
s, _, cleanup := mongoLocalSetup(t, "wordle")
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
for _, k := range []string{"user:1:state", "user:2:state", "config:daily", "user:1:history"} {
|
||||
if err := s.Put(ctx, k, []byte("x")); err != nil {
|
||||
t.Fatalf("Put %s: %v", k, err)
|
||||
}
|
||||
}
|
||||
|
||||
got, err := s.List(ctx, "user:")
|
||||
if err != nil {
|
||||
t.Fatalf("List: %v", err)
|
||||
}
|
||||
want := map[string]bool{"user:1:state": true, "user:2:state": true, "user:1:history": true}
|
||||
if len(got) != len(want) {
|
||||
t.Errorf("List: got %v (len=%d), want len=%d", got, len(got), len(want))
|
||||
}
|
||||
for _, k := range got {
|
||||
if !want[k] {
|
||||
t.Errorf("List: unexpected key %q", k)
|
||||
}
|
||||
}
|
||||
|
||||
// Empty prefix returns everything in the collection.
|
||||
all, err := s.List(ctx, "")
|
||||
if err != nil {
|
||||
t.Fatalf("List empty: %v", err)
|
||||
}
|
||||
if len(all) != 4 {
|
||||
t.Errorf("List(\"\"): got %d keys, want 4", len(all))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMongoKVStore_Versioned(t *testing.T) {
|
||||
s, _, cleanup := mongoLocalSetup(t, "gold")
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
// Create-if-absent (version 0) succeeds once, then conflicts.
|
||||
if err := s.PutVersioned(ctx, "user:1", 0, []byte("v1")); err != nil {
|
||||
t.Fatalf("PutVersioned create: %v", err)
|
||||
}
|
||||
if err := s.PutVersioned(ctx, "user:1", 0, []byte("v2")); !errors.Is(err, ErrConflict) {
|
||||
t.Errorf("create over existing: got %v, want ErrConflict", err)
|
||||
}
|
||||
|
||||
val, ver, err := s.GetVersioned(ctx, "user:1")
|
||||
if err != nil || string(val) != "v1" || ver != 1 {
|
||||
t.Fatalf("GetVersioned: got (%q, v=%d, %v), want (v1, 1, nil)", val, ver, err)
|
||||
}
|
||||
|
||||
// Swap with matching version succeeds and bumps; stale version conflicts.
|
||||
if err := s.PutVersioned(ctx, "user:1", 1, []byte("v2")); err != nil {
|
||||
t.Fatalf("swap matching version: %v", err)
|
||||
}
|
||||
if err := s.PutVersioned(ctx, "user:1", 1, []byte("v3")); !errors.Is(err, ErrConflict) {
|
||||
t.Errorf("stale version: got %v, want ErrConflict", err)
|
||||
}
|
||||
if got, _ := s.Get(ctx, "user:1"); string(got) != "v2" {
|
||||
t.Errorf("stored value = %q, want %q", got, "v2")
|
||||
}
|
||||
|
||||
// Swap on a missing key conflicts.
|
||||
if err := s.PutVersioned(ctx, "user:missing", 1, []byte("v2")); !errors.Is(err, ErrConflict) {
|
||||
t.Errorf("swap missing key: got %v, want ErrConflict", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMongoKVStore_PutVersioned_AdoptsLegacyDoc proves a document written by an
|
||||
// older build (no version field) is treated as version 0 and updated cleanly,
|
||||
// so existing users' portfolios keep working after the rollout.
|
||||
func TestMongoKVStore_PutVersioned_AdoptsLegacyDoc(t *testing.T) {
|
||||
s, db, cleanup := mongoLocalSetup(t, "coin")
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
// Seed a legacy doc: value string, NO version field.
|
||||
if _, err := db.Collection("coin").InsertOne(ctx, bson.M{"_id": "user:9", "value": "legacy"}); err != nil {
|
||||
t.Fatalf("seed legacy doc: %v", err)
|
||||
}
|
||||
val, ver, err := s.GetVersioned(ctx, "user:9")
|
||||
if err != nil || string(val) != "legacy" || ver != 0 {
|
||||
t.Fatalf("GetVersioned legacy: got (%q, v=%d, %v), want (legacy, 0, nil)", val, ver, err)
|
||||
}
|
||||
// Claim it at version 0 — must succeed (no spurious conflict).
|
||||
if err := s.PutVersioned(ctx, "user:9", 0, []byte("migrated")); err != nil {
|
||||
t.Fatalf("adopt legacy doc: %v", err)
|
||||
}
|
||||
if got, ver, _ := s.GetVersioned(ctx, "user:9"); string(got) != "migrated" || ver != 1 {
|
||||
t.Errorf("after adopt: got (%q, v=%d), want (migrated, 1)", got, ver)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMongoKVStore_PutVersioned_ConcurrentCreate proves the absent-create race
|
||||
// is linearizable: N goroutines racing PutVersioned(_, 0, _) on one key yield
|
||||
// exactly one winner; losers get ErrConflict. Blocking correctness gate.
|
||||
func TestMongoKVStore_PutVersioned_ConcurrentCreate(t *testing.T) {
|
||||
s, _, cleanup := mongoLocalSetup(t, "coin")
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
const n = 16
|
||||
var wg sync.WaitGroup
|
||||
var mu sync.Mutex
|
||||
var wins, conflicts, others int
|
||||
start := make(chan struct{})
|
||||
for i := 0; i < n; i++ {
|
||||
wg.Add(1)
|
||||
go func(i int) {
|
||||
defer wg.Done()
|
||||
<-start
|
||||
err := s.PutVersioned(ctx, "race", 0, []byte(fmt.Sprintf("v%d", i)))
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
switch {
|
||||
case err == nil:
|
||||
wins++
|
||||
case errors.Is(err, ErrConflict):
|
||||
conflicts++
|
||||
default:
|
||||
others++
|
||||
t.Errorf("unexpected error: %v", err)
|
||||
}
|
||||
}(i)
|
||||
}
|
||||
close(start)
|
||||
wg.Wait()
|
||||
|
||||
if wins != 1 {
|
||||
t.Errorf("concurrent create: got %d winners, want exactly 1", wins)
|
||||
}
|
||||
if conflicts != n-1 {
|
||||
t.Errorf("concurrent create: got %d conflicts, want %d", conflicts, n-1)
|
||||
}
|
||||
if others != 0 {
|
||||
t.Errorf("concurrent create produced %d non-conflict errors", others)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMongoKVStore_GetDocWithoutValueField covers a malformed document edge
|
||||
// case (a doc that exists but lacks the value field) — Get must surface a clear
|
||||
// error, not panic or return empty bytes silently.
|
||||
func TestMongoKVStore_GetDocWithoutValueField(t *testing.T) {
|
||||
s, db, cleanup := mongoLocalSetup(t, "wordle")
|
||||
defer cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
// Insert a doc directly with no `value` field.
|
||||
if _, err := db.Collection("wordle").InsertOne(ctx, bson.M{"_id": "novalue", "other": 1}); err != nil {
|
||||
t.Fatalf("seed InsertOne: %v", err)
|
||||
}
|
||||
if _, err := s.Get(ctx, "novalue"); err == nil {
|
||||
t.Errorf("Get on doc without value field: got nil error, want a descriptive error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMongoKVStore_DeleteMissingNoError(t *testing.T) {
|
||||
s, _, cleanup := mongoLocalSetup(t, "wordle")
|
||||
defer cleanup()
|
||||
|
||||
if err := s.Delete(context.Background(), "never-existed"); err != nil {
|
||||
t.Errorf("Delete missing key: got %v, want nil (idempotent)", err)
|
||||
}
|
||||
}
|
||||
@@ -4,28 +4,26 @@ import (
|
||||
"go.mongodb.org/mongo-driver/v2/mongo"
|
||||
)
|
||||
|
||||
// MongoProvider is a KVProvider that creates one collection per module,
|
||||
// mirroring FirestoreProvider. Collection-per-module IS the isolation — no key
|
||||
// prefix wrapping is needed at this layer.
|
||||
// MongoProvider is a Provider that creates one collection per module.
|
||||
// Collection-per-module IS the isolation — no key prefix wrapping is needed.
|
||||
type MongoProvider struct {
|
||||
db *mongo.Database
|
||||
}
|
||||
|
||||
// NewMongoProvider returns a provider over the given database handle. The
|
||||
// underlying client must outlive every KVStore the provider hands out; callers
|
||||
// underlying client must outlive every store the provider hands out; callers
|
||||
// own its Disconnect.
|
||||
func NewMongoProvider(db *mongo.Database) *MongoProvider {
|
||||
return &MongoProvider{db: db}
|
||||
}
|
||||
|
||||
// For returns a MongoKVStore writing to a collection named after the module.
|
||||
// moduleName is re-validated against collectionNameRe — defense in depth
|
||||
// against caller bugs that bypass modules.Build, identical to the firestore
|
||||
// and dynamodb providers. An invalid name yields an invalidStore whose every
|
||||
// op errors at first use.
|
||||
func (p *MongoProvider) For(moduleName string) KVStore {
|
||||
// Collection returns a handle to the collection named after the module.
|
||||
// moduleName is re-validated against collectionNameRe — defense in depth against
|
||||
// caller bugs that bypass modules.Build. An invalid name yields an
|
||||
// invalidCollection whose Typed store errors at first use.
|
||||
func (p *MongoProvider) Collection(moduleName string) Collection {
|
||||
if !collectionNameRe.MatchString(moduleName) {
|
||||
return invalidStore{name: moduleName}
|
||||
return invalidCollection{name: moduleName}
|
||||
}
|
||||
return NewMongoKVStore(p.db.Collection(moduleName), moduleName)
|
||||
return mongoCollection{coll: p.db.Collection(moduleName), module: moduleName}
|
||||
}
|
||||
@@ -6,10 +6,10 @@ import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// MongoProvider.For re-validates the module name as defense-in-depth. Invalid
|
||||
// names return invalidStore without touching the DB — the branch worth locking
|
||||
// even without a live Mongo (mirrors the firestore provider test).
|
||||
func TestMongoProvider_For_RejectsInvalidName(t *testing.T) {
|
||||
// MongoProvider.Collection re-validates the module name as defense-in-depth.
|
||||
// Invalid names yield an invalidCollection whose Typed store errors without
|
||||
// touching the DB — the branch worth locking even without a live Mongo.
|
||||
func TestMongoProvider_Collection_RejectsInvalidName(t *testing.T) {
|
||||
p := &MongoProvider{db: nil}
|
||||
bogus := []string{
|
||||
"",
|
||||
@@ -21,45 +21,45 @@ func TestMongoProvider_For_RejectsInvalidName(t *testing.T) {
|
||||
"with:colon",
|
||||
}
|
||||
for _, name := range bogus {
|
||||
store := p.For(name)
|
||||
if _, err := store.Get(context.Background(), "any-key"); !errors.Is(err, ErrInvalidModuleName) {
|
||||
t.Errorf("For(%q).Get → %v, want ErrInvalidModuleName", name, err)
|
||||
// Invalid names short-circuit to invalidCollection before touching the
|
||||
// (nil) db, so Typed yields a store whose ops report ErrInvalidModuleName.
|
||||
store := Typed[string](p.Collection(name))
|
||||
if _, _, err := store.Get(context.Background(), "any-key"); !errors.Is(err, ErrInvalidModuleName) {
|
||||
t.Errorf("Collection(%q) store Get → %v, want ErrInvalidModuleName", name, err)
|
||||
}
|
||||
}
|
||||
// Valid names pass validation against a real db in
|
||||
// TestMongoProvider_CrossModuleIsolation (gated on MONGODB_TEST_URL); a nil
|
||||
// db here would panic in the driver for a valid name.
|
||||
}
|
||||
|
||||
// TestMongoProvider_CrossModuleIsolation verifies collection-per-module
|
||||
// isolation: the same key written through two module stores yields independent
|
||||
// values. Gated on MONGODB_TEST_URL.
|
||||
func TestMongoProvider_CrossModuleIsolation(t *testing.T) {
|
||||
_, db, cleanup := mongoLocalSetup(t, "wordle")
|
||||
db, cleanup := mongoLocalSetup(t)
|
||||
defer cleanup()
|
||||
|
||||
p := NewMongoProvider(db)
|
||||
ctx := context.Background()
|
||||
|
||||
wordle := p.For("wordle")
|
||||
loldle := p.For("loldle")
|
||||
if err := wordle.Put(ctx, "shared", []byte("from-wordle")); err != nil {
|
||||
// Mongo payloads must be a struct or map (bson inline cannot hoist a
|
||||
// scalar); a wrapper struct is the same shape modules use.
|
||||
wordle := Typed[wrappedScalar](p.Collection("wordle"))
|
||||
loldle := Typed[wrappedScalar](p.Collection("loldle"))
|
||||
if err := wordle.Put(ctx, "shared", wrappedScalar{Date: "from-wordle"}); err != nil {
|
||||
t.Fatalf("wordle Put: %v", err)
|
||||
}
|
||||
if err := loldle.Put(ctx, "shared", []byte("from-loldle")); err != nil {
|
||||
if err := loldle.Put(ctx, "shared", wrappedScalar{Date: "from-loldle"}); err != nil {
|
||||
t.Fatalf("loldle Put: %v", err)
|
||||
}
|
||||
|
||||
gotW, _ := wordle.Get(ctx, "shared")
|
||||
gotL, _ := loldle.Get(ctx, "shared")
|
||||
if string(gotW) != "from-wordle" {
|
||||
t.Errorf("wordle key leaked: got %q", gotW)
|
||||
gotW, _, _ := wordle.Get(ctx, "shared")
|
||||
gotL, _, _ := loldle.Get(ctx, "shared")
|
||||
if gotW.Date != "from-wordle" {
|
||||
t.Errorf("wordle key leaked: got %q", gotW.Date)
|
||||
}
|
||||
if string(gotL) != "from-loldle" {
|
||||
t.Errorf("loldle key leaked: got %q", gotL)
|
||||
}
|
||||
// Canonical names pass validation (not invalidStore); this also covers the
|
||||
// valid-name branch of For against a real database.
|
||||
for _, name := range []string{"misc", "demo-mod", "x", "a1_b-2"} {
|
||||
if _, ok := p.For(name).(invalidStore); ok {
|
||||
t.Errorf("For(%q) returned invalidStore; expected validation to pass", name)
|
||||
}
|
||||
if gotL.Date != "from-loldle" {
|
||||
t.Errorf("loldle key leaked: got %q", gotL.Date)
|
||||
}
|
||||
}
|
||||
@@ -1,116 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
)
|
||||
|
||||
// This file converts between the KVStore's opaque JSON []byte contract and a
|
||||
// native BSON representation, so values render as expandable documents in the
|
||||
// Atlas/Compass UI (and are queryable) rather than as opaque blobs.
|
||||
//
|
||||
// Fidelity: JSON is decoded with json.Number so integral numbers persist as
|
||||
// BSON int64 (NOT float64) — a struct's int64 field round-trips without
|
||||
// precision loss. Only JSON objects and arrays become native; bare scalars and
|
||||
// non-JSON values (e.g. a plain date string) fall back to a BSON string.
|
||||
|
||||
// encodeValue produces the stored BSON representation of a value. A JSON object
|
||||
// or array → native (expandable/queryable); anything else → a string (the
|
||||
// human-readable fallback, also covers non-UTF-8-free callers identically to
|
||||
// the previous string encoding).
|
||||
func encodeValue(val []byte) any {
|
||||
if native, ok := jsonToNative(val); ok {
|
||||
return native
|
||||
}
|
||||
return string(val)
|
||||
}
|
||||
|
||||
// jsonToNative decodes val into a native BSON value (bson.M / bson.A) when it is
|
||||
// a JSON object or array, preserving integers as int64. Returns ok=false for
|
||||
// scalars, non-JSON, or decode errors (caller stores those as a string).
|
||||
func jsonToNative(val []byte) (any, bool) {
|
||||
trimmed := bytes.TrimSpace(val)
|
||||
if len(trimmed) == 0 || (trimmed[0] != '{' && trimmed[0] != '[') {
|
||||
return nil, false
|
||||
}
|
||||
dec := json.NewDecoder(bytes.NewReader(val))
|
||||
dec.UseNumber()
|
||||
var v any
|
||||
if err := dec.Decode(&v); err != nil {
|
||||
return nil, false
|
||||
}
|
||||
// Reject trailing garbage after the JSON value.
|
||||
if dec.More() {
|
||||
return nil, false
|
||||
}
|
||||
return numbersToBSON(v), true
|
||||
}
|
||||
|
||||
// numbersToBSON walks a json.Unmarshal(UseNumber) tree, converting maps/slices
|
||||
// to bson.M/bson.A and json.Number to int64 (when integral) else float64.
|
||||
func numbersToBSON(v any) any {
|
||||
switch t := v.(type) {
|
||||
case map[string]any:
|
||||
m := make(bson.M, len(t))
|
||||
for k, val := range t {
|
||||
m[k] = numbersToBSON(val)
|
||||
}
|
||||
return m
|
||||
case []any:
|
||||
a := make(bson.A, len(t))
|
||||
for i, val := range t {
|
||||
a[i] = numbersToBSON(val)
|
||||
}
|
||||
return a
|
||||
case json.Number:
|
||||
if i, err := t.Int64(); err == nil {
|
||||
return i
|
||||
}
|
||||
if f, err := t.Float64(); err == nil {
|
||||
return f
|
||||
}
|
||||
return t.String()
|
||||
default:
|
||||
return v // string, bool, nil
|
||||
}
|
||||
}
|
||||
|
||||
// nativeToJSON re-serializes a natively-stored value (bson.M / bson.A / bson.D
|
||||
// / scalar) back to JSON bytes. bson.M is map[string]any and bson.A is []any,
|
||||
// both of which json.Marshal handles directly; bson.D (ordered) is normalized
|
||||
// to a map first so it marshals as an object, not an array of key/value pairs.
|
||||
func nativeToJSON(raw any) ([]byte, error) {
|
||||
out, err := json.Marshal(normalizeBSON(raw))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("mongo: re-encode native value: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// normalizeBSON converts bson.D to bson.M recursively so json.Marshal emits an
|
||||
// object. bson.M/bson.A children are recursed; scalars pass through.
|
||||
func normalizeBSON(v any) any {
|
||||
switch t := v.(type) {
|
||||
case bson.D:
|
||||
m := make(bson.M, len(t))
|
||||
for _, e := range t {
|
||||
m[e.Key] = normalizeBSON(e.Value)
|
||||
}
|
||||
return m
|
||||
case bson.M:
|
||||
for k, val := range t {
|
||||
t[k] = normalizeBSON(val)
|
||||
}
|
||||
return t
|
||||
case bson.A:
|
||||
for i, val := range t {
|
||||
t[i] = normalizeBSON(val)
|
||||
}
|
||||
return t
|
||||
default:
|
||||
return v
|
||||
}
|
||||
}
|
||||
@@ -1,75 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Prefixed returns a KVStore that transparently prepends prefix+":" to every
|
||||
// key. List() strips the prefix from returned keys so callers see their own
|
||||
// flat namespace. The prefix must be non-empty.
|
||||
func Prefixed(inner KVStore, prefix string) KVStore {
|
||||
if prefix == "" {
|
||||
panic("storage: Prefixed requires non-empty prefix")
|
||||
}
|
||||
return &prefixedStore{inner: inner, prefix: prefix + ":"}
|
||||
}
|
||||
|
||||
type prefixedStore struct {
|
||||
inner KVStore
|
||||
prefix string
|
||||
}
|
||||
|
||||
func (p *prefixedStore) k(key string) string { return p.prefix + key }
|
||||
|
||||
func (p *prefixedStore) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
return p.inner.Get(ctx, p.k(key))
|
||||
}
|
||||
|
||||
func (p *prefixedStore) GetJSON(ctx context.Context, key string, dst any) error {
|
||||
return p.inner.GetJSON(ctx, p.k(key), dst)
|
||||
}
|
||||
|
||||
func (p *prefixedStore) Put(ctx context.Context, key string, val []byte) error {
|
||||
return p.inner.Put(ctx, p.k(key), val)
|
||||
}
|
||||
|
||||
func (p *prefixedStore) GetVersioned(ctx context.Context, key string) ([]byte, int64, error) {
|
||||
vs, ok := p.inner.(VersionedStore)
|
||||
if !ok {
|
||||
return nil, 0, fmt.Errorf("storage: inner store does not support versioned reads: %w", errors.ErrUnsupported)
|
||||
}
|
||||
return vs.GetVersioned(ctx, p.k(key))
|
||||
}
|
||||
|
||||
func (p *prefixedStore) PutVersioned(ctx context.Context, key string, expectedVersion int64, val []byte) error {
|
||||
vs, ok := p.inner.(VersionedStore)
|
||||
if !ok {
|
||||
// A missing capability is permanent: report it as unsupported so
|
||||
// callers fail fast instead of retrying it as a write conflict.
|
||||
return fmt.Errorf("storage: inner store does not support versioned writes: %w", errors.ErrUnsupported)
|
||||
}
|
||||
return vs.PutVersioned(ctx, p.k(key), expectedVersion, val)
|
||||
}
|
||||
|
||||
func (p *prefixedStore) PutJSON(ctx context.Context, key string, val any) error {
|
||||
return p.inner.PutJSON(ctx, p.k(key), val)
|
||||
}
|
||||
|
||||
func (p *prefixedStore) Delete(ctx context.Context, key string) error {
|
||||
return p.inner.Delete(ctx, p.k(key))
|
||||
}
|
||||
|
||||
func (p *prefixedStore) List(ctx context.Context, prefix string) ([]string, error) {
|
||||
keys, err := p.inner.List(ctx, p.k(prefix))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := make([]string, len(keys))
|
||||
for i, k := range keys {
|
||||
out[i] = strings.TrimPrefix(k, p.prefix)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -1,134 +0,0 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestPrefixed_RoundTrip(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
base := NewMemoryKVStore()
|
||||
a := Prefixed(base, "modA")
|
||||
b := Prefixed(base, "modB")
|
||||
|
||||
if err := a.Put(ctx, "score", []byte("10")); err != nil {
|
||||
t.Fatalf("a.Put: %v", err)
|
||||
}
|
||||
if err := b.Put(ctx, "score", []byte("20")); err != nil {
|
||||
t.Fatalf("b.Put: %v", err)
|
||||
}
|
||||
|
||||
got, err := a.Get(ctx, "score")
|
||||
if err != nil {
|
||||
t.Fatalf("a.Get: %v", err)
|
||||
}
|
||||
if string(got) != "10" {
|
||||
t.Errorf("a.Get score = %q, want %q", got, "10")
|
||||
}
|
||||
|
||||
got, err = b.Get(ctx, "score")
|
||||
if err != nil {
|
||||
t.Fatalf("b.Get: %v", err)
|
||||
}
|
||||
if string(got) != "20" {
|
||||
t.Errorf("b.Get score = %q, want %q", got, "20")
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrefixed_ListStripsPrefix(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
base := NewMemoryKVStore()
|
||||
mod := Prefixed(base, "wordle")
|
||||
|
||||
for _, k := range []string{"u:1", "u:2", "session:abc"} {
|
||||
if err := mod.Put(ctx, k, []byte("x")); err != nil {
|
||||
t.Fatalf("Put %q: %v", k, err)
|
||||
}
|
||||
}
|
||||
|
||||
got, err := mod.List(ctx, "u:")
|
||||
if err != nil {
|
||||
t.Fatalf("List: %v", err)
|
||||
}
|
||||
want := []string{"u:1", "u:2"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("List u: = %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrefixed_NotFoundPropagates(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
base := NewMemoryKVStore()
|
||||
mod := Prefixed(base, "modA")
|
||||
|
||||
if _, err := mod.Get(ctx, "missing"); err != ErrNotFound {
|
||||
t.Errorf("Get missing = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrefixed_VersionedDelegatesWithPrefixedKey(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
base := NewMemoryKVStore()
|
||||
mod := Prefixed(base, "gold")
|
||||
|
||||
vs, ok := mod.(VersionedStore)
|
||||
if !ok {
|
||||
t.Fatal("Prefixed store does not implement VersionedStore")
|
||||
}
|
||||
if err := vs.PutVersioned(ctx, "user:1", 0, []byte("v1")); err != nil {
|
||||
t.Fatalf("PutVersioned create: %v", err)
|
||||
}
|
||||
|
||||
// The write must land under the prefixed key on the inner store.
|
||||
got, err := base.Get(ctx, "gold:user:1")
|
||||
if err != nil {
|
||||
t.Fatalf("inner Get: %v", err)
|
||||
}
|
||||
if string(got) != "v1" {
|
||||
t.Errorf("inner value = %q, want %q", got, "v1")
|
||||
}
|
||||
|
||||
// GetVersioned reads through the prefix and returns the inner version.
|
||||
val, ver, err := vs.GetVersioned(ctx, "user:1")
|
||||
if err != nil || string(val) != "v1" || ver != 1 {
|
||||
t.Fatalf("GetVersioned: got (%q, v=%d, %v), want (v1, 1, nil)", val, ver, err)
|
||||
}
|
||||
|
||||
// Stale version must surface the inner store's conflict.
|
||||
if err := vs.PutVersioned(ctx, "user:1", 99, []byte("v2")); !errors.Is(err, ErrConflict) {
|
||||
t.Errorf("stale version: got %v, want ErrConflict", err)
|
||||
}
|
||||
if err := vs.PutVersioned(ctx, "user:1", 1, []byte("v2")); err != nil {
|
||||
t.Errorf("matching version: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// plainStore hides the inner store's versioned methods so the wrapper's
|
||||
// missing-capability path can be exercised.
|
||||
type plainStore struct {
|
||||
KVStore
|
||||
}
|
||||
|
||||
func TestPrefixed_VersionedUnsupportedInner(t *testing.T) {
|
||||
mod := Prefixed(&plainStore{KVStore: NewMemoryKVStore()}, "gold")
|
||||
vs := mod.(VersionedStore)
|
||||
|
||||
err := vs.PutVersioned(context.Background(), "user:1", 0, []byte("v1"))
|
||||
if !errors.Is(err, errors.ErrUnsupported) {
|
||||
t.Errorf("got %v, want errors.ErrUnsupported", err)
|
||||
}
|
||||
if errors.Is(err, ErrConflict) {
|
||||
t.Error("missing versioned capability must not be reported as a retryable conflict")
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrefixed_PanicsOnEmptyPrefix(t *testing.T) {
|
||||
defer func() {
|
||||
if recover() == nil {
|
||||
t.Error("Prefixed(_, \"\") did not panic")
|
||||
}
|
||||
}()
|
||||
Prefixed(NewMemoryKVStore(), "")
|
||||
}
|
||||
@@ -6,7 +6,7 @@ priority: P2
|
||||
branch: "feature/selfhosted"
|
||||
tags: [selfhost, coolify, mongodb, migration]
|
||||
blockedBy: []
|
||||
blocks: [260628-1113-mongo-native-value-documents]
|
||||
blocks: [260628-1113-mongo-native-value-documents, 260628-1310-flatten-mongo-value-documents]
|
||||
created: "2026-06-27T12:01:53.894Z"
|
||||
createdBy: "ck:plan"
|
||||
source: skill
|
||||
|
||||
+136
@@ -0,0 +1,136 @@
|
||||
---
|
||||
phase: 1
|
||||
title: "Schema Contract and Regression Tests"
|
||||
status: pending
|
||||
priority: P2
|
||||
dependencies: []
|
||||
effort: "M"
|
||||
---
|
||||
|
||||
# Phase 1: Schema Contract and Regression Tests
|
||||
|
||||
## Overview
|
||||
|
||||
Define the exact Mongo document contract and lock it with failing tests before
|
||||
changing the codec. This phase prevents a vague "flatten value" implementation
|
||||
from preserving stale fields, losing type fidelity, or silently breaking legacy
|
||||
reads.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Functional: specify new raw document shapes for JSON object, array, scalar,
|
||||
non-JSON string, reserved-key collision, and legacy `value` docs.
|
||||
- Functional: preserve current `KVStore` and `VersionedStore` method contracts.
|
||||
- Functional: prove `updatedAt` is BSON Date / Go `time.Time`.
|
||||
- Non-functional: tests must be explicit enough to fail against the current
|
||||
`{ _id, value, version, updatedAt int64 }` shape.
|
||||
|
||||
## Architecture
|
||||
|
||||
Target document examples:
|
||||
|
||||
```javascript
|
||||
// Normal JSON object payload, flattened.
|
||||
{
|
||||
_id: "user:7",
|
||||
usd: 1000.25,
|
||||
assets: { BTC: 1 },
|
||||
meta: { createdAt: 1782604800000 },
|
||||
version: 4,
|
||||
updatedAt: ISODate("2026-06-28T06:10:29Z"),
|
||||
schemaVersion: 2
|
||||
}
|
||||
|
||||
// Non-object fallback. Mongo root must be a document, so array/scalar payloads
|
||||
// need a reserved payload field. This is not the old generic `value` field.
|
||||
{
|
||||
_id: "daily_push:last_date",
|
||||
_payload: "2026-06-28",
|
||||
payloadKind: "string",
|
||||
version: 1,
|
||||
updatedAt: ISODate("2026-06-28T06:10:29Z"),
|
||||
schemaVersion: 2
|
||||
}
|
||||
```
|
||||
|
||||
Reserved fields: `_id`, `value`, `version`, `updatedAt`, `schemaVersion`,
|
||||
`payloadKind`, `_payload`. If an object payload contains one of these keys, the
|
||||
codec must preserve data by storing the whole object in `_payload` with
|
||||
`payloadKind: "object"` instead of flattening. Object payloads with keys that
|
||||
contain null, contain `.`, or start with `$` also use `_payload`; MongoDB
|
||||
permits some of these in modern versions but documents restrictions, so fallback
|
||||
keeps query/index behavior predictable. <!-- Updated: Validation Session 1 - reserve legacy value + unsafe Mongo field names -->
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Modify: `internal/storage/mongodb_kv_test.go` — raw-shape, legacy-read,
|
||||
stale-field, updatedAt, and CAS tests.
|
||||
- Modify: `internal/storage/mongodb_value_codec.go` — only if test helpers need
|
||||
exported/unexported constants clarified later.
|
||||
- Read: `internal/storage/mongodb_kv.go`
|
||||
- Read: `internal/storage/kv_store.go`
|
||||
- Read: `plans/reports/research-260628-0605-mongodb-document-design-standards.md`
|
||||
|
||||
## Tests Before
|
||||
|
||||
Add/adjust Mongo integration tests gated by `MONGODB_TEST_URL`:
|
||||
|
||||
1. `TestMongoKVStore_RootObjectRepresentation`
|
||||
- `PutJSON` a coin-like portfolio.
|
||||
- Raw Mongo doc has root `usd`, `assets`, `meta`.
|
||||
- Raw Mongo doc has no `value`.
|
||||
- `updatedAt` decodes as `time.Time`.
|
||||
2. `TestMongoKVStore_RootObjectOverwriteRemovesStaleFields`
|
||||
- Write `{a:1,b:2}` then overwrite `{a:3}`.
|
||||
- Raw doc has `a`, no stale `b`.
|
||||
3. `TestMongoKVStore_NonObjectPayloadFallback`
|
||||
- Write bare string and JSON array.
|
||||
- Raw docs use `_payload` + `payloadKind`, no `value`.
|
||||
- `Get` round-trips.
|
||||
4. `TestMongoKVStore_ReservedRootFieldCollision`
|
||||
- Write object with `value`, `version`, or `updatedAt` in payload.
|
||||
- Raw doc preserves payload under `_payload`; `GetJSON` round-trips.
|
||||
5. `TestMongoKVStore_UnsafeMongoFieldNameFallback`
|
||||
- Write object payloads with keys containing null, containing `.`, and
|
||||
starting with `$`.
|
||||
- Raw docs preserve payload under `_payload`; `GetJSON` round-trips.
|
||||
6. `TestMongoKVStore_LegacyValueDocsStillRead`
|
||||
- Seed old docs directly with `value` as string, `bson.Binary`, object,
|
||||
and array.
|
||||
- `Get`/`GetJSON` decode all.
|
||||
7. Keep/strengthen `TestMongoKVStore_PutVersioned_ConcurrentCreate`.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Add named constants in tests for reserved fields expected in the new shape.
|
||||
2. Add raw Mongo assertions before changing implementation.
|
||||
3. Audit current persisted structs for reserved top-level key collisions and
|
||||
document the result in test comments.
|
||||
4. Confirm all new tests fail for the intended reason against current code.
|
||||
5. Keep existing int64-fidelity and legacy-version tests in place.
|
||||
|
||||
## Tests After
|
||||
|
||||
- Re-run the new Mongo storage tests after Phase 2 implementation.
|
||||
- Ensure old native-`value` tests are either rewritten to the new contract or
|
||||
retained only as legacy-read tests.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] New raw-shape tests exist and fail before implementation.
|
||||
- [ ] Legacy-read coverage includes the old `value` field.
|
||||
- [ ] Reserved-field coverage includes payload key `value`.
|
||||
- [ ] Unsafe Mongo field-name coverage includes null, `.`, and `$` cases.
|
||||
- [ ] Stale-field removal has a dedicated regression test.
|
||||
- [ ] `updatedAt` BSON Date behavior has a dedicated assertion.
|
||||
- [ ] Existing version-CAS tests remain in the suite.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Risk: tests assert driver-specific decoded types too tightly. Mitigation:
|
||||
assert behavior and accepted BSON-decoded Go types, not exact internal map
|
||||
ordering.
|
||||
- Risk: reserved-field collision behavior gets forgotten because current structs
|
||||
do not collide. Mitigation: synthetic collision test is required.
|
||||
- Risk: stale-field issue appears only on overwrite. Mitigation: explicit
|
||||
overwrite test before implementation.
|
||||
+139
@@ -0,0 +1,139 @@
|
||||
---
|
||||
phase: 2
|
||||
title: "Root Document Storage Encoding"
|
||||
status: pending
|
||||
priority: P2
|
||||
dependencies: [1]
|
||||
effort: "L"
|
||||
---
|
||||
|
||||
# Phase 2: Root Document Storage Encoding
|
||||
|
||||
## Overview
|
||||
|
||||
Implement the new root-payload Mongo encoding in `MongoKVStore` while preserving
|
||||
all caller-facing storage interfaces. Writes should stop producing `value`,
|
||||
reads should support both new root docs and legacy `value` docs.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Functional: `Put`, `PutJSON`, `PutVersioned`, `Get`, `GetJSON`,
|
||||
`GetVersioned`, `Delete`, and `List` signatures remain unchanged.
|
||||
- Functional: JSON object values flatten to root unless they collide with
|
||||
reserved metadata fields, the legacy `value` field, or unsafe Mongo field
|
||||
names.
|
||||
- Functional: arrays/scalars/non-JSON values use `_payload` and `payloadKind`,
|
||||
never the old `value` field.
|
||||
- Functional: `updatedAt` is stored as `time.Time`.
|
||||
- Functional: `version` increments on plain and versioned writes.
|
||||
- Non-functional: no stale payload fields after overwrite.
|
||||
|
||||
## Architecture
|
||||
|
||||
Split the codec into document-level helpers:
|
||||
|
||||
```go
|
||||
type payloadKind string
|
||||
|
||||
const (
|
||||
payloadKindObject payloadKind = "object"
|
||||
payloadKindArray payloadKind = "array"
|
||||
payloadKindString payloadKind = "string"
|
||||
)
|
||||
|
||||
func encodeRootDocument(key string, val []byte, version int64, now time.Time) (bson.M, error)
|
||||
func decodeRootDocument(key string, doc bson.M) ([]byte, error)
|
||||
func payloadFieldsFromRoot(doc bson.M) bson.M
|
||||
```
|
||||
|
||||
Decode order:
|
||||
|
||||
1. Legacy `value` field exists -> current `decodeValue` compatibility path.
|
||||
2. `_payload` exists -> decode `_payload` by `payloadKind`.
|
||||
3. Else -> collect all root fields except reserved metadata and marshal as JSON.
|
||||
|
||||
Write strategy:
|
||||
|
||||
- `PutVersioned(expectedVersion > 0)`: `ReplaceOne({_id:key, version:expected}, replacement(version=expected+1))`.
|
||||
- `PutVersioned(expectedVersion == 0)`: replace/upsert with filter matching
|
||||
`{_id:key, version missing or 0}`, duplicate key -> `ErrConflict`.
|
||||
- `Put`: implement as a bounded loop over `GetVersioned` + `PutVersioned` so it
|
||||
overwrites unconditionally while still removing stale fields and bumping
|
||||
version. Use existing retry style (`portfolioUpdateAttempts` is 5) or a
|
||||
small storage-local constant.
|
||||
|
||||
Use `ReplaceOne`, not partial `$set`, for root-payload writes. Partial updates
|
||||
are the foot-gun: `{a:1,b:2}` overwritten by `{a:3}` would otherwise leave `b`.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Modify: `internal/storage/mongodb_kv.go` — write/read methods and version
|
||||
replacement logic.
|
||||
- Modify: `internal/storage/mongodb_value_codec.go` — convert byte payload to
|
||||
flattened root docs and reconstruct JSON.
|
||||
- Modify: `internal/storage/mongodb_kv_test.go` — make Phase 1 tests pass.
|
||||
- Read: `internal/storage/memory_kv.go` — keep interface behavior aligned.
|
||||
- Read: `internal/storage/dynamodb_kv.go` — ensure migrate-only DynamoDB stays
|
||||
unaffected.
|
||||
|
||||
## Tests Before
|
||||
|
||||
- Run Phase 1 Mongo tests and confirm failure before implementation.
|
||||
- Run hermetic storage tests to ensure no memory/DynamoDB API changes are
|
||||
required.
|
||||
|
||||
## Refactor
|
||||
|
||||
1. Rename legacy `decodeValue` to make compatibility explicit, e.g.
|
||||
`decodeLegacyValueField`.
|
||||
2. Add reserved-field constants and helper `isMongoRootMetadataField`.
|
||||
3. Add root-document encoder:
|
||||
- trim/decode JSON with `json.Decoder.UseNumber`;
|
||||
- object -> root fields if no reserved/legacy `value` collision and no
|
||||
unsafe key containing null, containing `.`, or starting with `$`;
|
||||
- array -> `_payload` + `payloadKind:"array"`;
|
||||
- scalar/non-JSON -> `_payload` + `payloadKind:"string"` or scalar kind;
|
||||
- always add `_id`, `version`, `updatedAt`, `schemaVersion`.
|
||||
<!-- Updated: Validation Session 1 - reserve legacy value + unsafe Mongo field names -->
|
||||
4. Add root-document decoder:
|
||||
- legacy `value` first;
|
||||
- `_payload` fallback second;
|
||||
- root object reconstruction third.
|
||||
5. Replace `$set`/`$inc` writes with whole-document replacement in
|
||||
`PutVersioned`.
|
||||
6. Rework `Put` to bump versions and replace whole docs without exposing
|
||||
conflicts to ordinary callers unless retries are exhausted.
|
||||
7. Keep `Delete` and `List` unchanged.
|
||||
8. Tighten error messages so malformed root docs name the module/key and
|
||||
missing payload reason.
|
||||
|
||||
## Tests After
|
||||
|
||||
- `go test ./internal/storage`
|
||||
- `MONGODB_TEST_URL=mongodb://127.0.0.1:27017 go test ./internal/storage -run 'MongoKVStore'`
|
||||
- `go test ./internal/modules/coin ./internal/modules/gold ./internal/modules/lolschedule`
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] All Phase 1 tests pass.
|
||||
- [ ] No new write path emits `value`.
|
||||
- [ ] Payload objects with key `value` use `_payload` fallback, not root
|
||||
flattening.
|
||||
- [ ] Payload objects with null/`.`/`$` field names use `_payload` fallback.
|
||||
- [ ] Legacy `value` docs continue to read.
|
||||
- [ ] Plain `Put` and `PutVersioned` both bump `version`.
|
||||
- [ ] Concurrent create/update tests keep exactly-one-winner semantics.
|
||||
- [ ] Stale payload fields are removed on overwrite.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Risk: bounded `Put` loop can conflict under extreme contention. Mitigation:
|
||||
keep retries small but adequate; if tests show trouble, switch to a
|
||||
single-write aggregation pipeline with `$replaceRoot` and `$literal` payload
|
||||
expressions.
|
||||
- Risk: root reconstruction accidentally includes metadata fields. Mitigation:
|
||||
centralize reserved field list and test it.
|
||||
- Risk: `time.Time` changes raw `updatedAt` type. Mitigation: no app reader uses
|
||||
it today; integration tests assert new date behavior.
|
||||
- Risk: malformed existing docs without `value` or root payload become harder
|
||||
to diagnose. Mitigation: descriptive errors in decoder.
|
||||
+126
@@ -0,0 +1,126 @@
|
||||
---
|
||||
phase: 3
|
||||
title: "Migration and Documentation"
|
||||
status: pending
|
||||
priority: P2
|
||||
dependencies: [2]
|
||||
effort: "M"
|
||||
---
|
||||
|
||||
# Phase 3: Migration and Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
Add an operator-safe path to rewrite existing Mongo docs from the old `value`
|
||||
envelope into the new flattened root shape, and update docs that currently
|
||||
describe the old layout.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Functional: existing Atlas documents can be migrated in place without relying
|
||||
on DynamoDB still existing.
|
||||
- Functional: migration is idempotent and reports per-collection counts.
|
||||
- Functional: migration preserves logical `Get`/`GetJSON` values and bumps or
|
||||
normalizes `version` consistently.
|
||||
- Non-functional: migration must not read or print secrets; `MONGO_URL` remains
|
||||
secret.
|
||||
- Non-functional: docs must stop calling int64 `updatedAt` TTL-ready.
|
||||
|
||||
## Architecture
|
||||
|
||||
Add a small Mongo-to-Mongo schema migration command rather than overloading the
|
||||
DynamoDB migrator:
|
||||
|
||||
```text
|
||||
cmd/migrate-mongo-schema/
|
||||
main.go
|
||||
main_test.go
|
||||
```
|
||||
|
||||
Tool behavior:
|
||||
|
||||
1. Connect using `MONGO_URL` + `MONGO_DATABASE`.
|
||||
2. Iterate collections or an allow-list flag.
|
||||
3. For each document:
|
||||
- skip if it already has `schemaVersion >= 2` and no `value`;
|
||||
- read through `MongoKVStore.GetVersioned` so legacy decode paths are used;
|
||||
- rewrite through `PutVersioned` so root encoding is shared with app writes;
|
||||
- on `ErrConflict`, re-read the document: skip it if a live app write
|
||||
already migrated it, otherwise retry up to a small bounded limit;
|
||||
- count migrated/skipped/errors.
|
||||
4. `--dry-run` prints counts without writes.
|
||||
5. `--verify` confirms no documents with `value` remain, except explicitly
|
||||
allowed malformed docs if any are reported.
|
||||
|
||||
Do not drop or rename collections. Keep one collection per module.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Create: `cmd/migrate-mongo-schema/main.go`
|
||||
- Create: `cmd/migrate-mongo-schema/main_test.go`
|
||||
- Modify: `Makefile` — add a migration target, e.g. `migrate-mongo-schema`.
|
||||
- Modify: `README.md` — update storage layout summary.
|
||||
- Modify: `docs/deploy-coolify-selfhosted.md` — update Atlas storage layout
|
||||
and migration note.
|
||||
- Modify: `cmd/migrate-dynamo-to-mongo/README.md` — update destination shape.
|
||||
- Modify: `cmd/migrate-dynamo-to-mongo/main.go` comments only if stale.
|
||||
- Read: `cmd/migrate-dynamo-to-mongo/main.go` — reuse connection/config style.
|
||||
|
||||
## Tests Before
|
||||
|
||||
- Add migrator e2e test that seeds old `value` docs directly, runs migration,
|
||||
then asserts:
|
||||
- logical values round-trip;
|
||||
- raw docs have no `value`;
|
||||
- `updatedAt` is `time.Time`;
|
||||
- counts match.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Implement CLI flags:
|
||||
- `--dry-run`
|
||||
- `--verify`
|
||||
- `--collection` optional repeat/single collection selector.
|
||||
2. Reuse `storage.NewMongoClient`, `storage.NewMongoDatabase`, and
|
||||
`storage.NewMongoProvider`.
|
||||
3. Add collection listing using the Mongo driver, excluding system collections.
|
||||
4. Rewrite docs through the storage layer, not raw BSON mutation.
|
||||
5. Print a concise table: collection, total, migrated, skipped, errors.
|
||||
6. Add Makefile target with env examples.
|
||||
7. Update docs and remove stale wording:
|
||||
- no old `value` envelope in new writes;
|
||||
- `updatedAt` is BSON Date;
|
||||
- legacy `value` docs dual-read until migrated.
|
||||
8. Add migration conflict tests: simulate a version change between read and
|
||||
write; assert the migrator re-reads and either skips already-migrated docs or
|
||||
retries legacy docs without clobbering the concurrent update.
|
||||
<!-- Updated: Validation Session 1 - migration must handle live-write conflicts -->
|
||||
|
||||
## Tests After
|
||||
|
||||
- `go test ./cmd/migrate-mongo-schema`
|
||||
- `MONGODB_TEST_URL=mongodb://127.0.0.1:27017 go test ./cmd/migrate-mongo-schema ./internal/storage`
|
||||
- `make test`
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] In-place migration rewrites old `value` docs without changing logical app
|
||||
values.
|
||||
- [ ] Migration is idempotent.
|
||||
- [ ] `--dry-run` performs no writes.
|
||||
- [ ] `--verify` exits non-zero when old `value` docs remain.
|
||||
- [ ] Migration handles version conflicts by re-reading and never clobbers a
|
||||
concurrent app write.
|
||||
- [ ] README and Coolify deploy docs show the new flattened layout.
|
||||
- [ ] DynamoDB-to-Mongo migrator docs no longer mention BSON binary/string as
|
||||
the destination `value` representation.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Risk: migration touches live data. Mitigation: dry-run + verify + local e2e
|
||||
test; recommend backup/export before production run.
|
||||
- Risk: old DynamoDB source may be gone. Mitigation: in-place Mongo migration is
|
||||
independent of DynamoDB.
|
||||
- Risk: malformed documents stop migration. Mitigation: report collection/key
|
||||
and continue only if an explicit `--continue-on-error` is later accepted;
|
||||
default should fail loud.
|
||||
@@ -0,0 +1,121 @@
|
||||
---
|
||||
phase: 4
|
||||
title: "Verification and Rollout"
|
||||
status: pending
|
||||
priority: P2
|
||||
dependencies: [3]
|
||||
effort: "M"
|
||||
---
|
||||
|
||||
# Phase 4: Verification and Rollout
|
||||
|
||||
## Overview
|
||||
|
||||
Run the full validation suite, check raw Mongo documents, and define the safe
|
||||
production rollout order. This phase is the gate before deploying a storage
|
||||
schema change.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Functional: app behavior remains unchanged for all Telegram commands.
|
||||
- Functional: Mongo raw shape is verified for representative module documents.
|
||||
- Functional: legacy docs remain readable before migration and absent after
|
||||
migration verify.
|
||||
- Non-functional: rollout includes backup, deploy, migration, verify, and
|
||||
rollback notes.
|
||||
|
||||
## Architecture
|
||||
|
||||
Rollout order:
|
||||
|
||||
1. Backup/export current Atlas database or snapshot if available.
|
||||
2. Deploy app version with dual-read + new-write root encoding.
|
||||
3. Let app run; new/touched docs rewrite naturally.
|
||||
4. Run `migrate-mongo-schema --dry-run`.
|
||||
5. Run `migrate-mongo-schema`.
|
||||
6. Run `migrate-mongo-schema --verify`.
|
||||
7. Inspect sample docs in Atlas/Compass:
|
||||
- coin/gold/stock portfolio root fields;
|
||||
- wordle/loldle/twentyq game state root fields;
|
||||
- lolschedule scalar date guard under `_payload`;
|
||||
- no old `value` on migrated docs.
|
||||
|
||||
Rollback:
|
||||
|
||||
- If the new app fails before it handles writes, roll back binary; old `value`
|
||||
docs are still readable by the previous build.
|
||||
- Once the new app has handled writes, rollback to the previous binary is unsafe
|
||||
for any touched flattened docs. Prefer fixing forward, or restore the backup.
|
||||
- If migration ran, previous build may not read flattened docs at all. Rollback
|
||||
after migration requires restoring backup or a reverse migrator. Prefer fixing
|
||||
forward unless the migration corrupted data.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Modify: tests only if verification gaps are found.
|
||||
- Read: `Makefile`
|
||||
- Read: `README.md`
|
||||
- Read: `docs/deploy-coolify-selfhosted.md`
|
||||
- Read: `cmd/migrate-mongo-schema/main.go`
|
||||
- Read: `internal/storage/mongodb_kv_test.go`
|
||||
|
||||
## Tests Before
|
||||
|
||||
- Confirm Phase 1-3 tests run locally.
|
||||
- Start local Mongo with `make mongo-local` for integration tests.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Run focused storage tests.
|
||||
2. Run module tests that rely on versioned storage flows:
|
||||
- coin portfolio;
|
||||
- gold portfolio;
|
||||
- lolschedule daily-push claim.
|
||||
3. Run full hermetic suite.
|
||||
4. Run Mongo integration suite.
|
||||
5. Run DynamoDB tests only if the DynamoDB migrator or shared storage contracts
|
||||
changed.
|
||||
6. Manually inspect raw docs or add a small test assertion report for:
|
||||
- no `value`;
|
||||
- expected root fields;
|
||||
- Date `updatedAt`;
|
||||
- stable `version`.
|
||||
7. Document production run commands in the final completion report.
|
||||
|
||||
## Tests After
|
||||
|
||||
Required gates:
|
||||
|
||||
```sh
|
||||
make vet
|
||||
make test
|
||||
MONGODB_TEST_URL=mongodb://127.0.0.1:27017 go test ./internal/storage ./cmd/migrate-mongo-schema
|
||||
```
|
||||
|
||||
Conditional gate if DynamoDB migrator comments/tests changed:
|
||||
|
||||
```sh
|
||||
make test-dynamodb
|
||||
```
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] `make vet` passes.
|
||||
- [ ] `make test` passes.
|
||||
- [ ] Mongo integration tests pass against local Mongo.
|
||||
- [ ] New migrator e2e test passes.
|
||||
- [ ] Raw docs verified no longer use `value` after migration.
|
||||
- [ ] Production rollout notes include backup and rollback constraints.
|
||||
- [ ] No user-visible Telegram command behavior changes.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Risk: previous binary cannot read newly-written or post-migration flattened
|
||||
docs. Mitigation: deploy dual-read binary first; backup before rollout;
|
||||
rollback old binary only before new writes happen; fix forward preferred.
|
||||
- Risk: Atlas M0 resource limits during migration. Mitigation: tiny dataset
|
||||
expected; still scan collection-by-collection and report progress.
|
||||
- Risk: hidden module stores non-object values. Mitigation: fallback `_payload`
|
||||
contract and migration tests for scalar/array docs.
|
||||
- Risk: docs drift. Mitigation: grep for old `{ _id, value, version, updatedAt }`
|
||||
wording before finalizing.
|
||||
@@ -0,0 +1,221 @@
|
||||
---
|
||||
title: "Flatten Mongo value documents"
|
||||
description: "Remove Mongo's generic value envelope for JSON object values, keep KVStore callers stable, and store Mongo metadata in queryable root fields."
|
||||
status: superseded
|
||||
supersededBy: 260628-1318-mongo-native-typed-stores
|
||||
priority: P2
|
||||
branch: "feature/selfhosted"
|
||||
tags: [database, mongodb, refactor, storage, tdd]
|
||||
blockedBy: [260627-1849-selfhost-coolify-mongodb]
|
||||
blocks: []
|
||||
created: "2026-06-28T06:10:29.184Z"
|
||||
createdBy: "ck:plan"
|
||||
source: skill
|
||||
---
|
||||
|
||||
# Flatten Mongo value documents
|
||||
|
||||
> **Superseded (2026-06-28)** by
|
||||
> [`260628-1318-mongo-native-typed-stores`](../260628-1318-mongo-native-typed-stores/plan.md).
|
||||
> User chose the full Mongo-native direction this plan deferred: delete the
|
||||
> `KVStore` abstraction in favor of typed stores, MongoDB-only runtime (memory
|
||||
> kept for tests), and drop the legacy `value` dual-read + in-place Mongo
|
||||
> migrator (Atlas is empty until cutover, so the DynamoDB→Mongo migrator writes
|
||||
> the final flattened shape directly). Kept for history; do not implement.
|
||||
|
||||
## Overview
|
||||
|
||||
Change MongoDB's on-disk KV document shape from a generic envelope:
|
||||
|
||||
```javascript
|
||||
{ _id, value: { ... }, version, updatedAt }
|
||||
```
|
||||
|
||||
to a more Mongo-native root-document shape for JSON object values:
|
||||
|
||||
```javascript
|
||||
{ _id, ...payloadFields, version, updatedAt: ISODate(...), schemaVersion }
|
||||
```
|
||||
|
||||
Keep the public `KVStore` / `VersionedStore` interfaces stable so modules keep
|
||||
using `GetJSON`, `PutJSON`, `GetVersioned`, and `PutVersioned`. This plan is a
|
||||
storage-representation refactor, not a full rewrite to typed repositories.
|
||||
|
||||
Non-object values cannot be Mongo root documents. New writes for arrays/scalars
|
||||
use a small reserved `_payload` fallback with `payloadKind`, but **new writes
|
||||
must not use the old `value` field**. Legacy docs with `value` remain readable
|
||||
and are rewritten to the new shape by normal updates or by the in-place
|
||||
migration tool.
|
||||
|
||||
## Scope Challenge
|
||||
|
||||
- Existing code: `MongoKVStore` already owns all Mongo encoding/decoding and
|
||||
version CAS; `mongodb_value_codec.go` already preserves `int64` via
|
||||
`json.Number`; modules do not need direct Mongo access.
|
||||
- Minimum changes: storage codec + raw-shape tests + migrator/docs. Defer
|
||||
per-module typed repositories until there is a real query/report need.
|
||||
- Complexity: touches storage, one migration command, docs, and tests. Avoid
|
||||
changing module handlers or public storage interfaces.
|
||||
- Selected mode: HOLD SCOPE. Refactor the persisted Mongo shape; do not expand
|
||||
into a full domain repository rewrite.
|
||||
|
||||
## Design Decision
|
||||
|
||||
Chosen approach: **root-payload KV documents**.
|
||||
|
||||
Why not full typed repositories now: it would touch most modules, duplicate
|
||||
test plumbing, and remove the memory backend value before there is a query need.
|
||||
Why not keep `value`: it defeats the user's Mongo document-design goal and
|
||||
keeps domain fields one level away from simple Compass/query/index use.
|
||||
|
||||
Reserved root fields:
|
||||
|
||||
| Field | Purpose |
|
||||
|---|---|
|
||||
| `_id` | Module-local KV key, unchanged |
|
||||
| `value` | Legacy envelope field; forbidden in new root payloads |
|
||||
| `version` | Optimistic-lock token, unchanged semantics |
|
||||
| `updatedAt` | BSON Date (`time.Time`), not unix nanos |
|
||||
| `schemaVersion` | Storage document shape version |
|
||||
| `payloadKind` | Only needed for non-object fallback or collision fallback |
|
||||
| `_payload` | Array/scalar payload fallback; never used for ordinary JSON objects |
|
||||
|
||||
Mongo field-name guard: object payloads also fall back to `_payload` if any
|
||||
flattened key contains the null character, contains `.`, or starts with `$`.
|
||||
MongoDB permits `.` and `$` in modern versions but documents restrictions around
|
||||
them; fallback keeps normal query/index/schema behavior predictable.
|
||||
Reference: https://www.mongodb.com/docs/manual/core/document/#field-names
|
||||
|
||||
Decoder rules:
|
||||
|
||||
1. If old `value` exists, decode using the legacy path.
|
||||
2. Else if `_payload` exists, decode `_payload` according to `payloadKind`.
|
||||
3. Else reconstruct the JSON object from all non-reserved root fields.
|
||||
|
||||
Writer rules:
|
||||
|
||||
1. JSON object without reserved-field collision writes payload fields at root.
|
||||
2. JSON array, scalar, invalid JSON, or object with reserved-field collision
|
||||
or unsafe Mongo field names writes `_payload` + `payloadKind`.
|
||||
3. Writes replace the whole document (except the `_id` key value is preserved in
|
||||
the replacement) so stale old payload fields cannot survive an overwrite.
|
||||
4. `PutVersioned` keeps single-document CAS semantics by filtering on `_id` and
|
||||
`version`.
|
||||
5. Plain `Put` must still bump `version`; implement as a bounded versioned
|
||||
write loop or a carefully tested aggregation update pipeline. Prefer the
|
||||
simpler loop unless tests prove unacceptable contention.
|
||||
|
||||
## Cross-Plan Dependencies
|
||||
|
||||
| Relationship | Plan | Status | Rationale |
|
||||
|---|---|---|---|
|
||||
| Blocked by | `260627-1849-selfhost-coolify-mongodb` | in-progress | Created the Mongo provider, migrator, and self-host runtime this refactor modifies. |
|
||||
| Supersedes storage choice from | `260628-1113-mongo-native-value-documents` | completed | Keeps its version CAS + native BSON goal, but replaces the `value` envelope with root payload fields. |
|
||||
|
||||
## Phases
|
||||
|
||||
| Phase | Name | Status |
|
||||
|-------|------|--------|
|
||||
| 1 | [Schema Contract and Regression Tests](./phase-01-schema-contract-and-regression-tests.md) | Pending |
|
||||
| 2 | [Root Document Storage Encoding](./phase-02-root-document-storage-encoding.md) | Pending |
|
||||
| 3 | [Migration and Documentation](./phase-03-migration-and-documentation.md) | Pending |
|
||||
| 4 | [Verification and Rollout](./phase-04-verification-and-rollout.md) | Pending |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Existing Mongo provider and version-CAS code from
|
||||
`plans/260628-1113-mongo-native-value-documents/`.
|
||||
- Research report:
|
||||
`plans/reports/research-260628-0605-mongodb-document-design-standards.md`.
|
||||
- MongoDB official guidance used in the research report:
|
||||
- https://www.mongodb.com/docs/manual/data-modeling/
|
||||
- https://www.mongodb.com/docs/manual/data-modeling/best-practices/
|
||||
- https://www.mongodb.com/docs/manual/core/write-operations-atomicity/
|
||||
- https://www.mongodb.com/docs/manual/core/index-ttl/
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] New JSON object writes contain no top-level `value` field; payload fields
|
||||
live at root with `_id`, `version`, `updatedAt`, and `schemaVersion`.
|
||||
- [ ] Payload objects with top-level `value`, reserved metadata keys, null
|
||||
characters, `.`, or `$`-prefixed keys use `_payload` fallback and round-trip.
|
||||
- [ ] New array/scalar/non-JSON writes contain no old `value` field; they use
|
||||
`_payload` + `payloadKind` and round-trip byte-equivalent where required.
|
||||
- [ ] `updatedAt` stores as BSON Date / Go `time.Time`, not int64 nanos.
|
||||
- [ ] Legacy docs with `value` as string, binary, object, or array still read.
|
||||
- [ ] Overwriting a document removes stale payload fields from prior values.
|
||||
- [ ] Versioned writes remain single-winner under concurrent create/update.
|
||||
- [ ] A Mongo in-place migration can rewrite existing `value` docs to the new
|
||||
shape without changing logical values or document counts.
|
||||
- [ ] `make test`, `make vet`, and Mongo integration tests pass.
|
||||
|
||||
## Not In Scope
|
||||
|
||||
- Replacing every module's `KVStore` usage with typed Mongo repositories.
|
||||
- Adding Mongo schema validation/indexes for every module.
|
||||
- Deleting DynamoDB migrator support.
|
||||
- Changing Telegram bot behavior or command outputs.
|
||||
|
||||
## Validation Log
|
||||
|
||||
### Session 1 — 2026-06-28
|
||||
**Trigger:** `/ck:plan validate /config/workspace/tiennm99/miti99bot/plans/260628-1310-flatten-mongo-value-documents/plan.md`
|
||||
**Questions asked:** 0 — validation found technical plan gaps with clear repo/doc evidence; no user-facing trade-off needed.
|
||||
|
||||
#### Plan Recap
|
||||
|
||||
- Refactor Mongo on-disk KV documents from `{ _id, value, version, updatedAt }`
|
||||
to root payload fields.
|
||||
- Keep `KVStore` / `VersionedStore` caller contracts stable.
|
||||
- Keep legacy `value` docs readable.
|
||||
- Use `_payload` only for non-object or unsafe/colliding object payloads.
|
||||
- Store `updatedAt` as BSON Date.
|
||||
- Add an in-place Mongo schema migration and rollout guardrails.
|
||||
|
||||
#### Verification Results
|
||||
|
||||
- **Tier:** Standard (4 phases)
|
||||
- **Claims checked:** 32
|
||||
- **Verified:** 29 | **Failed:** 3 | **Unverified:** 0
|
||||
|
||||
Failures fixed in this validation:
|
||||
|
||||
1. [Contract Verifier] Reserved fields omitted `value`. Evidence:
|
||||
`internal/storage/mongodb_kv.go:19` uses `mongoValueField = "value"` and
|
||||
`decodeValue` treats any top-level `value` as legacy payload
|
||||
(`internal/storage/mongodb_kv.go:46`). A flattened payload containing
|
||||
top-level `value` would violate the no-`value` contract and decode wrong.
|
||||
2. [Fact Checker] Field-name guard omitted null/`.`/`$` cases. Evidence:
|
||||
MongoDB documents forbid null in field names and document restrictions for
|
||||
`.`/`$` field names: https://www.mongodb.com/docs/manual/core/document/#field-names.
|
||||
The plan now falls back to `_payload` for those keys.
|
||||
3. [Contract Verifier] Migration rewrite via `PutVersioned` lacked conflict
|
||||
retry handling. Evidence: `PutVersioned` returns `ErrConflict` on version
|
||||
mismatch (`internal/storage/kv_store.go:35`) and concurrent app writes are
|
||||
possible after deploying the dual-read/new-write binary. Phase 3 now requires
|
||||
re-read/skip/retry behavior.
|
||||
|
||||
#### Confirmed Decisions
|
||||
|
||||
- Keep `value` reserved forever for backward-compatible legacy decode.
|
||||
- Use `_payload` fallback for reserved keys and Mongo-awkward field names.
|
||||
- Keep migration writes through storage encoding and handle conflicts by
|
||||
re-reading, skipping already-migrated docs, or retrying.
|
||||
|
||||
#### Impact on Phases
|
||||
|
||||
- Phase 1: add tests for payload key `value`, null/`.`/`$` field names, and
|
||||
migration conflict setup.
|
||||
- Phase 2: update reserved-field and unsafe-field encoder rules.
|
||||
- Phase 3: add conflict retry semantics to the migration command.
|
||||
|
||||
### Whole-Plan Consistency Sweep
|
||||
|
||||
- Files reread: `plan.md`, all four `phase-*.md` files.
|
||||
- Decision deltas checked: 3 (`value` reserved, unsafe field fallback, migration conflict retry).
|
||||
- Reconciled stale references: 6.
|
||||
- Unresolved contradictions: 0.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None.
|
||||
+107
@@ -0,0 +1,107 @@
|
||||
---
|
||||
phase: 1
|
||||
title: "Typed DocStore contract + memory backend"
|
||||
status: done
|
||||
priority: P2
|
||||
dependencies: []
|
||||
effort: "M"
|
||||
---
|
||||
|
||||
# Phase 1: Typed DocStore contract + memory backend
|
||||
|
||||
## Overview
|
||||
|
||||
Define the new typed storage contract and a memory implementation first, with
|
||||
tests, before touching Mongo or any module. This nails the interface every
|
||||
module and the Mongo backend will target.
|
||||
|
||||
## Requirements
|
||||
|
||||
- `DocStore[T]` generic interface: `Get`, `Put`, `PutVersioned`, `Delete`, `List`.
|
||||
- `Provider.Collection(module)` returns an opaque per-module `Collection`.
|
||||
- `Typed[T](Collection) DocStore[T]` free function (methods cannot be generic).
|
||||
- Memory implementation: hermetic, concurrency-safe, version-CAS correct.
|
||||
- Reserved-field name guard: payload BSON tags must not collide with `_id`,
|
||||
`version`, `updatedAt`.
|
||||
- `ErrNotFound`, `ErrConflict` reused (move to this package if `kv_store.go` is deleted later).
|
||||
|
||||
## Architecture
|
||||
|
||||
```go
|
||||
// doc_store.go
|
||||
type DocStore[T any] interface {
|
||||
Get(ctx, id) (T, int64, error)
|
||||
Put(ctx, id, T) error
|
||||
PutVersioned(ctx, id, expectedVersion int64, T) error
|
||||
Delete(ctx, id) error
|
||||
List(ctx, prefix string) ([]string, error)
|
||||
}
|
||||
|
||||
type Provider interface{ Collection(module string) Collection }
|
||||
type Collection interface{ collection() } // sealed marker
|
||||
|
||||
func Typed[T any](c Collection) DocStore[T] // switch concrete type → mongo|memory store
|
||||
```
|
||||
|
||||
Memory backend:
|
||||
|
||||
```go
|
||||
// memory_provider.go (replaces kv_provider.go's MemoryProvider)
|
||||
type MemoryProvider struct{ mu sync.Mutex; cols map[string]*memoryCollection }
|
||||
type memoryCollection struct{ mu sync.Mutex; rows map[string]memoryRow }
|
||||
type memoryRow struct{ version int64; val any } // val holds the typed value
|
||||
|
||||
type MemoryDocStore[T any] struct{ c *memoryCollection }
|
||||
```
|
||||
|
||||
- `Get`: type-assert `row.val.(T)`; missing → `ErrNotFound`.
|
||||
- `Put`: overwrite, `version++`.
|
||||
- `PutVersioned`: compare stored version to expected; mismatch → `ErrConflict`;
|
||||
`expected==0` requires absent/zero-version row.
|
||||
- `Delete`: idempotent.
|
||||
- `List(prefix)`: sorted keys with prefix (reuse `validatePrefix`).
|
||||
|
||||
Keys: keep `validateKey`/`validatePrefix` from `keys.go`/`prefix.go`.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Create: `internal/storage/doc_store.go` (interface, Provider, Collection, `Typed`, errors).
|
||||
- Create: `internal/storage/memory_provider.go` (MemoryProvider + memoryCollection + MemoryDocStore).
|
||||
- Create: `internal/storage/memory_doc_store_test.go`.
|
||||
- Create: `internal/storage/reserved_fields.go` + test (reserved-name guard helper).
|
||||
- Read: `internal/storage/keys.go`, `prefix.go`, `kv_store.go` (for error vars to migrate).
|
||||
|
||||
## Tests Before
|
||||
|
||||
Write failing tests first:
|
||||
|
||||
1. `TestMemoryDocStore_PutGetRoundTrip` — typed struct round-trips; version starts at 1.
|
||||
2. `TestMemoryDocStore_PutBumpsVersion` — repeated Put increments version.
|
||||
3. `TestMemoryDocStore_PutVersioned_CAS` — stale expected → `ErrConflict`; fresh succeeds.
|
||||
4. `TestMemoryDocStore_PutVersioned_CreateZero` — `expected==0` creates; second create → `ErrConflict`.
|
||||
5. `TestMemoryDocStore_Delete_Idempotent`.
|
||||
6. `TestMemoryDocStore_List_Prefix` — disjoint key prefixes isolate types in one collection.
|
||||
7. `TestReservedFields_RejectsCollision` — a payload type tagged `version` is flagged.
|
||||
8. `TestMemoryProvider_CollectionIsolation` — two modules' collections don't share keys.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Add `doc_store.go` with interface, sealed `Collection`, `Typed[T]`, and the
|
||||
`ErrNotFound`/`ErrConflict` vars (kept here so `kv_store.go` can be deleted).
|
||||
2. Implement memory provider + store.
|
||||
3. Implement reserved-field reflection guard (`bson` tag scan).
|
||||
4. Make all Phase 1 tests pass.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] `DocStore[T]`, `Provider`, `Collection`, `Typed` compile and are documented.
|
||||
- [ ] Memory store passes CAS/version/list/delete tests.
|
||||
- [ ] Reserved-name guard catches `_id`/`version`/`updatedAt` collisions.
|
||||
- [ ] No dependency on the soon-to-be-deleted byte `KVStore`.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Risk: generic free-function `Typed[T]` is unusual. Mitigation: document why
|
||||
(Go methods can't be generic); single switch point, well tested.
|
||||
- Risk: memory `val any` type mismatch panics. Mitigation: `Get` returns a typed
|
||||
error on failed assertion, not a panic.
|
||||
@@ -0,0 +1,116 @@
|
||||
---
|
||||
phase: 2
|
||||
title: "Mongo typed store + integration tests"
|
||||
status: done
|
||||
priority: P2
|
||||
dependencies: [1]
|
||||
effort: "L"
|
||||
---
|
||||
|
||||
# Phase 2: Mongo typed store + integration tests
|
||||
|
||||
## Overview
|
||||
|
||||
Implement `MongoDocStore[T]` and a `MongoProvider` that satisfies the Phase 1
|
||||
contract, persisting typed payloads as flattened native root documents. Lock the
|
||||
raw shape and CAS semantics with Mongo integration tests.
|
||||
|
||||
## Requirements
|
||||
|
||||
- `MongoProvider.Collection(module)` returns a Mongo-backed `Collection`.
|
||||
- `Typed[T]` over a Mongo collection yields `MongoDocStore[T]`.
|
||||
- Write shape: `{ _id, ...payload, version, updatedAt(Date) }`; no `value`, no `_payload`.
|
||||
- Whole-document `ReplaceOne` (no `$set`) so overwrites drop stale fields.
|
||||
- Version CAS identical in behavior to the current `MongoKVStore`.
|
||||
- `updatedAt` stored as `time.Time`.
|
||||
- Plain `Put` bumps version via a bounded versioned-write retry loop.
|
||||
|
||||
## Architecture
|
||||
|
||||
```go
|
||||
type storedDoc[T any] struct {
|
||||
ID string `bson:"_id"`
|
||||
Version int64 `bson:"version"`
|
||||
UpdatedAt time.Time `bson:"updatedAt"`
|
||||
Payload T `bson:",inline"`
|
||||
}
|
||||
|
||||
type MongoDocStore[T any] struct{ coll *mongo.Collection }
|
||||
```
|
||||
|
||||
- `Get`: `FindOne({_id})` → decode `storedDoc[T]`; `ErrNoDocuments`→`ErrNotFound`;
|
||||
return `Payload`, `Version`.
|
||||
- `PutVersioned(expected>0)`: `ReplaceOne({_id, version:expected}, storedDoc{version:expected+1, now, payload})`;
|
||||
`MatchedCount==0`→`ErrConflict`.
|
||||
- `PutVersioned(0)`: upsert filter `{_id, $or:[version absent, version:0]}` with
|
||||
replacement `version:1`; duplicate-key→`ErrConflict`. (Port existing logic.)
|
||||
- `Put`: loop `Get`→`PutVersioned(version)` up to N attempts (reuse the small
|
||||
retry constant style already in coin/gold); first write uses `expected=0`.
|
||||
- `Delete`: `DeleteOne` idempotent.
|
||||
- `List(prefix)`: half-open `_id` range scan via `prefixSuccessor`, project `_id`
|
||||
only. (Port from current `MongoKVStore.List`.)
|
||||
- Error messages name module + key + reason.
|
||||
|
||||
`MongoProvider` wraps `*mongo.Database`; `Collection(module)` validates the
|
||||
module name (reuse existing validation) and returns `mongoCollection{db.Collection(module)}`.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Create: `internal/storage/mongo_doc_store.go` (MongoDocStore + storedDoc).
|
||||
- Modify: `internal/storage/mongodb_provider.go` (implement new `Provider`/`Collection`).
|
||||
- Create: `internal/storage/mongo_doc_store_test.go` (integration, gated by `MONGODB_TEST_URL`).
|
||||
- Delete (this phase): `mongodb_kv.go`, `mongodb_value_codec.go`, `mongodb_kv_test.go`.
|
||||
- Read: current `mongodb_kv.go` (port CAS + List), `mongodb_provider.go`.
|
||||
|
||||
## Tests Before
|
||||
|
||||
Gated by `MONGODB_TEST_URL`:
|
||||
|
||||
1. `TestMongoDocStore_RootShape` — Put a portfolio-like struct; raw `bson.M` has
|
||||
root payload fields, `version`, `updatedAt`; **no** `value`, **no** `_payload`;
|
||||
`updatedAt` decodes as `time.Time`.
|
||||
2. `TestMongoDocStore_OverwriteRemovesStaleFields` — write struct A with field
|
||||
set X, overwrite with struct B lacking some of X; raw doc has no stale field.
|
||||
3. `TestMongoDocStore_PutVersioned_ConcurrentCreate` — N goroutines create same
|
||||
`_id`; exactly one wins, rest get `ErrConflict`.
|
||||
4. `TestMongoDocStore_PutVersioned_Update` — stale expected → `ErrConflict`.
|
||||
5. `TestMongoDocStore_Put_BumpsVersion` — plain Put increments version, survives
|
||||
a concurrent bump (retry loop succeeds or conflicts deterministically).
|
||||
6. `TestMongoDocStore_List_Prefix`.
|
||||
7. `TestMongoDocStore_WrappedScalar` — a `struct{ Date string }` payload stores
|
||||
`date` at root (proves the lolschedule wrapping works).
|
||||
8. `TestMongoDocStore_WrappedArray` — a `struct{ Subscribers []sub }` payload
|
||||
stores `subscribers` array at root.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Implement `MongoDocStore[T]` (port CAS + List from `MongoKVStore`, typed).
|
||||
2. Reshape `MongoProvider` to the new `Provider`/`Collection`; wire `Typed[T]`.
|
||||
3. Delete `mongodb_kv.go`, `mongodb_value_codec.go` and their tests.
|
||||
4. `go build ./internal/storage` — expect module/server breakage deferred to
|
||||
Phase 3 (storage package itself must compile).
|
||||
5. Make Phase 2 integration tests pass against `make mongo-local`.
|
||||
|
||||
## Tests After
|
||||
|
||||
```sh
|
||||
go test ./internal/storage
|
||||
MONGODB_TEST_URL=mongodb://127.0.0.1:27017 go test ./internal/storage -run 'MongoDocStore'
|
||||
```
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Raw Mongo doc is flattened root payload + `version` + Date `updatedAt`; no `value`/`_payload`.
|
||||
- [ ] Overwrite removes stale fields.
|
||||
- [ ] Concurrent create/update keep single-winner CAS.
|
||||
- [ ] Wrapped scalar/array payloads store as named root fields.
|
||||
- [ ] Old `MongoKVStore`/codec deleted; storage package builds.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Risk: `bson:",inline"` edge cases (pointer payloads, embedded maps). Mitigation:
|
||||
require payload `T` to be a concrete struct; test scalar/array wrappers.
|
||||
- Risk: bounded `Put` loop contention. Mitigation: small retry count; CAS users
|
||||
call `PutVersioned` directly anyway.
|
||||
- Risk: driver decodes `version` as int32. Mitigation: `storedDoc.Version int64`
|
||||
+ decode test.
|
||||
@@ -0,0 +1,115 @@
|
||||
---
|
||||
phase: 3
|
||||
title: "Migrate modules + wiring; delete KVStore"
|
||||
status: done
|
||||
priority: P2
|
||||
dependencies: [2]
|
||||
effort: "XL"
|
||||
---
|
||||
|
||||
# Phase 3: Migrate modules + wiring; delete KVStore
|
||||
|
||||
## Overview
|
||||
|
||||
Switch every module from the byte `KVStore` to typed `DocStore[T]`, rewire
|
||||
`Deps`/registry/server, and delete the old byte abstraction and DynamoDB runtime
|
||||
backend. This is the largest phase; do it module-by-module with tests green
|
||||
after each.
|
||||
|
||||
## Requirements
|
||||
|
||||
- `Deps.KV storage.KVStore` → `Deps.Store storage.Collection`.
|
||||
- `registry.Build` takes `storage.Provider` instead of `storage.KVProvider`.
|
||||
- Each module builds typed stores via `storage.Typed[T](deps.Store)`.
|
||||
- CAS behavior preserved for coin, gold, lolschedule last-push.
|
||||
- `lolschedule` subscribers + last-push wrapped in named structs.
|
||||
- No remaining references to `KVStore`, `VersionedStore`, `PutJSON`, `GetJSON`,
|
||||
`kv.Put`/`kv.Get` byte calls, or the DynamoDB runtime backend.
|
||||
- `cmd/server/main.go` `buildProvider` returns a Mongo `Provider` (or memory for
|
||||
`MODULES=`/no-`MONGO_URL` local runs); the `dynamodb` runtime branch is removed.
|
||||
|
||||
## Per-module migration map
|
||||
|
||||
| Module | Keys / payload types | Store ops | Notes |
|
||||
|---|---|---|---|
|
||||
| coin | `Portfolio` (per user) | Get/PutVersioned (CAS loop) | typed CAS via `DocStore[Portfolio]` |
|
||||
| gold | `Portfolio` (per user) | Get/PutVersioned (CAS loop) | same |
|
||||
| stock | `Portfolio` (per user) | Get/Put | no CAS today; keep plain Put |
|
||||
| wordle | `GameState`, `Stats` | Get/Put | two typed views, disjoint key prefixes |
|
||||
| loldle | `gameState`, `stats`, `roundConfig` | Get/Put/Delete | three typed views |
|
||||
| twentyq | `GameState`, `Stats` | Get/Put/Delete | two typed views |
|
||||
| lolschedule | `subscribers` (wrap `[]Subscriber`), `lastPush` (wrap date string) | List/Put + CAS on last-push | **wrap array+scalar in named structs** |
|
||||
| stats | counter entry struct(s) | Get/Put/List | List-heavy (`views.go`) |
|
||||
| misc | `lastPing` struct | Get/Put | |
|
||||
| util | none | — | no storage |
|
||||
|
||||
`lolschedule` wrappers:
|
||||
|
||||
```go
|
||||
type subscribersDoc struct { Subscribers []Subscriber `bson:"subscribers" json:"subscribers"` }
|
||||
type lastPushDoc struct { Date string `bson:"date" json:"date"` }
|
||||
```
|
||||
|
||||
`claimDailyPush` uses `DocStore[lastPushDoc].Get`/`PutVersioned` for the
|
||||
single-winner daily claim (replaces the current `VersionedStore` byte path).
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Modify: `internal/modules/module.go` (Deps.Store), `registry.go` (Build signature, `Collection(name)`).
|
||||
- Modify: every module's storage-touching file (see map) + their factories
|
||||
(`coin.go`, `gold.go`, … `New(deps)` build typed stores).
|
||||
- Modify: `cmd/server/main.go` `buildProvider` (Mongo|memory only).
|
||||
- Modify: all module `*_test.go` that construct stores (use `MemoryProvider`/`Typed`).
|
||||
- Delete: `kv_store.go`, `kv_provider.go`, `memory_kv.go`, `memory_kv_test.go`,
|
||||
`dynamodb_kv.go`, `dynamodb_provider.go`, `dynamodb_kv_test.go`,
|
||||
`dynamodb_provider_test.go`, `invalid_store.go`.
|
||||
|
||||
## Tests Before / After (per module)
|
||||
|
||||
For each module: update its tests to build a `MemoryProvider`, get
|
||||
`deps.Store = provider.Collection(name)`, then run existing behavior assertions.
|
||||
Run `go test ./internal/modules/<m>/...` green before moving on. Order:
|
||||
misc → stock → wordle → loldle → twentyq → coin → gold → stats → lolschedule
|
||||
(simplest first; CAS + List modules last).
|
||||
|
||||
After all modules:
|
||||
|
||||
```sh
|
||||
go build ./...
|
||||
go test ./internal/modules/...
|
||||
grep -rn "KVStore\|VersionedStore\|PutJSON\|GetJSON\|\.For(" internal/ cmd/ # expect: none (except migrator handled in Phase 4)
|
||||
```
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Add `Deps.Store storage.Collection`; change `registry.Build` to take
|
||||
`storage.Provider` and set `Deps.Store = provider.Collection(name)`.
|
||||
2. Migrate modules one at a time per the order above; keep the suite green.
|
||||
3. For coin/gold, replace the `kv.(VersionedStore)` assertion + byte CAS loop
|
||||
with `DocStore[Portfolio]` Get/PutVersioned (same retry count).
|
||||
4. For lolschedule, introduce the wrapper structs and migrate subscribers (List +
|
||||
Put) and last-push (CAS).
|
||||
5. Update `cmd/server/main.go`: drop the `dynamodb` runtime branch; auto-detect =
|
||||
Mongo when `MONGO_URL` set, else memory (warn). Keep secret-safe logging.
|
||||
6. Delete the dead files listed above; ensure `go build ./...` is clean.
|
||||
7. Run full module + storage suites.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] All modules compile and pass tests using `DocStore[T]`.
|
||||
- [ ] No `KVStore`/`VersionedStore`/`PutJSON`/`GetJSON` references outside the migrator.
|
||||
- [ ] DynamoDB runtime backend deleted; `dynamodb_client.go` retained for migrator.
|
||||
- [ ] coin/gold/lolschedule CAS behavior unchanged (tests prove single-winner).
|
||||
- [ ] lolschedule subscribers/last-push persist as named root fields.
|
||||
- [ ] `cmd/server` builds a Mongo-only runtime (memory fallback for local/no-DB).
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Risk: large blast radius. Mitigation: strict module-by-module order, suite
|
||||
green between each; no behavior changes, only the store type.
|
||||
- Risk: a module silently relied on byte-identical round-trip. Mitigation: typed
|
||||
structs already define the JSON contract; assert decoded values, not bytes.
|
||||
- Risk: stats List semantics differ. Mitigation: `List(prefix)` ported verbatim;
|
||||
reuse stats' existing key prefixes.
|
||||
- Risk: cron Deps scoping regresses. Mitigation: keep registry's per-module Deps
|
||||
cloning; `Deps.Store` is already per-module.
|
||||
@@ -0,0 +1,115 @@
|
||||
---
|
||||
phase: 4
|
||||
title: "DynamoDB→Mongo migrator + docs"
|
||||
status: done
|
||||
priority: P2
|
||||
dependencies: [3]
|
||||
effort: "M"
|
||||
---
|
||||
|
||||
# Phase 4: DynamoDB→Mongo migrator + docs
|
||||
|
||||
## Overview
|
||||
|
||||
Keep `cmd/migrate-dynamo-to-mongo` working, but make it write the new flattened
|
||||
native shape directly. The migrator is schema-agnostic over modules, so it
|
||||
cannot use the typed `DocStore[T]` (it has no compile-time `T` per row). It
|
||||
writes documents with a small generic JSON→root-document encoder plus a tiny
|
||||
explicit table for the only two non-object values.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Migrator still Scans DynamoDB (read-only, `dynamodb:Scan` only) — unchanged.
|
||||
- It writes `{ _id: sk, ...payloadFields, version:1, updatedAt(Date) }` per item.
|
||||
- JSON object values flatten to root.
|
||||
- The two known non-object DynamoDB values are wrapped to match the module's
|
||||
typed shape:
|
||||
- `lolschedule` subscribers (JSON array) → `{ subscribers: [...] }`
|
||||
- `lolschedule` last-push date (JSON/scalar string) → `{ date: "..." }`
|
||||
- Idempotent (re-run overwrites by `_id`, never duplicates).
|
||||
- `--dry-run` (counts only) and `--verify` (per-collection count parity) kept.
|
||||
- No secrets read/printed; `MONGO_URL` stays secret.
|
||||
|
||||
## Architecture
|
||||
|
||||
Add a migrator-local encoder (NOT in `internal/storage`, to keep the runtime
|
||||
typed and the generic JSON path confined to migration):
|
||||
|
||||
```go
|
||||
// cmd/migrate-dynamo-to-mongo/encode.go
|
||||
func rootDocForItem(module, key string, value []byte, now time.Time) (bson.M, error)
|
||||
```
|
||||
|
||||
- Decode `value` with `json.Decoder.UseNumber` (int64 fidelity).
|
||||
- If a `(module,key-prefix)` wrap rule matches → `{ wrapField: decoded }`.
|
||||
- Else if decoded is a JSON object → spread its fields at root (reject keys
|
||||
colliding with `_id`/`version`/`updatedAt`; none expected — fail loud if seen).
|
||||
- Else → error (an unexpected non-object value means a new wrap rule is needed;
|
||||
fail loud rather than guess).
|
||||
- Always set `_id`, `version:int64(1)`, `updatedAt: now`.
|
||||
|
||||
Wrap rules table (explicit, documented):
|
||||
|
||||
```go
|
||||
var wrapRules = []struct{ module, keyPrefix, field string }{
|
||||
{"lolschedule", "subscribers", "subscribers"},
|
||||
{"lolschedule", "last_push_date", "date"},
|
||||
}
|
||||
```
|
||||
|
||||
Write with `Collection(module).ReplaceOne({_id:key}, doc, upsert=true)` directly
|
||||
via the driver (the migrator already holds `*mongo.Database`).
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Modify: `cmd/migrate-dynamo-to-mongo/main.go` — replace `provider.For(pk).Put`
|
||||
with `rootDocForItem` + `ReplaceOne` upsert; validate module/key names inline
|
||||
(the old `For` returned an erroring store for bad names — replicate that guard).
|
||||
- Create: `cmd/migrate-dynamo-to-mongo/encode.go` + `encode_test.go`.
|
||||
- Modify: `cmd/migrate-dynamo-to-mongo/main_test.go` — assert flattened shape +
|
||||
wrapped lolschedule docs.
|
||||
- Modify: `cmd/migrate-dynamo-to-mongo/README.md` — new destination shape.
|
||||
- Modify: `README.md` (storage layout summary), `docs/deploy-coolify-selfhosted.md`
|
||||
(Atlas layout + cutover note), `Makefile` if migrator target wording changed.
|
||||
|
||||
## Tests Before
|
||||
|
||||
1. `TestRootDocForItem_Object` — object JSON → root fields + meta; no `value`.
|
||||
2. `TestRootDocForItem_Int64Fidelity` — large integral number stays int64.
|
||||
3. `TestRootDocForItem_LolscheduleSubscribers` — array → `{subscribers:[...]}`.
|
||||
4. `TestRootDocForItem_LolscheduleLastPush` — date string → `{date:"..."}`.
|
||||
5. `TestRootDocForItem_UnknownScalar_Errors` — bare scalar w/o wrap rule fails loud.
|
||||
6. Migrator e2e (`MONGODB_TEST_URL` + DynamoDB Local): seed DynamoDB rows for a
|
||||
few modules incl. lolschedule array+scalar → migrate → assert raw Mongo docs
|
||||
are flattened/wrapped, no `value`; re-run idempotent; `--verify` counts match;
|
||||
values readable by the module's `DocStore[T]`.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Implement `rootDocForItem` + wrap rules.
|
||||
2. Rewrite `runMigrate` to encode + `ReplaceOne` upsert; keep `--dry-run`.
|
||||
3. Keep `runVerify` (count parity) — unchanged logic.
|
||||
4. Add inline module/key validation (reuse `storage.ValidateKey` if exported, or
|
||||
replicate the rule) so bad rows fail loud.
|
||||
5. Make migrator tests pass.
|
||||
6. Update README/deploy docs to the flattened layout; remove BSON-string/binary
|
||||
destination wording.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Migrator writes flattened native docs (no `value`) for all modules.
|
||||
- [ ] lolschedule array + scalar land as `subscribers`/`date` root fields.
|
||||
- [ ] Migrated docs are read back correctly by the typed module stores.
|
||||
- [ ] Idempotent; `--dry-run` writes nothing; `--verify` exits non-zero on mismatch.
|
||||
- [ ] Unknown non-object value fails loud (no silent guess).
|
||||
- [ ] README + Coolify deploy docs show the flattened layout.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Risk: a module other than lolschedule stores a non-object (missed in survey).
|
||||
Mitigation: encoder fails loud on un-wrapped scalars/arrays; e2e covers the
|
||||
known set; the error names module/key so a new wrap rule is a one-liner.
|
||||
- Risk: re-introducing a generic JSON codec invites runtime reuse. Mitigation:
|
||||
it lives only in `cmd/migrate-dynamo-to-mongo`, not `internal/storage`.
|
||||
- Risk: DynamoDB source already gone at run time. Mitigation: migrator is the
|
||||
cutover step; if DynamoDB is empty/absent the operator skips it (documented).
|
||||
@@ -0,0 +1,89 @@
|
||||
---
|
||||
phase: 5
|
||||
title: "Verification + Mongo-only rollout"
|
||||
status: done
|
||||
priority: P2
|
||||
dependencies: [4]
|
||||
effort: "M"
|
||||
---
|
||||
|
||||
# Phase 5: Verification + Mongo-only rollout
|
||||
|
||||
## Overview
|
||||
|
||||
Full validation and a simple cutover. Because Atlas is empty until cutover and
|
||||
there is no legacy Mongo data, rollout is one-way and clean: migrate from
|
||||
DynamoDB, then run Mongo-only. No in-place Mongo migration, no dual-read.
|
||||
|
||||
## Requirements
|
||||
|
||||
- All test gates pass (`make vet`, `make test`, Mongo integration, migrator e2e).
|
||||
- Raw Mongo docs verified flattened for representative modules.
|
||||
- App behavior unchanged for all Telegram commands.
|
||||
- Cutover + rollback notes recorded.
|
||||
|
||||
## Validation gates
|
||||
|
||||
```sh
|
||||
make vet
|
||||
make test
|
||||
MONGODB_TEST_URL=mongodb://127.0.0.1:27017 go test ./internal/storage ./cmd/migrate-dynamo-to-mongo
|
||||
# migrator e2e additionally needs DynamoDB Local (DYNAMODB_LOCAL_URL)
|
||||
```
|
||||
|
||||
Manual raw-doc inspection (Atlas/Compass or a small test assertion) for:
|
||||
|
||||
- coin/gold/stock portfolio: root fields, `version`, Date `updatedAt`, no `value`.
|
||||
- wordle/loldle/twentyq game + stats: root fields.
|
||||
- lolschedule: `subscribers` array field; last-push `date` field.
|
||||
- stats counters: root fields + List works.
|
||||
|
||||
## Cutover order (operator)
|
||||
|
||||
1. Backup/export current DynamoDB table (source of truth) before cutover.
|
||||
2. Disable EventBridge / stop the old AWS app so DynamoDB writes stop.
|
||||
3. `migrate-dynamo-to-mongo --dry-run` → review per-module counts.
|
||||
4. `migrate-dynamo-to-mongo` → writes flattened native docs to Atlas.
|
||||
5. `migrate-dynamo-to-mongo --verify` → per-collection count parity (exit 0).
|
||||
6. Start the Mongo-only container (`MONGO_URL`+`MONGO_DATABASE`); confirm health.
|
||||
7. Spot-check a few commands (coin balance, wordle state, lolschedule subscribe).
|
||||
|
||||
## Rollback
|
||||
|
||||
- Before step 6 (no Mongo writes yet): re-enable the old AWS app on DynamoDB.
|
||||
DynamoDB is untouched (migrator is Scan-only), so this is safe.
|
||||
- After step 6 (Mongo has taken live writes): forward-fix preferred. To revert to
|
||||
DynamoDB you would need a reverse Mongo→DynamoDB export of anything written
|
||||
after cutover. Keep the DynamoDB backup from step 1 as the floor.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Read: `Makefile`, `README.md`, `docs/deploy-coolify-selfhosted.md`,
|
||||
`docs/aws-decommission-runbook.md`.
|
||||
- Modify: tests only if verification finds gaps.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Run all gates above; fix regressions (do not weaken tests).
|
||||
2. Run the migrator e2e against local Mongo + DynamoDB Local.
|
||||
3. Inspect raw docs per the list; optionally add a small assertion test.
|
||||
4. Record the cutover commands + rollback constraints in the completion report
|
||||
under `plans/reports/`.
|
||||
5. Grep docs for stale `{ _id, value, version, updatedAt }` wording; fix.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] `make vet`, `make test` pass.
|
||||
- [ ] Mongo integration + migrator e2e pass.
|
||||
- [ ] Raw docs verified flattened (no `value`/`_payload`) for representative modules.
|
||||
- [ ] No Telegram command behavior change.
|
||||
- [ ] Cutover + rollback documented in the completion report.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Risk: a module's data didn't round-trip through migration. Mitigation: e2e
|
||||
reads migrated docs back through the typed store; spot-check live commands.
|
||||
- Risk: Atlas M0 limits during migrate. Mitigation: tiny dataset; collection-by-
|
||||
collection counts in `--verify`.
|
||||
- Risk: one-way cutover. Mitigation: DynamoDB backup + Scan-only migrator means
|
||||
pre-write rollback is safe; post-write rollback documented as forward-fix.
|
||||
Loaded 100 of 104 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user