chore(plans): remove completed migration & mapillary plans

This commit is contained in:
tiennm99 committed 2026-05-10 02:33:01 +07:00
1 parent 1e16bd8d42
commit 10cb776ecc
8 files changed
-799

No files matched your search

@@ -1,134 +0,0 @@
---
phase: 1
title: "Adapter Refactor with Prefix"
status: pending
priority: P1
effort: "1h"
dependencies: []
---
# Phase 1: Adapter Refactor with Prefix
## Overview
Replace the node-redis client (`src/lib/redis.js`) with an `@upstash/redis` REST adapter that bundles client + `KEY_PREFIX` into a single handle. Refactor `session.js` and `leaderboard.js` to call helpers that prepend the prefix, mirroring the store-scraper-bot adapter pattern. After this phase, the runtime targets the new Upstash DB and is multi-tenant safe.
## Requirements
- Public API of `session.js` and `leaderboard.js` (function names + arg shapes) **must not change**. API routes must not be modified beyond import paths.
- All physical Redis keys carry `KEY_PREFIX` (default `vngeoguessr:`). Logical keys at call sites stay raw (`session:abc`, `leaderboard:vietnam`, etc.).
- Prefix applied centrally in the adapter — no caller may concatenate the prefix manually.
- Env var naming: accept both `UPSTASH_REDIS_REST_URL`/`UPSTASH_REDIS_REST_TOKEN` and the Vercel Marketplace aliases `KV_REST_API_URL`/`KV_REST_API_TOKEN`. Throw on first call if neither set.
- Sorted-set ordering semantics preserved: scores leaderboard returns highest-first; distance leaderboard returns lowest-first.
- TTL semantics preserved on session keys (30 min via `ex`).
- Followed: JavaScript only, individual parameters (no destructuring in public function signatures).
## Architecture
```
src/lib/
├── upstash.js ← NEW: createUpstashClient + getJson/putJson/del
│ + zAdd/zRange/zScore/zRank/zRevRank/zRemRangeByRank
├── redis.js ← DELETE
├── session.js ← imports getUpstash() handle; uses putJson/getJson/del
└── leaderboard.js ← imports getUpstash() handle; uses sorted-set helpers
```
### Adapter shape
```js
// src/lib/upstash.js
import { Redis } from '@upstash/redis';
const DEFAULT_KEY_PREFIX = 'vngeoguessr:';
let handle = null;
export function getUpstash() {
if (handle) return handle;
const url = process.env.UPSTASH_REDIS_REST_URL ?? process.env.KV_REST_API_URL;
const token = process.env.UPSTASH_REDIS_REST_TOKEN ?? process.env.KV_REST_API_TOKEN;
if (!url) throw new Error('UPSTASH_REDIS_REST_URL or KV_REST_API_URL is required');
if (!token) throw new Error('UPSTASH_REDIS_REST_TOKEN or KV_REST_API_TOKEN is required');
const client = new Redis({ url, token });
const prefix = process.env.KEY_PREFIX ?? DEFAULT_KEY_PREFIX;
handle = { client, prefix };
return handle;
}
function pkey(h, key) { return `${h.prefix}${key}`; }
```
### Helper inventory (all accept handle as first arg)
| Helper | Wraps | Caller |
|---|---|---|
| `getJson(h, key)` | `client.get` (auto-parses) | session read |
| `putJson(h, key, value, ttlSeconds)` | `client.set(..., { ex })` | session write |
| `del(h, key)` | `client.del` | session delete |
| `zAdd(h, key, score, member)` | `client.zadd(key, { score, member })` | leaderboard write |
| `zScore(h, key, member)` | `client.zscore` | leaderboard read |
| `zRangeWithScores(h, key, start, stop, rev)` | `client.zrange(..., { withScores, rev })` | leaderboard read |
| `zRank(h, key, member)` | `client.zrank` | distance rank |
| `zRevRank(h, key, member)` | `client.zrevrank` | score rank |
| `zRemRangeByRank(h, key, start, stop)` | `client.zremrangebyrank` | leaderboard trim |
`@upstash/redis` returns `zrange` with scores as a flat `[member, score, member, score, ...]` array — helper must reshape to `[{ value, score }, ...]` to match what `leaderboard.js` consumes today.
### Caller migration (mechanical)
Each `await getRedis()` becomes `getUpstash()`. Each `redis.method(physicalKey, ...)` becomes `helper(handle, logicalKey, ...)`. Examples:
| Before | After |
|---|---|
| `redis.setEx('session:' + id, 1800, JSON.stringify(data))` | `await putJson(h, 'session:' + id, data, 1800)` |
| `redis.get('session:' + id)` then `JSON.parse` | `await getJson(h, 'session:' + id)` |
| `redis.zAdd(key, { score, value })` | `await zAdd(h, key, score, value)` |
| `redis.zRangeWithScores(key, 0, n-1, { REV: true })` | `await zRangeWithScores(h, key, 0, n-1, true)` |
| `redis.zRemRangeByRank(key, 0, -201)` | `await zRemRangeByRank(h, key, 0, -201)` |
Note: `session.js` currently `JSON.stringify`s the value before `setEx` and `JSON.parse`s after `get`. The new helpers do this internally; remove the manual stringify/parse at call sites.
## Related Code Files
- Create: `src/lib/upstash.js`
- Modify: `src/lib/session.js` (swap to handle + helpers, drop manual JSON marshal)
- Modify: `src/lib/leaderboard.js` (swap to handle + sorted-set helpers; reshape `zrange` results)
- Delete: `src/lib/redis.js`
- Modify: `package.json` — add `@upstash/redis`, remove `redis`. **Highlight to user for manual install** per project's `File Modification Policy` (configuration changes).
## Implementation Steps
1. Add `@upstash/redis` dep, drop `redis` dep.
- User runs: `npm install @upstash/redis && npm uninstall redis`
- Agent must NOT auto-modify `package.json` per project rules; flag the install command in the final report.
2. Write `src/lib/upstash.js` with the singleton `getUpstash()` and all helpers listed above. Verify the `zRangeWithScores` reshape matches what `leaderboard.js` expects (`[{ value, score }]`).
3. Rewrite `src/lib/session.js` to use `getUpstash()` + `putJson`/`getJson`/`del`. Keep function signatures identical (`storeGameSession(sessionId, sessionData)`, etc.).
4. Rewrite `src/lib/leaderboard.js` to use `getUpstash()` + sorted-set helpers. Keep `getLeaderboard(cityCode, limit, type)`, `submitScore(username, score, cityCode)`, `submitDistanceRecord(username, distance, cityCode)` signatures intact.
5. Delete `src/lib/redis.js`.
6. Run `npm run lint` and `npm run build` — no compile errors, no `from 'redis'` references remain.
7. Local sanity (user-driven, optional): `npm run dev`, hit `/api/leaderboard` (GET) — should return empty leaderboard from new DB without throwing.
## Success Criteria
- [ ] `src/lib/upstash.js` exports `getUpstash` + the helper inventory above
- [ ] `getUpstash()` reads `KEY_PREFIX` (default `vngeoguessr:`) and accepts both env-var naming conventions
- [ ] `src/lib/session.js` and `src/lib/leaderboard.js` import only from `./upstash.js`
- [ ] `src/lib/redis.js` deleted; `grep -r "from './redis" src/` empty
- [ ] `grep -r "REDIS_URL\|node:redis\|from 'redis'" src/` empty
- [ ] `npm run build` passes
- [ ] API route files (`api/guess/route.js`, `api/cron/route.js`, `api/leaderboard/route.js`) untouched
## Risk Assessment
- **Risk:** `@upstash/redis` `zrange` shape with `withScores: true` differs from `node-redis` `zRangeWithScores`. **Mitigation:** Helper normalizes to `[{ value, score }]`; verify with one log line during dev smoke before proceeding to Phase 2.
- **Risk:** `@upstash/redis` auto-parses JSON only when value was set as an object via the SDK. Mixing manual `JSON.stringify` with later `client.get` may double-encode. **Mitigation:** centralize encode/decode in `putJson`/`getJson` — pass raw objects in, receive raw objects out. No caller does its own marshal.
- **Risk:** Caller forgets to `await getUpstash()` — but it's synchronous (no connection step in REST SDK). **Mitigation:** helper is sync; no chance of unawaited promise here.
- **Risk:** Hardcoded prefix drift between adapter and migration script (Phase 2). **Mitigation:** both read `process.env.KEY_PREFIX ?? 'vngeoguessr:'`. Document the contract in `plan.md`.
- **Risk:** `next build` triggers env-var lookup on import — `getUpstash()` must be lazy. **Mitigation:** call from request handlers, not module top-level. Pattern matches existing `getRedis()`.
## Out of Scope
- Migrating data from old DB → new DB (Phase 2).
- Removing old DB env vars from Vercel (Phase 3, user-driven).
- Updating `docs/tech-stack.md` (Phase 3).
@@ -1,104 +0,0 @@
---
phase: 2
title: "Data Migration old → new Upstash"
status: pending
priority: P1
effort: "30 min"
dependencies: [1]
---
# Phase 2: Data Migration old → new Upstash
## Overview
One-shot script that copies leaderboard sorted-set data from the OLD Upstash DB to the NEW Upstash DB, prepending `KEY_PREFIX` (default `vngeoguessr:`) on write. Sessions are not migrated — they are 30-min ephemeral and any in-flight games will resubmit cleanly post-cutover. The OLD DB is read-only during this phase.
## Requirements
- Migrate exactly these logical keys (sorted sets):
- `leaderboard:vietnam`
- `leaderboard:city:hanoi`, `leaderboard:city:danang`, `leaderboard:city:hcm`, `leaderboard:city:dalat`, `leaderboard:city:duchoa` (and any other city codes present)
- `distance:vietnam`
- `distance:city:*` (same set of city codes)
- Discover keys via `SCAN MATCH 'leaderboard:*'` and `SCAN MATCH 'distance:*'` against the OLD DB rather than hardcoding city codes — script stays correct if a city is added later.
- Write to NEW DB with prefix prepended: `vngeoguessr:leaderboard:vietnam` etc.
- Idempotent: re-runs produce the same final state. Use `DEL <prefix>+key` on the destination before re-populating, or rely on `ZADD` overwriting score-by-member (safe).
- `--dry-run` flag: log counts and sample keys, no writes.
- Log effective `KEY_PREFIX` and per-key cardinality so operator can verify alignment with runtime.
## Architecture
```
scripts/migrate-upstash.js ← NEW
Reads: OLD_UPSTASH_REDIS_REST_URL / OLD_UPSTASH_REDIS_REST_TOKEN
Writes: UPSTASH_REDIS_REST_URL / UPSTASH_REDIS_REST_TOKEN (NEW DB, same as runtime)
PREFIX = process.env.KEY_PREFIX ?? 'vngeoguessr:'
steps:
1. SCAN old DB for keys matching 'leaderboard:*' and 'distance:*'
2. For each key:
members = oldClient.zrange(key, 0, -1, { withScores: true })
newClient.zadd(PREFIX + key, ...members) // multi-add
3. Log: { source_keys: N, total_members: M, prefix: 'vngeoguessr:' }
4. Print sample physical keys written so operator can spot-check the dashboard
```
Single Node.js file, no test framework needed. Run with `node --env-file=.env.migrate scripts/migrate-upstash.js`.
### Source/destination protocols (confirmed via `vercel env ls`)
- OLD DB → TCP only, env var `OLD_STORAGE_REDIS_URL` → script uses `redis` (node-redis)
- NEW DB → REST + TCP; env vars `KV_REST_API_URL` + `KV_REST_API_TOKEN` → script uses `@upstash/redis`
### `.env.migrate` shape (gitignored; populated via `vercel env pull .env.migrate`)
```
OLD_STORAGE_REDIS_URL=redis://... # OLD DB (source, TCP)
KV_REST_API_URL=https://... # NEW DB (dest, REST)
KV_REST_API_TOKEN=...
KEY_PREFIX=vngeoguessr: # optional, defaults to vngeoguessr:
```
## Related Code Files
- Create: `scripts/migrate-upstash.js`
- Modify: `.gitignore` — ensure `.env.migrate` is ignored. **Highlight to user** if `.gitignore` already covers `.env*`.
- No `package.json` script entry needed (one-shot; user invokes directly).
## Implementation Steps
1. Write `scripts/migrate-upstash.js`:
- Two `Redis` instances (`oldClient`, `newClient`) using `@upstash/redis`.
- `parseArgs` for `--dry-run`.
- `scanAll(client, match)` helper that paginates `client.scan(cursor, { match, count: 200 })` until cursor returns `0`.
- For each key: `members = await oldClient.zrange(key, 0, -1, { withScores: true })`. The SDK returns `[member, score, member, score, ...]`; reshape into the form `zadd` accepts.
- Multi-add: `newClient.zadd(physicalKey, ...members.map(m => ({ score: m.score, member: m.value })))`.
- Skip empty sets.
- In `--dry-run` mode, do not call `zadd`; print what would be written.
2. Verify `.env.migrate` is gitignored (`grep -F '.env' .gitignore`). If not covered, append `.env.migrate` to `.gitignore` and flag for user.
3. Local run: `node --env-file=.env.migrate scripts/migrate-upstash.js --dry-run`. Verify counts match expected (compare to old DB Upstash dashboard).
4. Real run: `node --env-file=.env.migrate scripts/migrate-upstash.js`. Verify NEW DB dashboard shows prefixed keys with matching cardinality.
5. Read-back compare: pick one user from `leaderboard:vietnam`, confirm same score in NEW DB (`vngeoguessr:leaderboard:vietnam`).
## Success Criteria
- [ ] `scripts/migrate-upstash.js` exists; runs cleanly with `--dry-run` and without
- [ ] Effective `KEY_PREFIX` printed at start of run (operator-visible alignment check)
- [ ] All `leaderboard:*` and `distance:*` source keys discovered via SCAN (no hardcoded city list)
- [ ] Per-key member count in NEW DB equals OLD DB
- [ ] Physical keys in NEW DB visibly carry the `vngeoguessr:` prefix
- [ ] Re-run is idempotent: `ZCARD` values unchanged on second run
- [ ] `.env.migrate` is gitignored; not committed
## Risk Assessment
- **Risk:** Old DB is the node-redis-protocol flavor and has no REST creds. **Mitigation:** if OLD DB lacks REST endpoint, use `redis` npm package (still installed pre-Phase 1 cleanup) for the read side, REST SDK for write side. Confirm during user data-collection step before writing the script.
- **Risk:** `zrange` cursor / `withScores` SDK shape mismatch. **Mitigation:** log first key's payload before the loop; abort on shape mismatch with a clear error.
- **Risk:** Race — players keep submitting to OLD DB while migration runs (writes get lost in NEW DB). **Mitigation:** Phase 3 cutover swaps env vars in one Vercel redeploy; migration runs at low-traffic time. Acceptable: at most a handful of new entries lost; users can re-play.
- **Risk:** Prefix drift between script and runtime (script uses `vngeoguessr:`, runtime uses `vngeoguessr-prod:` etc.). **Mitigation:** script logs prefix; Phase 3 success criteria includes `/api/leaderboard` returning a non-empty list post-cutover (end-to-end alignment check).
- **Risk:** Sessions in flight at cutover see stale state. **Mitigation:** sessions are 30-min TTL and not migrated; affected players just resubmit. Documented expectation, not a defect.
## Out of Scope
- Migrating `session:*` keys (intentionally skipped — ephemeral).
- Updating Vercel env vars or removing the OLD DB integration (Phase 3, user-driven).
@@ -1,95 +0,0 @@
---
phase: 3
title: "Cutover and Cleanup"
status: pending
priority: P1
effort: "30 min"
dependencies: [1, 2]
---
# Phase 3: Cutover and Cleanup
## Overview
Deploy the refactored app pointed at the NEW Upstash DB, validate end-to-end, then remove the OLD DB integration and stale dependencies. After this phase, the project depends only on the new shared Upstash DB and the `redis` npm package is gone.
## Requirements
- Production reads/writes hit the NEW DB (with prefix) — verified via real user-flow.
- OLD DB integration disconnected from the Vercel project (user-driven).
- `redis` npm dep removed from `package.json` and lockfile.
- Documentation in `docs/tech-stack.md` reflects the new SDK and prefix model.
## Architecture
No code architecture changes here — this phase is operations + documentation. Sequence:
```
1. Deploy preview with refactor (Phase 1) → smoke test against NEW DB
2. Promote preview → production
3. Validate production:
POST /api/guess (full game flow) → leaderboard updates in NEW DB
GET /api/leaderboard → returns migrated entries
4. Disconnect OLD Upstash integration in Vercel dashboard (user)
5. Remove `redis` dep, run `npm install`, commit lockfile
6. Update docs/tech-stack.md
```
## Related Code Files
- Modify: `package.json` — remove `"redis"` dependency. **Highlight to user** per File Modification Policy.
- Modify: `package-lock.json` — regenerated by `npm install` (user runs).
- Modify: `docs/tech-stack.md` — replace "Redis: Complete session and leaderboard storage" + node-redis line with `@upstash/redis` REST SDK + `KEY_PREFIX` multi-tenant note.
- No source-code changes in this phase.
## Implementation Steps
1. **Deploy preview** (user runs `vercel` or pushes to a preview branch). Confirm preview reads from NEW DB.
2. **Smoke test on preview:**
- GET `/api/leaderboard?type=score` → should list migrated users.
- Play one full game → POST `/api/guess` → leaderboard rank updates.
- Optional: check Upstash dashboard for new key like `vngeoguessr:leaderboard:vietnam` with the new score reflected.
3. **Promote to production** (`vercel --prod` or merge to `main`).
4. **Production smoke test** (same as step 2, on the production URL).
5. **Disconnect OLD Upstash integration** — user action in Vercel dashboard:
- Project → Storage → remove the OLD Upstash integration.
- Remove `REDIS_URL` env var if it lingers.
6. **Remove `redis` dep:**
- User runs: `npm uninstall redis`
- Verify `npm ls redis` is empty and `package-lock.json` no longer references `node-redis`.
7. **Update `docs/tech-stack.md`:** replace lines about node-redis with `@upstash/redis` REST SDK; add a sentence noting the `KEY_PREFIX` multi-tenant convention.
8. **Final grep sweep:**
- `grep -r "REDIS_URL\|from 'redis'\|node:redis" src/ scripts/` → empty
- `grep -r "vngeoguessr:" src/` → only in `src/lib/upstash.js` (the default constant)
## Success Criteria
- [ ] Production GET `/api/leaderboard` returns migrated entries from NEW DB
- [ ] Production POST `/api/guess` increments score in NEW DB (verified in Upstash dashboard)
- [ ] OLD Upstash integration removed from Vercel project (Storage tab clean)
- [ ] `redis` dep removed from `package.json`; `npm ls redis` empty
- [ ] `docs/tech-stack.md` updated to reflect `@upstash/redis` + `KEY_PREFIX`
- [ ] Final grep sweep clean (no `REDIS_URL` / `from 'redis'` references)
- [ ] `npm run build` passes on a fresh checkout
## Risk Assessment
- **Risk:** Production deploy reads from NEW DB before migration script ran ⇒ empty leaderboard for users. **Mitigation:** Phase 2 must complete and be verified before promoting Phase 3 to production. The plan ordering enforces this.
- **Risk:** OLD DB integration removed before production cutover settles ⇒ rollback impossible. **Mitigation:** keep OLD DB connected for at least one full traffic cycle (e.g., 24h) post-cutover before disconnecting.
- **Risk:** `KEY_PREFIX` env var missing in Vercel production ⇒ runtime falls back to default `vngeoguessr:`. This is fine (matches migration default), but if a different prefix was set at runtime, leaderboards appear empty. **Mitigation:** explicitly set `KEY_PREFIX=vngeoguessr:` in Vercel env vars during cutover, even though it's the default.
- **Risk:** `docs/tech-stack.md` drift (forgotten update). **Mitigation:** included in success criteria checklist; reviewer will catch.
## Out of Scope
- Adding more projects to the shared Upstash DB (the prefix system enables it; actual onboarding of other projects is separate work).
- Adopting `@upstash/redis` for any feature beyond leaderboard + session.
- Refactoring API route files — they remain untouched throughout.
## Rollback Plan
If production smoke fails after promotion:
1. In Vercel dashboard, redeploy the previous commit (one click).
2. Old commit still uses node-redis + `REDIS_URL`; OLD DB still connected → instant rollback.
3. Investigate failure offline, redo Phase 1 / 2 steps as needed before retrying.
This is why steps 5–7 (cleanup) only happen AFTER production has been stable for a full traffic cycle.
@@ -1,73 +0,0 @@
---
title: "Migrate Upstash DB with Project Key Prefix"
description: "Switch vngeoguessr from node-redis pointed at the old Upstash DB to @upstash/redis REST SDK pointed at the new Vercel-Marketplace Upstash DB. Introduce KEY_PREFIX adapter pattern (default vngeoguessr:) so the new DB can be safely shared with other Vercel projects (mirrors store-scraper-bot)."
status: pending
priority: P1
effort: 2h
branch: main
tags: [vercel, upstash, migration, redis, multi-tenant]
created: 2026-05-09
blockedBy: []
blocks: []
---
# Migrate Upstash DB with Project Key Prefix
## Goal
Cut vngeoguessr over from old Upstash DB → new Upstash DB while introducing a project-scoped key prefix. After this plan, the same Upstash DB can host data for multiple Vercel projects without key collisions.
## Current State
- Connection: `src/lib/redis.js` uses `redis@5.8.0` (TCP) via `process.env.REDIS_URL` — a single global client.
- Both old and new Upstash DBs are already linked to the Vercel project (per user); env vars exist for both. New DB connection must come via the Vercel Marketplace Upstash integration (`KV_REST_API_URL` / `KV_REST_API_TOKEN` or `UPSTASH_REDIS_REST_URL` / `UPSTASH_REDIS_REST_TOKEN`).
- Logical key namespaces in use:
- `session:{sessionId}` — string, TTL 30 min (`src/lib/session.js`)
- `leaderboard:vietnam`, `leaderboard:city:{cityCode}` — sorted sets (`src/lib/leaderboard.js`)
- `distance:vietnam`, `distance:city:{cityCode}` — sorted sets (`src/lib/leaderboard.js`)
- Files touching Redis: `src/lib/redis.js`, `src/lib/session.js`, `src/lib/leaderboard.js`, `src/app/api/guess/route.js`, `src/app/api/cron/route.js`, `src/app/api/leaderboard/route.js`.
## Target State
- Connection: `@upstash/redis` REST SDK; client + prefix bundled in a single handle returned by `createUpstashClient(env)`.
- Default `KEY_PREFIX = 'vngeoguessr:'`. All physical keys carry it; logical keys at call sites stay unchanged.
- No `process.env.REDIS_URL` references remain. `redis` npm dep removed.
- One-shot migration script copies sorted-set leaderboard data from old DB → new DB, prepending the prefix on write. Sessions are not migrated (30-min TTL, ephemeral).
## Key Decisions
- **REST SDK over node-redis.** Vercel Marketplace Upstash exposes REST creds; `@upstash/redis` is fluid-compute friendly (no socket pooling), matches the store-scraper-bot reference. Sorted-set commands (`zadd`, `zrange`, `zscore`, `zrank`, `zrevrank`, `zremrangebyrank`) are all supported.
- **Adapter shape mirrors store-scraper-bot.** `createUpstashClient(env) → { client, prefix }`. Helper functions accept the handle and prepend the prefix. Repositories stay prefix-unaware. Difference vs store-scraper-bot: we expose sorted-set helpers (`zAdd`, `zRange`, `zScore`, `zRank`, `zRevRank`, `zRemRangeByRank`) in addition to JSON `get/set/del`.
- **Env var naming.** Adapter accepts both `UPSTASH_REDIS_REST_URL` / `UPSTASH_REDIS_REST_TOKEN` and `KV_REST_API_URL` / `KV_REST_API_TOKEN` (Vercel Marketplace alias) — same as store-scraper-bot.
- **Migration scope.** Copy leaderboard sorted sets only. Sessions are not migrated. Old DB stays online briefly as fallback during cutover, then gets disconnected from Vercel by the user.
- **Prefix value.** `vngeoguessr:`. Single source of truth; only override if explicitly multi-tenant beyond this project.
## Phases
| # | Phase | Effort | Status |
|---|---|---|---|
| 01 | [Adapter Refactor with Prefix](./phase-01-adapter-refactor-with-prefix.md) | 1h | pending |
| 02 | [Data Migration old → new Upstash](./phase-02-data-migration.md) | 30 min | pending |
| 03 | [Cutover and Cleanup](./phase-03-cutover-and-cleanup.md) | 30 min | pending |
## Success Criteria (overall)
- App reads/writes against the new Upstash DB via REST SDK.
- All physical keys in new DB carry the `vngeoguessr:` prefix.
- Leaderboard counts (per sorted set, per city) match between old and new DB after migration.
- `npm ls redis` returns nothing; `package.json` lists `@upstash/redis` instead.
- `grep -r "REDIS_URL\|node:redis\|from 'redis'" src/` returns zero hits.
- Production smoke: submit a guess, view leaderboard → both succeed against the new DB.
## Manual Steps Required (User)
These cannot be performed by the agent and must be done by the user:
- Confirm the new Upstash DB env vars present in Vercel project (preview + production).
- Set `KEY_PREFIX=vngeoguessr:` in Vercel env (preview + production). Optional but recommended to make the prefix explicit; default also works.
- Provide the OLD DB REST URL+token to a local `.env.migrate` (read-only, for one-shot migration script).
- After cutover validated, disconnect the old Upstash DB from the Vercel project.
## Dependencies
- `@upstash/redis` npm package (added in Phase 1).
- New Upstash DB already linked via Vercel Marketplace (per user — confirmed pre-plan).
@@ -1,176 +0,0 @@
---
phase: 1
title: "Per-city Delta + Random-Point Fetch"
status: pending
priority: P1
effort: "30m"
dependencies: []
---
# Phase 1: Per-city Delta + Random-Point Fetch
## Overview
Replace the adaptive bbox loop in `src/lib/mapillary.js` with viguessr's random-point dart-throw using a per-city fixed `delta`. Add `mapillaryDelta` to each `CITIES` entry. Update `new-game` route to pass delta and use `thumb_original_url`.
## Related Code Files
- Modify: `src/lib/game.js` — add `mapillaryDelta` field on each `CITIES` entry; export `cityDeltas`
- Rewrite: `src/lib/mapillary.js` — random-point + fixed-delta + retry-position
- Modify: `src/app/api/new-game/route.js` — import `cityDeltas`, pass to fetch, switch back to `thumb_original_url`
## Implementation Steps
### 1. `src/lib/game.js`
Inside each `CITIES.<code>` object, add a numeric field `mapillaryDelta`:
```js
HN: { ..., mapillaryDelta: 0.003 },
TPHCM: { ..., mapillaryDelta: 0.005 },
DL: { ..., mapillaryDelta: 0.005 },
DH: { ..., mapillaryDelta: 0.005 },
DN: { ..., mapillaryDelta: 0.005 }, // disabled but populated
```
Below the existing `cityBboxes` export, add:
```js
export const cityDeltas = Object.fromEntries(
Object.values(CITIES).map(city => [city.code, city.mapillaryDelta])
);
```
### 2. `src/lib/mapillary.js` (full rewrite)
```js
// Mapillary API utilities
//
// Strategy ported from viguessr (https://github.com/luuvanduc1999/viguessr):
// pick a random point inside the city bbox, query a small fixed-size sub-bbox
// (side = 2*delta) centered on that point, and re-roll the point on empty
// results or transient 5xx. The fixed small size keeps Mapillary query cost
// below their cap; the dart-throw eventually lands on a pano-rich window.
const MAX_RETRIES = 10;
export async function fetchMapillaryImages(bbox, delta) {
const accessToken = process.env.MAPILLARY_ACCESS_TOKEN;
if (!accessToken) {
throw new Error('MAPILLARY_ACCESS_TOKEN environment variable is not set');
}
const [minLng, minLat, maxLng, maxLat] = bbox;
let lastError = null;
for (let attempt = 1; attempt <= MAX_RETRIES; attempt++) {
const lat = Math.random() * (maxLat - minLat) + minLat;
const lng = Math.random() * (maxLng - minLng) + minLng;
const queryBbox = [
(lng - delta).toFixed(4),
(lat - delta).toFixed(4),
(lng + delta).toFixed(4),
(lat + delta).toFixed(4),
].join(',');
const apiUrl = `https://graph.mapillary.com/images?access_token=${accessToken}&fields=id,thumb_original_url,geometry,is_pano&limit=3&bbox=${queryBbox}&is_pano=true`;
try {
const response = await fetch(apiUrl, {
headers: { Accept: 'application/json' },
next: { revalidate: 0 },
});
if (response.status === 401) {
throw new Error('Mapillary authentication failed');
}
if (!response.ok) {
const body = await response.text().catch(() => '<unreadable>');
lastError = `${response.status}: ${body.slice(0, 200)}`;
console.error(`Attempt ${attempt}/${MAX_RETRIES}: Mapillary ${response.status}; re-rolling`);
continue;
}
const data = await response.json();
const images = data.data || [];
const panos = images.filter(img => img.is_pano);
if (panos.length > 0) {
console.log(`Found ${panos.length} panos on attempt ${attempt}`);
return { success: true, data: panos };
}
console.log(`Attempt ${attempt}/${MAX_RETRIES}: no panos; re-rolling`);
} catch (error) {
if (error.message === 'Mapillary authentication failed') {
throw error;
}
lastError = error.message;
console.error(`Attempt ${attempt}/${MAX_RETRIES} threw: ${error.message}; re-rolling`);
}
}
return {
success: false,
error: lastError
? `No panos after ${MAX_RETRIES} attempts (last: ${lastError})`
: 'No panos found',
};
}
```
Removed:
- `MAX_BBOX_AREA` / `MIN_BBOX_AREA` / `GROW_FACTOR` / `SHRINK_FACTOR`
- `getRandomSubBbox` helper (replaced inline)
- Adaptive shrink/grow loop
### 3. `src/app/api/new-game/route.js`
Update imports + call:
```js
// Before:
import { cityNames, cityBboxes } from '../../../lib/game.js';
const bbox = cityBboxes[cityCode];
const imageResult = await fetchMapillaryImages(bbox);
const imageUrl = selectedImage.thumb_2048_url;
// After:
import { cityNames, cityBboxes, cityDeltas } from '../../../lib/game.js';
const bbox = cityBboxes[cityCode];
const delta = cityDeltas[cityCode];
const imageResult = await fetchMapillaryImages(bbox, delta);
const imageUrl = selectedImage.thumb_original_url;
```
Add a guard for missing delta:
```js
if (!delta) {
return NextResponse.json({
success: false,
error: `No Mapillary delta configured for city: ${cityCode}`,
}, { status: 400 });
}
```
## Success Criteria
- [ ] `npm run build` passes
- [ ] HN: 3 sample requests → ≥2 success
- [ ] TPHCM: 3 sample requests → ≥2 success
- [ ] DL: 3 sample requests → ≥2 success
- [ ] DH: 3 sample requests → ≥2 success
- [ ] No 'reduce data' 500s in production logs after deploy
- [ ] `grep "MAX_BBOX_AREA\|GROW_FACTOR\|SHRINK_FACTOR" src/` empty
## Risk Assessment
- **Risk**: viguessr's HN delta=0.003 (666m) might still occasionally hit 'reduce data' if Mapillary tightens further. **Mitigation**: 10 retries; if persistent, drop HN delta to 0.002.
- **Risk**: Per-city delta requires a value for each enabled city; missing entry crashes. **Mitigation**: explicit guard in route; default fallback to 0.005 if undefined.
- **Risk**: `thumb_original_url` may be deprecated. **Mitigation**: viguessr is shipping it in production today; if Mapillary deprecates, swap to `thumb_2048_url`.
- **Risk**: Rare cities with sparse pano coverage (DH 666m windows) might exhaust retries. **Mitigation**: keep delta=0.005 (1.1km) for non-HN cities.
## Rollback
Single commit; revert HEAD to restore current adaptive code. Mapillary deploys with no schema/state changes, so rollback is instant.
@@ -1,94 +0,0 @@
---
title: "Port viguessr Mapillary strategy"
description: "Replace adaptive shrink/grow bbox loop with viguessr's proven approach: per-city fixed `delta` (half-bbox-side) + random-point dart-throw + simple retry-with-new-point. Fixes 'reduce data' 500s in HN/HCM dense areas."
status: pending
priority: P1
effort: 30m
created: 2026-05-10
source: https://github.com/luuvanduc1999/viguessr (assets/js/game.js, obfuscated)
blockedBy: []
blocks: []
---
# Port viguessr Mapillary strategy
## Why
Current `src/lib/mapillary.js` adaptive bbox sizing fails consistently in dense HN/HCM — Mapillary returns 500 'reduce data' when query bbox costs too much to scan. Adaptive grow on empty results worsens this.
viguessr (a sibling vngeoguessr project) ships a working implementation for the same cities. Their approach: **per-city fixed `delta`**, query a small bbox centered on a random point, retry only by re-rolling the random point (never resizing the bbox). The small fixed bbox keeps Mapillary query cost under their cap; the dart-throw eventually finds pano-rich windows.
## Source decoded (from obfuscated game.js)
```js
boundingBoxVN = {
'HN': [105.77, 20.96, 105.88, 21.05, delta=0.003],
'TPHCM': [106.62, 10.71, 106.75, 10.83, delta=0.005],
'HP': [106.65, 20.80, 106.75, 20.90, delta=0.05],
'ND': [106.00, 20.35, 106.25, 20.50, delta=0.005],
'DN': [108.17, 16.00, 108.25, 16.10, delta=0.005],
};
async function getRandomMapillaryImage() {
const lat = Math.random() * (maxLat - minLat) + minLat;
const lng = Math.random() * (maxLong - minLong) + minLong;
const bbox = [lng-delta, lat-delta, lng+delta, lat+delta].map(n => n.toFixed(4)).join(',');
const url = `https://graph.mapillary.com/images?access_token=${TOKEN}&fields=id,thumb_original_url,geometry,is_pano&limit=3&bbox=${bbox}&is_pano=true`;
const json = await (await fetch(url)).json();
if (json.data?.length) {
const panos = json.data.filter(d => d.is_pano);
return panos.length ? panos[rand(panos.length)] : json.data[0];
}
return getRandomMapillaryImage(); // re-roll, same delta
}
```
## What changes locally
### `src/lib/game.js` (CITIES schema)
Add `mapillaryDelta` field to each enabled `CITIES` entry:
| Code | New `mapillaryDelta` | Source rationale |
|---|---|---|
| HN | `0.003` | viguessr value |
| TPHCM | `0.005` | viguessr value |
| DL | `0.005` | not in viguessr; use TPHCM-class default |
| DH | `0.005` | not in viguessr; use TPHCM-class default |
| DN | `0.005` (still disabled) | viguessr value, future-proof |
Export a `cityDeltas` map alongside existing `cityBboxes`/`cityNames`/`cityCenters`.
### `src/lib/mapillary.js` (rewrite)
Replace adaptive loop with viguessr-style: random point inside city bbox, fixed-size sub-bbox of side `2*delta`, re-roll position on empty/error.
Signature change: `fetchMapillaryImages(bbox, delta)`.
### `src/app/api/new-game/route.js`
- Import `cityDeltas` from `lib/game.js`.
- Pass `cityDeltas[cityCode]` to `fetchMapillaryImages`.
- Use `selectedImage.thumb_original_url` (revert from `thumb_2048_url`).
## Phases
| # | Phase | Effort | Status |
|---|---|---|---|
| 01 | [Per-city Delta + Random-Point Fetch](./phase-01-per-city-delta-random-point-fetch.md) | 30m | pending |
## Success Criteria
- HN, TPHCM, DL, DH all return panoramic image on first request (sample ≥ 3 attempts each).
- No `MAX_BBOX_AREA` / `MIN_BBOX_AREA` / shrink/grow logic remains in `src/lib/mapillary.js`.
- `npm run build` passes.
## Out of scope
- Token rotation or rate-limit handling
- Caching successful responses
- Adding more cities (DN remains disabled, future work)
## Manual cleanup after success
- Remove the diagnostic `console.error` of Mapillary URL/body once confirmed stable (or keep as observability).
@@ -1,63 +0,0 @@
---
phase: 1
title: "Cleanup"
status: pending
priority: P2
effort: "15m"
dependencies: []
---
# Phase 1: Cleanup
## Overview
Single-pass cleanup of Upstash migration scaffolding. No code logic changes, only file/dependency removal and plan-status update.
## Related Code Files
- Modify: `package.json` (remove `redis` dep) — **highlight to user** per File Modification Policy
- Modify: `package-lock.json` (regenerated by `npm uninstall`) — **highlight to user**
- Delete: `scripts/migrate-upstash.js`
- Delete: `.env.migrate` (local only, gitignored)
- Update: `plans/260509-2234-upstash-db-migration-with-prefix/phase-03-cutover-and-cleanup.md` (mark complete via `ck plan check 3`)
## Implementation Steps
1. **User runs**: `npm uninstall redis`
- Verify: `npm ls redis` returns empty.
- This regenerates `package-lock.json`.
2. **Agent deletes**: `scripts/migrate-upstash.js`.
- Recoverable from git history (commit `53ee795`) if ever needed.
- The directory `scripts/` will be left empty; remove the directory too.
3. **Agent deletes**: local `.env.migrate`.
- Contains old DB credentials no longer needed; sensitive even if gitignored.
4. **Agent runs**: `cd plans/260509-2234-upstash-db-migration-with-prefix && ck plan check 3` to mark Phase 3 of the migration plan complete.
5. **Agent runs**: `npm run build` to confirm nothing else depends on `redis`.
6. **Agent commits + pushes** the deletion changes (single commit).
## What stays untouched
- Vercel env vars `REDIS_URL`, `KV_URL`, `KV_REST_API_READ_ONLY_TOKEN` — auto-provisioned by the Vercel Marketplace Upstash integration. Removing causes drift.
- `src/lib/mapillary.js` console.error lines — kept-on-purpose error observability, not temporary instrumentation.
- The migration plan files themselves (kept as record).
## Success Criteria
- [ ] `npm ls redis` empty
- [ ] `scripts/migrate-upstash.js` not present
- [ ] `.env.migrate` not present locally
- [ ] `npm run build` passes
- [ ] Original migration plan Phase 3 marked completed
- [ ] Single commit pushed to main
## Risk Assessment
- **Risk**: deleting the migration script loses the bridge if a re-migration becomes necessary. **Mitigation**: git history preserves it (commit `53ee795`). Restoration is `git checkout 53ee795 -- scripts/migrate-upstash.js`.
- **Risk**: removing `redis` dep breaks something we missed. **Mitigation**: `npm run build` step after uninstall surfaces any stray imports. Grep already confirmed no `from 'redis'` or `require('redis')` references in `src/`.
- **Risk**: `.env.migrate` deletion is irreversible. **Mitigation**: file just contains creds copy-pastable from Vercel dashboard if ever needed; no original-source data.
## Out of Scope
- Removing Vercel auto-provisioned env vars.
- Refactoring observability in `mapillary.js`.
- Any change to the new Upstash integration or its credentials.
@@ -1,60 +0,0 @@
---
title: "Cleanup Upstash migration artifacts"
description: "Remove leftover artifacts from the completed Upstash migration: legacy redis npm dep, one-shot migration script, local .env.migrate, and close out the original migration plan's Phase 3."
status: pending
priority: P2
effort: 15m
created: 2026-05-10
related: 260509-2234-upstash-db-migration-with-prefix (closes Phase 3)
blockedBy: []
blocks: []
---
# Cleanup Upstash migration artifacts
## Goal
The Upstash migration is complete and verified in production (15 leaderboard users, 425 sorted-set members migrated, all 4 cities returning panoramas). Old Upstash integration is disconnected from Vercel. This plan retires the migration scaffolding.
## Current state (verified)
| Artifact | State |
|---|---|
| `redis` npm dep | still in `package.json` ^5.8.0 (last needed by migration script) |
| `scripts/migrate-upstash.js` | committed; one-shot job done |
| `.env.migrate` | exists locally (gitignored) with old + new DB credentials |
| `OLD_STORAGE_REDIS_URL` env in Vercel | already removed by user |
| `REDIS_URL` env in Vercel | auto-provisioned by Marketplace integration; unused by app |
| `KV_URL` env in Vercel | auto-provisioned; unused |
| `KV_REST_API_READ_ONLY_TOKEN` env in Vercel | auto-provisioned; unused |
| `KV_REST_API_URL` / `KV_REST_API_TOKEN` env in Vercel | **in active use** — keep |
| Mapillary diagnostic logging | already removed (viguessr rewrite replaced it; current logs are kept-on-purpose error observability) |
| Plan `260509-2234-upstash-db-migration-with-prefix` Phase 3 | marked in-progress in tracker; needs close |
## What this plan does
| Action | Reason |
|---|---|
| `npm uninstall redis` | App uses `@upstash/redis` exclusively; node-redis no longer needed |
| Delete `scripts/migrate-upstash.js` | One-shot done; recoverable from git history if ever needed |
| Delete local `.env.migrate` | Contains old DB credentials no longer in use; sensitive even if gitignored |
| Mark migration plan Phase 3 complete | Operational status accuracy |
## What this plan does NOT do
- **Don't touch `REDIS_URL`, `KV_URL`, `KV_REST_API_READ_ONLY_TOKEN`** — they're auto-provisioned by the Vercel Marketplace Upstash integration. Removing them via CLI causes drift; integration may re-sync them. They're harmless when unused.
- Don't change the diagnostic-but-still-useful console.error lines in `src/lib/mapillary.js` — those provide error observability for the Mapillary 500s and aren't temporary instrumentation.
## Phases
| # | Phase | Effort | Status |
|---|---|---|---|
| 01 | [Cleanup](./phase-01-cleanup.md) | 15m | pending |
## Success Criteria
- `npm ls redis` returns nothing
- `scripts/` directory empty (or removed entirely)
- `.env.migrate` not present locally
- `npm run build` passes
- Original migration plan reports Phase 3 completed