mirror of
https://github.com/tiennm99/vngeoguessr.git
synced 2026-10-11 03:13:56 +00:00
chore: remove leaderboard migration scripts and close plan
Completed one-shot leaderboard backfill migration verified in production. Removed migration scripts, test, unused export, and npm script. Updated documentation and closed UI/UX flow polish plan with session journal.
This commit is contained in:
tiennm99 committed
1 parent
0ce56737f3
commit
7212b67feb
11 files changed
+81
-536
No files matched your search
@@ -184,21 +184,6 @@ before running. Two things worth knowing up front:
|
||||
Deploy the app and the seed in either order: the schema is unchanged, and
|
||||
running processes keep serving their cached counts until they recycle.
|
||||
|
||||
### Migrating leaderboards
|
||||
|
||||
`scripts/migrate-leaderboards.mjs` backfills the two boards whose region code
|
||||
changed (Da Lat into Lam Dong, Duc Hoa into Long An). It is **dry-run by
|
||||
default** and copies rather than moves, so the source boards survive.
|
||||
|
||||
Deploy first, then migrate. The new code writes to the new keys immediately; a
|
||||
migration run before the deploy would copy a board that is still being written
|
||||
to under its old name. Run `npm run leaderboard:migrate` with no flags for a
|
||||
dry run; add `--apply --confirm-prefix=<key prefix>` to write (the confirm
|
||||
flag must echo the deployment's own `KEY_PREFIX`, or the script refuses), and
|
||||
`--restore=<backup.json> --confirm-prefix=<key prefix>` to undo from the
|
||||
backup file the apply run wrote. `--dry-run` is a guard, not a mode switch --
|
||||
combining it with `--apply` throws instead of guessing which one you meant.
|
||||
|
||||
## Styling Conventions
|
||||
|
||||
One design system for every page, including debug pages. It is Tailwind 4 +
|
||||
|
||||
+3
-3
@@ -88,9 +88,9 @@ precision. Each level's added points are returned as `points` on its
|
||||
`leaderboard:vietnam` / `distance:vietnam` keys rather than starting a new
|
||||
`leaderboard:city:vn`. `HN`, `DN` and `TPHCM` kept their codes, so their boards
|
||||
carried over untouched. Only two codes moved — Da Lat's board was backfilled
|
||||
into Lam Dong's and Duc Hoa's into Long An's by
|
||||
`scripts/migrate-leaderboards.mjs`, which copies rather than moves. No score
|
||||
was reset
|
||||
into Lam Dong's and Duc Hoa's into Long An's by a one-shot copy script
|
||||
(applied and verified 2026-09-01, then removed; it survives in git history).
|
||||
No score was reset
|
||||
- **Redis Sorted Sets**: persistent leaderboard data using ZADD/ZRANGE
|
||||
- **Top 200 Entries**: automatic trimming per leaderboard
|
||||
- **Real-time Ranking**: rank calculated with ZREVRANK/ZRANK
|
||||
|
||||
@@ -120,11 +120,9 @@ Each carries a header comment with its flags and its cost.
|
||||
- `build-pano-index.mjs` - Mapillary z14 tiles to per-province panorama artifacts
|
||||
- `assign-pano-districts.mjs` - Clips and partitions panoramas by district
|
||||
- `seed-pano-db.mjs` - Validates the artifacts and uploads them to Neon
|
||||
- `migrate-leaderboards.mjs` - Backfills the two boards whose code changed
|
||||
- `build-check.mjs` - Production build into `.next-check`
|
||||
- `lib/assign-districts.mjs` - District assignment shared by the two pano scripts
|
||||
- `lib/pano-schema.mjs` - Panorama table DDL shared by the seed and the tests
|
||||
- `lib/leaderboard-migration.mjs` - Copy, verify, regression-check and restore
|
||||
|
||||
## Tests (`tests/`)
|
||||
Vitest, mostly one file per `src/lib/` module, plus a route test for
|
||||
|
||||
@@ -18,7 +18,6 @@
|
||||
"data:panos": "node scripts/build-pano-index.mjs",
|
||||
"data:districts": "node scripts/assign-pano-districts.mjs",
|
||||
"data:repartition": "npm run data:boundaries -- --regenerate && npm run data:districts",
|
||||
"leaderboard:migrate": "node scripts/migrate-leaderboards.mjs",
|
||||
"data:seed": "node scripts/seed-pano-db.mjs",
|
||||
"test:e2e": "playwright test"
|
||||
},
|
||||
|
||||
@@ -58,15 +58,17 @@ compat that must NOT be touched.
|
||||
## Todo
|
||||
|
||||
- [x] Step 1 (ungated) — fallbacks removed 2026-09-01, compat test deleted
|
||||
- [ ] Operator runs backfill; verify output (dry run 2026-09-01 confirmed NOT yet applied)
|
||||
- [ ] Steps 2-3
|
||||
- [x] `npm test`, `npm run lint`, `npm run build` green (after step 1)
|
||||
- [x] Operator ran backfill 2026-09-01; dry-run re-check showed every destination
|
||||
matching its source (5/18/6/101 members, replacing same counts)
|
||||
- [x] Steps 2-3 — scripts, test, npm script deleted; docs updated; grep sweep
|
||||
clean (only the permanent-compat comments remain)
|
||||
- [x] `npm test` (233), `npm run lint`, `npm run build` green
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] No `cityCode` session fallback in API routes; tests reflect current shape
|
||||
- [ ] Migration scripts gone only after verified backfill
|
||||
- [ ] Permanent compat untouched (assert `?city=` and legacy response fields
|
||||
- [x] No `cityCode` session fallback in API routes; tests reflect current shape
|
||||
- [x] Migration scripts gone only after verified backfill
|
||||
- [x] Permanent compat untouched (assert `?city=` and legacy response fields
|
||||
still work via existing tests)
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "UI/UX flow polish and legacy cleanup"
|
||||
description: "Newbie-friendly flow, consistent presentation, migration-era code removal"
|
||||
status: in-progress
|
||||
status: completed
|
||||
priority: P1
|
||||
effort: "2-3d"
|
||||
tags: [ui-ux, cleanup]
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: UI/UX flow polish and legacy cleanup
|
||||
date: 2026-09-01
|
||||
summary: "Newbie-friendly flow rework, semantic tokens, session cityCode fallback removal; migration-script deletion gated on unapplied prod backfill"
|
||||
---
|
||||
|
||||
# UI/UX flow polish and legacy cleanup
|
||||
|
||||
## What happened
|
||||
|
||||
Delivered the accepted brainstorm (UI/UX presentation audit + legacy scout, both
|
||||
agent-reviewed) as plan `plans/260901-1636-uiux-flow-polish-and-legacy-cleanup`.
|
||||
Six commits on main (0876a9d..0ce5673), not pushed.
|
||||
|
||||
- **Phase 1 (quick wins):** editable "Playing as X" header chip (mobile too),
|
||||
honest modal CTAs, skip tooltip, result dialog reorder (map above fold,
|
||||
labeled "It was in" reveal, captioned grids), blue guess dot + legend,
|
||||
labeled pano counts ("spots") and badge tooltips, DialogDescription added to
|
||||
Donate/Leaderboard, ThemeToggle 44px, credits page parity.
|
||||
- **Phase 2 (flow):** username prompt deferred to first Play click
|
||||
(RegionPicker interception, navigation resumes after save/skip); skip and
|
||||
deep-link first-submit generate a persisted `Player-xxxxxx` name (6 base36
|
||||
chars — 4 had birthday-collision odds since the name is the ZSET member);
|
||||
one-time FirstRoundHint banner (`vngeoguessr_hint_seen`); desktop "Click to
|
||||
place your guess" ghost label; session rounds/points badge in game header;
|
||||
leaderboard bookkeeping collapsed into a `<details>` section.
|
||||
- **Phase 3 (consistency):** semantic tokens `--success/--warning/--danger/
|
||||
--rank-*` (light+dark) in globals.css replace raw Tailwind palettes in
|
||||
RoundResultDialog/LeaderboardList; ResultMap exports MARKER_COLORS shared
|
||||
with the dialog legend; emoji → Lucide (ThemeToggle Sun/Moon/Monitor, Beer);
|
||||
`.vn-gradient-bg` → `.vn-surface`.
|
||||
- **Phase 4 (partial):** session `cityCode` fallbacks removed from guess/
|
||||
new-game routes + compat test deleted (provably unreachable: new-game rejects
|
||||
rounds without regionCode before the session write, 30-min TTL).
|
||||
|
||||
Code review (DONE_WITH_CONCERNS) caught H1: the hint banner painted over the
|
||||
expanded mobile map's search/collapse controls — the panel's z-[1200] children
|
||||
are clamped inside its z-[500] stacking context. Fixed by hiding the banner
|
||||
while the minimap is expanded. Also fixed: header overflow on small phones
|
||||
(chip truncates, row wraps), modified-click hijack on Play rows, summary focus
|
||||
ring, stale docs (development.md styling conventions, game-flow.md username
|
||||
step, project-structure.md).
|
||||
|
||||
Verified: 252 unit tests, 12 Playwright e2e, lint 0 errors (19 warnings, +1 vs
|
||||
baseline in the codebase's accepted localStorage-read-in-effect idiom), build
|
||||
clean.
|
||||
|
||||
## Decision
|
||||
|
||||
- Random-name fallback replaces the shared "Anonymous" leaderboard bucket going
|
||||
forward; existing "Anonymous" board rows stay (data, not code).
|
||||
- Score-total rainbow in LeaderboardList dropped to neutral — a running total
|
||||
has no ladder to grade against. Distance tints kept, 3 semantic states.
|
||||
- Permanent compat kept forever: `:city:` Redis key prefixes, API aliases
|
||||
(`cityRank` etc.), `?city=`/`?location=` params — they guard player history.
|
||||
|
||||
## Next steps
|
||||
|
||||
- **Gate open:** run
|
||||
`node --env-file=.env scripts/migrate-leaderboards.mjs --apply --confirm-prefix=vngeoguessr:`
|
||||
(dry run 2026-09-01 proved the Da Lat→Lam Dong / Duc Hoa→Long An backfill has
|
||||
NOT been applied; destinations hold 0-2 members vs 5/18/6/101 sources). A
|
||||
permission classifier blocked the agent from running it; operator must.
|
||||
- After verified apply: delete the two migration scripts, their test, the
|
||||
`leaderboard:migrate` npm script; update docs (phase 4 steps 2-3).
|
||||
- Manual UI pass in light + dark themes (user-owned per project convention).
|
||||
- Deferred: Vietnamese font subset (F20), mobile expanded-map panorama peek.
|
||||
|
||||
> Historical work record — not durable authority. Prefer docs/specs/ADRs for current decisions.
|
||||
@@ -1,161 +0,0 @@
|
||||
// Backfill logic for the leaderboard migration.
|
||||
//
|
||||
// Separated from the CLI in scripts/migrate-leaderboards.mjs so every guard is
|
||||
// callable from a test. This is the one place in the region-tree change where
|
||||
// existing player points can be destroyed, and a script whose safety checks are
|
||||
// only reachable by running it against a live database is not testable at all.
|
||||
|
||||
import {
|
||||
scanKeys,
|
||||
zAdd,
|
||||
zRangeWithScores,
|
||||
zRemRangeByRank,
|
||||
} from '../../src/lib/upstash.js';
|
||||
import { leaderboardKeys } from '../../src/lib/leaderboard.js';
|
||||
|
||||
// child -> parent. Lam Dong and Long An are new nodes whose only child already
|
||||
// holds the full history.
|
||||
export const BACKFILLS = [
|
||||
{ from: 'DL', to: 'LD', label: 'Da Lat -> Lam Dong' },
|
||||
{ from: 'DH', to: 'LA', label: 'Duc Hoa -> Long An' },
|
||||
];
|
||||
|
||||
export const PATTERNS = ['leaderboard:*', 'distance:*'];
|
||||
|
||||
/**
|
||||
* Every key each backfill pair touches, score and distance.
|
||||
*
|
||||
* Built through the same key functions the app uses, so a change to key naming
|
||||
* cannot leave the migration writing to the old names.
|
||||
* @returns {Array<{fromKey: string, toKey: string, label: string}>} Pairs.
|
||||
*/
|
||||
export function backfillPairs() {
|
||||
return BACKFILLS.flatMap(({ from, to, label }) => [
|
||||
{ fromKey: leaderboardKeys.score(from), toKey: leaderboardKeys.score(to), label },
|
||||
{ fromKey: leaderboardKeys.distance(from), toKey: leaderboardKeys.distance(to), label },
|
||||
]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Read every leaderboard key with its members and scores.
|
||||
* @param {Object} h Upstash handle.
|
||||
* @returns {Promise<Object>} Logical key -> [{value, score}].
|
||||
*/
|
||||
export async function exportAll(h) {
|
||||
const snapshot = {};
|
||||
for (const pattern of PATTERNS) {
|
||||
for (const key of await scanKeys(h, pattern)) {
|
||||
snapshot[key] = await zRangeWithScores(h, key, 0, -1, false);
|
||||
}
|
||||
}
|
||||
return snapshot;
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy one sorted set over another, replacing what was there.
|
||||
*
|
||||
* Absolute scores, not increments, so re-running converges instead of doubling.
|
||||
* The destination is emptied first: writing member-by-member would leave behind
|
||||
* anyone present only in the destination, which the 200-entry trim makes
|
||||
* reachable -- a player can be trimmed out of the source while surviving in the
|
||||
* copy, and would then keep a stale score forever.
|
||||
*
|
||||
* An empty source is refused BEFORE the delete. Otherwise this wipes the
|
||||
* destination and writes nothing back, which is indistinguishable from a
|
||||
* successful no-op in the output and destroys real points.
|
||||
* @param {Object} h Upstash handle.
|
||||
* @param {string} fromKey Source logical key.
|
||||
* @param {string} toKey Destination logical key.
|
||||
* @param {boolean} apply False to report without writing.
|
||||
* @returns {Promise<{members: number, replaced: number, skipped: boolean}>}
|
||||
*/
|
||||
export async function copySortedSet(h, fromKey, toKey, apply) {
|
||||
const source = await zRangeWithScores(h, fromKey, 0, -1, false);
|
||||
const destination = await zRangeWithScores(h, toKey, 0, -1, false);
|
||||
|
||||
if (source.length === 0) {
|
||||
if (destination.length > 0) {
|
||||
throw new Error(
|
||||
`${fromKey} is empty but ${toKey} holds ${destination.length} members. ` +
|
||||
'Copying would delete them. Refusing: check KEY_PREFIX and the source key.'
|
||||
);
|
||||
}
|
||||
// Both empty is the ordinary case for a board nobody has played yet.
|
||||
return { members: 0, replaced: 0, skipped: true };
|
||||
}
|
||||
|
||||
if (apply) {
|
||||
await zRemRangeByRank(h, toKey, 0, -1);
|
||||
for (const { value, score } of source) await zAdd(h, toKey, score, value);
|
||||
}
|
||||
return { members: source.length, replaced: destination.length, skipped: false };
|
||||
}
|
||||
|
||||
/**
|
||||
* Confirm the backfill landed: each destination must now equal its source.
|
||||
*
|
||||
* The earlier version verified only the keys it did NOT write, which meant the
|
||||
* copy itself was never checked.
|
||||
* @param {Object} h Upstash handle.
|
||||
* @returns {Promise<string[]>} Pairs that disagree, empty when all match.
|
||||
*/
|
||||
export async function verifyTargets(h) {
|
||||
const mismatched = [];
|
||||
for (const { fromKey, toKey } of backfillPairs()) {
|
||||
const source = await zRangeWithScores(h, fromKey, 0, -1, false);
|
||||
const destination = await zRangeWithScores(h, toKey, 0, -1, false);
|
||||
if (JSON.stringify(source) !== JSON.stringify(destination)) {
|
||||
mismatched.push(`${toKey} does not match ${fromKey}`);
|
||||
}
|
||||
}
|
||||
return mismatched;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check that no untouched key lost anything while the migration ran.
|
||||
*
|
||||
* Ordering is deploy-then-migrate, so the app is serving guesses throughout and
|
||||
* boards legitimately grow mid-run. Demanding byte-equality would throw on a
|
||||
* healthy migration the moment one player scores. Forward-only drift -- members
|
||||
* added, scores never reduced, nobody removed -- is expected; anything else is
|
||||
* not.
|
||||
* @param {Object} before Snapshot taken before the writes.
|
||||
* @param {Object} after Snapshot taken after.
|
||||
* @returns {string[]} Keys that regressed, empty when all are intact.
|
||||
*/
|
||||
export function findRegressions(before, after) {
|
||||
const targets = new Set(backfillPairs().map(({ toKey }) => toKey));
|
||||
const regressed = [];
|
||||
|
||||
for (const [key, entries] of Object.entries(before)) {
|
||||
if (targets.has(key)) continue;
|
||||
const now = after[key];
|
||||
if (!now) {
|
||||
regressed.push(`${key} disappeared`);
|
||||
continue;
|
||||
}
|
||||
const nowByMember = new Map(now.map((e) => [e.value, e.score]));
|
||||
for (const { value, score } of entries) {
|
||||
if (!nowByMember.has(value)) {
|
||||
regressed.push(`${key}: member ${value} removed`);
|
||||
} else if (nowByMember.get(value) < score) {
|
||||
regressed.push(`${key}: ${value} fell from ${score} to ${nowByMember.get(value)}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
return regressed;
|
||||
}
|
||||
|
||||
/**
|
||||
* Restore a snapshot, making the backup an actual rollback rather than a file.
|
||||
* @param {Object} h Upstash handle.
|
||||
* @param {Object} snapshot Output of exportAll.
|
||||
* @returns {Promise<number>} Keys restored.
|
||||
*/
|
||||
export async function restore(h, snapshot) {
|
||||
for (const [key, entries] of Object.entries(snapshot)) {
|
||||
await zRemRangeByRank(h, key, 0, -1);
|
||||
for (const { value, score } of entries) await zAdd(h, key, score, value);
|
||||
}
|
||||
return Object.keys(snapshot).length;
|
||||
}
|
||||
@@ -1,123 +0,0 @@
|
||||
// Seed the two new province leaderboards from the towns they inherited.
|
||||
//
|
||||
// node scripts/migrate-leaderboards.mjs # dry run
|
||||
// node scripts/migrate-leaderboards.mjs --apply --confirm-prefix=vngeoguessr:
|
||||
// node scripts/migrate-leaderboards.mjs --restore=<backup.json> --confirm-prefix=...
|
||||
//
|
||||
// Lam Dong and Long An are new nodes. Their only child -- Da Lat and Duc Hoa --
|
||||
// already holds the full history, so without a backfill each province would
|
||||
// disagree with its own single child from day one.
|
||||
//
|
||||
// RUN THIS AFTER DEPLOYING THE FAN-OUT, not before. Migrating first opens a
|
||||
// window where a Da Lat guess credits DL and Vietnam but not Lam Dong, and the
|
||||
// province is permanently short by that window. Deploying first is safe: a
|
||||
// pre-migration LD only accumulates a subset of DL's deltas, which the copy
|
||||
// then overwrites.
|
||||
//
|
||||
// Nothing else is touched. Ha Noi, Da Nang and Ho Chi Minh keep their history
|
||||
// where it is: those points predate districts and cannot be attributed to one.
|
||||
//
|
||||
// The logic lives in scripts/lib/leaderboard-migration.mjs so its guards are
|
||||
// testable; this file is argument handling and reporting.
|
||||
|
||||
import { writeFileSync, readFileSync } from 'node:fs';
|
||||
import { getUpstash } from '../src/lib/upstash.js';
|
||||
import {
|
||||
backfillPairs,
|
||||
copySortedSet,
|
||||
exportAll,
|
||||
findRegressions,
|
||||
restore,
|
||||
verifyTargets,
|
||||
} from './lib/leaderboard-migration.mjs';
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
const apply = args.includes('--apply');
|
||||
const dryRunFlag = args.includes('--dry-run');
|
||||
const restorePath = args.find((arg) => arg.startsWith('--restore='))?.split('=')[1];
|
||||
const confirmed = args.find((arg) => arg.startsWith('--confirm-prefix='))?.split('=')[1];
|
||||
|
||||
if (dryRunFlag && apply) {
|
||||
// Typing both is a reasonable belt-and-braces instinct, and silently writing
|
||||
// would be the worst possible answer to it.
|
||||
throw new Error('--dry-run and --apply are mutually exclusive');
|
||||
}
|
||||
|
||||
const h = getUpstash();
|
||||
const writing = apply || Boolean(restorePath);
|
||||
console.log(`key prefix: ${JSON.stringify(h.prefix)}`);
|
||||
console.log(writing ? 'mode: WRITES ENABLED' : 'mode: dry run (no writes)');
|
||||
|
||||
if (writing && confirmed !== h.prefix) {
|
||||
// The prefix decides which namespace this touches, and getting it wrong is
|
||||
// silent: every read returns empty and every write lands somewhere nothing
|
||||
// reads. Make the operator name it.
|
||||
throw new Error(
|
||||
`Refusing to write: pass --confirm-prefix=${h.prefix} to confirm the target namespace` +
|
||||
(confirmed !== undefined ? ` (got ${JSON.stringify(confirmed)})` : '')
|
||||
);
|
||||
}
|
||||
|
||||
/** Put a snapshot back. The backup is only a rollback if something reads it. */
|
||||
async function runRestore(path) {
|
||||
const snapshot = JSON.parse(readFileSync(path, 'utf8'));
|
||||
const count = await restore(h, snapshot);
|
||||
console.log(`restored ${count} keys from ${path}`);
|
||||
}
|
||||
|
||||
/** Back-fill the two new province boards from their single child. */
|
||||
async function runBackfill() {
|
||||
const before = await exportAll(h);
|
||||
const keyCount = Object.keys(before).length;
|
||||
console.log(`exported ${keyCount} leaderboard keys`);
|
||||
|
||||
if (keyCount === 0) {
|
||||
// An empty export is far more likely to be a wrong prefix than an empty
|
||||
// database, and it would make the backup -- the only rollback -- worthless.
|
||||
throw new Error(
|
||||
'Export found no leaderboard keys. That usually means KEY_PREFIX does not ' +
|
||||
'match the deployment. Refusing to continue: the backup would be empty.'
|
||||
);
|
||||
}
|
||||
|
||||
const stamp = new Date().toISOString().replace(/[:.]/g, '-');
|
||||
const backupPath = `leaderboard-backup-${stamp}.json`;
|
||||
writeFileSync(backupPath, JSON.stringify(before, null, 1) + '\n');
|
||||
console.log(`backup -> ${backupPath} (restore with --restore=${backupPath})`);
|
||||
|
||||
console.log('');
|
||||
for (const { fromKey, toKey, label } of backfillPairs()) {
|
||||
const { members, replaced, skipped } = await copySortedSet(h, fromKey, toKey, apply);
|
||||
console.log(
|
||||
`${label.padEnd(20)} ${fromKey} -> ${toKey}: ` +
|
||||
(skipped ? 'both empty, nothing to do' : `${members} members`) +
|
||||
(replaced ? `, replacing ${replaced} already there` : '') +
|
||||
(apply || skipped ? '' : ' [dry run]')
|
||||
);
|
||||
}
|
||||
|
||||
if (!apply) {
|
||||
console.log(`\nNothing written. Re-run with --apply --confirm-prefix=${h.prefix}`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Two checks, because they catch different failures: the destinations must
|
||||
// now match their sources, and nothing else may have gone backwards.
|
||||
const mismatched = await verifyTargets(h);
|
||||
if (mismatched.length > 0) {
|
||||
throw new Error(`Backfill did not land: ${mismatched.join('; ')}`);
|
||||
}
|
||||
|
||||
const regressed = findRegressions(before, await exportAll(h));
|
||||
if (regressed.length > 0) {
|
||||
throw new Error(
|
||||
`Existing data regressed: ${regressed.join('; ')}. ` +
|
||||
`Restore with --restore=${backupPath} --confirm-prefix=${h.prefix}`
|
||||
);
|
||||
}
|
||||
|
||||
console.log(`\napplied. Backfill verified, and ${keyCount} pre-existing keys intact.`);
|
||||
}
|
||||
|
||||
if (restorePath) await runRestore(restorePath);
|
||||
else await runBackfill();
|
||||
@@ -63,14 +63,6 @@ function getDistanceLeaderboardKey(regionCode) {
|
||||
return `${DISTANCE_CITY_PREFIX}${regionCode.toLowerCase()}`;
|
||||
}
|
||||
|
||||
// Both key builders, so scripts/lib/leaderboard-migration.mjs derives the keys
|
||||
// it copies from the same code the app writes through. Hardcoding the prefixes
|
||||
// there would let a change here silently point the migration at old names.
|
||||
export const leaderboardKeys = {
|
||||
score: getRegionLeaderboardKey,
|
||||
distance: getDistanceLeaderboardKey,
|
||||
};
|
||||
|
||||
/**
|
||||
* Get a leaderboard for one region.
|
||||
* @param {string|null} regionCode Region code, or null for the country.
|
||||
|
||||
@@ -1,216 +0,0 @@
|
||||
import { describe, it, expect, beforeEach, vi } from 'vitest';
|
||||
vi.mock('@upstash/redis', async (importOriginal) => {
|
||||
const { upstashModule } = await import('./mock-upstash.js');
|
||||
return upstashModule(importOriginal);
|
||||
});
|
||||
|
||||
import { getUpstash, scanKeys, zAdd, zRangeWithScores } from '../src/lib/upstash.js';
|
||||
import {
|
||||
backfillPairs,
|
||||
copySortedSet,
|
||||
exportAll,
|
||||
findRegressions,
|
||||
restore,
|
||||
verifyTargets,
|
||||
} from '../scripts/lib/leaderboard-migration.mjs';
|
||||
import { resetStore } from './redis-harness.js';
|
||||
|
||||
// The migration is the one place in this change where existing points can be
|
||||
// lost, so its guards are exercised against the real functions the script runs.
|
||||
// An earlier version of this file reimplemented the copy locally, which meant
|
||||
// the missing empty-source guard was invisible to the suite.
|
||||
|
||||
const h = () => getUpstash();
|
||||
|
||||
describe('scanKeys', () => {
|
||||
beforeEach(async () => {
|
||||
await resetStore();
|
||||
});
|
||||
|
||||
it('finds leaderboard keys and hands them back unprefixed', async () => {
|
||||
// Callers work in the logical namespace. A migration that reached past the
|
||||
// adapter and scanned 'leaderboard:*' directly would match nothing, because
|
||||
// every physical key carries KEY_PREFIX.
|
||||
await zAdd(h(), 'leaderboard:city:dl', 5, 'mai');
|
||||
await zAdd(h(), 'leaderboard:vietnam', 5, 'mai');
|
||||
await zAdd(h(), 'distance:city:dl', 120, 'mai:120:1');
|
||||
|
||||
const found = await scanKeys(h(), 'leaderboard:*');
|
||||
expect(found.sort()).toEqual(['leaderboard:city:dl', 'leaderboard:vietnam']);
|
||||
for (const key of found) expect(key.startsWith(h().prefix)).toBe(false);
|
||||
});
|
||||
|
||||
it('keeps one pattern out of another namespace', async () => {
|
||||
await zAdd(h(), 'leaderboard:city:dl', 5, 'mai');
|
||||
await zAdd(h(), 'distance:city:dl', 120, 'mai:120:1');
|
||||
expect(await scanKeys(h(), 'distance:*')).toEqual(['distance:city:dl']);
|
||||
});
|
||||
|
||||
it('returns nothing for a pattern that matches nothing', async () => {
|
||||
await zAdd(h(), 'leaderboard:city:dl', 5, 'mai');
|
||||
expect(await scanKeys(h(), 'nosuch:*')).toEqual([]);
|
||||
});
|
||||
|
||||
it('never returns a duplicate', async () => {
|
||||
// SCAN promises each key at least once, not exactly once.
|
||||
await zAdd(h(), 'leaderboard:city:dl', 5, 'mai');
|
||||
const found = await scanKeys(h(), 'leaderboard:*');
|
||||
expect(new Set(found).size).toBe(found.length);
|
||||
});
|
||||
});
|
||||
|
||||
describe('backfill safety', () => {
|
||||
beforeEach(async () => {
|
||||
await resetStore();
|
||||
});
|
||||
|
||||
it('refuses an empty source that would wipe a populated destination', async () => {
|
||||
// The dangerous case: DEL-then-copy with nothing to copy back empties the
|
||||
// destination and looks like a clean no-op in the output.
|
||||
await zAdd(h(), 'leaderboard:city:ld', 42, 'mai');
|
||||
|
||||
await expect(
|
||||
copySortedSet(h(), 'leaderboard:city:dl', 'leaderboard:city:ld', true)
|
||||
).rejects.toThrow(/Refusing/);
|
||||
|
||||
const survived = await zRangeWithScores(h(), 'leaderboard:city:ld', 0, -1, false);
|
||||
expect(survived).toEqual([{ value: 'mai', score: 42 }]);
|
||||
});
|
||||
|
||||
it('treats both-empty as an ordinary no-op', async () => {
|
||||
const result = await copySortedSet(
|
||||
h(),
|
||||
'leaderboard:city:dh',
|
||||
'leaderboard:city:la',
|
||||
true
|
||||
);
|
||||
expect(result.skipped).toBe(true);
|
||||
});
|
||||
|
||||
it('writes nothing in dry-run mode', async () => {
|
||||
await zAdd(h(), 'leaderboard:city:dl', 12, 'mai');
|
||||
await copySortedSet(h(), 'leaderboard:city:dl', 'leaderboard:city:ld', false);
|
||||
expect(await zRangeWithScores(h(), 'leaderboard:city:ld', 0, -1, false)).toEqual([]);
|
||||
});
|
||||
|
||||
it('copies absolute scores, so running twice is a no-op', async () => {
|
||||
await zAdd(h(), 'leaderboard:city:dl', 12, 'mai');
|
||||
await zAdd(h(), 'leaderboard:city:dl', 7, 'linh');
|
||||
|
||||
await copySortedSet(h(), 'leaderboard:city:dl', 'leaderboard:city:ld', true);
|
||||
const once = await zRangeWithScores(h(), 'leaderboard:city:ld', 0, -1, false);
|
||||
await copySortedSet(h(), 'leaderboard:city:dl', 'leaderboard:city:ld', true);
|
||||
const twice = await zRangeWithScores(h(), 'leaderboard:city:ld', 0, -1, false);
|
||||
|
||||
// Adding instead of replacing would double every score on the second run.
|
||||
expect(twice).toEqual(once);
|
||||
expect(once.find((e) => e.value === 'mai').score).toBe(12);
|
||||
});
|
||||
|
||||
it('removes a destination-only member instead of stranding it', async () => {
|
||||
// Reachable in practice: the 200-entry trim can drop a player from the
|
||||
// source while they survive in the copy, and a member-by-member write would
|
||||
// leave them there with a stale score forever.
|
||||
await zAdd(h(), 'leaderboard:city:dl', 12, 'mai');
|
||||
await zAdd(h(), 'leaderboard:city:ld', 999, 'orphan');
|
||||
|
||||
await copySortedSet(h(), 'leaderboard:city:dl', 'leaderboard:city:ld', true);
|
||||
|
||||
const after = await zRangeWithScores(h(), 'leaderboard:city:ld', 0, -1, false);
|
||||
expect(after.map((e) => e.value)).toEqual(['mai']);
|
||||
});
|
||||
|
||||
it('leaves the source untouched', async () => {
|
||||
await zAdd(h(), 'leaderboard:city:dh', 8, 'mai');
|
||||
const before = await zRangeWithScores(h(), 'leaderboard:city:dh', 0, -1, false);
|
||||
await copySortedSet(h(), 'leaderboard:city:dh', 'leaderboard:city:la', true);
|
||||
expect(await zRangeWithScores(h(), 'leaderboard:city:dh', 0, -1, false)).toEqual(before);
|
||||
});
|
||||
});
|
||||
|
||||
describe('backfill verification', () => {
|
||||
beforeEach(async () => {
|
||||
await resetStore();
|
||||
});
|
||||
|
||||
it('derives its keys from the app, not from hardcoded strings', async () => {
|
||||
const pairs = backfillPairs();
|
||||
expect(pairs.map((p) => `${p.fromKey}->${p.toKey}`)).toEqual([
|
||||
'leaderboard:city:dl->leaderboard:city:ld',
|
||||
'distance:city:dl->distance:city:ld',
|
||||
'leaderboard:city:dh->leaderboard:city:la',
|
||||
'distance:city:dh->distance:city:la',
|
||||
]);
|
||||
});
|
||||
|
||||
it('confirms each destination matches its source', async () => {
|
||||
await zAdd(h(), 'leaderboard:city:dl', 12, 'mai');
|
||||
expect(await verifyTargets(h())).not.toEqual([]);
|
||||
|
||||
await copySortedSet(h(), 'leaderboard:city:dl', 'leaderboard:city:ld', true);
|
||||
expect(await verifyTargets(h())).toEqual([]);
|
||||
});
|
||||
|
||||
it('reports a destination that never got written', async () => {
|
||||
// The earlier verification excluded exactly the keys the script writes, so
|
||||
// a failed copy passed silently.
|
||||
await zAdd(h(), 'leaderboard:city:dl', 12, 'mai');
|
||||
const mismatched = await verifyTargets(h());
|
||||
expect(mismatched.join()).toMatch(/leaderboard:city:ld/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('regression detection', () => {
|
||||
it('accepts forward-only drift from live traffic', () => {
|
||||
// Ordering is deploy-then-migrate, so players keep scoring while the script
|
||||
// runs. Demanding byte-equality would throw on a healthy migration.
|
||||
const before = { 'leaderboard:vietnam': [{ value: 'mai', score: 10 }] };
|
||||
const after = {
|
||||
'leaderboard:vietnam': [
|
||||
{ value: 'mai', score: 13 },
|
||||
{ value: 'linh', score: 4 },
|
||||
],
|
||||
};
|
||||
expect(findRegressions(before, after)).toEqual([]);
|
||||
});
|
||||
|
||||
it('flags a score that went backwards', () => {
|
||||
const before = { 'leaderboard:vietnam': [{ value: 'mai', score: 10 }] };
|
||||
const after = { 'leaderboard:vietnam': [{ value: 'mai', score: 3 }] };
|
||||
expect(findRegressions(before, after).join()).toMatch(/fell from 10 to 3/);
|
||||
});
|
||||
|
||||
it('flags a member that disappeared', () => {
|
||||
const before = { 'leaderboard:city:hn': [{ value: 'mai', score: 10 }] };
|
||||
const after = { 'leaderboard:city:hn': [] };
|
||||
expect(findRegressions(before, after).join()).toMatch(/member mai removed/);
|
||||
});
|
||||
|
||||
it('flags a key that disappeared', () => {
|
||||
const before = { 'leaderboard:city:hn': [{ value: 'mai', score: 10 }] };
|
||||
expect(findRegressions(before, {}).join()).toMatch(/disappeared/);
|
||||
});
|
||||
|
||||
it('ignores the backfill destinations, which are meant to change', () => {
|
||||
const before = { 'leaderboard:city:ld': [{ value: 'orphan', score: 99 }] };
|
||||
expect(findRegressions(before, { 'leaderboard:city:ld': [] })).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('restore', () => {
|
||||
beforeEach(async () => {
|
||||
await resetStore();
|
||||
});
|
||||
|
||||
it('puts a snapshot back, making the backup an actual rollback', async () => {
|
||||
await zAdd(h(), 'leaderboard:city:hn', 50, 'mai');
|
||||
await zAdd(h(), 'leaderboard:city:hn', 20, 'linh');
|
||||
const snapshot = await exportAll(h());
|
||||
|
||||
// Simulate the damage a bad run would do.
|
||||
await zAdd(h(), 'leaderboard:city:hn', 1, 'mai');
|
||||
await restore(h(), snapshot);
|
||||
|
||||
expect(await exportAll(h())).toEqual(snapshot);
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user