Commit Graph
18 Commits
Author SHA1 Message Date
tiennm99 5765055588 test: add DO storage + integration coverage via wrangler unstable_dev
Pure-function unit tests for chunk-storage (BLOB-grow, orphan-row read,
multi-chunk grouping) and cooldown-store (TTL math, INSERT cursor drain,
GC sampling, release/refund) cover the bugs Phase 1 fixed.

Integration suite boots a local Worker + real DO via wrangler
unstable_dev and asserts cookie issuance, cooldown isolation across
identities, Content-Length 411/413 guards, WS upgrade semantics, WS
broadcast frame shape with monotonic seq, ping/pong, per-identity WS
cap, and Origin allowlist. WS uses the ws package directly since
unstable_dev's fetch strips CF's webSocket Response field.

156 tests pass in ~3.3s; verified stable across 3 consecutive runs.
2026-05-11 16:17:35 +07:00
tiennm99 4f02d30393 feat(canvas): WS hardening, client race fix, and ping/pong heartbeat
Server:
- Origin allowlist on /api/ws (env.ALLOWED_ORIGINS, comma-separated; empty
  = allow all for dev/preview)
- per-identity WS connection cap (MAX_WS_PER_IDENTITY = 5) using
  acceptWebSocket(socket, [identity]) tagging; 6th upgrade returns 429
- ws.send 'ping' triggers a {type:'pong'} reply so dead connections fire
  onclose promptly instead of waiting on TCP keepalive

Client:
- buffer WS pixels arriving during the initial canvas fetch and replay
  them after committedColors is replaced; previously the post-fetch
  Uint8Array assignment silently overwrote any pixels broadcast in the
  fetch window (the documented C2 race)
- 30s ping / 60s pong watchdog closes the socket if pong stops arriving,
  routing through the existing exponential-backoff reconnect path

Tests:
- four /api/ws cases: missing upgrade, disallowed origin, allowed origin,
  empty allowlist (dev default). Sentinel uses status 200 because undici
  rejects 101 in Node-side Response constructors.

Plan: plans/260510-0232-fix-do-migration-followups/phase-03-ws-hardening-client-race.md
2026-05-10 03:05:56 +07:00
tiennm99 42d1ca19ee feat(canvas): cookie+IP rate-limit identity and broadcast sequence numbers
- resolveIdentity prefers an opaque rplace_id cookie; falls back to a
  cf-connecting-ip hash; in production a request with neither now returns
  500 no_identity instead of bucketing all such traffic together
- /api/canvas issues Set-Cookie when no cookie is present so subsequent
  requests escape NAT-shared IP buckets (mobile/CGNAT users)
- DO maintains an in-memory monotonic broadcast counter; broadcast frames
  carry { seq } so the client can detect missed pixels and refetch
- client tracks lastSeq, refetches on gap, resets on every (re)connect

NAT/CGNAT users previously shared a single 1Hz bucket per egress IP. With
cookie identity they each get their own bucket. Cookie is HttpOnly, Secure,
SameSite=Lax, 1y Max-Age. Stripped/cleared cookies fall through to IP.

The seq counter resets on DO hibernation rehydrate; client always refetches
on reconnect, so a reset is indistinguishable from a fresh connect.

Plan: plans/260510-0232-fix-do-migration-followups/phase-02-cookie-ip-identity.md
2026-05-10 03:00:39 +07:00
tiennm99 3c1263add6 fix(canvas): make multi-chunk pixel writes atomic and refund cooldown on failure
- wrap writePixels in state.storage.transactionSync so a partial multi-chunk
  failure doesn't leave the canvas half-written or out of sync with the WS
  broadcast
- size new chunk buffer against chunkSize(chunkId) instead of the persisted
  blob's length so writes after a canvas-grow no longer silently drop OOB
  bytes in the formerly-last short chunk
- refund the cooldown row when writePixels throws so transient storage
  errors stop soft-DOSing the user (and halving image-uploader throughput)
- bound readAllChunks by chunk_id < CHUNK_COUNT and trim oversized blobs so
  orphan rows from a future shrink no longer crash GET /api/canvas
- require a positive Content-Length on /api/place (411) and reject above the
  pre-parse cap (413); previously a missing or zero header bypassed the cap
- drop String(err) from the 500 response body
- drain the INSERT cursor symmetrically with the UPDATE branch in tryAcquire
- assert CHUNK_BYTES <= 2 MB at module load (DO SQLite per-cell BLOB cap)
- correct the inverted webSocketClose comment and guard the re-close call
- add tests for missing / zero / oversized Content-Length

Plan: plans/260510-0232-fix-do-migration-followups/phase-01-do-storage-atomicity.md
2026-05-10 02:57:30 +07:00
tiennm99 a977adc62d refactor(canvas): drop Upstash entirely after successful DO migration
Production migration ran cleanly (samples_checked: 9, mismatches: []),
canvas data preserved, drawing verified by user. Phase 4 cleanup:

Removed code:
- src/lib/canvas-storage.js (Upstash GETRANGE reader)
- src/lib/redis-client.js   (Upstash REST helpers)
- src/lib/rate-limiter.js   (orphan since Phase 2)
- src/admin/migrate-from-upstash.js (one-shot importer)
- src/durable-objects/canvas-room.js #handleImport route
- src/durable-objects/lib/chunk-storage.js importFullCanvas helper
- src/worker.js /admin/migrate-from-upstash mount + import

Removed tests:
- test/lib/canvas-storage.test.js
- test/lib/redis-client.test.js
- test/integration/redis-canvas-roundtrip.test.js
- test/durable-objects/canvas-room.test.js (was skipped pending rewrite)
- vitest.integration.config.js (only Redis testcontainers used it)

Removed deps:
- @upstash/redis, ioredis, testcontainers (-184 packages)

Removed constants:
- REDIS_KEY_PREFIX, REDIS_CANVAS_KEY (only used by deleted code)

Removed package.json scripts: test:integration, test:all
Removed CF Worker secrets in production:
  UPSTASH_REDIS_REST_URL, UPSTASH_REDIS_REST_TOKEN, MIGRATION_TOKEN

Tests: 94/94 pass (down from 112 — 18 deleted Upstash-specific).
Bundle: 71.90 KiB (down from 79.43 KiB).
Production verified: canvas data intact (38% non-zero bytes), migration
endpoint returns 404, secret list empty.
2026-05-10 02:05:43 +07:00
tiennm99 c3f7c02f6d feat(canvas): migrate canvas + cooldown storage to DO SQLite
Move pixel state and rate-limit cooldowns out of Upstash Redis and into
the existing CanvasRoom Durable Object's SQLite-backed storage. Worker
becomes a thin validation/proxy; the DO does atomic cooldown check +
pixel write + WS broadcast in one in-memory step.

Why: eliminate external dependency, keep $0/month free-tier forever,
exploit single-threaded actor for strong consistency without round-trips.

Architecture:
- canvas_chunks: 256 BLOB rows × 64 KB; CHUNK_COUNT derived from
  CANVAS_WIDTH × CANVAS_HEIGHT / CHUNK_BYTES so resize is config-only.
- cooldowns: user_id → expires_at, 1% sample-rate lazy GC.
- Worker forwards /api/canvas, /api/place, /api/ws to DO endpoints.
- POST /admin/migrate-from-upstash: token-gated one-shot importer.

Phases 1-3 code complete; Upstash dependency stays until prod migration
runs and 7-day rollback window passes (Phase 4).

Plan: plans/260509-2309-canvas-on-do-storage/
2026-05-09 23:53:21 +07:00
tiennm99 6b790f3892 chore: prune dead code, unused exports, and orphan scripts
- scripts/: remove upload-colors.js (broken — imported removed constants)
  and image-to-colors.js (orphan — flow moved fully client-side to
  ImageImporter). Drop sharp from devDependencies.
- constants.js: drop unused BITS_PER_PIXEL export.
- dither-kernels.js: make KERNEL_* consts internal; only the
  ERROR_DIFFUSION_KERNELS map is consumed externally.
- image-pipeline.js: drop unused pipeline.source() accessor.
- image-to-palette.js: drop `options.dither` legacy boolean shim
  (no external caller uses it) and its test.
- canvas-storage.js, App.svelte: drop comments that restated names.
2026-04-18 15:01:49 +07:00
tiennm99 cfbac2a586 feat(canvas): 4096^2 canvas, 256-color palette (u8 byte-aligned), custom picker (#5)
Canvas:
- CANVAS_W/H = 4096, total 16,777,216 pixels
- BITS_PER_PIXEL = 8 (byte-aligned) — raw Redis bytes are palette indices
- Canvas-decoder becomes an identity wrap/copy
- Storage BITFIELD uses u8; offset = y*W + x
- Redis key versioned to rplace:canvas:v2 so old 32-color/2048^2 data is
  orphaned (operators can DEL the old key to reclaim memory)

Palette:
- 256 entries, generated deterministically:
  - 0..15  = 16-step grayscale ramp (pure black -> pure white)
  - 16..255 = 240 HSL wheel (4 lightness rings x 60 hues @ 82% saturation)
- nearestPaletteIndex(r,g,b) helper for custom-color snapping

UI:
- ColorPicker: 16-swatch favorites strip (grays + 8 accents) + current-color
  swatch + expand toggle for the full 16x16 grid + "Custom..." button that
  opens the native <input type="color"> and snaps to nearest palette entry
- Default selected color bumped to index 0 (black)

Tests + docs:
- canvas-decoder tests rewritten for identity u8 decode
- canvas-storage tests updated for u8 offsets
- image-to-palette tests anchored to PALETTE_BLACK=0 / PALETTE_WHITE=15 and
  COLORS_RGBA[i] probes (no more hardcoded old 32-color indices)
- integration test uses u8 BITFIELD and canvas-aware bounds
- README, system-architecture, deployment-guide updated (storage math,
  migration note for orphaned old key)
2026-04-18 13:47:01 +07:00
tiennm99 f59e55a852 refactor(rate-limit): 1 req/sec cooldown, batch up to 2048 (#3)
* refactor(rate-limit): switch to 1 req/sec cooldown, batch size up to 2048

Replace per-pixel credit/token-bucket model with a simple per-user cooldown
(SET NX EX 1). Batch size is now independent of the rate limit and capped
at MAX_BATCH_SIZE = 2048.

- rate-limiter: SET NX EX replaces Lua credit script
- worker: response shape { ok: true } (no credits field)
- client: drop credit state/timer/UserInfo; uploader paces by cooldown
- tests: mock checkRateLimit; integration test exercises SET NX EX
- docs: README, system-architecture, code-standards, deployment-guide

* chore(plans): remove implemented plan directories

rplace-implementation (base build), review-fixes, and
image-importer-enhancements are all shipped. Keep plans/reports/ as
historical code-review and research references.
2026-04-18 10:19:00 +07:00
tiennm99 d924a66c50 feat(importer): phase 6 — skip-white + paint-transparent toggles
Two quick quality-of-life options for logo and background-heavy imports:

- Skip-white: near-white pixels (r,g,b >= threshold, default 230) are
  marked transparent in the output so they don't consume credits. Useful
  when the source has a white background you don't want to paint.
- Paint-transparent: fully-transparent source pixels are treated as opaque
  white before quantization. Useful if the canvas background is white and
  you want empty regions of a logo to blend in.

Both toggles compose with every existing dither method; paint-transparent
runs first, so combining the two leaves originally-transparent pixels as
skip (since they're synthesized white, then the skip-white pass drops
them).

UI adds the two checkboxes and a threshold slider shown only when
skip-white is on. CLI gains --skip-white, --white-threshold,
--paint-transparent. 4 new unit tests pin the behavior including the
paint+skip interaction.
2026-04-17 11:58:55 +07:00
tiennm99 6c35771445 feat(importer): phase 5 — color correction sliders
Brightness, contrast, saturation, and gamma adjustments before palette
quantization. Pure-function applyColorCorrection in src/lib keeps the CLI
and browser importer in sync. Inserted post-resize so slider moves stay
responsive on large sources; precision loss is negligible through a
32-color palette.

UI is a collapsible "Color correction" section with four sliders, live
values, a "Reset" button, and an "on" badge when any value differs from
default. CLI gains --brightness, --contrast, --saturation, --gamma
(all -100..+100 except gamma which is 0.1..3.0).

9 unit tests pin down identity, saturate-to-0-or-255 clamping, greyscale
at saturation=-100, gamma midtone direction, and alpha preservation.
2026-04-17 11:56:35 +07:00
tiennm99 4a5b6ef903 feat(importer): phase 4 — more dithering algorithms
Replace the single Floyd-Steinberg toggle with a method dropdown offering
none, floyd, atkinson, jarvis, burkes, sierra, sierra-lite (error diffusion)
and bayer-2 / bayer-4 / bayer-8 (ordered). Kernels live in
src/lib/dither-kernels.js as plain data; rgbaToPalette dispatches via a
shared runErrorDiffusion runner and a runOrderedDither for Bayer.

The legacy `dither: true` option keeps working (maps to `method: 'floyd'`),
so existing CLI invocations and tests stay green. CLI gains
`--dither-method <name>` which takes precedence over `--dither`.

13 unit tests cover exact-palette hits, transparent pixels, all-method
smoke, kernel weight sums (Atkinson=0.75, others=1.0), and verify Bayer
actually scatters on mid-grey where plain nearest would produce one color.
2026-04-17 11:53:52 +07:00
tiennm99 fc0121fd48 feat(importer): phase 3 — flip / rotate transforms
Pure-function transformRgba in src/lib/image-transform.js supports flip H,
flip V, and rotation in 90° steps (fixed application order: rotate → flipH
→ flipV). Wired into the importer pipeline before resize with buttons for
each op; ±90° rotations auto-swap the resize dims so output aspect tracks
the rotation. CLI gains --flip-h, --flip-v, --rotate flags using the same
shared module.

8 new unit tests cover identity, each flip, 90/180/270 rotation, the
flipH+flipV ≡ 180° identity, and invalid rotation rejection.
2026-04-17 11:49:54 +07:00
tiennm99 f94f1dc5f5 feat(importer): phase 1 — resize controls with multiple methods
Add W/H inputs, aspect-ratio lock, resampling method dropdown (nearest /
bilinear / box), "Fit to canvas" (respects current origin) and "1:1" reset
buttons to the import panel. Pipeline now runs resize before palette
quantization so dither/skip-matching operate on the final dimensions.

Shared src/lib/image-resize.js is reused by scripts/image-to-colors.js,
which gains --width, --height, --method flags with aspect-preserve when
only one dimension is provided.
2026-04-17 11:43:11 +07:00
tiennm99 a823f8527d fix: address ultrareview findings across backend and frontend
Backend:
- rate-limiter: retryAfter now in seconds; ms-precision lu preserves
  fractional regen residue across calls (C1, C2)
- redis-client: throw on Upstash 200-with-error envelope; redisRaw
  returns body.result (NH1)
- constants: MAX_BATCH_SIZE = MAX_CREDITS = 256 (was 512 vs 256)
- worker: content-length cap, gzip + s-maxage=10 on /api/canvas,
  broadcast via executionCtx.waitUntil with r.ok check (NC2, H4, H5)
- canvas-storage: warn on truncated Upstash read instead of silent
  zero-pad (NH2)
- get-user-id: SHA-256 (16 hex chars) replaces 32-bit string hash;
  missing cf-connecting-ip routes to anon:dev with warn (H1, H2);
  function is now async
- canvas-room: log unclean WS closes and errors; defensive close on
  unexpected client message (NH4, N5)

Frontend:
- pixel buffer capped at MAX_BATCH_SIZE with toast (NC2)
- Submit error UX: toast for 429/413/400/5xx/network; honor
  retryAfter (NC1)
- committedColors allocated upfront so WS updates during initial
  fetch no longer null-deref (NC3)
- handleWheel always renders even when zoom is clamped (C1, C2)
- canvas-decoder throws on truncated input instead of reading past
  end with || 0 (C3)
- WS reconnect refetches canvas to recover missed pixels (C4)
- pixel-buffer Map cache for O(1) getColorAt/pixelCount (NH1)
- cancel in-progress stroke on mode switch (NH3)
- canvas load error overlay with Retry button (NH5)
- DPR-aware canvas sizing (H1)
- onMount cleanup is now sync (no leaked resize listener) (H2)

Tests:
- update for async getUserId, decoder bounds check, redis error
  format; add Upstash error-envelope coverage
2026-04-17 10:14:45 +07:00
tiennm99 8e1f8c4049 test: add Redis integration tests with Testcontainers
Docker-based tests with real Redis verifying:
- BITFIELD write → GETRANGE read round-trip for all 32 colors
- Pixel placement at canvas boundaries and various positions
- Pixel overwrite correctness
- 5-bit boundary isolation (adjacent pixels don't corrupt)
- Batch atomicity (100 pixels in single BITFIELD)
- Rate limiter Lua script: credit grant, deduction, regen, cap, rejection

Test commands: npm test (unit), npm run test:integration (Docker),
npm run test:all (both). 82 total tests.
2026-04-16 22:52:36 +07:00
tiennm99 fcddb1f9c5 test: add unit tests for Durable Object and clean up test setup
- Add CanvasRoom broadcast/close/error tests with mock WebSockets
- Remove @cloudflare/vitest-pool-workers (incompatible with Vitest 4)
- Clean up vitest config (single config, no integration workspace)
- 71 tests across 7 test files, all passing
2026-04-16 22:45:16 +07:00
tiennm99 33cfd3d7b3 test: add comprehensive unit tests with Vitest
65 tests covering canvas decoder, pixel buffer, user ID hashing,
canvas storage (with binary encoding regression test), Redis client
API shape, and worker endpoint validation.
2026-04-16 22:37:12 +07:00