tiennm99/rplace
1
0 Fork 0
mirror of https://github.com/tiennm99/rplace.git synced 2026-10-15 06:16:16 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(canvas): code-review fixes + sync user-facing docs to DO storage

Browse Source 
Fixes from code review of canvas-on-do migration (commit c3f7c02):

- worker.js /api/ws: rewrite request URL to '/ws' so the DO pathname
  switch dispatches correctly. The original c.req.raw kept '/api/ws'
  which the DO never matched → 404 on every WS upgrade.
- migrate-from-upstash.js pickSampleOffsets: use TOTAL_PIXELS - 1 for the
  last byte instead of CANVAS_WIDTH * CANVAS_WIDTH (only correct when
  the canvas is square; constants explicitly invite non-square).
- chunk-storage.js writePixels: clarify atomicity comment — the loop is
  atomic *because it has no awaits*, not because of any implicit DO
  transaction. Added guidance for future maintainers.
- cooldown-store.js tryAcquire: GC sweep wrapped in try/catch so a
  transient failure can't drop the user's allowed: true response.

Docs:
- README.md: drop Upstash from tech stack, redraw architecture,
  document new project layout (durable-objects/lib, admin/), add
  CHUNK_BYTES to configuration table.
- docs/system-architecture.md: full rewrite for DO-storage data flow,
  document SQLite schema, race-safe rate-limit pattern, free-tier table.
- docs/deployment-guide.md: drop Upstash setup, add optional one-shot
  migration runbook, update free-tier table to actual May 2026 limits.

Tests: 112 pass, 6 skipped (pending Phase 4 rewrite via
@cloudflare/vitest-pool-workers). Bundle dry-run clean.

Local wrangler dev smoke test was attempted but the sandboxed env
hangs HTTP requests at the workerd layer (TCP connects, no response).
Routing fix verified by code inspection; user must verify in their
own dev or production.
This commit is contained in:
tiennm99 committed 2026-05-10 00:37:27 +07:00
1 parent cdf2295ef6
commit b890dfb3b7
7 files changed
+271 -137
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/deployment-guide.md
+82 -43
View File
@@ -3,46 +3,66 @@
## Prerequisites
- Node.js 18+
- Cloudflare account (free tier)
- Upstash Redis database (free tier)
- Cloudflare account (Free plan is sufficient)
- `wrangler` CLI (installed as dev dependency)
## Step 1: Create Upstash Redis Database
No external storage to provision. Canvas pixels and rate-limit cooldowns
live inside the `CanvasRoom` Durable Object's SQLite-backed storage.
1. Go to [console.upstash.com](https://console.upstash.com/)
2. Create a new Redis database
3. Choose a region close to your users
4. Copy the **REST URL** and **REST Token**
## Step 2: Configure Secrets
```bash
# Set secrets in Cloudflare (not in code)
npx wrangler secret put UPSTASH_REDIS_REST_URL
npx wrangler secret put UPSTASH_REDIS_REST_TOKEN
```
For local development, create `.dev.vars`:
```
UPSTASH_REDIS_REST_URL=https://your-url.upstash.io
UPSTASH_REDIS_REST_TOKEN=your-token
```
## Step 3: Deploy
## Step 1: Deploy
```bash
npm install
npm run deploy
```
This runs `vite build` (compiles Svelte → dist/) then `wrangler deploy` (uploads Worker + static assets).
This runs `vite build` (compiles Svelte → `dist/`) then `wrangler deploy`
(uploads the Worker, static assets, and Durable Object class).
## Step 4: Verify
The first deploy applies the `wrangler.json` migration that registers
`CanvasRoom` as a SQLite-backed DO class.
1. Visit your Worker URL (e.g., `https://rplace.your-subdomain.workers.dev`)
2. Canvas should load (empty/dark on first visit)
3. Select a color, click to place a pixel
4. Open a second browser tab — pixel should appear via WebSocket
## Step 2: Verify
1. Visit your Worker URL (e.g., `https://rplace.your-subdomain.workers.dev`).
2. Canvas should load (empty / black on first visit — palette index 0).
3. Select a color, click to place a pixel.
4. Open a second browser tab — pixel should appear via WebSocket.
5. `curl -I https://your-url/api/canvas` should report
`cf-cache-status: HIT` after a couple of warm requests (10 s edge cache).
## (Optional) One-Shot Migration from Upstash
Only if you have an existing Upstash-backed deployment to import.
```bash
# 1. Set credentials for the legacy Upstash instance
npx wrangler secret put UPSTASH_REDIS_REST_URL
npx wrangler secret put UPSTASH_REDIS_REST_TOKEN
# 2. Generate and set a migration token
npx wrangler secret put MIGRATION_TOKEN
# Paste a random 32-byte hex value
# 3. Deploy
npm run deploy
# 4. Run the import once
curl -X POST -H "Authorization: Bearer $TOKEN" \
https://your-worker.workers.dev/admin/migrate-from-upstash
# Expect: {"ok":true,"bytes_imported":16777216,"samples_checked":N,"mismatches":[]}
# 5. Verify in browser; wait 7 days as rollback safety
# 6. Run Phase 4 cleanup (see plans/260509-2309-canvas-on-do-storage)
```
After Phase 4 cleanup deletes the migration code, also delete the
secrets:
```bash
npx wrangler secret delete UPSTASH_REDIS_REST_URL
npx wrangler secret delete UPSTASH_REDIS_REST_TOKEN
npx wrangler secret delete MIGRATION_TOKEN
```
## Custom Domain
@@ -53,23 +73,42 @@ npx wrangler domains add rplace.yourdomain.com
## Monitoring
- **Cloudflare dashboard**: Worker analytics, request logs, DO metrics
- **Upstash console**: Redis command count, memory usage, latency
- **Cloudflare dashboard**:
- Workers analytics — requests/day, errors, CPU time
- Durable Object metrics — storage size, request rate
- Cache analytics — `cf-cache-status` HIT ratio on `/api/canvas`
## Cost Estimates (Free Tier)
## Free-tier Footprint
| Resource | Free Limit | rplace Usage |
| Resource | Free Cap (May 2026) | rplace at hobby scale |
|---|---|---|
| CF Workers | 100K requests/day | Canvas reads + pixel placements |
| CF Durable Objects | Free with Workers | WebSocket connections |
| Upstash Redis | 10K commands/day | BITFIELD reads/writes + cooldown SET NX EX |
| Workers requests | 100,000 / day | ~100 / day @ 50 users |
| DO storage / object | 10 GB | 16 MB canvas |
| DO storage / account | 5 GB | 16 MB total |
| BLOB row size | 2 MB | 64 KB chunks (32× under) |
| Per-DO request rate | 1,000 / s soft | ~1 / s |
For hobby traffic (< few hundred users/day), free tiers are sufficient. Upstash pay-as-you-go ($0.2/100K commands) is the first thing to hit limits.
Bandwidth is unlimited on Workers. With the 10 s edge cache on
`/api/canvas`, the dominant request driver is `/api/place` (1 per
placement). At 1-req-per-second-per-user rate-limit, 50 concurrent
users × 24h × 3600s = 4.3 M theoretical max — but realistic hobby
sessions stay well under 100K/day.
## Troubleshooting
- **Canvas loads empty**: Check Upstash credentials in secrets
- **Pixels don't persist**: Verify BITFIELD support — test with `redis-cli BITFIELD rplace:canvas:v2 SET u8 #0 42`
- **Old 32-color canvas still visible after deploy**: the canvas key is versioned (`rplace:canvas:v2`). The old `rplace:canvas` key is orphaned — run `redis-cli DEL rplace:canvas` once to reclaim memory.
- **WebSocket not connecting**: Ensure Durable Object migration ran (check `wrangler.json` migrations)
- **Rate limiting not working**: Verify `SET key value NX EX 1` returns `"OK"` / `null` as expected on your Upstash tier
- **Canvas loads empty**: expected on first deploy — DO `canvas_chunks`
table is empty until pixels are placed (or migration runs).
- **WebSocket not connecting**: verify the wrangler migration applied
via `wrangler tail` — should see no errors on DO instantiation.
- **`cf-cache-status` shows MISS**: edge caching may need an extra
`caches.default.put` wrap if `Cache-Control` headers aren't honored
through the worker → DO → response chain. Verify with two consecutive
`curl -I` requests; second should HIT.
- **Migration import fails with `size_mismatch`**: the legacy Upstash
data isn't 16 MB. Resize CHUNK constants or delete the partial Upstash
data and start fresh.
- **`already_populated` from migration endpoint**: pass `?force=1` to
overwrite. Use only when you're certain.
- **Storage billing meter ticking up**: per-account 5 GB free cap. A
16 MB canvas is harmless; the worry only appears if you stand up many
rooms or hit a runaway insert.
docs/system-architecture.md
+127 -44
View File
@@ -2,80 +2,163 @@
## Overview
rplace is a collaborative pixel canvas deployed as a single Cloudflare Worker. The frontend (Svelte SPA) is served as static assets, the API (Hono) handles pixel operations, and a Durable Object manages WebSocket broadcasting.
rplace is a collaborative pixel canvas deployed as a single Cloudflare Worker.
The frontend (Svelte SPA) is served as static assets, and a single
**Durable Object** (`CanvasRoom`, `idFromName('main')`) owns canvas state,
rate-limit cooldowns, and the WebSocket broadcast hub. The Worker is a
thin validation/routing proxy.
## Component Map
```
Browser (Svelte SPA + WebSocket)
| GET /api/canvas → 16 MB binary, edge-cached 10s
| POST /api/place → batch pixel placement (validated at edge)
| WS /api/ws → real-time pixel deltas
v
Cloudflare Worker (Hono — thin proxy)
└─▶ CanvasRoom Durable Object (single instance, 'main')
├── canvas_chunks SQLite BLOB rows (256 × 64 KB = 16 MB)
├── cooldowns SQLite TTL rows (1s rate-limit)
└── WebSocket hub Hibernation API broadcasts pixel deltas
```
## Data Flow
### Pixel Placement
```
1. User draws on canvas → optimistic render into a pending buffer
2. User hits Submit → POST /api/place { pixels: [{x, y, color}] }
3. Worker validates input (bounds, types, batch size ≤ 2048)
4. Worker checks cooldown via SET NX EX (atomic per-user lock)
5. Worker writes pixels via Redis BITFIELD (atomic batch)
6. Worker sends pixels to Durable Object /broadcast
7. Durable Object fans out to all WebSocket clients
8. Response: { ok: true }
1. User draws on canvas → optimistic render into a pending buffer.
2. User hits Submit → POST /api/place { pixels: [{x, y, color}, ...] }.
3. Worker validates input (bounds, types, batch ≤ 2048, body ≤ 128 KB).
4. Worker resolves userId from CF-Connecting-IP and forwards to DO /place.
5. DO atomically:
a. cooldowns.tryAcquire(userId, 1s) — UPDATE expired or INSERT new row.
On conflict (active claim), responds 429.
b. chunk_storage.writePixels — group pixels by chunk_id, read each
touched chunk's BLOB, modify in memory, INSERT OR REPLACE.
c. Broadcast `{type:'pixels', pixels}` to all hibernating WebSockets.
6. Response: { ok: true } (or { error, retryAfter } on rate limit).
```
The DO is single-threaded; the entire 5a-c sequence runs without
preemption, so cooldown check + write + broadcast are effectively atomic
without explicit transactions.
### Canvas Loading
```
1. Client fetches GET /api/canvas
2. Worker reads Redis key via GETRANGE → raw binary (16 MB, gzip-compressed by CF edge)
3. Client receives 16 MB of bytes — each byte is a palette index (u8, byte-aligned)
4. Client maps indices → RGBA ImageData via COLORS_RGBA lookup
5. Renders onto HTML5 Canvas with OffscreenCanvas
1. Client fetches GET /api/canvas (10s edge-cache; CDN serves 99% of hits).
2. Worker forwards to DO /canvas on miss.
3. DO chunk_storage.readAllChunks: SELECT all canvas_chunks rows, copy each
BLOB into a single 16 MB Uint8Array offset by chunk_id × CHUNK_BYTES.
4. Client receives raw bytes (CF auto-gzip), maps each byte → COLORS_RGBA.
5. Renders via OffscreenCanvas + ImageData.
```
### Real-time Updates
```
1. Client connects WS /api/ws
2. Worker upgrades to Durable Object WebSocket
3. DO stores connection in memory Set
4. On pixel placement → DO broadcasts JSON to all connections
5. Client updates local ImageData + re-renders
6. On disconnect → auto-reconnect with exponential backoff (1s→30s)
1. Client connects WS /api/ws.
2. Worker forwards upgrade to DO /ws (URL rewritten to /ws so the DO
pathname dispatch matches).
3. DO state.acceptWebSocket(server) — Hibernation API; idle sockets
survive DO eviction at zero CPU cost.
4. On placement → DO iterates state.getWebSockets() and sends JSON.
5. Client merges received pixels into local ImageData; re-renders dirty
region.
6. On disconnect → exponential-backoff reconnect (1 s → 30 s).
```
## Storage
### Redis STRING / BITFIELD (Canvas)
### `canvas_chunks` (canvas pixels)
- Key: `rplace:canvas:v2` (bumped from the old `rplace:canvas` so the 32-color/2048² data is ignored on rollout)
- Encoding: 8 bits per pixel (u8), 256-color palette — byte-aligned, so raw Redis bytes are the pixel indices directly
- Size: `4096 × 4096 × 1 = 16,777,216 bytes` (16 MB)
- Offset: `y * CANVAS_WIDTH + x`
- Atomic batch writes: single BITFIELD command chaining `SET u8 #offset color` per pixel
- Reads via GETRANGE return the whole buffer; Cloudflare edge handles gzip
| Column | Type | Notes |
|---|---|---|
| `chunk_id` | INTEGER PRIMARY KEY | 0 .. CHUNK_COUNT − 1 |
| `bytes` | BLOB NOT NULL | Exactly CHUNK_BYTES bytes (last chunk may be short) |
### Redis STRING (Cooldown)
- Linear byte layout: `offset = y * CANVAS_WIDTH + x`
- `chunk_id = floor(offset / CHUNK_BYTES)`
- `CHUNK_BYTES = 65536`, `CHUNK_COUNT = ceil(TOTAL_PIXELS / CHUNK_BYTES)`
- **Lazy initialization:** missing rows read as zero-filled buffers. Resizing
the canvas is just a constants change — new chunks materialize on first
read. No migration script needed.
- **Hard caps:** CF DO BLOB row size 2 MB → 64 KB chunks have 32× headroom.
- Key pattern: `cooldown:{userId}`
- Value: `"1"` (presence is the signal; content is irrelevant)
- TTL: `REQUEST_COOLDOWN_SEC` (1s) — auto-expires, no explicit cleanup
- Atomic via `SET key "1" NX EX 1`
### `cooldowns` (rate-limit windows)
| Column | Type | Notes |
|---|---|---|
| `user_id` | TEXT PRIMARY KEY | Hashed CF-Connecting-IP |
| `expires_at` | INTEGER NOT NULL | ms epoch; row becomes stale past this point |
- `idx_cooldowns_expires` keeps lazy GC sweeps cheap.
- GC runs at 1% sample rate inside `tryAcquire`, wrapped in `try/catch` —
best-effort; never blocks the rate-limit decision.
## Rate Limiting
Fixed-window cooldown, one request per second per user:
Single-row per user, 1 s window. Race-safe inside the DO via:
```
On placement request:
1. SET cooldown:{userId} "1" NX EX 1
2. If reply == "OK" → allow (key set, TTL 1s)
3. If reply == null → reject (429, retryAfter = 1)
UPDATE cooldowns SET expires_at = ? WHERE user_id = ? AND expires_at <= ?
→ if rowsWritten > 0: claim acquired (existing row was expired).
→ else: INSERT INTO cooldowns ...
if INSERT throws on PK conflict → claim denied (active row exists).
```
Batch size is independent of the cooldown; it is validated separately
(MAX_BATCH_SIZE = 2048). Anonymous identity via CF-Connecting-IP hash.
Hardness comes from the DO single-threaded model; no SQL-level locking
needed.
## Migration Endpoint (transitional)
`POST /admin/migrate-from-upstash` (token-gated): one-shot importer that
pulls the legacy Upstash canvas via `lib/canvas-storage.js` (4 chunked
GETRANGE calls) and posts the raw 16 MB bytes to DO `/import`. The DO
splits into CHUNK_COUNT BLOB rows in a single sync transaction, then
the worker round-trips a sample-byte verification.
Removed in Phase 4 of `plans/260509-2309-canvas-on-do-storage` along with
`@upstash/redis` and `ioredis` dependencies.
## Free-tier Footprint (CF, 2026)
| Resource | Quota | rplace usage at hobby scale | Headroom |
|---|---|---|---|
| Workers requests | 100K/day | ~100/day (50 users) | 1000× |
| DO storage / object | 10 GB | 16 MB | 600× |
| DO storage / account | 5 GB | 16 MB | 300× |
| BLOB row size | 2 MB | 64 KB chunks | 32× |
| Per-DO request rate | 1,000 / s soft cap | <1 / s | 1000× |
| WS connections / DO | tens of thousands | ~50 | huge |
`/api/canvas` carries `Cache-Control: public, s-maxage=10, stale-while-revalidate=30`.
Verify edge HIT in production via `cf-cache-status: HIT`; if absent, wrap
the worker handler with the Cache API to enforce caching.
## Security
- **Rate limiting**: Atomic `SET NX EX` prevents race conditions
- **Identity**: CF-Connecting-IP (set by Cloudflare, unspoofable)
- **Input validation**: Bounds checking, type checking, integer validation on all pixel data
- **Batch cap**: Max 2048 pixels per request + request body size guard
- **DO isolation**: /broadcast route only reachable via DO stub, not externally
- **Rate limiting**: race-safe at the actor (single-threaded DO).
- **Identity**: CF-Connecting-IP hashed to userId — unspoofable.
- **Input validation**: strict bounds + type checks at the worker edge,
re-validated at the DO trust boundary.
- **Body cap**: ~128 KB per `/api/place` (2048 pixels × ~64 B JSON each).
- **Migration endpoint**: Bearer-token gated. Token-comparison is direct
string equality — adequate at hobby scale; consider constant-time
comparison if exposure profile changes.
- **DO isolation**: `/canvas`, `/place`, `/import`, `/ws` are intra-DO
paths only — not internet-reachable except via the worker.
## Operational Notes
- **Single-region.** DO is anchored to one Cloudflare colo; latency for
far-away users mirrors the previous Upstash Redis topology — no
regression.
- **DO eviction mid-place** is safe: the SQLite write commits before
broadcast. If the DO evicts before broadcast fires, client WS receives
no message but auto-reconnects and re-fetches `/api/canvas`. No data
loss; minor latency tail.
- **Storage billing.** Per-account 5 GB free cap on Free plan as of
Jan 7 2026 — current 16 MB is far under. Monitor on resize.