mirror of
https://github.com/tiennm99/rplace.git
synced 2026-10-11 03:13:48 +00:00
fix(canvas): code-review fixes + sync user-facing docs to DO storage
Fixes from code review of canvas-on-do migration (commit c3f7c02):
- worker.js /api/ws: rewrite request URL to '/ws' so the DO pathname
switch dispatches correctly. The original c.req.raw kept '/api/ws'
which the DO never matched → 404 on every WS upgrade.
- migrate-from-upstash.js pickSampleOffsets: use TOTAL_PIXELS - 1 for the
last byte instead of CANVAS_WIDTH * CANVAS_WIDTH (only correct when
the canvas is square; constants explicitly invite non-square).
- chunk-storage.js writePixels: clarify atomicity comment — the loop is
atomic *because it has no awaits*, not because of any implicit DO
transaction. Added guidance for future maintainers.
- cooldown-store.js tryAcquire: GC sweep wrapped in try/catch so a
transient failure can't drop the user's allowed: true response.
Docs:
- README.md: drop Upstash from tech stack, redraw architecture,
document new project layout (durable-objects/lib, admin/), add
CHUNK_BYTES to configuration table.
- docs/system-architecture.md: full rewrite for DO-storage data flow,
document SQLite schema, race-safe rate-limit pattern, free-tier table.
- docs/deployment-guide.md: drop Upstash setup, add optional one-shot
migration runbook, update free-tier table to actual May 2026 limits.
Tests: 112 pass, 6 skipped (pending Phase 4 rewrite via
@cloudflare/vitest-pool-workers). Bundle dry-run clean.
Local wrangler dev smoke test was attempted but the sandboxed env
hangs HTTP requests at the workerd layer (TCP connects, no response).
Routing fix verified by code inspection; user must verify in their
own dev or production.
This commit is contained in:
1 parent
cdf2295ef6
commit
b890dfb3b7
7 files changed
+271
-137
No files matched your search
@@ -18,60 +18,54 @@ A collaborative pixel art canvas inspired by [Reddit's r/place](https://www.redd
|
||||
|---|---|
|
||||
| Frontend | [Svelte 5](https://svelte.dev/) (runes) + HTML5 Canvas |
|
||||
| Backend | [Hono](https://hono.dev/) on Cloudflare Workers |
|
||||
| Real-time | WebSocket via Cloudflare Durable Objects |
|
||||
| Storage | [Upstash Redis](https://upstash.com/) (BITFIELD for canvas, SET NX EX for rate limiting) |
|
||||
| Real-time | WebSocket via Cloudflare Durable Objects (Hibernation API) |
|
||||
| Storage | Durable Object SQLite — chunked BLOB rows for canvas, TTL rows for cooldowns |
|
||||
| Build | [Vite](https://vite.dev/) |
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Browser (Svelte SPA + WebSocket)
|
||||
| GET /api/canvas → full canvas binary (16MB raw, ~5MB gzip)
|
||||
| POST /api/place → batch pixel placement
|
||||
| WS /api/ws → Durable Object broadcast room
|
||||
| GET /api/canvas → full canvas binary (16 MB, edge-cached 10s)
|
||||
| POST /api/place → batch pixel placement (validated at edge)
|
||||
| WS /api/ws → CanvasRoom Durable Object broadcast
|
||||
v
|
||||
Cloudflare Worker (Hono)
|
||||
├── Canvas API (read/write pixels via Redis BITFIELD)
|
||||
├── Rate Limiter (SET NX EX — atomic per-user cooldown)
|
||||
└── Durable Object (WebSocket broadcast to all clients)
|
||||
↕
|
||||
Upstash Redis
|
||||
├── STRING "canvas:v2" (1 byte per pixel, 4096×4096 = 16 MB)
|
||||
└── STRING "cooldown:{userId}" (1s TTL, blocks repeat requests)
|
||||
Cloudflare Worker (Hono — thin proxy)
|
||||
└─▶ CanvasRoom Durable Object (single instance, idFromName('main'))
|
||||
├── canvas_chunks SQLite BLOB rows × 256 (64 KB each = 16 MB)
|
||||
├── cooldowns SQLite TTL rows (1s rate-limit, lazy GC)
|
||||
└── WebSocket hub Hibernation API broadcasts pixel deltas
|
||||
```
|
||||
|
||||
`CHUNK_COUNT = ceil(CANVAS_WIDTH × CANVAS_HEIGHT / CHUNK_BYTES)` — bumping
|
||||
canvas dimensions in `src/lib/constants.js` and redeploying lazy-allocates new
|
||||
chunks on first read. See [`docs/canvas-resize-procedure.md`](docs/canvas-resize-procedure.md).
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- [Node.js](https://nodejs.org/) 18+
|
||||
- [Cloudflare account](https://dash.cloudflare.com/) (free tier works)
|
||||
- [Upstash Redis](https://console.upstash.com/) database (free tier works)
|
||||
- [Cloudflare account](https://dash.cloudflare.com/) (free tier works — Workers Free + DO SQLite Free)
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
# Clone and install
|
||||
git clone <repo-url>
|
||||
cd rplace
|
||||
npm install
|
||||
|
||||
# Configure environment
|
||||
cp .env.example .env
|
||||
# Edit .env with your Upstash Redis credentials
|
||||
|
||||
# For wrangler (Cloudflare Workers CLI)
|
||||
npx wrangler secret put UPSTASH_REDIS_REST_URL
|
||||
npx wrangler secret put UPSTASH_REDIS_REST_TOKEN
|
||||
```
|
||||
|
||||
No external storage to configure. The canvas and rate-limit state live inside
|
||||
the Durable Object.
|
||||
|
||||
### Development
|
||||
|
||||
```bash
|
||||
# Run worker locally (serves both API and frontend)
|
||||
# Run worker locally (serves API + static frontend)
|
||||
npm run dev
|
||||
|
||||
# Or run frontend and worker separately
|
||||
# Or split frontend + worker
|
||||
npm run dev:client # Vite dev server on :5173 (proxies /api to :8787)
|
||||
npm run dev # Wrangler dev server on :8787
|
||||
```
|
||||
@@ -86,15 +80,21 @@ npm run deploy # Builds frontend + deploys worker to Cloudflare
|
||||
|
||||
```
|
||||
src/
|
||||
├── worker.js # Hono API entry point
|
||||
├── worker.js # Hono entry — thin proxy + edge validation
|
||||
├── admin/
|
||||
│ └── migrate-from-upstash.js # One-shot Upstash → DO importer (token-gated)
|
||||
├── durable-objects/
|
||||
│ └── canvas-room.js # WebSocket broadcast room
|
||||
│ ├── canvas-room.js # DO: storage + cooldown + WS hub
|
||||
│ └── lib/
|
||||
│ ├── schema.js # Idempotent CREATE TABLE
|
||||
│ ├── chunk-storage.js # BLOB chunk read/write/import
|
||||
│ └── cooldown-store.js # Rate-limit acquire + lazy GC
|
||||
├── lib/
|
||||
│ ├── constants.js # Config, palette, limits (shared)
|
||||
│ ├── redis-client.js # Upstash Redis factory
|
||||
│ ├── canvas-storage.js # BITFIELD read/write
|
||||
│ ├── canvas-decoder.js # Raw bytes → RGBA (client-side; u8 = identity indices)
|
||||
│ ├── rate-limiter.js # SET NX EX cooldown
|
||||
│ ├── constants.js # CANVAS_WIDTH/HEIGHT, CHUNK_BYTES, palette
|
||||
│ ├── canvas-decoder.js # Raw bytes → RGBA (client-side)
|
||||
│ ├── canvas-storage.js # Legacy Upstash reader (used by migration only)
|
||||
│ ├── redis-client.js # Legacy Upstash REST helpers (migration only)
|
||||
│ ├── rate-limiter.js # Legacy Upstash cooldown (orphaned, awaits removal)
|
||||
│ ├── image-uploader.js # Browser-side batched uploader
|
||||
│ └── get-user-id.js # IP-based identity
|
||||
├── client/
|
||||
@@ -103,7 +103,7 @@ src/
|
||||
│ ├── app.css # Global styles
|
||||
│ └── components/
|
||||
│ ├── CanvasRenderer.svelte # Canvas + zoom/pan + touch
|
||||
│ ├── ColorPicker.svelte # Favorites strip + 256-color grid + custom picker
|
||||
│ ├── ColorPicker.svelte # Favorites + 256-color grid + custom picker
|
||||
│ ├── CanvasControls.svelte # Zoom buttons + coordinates
|
||||
│ ├── DrawToolbar.svelte # Paint / submit / undo / redo
|
||||
│ └── ImageImporter.svelte # Image-to-canvas uploader
|
||||
@@ -114,7 +114,7 @@ src/
|
||||
|
||||
### `GET /api/canvas`
|
||||
|
||||
Returns the full canvas as raw binary (1 byte per pixel, 16 MB — Cloudflare gzips it on the edge).
|
||||
Returns the full canvas as raw binary (1 byte per pixel, 16 MB — Cloudflare gzips it on the edge). Cached for 10s at the edge.
|
||||
|
||||
### `POST /api/place`
|
||||
|
||||
@@ -143,6 +143,13 @@ WebSocket for real-time pixel updates. Messages are JSON:
|
||||
{ "type": "pixels", "pixels": [{ "x": 100, "y": 200, "color": 27 }] }
|
||||
```
|
||||
|
||||
### `POST /admin/migrate-from-upstash` (transitional)
|
||||
|
||||
Token-gated one-shot endpoint that pulls the canvas from a legacy Upstash
|
||||
Redis instance and imports it into the Durable Object. Slated for removal
|
||||
after the production migration completes (Phase 4 of
|
||||
[`plans/260509-2309-canvas-on-do-storage`](plans/260509-2309-canvas-on-do-storage)).
|
||||
|
||||
## Configuration
|
||||
|
||||
Key constants in `src/lib/constants.js`:
|
||||
@@ -152,22 +159,19 @@ Key constants in `src/lib/constants.js`:
|
||||
| `CANVAS_WIDTH` | 4096 | Canvas width in pixels |
|
||||
| `CANVAS_HEIGHT` | 4096 | Canvas height in pixels |
|
||||
| `MAX_COLORS` | 256 | Number of palette entries |
|
||||
| `BITS_PER_PIXEL` | 8 | Byte-aligned (storage = W × H bytes) |
|
||||
| `MAX_BATCH_SIZE` | 2048 | Max pixels per placement request |
|
||||
| `REQUEST_COOLDOWN_SEC` | 1 | Minimum seconds between requests per user |
|
||||
| `CHUNK_BYTES` | 65536 | Bytes per SQLite BLOB chunk (must stay ≤ 2 MB CF DO row cap) |
|
||||
| `CHUNK_COUNT` | derived | `ceil(TOTAL_PIXELS / CHUNK_BYTES)` — auto-recomputed on resize |
|
||||
|
||||
## Credits & References
|
||||
|
||||
- [Reddit on Building & Scaling r/place (Fastly)](https://www.fastly.com/blog/reddit-on-building-scaling-rplace)
|
||||
- [Engineering Behind r/place (Sai Kumar Chintada)](https://saikumarchintada.medium.com/engineering-behind-r-place-a7eb53bcf5f1)
|
||||
- [Redis Place: Building r/place with 9 Redis Data Structures (Mehdi Amrane)](https://dev.to/mehdi/redis-place-building-rplace-with-9-redis-data-structures-3lj8)
|
||||
- [Redis Pixel War (Alfredo Salzillo)](https://dev.to/alfredosalzillo/redis-pixel-war-3i7a)
|
||||
- [Redis BITFIELD Command](https://redis.io/docs/latest/commands/bitfield/)
|
||||
- [Cloudflare Durable Objects](https://developers.cloudflare.com/durable-objects/)
|
||||
- [SQLite-backed Durable Object Storage](https://developers.cloudflare.com/durable-objects/api/sqlite-storage-api/)
|
||||
- [reddit-plugin-place-opensource](https://github.com/reddit-archive/reddit-plugin-place-opensource)
|
||||
- [rPlace by anthonytedja](https://github.com/anthonytedja/rPlace)
|
||||
- [redis-place by mehdiamrane](https://github.com/mehdiamrane/redis-place)
|
||||
- [redis-challenge by alfredosalzillo](https://github.com/alfredosalzillo/redis-challenge)
|
||||
- [place by dynastic](https://github.com/dynastic/place)
|
||||
- [rplace.live](https://rplace.live/) — original 32-color palette reference (since superseded by our 256-color HSL wheel)
|
||||
|
||||
## License
|
||||
|
||||
+82
-43
@@ -3,46 +3,66 @@
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 18+
|
||||
- Cloudflare account (free tier)
|
||||
- Upstash Redis database (free tier)
|
||||
- Cloudflare account (Free plan is sufficient)
|
||||
- `wrangler` CLI (installed as dev dependency)
|
||||
|
||||
## Step 1: Create Upstash Redis Database
|
||||
No external storage to provision. Canvas pixels and rate-limit cooldowns
|
||||
live inside the `CanvasRoom` Durable Object's SQLite-backed storage.
|
||||
|
||||
1. Go to [console.upstash.com](https://console.upstash.com/)
|
||||
2. Create a new Redis database
|
||||
3. Choose a region close to your users
|
||||
4. Copy the **REST URL** and **REST Token**
|
||||
|
||||
## Step 2: Configure Secrets
|
||||
|
||||
```bash
|
||||
# Set secrets in Cloudflare (not in code)
|
||||
npx wrangler secret put UPSTASH_REDIS_REST_URL
|
||||
npx wrangler secret put UPSTASH_REDIS_REST_TOKEN
|
||||
```
|
||||
|
||||
For local development, create `.dev.vars`:
|
||||
|
||||
```
|
||||
UPSTASH_REDIS_REST_URL=https://your-url.upstash.io
|
||||
UPSTASH_REDIS_REST_TOKEN=your-token
|
||||
```
|
||||
|
||||
## Step 3: Deploy
|
||||
## Step 1: Deploy
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run deploy
|
||||
```
|
||||
|
||||
This runs `vite build` (compiles Svelte → dist/) then `wrangler deploy` (uploads Worker + static assets).
|
||||
This runs `vite build` (compiles Svelte → `dist/`) then `wrangler deploy`
|
||||
(uploads the Worker, static assets, and Durable Object class).
|
||||
|
||||
## Step 4: Verify
|
||||
The first deploy applies the `wrangler.json` migration that registers
|
||||
`CanvasRoom` as a SQLite-backed DO class.
|
||||
|
||||
1. Visit your Worker URL (e.g., `https://rplace.your-subdomain.workers.dev`)
|
||||
2. Canvas should load (empty/dark on first visit)
|
||||
3. Select a color, click to place a pixel
|
||||
4. Open a second browser tab — pixel should appear via WebSocket
|
||||
## Step 2: Verify
|
||||
|
||||
1. Visit your Worker URL (e.g., `https://rplace.your-subdomain.workers.dev`).
|
||||
2. Canvas should load (empty / black on first visit — palette index 0).
|
||||
3. Select a color, click to place a pixel.
|
||||
4. Open a second browser tab — pixel should appear via WebSocket.
|
||||
5. `curl -I https://your-url/api/canvas` should report
|
||||
`cf-cache-status: HIT` after a couple of warm requests (10 s edge cache).
|
||||
|
||||
## (Optional) One-Shot Migration from Upstash
|
||||
|
||||
Only if you have an existing Upstash-backed deployment to import.
|
||||
|
||||
```bash
|
||||
# 1. Set credentials for the legacy Upstash instance
|
||||
npx wrangler secret put UPSTASH_REDIS_REST_URL
|
||||
npx wrangler secret put UPSTASH_REDIS_REST_TOKEN
|
||||
|
||||
# 2. Generate and set a migration token
|
||||
npx wrangler secret put MIGRATION_TOKEN
|
||||
# Paste a random 32-byte hex value
|
||||
|
||||
# 3. Deploy
|
||||
npm run deploy
|
||||
|
||||
# 4. Run the import once
|
||||
curl -X POST -H "Authorization: Bearer $TOKEN" \
|
||||
https://your-worker.workers.dev/admin/migrate-from-upstash
|
||||
# Expect: {"ok":true,"bytes_imported":16777216,"samples_checked":N,"mismatches":[]}
|
||||
|
||||
# 5. Verify in browser; wait 7 days as rollback safety
|
||||
# 6. Run Phase 4 cleanup (see plans/260509-2309-canvas-on-do-storage)
|
||||
```
|
||||
|
||||
After Phase 4 cleanup deletes the migration code, also delete the
|
||||
secrets:
|
||||
```bash
|
||||
npx wrangler secret delete UPSTASH_REDIS_REST_URL
|
||||
npx wrangler secret delete UPSTASH_REDIS_REST_TOKEN
|
||||
npx wrangler secret delete MIGRATION_TOKEN
|
||||
```
|
||||
|
||||
## Custom Domain
|
||||
|
||||
@@ -53,23 +73,42 @@ npx wrangler domains add rplace.yourdomain.com
|
||||
|
||||
## Monitoring
|
||||
|
||||
- **Cloudflare dashboard**: Worker analytics, request logs, DO metrics
|
||||
- **Upstash console**: Redis command count, memory usage, latency
|
||||
- **Cloudflare dashboard**:
|
||||
- Workers analytics — requests/day, errors, CPU time
|
||||
- Durable Object metrics — storage size, request rate
|
||||
- Cache analytics — `cf-cache-status` HIT ratio on `/api/canvas`
|
||||
|
||||
## Cost Estimates (Free Tier)
|
||||
## Free-tier Footprint
|
||||
|
||||
| Resource | Free Limit | rplace Usage |
|
||||
| Resource | Free Cap (May 2026) | rplace at hobby scale |
|
||||
|---|---|---|
|
||||
| CF Workers | 100K requests/day | Canvas reads + pixel placements |
|
||||
| CF Durable Objects | Free with Workers | WebSocket connections |
|
||||
| Upstash Redis | 10K commands/day | BITFIELD reads/writes + cooldown SET NX EX |
|
||||
| Workers requests | 100,000 / day | ~100 / day @ 50 users |
|
||||
| DO storage / object | 10 GB | 16 MB canvas |
|
||||
| DO storage / account | 5 GB | 16 MB total |
|
||||
| BLOB row size | 2 MB | 64 KB chunks (32× under) |
|
||||
| Per-DO request rate | 1,000 / s soft | ~1 / s |
|
||||
|
||||
For hobby traffic (< few hundred users/day), free tiers are sufficient. Upstash pay-as-you-go ($0.2/100K commands) is the first thing to hit limits.
|
||||
Bandwidth is unlimited on Workers. With the 10 s edge cache on
|
||||
`/api/canvas`, the dominant request driver is `/api/place` (1 per
|
||||
placement). At 1-req-per-second-per-user rate-limit, 50 concurrent
|
||||
users × 24h × 3600s = 4.3 M theoretical max — but realistic hobby
|
||||
sessions stay well under 100K/day.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Canvas loads empty**: Check Upstash credentials in secrets
|
||||
- **Pixels don't persist**: Verify BITFIELD support — test with `redis-cli BITFIELD rplace:canvas:v2 SET u8 #0 42`
|
||||
- **Old 32-color canvas still visible after deploy**: the canvas key is versioned (`rplace:canvas:v2`). The old `rplace:canvas` key is orphaned — run `redis-cli DEL rplace:canvas` once to reclaim memory.
|
||||
- **WebSocket not connecting**: Ensure Durable Object migration ran (check `wrangler.json` migrations)
|
||||
- **Rate limiting not working**: Verify `SET key value NX EX 1` returns `"OK"` / `null` as expected on your Upstash tier
|
||||
- **Canvas loads empty**: expected on first deploy — DO `canvas_chunks`
|
||||
table is empty until pixels are placed (or migration runs).
|
||||
- **WebSocket not connecting**: verify the wrangler migration applied
|
||||
via `wrangler tail` — should see no errors on DO instantiation.
|
||||
- **`cf-cache-status` shows MISS**: edge caching may need an extra
|
||||
`caches.default.put` wrap if `Cache-Control` headers aren't honored
|
||||
through the worker → DO → response chain. Verify with two consecutive
|
||||
`curl -I` requests; second should HIT.
|
||||
- **Migration import fails with `size_mismatch`**: the legacy Upstash
|
||||
data isn't 16 MB. Resize CHUNK constants or delete the partial Upstash
|
||||
data and start fresh.
|
||||
- **`already_populated` from migration endpoint**: pass `?force=1` to
|
||||
overwrite. Use only when you're certain.
|
||||
- **Storage billing meter ticking up**: per-account 5 GB free cap. A
|
||||
16 MB canvas is harmless; the worry only appears if you stand up many
|
||||
rooms or hit a runaway insert.
|
||||
+127
-44
@@ -2,80 +2,163 @@
|
||||
|
||||
## Overview
|
||||
|
||||
rplace is a collaborative pixel canvas deployed as a single Cloudflare Worker. The frontend (Svelte SPA) is served as static assets, the API (Hono) handles pixel operations, and a Durable Object manages WebSocket broadcasting.
|
||||
rplace is a collaborative pixel canvas deployed as a single Cloudflare Worker.
|
||||
The frontend (Svelte SPA) is served as static assets, and a single
|
||||
**Durable Object** (`CanvasRoom`, `idFromName('main')`) owns canvas state,
|
||||
rate-limit cooldowns, and the WebSocket broadcast hub. The Worker is a
|
||||
thin validation/routing proxy.
|
||||
|
||||
## Component Map
|
||||
|
||||
```
|
||||
Browser (Svelte SPA + WebSocket)
|
||||
| GET /api/canvas → 16 MB binary, edge-cached 10s
|
||||
| POST /api/place → batch pixel placement (validated at edge)
|
||||
| WS /api/ws → real-time pixel deltas
|
||||
v
|
||||
Cloudflare Worker (Hono — thin proxy)
|
||||
└─▶ CanvasRoom Durable Object (single instance, 'main')
|
||||
├── canvas_chunks SQLite BLOB rows (256 × 64 KB = 16 MB)
|
||||
├── cooldowns SQLite TTL rows (1s rate-limit)
|
||||
└── WebSocket hub Hibernation API broadcasts pixel deltas
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Pixel Placement
|
||||
|
||||
```
|
||||
1. User draws on canvas → optimistic render into a pending buffer
|
||||
2. User hits Submit → POST /api/place { pixels: [{x, y, color}] }
|
||||
3. Worker validates input (bounds, types, batch size ≤ 2048)
|
||||
4. Worker checks cooldown via SET NX EX (atomic per-user lock)
|
||||
5. Worker writes pixels via Redis BITFIELD (atomic batch)
|
||||
6. Worker sends pixels to Durable Object /broadcast
|
||||
7. Durable Object fans out to all WebSocket clients
|
||||
8. Response: { ok: true }
|
||||
1. User draws on canvas → optimistic render into a pending buffer.
|
||||
2. User hits Submit → POST /api/place { pixels: [{x, y, color}, ...] }.
|
||||
3. Worker validates input (bounds, types, batch ≤ 2048, body ≤ 128 KB).
|
||||
4. Worker resolves userId from CF-Connecting-IP and forwards to DO /place.
|
||||
5. DO atomically:
|
||||
a. cooldowns.tryAcquire(userId, 1s) — UPDATE expired or INSERT new row.
|
||||
On conflict (active claim), responds 429.
|
||||
b. chunk_storage.writePixels — group pixels by chunk_id, read each
|
||||
touched chunk's BLOB, modify in memory, INSERT OR REPLACE.
|
||||
c. Broadcast `{type:'pixels', pixels}` to all hibernating WebSockets.
|
||||
6. Response: { ok: true } (or { error, retryAfter } on rate limit).
|
||||
```
|
||||
|
||||
The DO is single-threaded; the entire 5a-c sequence runs without
|
||||
preemption, so cooldown check + write + broadcast are effectively atomic
|
||||
without explicit transactions.
|
||||
|
||||
### Canvas Loading
|
||||
|
||||
```
|
||||
1. Client fetches GET /api/canvas
|
||||
2. Worker reads Redis key via GETRANGE → raw binary (16 MB, gzip-compressed by CF edge)
|
||||
3. Client receives 16 MB of bytes — each byte is a palette index (u8, byte-aligned)
|
||||
4. Client maps indices → RGBA ImageData via COLORS_RGBA lookup
|
||||
5. Renders onto HTML5 Canvas with OffscreenCanvas
|
||||
1. Client fetches GET /api/canvas (10s edge-cache; CDN serves 99% of hits).
|
||||
2. Worker forwards to DO /canvas on miss.
|
||||
3. DO chunk_storage.readAllChunks: SELECT all canvas_chunks rows, copy each
|
||||
BLOB into a single 16 MB Uint8Array offset by chunk_id × CHUNK_BYTES.
|
||||
4. Client receives raw bytes (CF auto-gzip), maps each byte → COLORS_RGBA.
|
||||
5. Renders via OffscreenCanvas + ImageData.
|
||||
```
|
||||
|
||||
### Real-time Updates
|
||||
|
||||
```
|
||||
1. Client connects WS /api/ws
|
||||
2. Worker upgrades to Durable Object WebSocket
|
||||
3. DO stores connection in memory Set
|
||||
4. On pixel placement → DO broadcasts JSON to all connections
|
||||
5. Client updates local ImageData + re-renders
|
||||
6. On disconnect → auto-reconnect with exponential backoff (1s→30s)
|
||||
1. Client connects WS /api/ws.
|
||||
2. Worker forwards upgrade to DO /ws (URL rewritten to /ws so the DO
|
||||
pathname dispatch matches).
|
||||
3. DO state.acceptWebSocket(server) — Hibernation API; idle sockets
|
||||
survive DO eviction at zero CPU cost.
|
||||
4. On placement → DO iterates state.getWebSockets() and sends JSON.
|
||||
5. Client merges received pixels into local ImageData; re-renders dirty
|
||||
region.
|
||||
6. On disconnect → exponential-backoff reconnect (1 s → 30 s).
|
||||
```
|
||||
|
||||
## Storage
|
||||
|
||||
### Redis STRING / BITFIELD (Canvas)
|
||||
### `canvas_chunks` (canvas pixels)
|
||||
|
||||
- Key: `rplace:canvas:v2` (bumped from the old `rplace:canvas` so the 32-color/2048² data is ignored on rollout)
|
||||
- Encoding: 8 bits per pixel (u8), 256-color palette — byte-aligned, so raw Redis bytes are the pixel indices directly
|
||||
- Size: `4096 × 4096 × 1 = 16,777,216 bytes` (16 MB)
|
||||
- Offset: `y * CANVAS_WIDTH + x`
|
||||
- Atomic batch writes: single BITFIELD command chaining `SET u8 #offset color` per pixel
|
||||
- Reads via GETRANGE return the whole buffer; Cloudflare edge handles gzip
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `chunk_id` | INTEGER PRIMARY KEY | 0 .. CHUNK_COUNT − 1 |
|
||||
| `bytes` | BLOB NOT NULL | Exactly CHUNK_BYTES bytes (last chunk may be short) |
|
||||
|
||||
### Redis STRING (Cooldown)
|
||||
- Linear byte layout: `offset = y * CANVAS_WIDTH + x`
|
||||
- `chunk_id = floor(offset / CHUNK_BYTES)`
|
||||
- `CHUNK_BYTES = 65536`, `CHUNK_COUNT = ceil(TOTAL_PIXELS / CHUNK_BYTES)`
|
||||
- **Lazy initialization:** missing rows read as zero-filled buffers. Resizing
|
||||
the canvas is just a constants change — new chunks materialize on first
|
||||
read. No migration script needed.
|
||||
- **Hard caps:** CF DO BLOB row size 2 MB → 64 KB chunks have 32× headroom.
|
||||
|
||||
- Key pattern: `cooldown:{userId}`
|
||||
- Value: `"1"` (presence is the signal; content is irrelevant)
|
||||
- TTL: `REQUEST_COOLDOWN_SEC` (1s) — auto-expires, no explicit cleanup
|
||||
- Atomic via `SET key "1" NX EX 1`
|
||||
### `cooldowns` (rate-limit windows)
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `user_id` | TEXT PRIMARY KEY | Hashed CF-Connecting-IP |
|
||||
| `expires_at` | INTEGER NOT NULL | ms epoch; row becomes stale past this point |
|
||||
|
||||
- `idx_cooldowns_expires` keeps lazy GC sweeps cheap.
|
||||
- GC runs at 1% sample rate inside `tryAcquire`, wrapped in `try/catch` —
|
||||
best-effort; never blocks the rate-limit decision.
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
Fixed-window cooldown, one request per second per user:
|
||||
Single-row per user, 1 s window. Race-safe inside the DO via:
|
||||
|
||||
```
|
||||
On placement request:
|
||||
1. SET cooldown:{userId} "1" NX EX 1
|
||||
2. If reply == "OK" → allow (key set, TTL 1s)
|
||||
3. If reply == null → reject (429, retryAfter = 1)
|
||||
UPDATE cooldowns SET expires_at = ? WHERE user_id = ? AND expires_at <= ?
|
||||
→ if rowsWritten > 0: claim acquired (existing row was expired).
|
||||
→ else: INSERT INTO cooldowns ...
|
||||
if INSERT throws on PK conflict → claim denied (active row exists).
|
||||
```
|
||||
|
||||
Batch size is independent of the cooldown; it is validated separately
|
||||
(MAX_BATCH_SIZE = 2048). Anonymous identity via CF-Connecting-IP hash.
|
||||
Hardness comes from the DO single-threaded model; no SQL-level locking
|
||||
needed.
|
||||
|
||||
## Migration Endpoint (transitional)
|
||||
|
||||
`POST /admin/migrate-from-upstash` (token-gated): one-shot importer that
|
||||
pulls the legacy Upstash canvas via `lib/canvas-storage.js` (4 chunked
|
||||
GETRANGE calls) and posts the raw 16 MB bytes to DO `/import`. The DO
|
||||
splits into CHUNK_COUNT BLOB rows in a single sync transaction, then
|
||||
the worker round-trips a sample-byte verification.
|
||||
|
||||
Removed in Phase 4 of `plans/260509-2309-canvas-on-do-storage` along with
|
||||
`@upstash/redis` and `ioredis` dependencies.
|
||||
|
||||
## Free-tier Footprint (CF, 2026)
|
||||
|
||||
| Resource | Quota | rplace usage at hobby scale | Headroom |
|
||||
|---|---|---|---|
|
||||
| Workers requests | 100K/day | ~100/day (50 users) | 1000× |
|
||||
| DO storage / object | 10 GB | 16 MB | 600× |
|
||||
| DO storage / account | 5 GB | 16 MB | 300× |
|
||||
| BLOB row size | 2 MB | 64 KB chunks | 32× |
|
||||
| Per-DO request rate | 1,000 / s soft cap | <1 / s | 1000× |
|
||||
| WS connections / DO | tens of thousands | ~50 | huge |
|
||||
|
||||
`/api/canvas` carries `Cache-Control: public, s-maxage=10, stale-while-revalidate=30`.
|
||||
Verify edge HIT in production via `cf-cache-status: HIT`; if absent, wrap
|
||||
the worker handler with the Cache API to enforce caching.
|
||||
|
||||
## Security
|
||||
|
||||
- **Rate limiting**: Atomic `SET NX EX` prevents race conditions
|
||||
- **Identity**: CF-Connecting-IP (set by Cloudflare, unspoofable)
|
||||
- **Input validation**: Bounds checking, type checking, integer validation on all pixel data
|
||||
- **Batch cap**: Max 2048 pixels per request + request body size guard
|
||||
- **DO isolation**: /broadcast route only reachable via DO stub, not externally
|
||||
- **Rate limiting**: race-safe at the actor (single-threaded DO).
|
||||
- **Identity**: CF-Connecting-IP hashed to userId — unspoofable.
|
||||
- **Input validation**: strict bounds + type checks at the worker edge,
|
||||
re-validated at the DO trust boundary.
|
||||
- **Body cap**: ~128 KB per `/api/place` (2048 pixels × ~64 B JSON each).
|
||||
- **Migration endpoint**: Bearer-token gated. Token-comparison is direct
|
||||
string equality — adequate at hobby scale; consider constant-time
|
||||
comparison if exposure profile changes.
|
||||
- **DO isolation**: `/canvas`, `/place`, `/import`, `/ws` are intra-DO
|
||||
paths only — not internet-reachable except via the worker.
|
||||
|
||||
## Operational Notes
|
||||
|
||||
- **Single-region.** DO is anchored to one Cloudflare colo; latency for
|
||||
far-away users mirrors the previous Upstash Redis topology — no
|
||||
regression.
|
||||
- **DO eviction mid-place** is safe: the SQLite write commits before
|
||||
broadcast. If the DO evicts before broadcast fires, client WS receives
|
||||
no message but auto-reconnects and re-fetches `/api/canvas`. No data
|
||||
loss; minor latency tail.
|
||||
- **Storage billing.** Per-account 5 GB free cap on Free plan as of
|
||||
Jan 7 2026 — current 16 MB is far under. Monitor on resize.
|
||||
@@ -80,8 +80,8 @@ export async function migrateFromUpstash(env, roomStub, { force = false } = {})
|
||||
function pickSampleOffsets(srcBytes) {
|
||||
const offsets = new Set([
|
||||
0, // (0, 0)
|
||||
CANVAS_WIDTH - 1, // top-right
|
||||
(CANVAS_WIDTH * CANVAS_WIDTH) - 1, // bottom-right (square canvas)
|
||||
CANVAS_WIDTH - 1, // first-row right edge
|
||||
TOTAL_PIXELS - 1, // last byte
|
||||
Math.floor(TOTAL_PIXELS / 2), // middle
|
||||
Math.floor(TOTAL_PIXELS / 2) + CANVAS_WIDTH + 1, // off-middle
|
||||
]);
|
||||
|
||||
@@ -81,8 +81,10 @@ export function writePixels(sql, pixels) {
|
||||
|
||||
// For each touched chunk: read current bytes (or zero-fill), apply edits,
|
||||
// write back. INSERT OR REPLACE upserts the row.
|
||||
// Note: CF DO transactions are implicit per-fetch handler invocation —
|
||||
// multiple sql.exec calls within the same handler are atomic.
|
||||
// Atomicity: this loop is fully synchronous (no `await`) and the DO is
|
||||
// single-threaded, so the chunk updates are atomic with respect to other
|
||||
// requests. If anyone adds an `await` inside this loop, wrap the whole
|
||||
// block in `state.storage.transactionSync(() => { ... })` to preserve it.
|
||||
for (const [chunkId, edits] of groups) {
|
||||
const buf = readChunk(sql, chunkId);
|
||||
// readChunk returns a fresh Uint8Array (or wraps a buffer); writes must
|
||||
|
||||
@@ -33,7 +33,9 @@ export function tryAcquire(sql, userId, now = Date.now()) {
|
||||
// Drain the cursor so rowsWritten is finalized.
|
||||
updateCursor.toArray();
|
||||
if (updateCursor.rowsWritten > 0) {
|
||||
if (Math.random() < GC_SAMPLE_RATE) gc(sql, now);
|
||||
if (Math.random() < GC_SAMPLE_RATE) {
|
||||
try { gc(sql, now); } catch { /* GC is best-effort */ }
|
||||
}
|
||||
return { allowed: true, retryAfter: 0 };
|
||||
}
|
||||
|
||||
@@ -45,7 +47,9 @@ export function tryAcquire(sql, userId, now = Date.now()) {
|
||||
userId,
|
||||
expiresAt,
|
||||
);
|
||||
if (Math.random() < GC_SAMPLE_RATE) gc(sql, now);
|
||||
if (Math.random() < GC_SAMPLE_RATE) {
|
||||
try { gc(sql, now); } catch { /* GC is best-effort */ }
|
||||
}
|
||||
return { allowed: true, retryAfter: 0 };
|
||||
} catch {
|
||||
return { allowed: false, retryAfter: REQUEST_COOLDOWN_SEC };
|
||||
|
||||
+4
-2
@@ -62,13 +62,15 @@ app.post('/api/place', async (c) => {
|
||||
});
|
||||
});
|
||||
|
||||
/** GET /api/ws — WebSocket upgrade routed to the DO. */
|
||||
/** GET /api/ws — WebSocket upgrade routed to the DO.
|
||||
* The DO routes by url.pathname, so we rewrite the URL to `/ws` while
|
||||
* preserving the original headers (including Upgrade) via the request init. */
|
||||
app.get('/api/ws', async (c) => {
|
||||
const upgradeHeader = c.req.header('Upgrade');
|
||||
if (upgradeHeader !== 'websocket') {
|
||||
return c.text('Expected WebSocket', 426);
|
||||
}
|
||||
return room(c.env).fetch(c.req.raw);
|
||||
return room(c.env).fetch('http://do/ws', c.req.raw);
|
||||
});
|
||||
|
||||
/**
|
||||
|
||||
Reference in new issue
Block a user