mirror of
https://github.com/tiennm99/tiennm99bot.git
synced 2026-10-11 03:13:46 +00:00
docs: add lol schedule research, tgs feasibility reports, and pandascore journal
This commit is contained in:
1 parent
5ad388c597
commit
d1ef691ff6
6 files changed
+1084
No files matched your search
@@ -0,0 +1,32 @@
|
||||
# Decade-Old Riot API Dies; Twice-Rewritten Transport in One Day
|
||||
|
||||
**Date**: 2026-08-05 morning/afternoon
|
||||
**Severity**: Critical
|
||||
**Component**: lol module (Go Telegram bot, LoL esports schedules)
|
||||
**Status**: Resolved
|
||||
|
||||
## What Happened
|
||||
|
||||
The bot's lol schedule fetch started failing HTTP 403. Investigation revealed Riot revoked the decade-old x-api-key for esports-api.lolesports.com. The new lolesports.com is a Next.js/Netlify site keeping its API key server-side behind /api/gql persisted-query proxies. Official replacement (GRID/formerly Bayes LDP) is paid-only. Two transports were implemented and shipped the same day.
|
||||
|
||||
## The Brutal Truth
|
||||
|
||||
This is incredibly frustrating because a clever same-day reverse-engineering win was immediately exposed as unmaintainable and abandoned. Commit b76d0ca reverse-engineered lolesports.com /api/gql by extracting the persisted-query ID manifest from webpack chunk 29—extracting sha256 hashes failed; the Apollo client uses `generatePersistedQueryIdsFromManifest`, and the manifest URL came from the webpack runtime's chunk-hash map. It worked. We verified it live on a headless ARM64 server without a browser. And then we threw it away. The second rewrite took a fraction of the effort because the ScheduleEvent boundary held—architecture won out over cleverness.
|
||||
|
||||
## Technical Details
|
||||
|
||||
**b76d0ca (reverse-engineering, abandoned)**: Persisted query approach worked on first live probe, pulling real homeEvents data from lolesports.com /api/gql proxy.
|
||||
|
||||
**15f7e50 (PandaScore, shipped)**: REST transport to /lol/matches with Bearer token auth. Persisted ScheduleEvent contract and bson cache untouched. League slugs canonicalized via 12-entry mapping (e.g., `league-of-legends-lck-champions-korea` → `lck`). Results joined by team_id; outcome only set when winner_id present (preserves "score pending" semantics). Code review caught silent page-budget truncation; fixed with warn log and budget increase (3→5). Live probe rendered a real week with LIVE JDG 1–0 LGD—bracket-encoding verified correct.
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
Riot's infrastructure changed, not our code. But the reverse-engineered approach depended on webpack chunk IDs and manifest URLs rotating with each frontend deploy—a hidden maintenance tax. The persisted-query ID schema is Riot's internal concern; every redeployment could break the mapping without warning. We should have recognized this immediately instead of celebrating the win.
|
||||
|
||||
## Lessons Learned
|
||||
|
||||
**Data-source sovereignty beats cleverness.** A PandaScore REST endpoint (versioned, documented, intentionally exposed) beats reverse-engineered gql IDs. **Stable boundaries matter.** The ScheduleEvent contract was unchanged between transport rewrites; that boundary made the second pass clean and fast. **Reverse-engineering is a temporary fix.** It solved a crisis, but architecture decides sustainability.
|
||||
|
||||
## Next Steps
|
||||
|
||||
None—migration complete and live-verified. Monitor PandaScore rate limits (1000 req/h free tier). Superseded pending Leaguepedia enrichment plan (260726-0952) since PandaScore results already carry scores.
|
||||
@@ -0,0 +1,144 @@
|
||||
---
|
||||
type: researcher
|
||||
date: 2026-07-10
|
||||
conducted_at: 2026-07-10T06:06:00Z
|
||||
---
|
||||
|
||||
# Research Report: Telegram Bot `.tgs` Support
|
||||
|
||||
## Navigation
|
||||
|
||||
- [Summary](#summary)
|
||||
- [Methodology](#methodology)
|
||||
- [Findings](#findings)
|
||||
- [Implementation Recommendation](#implementation-recommendation)
|
||||
- [References](#references)
|
||||
- [Next Steps](#next-steps)
|
||||
- [Unresolved Questions](#unresolved-questions)
|
||||
|
||||
## Summary
|
||||
|
||||
Yes. Telegram Bot API `sendSticker` accepts a new animated `.TGS` sticker as a
|
||||
multipart upload. It also accepts an existing Telegram `file_id`, which is the
|
||||
recommended path after first upload. A `.TGS` URL is not accepted for an
|
||||
animated sticker.
|
||||
|
||||
This repo already has everything needed. `github.com/go-telegram/bot` v1.20.0
|
||||
provides `SendSticker` and `models.InputFileUpload`; no dependency upgrade or
|
||||
new package required.
|
||||
|
||||
## Methodology
|
||||
|
||||
- Sources consulted: Telegram Bot API, Telegram sticker specification, pinned
|
||||
Go module source, current repo implementation
|
||||
- Research date: 2026-07-10
|
||||
- Terms: `sendSticker`, `.TGS`, `InputFile`, multipart upload, animated sticker
|
||||
- Boundary: send an existing `.TGS`; sticker-set creation excluded
|
||||
|
||||
## Findings
|
||||
|
||||
### API Support
|
||||
|
||||
Use `sendSticker`, with the `sticker` parameter as an `InputFile` multipart
|
||||
upload. Telegram explicitly lists `.WEBP`, `.TGS`, and `.WEBM` as supported
|
||||
uploads. For `.TGS`, do not pass an HTTP URL; upload bytes or reuse a `file_id`.
|
||||
The file does not need to be added to a sticker set first.
|
||||
|
||||
`emoji` is optional and applies to a newly uploaded sticker. Telegram returns a
|
||||
`Message`; its sticker contains the reusable `file_id`.
|
||||
|
||||
### `.TGS` Requirements
|
||||
|
||||
Telegram's animated-sticker requirements:
|
||||
|
||||
- 512 x 512 canvas
|
||||
- maximum 3 seconds
|
||||
- looped animation
|
||||
- maximum 64 KB after rendering
|
||||
- 60 FPS
|
||||
- supported Bodymovin-TG/Lottie features only
|
||||
|
||||
An arbitrary Lottie JSON renamed to `.tgs` is not sufficient. Telegram `.TGS`
|
||||
is its constrained, gzip-compressed Lottie format.
|
||||
|
||||
### Repo Compatibility
|
||||
|
||||
The repo pins `github.com/go-telegram/bot` v1.20.0. It currently sends stickers
|
||||
by `models.InputFileString` in `internal/modules/loldle/handlers.go`, and already
|
||||
uses `models.InputFileUpload` for animation bytes in
|
||||
`internal/modules/misc/wheelofnames_command.go`. Direct `.TGS` upload combines
|
||||
those existing patterns.
|
||||
|
||||
### `sendSticker` vs `sendDocument`
|
||||
|
||||
`sendSticker` makes Telegram render the file as a sticker. `sendDocument` can
|
||||
transport the `.tgs` as a generic downloadable file (currently up to 50 MB),
|
||||
but that is not the right method when sticker rendering is desired. A document
|
||||
`file_id` cannot later be used with `sendSticker`, because Telegram does not
|
||||
allow changing the media type represented by a `file_id`.
|
||||
|
||||
### Security and Performance
|
||||
|
||||
- Treat local/embedded `.TGS` assets as untrusted input until validated against
|
||||
Telegram's format and size constraints.
|
||||
- Avoid fetching user-supplied URLs server-side; `.TGS` sticker URLs are not
|
||||
supported by `sendSticker` anyway.
|
||||
- Upload once, persist the returned bot-scoped `file_id`, then resend by
|
||||
`file_id` to avoid repeated multipart transfers.
|
||||
|
||||
## Implementation Recommendation
|
||||
|
||||
For in-memory bytes, matching current repo conventions:
|
||||
|
||||
```go
|
||||
sent, err := b.SendSticker(ctx, &bot.SendStickerParams{
|
||||
ChatID: msg.Chat.ID,
|
||||
MessageThreadID: msg.MessageThreadID,
|
||||
Sticker: &models.InputFileUpload{
|
||||
Filename: "celebration.tgs",
|
||||
Data: bytes.NewReader(tgsData),
|
||||
},
|
||||
Emoji: "🎉",
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fileID := sent.Sticker.FileID
|
||||
```
|
||||
|
||||
For later sends, use the repo's existing pattern:
|
||||
|
||||
```go
|
||||
Sticker: &models.InputFileString{Data: fileID}
|
||||
```
|
||||
|
||||
Common pitfalls:
|
||||
|
||||
- Passing an HTTPS `.tgs` URL instead of multipart-uploading it
|
||||
- Using `sendAnimation`; `.TGS` is an animated sticker, not a GIF-style
|
||||
animation payload
|
||||
- Sending invalid or oversized Lottie data
|
||||
- Omitting `MessageThreadID` in forum-topic replies
|
||||
- Assuming another bot's `file_id` is reusable
|
||||
|
||||
## References
|
||||
|
||||
- [Telegram Bot API: `sendSticker`](https://core.telegram.org/bots/api#sendsticker)
|
||||
- [Telegram Bot API: sending files](https://core.telegram.org/bots/api#sending-files)
|
||||
- [Telegram Bot API: `sendDocument`](https://core.telegram.org/bots/api#senddocument)
|
||||
- [Telegram animated sticker requirements](https://core.telegram.org/stickers#animated-stickers-and-emoji)
|
||||
- [`SendStickerParams` in go-telegram/bot v1.20.0](https://github.com/go-telegram/bot/blob/v1.20.0/methods_params.go)
|
||||
- [`InputFileUpload` in go-telegram/bot v1.20.0](https://github.com/go-telegram/bot/blob/v1.20.0/models/input_file.go)
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Add the `.tgs` asset only if a concrete command/feature needs it.
|
||||
2. Validate it against Telegram's limits.
|
||||
3. Upload with `SendSticker`; retain the returned `file_id` for repeated sends.
|
||||
4. Add focused handler and multipart-call tests if implementing a command.
|
||||
|
||||
## Unresolved Questions
|
||||
|
||||
- Which command or event should send the sticker?
|
||||
- Should the asset ship with the binary or be configured by `file_id`?
|
||||
@@ -0,0 +1,291 @@
|
||||
---
|
||||
type: researcher
|
||||
date: 2026-07-10
|
||||
conducted_at: 2026-07-10T06:14:00Z
|
||||
---
|
||||
|
||||
# Research Report: Wheel of Names as a TGS Document
|
||||
|
||||
## Navigation
|
||||
|
||||
- [Executive Summary](#executive-summary)
|
||||
- [Research Methodology](#research-methodology)
|
||||
- [Current Architecture](#current-architecture)
|
||||
- [Key Findings](#key-findings)
|
||||
- [Comparative Analysis](#comparative-analysis)
|
||||
- [Timing Recommendation](#timing-recommendation)
|
||||
- [Implementation Impact](#implementation-impact)
|
||||
- [Recommendation](#recommendation)
|
||||
- [Resources and References](#resources-and-references)
|
||||
- [Next Steps](#next-steps)
|
||||
- [Unresolved Questions](#unresolved-questions)
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Sending `wheelofnames.tgs` through Telegram `sendDocument` is technically
|
||||
possible. Bots may multipart-upload general files up to 50 MB, and this repo's
|
||||
Go library already supports `SendDocument` plus `InputFileUpload`.
|
||||
|
||||
The exact proposal is not attractive for the current user experience. Telegram
|
||||
defines `sendDocument` as general-file delivery and does not promise inline or
|
||||
auto-playing TGS document previews. Inline TGS animation is part of the sticker
|
||||
contract. Therefore users may see a downloadable `.tgs` attachment rather than
|
||||
the wheel animation.
|
||||
|
||||
Producing TGS is also not a GIF conversion setting. The existing
|
||||
`tiennm99/wheelofnames` renderer uses React/Remotion, Chromium-rendered PNG
|
||||
frames, and Remotion's GIF encoder. Remotion does not emit Lottie/TGS. A TGS
|
||||
version needs a second vector renderer/serializer. Dynamic arbitrary labels are
|
||||
the hardest part because Telegram TGS disallows text layers; glyphs would need
|
||||
conversion to vector paths, threatening the 64 KB sticker-format target.
|
||||
|
||||
Verdict: **transmission feasible; useful inline wheel experience not reliably
|
||||
feasible with `sendDocument`; engineering/value ratio poor.** If the goal is a
|
||||
smaller inline animation with caption, H.264 MP4 through the existing
|
||||
`sendAnimation` method is the strongest option to benchmark first.
|
||||
|
||||
## Research Methodology
|
||||
|
||||
- Conducted: 2026-07-10 UTC
|
||||
- Sources: current `miti99bot`, `tiennm99/wheelofnames` at commit
|
||||
`f13055651e7a18a59b024f7b22266611cc843389`, Telegram Bot API and sticker
|
||||
specification, Remotion renderer documentation
|
||||
- Criteria: Telegram UX, file size, renderer compatibility, timing quality,
|
||||
implementation scope, operational cost
|
||||
- Boundary: research only; no implementation or live Telegram client test
|
||||
- Search terms: `sendDocument`, TGS, Lottie, Remotion codec, 3-second wheel,
|
||||
Telegram document preview
|
||||
|
||||
## Current Architecture
|
||||
|
||||
```text
|
||||
miti99bot
|
||||
POST /api/gif (options, winner, 6000ms spin, 1000ms hold, 20 FPS, 512px)
|
||||
-> wheelofnames React/Remotion composition
|
||||
-> Chromium renders PNG frames
|
||||
-> Remotion encodes GIF
|
||||
sendAnimation(GIF + spoiler caption)
|
||||
-> Telegram inline animation
|
||||
```
|
||||
|
||||
Current facts:
|
||||
|
||||
- Bot requests a 6.0s spin plus 1.0s result hold: 7.0s total.
|
||||
- Bot requests 20 FPS and 512x512.
|
||||
- Renderer accepts only 12, 15, or 20 FPS.
|
||||
- Renderer schema currently requires at least 3.0s spin and 0.5s hold, so its
|
||||
shortest accepted total is 3.5s.
|
||||
- Renderer supports up to 32 options with 40 characters each.
|
||||
- Recorded 384px/12 FPS/3.5s GIFs are about 298-316 KB. A production-equivalent
|
||||
512px/20 FPS/7s benchmark is not recorded.
|
||||
- Current bot sends the result as a spoiler caption attached to the animation.
|
||||
|
||||
The proposed architecture is materially different:
|
||||
|
||||
```text
|
||||
miti99bot
|
||||
POST /api/tgs (new contract and timing)
|
||||
-> new vector wheel model
|
||||
-> new Lottie JSON serializer
|
||||
-> validate Telegram-supported features
|
||||
-> gzip as .tgs
|
||||
sendDocument(TGS + spoiler caption)
|
||||
-> Telegram document attachment
|
||||
-> inline playback not guaranteed
|
||||
```
|
||||
|
||||
## Key Findings
|
||||
|
||||
### 1. Telegram Can Transport the File
|
||||
|
||||
`sendDocument` accepts an uploaded file of any type up to 50 MB and returns a
|
||||
message whose media is a `Document`. The pinned Go dependency exposes all
|
||||
required fields, including document upload, caption, thread ID, and optional
|
||||
content-type detection control.
|
||||
|
||||
The animated-sticker restrictions (3s, 64 KB, 512x512, 60 FPS) are not Bot API
|
||||
limits on a generic document. The document limit is 50 MB. Following the TGS
|
||||
restrictions still makes sense if the artifact should remain a valid Telegram
|
||||
animation or may later be sent as a sticker.
|
||||
|
||||
### 2. `sendDocument` Does Not Promise TGS Playback
|
||||
|
||||
Telegram documents inline animation for `Animation` (GIF or silent H.264 MP4)
|
||||
and `Sticker` (`.WEBP`, `.TGS`, `.WEBM`) message types. It documents
|
||||
`sendDocument` only as sending general files.
|
||||
|
||||
Inference from the official message contracts: a `.tgs` uploaded as a document
|
||||
cannot be relied on to auto-play or render like a sticker. Client behavior may
|
||||
vary and is not an API guarantee. This is the proposal's decisive risk. A
|
||||
downloadable file users must open elsewhere is worse than the current GIF.
|
||||
|
||||
### 3. TGS Requires a New Renderer
|
||||
|
||||
The current renderer's output path is frame-based:
|
||||
|
||||
```text
|
||||
React DOM/CSS -> Chromium -> PNG frames -> GIF encoder
|
||||
```
|
||||
|
||||
TGS is gzip-compressed Lottie vector-animation JSON. There is no reliable
|
||||
GIF-to-TGS conversion: raster frames do not contain the vector geometry,
|
||||
transforms, or typography needed for compact Lottie output. Remotion's media
|
||||
codecs include GIF and video formats such as H.264; TGS is not an output codec.
|
||||
|
||||
A TGS renderer would need to independently serialize:
|
||||
|
||||
- wedge vector paths and colors
|
||||
- hub and pointer vector paths
|
||||
- wheel rotation keyframes/easing
|
||||
- winner indication using supported vector properties
|
||||
- every label as glyph outlines, or omit labels
|
||||
- JSON normalization, gzip, and TGS validation
|
||||
|
||||
The current CSS shadow, drop-shadow winner glow, HTML text, and font fallback
|
||||
cannot be transferred directly because Telegram's TGS requirements disallow
|
||||
layer effects and text.
|
||||
|
||||
### 4. Dynamic Labels Undermine the Size Advantage
|
||||
|
||||
The wheel's value comes from arbitrary user labels, including Vietnamese and
|
||||
other Unicode text. Telegram-compatible TGS cannot use text layers. Converting
|
||||
every unique glyph to vector paths requires font shaping, path generation, and
|
||||
font coverage. Up to 32 x 40 characters creates a large worst case.
|
||||
|
||||
Consequences:
|
||||
|
||||
- 64 KB is plausible only for a simplified wheel with few short labels, or no
|
||||
labels; it is not a safe general contract without empirical prototypes.
|
||||
- Font files and fallback behavior become renderer concerns.
|
||||
- Unicode shaping and emoji are substantially harder than current browser text.
|
||||
- Repeated glyph/path data can erase much of TGS's size advantage.
|
||||
|
||||
If sent only as a document, exceeding 64 KB is allowed, but then the proposal
|
||||
loses both guaranteed playback and its strongest size target.
|
||||
|
||||
### 5. Operational and Security Considerations
|
||||
|
||||
- Keep current option-count and character-count bounds; vector path generation
|
||||
expands attacker-controlled text into CPU and memory work.
|
||||
- Validate generated JSON, compressed and uncompressed sizes, layer count,
|
||||
duration, frame rate, and unsupported features before upload.
|
||||
- Generate TGS internally; do not accept user-provided TGS payloads without
|
||||
decompression limits and schema validation.
|
||||
- A second render path duplicates layout logic unless wheel geometry is first
|
||||
extracted into a renderer-neutral model.
|
||||
|
||||
## Comparative Analysis
|
||||
|
||||
| Option | Telegram UX | Renderer effort | Expected size | Caption | Main risk |
|
||||
|---|---|---:|---:|---|---|
|
||||
| Current GIF + `sendAnimation` | Inline/autoplay | Existing | Largest | Yes | Bandwidth/render time |
|
||||
| 3s GIF + `sendAnimation` | Inline/autoplay | Low | Lower than current | Yes | Still GIF compression |
|
||||
| TGS + `sendDocument` | Generic attachment; playback not guaranteed | High | Unknown; potentially small | Yes | Users may not see animation |
|
||||
| TGS + `sendSticker` | Inline sticker | High | Must target 64 KB | No sticker caption | Labels/features may fail limits |
|
||||
| H.264 MP4 + `sendAnimation` | Inline/autoplay | Low-medium | Likely much smaller than GIF; benchmark needed | Yes | Encoding/client tuning |
|
||||
|
||||
H.264 is a closer fit to current architecture: Remotion already renders H.264,
|
||||
and Telegram explicitly accepts silent H.264/MPEG-4 AVC through
|
||||
`sendAnimation`. It preserves the spoiler caption and browser-rendered labels.
|
||||
No size claim should be finalized until identical 3s fixtures are benchmarked.
|
||||
|
||||
## Timing Recommendation
|
||||
|
||||
If a 3-second-compatible animation prototype is created, use:
|
||||
|
||||
| Phase | Duration | 60 FPS frames | Behavior |
|
||||
|---|---:|---:|---|
|
||||
| Spin/decelerate | 2300ms | 138 | Three full turns, cubic ease-out to winner |
|
||||
| Winner hold | 650ms | 39 | Stationary wheel; simple winner emphasis |
|
||||
| Total | 2950ms | 177 | 50ms safety margin below 3 seconds |
|
||||
|
||||
Why three turns: the current wheel uses seven turns over six seconds with a
|
||||
cubic ease-out. Compressing seven turns into 2.3s would start near nine
|
||||
rotations/second and be visually noisy. Three turns preserves roughly the
|
||||
current initial angular speed while making deceleration readable.
|
||||
|
||||
The 650ms hold is short but sufficient for confirmation because the caption
|
||||
also states the result. If no caption is present, favor 2200ms spin plus 750ms
|
||||
hold. Timing has little UX value when TGS is sent as a non-previewed document.
|
||||
|
||||
Required renderer changes even for a GIF/MP4 timing experiment:
|
||||
|
||||
- lower `durationMs` minimum below 3000
|
||||
- permit total duration below 3500ms
|
||||
- add 60 FPS only if targeting valid TGS; GIF/MP4 need not use 60 FPS
|
||||
- reduce `fullTurns` from 7 to 3 for the short profile
|
||||
- verify the winner lands exactly after frame rounding
|
||||
|
||||
## Implementation Impact
|
||||
|
||||
If this exact TGS-document option is later chosen:
|
||||
|
||||
### `tiennm99/wheelofnames`
|
||||
|
||||
- Add a renderer-neutral wheel geometry/timing model.
|
||||
- Add `/api/tgs` and a response MIME/validation contract.
|
||||
- Implement Lottie vector serialization and gzip packaging.
|
||||
- Convert supported text to glyph paths or define label restrictions.
|
||||
- Simplify unsupported shadow/glow effects.
|
||||
- Add format, timing, size, Unicode, and winner-alignment tests.
|
||||
- Benchmark sparse and worst-case option sets.
|
||||
|
||||
### `miti99bot`
|
||||
|
||||
- Change the API URL/Accept/content-type and magic validation from GIF to TGS.
|
||||
- Change the filename and response byte limit.
|
||||
- Replace `SendAnimationParams` with `SendDocumentParams`, retaining caption,
|
||||
parse mode, thread ID, and text fallback.
|
||||
- Update recording-bot assertions and all wheel handler/client tests.
|
||||
- Update user-facing GIF wording and deployment docs/config contract.
|
||||
|
||||
No dependency change is necessary in the bot.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Do not implement **TGS via `sendDocument`** as the primary wheel response unless
|
||||
a real Telegram Android/iOS/Desktop client matrix proves acceptable inline
|
||||
playback. The Bot API provides no such guarantee.
|
||||
|
||||
Decision order:
|
||||
|
||||
1. Reduce the existing animation to the 2.3s spin + 0.65s hold profile.
|
||||
2. Benchmark identical GIF and silent H.264 MP4 outputs at 512px.
|
||||
3. Prefer H.264 with `sendAnimation` if it materially reduces bytes while
|
||||
retaining inline playback and caption.
|
||||
4. Prototype TGS only if sticker-style delivery or vector rendering is itself
|
||||
a product requirement, and first test a no-label/few-label wheel against
|
||||
Telegram's validator and clients.
|
||||
|
||||
## Resources and References
|
||||
|
||||
### Official Documentation
|
||||
|
||||
- [Telegram Bot API: `sendDocument`](https://core.telegram.org/bots/api#senddocument)
|
||||
- [Telegram Bot API: `sendAnimation`](https://core.telegram.org/bots/api#sendanimation)
|
||||
- [Telegram Bot API: `sendSticker`](https://core.telegram.org/bots/api#sendsticker)
|
||||
- [Telegram animated TGS requirements](https://core.telegram.org/stickers#animated-stickers-and-emoji)
|
||||
- [Remotion `renderMedia()`](https://www.remotion.dev/docs/renderer/render-media)
|
||||
|
||||
### Current Implementations
|
||||
|
||||
- [`wheelofnames` GIF renderer](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/src/render/render-gif.js)
|
||||
- [`wheelofnames` request limits](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/src/schemas/wheel-request.js)
|
||||
- [`wheelofnames` Remotion composition](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/src/remotion/WheelComposition.jsx)
|
||||
- [`wheelofnames` recorded benchmarks](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/docs/render-benchmarks.md)
|
||||
|
||||
## Next Steps
|
||||
|
||||
No implementation now. If the idea advances:
|
||||
|
||||
1. Decide whether inline/autoplay is mandatory.
|
||||
2. Run a Telegram client-matrix spike with one hand-authored TGS document.
|
||||
3. Benchmark 2.95s GIF versus H.264 before building a new vector renderer.
|
||||
4. Only then estimate TGS implementation from a few-label vector prototype.
|
||||
|
||||
## Unresolved Questions
|
||||
|
||||
- Must the wheel auto-play inline on Telegram clients?
|
||||
- Must all option labels remain visible inside the wheel?
|
||||
- Is the target specifically under 64 KB, or only smaller than the current GIF?
|
||||
- Is silent H.264 MP4 acceptable if it preserves the current message UX?
|
||||
@@ -0,0 +1,476 @@
|
||||
---
|
||||
type: researcher
|
||||
date: 2026-07-10
|
||||
conducted_at: 2026-07-10T06:32:00Z
|
||||
---
|
||||
|
||||
# Research Report: In-House Wheel Rendering with GIF or Video
|
||||
|
||||
## Navigation
|
||||
|
||||
- [Executive Summary](#executive-summary)
|
||||
- [Research Scope](#research-scope)
|
||||
- [Current Constraints](#current-constraints)
|
||||
- [Technical Options](#technical-options)
|
||||
- [Format Decision](#format-decision)
|
||||
- [Recommended Architecture](#recommended-architecture)
|
||||
- [Performance and Quality](#performance-and-quality)
|
||||
- [Security and Reliability](#security-and-reliability)
|
||||
- [Implementation Impact](#implementation-impact)
|
||||
- [Validation Plan](#validation-plan)
|
||||
- [Resources and References](#resources-and-references)
|
||||
- [Next Steps](#next-steps)
|
||||
- [Unresolved Questions](#unresolved-questions)
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Best technical direction: **keep the existing React/Remotion wheel composition,
|
||||
render silent H.264 MP4 locally, and continue sending it with Telegram
|
||||
`sendAnimation`.** This preserves the current geometry, colors, shadows,
|
||||
winner glow, fonts, label layout, easing, caption, and inline/autoplay behavior.
|
||||
It removes the renderer HTTP service and its URL/token, while avoiding a risky
|
||||
visual rewrite.
|
||||
|
||||
"All in house" should mean one deployed container, no renderer HTTP listener,
|
||||
no `WHEELOFNAMES_API_URL`, no renderer authentication token, no runtime browser
|
||||
download, and no external render service. The Go bot starts a local persistent
|
||||
Node/Remotion worker and communicates through stdin/stdout plus controlled
|
||||
temporary files. Telegram upload remains the only media network step.
|
||||
|
||||
H.264 is preferred over GIF. Telegram explicitly accepts silent H.264/MPEG-4
|
||||
AVC as an animation. H.264 preserves far more color detail than GIF's
|
||||
256-color palette and should compress the mostly-static wheel frames much more
|
||||
efficiently. Exact size and latency remain benchmark gates; no successful
|
||||
H.264 benchmark exists yet.
|
||||
|
||||
Main trade-off: exact visual reuse requires Node, Chromium, fonts, and Remotion
|
||||
inside the bot container. The current tiny distroless/static image cannot run
|
||||
them. Expect a much larger image and materially higher render-time CPU/RAM.
|
||||
|
||||
## Research Scope
|
||||
|
||||
- Conducted: 2026-07-10 UTC
|
||||
- Goal: local rendering, GIF or video, current visual fidelity, no renderer API
|
||||
- Sources: current `miti99bot`; `tiennm99/wheelofnames` at commit
|
||||
`f13055651e7a18a59b024f7b22266611cc843389`; Telegram Bot API; Remotion
|
||||
renderer docs; Go `image/gif` docs
|
||||
- Compared: pure-Go GIF, Go frames plus FFmpeg, local Remotion worker
|
||||
- Boundary: research only; no project implementation
|
||||
|
||||
### Definition of "No API"
|
||||
|
||||
Recommended interpretation:
|
||||
|
||||
- no HTTP call to a wheel renderer
|
||||
- no renderer TCP port or public/private service
|
||||
- no external rendering vendor
|
||||
- no runtime download of code, fonts, Chromium, or media tools
|
||||
- local child-process IPC is allowed
|
||||
|
||||
If "no API" instead means one Go process with no child runtime, exact parity
|
||||
with the current Remotion composition is not realistic. The pure-Go option can
|
||||
approximate it but becomes a separate renderer requiring visual maintenance.
|
||||
|
||||
## Current Constraints
|
||||
|
||||
### Bot Deployment
|
||||
|
||||
The current image is:
|
||||
|
||||
```text
|
||||
Go 1.26 static build -> distroless/static non-root runtime -> /server only
|
||||
```
|
||||
|
||||
It contains no shell, Node, Chromium, fonts, or FFmpeg. It cannot execute the
|
||||
current renderer locally without a Docker runtime redesign.
|
||||
|
||||
### Wheel Renderer
|
||||
|
||||
The current `wheelofnames` project uses:
|
||||
|
||||
- React 19 and Remotion 4.0.485
|
||||
- Chromium-rendered DOM/CSS frames
|
||||
- Remotion `renderMedia()`
|
||||
- PNG frame rendering, then GIF encoding
|
||||
- Noto fonts, including Vietnamese and color emoji
|
||||
- one render at a time by default
|
||||
|
||||
Current bot profile:
|
||||
|
||||
| Setting | Value |
|
||||
|---|---:|
|
||||
| Canvas | 512x512 |
|
||||
| Spin | 6000ms |
|
||||
| Winner hold | 1000ms |
|
||||
| Total | 7000ms |
|
||||
| FPS | 20 |
|
||||
| Frames | 140 |
|
||||
| Full turns | 7 |
|
||||
|
||||
Recorded renderer measurements cover only a smaller fixture:
|
||||
|
||||
| Fixture | Output | Bytes | Render time |
|
||||
|---|---:|---:|---:|
|
||||
| 384px, 12 FPS, 3.5s cold | GIF | 315,661 | 13.9s |
|
||||
| 384px, 12 FPS, 3.5s warm service | GIF | 298,374 | 4.1s |
|
||||
|
||||
No production-equivalent 512px/20 FPS/7s GIF measurement or H.264 measurement
|
||||
is recorded.
|
||||
|
||||
## Technical Options
|
||||
|
||||
### Option A: Pure-Go Raster Renderer and GIF
|
||||
|
||||
Flow:
|
||||
|
||||
```text
|
||||
Go wheel geometry + Go font/layout + raster frames + image/gif -> sendAnimation
|
||||
```
|
||||
|
||||
Advantages:
|
||||
|
||||
- retains one static Go binary and distroless image
|
||||
- no child process or browser
|
||||
- simplest production runtime
|
||||
- deterministic, fully offline renderer
|
||||
|
||||
Disadvantages:
|
||||
|
||||
- reimplement every current visual and animation behavior
|
||||
- browser-quality Unicode shaping, fallback fonts, and color emoji are hard
|
||||
- must recreate anti-aliased wedges, radial text, shadows, glow, and CSS effects
|
||||
- GIF allows at most 256 palette colors per frame
|
||||
- palette quantization/dithering can damage text edges, shadows, and glow
|
||||
- standard `image/gif.EncodeAll` holds all paletted frames before writing
|
||||
- duplicates visual logic now owned by the Remotion composition
|
||||
|
||||
Verdict: good only if minimal runtime matters more than exact visual parity.
|
||||
|
||||
### Option B: Pure-Go Frames Piped to Local FFmpeg H.264
|
||||
|
||||
Flow:
|
||||
|
||||
```text
|
||||
Go wheel geometry + Go font/layout -> raw frames -> local FFmpeg -> MP4
|
||||
```
|
||||
|
||||
Advantages:
|
||||
|
||||
- video size/color advantages
|
||||
- raw frames can stream to FFmpeg with bounded memory
|
||||
- no Chromium or Node
|
||||
- no HTTP API
|
||||
|
||||
Disadvantages:
|
||||
|
||||
- still requires the full visual rewrite
|
||||
- adds FFmpeg executable and Linux runtime libraries
|
||||
- loses current distroless/static simplicity
|
||||
- child-process lifecycle, cancellation, temp/output bounds still required
|
||||
- font shaping remains the largest fidelity risk
|
||||
|
||||
Verdict: inferior to reusing Remotion; it pays both rewrite and runtime costs.
|
||||
|
||||
### Option C: Existing Remotion Composition as a Local Worker
|
||||
|
||||
Flow:
|
||||
|
||||
```text
|
||||
Go bot -> local JSON-lines worker -> Remotion/Chromium -> H.264 MP4
|
||||
<- result metadata + controlled temp path
|
||||
Go bot -> Telegram sendAnimation
|
||||
```
|
||||
|
||||
Advantages:
|
||||
|
||||
- highest visual parity; reuses the current composition directly
|
||||
- browser keeps current Unicode/font behavior
|
||||
- H.264 is already a supported Remotion codec
|
||||
- no renderer HTTP surface, URL, token, or network failure mode
|
||||
- one composition can still render GIF during benchmarks/debugging
|
||||
- current text fallback remains available if rendering fails
|
||||
|
||||
Disadvantages:
|
||||
|
||||
- combined runtime needs Node 24, Chromium, fonts, and browser libraries
|
||||
- image size grows from a tiny static runtime to likely hundreds of MB
|
||||
- render CPU/RAM now shares the bot container's cgroup
|
||||
- local worker protocol and restart supervision must be implemented
|
||||
- Remotion bundle/browser cold start remains expensive unless reused
|
||||
|
||||
Verdict: best match for "current beauty + no renderer API."
|
||||
|
||||
### Option D: One-Shot Local Remotion CLI per Command
|
||||
|
||||
This also removes HTTP, but it repeats Node startup, bundling, and browser
|
||||
launch per command. Existing measurements show a large cold/warm gap. Remotion
|
||||
officially supports reusing an open browser to speed multiple renders.
|
||||
|
||||
Verdict: acceptable prototype, poor production architecture.
|
||||
|
||||
## Format Decision
|
||||
|
||||
### GIF versus H.264 MP4
|
||||
|
||||
| Criterion | GIF | Silent H.264 MP4 |
|
||||
|---|---|---|
|
||||
| Telegram method | `sendAnimation` | `sendAnimation` |
|
||||
| Inline/autoplay UX | Yes | Yes |
|
||||
| Caption/spoiler | Yes | Yes |
|
||||
| Colors | 256 per frame | Full-color input, compressed video output |
|
||||
| Shadows/glow | Palette banding/dither risk | Better gradients; lossy artifact risk |
|
||||
| Temporal compression | Limited | Strong |
|
||||
| Expected bytes | Larger | Smaller; must benchmark |
|
||||
| Current Remotion support | Existing | Native `codec: "h264"` |
|
||||
| Pure-Go production | Possible | Not with standard library |
|
||||
|
||||
**Choose H.264 MP4 as the production format.** Keep GIF only as a temporary
|
||||
benchmark/debug option, not an automatic production fallback. Re-rendering GIF
|
||||
after an MP4 failure doubles latency and resource use; retain the existing text
|
||||
winner fallback instead.
|
||||
|
||||
Suggested first parity profile:
|
||||
|
||||
```js
|
||||
await renderMedia({
|
||||
codec: 'h264',
|
||||
composition,
|
||||
crf: 18,
|
||||
imageFormat: 'png',
|
||||
inputProps,
|
||||
muted: true,
|
||||
outputLocation,
|
||||
overwrite: true,
|
||||
pixelFormat: 'yuv420p',
|
||||
puppeteerInstance: sharedBrowser,
|
||||
serveUrl,
|
||||
});
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Preserve 512px, 20 FPS, 6s spin, 1s hold, seven turns for the first parity
|
||||
release. Shortening is a separate UX decision.
|
||||
- Start at CRF 18; compare CRF 18, 20, and 22 visually and by bytes.
|
||||
- PNG source frames preserve browser rendering before H.264 encoding.
|
||||
- `yuv420p` is the conservative compatibility target; inspect small radial text
|
||||
and saturated slice edges for chroma-subsampling artifacts.
|
||||
- Do not add an audio track.
|
||||
|
||||
## Recommended Architecture
|
||||
|
||||
```text
|
||||
┌──────────────────── one Coolify container ────────────────────┐
|
||||
│ │
|
||||
│ Go /server │
|
||||
│ command handler │
|
||||
│ │ bounded request │
|
||||
│ ▼ │
|
||||
│ render supervisor ── JSON lines over pipes ──► Node worker │
|
||||
│ ▲ │ │
|
||||
│ │ metadata + temp path ▼ │
|
||||
│ │ cached Remotion bundle │
|
||||
│ │ shared Chromium instance │
|
||||
│ │ H.264 render, concurrency=1 │
|
||||
│ │ │ │
|
||||
│ └──────── read + validate + delete ◄── temp MP4 │
|
||||
│ │
|
||||
│ Go SendAnimation(MP4 + existing spoiler caption) ─► Telegram │
|
||||
│ │
|
||||
│ no renderer HTTP listener; no renderer network call │
|
||||
└───────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Worker Lifecycle
|
||||
|
||||
1. Go starts one Node worker lazily or warms it asynchronously after startup.
|
||||
2. Worker bundles Remotion once and opens one reusable browser.
|
||||
3. Go serializes render requests through a capacity-one queue.
|
||||
4. Worker renders to a unique directory under a controlled temp root.
|
||||
5. Worker returns request ID, path, MIME, dimensions, duration, and byte count.
|
||||
6. Go verifies path prefix, MP4 signature/metadata, and maximum size.
|
||||
7. Go reads/sends the file and always removes the temp directory.
|
||||
8. On timeout/crash, Go kills and restarts the worker, then sends text result.
|
||||
|
||||
Use stdout only for machine protocol and stderr for renderer logs. Never send
|
||||
binary/base64 through JSON lines; base64 adds memory and about 33% transfer
|
||||
overhead.
|
||||
|
||||
### Container Design
|
||||
|
||||
Recommended multi-stage build:
|
||||
|
||||
```text
|
||||
stage 1: golang:1.26.5 -> build static /server
|
||||
stage 2: node:24-bookworm-slim -> pnpm production install + browser download
|
||||
runtime: node:24-bookworm-slim + Chromium libs + Noto fonts
|
||||
copy /server, renderer bundle/source, node_modules, pinned browser
|
||||
entrypoint: /server (starts local Node worker)
|
||||
```
|
||||
|
||||
Requirements:
|
||||
|
||||
- install the same Chromium libraries/fonts as current `wheelofnames` Dockerfile
|
||||
- download/pin the Remotion browser during image build, never at runtime
|
||||
- run bot and worker as non-root
|
||||
- keep port 8080 only for bot health; expose no renderer port
|
||||
- allow controlled writable temp space; keep application/root filesystem
|
||||
read-only where practical
|
||||
- retain one bot replica and one render at a time
|
||||
|
||||
This removes the current distroless security/minimal-size benefit. That is the
|
||||
principal architecture cost.
|
||||
|
||||
## Performance and Quality
|
||||
|
||||
### What Is Verified
|
||||
|
||||
- Telegram accepts GIF and silent H.264/MPEG-4 AVC through `sendAnimation`, up
|
||||
to 50 MB.
|
||||
- Remotion `renderMedia()` supports `codec: "h264"`, in-memory or file output,
|
||||
cancellation, CRF, pixel format, and reusable browser instances.
|
||||
- Current warm GIF fixture render is 4.1s; cold is 13.9s.
|
||||
- The current production container lacks all browser/media runtime pieces.
|
||||
|
||||
### What Is Not Yet Verified
|
||||
|
||||
- H.264 bytes and render latency for this wheel
|
||||
- production 512px/20 FPS/7s GIF bytes and latency
|
||||
- Telegram-side appearance after upload/transcoding on Android, iOS, Desktop
|
||||
- combined container image size and idle/peak memory
|
||||
|
||||
A temporary local benchmark was attempted. Dependency installation and browser
|
||||
preparation succeeded, but Chromium could not launch because the research host
|
||||
lacks the shared libraries listed in the renderer Dockerfile. No codec numbers
|
||||
were fabricated from that failed run.
|
||||
|
||||
The temporary checkout plus installed dependencies occupied about 600 MB, and
|
||||
the downloaded headless browser executable alone was about 188 MB. These are
|
||||
research-host figures, not a final compressed container-image estimate, but
|
||||
they confirm the runtime increase is material.
|
||||
|
||||
### Expected Behavior Requiring Measurement
|
||||
|
||||
H.264 should beat GIF for this animation because most pixels remain spatially
|
||||
and temporally related while only wheel rotation and winner emphasis change.
|
||||
GIF stores palette-based images and has much weaker temporal compression. The
|
||||
size win must still be measured on real fixtures.
|
||||
|
||||
## Security and Reliability
|
||||
|
||||
- Preserve option count/length limits before invoking the worker.
|
||||
- Never interpolate option text into shell commands; send structured JSON over
|
||||
an already-open pipe.
|
||||
- Use request IDs and reject unsolicited/out-of-order worker responses.
|
||||
- Enforce render timeout, output byte cap, temp-root path containment, and file
|
||||
signature/ffprobe-style metadata checks.
|
||||
- Keep concurrency at one on the recommended 1-2 vCPU/1-2 GB starting profile.
|
||||
- Cancel render on Telegram/request context cancellation where practical.
|
||||
- Restart the worker after crash or repeated render failures.
|
||||
- Avoid worker stdout logging that can corrupt the IPC stream.
|
||||
- Pin Node, pnpm lock, Remotion, and browser versions.
|
||||
- Run without renderer network access if container policy supports it; the
|
||||
worker needs local files only.
|
||||
- Preserve current plain-text winner fallback on any render/send failure.
|
||||
|
||||
An out-of-memory kill can still terminate the whole container even though the
|
||||
renderer is a child process. Resource limits, serial rendering, and production
|
||||
peak-memory measurement are mandatory.
|
||||
|
||||
## Implementation Impact
|
||||
|
||||
If selected later, expected scope:
|
||||
|
||||
### Repository/runtime
|
||||
|
||||
- bring the required `wheelofnames` composition into this repository or a
|
||||
version-pinned build input
|
||||
- add Node renderer package/lock files and local worker entry point
|
||||
- replace distroless runtime with combined Node/Chromium runtime
|
||||
- remove wheel API URL/token configuration and deployment instructions
|
||||
|
||||
### Go bot
|
||||
|
||||
- replace HTTP wheel client with a local render supervisor
|
||||
- preserve command parsing, local winner selection, spoiler caption, thread ID,
|
||||
and text fallback
|
||||
- change upload filename from `.gif` to `.mp4`
|
||||
- keep `SendAnimationParams`; no Telegram method change
|
||||
- validate local worker output and clean temporary files
|
||||
|
||||
### Tests
|
||||
|
||||
- unit-test supervisor framing, timeouts, crash/restart, output validation, and
|
||||
cleanup with a small fake child executable
|
||||
- retain handler tests for caption, thread ID, and text fallback
|
||||
- test renderer composition and winner alignment in Node
|
||||
- add deterministic smoke renders for GIF/H.264 comparison
|
||||
- add Docker smoke test proving browser launch under the final non-root runtime
|
||||
- perform Telegram client visual matrix before rollout
|
||||
|
||||
### Maintainability Boundary
|
||||
|
||||
Extract renderer-neutral request/timing constants, but do not rewrite wheel
|
||||
geometry in Go. The Remotion composition remains the sole visual source of
|
||||
truth. This avoids GIF/video visual drift.
|
||||
|
||||
## Validation Plan
|
||||
|
||||
Before choosing implementation:
|
||||
|
||||
1. Build the combined runtime prototype with required browser libraries.
|
||||
2. Render identical fixtures as GIF and H.264:
|
||||
- 2, 8, 16, and 32 options
|
||||
- ASCII, Vietnamese, long labels, emoji
|
||||
- all themes
|
||||
- current 512px/20 FPS/7s profile
|
||||
3. Record output bytes, cold/warm latency, CPU time, peak RSS, and temp bytes.
|
||||
4. Compare H.264 CRF 18/20/22 at 512px.
|
||||
5. Inspect labels, slice edges, shadows, glow, easing, and winner frame.
|
||||
6. Upload representative files to a development bot and inspect Android, iOS,
|
||||
Desktop, slow network, reduced-motion/autoplay settings.
|
||||
7. Require the selected H.264 profile to be visually equivalent and materially
|
||||
smaller than GIF before removing GIF production output.
|
||||
|
||||
Suggested acceptance gates:
|
||||
|
||||
- no renderer network socket or runtime download
|
||||
- exact winner and caption behavior
|
||||
- no obvious text/chroma artifacts at normal Telegram display size
|
||||
- warm render finishes within existing 30s command timeout
|
||||
- output remains well below Telegram's 50 MB limit and current 12 MB bot cap
|
||||
- worker crash always yields text fallback and leaves no temp files
|
||||
- bot stays responsive during a render
|
||||
|
||||
## Resources and References
|
||||
|
||||
### Official Documentation
|
||||
|
||||
- [Telegram Bot API: `sendAnimation`](https://core.telegram.org/bots/api#sendanimation)
|
||||
- [Remotion `renderMedia()`](https://www.remotion.dev/docs/renderer/render-media)
|
||||
- [Remotion server-side rendering](https://www.remotion.dev/docs/ssr)
|
||||
- [Go `image/gif`](https://pkg.go.dev/image/gif)
|
||||
|
||||
### Current Source and Evidence
|
||||
|
||||
- [`wheelofnames` GIF renderer](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/src/render/render-gif.js)
|
||||
- [`wheelofnames` composition](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/src/remotion/WheelComposition.jsx)
|
||||
- [`wheelofnames` container runtime](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/Dockerfile)
|
||||
- [`wheelofnames` benchmarks](https://github.com/tiennm99/wheelofnames/blob/f13055651e7a18a59b024f7b22266611cc843389/docs/render-benchmarks.md)
|
||||
|
||||
## Next Steps
|
||||
|
||||
No implementation now. If approved later:
|
||||
|
||||
1. Build a disposable combined-container benchmark first.
|
||||
2. Measure identical GIF and H.264 outputs.
|
||||
3. Confirm H.264 Telegram client quality.
|
||||
4. Only then plan the local persistent-worker integration.
|
||||
|
||||
## Unresolved Questions
|
||||
|
||||
- Does "all in house" allow a local persistent Node child process, or require
|
||||
one Go process only?
|
||||
- Is the larger Node/Chromium container acceptable on the Coolify host?
|
||||
- Must the first version retain the current 7-second timing exactly, or use the
|
||||
previously researched 2.95-second profile?
|
||||
@@ -0,0 +1,76 @@
|
||||
# Debug: LoL schedule fetch 403 Forbidden — root cause
|
||||
|
||||
Date: 2026-08-05 | Module: `internal/modules/lol` | Status: root cause confirmed, no code changed
|
||||
|
||||
## Symptom
|
||||
|
||||
```
|
||||
WARN lol_fetch status=403 body={"message":"Forbidden"}
|
||||
ERROR lol_fetch_fail err="lol API HTTP 403"
|
||||
```
|
||||
|
||||
Every call to `https://esports-api.lolesports.com/persisted/gw/getSchedule` fails; stale cache (60 min) expires, so `/lol` commands and daily push break.
|
||||
|
||||
## Root cause
|
||||
|
||||
Riot revoked/rotated the long-standing public web-client API key hardcoded at `internal/modules/lol/api_client.go:34` (`0TvQnueqKa5mxJntVWt0w4LpLfEkrV1Ta8rQBb9Z`). No replacement public key exists: the rebuilt lolesports.com no longer ships a key to the browser at all.
|
||||
|
||||
## Evidence
|
||||
|
||||
1. Reproduced from this host: `getSchedule` with the key → HTTP 403, body `{"message":"Forbidden"}`, headers `x-amzn-errortype: ForbiddenException` + `x-amz-apigw-id` present. Request reached AWS API Gateway and Gateway itself rejected the key — not a CloudFront/WAF/IP edge block (an edge block would not carry `x-amz-apigw-id`).
|
||||
2. Same 403 with no key, and with browser UA + `Origin`/`Referer: lolesports.com` — rules out UA/referer/origin gating; the key itself is dead.
|
||||
3. Downloaded all 49 JS chunks from current lolesports.com (Next.js on Netlify). No `esports-api.lolesports.com` reference, no embedded key. Bundle shows Apollo client with `uri: () => ESPORTS_EDGE_URL ?? "/api/gql"` and key injected server-side from `process.env.ESPORTS_API_KEY` — key moved behind their BFF, never sent to browsers.
|
||||
4. Probed `POST https://lolesports.com/api/gql` anonymously → HTTP 400 `PERSISTED_QUERY_ID_REQUIRED`: proxy is publicly reachable but accepts only pre-registered persisted-query IDs, no freeform GraphQL.
|
||||
|
||||
## Why now
|
||||
|
||||
Riot migrated lolesports.com to a new Next.js/GraphQL stack; the legacy `persisted/gw` REST API's public key was revoked as part of that migration. The code comment at `api_client.go:5-7` ("if Riot ever rotates it, lift the new value from their public JS bundle") no longer works — there is nothing to lift.
|
||||
|
||||
## Fix options (not implemented)
|
||||
|
||||
1. **Call `lolesports.com/api/gql` with persisted-query IDs** — sniff the ID + variables the live site sends for its schedule query (browser devtools), replay with `extensions.persistedQuery.sha256Hash`. Anonymous access works. Fragile: IDs change on each frontend deploy; needs re-sniff on breakage. Response shape differs from current `schedulePage` → parser rewrite.
|
||||
2. **Community mirror APIs** (e.g. Leaguepedia/Fandom cargo API, third-party esports APIs) — stable contracts, but different data model and licensing/ToS review needed.
|
||||
3. **Official Riot esports data** — Riot has no public esports schedule API in the developer portal; production key does not cover lolesports data.
|
||||
4. **Keep module, degrade gracefully** — if no replacement chosen, surface "schedule source unavailable" to users instead of daily error logs; disable daily push until fixed.
|
||||
|
||||
Recommendation: option 1 for continuity (accept fragility, keep 60-min stale-cache fallback and add alerting on repeated 403/400), else option 4 short-term.
|
||||
|
||||
## Follow-up: RGAPI key + LoL Esports Data Portal (checked 2026-08-05)
|
||||
|
||||
- User's `RGAPI-…` dev-portal key: valid (200 on `vn2.api.riotgames.com/lol/status/v4`), still 403 on `esports-api.lolesports.com` with both `x-api-key` and `X-Riot-Token`. Dev portal keys only cover `*.api.riotgames.com`; no esports schedule product there.
|
||||
- Riot dev diary "Introducing the New LoL Esports Data Portal": LDP built with Bayes Esports, replaced LEGs/ACS; targets pro teams/partners; community access "planned".
|
||||
- Bayes Esports acquired by GRID (bayesesports.com 301 → grid.gg). LoL data now behind GRID's paid portal (`grid.gg/get-league-of-legends/`). GRID Open Access (free tier) covers CS2/Dota2 only, and Series Events (schedules) is paid-tier even there.
|
||||
- Net: the official replacement is commercially gated; not viable for the bot. Fix options unchanged (persisted-query proxy / Leaguepedia / graceful degrade).
|
||||
|
||||
## Solution options (validated 2026-08-05)
|
||||
|
||||
1. **lolesports.com `/api/gql` persisted queries** — recommended primary. Anonymous access verified; needs sniffed sha256Hash + variables from site frontend; parser rewrite; alert on `PERSISTED_QUERY_NOT_FOUND`; keep stale cache.
|
||||
2. **PandaScore free tier** — `/lol/matches/upcoming`, token auth, ~1k req/h hobby tier, non-commercial. Stable contract; naming differs from Riot. Verify signup still self-serve.
|
||||
3. **Leaguepedia Cargo API** (`MatchSchedule` table) — live test from this host: immediate `ratelimited` (anon datacenter IP throttled). Needs Fandom bot-password auth + CC BY-SA attribution.
|
||||
4. **GRID** — official successor, paid for LoL; Open Access free tier = CS2/Dota2 only, schedules excluded. Not viable now; recheck quarterly.
|
||||
5. **Scrape `lolesports.com/schedule` RSC payload** — fallback only; more fragile than option 1, no advantage.
|
||||
6. **Graceful degrade** — regardless of source: user-facing "source unavailable" message + pause daily push on repeated failure.
|
||||
|
||||
Recommendation: 1 primary, 2 as designed fallback.
|
||||
|
||||
## Implemented (2026-08-05): option 1, gql persisted-query client
|
||||
|
||||
No browser available (headless ARM64 server) — persisted-query ID extracted statically instead of via devtools:
|
||||
|
||||
1. Bundle analysis: lolesports.com Apollo client uses `generatePersistedQueryIdsFromManifest` — IDs come from a build manifest, NOT computed sha256 (both `__meta__.hash` and sha256(print(doc)) are rejected by the gateway).
|
||||
2. Manifest = webpack chunk 29 (`loadManifest:()=>o.e(29)…` in bundle); URL resolved from webpack runtime chunk-hash map → `/_next/static/chunks/29.<hash>.js`; contains `{name, id, body}` per operation.
|
||||
3. Operation chosen: `homeEvents` — date-window filter (`eventDateStart/End`), pagination (`pages.newer` + `pageToken`), events carry startTime/state/type/blockName/league/match.strategy/matchTeams(result.outcome/gameWins). Same semantic contract as dead REST API. States verified identical: unstarted/inProgress/completed; pending result = `{outcome:null,gameWins:0}` — `scoreIsPublished` logic still correct.
|
||||
4. Gateway requirements discovered: `apollographql-client-name` + `-version` headers mandatory ("No client headers set"); `sport` enum lowercase `["lol"]`; Date params `YYYY-MM-DD`.
|
||||
|
||||
Code changes (`internal/modules/lol/`):
|
||||
- `api_client.go`: transport rewritten — POST /api/gql persisted op `homeEvents` (ID const + refresh procedure in package doc). Removed dead public API key. `gqlEvent→ScheduleEvent` mapping keeps module contract + bson cache shape unchanged. Date window padded ±1 day, exact [from,to) filtered client-side (gateway date tz semantics unspecified). In-band GraphQL errors (HTTP 200 + errors[]) surface as fetch errors; `PERSISTED_QUERY_NOT_IN_LIST` logged as `lol_persisted_query_rotated` ERROR with refresh hint. Older-page walk removed (window is query-bounded; newer-walk only).
|
||||
- `api_client_test.go`, `handlers_test.go`: fixtures → gateway shape; new tests for gql-error-as-fetch-error and exact-window filtering.
|
||||
- format.go/handlers.go/cron.go/subscribers.go untouched.
|
||||
|
||||
Verification: module tests green; live probe against real gateway returned 72 events for today+tomorrow incl. an in-progress LCK match, FilterMajor→10; `go test ./...`, `go vet`, `golangci-lint` all clean. Stale-cache fallback (60 min) retained.
|
||||
|
||||
## Unresolved questions
|
||||
|
||||
- Whether Riot will publish an official replacement API (watch RiotGames/developer-relations).
|
||||
- Persisted-query ID churn rate on lolesports.com deploys (determines option 1 maintenance cost).
|
||||
- PandaScore free-tier availability/terms as of now (not yet verified with a signup).
|
||||
@@ -0,0 +1,65 @@
|
||||
# Research: stable free LoL esports schedule source (replace/back up gql client)
|
||||
|
||||
Date: 2026-08-05 16:27 ICT | Sources: 5 web searches + live probes from bot's server | Context: commit b76d0ca (gql persisted-query client) works but breaks when Riot rotates the persisted-query ID.
|
||||
|
||||
## Executive summary
|
||||
|
||||
No free source beats the current gql client on data quality (official, live state, scores, league slugs). The stability gain is available two ways: **Leaguepedia** (no signup, proven decade-old MediaWiki API, works anon from this server — verified live today) or **PandaScore free tier** (most stable API contract, 1000 req/h, but adds account+token dependency). Recommendation: keep gql client primary, add Leaguepedia as the fallback source; skip PandaScore unless the user prefers a commercial-grade contract over zero-dependency.
|
||||
|
||||
## Candidates
|
||||
|
||||
### 1. Leaguepedia Cargo API — recommended fallback
|
||||
- Endpoint: `https://lol.fandom.com/api.php?action=cargoquery`, table `MatchSchedule` (Team1, Team2, DateTime_UTC, BestOf, Winner, scores, OverviewPage, Stream…).
|
||||
- **Verified live from the bot's server (2026-08-05):** proper descriptive UA + single paced request → HTTP 200 with correct data. Earlier `ratelimited` was the anon throttle (~1 req/min); burst of 2 rapid requests trips it.
|
||||
- Cross-validated against lolesports gateway: same match, same start time (Estral vs Ei Nerd 00:00 UTC) — data agrees.
|
||||
- Auth: Fandom account + bot password raises limits (exact numbers undocumented; anon ~1/min per mediawiki-api list thread). Anon is enough for this bot: 1 daily push + user commands behind 60-min cache, if requests are serialized.
|
||||
- License: CC BY-SA — attribution required (one line in /help or bot bio suffices).
|
||||
- Cons: different data model — full team names not codes, league via `OverviewPage` string not slug (needs mapping for FilterMajor), no live `inProgress` state (has Winner/score for finished). Community-entered data (major leagues near-instant, minor leagues can lag).
|
||||
- Stability: MediaWiki+Cargo API unchanged for ~a decade; immune to Riot frontend deploys.
|
||||
|
||||
### 2. PandaScore free tier — most stable contract
|
||||
- 1000 req/h free, no credit card (pricing page). Versioned REST (`/lol/matches/upcoming`, `/lol/matches/past`), token auth, real docs.
|
||||
- Cons: signup + API token to manage (new secret in env); team/league naming differs; free-tier terms can change unilaterally; third-party (not official Riot data, though sourced professionally).
|
||||
- Best choice if a long-lived stable contract matters more than zero-dependency.
|
||||
|
||||
### 3. Liquipedia LPDB API — viable but stricter
|
||||
- Free tier 60 req/h; **requires open-source project + non-commercial**; CC-BY-SA attribution; custom UA with contact mandatory; MediaWiki api.php capped 1 req/2s.
|
||||
- No advantage over Leaguepedia for LoL specifically; stricter terms. Skip.
|
||||
|
||||
### 4. Community iCal feeds (zlypher/lol-events, olol/lolesp-cal)
|
||||
- Free, zero auth, but single-maintainer projects (staleness risk), ICS carries no scores/state. Emergency backup grade only. zlypher feed last updated 2026-07 and itself sources Liquipedia.
|
||||
|
||||
### 5. lolesportsapi.com
|
||||
- Commercial wrapper, free 500 calls/month; unknown operator that itself scrapes Riot. Adds a middleman with less stability than our own gql client. Skip.
|
||||
|
||||
## Comparative summary
|
||||
|
||||
| Source | Signup | Quota (free) | Live state | Scores | Deploy-proof | Data model fit |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Current gql client | none | ample | ✅ | ✅ | ❌ ID rotates | ✅ native |
|
||||
| Leaguepedia | none | ~1/min anon | ❌ | ✅ | ✅ | ⚠️ mapping needed |
|
||||
| PandaScore | account+token | 1000/h | ✅ | ✅ | ✅ | ⚠️ mapping needed |
|
||||
| Liquipedia | none | 60/h LPDB | ❌ | ✅ | ✅ | ⚠️ + strict terms |
|
||||
| iCal feeds | none | n/a | ❌ | ❌ | ⚠️ maintainer | ❌ |
|
||||
|
||||
## Recommendation
|
||||
|
||||
1. Keep gql client primary (richest data, official).
|
||||
2. Implement Leaguepedia as fallback in `GetEventsWithFallback`'s failure path (after stale cache): serialize requests, ≥60s spacing, descriptive UA, map OverviewPage→league slug for the major-league allowlist, render without live-state features. Zero new secrets or accounts.
|
||||
3. Reconsider PandaScore only if Leaguepedia's anon limit proves painful in practice.
|
||||
4. Regardless: alert on `lol_persisted_query_rotated` so ID refresh happens fast (fallback covers the gap).
|
||||
|
||||
## Sources
|
||||
|
||||
- [Leaguepedia rate limit thread (mediawiki-api list)](https://lists.wikimedia.org/hyperkitty/list/mediawiki-api@lists.wikimedia.org/thread/A6VWUYRHLGGJWZ3USGEBQJSDMX6A4YCM/)
|
||||
- [PandaScore pricing](https://www.pandascore.co/pricing)
|
||||
- [Liquipedia API terms](https://liquipedia.net/api-terms-of-use), [usage guidelines](https://liquipedia.net/commons/Liquipedia:API_Usage_Guidelines)
|
||||
- [zlypher/lol-events iCal](https://github.com/zlypher/lol-events), [olol/lolesp-cal](https://olol.github.io/lolesp-cal/)
|
||||
- [river-mwclient (Leaguepedia wrapper by its dev)](https://pypi.org/project/river-mwclient)
|
||||
- Live probes: Leaguepedia cargoquery 200 from bot host (2026-08-05), cross-checked vs lolesports gateway.
|
||||
|
||||
## Unresolved questions
|
||||
|
||||
- Leaguepedia authenticated (bot-password) rate limit exact number — undocumented; ask on their Discord if anon 1/min ever binds.
|
||||
- PandaScore free-tier ToS details (commercial-use clause) — not verified beyond "no credit card"; check at signup if pursued.
|
||||
- Whether the bot repo being public matters to the user (required only for Liquipedia, not Leaguepedia/PandaScore).
|
||||
Reference in new issue
Block a user