mirror of
https://github.com/tiennm99/awesome-ai-dev-tools.git
synced 2026-10-20 02:16:56 +00:00
feat: split data update from site build; publish via Cloudflare Pages
The updater fetched GitHub metadata and rendered the dashboard payload in a single step, so anything that published the site needed a GITHUB_TOKEN. Cloudflare Pages builds from a Git webhook and has no business holding one. Split the tool into three modes. The default update step still fetches GitHub and now records every API-sourced field in data/metadata.json, which is committed alongside README.md and data/history.jsonl. The new -build mode joins that snapshot with data/agents.yml and data/history.jsonl to render dist/ with no network access and no token; -check is unchanged. Supporting changes: - computeDeltaAt anchors the delta windows to when the data was fetched rather than the wall clock, so a redeploy days later reproduces the same Δ7d instead of sliding the window past its slack allowance. - writeSiteData takes updatedAt explicitly for the same reason: the timestamp labels data freshness, not build time. - sortStats is extracted from fetchStats so the build step re-ranks identically from committed metadata. - An agents.yml entry with no metadata yet is omitted with a warning instead of failing the build, which would otherwise block every deploy between merging a new entry and the next nightly run. - update.yml drops the GitHub Pages deploy steps and commits data/metadata.json; ci.yml runs `go run . -build` so a build that would break on deploy breaks in CI first. - site/_headers stops the edge serving a stale data.json after a refresh. - docs/DEPLOY.md covers the Cloudflare setup, including the one-time bootstrap of data/metadata.json that the build depends on. Also carries the in-progress curation work already in the tree: the module rename to awesome-ai-dev-tools, retagged entries, and removal of the archived Roo-Code, void, continue and suna entries.
This commit is contained in:
1 parent
3310ed36d4
commit
00c90c5be9
22 files changed
+728
-116
No files matched your search
+36
-4
@@ -9,25 +9,56 @@
|
||||
|
||||
1. Go to [https://github.com/settings/tokens](https://github.com/settings/tokens)
|
||||
2. Click "Generate new token" → "Generate new token (classic)"
|
||||
3. Give it a name (e.g., "ai-dev-tools")
|
||||
3. Give it a name (e.g., "awesome-ai-dev-tools")
|
||||
4. Select scope: **`public_repo`** (needed to read public repo metadata)
|
||||
5. Click "Generate token" and copy the value
|
||||
|
||||
The updater reads public repos only; `public_repo` scope is sufficient and safe.
|
||||
|
||||
## Running Locally
|
||||
## The two steps
|
||||
|
||||
The tool separates fetching data from rendering the site, so only the fetch
|
||||
needs a token. See [DEPLOY.md](./DEPLOY.md) for why.
|
||||
|
||||
### Update (needs a token, hits the network)
|
||||
|
||||
```bash
|
||||
export GITHUB_TOKEN=ghp_your_token_here
|
||||
go run .
|
||||
```
|
||||
|
||||
The tool will:
|
||||
This will:
|
||||
- Read `data/agents.yml`
|
||||
- Fetch live metadata from GitHub GraphQL API
|
||||
- Fetch live metadata from the GitHub GraphQL API
|
||||
- Append star counts to `data/history.jsonl`
|
||||
- Write the fetched fields to `data/metadata.json`
|
||||
- Regenerate `README.md` from `templates/readme.tmpl`
|
||||
|
||||
### Build (offline, no token)
|
||||
|
||||
```bash
|
||||
go run . -build
|
||||
```
|
||||
|
||||
This joins `data/agents.yml` (tags and notes) with `data/metadata.json` (stars
|
||||
and repo metadata) and renders `dist/` — a copy of `site/` plus the generated
|
||||
`dist/data.json`. This is exactly what Cloudflare Pages runs.
|
||||
|
||||
Preview it with any static server:
|
||||
|
||||
```bash
|
||||
go run . -build && python3 -m http.server -d dist 8080
|
||||
```
|
||||
|
||||
If you only touched `site/index.html` or the tags in `data/agents.yml`, the
|
||||
build step alone is enough — no token required.
|
||||
|
||||
### Validate (offline, no token)
|
||||
|
||||
```bash
|
||||
go run . -check
|
||||
```
|
||||
|
||||
## Reverting Local Changes
|
||||
|
||||
Before opening a PR, undo local modifications:
|
||||
@@ -48,3 +79,4 @@ Always set `GITHUB_TOKEN` when testing locally to avoid hitting the unauthentica
|
||||
## Next Steps
|
||||
|
||||
To add agents, see [CONTRIBUTING.md](./CONTRIBUTING.md).
|
||||
To publish the site, see [DEPLOY.md](./DEPLOY.md).
|
||||
Reference in new issue
Block a user