0 Fork 0

mirror of https://github.com/tiennm99/awesome-ai-dev-tools.git synced 2026-10-11 03:13:05 +00:00

Files

T

tiennm99 00c90c5be9 feat: split data update from site build; publish via Cloudflare Pages

The updater fetched GitHub metadata and rendered the dashboard payload in
a single step, so anything that published the site needed a GITHUB_TOKEN.
Cloudflare Pages builds from a Git webhook and has no business holding
one.

Split the tool into three modes. The default update step still fetches
GitHub and now records every API-sourced field in data/metadata.json,
which is committed alongside README.md and data/history.jsonl. The new
-build mode joins that snapshot with data/agents.yml and
data/history.jsonl to render dist/ with no network access and no token;
-check is unchanged.

Supporting changes:

- computeDeltaAt anchors the delta windows to when the data was fetched
  rather than the wall clock, so a redeploy days later reproduces the
  same Δ7d instead of sliding the window past its slack allowance.
- writeSiteData takes updatedAt explicitly for the same reason: the
  timestamp labels data freshness, not build time.
- sortStats is extracted from fetchStats so the build step re-ranks
  identically from committed metadata.
- An agents.yml entry with no metadata yet is omitted with a warning
  instead of failing the build, which would otherwise block every deploy
  between merging a new entry and the next nightly run.
- update.yml drops the GitHub Pages deploy steps and commits
  data/metadata.json; ci.yml runs `go run . -build` so a build that would
  break on deploy breaks in CI first.
- site/_headers stops the edge serving a stale data.json after a refresh.
- docs/DEPLOY.md covers the Cloudflare setup, including the one-time
  bootstrap of data/metadata.json that the build depends on.

Also carries the in-progress curation work already in the tree: the
module rename to awesome-ai-dev-tools, retagged entries, and removal of
the archived Roo-Code, void, continue and suna entries.

2026-09-16 22:24:48 +07:00

2.1 KiB

Raw Blame History

Local Development

Prerequisites

Go 1.23 or later
A GitHub personal access token (PAT)

Getting a GitHub Token

Go to https://github.com/settings/tokens
Click "Generate new token" → "Generate new token (classic)"
Give it a name (e.g., "awesome-ai-dev-tools")
Select scope: public_repo (needed to read public repo metadata)
Click "Generate token" and copy the value

The updater reads public repos only; public_repo scope is sufficient and safe.

The two steps

The tool separates fetching data from rendering the site, so only the fetch needs a token. See DEPLOY.md for why.

Update (needs a token, hits the network)

export GITHUB_TOKEN=ghp_your_token_here
go run .

This will:

Read data/agents.yml
Fetch live metadata from the GitHub GraphQL API
Append star counts to data/history.jsonl
Write the fetched fields to data/metadata.json
Regenerate README.md from templates/readme.tmpl

Build (offline, no token)

go run . -build

This joins data/agents.yml (tags and notes) with data/metadata.json (stars and repo metadata) and renders dist/ — a copy of site/ plus the generated dist/data.json. This is exactly what Cloudflare Pages runs.

Preview it with any static server:

go run . -build && python3 -m http.server -d dist 8080

If you only touched site/index.html or the tags in data/agents.yml, the build step alone is enough — no token required.

Validate (offline, no token)

go run . -check

Reverting Local Changes

Before opening a PR, undo local modifications:

git restore data/history.jsonl README.md

This removes the snapshot files generated by your local run, leaving only your edits to data/agents.yml.

Rate Limits

With GITHUB_TOKEN: 5,000 requests/hour
Without token: 60 requests/hour

Always set GITHUB_TOKEN when testing locally to avoid hitting the unauthenticated limit.

Next Steps

To add agents, see CONTRIBUTING.md. To publish the site, see DEPLOY.md.

2.1 KiB Raw Blame History