feat(gitea-mirror): add cleanup of archived copies of deleted repos

Deletes the archived-* Gitea copies gitea-mirror keeps when a source in the
given owners disappears, and removes their tracking rows so they are not
re-mirrored. Dry run by default.
This commit is contained in:
tiennm99 committed 2026-10-03 13:09:05 +07:00
1 parent 4576b4dfd5
commit efd0848143
2 files changed
+106 -2

No files matched your search

@@ -1,6 +1,6 @@
---
name: gitea-mirror-maintenance
description: Detect and clean up failed, broken, or empty Gitea mirror repositories in the Coolify-deployed gitea + gitea-mirror stack, using tea and the gitea-mirror API. Use when the user asks to check mirror health, find failed or empty repos, investigate why a mirror did not sync or clone, delete broken mirror repos, reclaim disk space from partial clones, re-mirror repos that failed, or run routine mirror upkeep. Not for Gitea setup, upgrades, or deployment problems — those belong to the service's compose definition.
description: Detect and clean up failed, broken, or empty Gitea mirror repositories in the Coolify-deployed gitea + gitea-mirror stack, using tea and the gitea-mirror API. Use when the user asks to check mirror health, find failed or empty repos, investigate why a mirror did not sync or clone, delete broken mirror repos, delete archived copies of the user's own deleted repos, reclaim disk space from partial clones, re-mirror repos that failed, or run routine mirror upkeep. Not for Gitea setup, upgrades, or deployment problems — those belong to the service's compose definition.
---
# Gitea Mirror Maintenance
@@ -55,7 +55,8 @@ from four signals; no single one is sufficient:
2. **Upstream probe** of `original_url`, public repos only — separates "retry"
from "the source is gone".
3. **gitea-mirror API** — `GET /api/github/repositories`, each repo's
`status` and `errorMessage`, matched to Gitea by `mirroredLocation`.
`status` and `errorMessage`, matched to Gitea by `mirroredLocation`, or by
`fullName` when a failed mirror has had its location cleared.
4. **Gitea log** — `[repo: <Repository N:owner/name>]` sync errors.
### 2. Review the classification
@@ -111,6 +112,25 @@ script warns when the plan is over 15 minutes old.
Re-run detect; an empty plan means the stack is clean. Case A repos re-mirror
in the background — confirm they come back non-empty rather than assuming it.
## Archived repo cleanup
When a GitHub source disappears, gitea-mirror keeps the Gitea copy, renames it
`archived-<name>`, and keeps tracking it. To drop those copies for the user's
own namespaces:
```bash
scripts/cleanup-archived-repos.sh --login <login> --owners <owner1,owner2,...>
scripts/cleanup-archived-repos.sh --login <login> --owners <owner1,owner2,...> --apply
```
`--owners` is the user's own GitHub users and orgs; ask for them if they are
not known. Only an `archived-*` repo whose gitea-mirror row points at it is
deleted, so a repo the user named that way by hand is skipped. For each one it
deletes the Gitea repo, then removes the tracking row
(`DELETE /api/repositories` with `{"ids": [...]}`) so it is not re-mirrored.
Third-party archived copies stay; they are the only remaining copy of a source
someone else deleted. Show the dry run and get confirmation before `--apply`.
## Mirror status overview
```bash
@@ -0,0 +1,84 @@
#!/usr/bin/env bash
# Deletes the `archived-*` repositories gitea-mirror left in Gitea after their
# GitHub source disappeared, together with gitea-mirror's tracking row.
# Dry run by default.
#
# Usage: cleanup-archived-repos.sh --login <tea-login> --owners a,b,c [--apply]
# Env: GITEA_MIRROR_URL, GITEA_MIRROR_API_KEY
set -euo pipefail
LOGIN=""
OWNERS=""
APPLY=0
while [ $# -gt 0 ]; do
case "$1" in
--login) LOGIN="$2"; shift 2 ;;
--owners) OWNERS="$2"; shift 2 ;;
--apply) APPLY=1; shift ;;
*) echo "unknown argument: $1" >&2; exit 2 ;;
esac
done
[ -n "$LOGIN" ] && [ -n "$OWNERS" ] || { echo "--login and --owners are required" >&2; exit 2; }
[ -n "${GITEA_MIRROR_URL:-}" ] && [ -n "${GITEA_MIRROR_API_KEY:-}" ] ||
{ echo "GITEA_MIRROR_URL and GITEA_MIRROR_API_KEY are required" >&2; exit 2; }
WORK=$(mktemp -d)
trap 'rm -rf "$WORK"' EXIT
tea_api() { (cd "$WORK" && timeout 60 tea api --login "$LOGIN" "$@" </dev/null); }
mirror_api() {
curl -fsS --max-time 60 -H @<(printf 'x-api-key: %s\n' "$GITEA_MIRROR_API_KEY") "$@"
}
# Archived copies in the given owners, from Gitea.
: > "$WORK/gitea.json"
for owner in ${OWNERS//,/ }; do
page=1
while :; do
tea_api "/repos/search?q=archived-&owner=$owner&limit=50&page=$page" | jq '.data' > "$WORK/p.json"
[ "$(jq length "$WORK/p.json")" -gt 0 ] || break
jq -c --arg o "$owner" '.[] | select(.owner.login == $o and (.name | startswith("archived-")))' "$WORK/p.json" >> "$WORK/gitea.json"
page=$((page + 1))
done
done
# Keep only those gitea-mirror itself archived: its row points at that location.
mirror_api "${GITEA_MIRROR_URL%/}/api/github/repositories" > "$WORK/app.json"
PLAN=$(jq -s --slurpfile app "$WORK/app.json" '
($app[0].repositories | map(select((.mirroredLocation // "") != "")
| {key: (.mirroredLocation | ascii_downcase), value: .}) | from_entries) as $m
| map(($m[.full_name | ascii_downcase]) as $a
| {full_name, owner: .owner.login, name, size_MB: ((.size / 1024 * 10 | round) / 10),
mirror_id: ($a.id // null), source: ($a.fullName // null)})' "$WORK/gitea.json")
echo "=== ARCHIVED REPOS ==="
jq -r '.[] | if .mirror_id then "DELETE \(.full_name) (\(.size_MB) MB, source \(.source))"
else "SKIP \(.full_name) (not archived by gitea-mirror)" end' <<<"$PLAN"
TARGETS=$(jq -c '[.[] | select(.mirror_id)]' <<<"$PLAN")
COUNT=$(jq length <<<"$TARGETS")
[ "$COUNT" -gt 0 ] || { echo "Nothing to delete."; exit 0; }
if [ "$APPLY" -eq 0 ]; then
echo "$COUNT repo(s) would be deleted from Gitea and gitea-mirror. Re-run with --apply to execute."
exit 0
fi
echo "=== APPLYING to $COUNT repo(s) ==="
IDS=()
while IFS=$'\t' read -r fn owner name id; do
if (cd "$WORK" && timeout 120 tea repos delete --login "$LOGIN" --owner "$owner" --name "$name" --force </dev/null >/dev/null 2>&1); then
echo " OK delete $fn"; IDS+=("$id")
else
echo " FAIL delete $fn"
fi
done < <(jq -r '.[] | [.full_name, .owner, .name, .mirror_id] | @tsv' <<<"$TARGETS")
# Drop the tracking rows of the deleted copies, so they are not re-mirrored.
if [ "${#IDS[@]}" -gt 0 ]; then
body=$(printf '%s\n' "${IDS[@]}" | jq -R . | jq -s '{ids: .}')
if mirror_api -X DELETE -H 'Content-Type: application/json' -d "$body" \
"${GITEA_MIRROR_URL%/}/api/repositories" >/dev/null; then
echo " OK removed ${#IDS[@]} gitea-mirror row(s)"
else
echo " FAIL removing gitea-mirror rows"
fi
fi