mirror of
https://github.com/tiennm99/composes.git
synced 2026-10-11 03:13:16 +00:00
feat(gitea-mirror): add cleanup of archived copies of deleted repos
Deletes the archived-* Gitea copies gitea-mirror keeps when a source in the given owners disappears, and removes their tracking rows so they are not re-mirrored. Dry run by default.
This commit is contained in:
1 parent
4576b4dfd5
commit
efd0848143
2 files changed
+106
-2
No files matched your search
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: gitea-mirror-maintenance
|
||||
description: Detect and clean up failed, broken, or empty Gitea mirror repositories in the Coolify-deployed gitea + gitea-mirror stack, using tea and the gitea-mirror API. Use when the user asks to check mirror health, find failed or empty repos, investigate why a mirror did not sync or clone, delete broken mirror repos, reclaim disk space from partial clones, re-mirror repos that failed, or run routine mirror upkeep. Not for Gitea setup, upgrades, or deployment problems — those belong to the service's compose definition.
|
||||
description: Detect and clean up failed, broken, or empty Gitea mirror repositories in the Coolify-deployed gitea + gitea-mirror stack, using tea and the gitea-mirror API. Use when the user asks to check mirror health, find failed or empty repos, investigate why a mirror did not sync or clone, delete broken mirror repos, delete archived copies of the user's own deleted repos, reclaim disk space from partial clones, re-mirror repos that failed, or run routine mirror upkeep. Not for Gitea setup, upgrades, or deployment problems — those belong to the service's compose definition.
|
||||
---
|
||||
|
||||
# Gitea Mirror Maintenance
|
||||
@@ -55,7 +55,8 @@ from four signals; no single one is sufficient:
|
||||
2. **Upstream probe** of `original_url`, public repos only — separates "retry"
|
||||
from "the source is gone".
|
||||
3. **gitea-mirror API** — `GET /api/github/repositories`, each repo's
|
||||
`status` and `errorMessage`, matched to Gitea by `mirroredLocation`.
|
||||
`status` and `errorMessage`, matched to Gitea by `mirroredLocation`, or by
|
||||
`fullName` when a failed mirror has had its location cleared.
|
||||
4. **Gitea log** — `[repo: <Repository N:owner/name>]` sync errors.
|
||||
|
||||
### 2. Review the classification
|
||||
@@ -111,6 +112,25 @@ script warns when the plan is over 15 minutes old.
|
||||
Re-run detect; an empty plan means the stack is clean. Case A repos re-mirror
|
||||
in the background — confirm they come back non-empty rather than assuming it.
|
||||
|
||||
## Archived repo cleanup
|
||||
|
||||
When a GitHub source disappears, gitea-mirror keeps the Gitea copy, renames it
|
||||
`archived-<name>`, and keeps tracking it. To drop those copies for the user's
|
||||
own namespaces:
|
||||
|
||||
```bash
|
||||
scripts/cleanup-archived-repos.sh --login <login> --owners <owner1,owner2,...>
|
||||
scripts/cleanup-archived-repos.sh --login <login> --owners <owner1,owner2,...> --apply
|
||||
```
|
||||
|
||||
`--owners` is the user's own GitHub users and orgs; ask for them if they are
|
||||
not known. Only an `archived-*` repo whose gitea-mirror row points at it is
|
||||
deleted, so a repo the user named that way by hand is skipped. For each one it
|
||||
deletes the Gitea repo, then removes the tracking row
|
||||
(`DELETE /api/repositories` with `{"ids": [...]}`) so it is not re-mirrored.
|
||||
Third-party archived copies stay; they are the only remaining copy of a source
|
||||
someone else deleted. Show the dry run and get confirmation before `--apply`.
|
||||
|
||||
## Mirror status overview
|
||||
|
||||
```bash
|
||||
|
||||
+84
@@ -0,0 +1,84 @@
|
||||
#!/usr/bin/env bash
|
||||
# Deletes the `archived-*` repositories gitea-mirror left in Gitea after their
|
||||
# GitHub source disappeared, together with gitea-mirror's tracking row.
|
||||
# Dry run by default.
|
||||
#
|
||||
# Usage: cleanup-archived-repos.sh --login <tea-login> --owners a,b,c [--apply]
|
||||
# Env: GITEA_MIRROR_URL, GITEA_MIRROR_API_KEY
|
||||
set -euo pipefail
|
||||
|
||||
LOGIN=""
|
||||
OWNERS=""
|
||||
APPLY=0
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--login) LOGIN="$2"; shift 2 ;;
|
||||
--owners) OWNERS="$2"; shift 2 ;;
|
||||
--apply) APPLY=1; shift ;;
|
||||
*) echo "unknown argument: $1" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
[ -n "$LOGIN" ] && [ -n "$OWNERS" ] || { echo "--login and --owners are required" >&2; exit 2; }
|
||||
[ -n "${GITEA_MIRROR_URL:-}" ] && [ -n "${GITEA_MIRROR_API_KEY:-}" ] ||
|
||||
{ echo "GITEA_MIRROR_URL and GITEA_MIRROR_API_KEY are required" >&2; exit 2; }
|
||||
|
||||
WORK=$(mktemp -d)
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
tea_api() { (cd "$WORK" && timeout 60 tea api --login "$LOGIN" "$@" </dev/null); }
|
||||
mirror_api() {
|
||||
curl -fsS --max-time 60 -H @<(printf 'x-api-key: %s\n' "$GITEA_MIRROR_API_KEY") "$@"
|
||||
}
|
||||
|
||||
# Archived copies in the given owners, from Gitea.
|
||||
: > "$WORK/gitea.json"
|
||||
for owner in ${OWNERS//,/ }; do
|
||||
page=1
|
||||
while :; do
|
||||
tea_api "/repos/search?q=archived-&owner=$owner&limit=50&page=$page" | jq '.data' > "$WORK/p.json"
|
||||
[ "$(jq length "$WORK/p.json")" -gt 0 ] || break
|
||||
jq -c --arg o "$owner" '.[] | select(.owner.login == $o and (.name | startswith("archived-")))' "$WORK/p.json" >> "$WORK/gitea.json"
|
||||
page=$((page + 1))
|
||||
done
|
||||
done
|
||||
|
||||
# Keep only those gitea-mirror itself archived: its row points at that location.
|
||||
mirror_api "${GITEA_MIRROR_URL%/}/api/github/repositories" > "$WORK/app.json"
|
||||
PLAN=$(jq -s --slurpfile app "$WORK/app.json" '
|
||||
($app[0].repositories | map(select((.mirroredLocation // "") != "")
|
||||
| {key: (.mirroredLocation | ascii_downcase), value: .}) | from_entries) as $m
|
||||
| map(($m[.full_name | ascii_downcase]) as $a
|
||||
| {full_name, owner: .owner.login, name, size_MB: ((.size / 1024 * 10 | round) / 10),
|
||||
mirror_id: ($a.id // null), source: ($a.fullName // null)})' "$WORK/gitea.json")
|
||||
|
||||
echo "=== ARCHIVED REPOS ==="
|
||||
jq -r '.[] | if .mirror_id then "DELETE \(.full_name) (\(.size_MB) MB, source \(.source))"
|
||||
else "SKIP \(.full_name) (not archived by gitea-mirror)" end' <<<"$PLAN"
|
||||
TARGETS=$(jq -c '[.[] | select(.mirror_id)]' <<<"$PLAN")
|
||||
COUNT=$(jq length <<<"$TARGETS")
|
||||
[ "$COUNT" -gt 0 ] || { echo "Nothing to delete."; exit 0; }
|
||||
|
||||
if [ "$APPLY" -eq 0 ]; then
|
||||
echo "$COUNT repo(s) would be deleted from Gitea and gitea-mirror. Re-run with --apply to execute."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "=== APPLYING to $COUNT repo(s) ==="
|
||||
IDS=()
|
||||
while IFS=$'\t' read -r fn owner name id; do
|
||||
if (cd "$WORK" && timeout 120 tea repos delete --login "$LOGIN" --owner "$owner" --name "$name" --force </dev/null >/dev/null 2>&1); then
|
||||
echo " OK delete $fn"; IDS+=("$id")
|
||||
else
|
||||
echo " FAIL delete $fn"
|
||||
fi
|
||||
done < <(jq -r '.[] | [.full_name, .owner, .name, .mirror_id] | @tsv' <<<"$TARGETS")
|
||||
|
||||
# Drop the tracking rows of the deleted copies, so they are not re-mirrored.
|
||||
if [ "${#IDS[@]}" -gt 0 ]; then
|
||||
body=$(printf '%s\n' "${IDS[@]}" | jq -R . | jq -s '{ids: .}')
|
||||
if mirror_api -X DELETE -H 'Content-Type: application/json' -d "$body" \
|
||||
"${GITEA_MIRROR_URL%/}/api/repositories" >/dev/null; then
|
||||
echo " OK removed ${#IDS[@]} gitea-mirror row(s)"
|
||||
else
|
||||
echo " FAIL removing gitea-mirror rows"
|
||||
fi
|
||||
fi
|
||||
Reference in new issue
Block a user