chore: tune defaults for a slow async-commit remote

Backends that finish an upload as a server-side task, pikpak among them,
leave it pending long enough that rclone abandons the transfer once
--low-level-retries polls run out. Default RCLONE_TRANSFERS to 2 to keep
that task queue short and RCLONE_LOW_LEVEL_RETRIES to 20 to wait a slow
commit out. Both use the :=default form, so an exported value still wins.

Shorten the --min-age default to 45s: excluding '*.tmp' is what actually
keeps an unfinished download from being uploaded, so the age guard only
needs to cover a rename racing a sweep, and two minutes of dwell made
staging larger than it had to be.

Sweep every 120s rather than 300s in the driver, which shrinks the backlog
each sweep has to move, and raise the pass limit to 30 to suit chats that
need more rounds than 20 to converge.

README.md

@@ -66,7 +66,7 @@ and warnings go to stderr; press Ctrl-C at any point and it stops cleanly.
 | `-f FILE` | Export JSON to download from (default `export-<chat>.json` with `-c`, else `export.json`) |
 | `-d DIR` | Staging directory (default `./staging`) |
 | `-i SECONDS` | Seconds between rclone sweeps (default `60`) |
-| `-a AGE` | rclone `--min-age`, a second guard against moving files still being written (default `2m`) |
+| `-a AGE` | rclone `--min-age`, a second guard against moving files still being written (default `45s`) |
 | `-m SIZE` | Cap the staging directory at `SIZE` (`K`/`M`/`G`/`T`, binary), e.g. `40G`. Unset means no cap (see below) |
 | `-h` | Help |
 
@@ -120,6 +120,28 @@ side is tunable without touching the script:
 RCLONE_TRANSFERS=8 RCLONE_BWLIMIT=20M ./run.sh -r s3:bucket/tg -c @mygroup
 ```
 
+`run.sh` sets two of those itself, and only when the caller has not:
+
+| Variable | Default here | rclone's own default | Why |
+|----------|--------------|----------------------|-----|
+| `RCLONE_TRANSFERS` | `2` | `4` | Backends that commit an upload as a server-side async task queue those tasks; less parallelism keeps the queue short |
+| `RCLONE_LOW_LEVEL_RETRIES` | `20` | `10` | Each retry re-polls a pending task, so a slow commit is waited out instead of failing the transfer |
+
+Both exist because of one failure mode. On pikpak an upload finishes in two
+phases — rclone sends the bytes, then a server-side task must reach
+`PHASE_TYPE_COMPLETE`. rclone waits 500 ms and then polls, giving up after
+`--low-level-retries` attempts with:
+
+```
+ERROR : <file>: Failed to copy: can't verify the task is completed: ... Phase:"PHASE_TYPE_PENDING"
+```
+
+Nothing is lost when that happens — the message is followed by `Not deleting
+source as copy failed`, the file stays in staging and the next sweep retries
+it. But it wastes the upload, and it counts against a `-m` cap, since a file
+that keeps failing can never be drained. Raise the retries further if you still
+see it.
+
 ### Capping the staging directory
 
 Without `-m`, staging grows whenever tdl downloads faster than rclone uploads,
@@ -236,9 +258,9 @@ subset, and repeat.
 | `-c CHAT` | Chat to export metadata for on the first pass |
 | `-f FILE` | Export JSON (default `export-<chat>.json`) |
 | `-d DIR` | Staging directory (default `./staging`) |
-| `-i SECONDS` | rclone sweep interval (default `300`) |
+| `-i SECONDS` | rclone sweep interval (default `120`) |
 | `-m SIZE` | Staging cap passed through to `run.sh`, e.g. `40G` |
-| `-p N` | Maximum passes (default `20`) |
+| `-p N` | Maximum passes (default `30`) |
 | `-q GIB` | Stop if remote free space falls below this (default `5`) |
 
 It stops when the verifier reports complete (exit `0`), when a pass fetches

export-until-complete.sh

@@ -13,15 +13,15 @@
 #   -c CHAT         chat id/username, used for the initial metadata export
 #   -f FILE         export JSON (default export-<chat>.json)
 #   -d DIR          staging directory (default ./staging)
-#   -i SECONDS      rclone sweep interval (default 300)
-#   -p N            maximum passes (default 20)
+#   -i SECONDS      rclone sweep interval (default 120)
+#   -p N            maximum passes (default 30)
 #   -m SIZE         cap the staging directory at SIZE, e.g. 40G (default: no cap)
 #   -q GIB          stop if remote free space falls below this (default 5)
 
 set -euo pipefail
 
-remote='' chat='' export_file='' staging='./staging' interval=300 max_staging=''
-max_passes=20 min_free_gib=5
+remote='' chat='' export_file='' staging='./staging' interval=120 max_staging=''
+max_passes=30 min_free_gib=5
 
 while getopts ':r:c:f:d:i:p:q:m:h' o; do case $o in
   r) remote=$OPTARG ;;   c) chat=$OPTARG ;;      f) export_file=$OPTARG ;;

run.sh

@@ -27,7 +27,7 @@ chat=''
 staging='./staging'
 remote=''
 interval=60
-min_age='2m'
+min_age='45s'
 max_staging=''      # empty: staging grows as fast as tdl fills it
 max_sync_failures=5
 cap_check_interval=10   # seconds between -m checks, independent of -i
@@ -132,6 +132,15 @@ if [[ -z $export_file ]]; then
   fi
 fi
 
+# pikpak-style backends finish an upload as a server-side async task, and
+# rclone abandons a still-pending one once --low-level-retries polls run out,
+# failing a transfer that would have succeeded. Fewer parallel transfers keep
+# that queue short and more retries wait it out. These are rclone's own
+# environment variables, so whatever the caller already exported wins.
+: "${RCLONE_TRANSFERS:=2}"
+: "${RCLONE_LOW_LEVEL_RETRIES:=20}"
+export RCLONE_TRANSFERS RCLONE_LOW_LEVEL_RETRIES
+
 for tool in tdl rclone; do
   command -v "$tool" >/dev/null || die "$tool is not installed or not on PATH"
 done