feat: target any rclone remote instead of only WebDAV

The pipeline never depended on WebDAV; only the naming, docs and preflight
did. Replace the WebDAV-root listing with a portable check: a named remote
must appear in 'rclone listremotes', and creating the destination proves
reachability and credentials. On-the-fly connection strings have no config
entry, so the name check is skipped for them.

Read the remote list into a variable rather than piping it into 'grep -q':
grep exits on the first match and kills rclone with SIGPIPE, which pipefail
reports as failure, intermittently rejecting a configured remote.

Document that rclone's flags are all settable through their RCLONE_*
environment variables, so the upload side is tunable without new options.

README.md

@@ -1,13 +1,14 @@
 # telegram-exporter
 
-Export a Telegram chat's media to a **WebDAV** server, using far less local disk
+Export a Telegram chat's media to **any rclone remote** — S3, Google Drive,
-than the chat's total size.
+Dropbox, Backblaze B2, SFTP, WebDAV, or anything else rclone supports — using
+far less local disk than the chat's total size.
 
 `run.sh` runs [tdl](https://github.com/iyear/tdl) and
 [rclone](https://rclone.org/) as a rolling pipeline: tdl downloads into a small
-staging directory while rclone concurrently moves finished files to WebDAV and
+staging directory while rclone concurrently moves finished files to the remote
-deletes the local copies. Local disk only ever holds the files in flight plus
+and deletes the local copies. Local disk only ever holds the files in flight
-one sync interval of throughput, so a multi-terabyte chat exports fine on a
+plus one sync interval of throughput, so a multi-terabyte chat exports fine on a
 small disk. Telegram caps a single file at 2 GB (4 GB from premium uploaders),
 so a few dozen GB of staging covers the worst case regardless of chat size.
 
@@ -31,18 +32,25 @@ Both steps are one-time.
 # 1) log in to Telegram with your user account (phone + code + 2FA)
 tdl login
 
-# 2) create the WebDAV remote
+# 2) configure the destination. The interactive wizard covers every backend:
-rclone config create tg-webdav webdav \
+rclone config
-  url=https://dav.example.com/remote.php/dav/files/you \
+
+# ...or create one non-interactively, e.g.
+rclone config create gdrive drive
+rclone config create b2 b2 account=KEY_ID key=APP_KEY
+rclone config create dav webdav url=https://dav.example.com/remote.php/dav/files/you \
   vendor=other user=YOU pass=SECRET
 
-rclone lsd tg-webdav:      # verify it works
+rclone listremotes          # confirm the name you will pass to -r
 ```
 
+Any rclone remote form works, including on-the-fly connection strings
+(`:webdav,url=https://...:/path`).
+
 ## Usage
 
 ```bash
-./run.sh -r tg-webdav:tg-export -c @mygroup
+./run.sh -r gdrive:telegram/media -c @mygroup
 ```
 
 That is the whole flow. It exports the chat's message metadata to
@@ -53,7 +61,7 @@ and warnings go to stderr; press Ctrl-C at any point and it stops cleanly.
 
 | Flag | Meaning |
 |------|---------|
-| `-r REMOTE:PATH` | **Required.** rclone destination, e.g. `tg-webdav:tg-export` |
+| `-r REMOTE:PATH` | **Required.** rclone destination, e.g. `gdrive:telegram/media`, `s3:bucket/tg`, `dav:tg-export` |
 | `-c CHAT` | Chat to export when the JSON does not exist yet: `@username` or a numeric chat id |
 | `-f FILE` | Export JSON to download from (default `export.json`) |
 | `-d DIR` | Staging directory (default `./staging`) |
@@ -64,14 +72,23 @@ and warnings go to stderr; press Ctrl-C at any point and it stops cleanly.
 Anything after `--` is passed straight to `tdl dl`:
 
 ```bash
-./run.sh -r tg-webdav:tg-export -- -t 4 -l 1      # calmer parallelism, fewer flood waits
+./run.sh -r gdrive:telegram/media -- -t 4 -l 1      # calmer parallelism, fewer flood waits
-./run.sh -r tg-webdav:tg-export -- -i mp4,mkv     # only these file extensions
+./run.sh -r gdrive:telegram/media -- -i mp4,mkv     # only these file extensions
-./run.sh -r tg-webdav:tg-export -- -e jpg,png     # skip these file extensions
+./run.sh -r gdrive:telegram/media -- -e jpg,png     # skip these file extensions
 ```
 
 tdl defaults to `-t 8 -l 4`, which is aggressive; lower it if you hit flood
 waits on a large export.
 
+### Tuning the upload
+
+rclone reads every one of its flags from an environment variable, so the upload
+side is tunable without touching the script:
+
+```bash
+RCLONE_TRANSFERS=8 RCLONE_BWLIMIT=20M ./run.sh -r s3:bucket/tg -c @mygroup
+```
+
 ### Exporting a subset
 
 Generate the JSON yourself when you want a narrower export, then point `-f` at
@@ -79,7 +96,7 @@ it:
 
 ```bash
 tdl chat export -c @mygroup -T id -i 1000,5000 --all --with-content -o part.json
-./run.sh -r tg-webdav:tg-export -f part.json
+./run.sh -r gdrive:telegram/media -f part.json
 ```
 
 `tdl chat export` takes `-T time|id|last` with `-i` as the range, and `-f` as an
@@ -91,7 +108,7 @@ Re-run the same command. Both legs resume independently and nothing is
 downloaded or uploaded twice.
 
 Keep the same `export.json` between runs: `--skip-same` compares against the
-**staging** directory, which is empty once files have moved to WebDAV, so
+**staging** directory, which is empty once files have moved to the remote, so
 cross-run deduplication rests on tdl's own `--continue` tracking. If you must
 start from a fresh export, narrow it to the missing message-id range
 (`-T id -i <last>,<max>`) rather than re-downloading everything.
@@ -112,13 +129,16 @@ start from a fresh export, narrow it to the missing message-id range
   and keeps staging for the next attempt.
 - **A dead remote filling the disk.** Five consecutive rclone failures abort the
   run instead of letting staging grow unbounded.
+- **Typos and bad credentials.** Before downloading anything, the remote must be
+  present in `rclone listremotes` (skipped for connection strings) and the
+  destination must be creatable, which proves both reachability and auth.
 
 Exit codes: `0` success, `2` usage error, `3` rclone failure, `130`/`143`
 interrupted, anything else is tdl's own exit code.
 
 ## Limits
 
-- Streaming with no staging at all (piping download chunks straight into a
+- Streaming with no staging at all (piping download chunks straight to the
-  WebDAV `PUT`) is not possible with tdl and would require custom code.
+  remote) is not possible with tdl and would require custom code.
 - The script is bash; the two tools it drives are cross-platform, but Windows
   needs WSL or Git Bash.

run.sh

@@ -1,11 +1,12 @@
 #!/usr/bin/env bash
 #
 # Rolling pipeline: tdl downloads Telegram media into a small staging directory
-# while rclone concurrently moves finished files to a WebDAV remote. Local disk
+# while rclone concurrently moves finished files to any rclone remote (S3,
-# only ever holds the files in flight plus one sync interval of throughput, so a
+# Google Drive, SFTP, WebDAV, B2, ...). Local disk only ever holds the files in
-# group larger than the local disk can still be exported.
+# flight plus one sync interval of throughput, so a chat larger than the local
+# disk can still be exported.
 #
-# See README.md ("Special case: exporting to WebDAV") for the background.
+# See README.md for the background.
 #
 # Exit codes: 0 ok, 2 usage error, 3 rclone failure, 130/143 interrupted,
 # anything else is tdl's own exit code.
@@ -34,7 +35,7 @@ usage() {
 Usage: $PROG -r REMOTE:PATH [options] [-- extra tdl dl args...]
 
 Required:
-  -r REMOTE:PATH  rclone destination, e.g. tg-webdav:tg-export
+  -r REMOTE:PATH  rclone destination, e.g. gdrive:telegram/media
 
 Options:
   -f FILE         tdl export JSON (default: $export_file)
@@ -46,7 +47,7 @@ Options:
   -h              this help
 
 Everything after -- is appended to the 'tdl dl' command, e.g.
-  $PROG -r tg-webdav:tg-export -- -t 4 -l 1
+  $PROG -r gdrive:telegram/media -- -t 4 -l 1
 USAGE
 }
 
@@ -80,8 +81,21 @@ for tool in tdl rclone; do
   command -v "$tool" >/dev/null || die "$tool is not installed or not on PATH"
 done
 
-rclone lsd "${remote%%:*}:" >/dev/null 2>&1 \
+# A named remote must exist in the config; a leading ':' means an on-the-fly
-  || die "rclone cannot reach remote '${remote%%:*}:' — check 'rclone config'"
+# connection string, which has no config entry to check. Listing the remote's
+# root is not portable (some backends refuse it), so reachability and
+# credentials are proven by creating the destination, which rclone would create
+# on the first move anyway.
+if [[ $remote != :* ]]; then
+  # Read the list into a variable first: piping it into 'grep -q' lets grep exit
+  # on the first match and kill rclone with SIGPIPE, which pipefail then reports
+  # as a failed pipeline -- rejecting a remote that is in fact configured.
+  remotes=$(rclone listremotes 2>/dev/null || true)
+  grep -qx -- "${remote%%:*}:" <<<"$remotes" \
+    || die "rclone remote '${remote%%:*}:' is not configured — see 'rclone listremotes'"
+fi
+rclone mkdir "$remote" >/dev/null 2>&1 \
+  || die "cannot reach '$remote' — check credentials and connectivity"
 
 # The export JSON only lists messages; it is cheap to keep and required for both
 # legs to stay resumable, so never regenerate it when it already exists.