feat: download, upload and drive a chat to completion in one process

Phases 4 through 6: the two legs and the command that joins them.

Downloads go to <name>.part and are renamed only once complete, so a file
without the suffix is always whole. That is what lets the upload leg treat
"exists" as "finished" — the property run.sh could only approximate with a
filename convention plus an age guard, because it could not see inside tdl.

Every finished file is checked against the size Telegram reported, and that
check rather than the error is the authoritative signal. core's downloader logs
a failed transfer and returns nil, and its completion callback is deferred on
that named return, so a failure arrives indistinguishable from a success.
Trusting it would promote a truncated file and archive it as complete.

The disk cap is a semaphore over bytes. A download reserves its own size before
starting and releases it only after the upload confirms, so a slow remote
stalls downloads by itself. Blocking the iterator is safe because the
downloader calls it from its dispatch loop while workers run in a group, so a
blocked iterator never stops the uploads that free the space. Gone with it: the
du polling, the SIGSTOP and SIGCONT suspension, the min-age guard, the
temp-file filter and the sweep-failure counter.

A cap smaller than the largest file is refused up front. The semaphore could
never admit it, and a run blocked on a file it can never start looks exactly
like a stalled remote.

Uploads re-state each object to prove its size before the local copy is gone,
closing a gap where a truncated upload was only noticed by a later verify.

The destination is created before the chat is read. It is also the credentials
check, and doing it first means a bad destination fails in seconds rather than
after a full history walk.

One invocation converges: each item is checked against the index immediately
before download, so there are no passes and re-running is the resume path.
Options that no longer exist say what replaced them instead of failing as
unknown flags.

Verified end to end against the live chat and a scratch remote path: two files
downloaded, uploaded, confirmed present at the right size, staging left empty.
This commit is contained in:
tiennm99 committed 2026-09-06 19:21:46 +07:00
1 parent 1393abdae2
commit a81e7eaacb
13 files changed
+1561 -1

No files matched your search

+14
View File
@@ -98,6 +98,20 @@ func Resolve(ctx context.Context, remote string) (fs.Fs, error) {
return f, nil
}
// EnsureDir creates the destination if it is not there yet.
//
// A destination that does not exist yet is the normal case for a first run, and
// the shell pipeline created it up front for the same reason — the call doubles
// as the reachability and credentials check, since a remote that refuses a
// mkdir will refuse the uploads too. Doing it before the chat is read means a
// bad destination fails in seconds rather than after a full history walk.
func EnsureDir(ctx context.Context, f fs.Fs) error {
if err := f.Mkdir(ctx, ""); err != nil {
return fmt.Errorf("cannot create %q — check credentials and connectivity: %w", f.String(), err)
}
return nil
}
// FreeBytes reports free space on the remote.
//
// Backends without quota reporting return ok=false rather than an error: the
+7
View File
@@ -2,6 +2,7 @@ package remote
import (
"context"
"errors"
"fmt"
"path"
"slices"
@@ -88,6 +89,12 @@ func BuildIndex(ctx context.Context, f fs.Fs, dialogID int64) (*Index, error) {
idx.byID[id] = append(idx.byID[id], name)
}
}); err != nil {
// A destination that does not exist yet holds nothing. That is an empty
// index, not a failure — it is what a first run against a new path looks
// like, and treating it as an error would make verify unusable there.
if errors.Is(err, fs.ErrorDirNotFound) {
return idx, nil
}
return nil, fmt.Errorf("list %s: %w", f.String(), err)
}
return idx, nil