feat: store session and exports in redis for cross-device portability

Replace the device-bound Telethon .session file and local CSV folders with
Redis-backed storage so the tool runs on any device from just .env.

- StringSession persisted at <prefix>:session:<phone> (no re-login per device)
- each group export saved as one self-contained JSON key per run:
  <prefix>:run:<phone>:<yyyymmddhhmmss>:<group_id>
- REDIS_PREFIX namespaces all keys; applied via a single key() helper
- no cross-key references: deleting any key cannot corrupt another's state
- list_exports() lists history, tolerant of deleted/corrupt keys
This commit is contained in:
tiennm99 committed 2026-06-08 16:49:34 +07:00
1 parent 294ecf571f
commit f03099a18e
7 files changed
+142 -29

No files matched your search

+2
View File
@@ -2,3 +2,5 @@ PHONE=+1234567890
API_ID=123456789 API_ID=123456789
API_HASH=0123456789abcdef0123456789abcdef API_HASH=0123456789abcdef0123456789abcdef
GROUP_IDS=1234567890,9876543210 GROUP_IDS=1234567890,9876543210
REDIS_URL=rediss://default:<password>@<host>:<port>
REDIS_PREFIX=telegram-export
+35 -9
View File
@@ -1,6 +1,6 @@
# export-telegram-group-members # export-telegram-group-members
Export Telegram group members to CSV using Telethon — admin auth required for full member visibility. Export Telegram group members using Telethon — admin auth required for full member visibility. Session and export history are stored in **Redis**, so you can run the tool on any device from just a `.env` file (no re-login, no copying session/CSV files).
## How to use ## How to use
@@ -17,24 +17,45 @@ pip install -r requirements.txt
``` ```
3. Create a new Telegram app at [https://my.telegram.org](https://my.telegram.org) and get the `api_id` and `api_hash`. 3. Create a new Telegram app at [https://my.telegram.org](https://my.telegram.org) and get the `api_id` and `api_hash`.
4. Copy `.env.example` to `.env` and fill in the `api_id`, `api_hash`, and `phone`. 4. Create a free Redis database (e.g. [Upstash](https://upstash.com)) and copy its `rediss://` connection URL.
5. Run the script: 5. Copy `.env.example` to `.env` and fill in `API_ID`, `API_HASH`, `PHONE`, `GROUP_IDS`, and `REDIS_URL`.
6. Run the script:
```bash ```bash
python main.py python main.py
``` ```
## Output format The first run asks for the Telegram login code once, then stores the session in Redis. Any later run — on any device pointed at the same Redis — reuses that session and **does not** prompt again.
Results are saved to a timestamped folder (e.g. `2025-01-15 10-30-00/`). Each group produces one CSV file named after the group title: ## Configuration
| Variable | Description |
|----------|-------------|
| `PHONE` | Telegram account phone (e.g. `+1234567890`) |
| `API_ID` / `API_HASH` | From [my.telegram.org](https://my.telegram.org) |
| `GROUP_IDS` | Comma-separated group IDs to export |
| `REDIS_URL` | Redis connection string (`rediss://default:<password>@<host>:<port>`) |
| `REDIS_PREFIX` | Key namespace (default `telegram-export`); isolates keys when sharing Redis with other projects |
## How data is stored
All data lives in Redis under `REDIS_PREFIX`. No key references another, so deleting any key can never corrupt another's state:
``` ```
id,username,first_name,last_name <prefix>:session:<phone> -> StringSession string (login)
123456789,johndoe,John,Doe <prefix>:run:<phone>:<yyyymmddhhmmss>:<group_id> -> one group's export as JSON:
987654321,,Jane,Smith { group_id, title, time,
members: [{ id, username, first_name, last_name }] }
``` ```
Columns: `id`, `username`, `first_name`, `last_name`. Members without a username have an empty `username` field. Each group is written as its own key per run; all groups in one run share the same `yyyymmddhhmmss` timestamp. Read the history programmatically:
```python
from common import list_exports
for rec in list_exports(): # sorted by (time, group_id)
print(rec['time'], rec['group_id'], rec['title'], len(rec['members']), 'members')
```
## Rate limits and visibility notes ## Rate limits and visibility notes
@@ -42,6 +63,11 @@ Columns: `id`, `username`, `first_name`, `last_name`. Members without a username
- For **supergroups**, only admins can retrieve the full member list — regular members see a partial list or get an error. - For **supergroups**, only admins can retrieve the full member list — regular members see a partial list or get an error.
- For private groups where you are not a member, access will be denied. - For private groups where you are not a member, access will be denied.
## Security
- The session string grants **full access to your Telegram account**. It lives only in Redis (use a TLS `rediss://` URL) and is never written to disk or committed to git.
- `.env` is git-ignored. Never commit your `REDIS_URL` or session string.
## License ## License
Apache-2.0 — see [LICENSE](LICENSE). Apache-2.0 — see [LICENSE](LICENSE).
+56 -16
View File
@@ -1,22 +1,62 @@
import csv """Export persistence in Redis.
import os
import re Each group's members are stored per export as one self-contained JSON key:
<prefix>:run:<phone>:<yyyymmddhhmmss>:<group_id>
-> {group_id, title, time, members:[{id,username,first_name,last_name}]}
All groups in a single run share one timestamp. No key references another, so
deleting any key can never corrupt another's state. Writing a group export is a
single atomic SET; listing scans the run keys.
"""
import json
from datetime import datetime from datetime import datetime
now = datetime.now() from config import key, phone, redis_client
folder_name = now.strftime('%Y-%m-%d %H-%M-%S')
os.mkdir(folder_name)
def remove_special_characters(string): def new_run_time():
return re.sub('[<>:"/\\\\|?*]', '', string) """Timestamp shared by every group in one run: yyyymmddhhmmss."""
return datetime.now().strftime('%Y%m%d%H%M%S')
def export_csv(group_name, members): def member_dict(member):
filename = remove_special_characters(group_name) + '.csv' return {
path = os.path.join(folder_name, filename) 'id': member.id,
with open(path, 'w', encoding='UTF-8') as f: 'username': member.username,
writer = csv.writer(f, delimiter=',', lineterminator='\n') 'first_name': member.first_name,
writer.writerow(['id', 'username', 'first_name', 'last_name']) 'last_name': member.last_name,
for member in members: }
writer.writerow([member.id, member.username, member.first_name, member.last_name])
def save_group_export(group_id, title, members, run_time):
"""Atomically persist one group's members for a run as a single JSON value."""
record = {
'group_id': group_id,
'title': title,
'time': run_time,
'members': [member_dict(m) for m in members],
}
redis_client.set(
key('run', phone, run_time, str(group_id)),
json.dumps(record, ensure_ascii=False),
)
def list_exports():
"""Return all group-export records, sorted by (time, group_id).
Tolerates keys deleted mid-scan and corrupt/non-JSON values (skips them).
"""
records = []
for export_key in redis_client.scan_iter(match=key('run', phone, '*')):
raw = redis_client.get(export_key)
if raw is None: # deleted between scan and get
continue
try:
records.append(json.loads(raw))
except (ValueError, TypeError): # corrupt / non-JSON value: skip, don't abort
continue
records.sort(key=lambda r: (r.get('time', ''), r.get('group_id', 0)))
return records
+16
View File
@@ -1,5 +1,6 @@
import os import os
import redis
from dotenv import load_dotenv from dotenv import load_dotenv
load_dotenv() load_dotenv()
@@ -9,3 +10,18 @@ api_id = os.getenv('API_ID')
api_hash = os.getenv('API_HASH') api_hash = os.getenv('API_HASH')
group_ids_str = os.getenv('GROUP_IDS', '') group_ids_str = os.getenv('GROUP_IDS', '')
group_ids = [int(id.strip()) for id in group_ids_str.split(',') if id.strip()] if group_ids_str else [] group_ids = [int(id.strip()) for id in group_ids_str.split(',') if id.strip()] if group_ids_str else []
# Shared Redis holds the Telegram session + export history so the tool runs on
# any device from just .env. REDIS_PREFIX namespaces every key so this project
# never collides with others sharing the same Redis instance.
redis_url = os.getenv('REDIS_URL')
if not redis_url:
raise SystemExit('REDIS_URL not set in .env (e.g. rediss://default:<password>@<host>:<port>)')
redis_prefix = os.getenv('REDIS_PREFIX', 'telegram-export')
redis_client = redis.from_url(redis_url, decode_responses=True)
def key(*parts):
"""Build a namespaced Redis key. Single source of the prefix (DRY)."""
return ':'.join([redis_prefix, *parts])
+13 -4
View File
@@ -1,13 +1,16 @@
import getpass import getpass
from telethon.errors import SessionPasswordNeededError from telethon.errors import SessionPasswordNeededError
from telethon.sessions import StringSession
from telethon.sync import TelegramClient from telethon.sync import TelegramClient
from telethon.tl.types import Chat, Channel from telethon.tl.types import Chat, Channel
from common import * from common import new_run_time, save_group_export
from config import * from config import api_hash, api_id, group_ids, phone
from session_store import load_session, save_session
client = TelegramClient(phone, api_id, api_hash) # Session loads from Redis: a saved string means no re-login on any device.
client = TelegramClient(StringSession(load_session()), api_id, api_hash)
client.connect() client.connect()
if not client.is_user_authorized(): if not client.is_user_authorized():
@@ -18,7 +21,10 @@ if not client.is_user_authorized():
client.sign_in(password=getpass.getpass()) client.sign_in(password=getpass.getpass())
client.start(phone) client.start(phone)
save_session(client.session.save())
run_time = new_run_time()
saved = 0
for group_id in group_ids: for group_id in group_ids:
try: try:
entity = client.get_entity(group_id) entity = client.get_entity(group_id)
@@ -28,7 +34,10 @@ for group_id in group_ids:
print(f'exporting {entity.title} (ID: {group_id})') print(f'exporting {entity.title} (ID: {group_id})')
members = client.get_participants(entity) members = client.get_participants(entity)
members.sort(key=lambda x: x.id) members.sort(key=lambda x: x.id)
export_csv(entity.title, members) save_group_export(group_id, entity.title, members, run_time)
saved += 1
print(f'export {entity.title} done.') print(f'export {entity.title} done.')
except Exception as e: except Exception as e:
print(f'error accessing group {group_id}: {e}') print(f'error accessing group {group_id}: {e}')
print(f'saved run {run_time}: {saved} group(s) to Redis.')
+1
View File
@@ -1,5 +1,6 @@
pyaes==1.6.1 pyaes==1.6.1
pyasn1==0.6.3 pyasn1==0.6.3
python-dotenv==1.2.2 python-dotenv==1.2.2
redis[hiredis]==5.2.1
rsa==4.9.1 rsa==4.9.1
Telethon==1.42.0 Telethon==1.42.0
+19
View File
@@ -0,0 +1,19 @@
"""Load/save the Telethon StringSession from Redis.
The session string grants full account access; it lives only in Redis (behind
TLS/auth), never on local disk. Deleting the key simply forces a clean re-login.
"""
from config import key, phone, redis_client
_SESSION_KEY = key('session', phone)
def load_session():
"""Return the stored StringSession string, or None if not yet authenticated."""
return redis_client.get(_SESSION_KEY)
def save_session(session_string):
"""Persist the StringSession string for reuse on any device."""
redis_client.set(_SESSION_KEY, session_string)