mirror of
https://github.com/tiennm99/export-telegram-group-members.git
synced 2026-10-11 03:13:26 +00:00
feat: store session and exports in redis for cross-device portability
Replace the device-bound Telethon .session file and local CSV folders with Redis-backed storage so the tool runs on any device from just .env. - StringSession persisted at <prefix>:session:<phone> (no re-login per device) - each group export saved as one self-contained JSON key per run: <prefix>:run:<phone>:<yyyymmddhhmmss>:<group_id> - REDIS_PREFIX namespaces all keys; applied via a single key() helper - no cross-key references: deleting any key cannot corrupt another's state - list_exports() lists history, tolerant of deleted/corrupt keys
This commit is contained in:
1 parent
294ecf571f
commit
f03099a18e
7 files changed
+142
-29
No files matched your search
@@ -2,3 +2,5 @@ PHONE=+1234567890
|
|||||||
API_ID=123456789
|
API_ID=123456789
|
||||||
API_HASH=0123456789abcdef0123456789abcdef
|
API_HASH=0123456789abcdef0123456789abcdef
|
||||||
GROUP_IDS=1234567890,9876543210
|
GROUP_IDS=1234567890,9876543210
|
||||||
|
REDIS_URL=rediss://default:<password>@<host>:<port>
|
||||||
|
REDIS_PREFIX=telegram-export
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# export-telegram-group-members
|
# export-telegram-group-members
|
||||||
|
|
||||||
Export Telegram group members to CSV using Telethon — admin auth required for full member visibility.
|
Export Telegram group members using Telethon — admin auth required for full member visibility. Session and export history are stored in **Redis**, so you can run the tool on any device from just a `.env` file (no re-login, no copying session/CSV files).
|
||||||
|
|
||||||
## How to use
|
## How to use
|
||||||
|
|
||||||
@@ -17,24 +17,45 @@ pip install -r requirements.txt
|
|||||||
```
|
```
|
||||||
|
|
||||||
3. Create a new Telegram app at [https://my.telegram.org](https://my.telegram.org) and get the `api_id` and `api_hash`.
|
3. Create a new Telegram app at [https://my.telegram.org](https://my.telegram.org) and get the `api_id` and `api_hash`.
|
||||||
4. Copy `.env.example` to `.env` and fill in the `api_id`, `api_hash`, and `phone`.
|
4. Create a free Redis database (e.g. [Upstash](https://upstash.com)) and copy its `rediss://` connection URL.
|
||||||
5. Run the script:
|
5. Copy `.env.example` to `.env` and fill in `API_ID`, `API_HASH`, `PHONE`, `GROUP_IDS`, and `REDIS_URL`.
|
||||||
|
6. Run the script:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python main.py
|
python main.py
|
||||||
```
|
```
|
||||||
|
|
||||||
## Output format
|
The first run asks for the Telegram login code once, then stores the session in Redis. Any later run — on any device pointed at the same Redis — reuses that session and **does not** prompt again.
|
||||||
|
|
||||||
Results are saved to a timestamped folder (e.g. `2025-01-15 10-30-00/`). Each group produces one CSV file named after the group title:
|
## Configuration
|
||||||
|
|
||||||
|
| Variable | Description |
|
||||||
|
|----------|-------------|
|
||||||
|
| `PHONE` | Telegram account phone (e.g. `+1234567890`) |
|
||||||
|
| `API_ID` / `API_HASH` | From [my.telegram.org](https://my.telegram.org) |
|
||||||
|
| `GROUP_IDS` | Comma-separated group IDs to export |
|
||||||
|
| `REDIS_URL` | Redis connection string (`rediss://default:<password>@<host>:<port>`) |
|
||||||
|
| `REDIS_PREFIX` | Key namespace (default `telegram-export`); isolates keys when sharing Redis with other projects |
|
||||||
|
|
||||||
|
## How data is stored
|
||||||
|
|
||||||
|
All data lives in Redis under `REDIS_PREFIX`. No key references another, so deleting any key can never corrupt another's state:
|
||||||
|
|
||||||
```
|
```
|
||||||
id,username,first_name,last_name
|
<prefix>:session:<phone> -> StringSession string (login)
|
||||||
123456789,johndoe,John,Doe
|
<prefix>:run:<phone>:<yyyymmddhhmmss>:<group_id> -> one group's export as JSON:
|
||||||
987654321,,Jane,Smith
|
{ group_id, title, time,
|
||||||
|
members: [{ id, username, first_name, last_name }] }
|
||||||
```
|
```
|
||||||
|
|
||||||
Columns: `id`, `username`, `first_name`, `last_name`. Members without a username have an empty `username` field.
|
Each group is written as its own key per run; all groups in one run share the same `yyyymmddhhmmss` timestamp. Read the history programmatically:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from common import list_exports
|
||||||
|
|
||||||
|
for rec in list_exports(): # sorted by (time, group_id)
|
||||||
|
print(rec['time'], rec['group_id'], rec['title'], len(rec['members']), 'members')
|
||||||
|
```
|
||||||
|
|
||||||
## Rate limits and visibility notes
|
## Rate limits and visibility notes
|
||||||
|
|
||||||
@@ -42,6 +63,11 @@ Columns: `id`, `username`, `first_name`, `last_name`. Members without a username
|
|||||||
- For **supergroups**, only admins can retrieve the full member list — regular members see a partial list or get an error.
|
- For **supergroups**, only admins can retrieve the full member list — regular members see a partial list or get an error.
|
||||||
- For private groups where you are not a member, access will be denied.
|
- For private groups where you are not a member, access will be denied.
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
- The session string grants **full access to your Telegram account**. It lives only in Redis (use a TLS `rediss://` URL) and is never written to disk or committed to git.
|
||||||
|
- `.env` is git-ignored. Never commit your `REDIS_URL` or session string.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
Apache-2.0 — see [LICENSE](LICENSE).
|
Apache-2.0 — see [LICENSE](LICENSE).
|
||||||
@@ -1,22 +1,62 @@
|
|||||||
import csv
|
"""Export persistence in Redis.
|
||||||
import os
|
|
||||||
import re
|
Each group's members are stored per export as one self-contained JSON key:
|
||||||
|
|
||||||
|
<prefix>:run:<phone>:<yyyymmddhhmmss>:<group_id>
|
||||||
|
-> {group_id, title, time, members:[{id,username,first_name,last_name}]}
|
||||||
|
|
||||||
|
All groups in a single run share one timestamp. No key references another, so
|
||||||
|
deleting any key can never corrupt another's state. Writing a group export is a
|
||||||
|
single atomic SET; listing scans the run keys.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
now = datetime.now()
|
from config import key, phone, redis_client
|
||||||
folder_name = now.strftime('%Y-%m-%d %H-%M-%S')
|
|
||||||
os.mkdir(folder_name)
|
|
||||||
|
|
||||||
|
|
||||||
def remove_special_characters(string):
|
def new_run_time():
|
||||||
return re.sub('[<>:"/\\\\|?*]', '', string)
|
"""Timestamp shared by every group in one run: yyyymmddhhmmss."""
|
||||||
|
return datetime.now().strftime('%Y%m%d%H%M%S')
|
||||||
|
|
||||||
|
|
||||||
def export_csv(group_name, members):
|
def member_dict(member):
|
||||||
filename = remove_special_characters(group_name) + '.csv'
|
return {
|
||||||
path = os.path.join(folder_name, filename)
|
'id': member.id,
|
||||||
with open(path, 'w', encoding='UTF-8') as f:
|
'username': member.username,
|
||||||
writer = csv.writer(f, delimiter=',', lineterminator='\n')
|
'first_name': member.first_name,
|
||||||
writer.writerow(['id', 'username', 'first_name', 'last_name'])
|
'last_name': member.last_name,
|
||||||
for member in members:
|
}
|
||||||
writer.writerow([member.id, member.username, member.first_name, member.last_name])
|
|
||||||
|
|
||||||
|
def save_group_export(group_id, title, members, run_time):
|
||||||
|
"""Atomically persist one group's members for a run as a single JSON value."""
|
||||||
|
record = {
|
||||||
|
'group_id': group_id,
|
||||||
|
'title': title,
|
||||||
|
'time': run_time,
|
||||||
|
'members': [member_dict(m) for m in members],
|
||||||
|
}
|
||||||
|
redis_client.set(
|
||||||
|
key('run', phone, run_time, str(group_id)),
|
||||||
|
json.dumps(record, ensure_ascii=False),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def list_exports():
|
||||||
|
"""Return all group-export records, sorted by (time, group_id).
|
||||||
|
|
||||||
|
Tolerates keys deleted mid-scan and corrupt/non-JSON values (skips them).
|
||||||
|
"""
|
||||||
|
records = []
|
||||||
|
for export_key in redis_client.scan_iter(match=key('run', phone, '*')):
|
||||||
|
raw = redis_client.get(export_key)
|
||||||
|
if raw is None: # deleted between scan and get
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
records.append(json.loads(raw))
|
||||||
|
except (ValueError, TypeError): # corrupt / non-JSON value: skip, don't abort
|
||||||
|
continue
|
||||||
|
records.sort(key=lambda r: (r.get('time', ''), r.get('group_id', 0)))
|
||||||
|
return records
|
||||||
@@ -1,5 +1,6 @@
|
|||||||
import os
|
import os
|
||||||
|
|
||||||
|
import redis
|
||||||
from dotenv import load_dotenv
|
from dotenv import load_dotenv
|
||||||
|
|
||||||
load_dotenv()
|
load_dotenv()
|
||||||
@@ -9,3 +10,18 @@ api_id = os.getenv('API_ID')
|
|||||||
api_hash = os.getenv('API_HASH')
|
api_hash = os.getenv('API_HASH')
|
||||||
group_ids_str = os.getenv('GROUP_IDS', '')
|
group_ids_str = os.getenv('GROUP_IDS', '')
|
||||||
group_ids = [int(id.strip()) for id in group_ids_str.split(',') if id.strip()] if group_ids_str else []
|
group_ids = [int(id.strip()) for id in group_ids_str.split(',') if id.strip()] if group_ids_str else []
|
||||||
|
|
||||||
|
# Shared Redis holds the Telegram session + export history so the tool runs on
|
||||||
|
# any device from just .env. REDIS_PREFIX namespaces every key so this project
|
||||||
|
# never collides with others sharing the same Redis instance.
|
||||||
|
redis_url = os.getenv('REDIS_URL')
|
||||||
|
if not redis_url:
|
||||||
|
raise SystemExit('REDIS_URL not set in .env (e.g. rediss://default:<password>@<host>:<port>)')
|
||||||
|
|
||||||
|
redis_prefix = os.getenv('REDIS_PREFIX', 'telegram-export')
|
||||||
|
redis_client = redis.from_url(redis_url, decode_responses=True)
|
||||||
|
|
||||||
|
|
||||||
|
def key(*parts):
|
||||||
|
"""Build a namespaced Redis key. Single source of the prefix (DRY)."""
|
||||||
|
return ':'.join([redis_prefix, *parts])
|
||||||
@@ -1,13 +1,16 @@
|
|||||||
import getpass
|
import getpass
|
||||||
|
|
||||||
from telethon.errors import SessionPasswordNeededError
|
from telethon.errors import SessionPasswordNeededError
|
||||||
|
from telethon.sessions import StringSession
|
||||||
from telethon.sync import TelegramClient
|
from telethon.sync import TelegramClient
|
||||||
from telethon.tl.types import Chat, Channel
|
from telethon.tl.types import Chat, Channel
|
||||||
|
|
||||||
from common import *
|
from common import new_run_time, save_group_export
|
||||||
from config import *
|
from config import api_hash, api_id, group_ids, phone
|
||||||
|
from session_store import load_session, save_session
|
||||||
|
|
||||||
client = TelegramClient(phone, api_id, api_hash)
|
# Session loads from Redis: a saved string means no re-login on any device.
|
||||||
|
client = TelegramClient(StringSession(load_session()), api_id, api_hash)
|
||||||
|
|
||||||
client.connect()
|
client.connect()
|
||||||
if not client.is_user_authorized():
|
if not client.is_user_authorized():
|
||||||
@@ -18,7 +21,10 @@ if not client.is_user_authorized():
|
|||||||
client.sign_in(password=getpass.getpass())
|
client.sign_in(password=getpass.getpass())
|
||||||
|
|
||||||
client.start(phone)
|
client.start(phone)
|
||||||
|
save_session(client.session.save())
|
||||||
|
|
||||||
|
run_time = new_run_time()
|
||||||
|
saved = 0
|
||||||
for group_id in group_ids:
|
for group_id in group_ids:
|
||||||
try:
|
try:
|
||||||
entity = client.get_entity(group_id)
|
entity = client.get_entity(group_id)
|
||||||
@@ -28,7 +34,10 @@ for group_id in group_ids:
|
|||||||
print(f'exporting {entity.title} (ID: {group_id})')
|
print(f'exporting {entity.title} (ID: {group_id})')
|
||||||
members = client.get_participants(entity)
|
members = client.get_participants(entity)
|
||||||
members.sort(key=lambda x: x.id)
|
members.sort(key=lambda x: x.id)
|
||||||
export_csv(entity.title, members)
|
save_group_export(group_id, entity.title, members, run_time)
|
||||||
|
saved += 1
|
||||||
print(f'export {entity.title} done.')
|
print(f'export {entity.title} done.')
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f'error accessing group {group_id}: {e}')
|
print(f'error accessing group {group_id}: {e}')
|
||||||
|
|
||||||
|
print(f'saved run {run_time}: {saved} group(s) to Redis.')
|
||||||
@@ -1,5 +1,6 @@
|
|||||||
pyaes==1.6.1
|
pyaes==1.6.1
|
||||||
pyasn1==0.6.3
|
pyasn1==0.6.3
|
||||||
python-dotenv==1.2.2
|
python-dotenv==1.2.2
|
||||||
|
redis[hiredis]==5.2.1
|
||||||
rsa==4.9.1
|
rsa==4.9.1
|
||||||
Telethon==1.42.0
|
Telethon==1.42.0
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
"""Load/save the Telethon StringSession from Redis.
|
||||||
|
|
||||||
|
The session string grants full account access; it lives only in Redis (behind
|
||||||
|
TLS/auth), never on local disk. Deleting the key simply forces a clean re-login.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from config import key, phone, redis_client
|
||||||
|
|
||||||
|
_SESSION_KEY = key('session', phone)
|
||||||
|
|
||||||
|
|
||||||
|
def load_session():
|
||||||
|
"""Return the stored StringSession string, or None if not yet authenticated."""
|
||||||
|
return redis_client.get(_SESSION_KEY)
|
||||||
|
|
||||||
|
|
||||||
|
def save_session(session_string):
|
||||||
|
"""Persist the StringSession string for reuse on any device."""
|
||||||
|
redis_client.set(_SESSION_KEY, session_string)
|
||||||
Reference in new issue
Block a user