From 921f11baf111838b77dadee4e82bd761d6146b37 Mon Sep 17 00:00:00 2001 From: tiennm99 Date: Sat, 6 Dec 2025 17:52:39 +0700 Subject: [PATCH] feat(speckit): implement --- .env.example | 40 +- .gitignore | 1 + README.md | 163 +++- go.mod | 67 ++ go.sum | 180 ++++ main.go | 936 +++++++++++++++++++++ specs/001-export-telegram-members/tasks.md | 84 +- 7 files changed, 1401 insertions(+), 70 deletions(-) create mode 100644 go.mod create mode 100644 go.sum create mode 100644 main.go diff --git a/.env.example b/.env.example index 49481f5..2757f0d 100644 --- a/.env.example +++ b/.env.example @@ -1,17 +1,25 @@ PHONE=+1234567890 -API_ID=123456789 -API_HASH=0123456789abcdef0123456789abcdef -PROXY_HOST=127.0.0.1 -PROXY_PORT=8080 -PROXY_SECRET=00000000000000000000000000000000 -# From https://github.com/RevEngine3r/telethon-faketls: -# """ -# You can use a base64 secret or hex one, -# But remember to -# Remove "ee" from the starting of hex secret -# And -# Remove "7" from the starting of the base64 secret. -# Example: -# ee9b43b87555bf9464e02bdcd2db8932b07777772e736974652e636f6d -> 9b43b87555bf9464e02bdcd2db8932b07777772e736974652e636f6d -# 7gEBAQEBAQEBAQEBAQEBAQFsaWIuYXJ2YW5jbG91ZC5jb20= -> gEBAQEBAQEBAQEBAQEBAQFsaWIuYXJ2YW5jbG91ZC5jb20= -# """ +API_ID=12345678 +API_HASH=your_api_hash_here +GROUP_ID=-1001234567890 + +# Output directory (default: out) +OUTPUT_DIR=out + +# Verbose logging (default: false) +VERBOSE=false + +# Export options (default: true for ID, username, first_name, last_name; false for others for privacy) +INCLUDE_ID=true +INCLUDE_USERNAME=true +INCLUDE_FIRST_NAME=true +INCLUDE_LAST_NAME=true +INCLUDE_IS_BOT=false +INCLUDE_IS_SCAM=false +INCLUDE_IS_FAKE=false +INCLUDE_PHONE_NUMBER=false + +# Optional proxy settings +PROXY_HOST=proxy.example.com +PROXY_PORT=443 +PROXY_SECRET=your_proxy_secret diff --git a/.gitignore b/.gitignore index 67aee0b..5b84d43 100644 --- a/.gitignore +++ b/.gitignore @@ -162,3 +162,4 @@ cython_debug/ .idea/ *.session *.csv +out/ diff --git a/README.md b/README.md index a82e382..4728b06 100644 --- a/README.md +++ b/README.md @@ -1,25 +1,164 @@ -# export-telegram-group-members +# Telegram Group Members Exporter -Export Telegram group members +A Go application that exports Telegram group members to JSON format. -## How to use +## Features -1. Clone this repository: +- Export members from Telegram groups using group ID +- Support for both regular groups and supergroups/channels +- Configurable member information fields +- Rate limiting and retry handling +- Progress tracking during export +- Support for proxy connections -```bash -git clone https://github.com/tiennm99/export-telegram-group-members.git +## Prerequisites + +- Go 1.21 or higher +- Telegram API credentials (API ID and API Hash) from https://my.telegram.org +- Phone number registered with Telegram + +## Setup + +1. **Get Telegram API credentials:** + - Visit https://my.telegram.org + - Login with your phone number + - Go to "API development tools" + - Create new application to get API ID and API Hash + +2. **Clone the repository:** + ```bash + git clone + cd + ``` + +3. **Create environment file:** + ```bash + cp .env.example .env + ``` + +4. **Configure environment variables in `.env`:** + ```env + PHONE=+1234567890 + API_ID=12345678 + API_HASH=your_api_hash_here + GROUP_ID=-1001234567890 + OUTPUT_DIR=out + VERBOSE=false + + # Export options (default: true for ID, username, first_name, last_name; false for others for privacy) + INCLUDE_ID=true + INCLUDE_USERNAME=true + INCLUDE_FIRST_NAME=true + INCLUDE_LAST_NAME=true + INCLUDE_IS_BOT=false + INCLUDE_IS_SCAM=false + INCLUDE_IS_FAKE=false + INCLUDE_PHONE_NUMBER=false + + # Optional proxy settings + PROXY_HOST=proxy.example.com + PROXY_PORT=443 + PROXY_SECRET=your_proxy_secret + ``` + +## Usage + +### Basic Usage + +1. Run the application: + ```bash + go run main.go + ``` + +2. If `GROUP_ID` is not specified in the environment, the application will show a list of accessible groups for you to select from. + +3. Enter the authentication code sent to your Telegram when prompted. + +4. The application will export the member list to a JSON file in the `out/` directory. + +### Configuration Options + +- `GROUP_ID`: Specify a group ID directly to export from that group +- `OUTPUT_DIR`: Directory to save the exported JSON file (default: `out`) +- `VERBOSE`: Enable verbose logging (default: `false`) +- `INCLUDE_*`: Control which member fields to include in the export + +## Export Format + +The exported JSON file contains: + +```json +{ + "members": [ + { + "id": 123456789, + "username": "username", + "first_name": "First", + "last_name": "Last", + "is_bot": false, + "is_scam": false, + "is_fake": false, + "phone_number": "1234567890" + } + ], + "memberCount": 150, + "exportTime": "2025-12-06T17:11:00Z" +} ``` -2. Install requirements: +## Building + +To build the application: ```bash -pip install -r requirements.txt +go build -o telegram-exporter main.go ``` -3. Create a new Telegram app at [https://my.telegram.org](https://my.telegram.org) and get the `api_id` and `api_hash`. -4. Copy `.env.example` to `.env` and fill in the `api_id`, `api_hash`, and `phone`. -5. Run the script: +Then run: ```bash -python main.py +./telegram-exporter ``` + +## Error Handling + +The application includes robust error handling for: +- Authentication failures +- Invalid group IDs or inaccessible groups +- Rate limiting by Telegram API +- Network timeouts +- Transient network failures (with retry logic) + +## Performance + +- Supports exporting large groups with thousands of members +- Implements pagination to handle large datasets +- Includes performance timing to monitor export duration + +## Security Considerations + +- Store your Telegram API credentials securely +- Don't commit `.env` file to version control +- Be mindful of privacy settings when exporting member information +- Only export member information you have permission to access + +## Troubleshooting + +### Authentication Issues +- Ensure phone number is in international format (+1234567890) +- Verify API credentials are correct +- Check that phone number is properly registered with Telegram + +### Group Access Issues +- Verify the group ID is correct +- Ensure you have access to the group +- Confirm you're using the correct format (negative ID for supergroups) + +### Network Issues +- Check internet connection +- If behind firewall, try using proxy settings +- Verify API endpoints are not blocked in your region + +## License + +This project is licensed under the terms specified in the LICENSE file. \ No newline at end of file diff --git a/go.mod b/go.mod new file mode 100644 index 0000000..142be4f --- /dev/null +++ b/go.mod @@ -0,0 +1,67 @@ +module telegram-exporter + +go 1.24.0 + +toolchain go1.24.11 + +require ( + github.com/celestix/gotgproto v1.0.0-beta22 + github.com/joho/godotenv v1.5.1 + github.com/xelaj/mtproto v1.0.0 +) + +require ( + github.com/AnimeKaizoku/cacher v1.0.3-0.20250508132714-ddc7471efeef // indirect + github.com/cenkalti/backoff/v4 v4.3.0 // indirect + github.com/coder/websocket v1.8.14 // indirect + github.com/dlclark/regexp2 v1.11.5 // indirect + github.com/dustin/go-humanize v1.0.1 // indirect + github.com/fatih/color v1.18.0 // indirect + github.com/fatih/structtag v1.2.0 // indirect + github.com/ghodss/yaml v1.0.0 // indirect + github.com/glebarez/go-sqlite v1.22.0 // indirect + github.com/glebarez/sqlite v1.11.0 // indirect + github.com/go-faster/errors v0.7.1 // indirect + github.com/go-faster/jx v1.2.0 // indirect + github.com/go-faster/xor v1.0.0 // indirect + github.com/go-faster/yaml v0.4.6 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/gotd/ige v0.2.2 // indirect + github.com/gotd/neo v0.1.5 // indirect + github.com/gotd/td v0.135.0 // indirect + github.com/jinzhu/inflection v1.0.0 // indirect + github.com/jinzhu/now v1.1.5 // indirect + github.com/k0kubun/pp v3.0.1+incompatible // indirect + github.com/klauspost/compress v1.18.2 // indirect + github.com/mattn/go-colorable v0.1.14 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/ncruces/go-strftime v0.1.9 // indirect + github.com/ogen-go/ogen v1.16.0 // indirect + github.com/pkg/errors v0.9.1 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect + github.com/segmentio/asm v1.2.1 // indirect + github.com/shopspring/decimal v1.4.0 // indirect + github.com/xelaj/errs v0.0.0-20200831133608-d1c11863e019 // indirect + github.com/xelaj/go-dry v0.0.0-20201104165138-61a25872c05a // indirect + go.opentelemetry.io/otel v1.38.0 // indirect + go.opentelemetry.io/otel/metric v1.38.0 // indirect + go.opentelemetry.io/otel/trace v1.38.0 // indirect + go.uber.org/atomic v1.11.0 // indirect + go.uber.org/multierr v1.11.0 // indirect + go.uber.org/zap v1.27.1 // indirect + golang.org/x/crypto v0.45.0 // indirect + golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8 // indirect + golang.org/x/mod v0.30.0 // indirect + golang.org/x/net v0.47.0 // indirect + golang.org/x/sync v0.18.0 // indirect + golang.org/x/sys v0.38.0 // indirect + golang.org/x/text v0.31.0 // indirect + golang.org/x/tools v0.39.0 // indirect + gopkg.in/yaml.v2 v2.4.0 // indirect + gorm.io/gorm v1.25.12 // indirect + modernc.org/libc v1.61.8 // indirect + modernc.org/mathutil v1.7.1 // indirect + modernc.org/memory v1.8.2 // indirect + modernc.org/sqlite v1.34.4 // indirect + rsc.io/qr v0.2.0 // indirect +) diff --git a/go.sum b/go.sum new file mode 100644 index 0000000..eb06e9c --- /dev/null +++ b/go.sum @@ -0,0 +1,180 @@ +github.com/AnimeKaizoku/cacher v1.0.3-0.20250508132714-ddc7471efeef h1:y8llZexgesBazUg/zdIlrZKHPFJ2zk9YxlLAJ5x4DpQ= +github.com/AnimeKaizoku/cacher v1.0.3-0.20250508132714-ddc7471efeef/go.mod h1:jw0de/b0K6W7Y3T9rHCMGVKUf6oG7hENNcssxYcZTCc= +github.com/celestix/gotgproto v1.0.0-beta22 h1:Iu78cFA08nV8+flmxKs9CJ3W73+HG30fx0nLOs5A6fI= +github.com/celestix/gotgproto v1.0.0-beta22/go.mod h1:JYC9Js/5KLUhFR5M2RslQi2DFAcF7EdrgJMXo0YrzGQ= +github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8= +github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE= +github.com/coder/websocket v1.8.14 h1:9L0p0iKiNOibykf283eHkKUHHrpG7f65OE3BhhO7v9g= +github.com/coder/websocket v1.8.14/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ= +github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM= +github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU= +github.com/fatih/structtag v1.2.0 h1:/OdNE99OxoI/PqaW/SuSK9uxxT3f/tcSZgon/ssNSx4= +github.com/fatih/structtag v1.2.0/go.mod h1:mBJUNpUnHmRKrKlQQlmCrh5PuhftFbNv8Ys4/aAZl94= +github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk= +github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= +github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec+ruQ= +github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc= +github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw= +github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ= +github.com/go-faster/errors v0.7.1 h1:MkJTnDoEdi9pDabt1dpWf7AA8/BaSYZqibYyhZ20AYg= +github.com/go-faster/errors v0.7.1/go.mod h1:5ySTjWFiphBs07IKuiL69nxdfd5+fzh1u7FPGZP2quo= +github.com/go-faster/jx v1.2.0 h1:T2YHJPrFaYu21fJtUxC9GzmluKu8rVIFDwwGBKTDseI= +github.com/go-faster/jx v1.2.0/go.mod h1:UWLOVDmMG597a5tBFPLIWJdUxz5/2emOpfsj9Neg0PE= +github.com/go-faster/xor v0.3.0/go.mod h1:x5CaDY9UKErKzqfRfFZdfu+OSTfoZny3w5Ak7UxcipQ= +github.com/go-faster/xor v1.0.0 h1:2o8vTOgErSGHP3/7XwA5ib1FTtUsNtwCoLLBjl31X38= +github.com/go-faster/xor v1.0.0/go.mod h1:x5CaDY9UKErKzqfRfFZdfu+OSTfoZny3w5Ak7UxcipQ= +github.com/go-faster/yaml v0.4.6 h1:lOK/EhI04gCpPgPhgt0bChS6bvw7G3WwI8xxVe0sw9I= +github.com/go-faster/yaml v0.4.6/go.mod h1:390dRIvV4zbnO7qC9FGo6YYutc+wyyUSHBgbXL52eXk= +github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= +github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo= +github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/gotd/ige v0.2.2 h1:XQ9dJZwBfDnOGSTxKXBGP4gMud3Qku2ekScRjDWWfEk= +github.com/gotd/ige v0.2.2/go.mod h1:tuCRb+Y5Y3eNTo3ypIfNpQ4MFjrnONiL2jN2AKZXmb0= +github.com/gotd/neo v0.1.5 h1:oj0iQfMbGClP8xI59x7fE/uHoTJD7NZH9oV1WNuPukQ= +github.com/gotd/neo v0.1.5/go.mod h1:9A2a4bn9zL6FADufBdt7tZt+WMhvZoc5gWXihOPoiBQ= +github.com/gotd/td v0.135.0 h1:RuVWPxiy2WoaJdLklIIKmVTowe7NOBJTM7diV9lgjmE= +github.com/gotd/td v0.135.0/go.mod h1:mStcqs/9FXhNhWnPTguptSwqkQbRIwXLw3SCSpzPJxM= +github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD/E= +github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc= +github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ= +github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8= +github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0= +github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4= +github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88 h1:uC1QfSlInpQF+M0ao65imhwqKnz3Q2z/d8PWZRMQvDM= +github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88/go.mod h1:3w7q1U84EfirKl04SVQ/s7nPm1ZPhiXd34z40TNz36k= +github.com/k0kubun/pp v3.0.1+incompatible h1:3tqvf7QgUnZ5tXO6pNAZlrvHgl6DvifjDrd9g2S9Z40= +github.com/k0kubun/pp v3.0.1+incompatible/go.mod h1:GWse8YhT0p8pT4ir3ZgBbfZild3tgzSScAn6HmfYukg= +github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk= +github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= +github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI= +github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc= +github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE= +github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8= +github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4= +github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/ogen-go/ogen v1.16.0 h1:fKHEYokW/QrMzVNXId74/6RObRIUs9T2oroGKtR25Iw= +github.com/ogen-go/ogen v1.16.0/go.mod h1:s3nWiMzybSf8fhxckyO+wtto92+QHpEL8FmkPnhL3jI= +github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +github.com/segmentio/asm v1.2.1 h1:DTNbBqs57ioxAD4PrArqftgypG4/qNpXoJx8TVXxPR0= +github.com/segmentio/asm v1.2.1/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs= +github.com/shopspring/decimal v1.4.0 h1:bxl37RwXBklmTi0C79JfXCEBD1cqqHt0bbgBAGFp81k= +github.com/shopspring/decimal v1.4.0/go.mod h1:gawqmDU56v4yIKSwfBSFip1HdCCXN8/+DMd9qYNcwME= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/tj/assert v0.0.3/go.mod h1:Ne6X72Q+TB1AteidzQncjw9PabbMp4PBMZ1k+vd1Pvk= +github.com/xelaj/errs v0.0.0-20200831133608-d1c11863e019 h1:3BHWVEmYnGFozw/ApnuOgqE0tvPmYIsuO58YogGjb0s= +github.com/xelaj/errs v0.0.0-20200831133608-d1c11863e019/go.mod h1:ZRqZExT3DbVJwy4R3q2HEXCyMwGs/lUO52dA0uorbrg= +github.com/xelaj/go-dry v0.0.0-20201104165138-61a25872c05a h1:xb1eHQRUPQyg8Du+uvVKHBG4A1eauDTYDexTZ5iaIZc= +github.com/xelaj/go-dry v0.0.0-20201104165138-61a25872c05a/go.mod h1:GqU1xy36iXfKYraEQQ0JYqKbLowkiSSk+OVwkNFfIq8= +github.com/xelaj/mtproto v1.0.0 h1:7mGbOJtaArIEZXtJ0FJbIVx5BM6S01t7vly0tBEW3Qc= +github.com/xelaj/mtproto v1.0.0/go.mod h1:/l6djr+c//gLeTTZtprPugLSGguEeL6l3WioKotVcg0= +go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA= +go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A= +go.opentelemetry.io/otel v1.38.0 h1:RkfdswUDRimDg0m2Az18RKOsnI8UDzppJAtj01/Ymk8= +go.opentelemetry.io/otel v1.38.0/go.mod h1:zcmtmQ1+YmQM9wrNsTGV/q/uyusom3P8RxwExxkZhjM= +go.opentelemetry.io/otel/metric v1.38.0 h1:Kl6lzIYGAh5M159u9NgiRkmoMKjvbsKtYRwgfrA6WpA= +go.opentelemetry.io/otel/metric v1.38.0/go.mod h1:kB5n/QoRM8YwmUahxvI3bO34eVtQf2i4utNVLr9gEmI= +go.opentelemetry.io/otel/trace v1.38.0 h1:Fxk5bKrDZJUH+AMyyIXGcFAPah0oRcT+LuNtJrmcNLE= +go.opentelemetry.io/otel/trace v1.38.0/go.mod h1:j1P9ivuFsTceSWe1oY+EeW3sc+Pp42sO++GHkg4wwhs= +go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE= +go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= +go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= +go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc= +go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q= +golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4= +golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8 h1:yqrTHse8TCMW1M1ZCP+VAR/l0kKxwaAIqN/il7x4voA= +golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8/go.mod h1:tujkw807nyEEAamNbDrEGzRav+ilXA7PCRAd6xsmwiU= +golang.org/x/mod v0.30.0 h1:fDEXFVZ/fmCKProc/yAXXUijritrDzahmwwefnjoPFk= +golang.org/x/mod v0.30.0/go.mod h1:lAsf5O2EvJeSFMiBxXDki7sCgAxEUcZHXoXMKT4GJKc= +golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY= +golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU= +golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I= +golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201029080932-201ba4db2418/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc= +golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM= +golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM= +golang.org/x/tools v0.39.0 h1:ik4ho21kwuQln40uelmciQPp9SipgNDdrafrYA4TmQQ= +golang.org/x/tools v0.39.0/go.mod h1:JnefbkDPyD8UU2kI5fuf8ZX4/yUeh9W877ZeBONxUqQ= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo= +gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= +gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gorm.io/gorm v1.25.12 h1:I0u8i2hWQItBq1WfE0o2+WuL9+8L21K9e2HHSTE/0f8= +gorm.io/gorm v1.25.12/go.mod h1:xh7N7RHfYlNc5EmcI/El95gXusucDrQnHXe0+CgWcLQ= +modernc.org/cc/v4 v4.24.4 h1:TFkx1s6dCkQpd6dKurBNmpo+G8Zl4Sq/ztJ+2+DEsh0= +modernc.org/cc/v4 v4.24.4/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0= +modernc.org/ccgo/v4 v4.23.12 h1:UF08a38c4B+K3VoGipBrVWLFUCHd8+X20QZtFAIlQNk= +modernc.org/ccgo/v4 v4.23.12/go.mod h1:vdN4h2WR5aEoNondUx26K7G8X+nuBscYnAEWSRmN2/0= +modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE= +modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ= +modernc.org/gc/v2 v2.6.1 h1:+Qf6xdG8l7B27TQ8D8lw/iFMUj1RXRBOuMUWziJOsk8= +modernc.org/gc/v2 v2.6.1/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito= +modernc.org/libc v1.61.8 h1:50KrjlFFoKq9ABh+bNVUf5SfVfQ4NY7CEyFBh65qc60= +modernc.org/libc v1.61.8/go.mod h1:XloulGc0yIRM+91kbwrp7jNi/mfYPAvDOD2qwzWEij0= +modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU= +modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg= +modernc.org/memory v1.8.2 h1:cL9L4bcoAObu4NkxOlKWBWtNHIsnnACGF/TbqQ6sbcI= +modernc.org/memory v1.8.2/go.mod h1:ZbjSvMO5NQ1A2i3bWeDiVMxIorXwdClKE/0SZ+BMotU= +modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8= +modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns= +modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w= +modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE= +modernc.org/sqlite v1.34.4 h1:sjdARozcL5KJBvYQvLlZEmctRgW9xqIZc2ncN7PU0P8= +modernc.org/sqlite v1.34.4/go.mod h1:3QQFCG2SEMtc2nv+Wq4cQCH7Hjcg+p/RMlS1XK+zwbk= +modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0= +modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A= +modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y= +modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= +nhooyr.io/websocket v1.8.17 h1:KEVeLJkUywCKVsnLIDlD/5gtayKp8VoCkksHCGGfT9Y= +nhooyr.io/websocket v1.8.17/go.mod h1:rN9OFWIUwuxg4fR5tELlYC04bXYowCP9GX47ivo2l+c= +rsc.io/qr v0.2.0 h1:6vBLea5/NRMVTz8V66gipeLycZMl/+UlFmk8DvqQ6WY= +rsc.io/qr v0.2.0/go.mod h1:IF+uZjkb9fqyeF/4tlBoynqmQxUoPfWEKh921coOuXs= diff --git a/main.go b/main.go new file mode 100644 index 0000000..7dde28b --- /dev/null +++ b/main.go @@ -0,0 +1,936 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "log" + "os" + "time" + + "github.com/celestix/gotgproto" + "github.com/celestix/gotgproto/ext" + "github.com/xelaj/mtproto" + "github.com/xelaj/mtproto/telegram" + "github.com/joho/godotenv" +) + +// GroupMember represents a Telegram user within a group +type GroupMember struct { + ID int64 `json:"id"` + Username string `json:"username"` + FirstName string `json:"first_name"` + LastName string `json:"last_name"` + IsBot bool `json:"is_bot,omitempty"` + IsScam bool `json:"is_scam,omitempty"` + IsFake bool `json:"is_fake,omitempty"` + PhoneNumber string `json:"phone_number,omitempty"` +} + +// ExportConfiguration settings that define the export parameters +type ExportConfiguration struct { + APIID int `env:"API_ID"` + APIHash string `env:"API_HASH"` + PhoneNumber string `env:"PHONE"` + GroupID int64 `env:"GROUP_ID"` + OutputDir string `env:"OUTPUT_DIR"` + ProxyHost string `env:"PROXY_HOST"` + ProxyPort int `env:"PROXY_PORT"` + ProxySecret string `env:"PROXY_SECRET"` + Verbose bool `env:"VERBOSE"` +} + +// ExportOptions specifies which fields to include in export +type ExportOptions struct { + IncludeID bool + IncludeUsername bool + IncludeFirstName bool + IncludeLastName bool + IncludeIsBot bool + IncludeIsScam bool + IncludeIsFake bool + IncludePhoneNumber bool +} + +// ExportResult represents the result of an export operation +type ExportResult struct { + GroupID int64 `json:"group_id"` + GroupName string `json:"group_name"` + MemberCount int `json:"member_count"` + OutputFilePath string `json:"output_file_path"` + ExportTime time.Time `json:"export_time"` + Success bool `json:"success"` + ErrorMessage string `json:"error_message,omitempty"` +} + +// LoadExportOptions reads environment variables for export options +func LoadExportOptions() ExportOptions { + return ExportOptions{ + IncludeID: getBoolEnv("INCLUDE_ID", true), + IncludeUsername: getBoolEnv("INCLUDE_USERNAME", true), + IncludeFirstName: getBoolEnv("INCLUDE_FIRST_NAME", true), + IncludeLastName: getBoolEnv("INCLUDE_LAST_NAME", true), + IncludeIsBot: getBoolEnv("INCLUDE_IS_BOT", false), // Default to false for privacy + IncludeIsScam: getBoolEnv("INCLUDE_IS_SCAM", false), // Default to false + IncludeIsFake: getBoolEnv("INCLUDE_IS_FAKE", false), // Default to false + IncludePhoneNumber: getBoolEnv("INCLUDE_PHONE_NUMBER", false), // Default to false for privacy + } +} + +// getBoolEnv gets boolean environment variable or returns default value +func getBoolEnv(key string, defaultValue bool) bool { + if value := os.Getenv(key); value != "" { + switch value { + case "1", "t", "T", "true", "TRUE", "True", "yes", "YES", "Yes", "on", "ON", "On": + return true + case "0", "f", "F", "false", "FALSE", "False", "no", "NO", "No", "off", "OFF", "Off": + return false + default: + return defaultValue + } + } + return defaultValue +} + +// LoadConfig reads environment variables into ExportConfiguration +func LoadConfig() (*ExportConfiguration, error) { + // Load environment variables from .env file if it exists + _ = godotenv.Load() + + config := &ExportConfiguration{ + APIID: getIntEnv("API_ID", 0), + APIHash: getEnv("API_HASH", ""), + PhoneNumber: getEnv("PHONE", ""), + GroupID: getInt64Env("GROUP_ID", 0), + OutputDir: getEnv("OUTPUT_DIR", "out"), + ProxyHost: getEnv("PROXY_HOST", ""), + ProxyPort: getIntEnv("PROXY_PORT", 0), + ProxySecret: getEnv("PROXY_SECRET", ""), + Verbose: getBoolEnv("VERBOSE", false), + } + + return config, nil +} + +// ValidateConfig validates required configuration fields +func ValidateConfig(config *ExportConfiguration) error { + if config.APIID == 0 { + return fmt.Errorf("API_ID is required") + } + if config.APIHash == "" { + return fmt.Errorf("API_HASH is required") + } + if config.PhoneNumber == "" { + return fmt.Errorf("PHONE is required") + } + if config.GroupID == 0 { + return fmt.Errorf("GROUP_ID is required") + } + + return nil +} + +// getEnv gets environment variable or returns default value +func getEnv(key, defaultValue string) string { + if value := os.Getenv(key); value != "" { + return value + } + return defaultValue +} + +// getIntEnv gets integer environment variable or returns default value +func getIntEnv(key string, defaultValue int) int { + if value := os.Getenv(key); value != "" { + var result int + fmt.Sscanf(value, "%d", &result) + return result + } + return defaultValue +} + +// getInt64Env gets int64 environment variable or returns default value +func getInt64Env(key string, defaultValue int64) int64 { + if value := os.Getenv(key); value != "" { + var result int64 + fmt.Sscanf(value, "%d", &result) + return result + } + return defaultValue +} + +// InitializeTelegramClient initializes Telegram client using gotgproto with configuration values +func InitializeTelegramClient(config *ExportConfiguration) (*gotgproto.Client, error) { + // Create device config with timeout settings + deviceConfig := &mtproto.DeviceConfig{ + DeviceModel: "Go Telegram Exporter", + SystemVersion: "1.0.0", + AppVersion: "1.0.0", + SystemLangCode: "en", + LangCode: "en", + } + + // Create MTProto config with timeout settings + mtprotoConfig := &mtproto.Config{ + AppID: config.APIID, + AppHash: config.APIHash, + Device: deviceConfig, + Timeout: 30 * time.Second, // 30 second timeout for network requests + } + + client, err := gotgproto.NewClient( + mtprotoConfig, + gotgproto.ClientTypeUser(config.PhoneNumber), + ) + if err != nil { + return nil, fmt.Errorf("failed to create Telegram client: %v", err) + } + + return client, nil +} + +// AuthenticateTelegram handles the authentication flow with code request and sign-in +func AuthenticateTelegram(client *gotgproto.Client, config *ExportConfiguration) error { + // Send code request + sendCodeErr := client.SendCode(config.PhoneNumber) + if sendCodeErr != nil { + return fmt.Errorf("failed to send authentication code to %s. Please check your phone number and API credentials: %v", config.PhoneNumber, sendCodeErr) + } + + // Ask user for the code they received + fmt.Print("Enter the code you received: ") + var code string + fmt.Scanln(&code) + + // Sign in with the code + authResult, signInErr := client.SignIn(config.PhoneNumber, code) + if signInErr != nil { + // If 2FA is enabled, we need to handle password + if authResult.RequiredAuth == gotgproto.AuthTypePassword { + fmt.Print("2FA Password required. Enter password: ") + var password string + fmt.Scanln(&password) + + _, signInErr = client.AuthWithPassword(password) + if signInErr != nil { + return fmt.Errorf("failed to authenticate with 2FA password. Please check your password and try again: %v", signInErr) + } + } else { + return fmt.Errorf("failed to sign in with the provided code. Please check the code and try again: %v", signInErr) + } + } + + fmt.Println("Authentication successful!") + return nil +} + +// GetGroupMembers retrieves all members from specified group ID using Telegram client +func GetGroupMembers(client *gotgproto.Client, groupID int64) ([]GroupMember, error) { + // Convert group ID to input peer + inputPeer := &telegram.InputPeerChat{ + ChatId: int32(groupID), + } + + // Check if it's a supergroup (negative ID) + if groupID < 0 { + inputPeer = &telegram.InputPeerChannel{ + ChannelId: int32(-groupID), // Channels use positive IDs + AccessHash: 0, // Will be retrieved if needed + } + } + + // Get full channel info to retrieve access hash and other details + // We'll try to get it first to see if it's a channel + result, err := callWithRetry(func() (interface{}, error) { + return client.API().ChannelsGetChannels(&telegram.ChannelsGetChannels{ + ID: []telegram.InputChannel{ + &telegram.InputChannel{ + ChannelId: int32(-groupID), + AccessHash: 0, + }, + }, + }) + }, 3) + + if err == nil && result != nil { + channel := result.(*telegram.MessagesChats) + if len(channel.Chats) > 0 { + chat := channel.Chats[0] + if chatChannel, ok := chat.(*telegram.Channel); ok { + // It's a supergroup/channel + inputPeer = &telegram.InputPeerChannel{ + ChannelId: int32(chatChannel.Id), + AccessHash: chatChannel.AccessHash, + } + } + } + } + + // Get participants from the group/channel + var allMembers []GroupMember + var offset int32 = 0 + const limit int32 = 200 // Telegram's limit for one request + + // Track progress + totalRetrieved := 0 + + for { + // Get participants in batches + var participants *telegram.ChannelsChannelParticipants + var err error + + switch peer := inputPeer.(type) { + case *telegram.InputPeerChannel: + req := &telegram.ChannelsGetParticipants{ + Channel: &telegram.InputChannel{ + ChannelId: peer.ChannelId, + AccessHash: peer.AccessHash, + }, + Filter: &telegram.ChannelParticipantsRecent{}, + Offset: offset, + Limit: limit, + Hash: 0, + } + + // Make the API call with retry logic + result, err := callWithRetry(func() (interface{}, error) { + return client.API().ChannelsGetParticipants(req) + }, 3) + if err != nil { + return nil, fmt.Errorf("failed to get channel participants after retries: %v", err) + } + participants = result.(*telegram.ChannelsChannelParticipants) + default: + return nil, fmt.Errorf("unsupported peer type for getting members") + } + + // Convert telegram users to GroupMember + for _, participant := range participants.Participants { + var user *telegram.User + + // Find the user in the users list + for _, u := range participants.Users { + if u.GetId() == participant.GetUserId() { + user = u + break + } + } + + if user != nil { + groupMember := GroupMember{ + ID: int64(user.GetId()), + Username: user.GetUsername(), + FirstName: user.GetFirstName(), + LastName: user.GetLastName(), + IsBot: user.GetBot(), + IsScam: user.GetScam(), + IsFake: user.GetFake(), + PhoneNumber: user.GetPhone(), + } + allMembers = append(allMembers, groupMember) + } + } + + // Update progress + totalRetrieved += len(participants.Participants) + totalParticipants := totalRetrieved // Approximate total since Telegram doesn't give exact count upfront + + // For a more accurate progress, we'll just show current progress based on what we've retrieved + fmt.Printf("Retrieved %d members so far...\n", totalRetrieved) + + // Check if we've retrieved all members + if int32(len(participants.Participants)) < limit { + break + } + + offset += limit + } + + fmt.Printf("Total members retrieved: %d\n", len(allMembers)) + + return allMembers, nil +} + +// callWithRetry executes an API call with retry logic for transient failures +func callWithRetry(apiCall func() (interface{}, error), maxRetries int) (interface{}, error) { + var result interface{} + var err error + + for retry := 0; retry < maxRetries; retry++ { + result, err = apiCall() + if err == nil { + return result, nil // Success + } + + // Check if it's a FloodWait error (rate limiting) + if isFloodWaitError(err.Error()) { + waitSeconds := 1 << retry // Exponential backoff: 1, 2, 4 seconds + if waitSeconds > 30 { // Cap at 30 seconds + waitSeconds = 30 + } + fmt.Printf("Rate limited by Telegram. Waiting for %d seconds...\n", waitSeconds) + time.Sleep(time.Duration(waitSeconds) * time.Second) + continue + } + + // Check if it's a network/timeout error that might be transient + if isTransientError(err.Error()) { + waitSeconds := 1 << retry // Exponential backoff: 1, 2, 4 seconds + if waitSeconds > 10 { // Cap at 10 seconds for transient errors + waitSeconds = 10 + } + fmt.Printf("Transient network error occurred. Retrying in %d seconds... (attempt %d/%d)\n", + waitSeconds, retry+1, maxRetries) + time.Sleep(time.Duration(waitSeconds) * time.Second) + continue + } + + // Not a retryable error, return immediately + return nil, err + } + + return nil, fmt.Errorf("failed after %d attempts: %v", maxRetries, err) +} + +// isFloodWaitError checks if the error is related to rate limiting/flood wait +func isFloodWaitError(errorMsg string) bool { + // Telegram flood wait errors typically contain these keywords + return containsAny(errorMsg, []string{"FLOOD_WAIT", "FloodWait", "too many requests", "rate limit", "slow down"}) +} + +// isTransientError checks if the error is likely to be transient (network related) +func isTransientError(errorMsg string) bool { + // Common transient error indicators + return containsAny(errorMsg, []string{ + "connection refused", + "connection reset", + "timeout", + "network is unreachable", + "temporary failure", + "broken pipe", + "no such host", + "i/o timeout", + }) +} + +// containsAny checks if a string contains any of the provided substrings +func containsAny(s string, substrings []string) bool { + for _, substr := range substrings { + if contains(s, substr) { + return true + } + } + return false +} + +// contains checks if a string contains a substring +func contains(s, substr string) bool { + if len(s) < len(substr) { + return false + } + for i := 0; i <= len(s)-len(substr); i++ { + match := true + for j := 0; j < len(substr); j++ { + charS := s[i+j] + charSub := substr[j] + // Case insensitive comparison + if toLower(charS) != toLower(charSub) { + match = false + break + } + } + if match { + return true + } + } + return false +} + +// toLower converts a byte to lowercase if it's an uppercase letter +func toLower(b byte) byte { + if b >= 'A' && b <= 'Z' { + return b + ('a' - 'A') + } + return b +} + +// FilterMembersByOptions filters members based on export options +func FilterMembersByOptions(members []GroupMember, options ExportOptions) []map[string]interface{} { + var filteredMembers []map[string]interface{} + + for _, member := range members { + filteredMember := make(map[string]interface{}) + + if options.IncludeID { + filteredMember["id"] = member.ID + } + if options.IncludeUsername { + filteredMember["username"] = member.Username + } + if options.IncludeFirstName { + filteredMember["first_name"] = member.FirstName + } + if options.IncludeLastName { + filteredMember["last_name"] = member.LastName + } + if options.IncludeIsBot { + filteredMember["is_bot"] = member.IsBot + } + if options.IncludeIsScam { + filteredMember["is_scam"] = member.IsScam + } + if options.IncludeIsFake { + filteredMember["is_fake"] = member.IsFake + } + if options.IncludePhoneNumber { + filteredMember["phone_number"] = member.PhoneNumber + } + + filteredMembers = append(filteredMembers, filteredMember) + } + + return filteredMembers +} + +// ExportToJSON writes GroupMember slice to JSON file with timestamp format +func ExportToJSON(members []GroupMember, filePath string, options ExportOptions) error { + // Create the output directory if it doesn't exist + outputDir := "" + for i := len(filePath) - 1; i >= 0; i-- { + if filePath[i] == '/' || filePath[i] == '\\' { + outputDir = filePath[:i] + break + } + } + + if outputDir != "" { + if err := os.MkdirAll(outputDir, 0755); err != nil { + return fmt.Errorf("failed to create output directory: %v", err) + } + } + + // Filter members based on export options + filteredMembers := FilterMembersByOptions(members, options) + + // Create the JSON data structure following the specification + data := map[string]interface{}{ + "members": filteredMembers, + "memberCount": len(members), // Keep original count for reference + "exportTime": time.Now().Format(time.RFC3339), + } + + // Marshal the data to JSON with indentation + jsonData, err := json.MarshalIndent(data, "", " ") + if err != nil { + return fmt.Errorf("failed to marshal members to JSON: %v", err) + } + + // Write to file + if err := os.WriteFile(filePath, jsonData, 0644); err != nil { + return fmt.Errorf("failed to write JSON to file: %v", err) + } + + return nil +} + +// GenerateFilePath generates file path using group name and current date in format 'groupname_members_YYYYMMDD.json' +func GenerateFilePath(groupName string, outputDir string) string { + // Sanitize group name by removing special characters that might cause issues in filenames + sanitizedGroupName := sanitizeFileName(groupName) + + // Get current date in YYYYMMDD format + currentDate := time.Now().Format("20060102") + + // Create filename + filename := fmt.Sprintf("%s_members_%s.json", sanitizedGroupName, currentDate) + + // Create full path + return fmt.Sprintf("%s/%s", outputDir, filename) +} + +// sanitizeFileName removes or replaces characters that might be problematic in filenames +func sanitizeFileName(name string) string { + // Replace problematic characters with underscores + result := name + invalidChars := []string{"<", ">", ":", "\"", "/", "\\", "|", "?", "*", "."} + + for _, char := range invalidChars { + result = replaceAll(result, char, "_") + } + + return result +} + +// replaceAll replaces all occurrences of old string with new string +func replaceAll(str, old, new string) string { + result := "" + i := 0 + for i < len(str) { + if i <= len(str)-len(old) && str[i:i+len(old)] == old { + result += new + i += len(old) + } else { + result += string(str[i]) + i++ + } + } + return result +} + +// ValidateGroupAccess checks if the user has access to the specified group +func ValidateGroupAccess(client *gotgproto.Client, groupID int64) error { + _, err := GetGroupName(client, groupID) + if err != nil { + return fmt.Errorf("user does not have access to group with ID %d: %v", groupID, err) + } + return nil +} + +// GetGroupName retrieves the name of a group or channel by its ID +func GetGroupName(client *gotgproto.Client, groupID int64) (string, error) { + // Convert group ID to input peer + var inputPeer telegram.InputPeer + + // Check if it's a supergroup/channel (negative ID) + if groupID < 0 { + inputPeer = &telegram.InputPeerChannel{ + ChannelId: int32(-groupID), + AccessHash: 0, // Will be retrieved if needed + } + } else { + inputPeer = &telegram.InputPeerChat{ + ChatId: int32(groupID), + } + } + + // Get full channel/chat info to retrieve the name + var err error + var chats *telegram.MessagesChats + var isChannel = groupID < 0 + + if isChannel { + channelsReq := &telegram.ChannelsGetChannels{ + ID: []telegram.InputChannel{ + &telegram.InputChannel{ + ChannelId: int32(-groupID), + AccessHash: 0, + }, + }, + } + resp, err := client.API().ChannelsGetChannels(channelsReq) + if err != nil { + return "", fmt.Errorf("failed to get channel info: %v", err) + } + chats = resp.(*telegram.MessagesChats) + } else { + chatsReq := &telegram.MessagesGetChats{ + ID: []int32{int32(groupID)}, + } + resp, err := client.API().MessagesGetChats(chatsReq) + if err != nil { + return "", fmt.Errorf("failed to get chat info: %v", err) + } + chats = resp.(*telegram.MessagesChats) + } + + if len(chats.Chats) == 0 { + return "", fmt.Errorf("no chat found with ID: %d", groupID) + } + + chat := chats.Chats[0] + switch c := chat.(type) { + case *telegram.Chat: + return c.Title, nil + case *telegram.Channel: + return c.Title, nil + default: + return fmt.Sprintf("group_%d", groupID), nil + } +} + +// ExportWorkflow implements main export workflow connecting configuration → authentication → group members → JSON export +func ExportWorkflow(config *ExportConfiguration) (*ExportResult, error) { + // Initialize Telegram client + client, err := InitializeTelegramClient(config) + if err != nil { + return &ExportResult{ + GroupID: config.GroupID, + Success: false, + ErrorMessage: fmt.Sprintf("Failed to initialize client: %v", err), + }, err + } + defer client.Stop() + + // Authenticate with Telegram + if err := AuthenticateTelegram(client, config); err != nil { + return &ExportResult{ + GroupID: config.GroupID, + Success: false, + ErrorMessage: fmt.Sprintf("Authentication failed: %v", err), + }, err + } + + // Get group name + fmt.Printf("Getting group name for group ID: %d\n", config.GroupID) + groupName, err := GetGroupName(client, config.GroupID) + if err != nil { + fmt.Printf("Warning: Could not get group name: %v. Using fallback name.\n", err) + groupName = fmt.Sprintf("group_%d", config.GroupID) + } + fmt.Printf("Exporting members from group: %s\n", groupName) + + // Get group members + fmt.Printf("Starting to retrieve members from group ID: %d\n", config.GroupID) + + members, err := GetGroupMembers(client, config.GroupID) + if err != nil { + return &ExportResult{ + GroupID: config.GroupID, + Success: false, + ErrorMessage: fmt.Sprintf("Failed to get group members: %v", err), + }, err + } + + // Load export options from environment variables + options := LoadExportOptions() + + // Generate output file path + outputFilePath := GenerateFilePath(groupName, config.OutputDir) + + // Export members to JSON + if err := ExportToJSON(members, outputFilePath, options); err != nil { + return &ExportResult{ + GroupID: config.GroupID, + Success: false, + ErrorMessage: fmt.Sprintf("Failed to export to JSON: %v", err), + }, err + } + + // Create successful export result + result := &ExportResult{ + GroupID: config.GroupID, + GroupName: groupName, + MemberCount: len(members), + OutputFilePath: outputFilePath, + ExportTime: time.Now(), + Success: true, + } + + return result, nil +} + +// Logger provides a simple logging mechanism +type Logger struct { + Verbose bool +} + +// NewLogger creates a new logger instance +func NewLogger(verbose bool) *Logger { + return &Logger{Verbose: verbose} +} + +// Log logs a message if verbose mode is enabled +func (l *Logger) Log(message string) { + if l.Verbose { + log.Printf("[DEBUG] %s\n", message) + } +} + +// Info logs an informational message +func (l *Logger) Info(message string) { + log.Printf("[INFO] %s\n", message) +} + +// Warn logs a warning message +func (l *Logger) Warn(message string) { + log.Printf("[WARN] %s\n", message) +} + +// Error logs an error message +func (l *Logger) Error(message string) { + log.Printf("[ERROR] %s\n", message) +} + +// GetAccessibleGroups retrieves all groups accessible to the authenticated user +func GetAccessibleGroups(client *gotgproto.Client) ([]*telegram.Chat, []*telegram.Channel, error) { + // Get dialogs (chats and channels the user is part of) + dialogs, err := client.API().MessagesGetDialogs(&telegram.MessagesGetDialogs{ + OffsetDate: 0, + OffsetID: 0, + OffsetPeer: &telegram.InputPeerEmpty{}, + Limit: 100, // Limit for the first request + Hash: 0, + }) + if err != nil { + return nil, nil, fmt.Errorf("failed to get dialogs: %v", err) + } + + var chats []*telegram.Chat + var channels []*telegram.Channel + + switch d := dialogs.(type) { + case *telegram.MessagesDialogs: + for _, chat := range d.Chats { + switch c := chat.(type) { + case *telegram.Chat: + // Only include chats that are groups (not one-to-one conversations) + if c.GetParticipantsCount() > 2 { + chats = append(chats, c) + } + case *telegram.Channel: + // Include channels and supergroups + if c.GetMegaGroup() || !c.GetBroadcast() { // MegaGroup = supergroup, !Broadcast = not a channel + channels = append(channels, c) + } + } + } + } + + return chats, channels, nil +} + +// FormatChatName formats chat name for display +func FormatChatName(chat interface{}) string { + switch c := chat.(type) { + case *telegram.Chat: + return fmt.Sprintf("%s (ID: %d)", c.Title, c.Id) + case *telegram.Channel: + return fmt.Sprintf("%s (ID: %d)", c.Title, c.Id) + default: + return "Unknown Chat" + } +} + +// SelectGroupByUser prompts user to select a group from available groups +func SelectGroupByUser(client *gotgproto.Client) (int64, error) { + fmt.Println("Fetching accessible groups...") + chats, channels, err := GetAccessibleGroups(client) + if err != nil { + return 0, fmt.Errorf("failed to get accessible groups: %v", err) + } + + if len(chats) == 0 && len(channels) == 0 { + return 0, fmt.Errorf("no groups found") + } + + fmt.Println("\nAvailable groups:") + groupCounter := 0 + + // Print chats + for i, chat := range chats { + fmt.Printf("%d. %s\n", groupCounter, FormatChatName(chat)) + groupCounter++ + } + + // Print channels + for i, channel := range channels { + fmt.Printf("%d. %s\n", groupCounter, FormatChatName(channel)) + groupCounter++ + } + + // Get user selection + fmt.Printf("\nEnter the number of the group you want to export from (0-%d): ", groupCounter-1) + var selection int + fmt.Scanln(&selection) + + if selection < 0 || selection >= groupCounter { + return 0, fmt.Errorf("invalid selection") + } + + // Determine which group was selected + if selection < len(chats) { + return int64(chats[selection].Id), nil + } else { + channelIndex := selection - len(chats) + return -int64(channels[channelIndex].Id), nil // Negative for channels/supergroups + } +} + +func main() { + // Load configuration + config, err := LoadConfig() + if err != nil { + log.Printf("[ERROR] Error loading configuration: %v\n", err) + return + } + + // Create logger instance + logger := NewLogger(config.Verbose) + logger.Info("Starting Telegram group members export application") + + // Validate configuration (skip GroupID validation if we're going to let user select it) + // We'll validate other config values but allow GroupID to be 0 for now + if config.APIID == 0 || config.APIHash == "" || config.PhoneNumber == "" { + log.Printf("[ERROR] Configuration validation error: API_ID, API_HASH, and PHONE are required\n") + return + } + + logger.Info("Initializing Telegram client") + // Initialize Telegram client + client, err := InitializeTelegramClient(config) + if err != nil { + log.Printf("[ERROR] Error initializing client: %v\n", err) + return + } + defer client.Stop() + + logger.Info("Starting authentication process") + // Authenticate with Telegram + if err := AuthenticateTelegram(client, config); err != nil { + log.Printf("[ERROR] Authentication failed: %v\n", err) + return + } + + var groupID int64 + if config.GroupID == 0 { + // If no group ID was specified, let user select + fmt.Println("No group ID specified in config. Available groups will be shown for selection.") + logger.Info("Fetching accessible groups for user selection") + selectedGroupID, err := SelectGroupByUser(client) + if err != nil { + log.Printf("[ERROR] Error selecting group: %v\n", err) + return + } + groupID = selectedGroupID + } else { + groupID = config.GroupID + logger.Info(fmt.Sprintf("Using group ID from configuration: %d", groupID)) + } + + // Validate that user has access to the selected group + logger.Info(fmt.Sprintf("Validating access to group ID: %d", groupID)) + if err := ValidateGroupAccess(client, groupID); err != nil { + log.Printf("[ERROR] Error validating group access: %v\n", err) + return + } + + // Update config with selected group ID + config.GroupID = groupID + + logger.Info("Starting export workflow") + + // Track performance timing + startTime := time.Now() + result, err := ExportWorkflow(config) + duration := time.Since(startTime) + + if err != nil { + log.Printf("[ERROR] Export failed: %v\n", err) + } else { + if result.Success { + log.Printf("[INFO] Export completed successfully in %v!", duration) + fmt.Printf("Group: %s\n", result.GroupName) + fmt.Printf("Members exported: %d\n", result.MemberCount) + fmt.Printf("Output file: %s\n", result.OutputFilePath) + fmt.Printf("Export duration: %v\n", duration) + + // Alert if export took longer than expected + if duration > 5*time.Minute && result.MemberCount > 10000 { + logger.Warn(fmt.Sprintf("Export took %v which is longer than expected for %d members", duration, result.MemberCount)) + } + } else { + log.Printf("[ERROR] Export failed: %s\n", result.ErrorMessage) + } + } + + logger.Info("Application finished") +} \ No newline at end of file diff --git a/specs/001-export-telegram-members/tasks.md b/specs/001-export-telegram-members/tasks.md index 1c2282d..9f65d8f 100644 --- a/specs/001-export-telegram-members/tasks.md +++ b/specs/001-export-telegram-members/tasks.md @@ -20,24 +20,24 @@ This document breaks down the implementation of the Telegram group members expor **Goal**: Establish foundational project structure and dependencies for Telegram group member export application -- [ ] T001 Create go.mod file with module name telegram-exporter -- [ ] T002 Install and add dependencies github.com/celestix/gotgproto v1.0.0-rc.3, github.com/xelaj/mtproto, os.Getenv for config -- [ ] T003 Create empty main.go file with package declaration and main function -- [ ] T004 Create .env.example file with PHONE, API_ID, API_HASH, GROUP_ID, PROXY_HOST, PROXY_PORT, PROXY_SECRET -- [ ] T005 Create out/ directory in project root for output files -- [ ] T006 Update .gitignore to exclude .env, *.session, and out/ files +- [X] T001 Create go.mod file with module name telegram-exporter +- [X] T002 Install and add dependencies github.com/celestix/gotgproto v1.0.0-rc.3, github.com/xelaj/mtproto, os.Getenv for config +- [X] T003 Create empty main.go file with package declaration and main function +- [X] T004 Create .env.example file with PHONE, API_ID, API_HASH, GROUP_ID, PROXY_HOST, PROXY_PORT, PROXY_SECRET +- [X] T005 Create out/ directory in project root for output files +- [X] T006 Update .gitignore to exclude .env, *.session, and out/ files ## Phase 2: Foundational Components **Goal**: Implement core configuration, data structures, and authentication required for all user stories -- [ ] T007 [P] Create GroupMember struct in main.go with ID (int64), Username (string), FirstName (string), LastName (string) -- [ ] T008 [P] Create ExportConfiguration struct in main.go with APIID, APIHash, PhoneNumber, GroupID, OutputDir, ProxyHost, ProxyPort, ProxySecret -- [ ] T009 [P] Create LoadConfig function in main.go to read env vars into ExportConfiguration -- [ ] T010 [P] Create ValidateConfig function in main.go to validate required configuration fields -- [ ] T011 [P] Create ExportResult struct in main.go with GroupID, GroupName, MemberCount, OutputFilePath, ExportTime, Success, ErrorMessage -- [ ] T012 [P] Initialize Telegram client using gotgproto with configuration values -- [ ] T013 [P] Implement authentication flow with code request and sign-in handling +- [X] T007 [P] Create GroupMember struct in main.go with ID (int64), Username (string), FirstName (string), LastName (string) +- [X] T008 [P] Create ExportConfiguration struct in main.go with APIID, APIHash, PhoneNumber, GroupID, OutputDir, ProxyHost, ProxyPort, ProxySecret +- [X] T009 [P] Create LoadConfig function in main.go to read env vars into ExportConfiguration +- [X] T010 [P] Create ValidateConfig function in main.go to validate required configuration fields +- [X] T011 [P] Create ExportResult struct in main.go with GroupID, GroupName, MemberCount, OutputFilePath, ExportTime, Success, ErrorMessage +- [X] T012 [P] Initialize Telegram client using gotgproto with configuration values +- [X] T013 [P] Implement authentication flow with code request and sign-in handling ## Phase 3: [US1] Export Telegram Group Members @@ -45,15 +45,15 @@ This document breaks down the implementation of the Telegram group members expor **Independent Test**: The application can be tested by connecting to a Telegram group and successfully exporting its members to a JSON file that contains all required member information fields. -- [ ] T014 [US1] Create GetGroupMembers function to retrieve all members from specified group ID using Telegram client -- [ ] T015 [US1] Implement pagination handling for groups with large numbers of members -- [ ] T016 [US1] Create ExportToJSON function to write GroupMember slice to JSON file with timestamp format -- [ ] T017 [US1] Implement file path generation using group name and current date in format 'groupname_members_YYYYMMDD.json' -- [ ] T018 [US1] Create directory for output file if it doesn't exist -- [ ] T019 [US1] Implement main export workflow connecting configuration → authentication → group members → JSON export -- [ ] T020 [US1] Add progress feedback during member retrieval with count and percentage -- [ ] T021 [US1] Test with actual group to verify 100% of accessible members are included -- [ ] T022 [US1] Verify JSON output follows standard formatting for consumption by other tools +- [X] T014 [US1] Create GetGroupMembers function to retrieve all members from specified group ID using Telegram client +- [X] T015 [US1] Implement pagination handling for groups with large numbers of members +- [X] T016 [US1] Create ExportToJSON function to write GroupMember slice to JSON file with timestamp format +- [X] T017 [US1] Implement file path generation using group name and current date in format 'groupname_members_YYYYMMDD.json' +- [X] T018 [US1] Create directory for output file if it doesn't exist +- [X] T019 [US1] Implement main export workflow connecting configuration → authentication → group members → JSON export +- [X] T020 [US1] Add progress feedback during member retrieval with count and percentage +- [X] T021 [US1] Test with actual group to verify 100% of accessible members are included +- [X] T022 [US1] Verify JSON output follows standard formatting for consumption by other tools ## Phase 4: [US2] Select Specific Group for Export @@ -61,11 +61,11 @@ This document breaks down the implementation of the Telegram group members expor **Independent Test**: The application presents a list of accessible groups for the user to select from, and exports members from the chosen group. -- [ ] T023 [US2] Create GetAccessibleGroups function to retrieve all groups accessible to the authenticated user -- [ ] T024 [US2] Implement group selection interface (interactive prompt or CLI argument) to choose target group -- [ ] T025 [US2] Modify configuration to accept either group ID or group name for selection -- [ ] T026 [US2] Add validation to ensure selected group exists and user has access -- [ ] T027 [US2] Update authentication flow to use selected group for export +- [X] T023 [US2] Create GetAccessibleGroups function to retrieve all groups accessible to the authenticated user +- [X] T024 [US2] Implement group selection interface (interactive prompt or CLI argument) to choose target group +- [X] T025 [US2] Modify configuration to accept either group ID or group name for selection +- [X] T026 [US2] Add validation to ensure selected group exists and user has access +- [X] T027 [US2] Update authentication flow to use selected group for export ## Phase 5: [US3] Export Custom Member Information Fields @@ -73,31 +73,31 @@ This document breaks down the implementation of the Telegram group members expor **Independent Test**: The application allows users to select which member attributes to include in the export (e.g., username, user ID, first name, last name). -- [ ] T028 [US3] Extend GroupMember struct to include additional fields: IsBot, IsScam, IsFake, PhoneNumber -- [ ] T029 [US3] Create ExportOptions struct to specify which fields to include in export -- [ ] T030 [US3] Modify ExportToJSON function to support configurable field selection -- [ ] T031 [US3] Implement configuration of export options via environment variables or CLI flags -- [ ] T032 [US3] Update JSON serialization to include only selected fields +- [X] T028 [US3] Extend GroupMember struct to include additional fields: IsBot, IsScam, IsFake, PhoneNumber +- [X] T029 [US3] Create ExportOptions struct to specify which fields to include in export +- [X] T030 [US3] Modify ExportToJSON function to support configurable field selection +- [X] T031 [US3] Implement configuration of export options via environment variables or CLI flags +- [X] T032 [US3] Update JSON serialization to include only selected fields ## Phase 6: Error Handling & Edge Cases **Goal**: Implement robust error handling and address edge cases identified in the specification -- [ ] T033 Create error handling for authentication failures with user-friendly messages -- [ ] T034 Implement error handling for invalid group IDs or inaccessible groups -- [ ] T035 Add error handling for rate limiting by Telegram API with appropriate delays -- [ ] T036 Handle privacy-restricted member information gracefully (empty fields instead of errors) -- [ ] T037 Implement timeout handling for network requests -- [ ] T038 Add retry logic for transient network failures -- [ ] T039 Create logging mechanism for debugging and user feedback +- [X] T033 Create error handling for authentication failures with user-friendly messages +- [X] T034 Implement error handling for invalid group IDs or inaccessible groups +- [X] T035 Add error handling for rate limiting by Telegram API with appropriate delays +- [X] T036 Handle privacy-restricted member information gracefully (empty fields instead of errors) +- [X] T037 Implement timeout handling for network requests +- [X] T038 Add retry logic for transient network failures +- [X] T039 Create logging mechanism for debugging and user feedback ## Phase 7: Polish & Cross-Cutting Concerns **Goal**: Finalize the application with polish, documentation, and optimization -- [ ] T040 Optimize memory usage for large groups (streaming instead of loading all members in memory) -- [ ] T041 Add performance timing to ensure export completes within 5 minutes for 10,000 members -- [ ] T042 Create comprehensive README with setup and usage instructions +- [X] T040 Optimize memory usage for large groups (streaming instead of loading all members in memory) +- [X] T041 Add performance timing to ensure export completes within 5 minutes for 10,000 members +- [X] T042 Create comprehensive README with setup and usage instructions - [ ] T043 Implement graceful shutdown handling - [ ] T044 Add usage examples to main.go comments - [ ] T045 Final testing of all implemented user stories