Best Practices for Secure Bot File Handling

Why Performance and Cost Define Secure Bot File Handling

Telegram Bot API 8.0 (Nov 2025) lets a single message carry 4 GB, but every byte still rides on your own bandwidth quota and CPU. Treating "secure" as a pure crypto problem ignores the first place most projects bleed: unbounded downloads, duplicate caching and un-scanned payloads that quietly inflate VPS bills. This guide anchors every recommendation to two measurable baselines—median response time ≤ 300 ms and egress cost ≤ 0.05 USD per 1 000 unique files—so you can prove security without sabotaging economics.

Version Differences That Change the Rules

Bot API 7.x → 8.0: what actually moved

The jump from 7.9 to 8.0 introduced streaming download (getFile?offset=&limit=) and raised the hard size ceiling from 2 GB to 4 GB. If your code still calls download_file() without offset, the library now pulls the entire payload into RAM before writing disk—an easy denial-of-service vector on 1 vCPU boxes. Update your wrapper first; python-telegram-bot ≥ 21.0 and node-telegram-bot-api ≥ 0.70 expose the new parameter.

Desktop vs mobile path to observe the limit

There is no admin toggle inside Telegram clients that caps what users can send your bot; enforcement must live server-side. Desktop (macOS, Windows, Linux) shows a 4 GB progress bar, while Android 10.9.2 silently compresses videos > 2 GB unless the user taps “Send uncompressed.” Your back-end should therefore expect both raw and re-encoded streams for the same MIME type.

Migration Steps: From Naïve Download to Zero-Trust Stream

1. Inventory current traffic. Enable --stats 60 on nginx or Caddy in front of your bot; log $body_bytes_sent and $request_time for 24 h. Identify the 95th-percentile file size; if it is above 200 MB you are a candidate for streaming.
2. Flip to chunked download. In Python:
   async with aiohttp.ClientSession() as s: headers={"Range":"bytes=0-16383"} while offset < file_size: async with s.get(url,headers=headers) as r: chunk=await r.read() f.write(chunk) offset+=len(chunk) headers["Range"]=f"bytes={offset}-{offset+16383}"
   This keeps resident memory under 32 kB regardless of file size.
3. Scan before you store. Pipe the first 16 MB through clamscan --stdout -; abort on non-zero exit. Because ClamAV uses ≤ 120 MB RAM during the scan, you stay within the 300 ms target for 95 % of files ≤ 20 MB.
4. Write-then-verify hash. After final chunk, compute SHA-256 incrementally and compare with file.file_unique_id provided by Telegram. On mismatch, delete local copy and answer with retry keyboard; this prevents poisoned edge-cache entries.

Roll-out order: staging bot → 5 % canary → 100 % production. Expect a 15 % drop in peak RAM and ≈ 0.02 USD/GB saved on egress because duplicates are rejected before full download.

Compatibility Matrix: Wrapper, Language, OS

If your language is stuck in the red zone, proxy downloads through a micro-service written in a supported stack; the latency cost (≈ 20 ms localhost) is cheaper than refactoring the entire bot.

Risk Control: Limits, Quotas, Kill-Switch

Per-user rate limiter

Use a sliding-window counter in Redis: key=user_id:hour, value=total bytes, TTL=3 600 s. Cap at 500 MB/hour for normal users, 2 GB/hour for premium. When the ceiling is hit, reply with a non-retry inline button; this prevents Telegram from re-sending the same file endlessly.

Global file-type blacklist

Reject mime.startswith("application/x-dosexec") or .scr extensions at the edge—before any disk write. Empirical observation: 0.4 % of spam sent to public bots carries trojanized Windows binaries; early abort saves ≈ 0.01 USD per blocked payload on scan and storage.

Caching Without Exploding Your Disk

Telegram already hosts each file for at least 60 minutes on its CDN, so duplicating every download to local SSD is usually waste. A workable pattern:

• Keep a LRU cache of ≤ 5 GB on fast disk keyed by file_unique_id.
• Set TTL=30 min; after expiry, serve a 302 redirect back to Telegram’s file_path URL—your bandwidth, zero egress.
• Log hit ratio; if it drops below 15 %, shrink cache to 2 GB—empirical cut-off where flash wear cost outweighs repeat-download savings.

For containers, mount the cache volume as tmpfs on hosts with ≥ 16 GB RAM; this removes I/O variance and keeps P99 latency under 200 ms for cache hits.

Logging and Audit: Store Hashes, Not Content

Regulatory frameworks (GDPR, CPRA) rarely let you keep user files indefinitely. Instead, persist:

This tuple is < 300 bytes yet proves file provenance if abuse is reported later. Rotate logs daily and compress with zstd; 90 days retention fits most jurisdiction requirements while keeping storage cost < 0.50 USD/month per million records.

Third-Party Scanners and Permission Minimisation

If you pipe files to a cloud sandbox (e.g., VirusTotal, MetaDefender), create a dedicated sub-bot with only file read rights—no group messages, no delete scope. Use a one-way queue (SQS, NATS) so the scanner’s compromise cannot callback into your main bot token. This separation reduced blast radius in a 2024 case study where a malicious sample exploited an outdated sandbox API to harvest tokens.

Troubleshooting High Memory or 502 Gateway

Checklist: Go-Live Criteria for File-Handling Bots

Case Study #1: 300 k-User Public Sticker Bot

Context: A community sticker bot running on a 2 vCPU/4 GB VPS saw nightly RAM exhaustion after API 8.0 launched because users began forwarding 3 GB video stickers.

Intervention: Migrated from download_file() to 16 KB chunked stream, added LRU tmpfs cache (3 GB), and introduced a 1 GB/hour per-user Redis cap.

Result: Median response dropped from 1 100 ms to 260 ms, 95th-percentile RAM stabilized at 1.8 GB, and monthly egress cost fell from 18 USD to 4 USD despite traffic doubling.

Post-mortem: The biggest win was rejecting duplicates before the 30 % mark of the download; earlier sampling at 8 MB instead of 16 MB would have saved another 0.8 USD but at the cost of two additional false-negative scans—an acceptable trade-off for this use-case.

Case Study #2: Enterprise 5 k-User Internal Scanner

Context: A Fortune-500 security team needed an on-prem bot to accept zips ≤ 4 GB from employees, scan with ClamAV + YARA, and push results to Splunk.

Intervention: Deployed a Go-tgbot micro-service using io.CopyN with 32 KB chunks, wrote straight to an ephemeral 10 GB NVMe, and streamed the first 25 MB to ClamAV while the remainder uploaded to S3-compatible storage for 7-day quarantine.

Result: 100 % of files ≤ 2 GB processed under 400 ms; P99 for 4 GB files landed at 1.9 s. Hash-only audit logs kept disk growth under 600 MB for 90 days. The pipeline passed internal pen-testing with zero critical findings and is now templated for other regions.

Post-mortem: Initial YARA rules were too greedy and doubled CPU; pruning to 120 high-confidence rules brought utilization back to 35 % without losing detection rate.

Runbook: Monitor, Diagnose, Roll Back

1. Alerting Signals

Watch for sustained > 80 % RAM, p99 download latency > 1 s, or ClamAV queue depth > 50. Any two within 5 minutes triggers a PAGE.

2. Locating the Fault

1. Check journalctl -u your-bot | grep -E "oom|killed" for OOM kills.
2. If absent, pull last 10 min of nginx $request_time and correlate with user_id that exceeds byte quota.
3. Should a single user dominate traffic, temporarily shadow-ban via Redis flag banned:{user_id} and observe relief.

3. Controlled Rollback

Your deployment tool (e.g., Helm) should keep the previous ReplicaSet live. Run:

This reverts code and clears any temporary bans within 60 s.

4. Drill Calendar

Run a chaos test every quarter: inject a 3 GB random blob through a canary bot, observe auto-throttle, verify kill-switch, and record RTO. Target RTO ≤ 5 min has been met in all 2025 drills to date.

FAQ – Quick Answers with Evidence

Q: Does Telegram guarantee the file_unique_id is really unique?

A: Yes, within the same bot scope and time window. Source: Bot API 8.0 docs, field description. Collisions have not been reported in production telemetry covering 10^9 files.

Q: Is Range request always respected?

A: The CDN honours RFC 7233; if denied you receive 200. Empirically this occurs < 0.01 %, usually during regional fail-over—fallback code must handle full-file read.

Q: Can I disable file uploads entirely for some users?

A: No native switch; implement an allow-list in your update handler and reply with BadRequest to drop the message pre-download.

Q: Why 16 KB chunk size?

A: Benchmarks on e2-micro show 16 KB balances syscall overhead and memory pressure; 8 KB adds 7 % CPU, 64 KB adds 12 % RAM—16 KB is the sweet spot.

Q: Does tmpfs survive container restart?

A: No; this is intentional—cache becomes cold and re-hydrates from Telegram CDN, avoiding stale data after code deploy.

Q: ClamAV found a virus—am I legally liable?

A: Jurisdiction-dependent. Retain the hash log and your immediate deletion record; consult counsel, but possession time is near-zero which limits exposure.

Q: Is Python asyncio mandatory?

A: Not strictly, but synchronous code will block the event loop and inflate concurrency. Any library listed green in the matrix already wraps asyncio or Worker Threads.

Q: Can Telegram revoke a file_path URL?

A: Yes, after ~60 min or earlier if abuse is detected. Never hard-code URLs for long-term hot-linking.

Q: What happens if I exceed the 50 msg/sec global limit?

A: API returns 429; retries must use exponential back-off. Large file notifications can push you over—batch alerts or use a separate channel.

Q: Do I need to scan images?

A: Stuxnet-style exploits via malformed EXIF have been demoed but remain rare. Cost-benefit says scan if you serve Windows clients; otherwise magic-byte sanity is often enough.

Terminology Reference

file_unique_id

Opaque string identifying a file within a bot; persists even if file_path changes.

file_path

Temporary CDN URL returned by getFile; valid ≈ 60 min.

Range header

HTTP request header enabling partial content fetch; cornerstone of streaming.

Magic bytes

First 4–8 bytes of a file signaling its format (e.g., FF D8 FF for JPEG).

Sliding-window rate limit

Algorithm counting events in the last N seconds, allowing smooth throttling.

LRU cache

Least-Recently-Used eviction policy keeping hottest items in bounded memory.

tmpfs

In-memory filesystem backed by RAM+swap; I/O latency near zero.

Egress cost

Cloud provider charge for outbound traffic; dominant cost in file-heavy bots.

SHA-256

Cryptographic hash producing 32-byte digest; used for integrity verification.

ClamAV StreamMaxLength

Config parameter capping the number of bytes scanned in INSTREAM mode.

archive bomb

Malicious archive designed to inflate exponentially, exhausting scanner memory.

Proof-of-clean hash

Hypothetical future checksum signifying client-side scan approval (not yet shipped).

Blast radius

Extent of damage caused by a security event; minimized via scope segregation.

RTO

Recovery Time Objective; maximum acceptable downtime after failure.

Canary deployment

Release technique exposing a small subset of traffic to new code for early validation.

Risk & Boundary Summary

Streaming, scanning and caching mitigate most abuse, yet edge cases remain. Files < 8 MB bypass chunk efficiency; archival formats can still archive-bomb ClamAV below the 25 MB limit; and zero-day malware may evade signature DB for hours. If your threat model includes nation-state payloads, augment with a sandboxed VM or commercial EDR pipeline. Finally, Telegram may tighten the 50 msg/sec or 4 GB ceiling without notice—design your policy engine to accept dynamic limits fetched from a config map so future tightening does not equal downtime.

Future Outlook: What 2026 May Bring

Pre-release notes circulated in Telegram’s beta channel hint at partial offloading of virus scanning to client-side TON workers—a sort of “proof-of-clean” hash that could remove the need for your own ClamAV farm. Until that ships, the performance-and-cost framework above remains the safest, cheapest constant.

Secure bot file handling is not a one-time checkbox; it is a budgeted pipeline where every extra second or gigabyte has a name and a price. Measure first, stream second, scan always, and you can welcome 4 GB uploads without welcoming 4 GB headaches.