The Great 2 PM Bandwidth Mystery

At 2:03 PM every weekday for two weeks, the network achieved a state previously thought impossible: worse than dial-up.

The Pattern

The tickets arrived with clockwork precision. "Network down AGAIN." "Internet completely broken." "Can't access ANYTHING urgent business critical."

Always 2 PM. Always the same symptoms. Always the same users—specifically, everyone except one person in Marketing who seemed suspiciously unaffected by what I was now calling "The Daily Bandwidth Apocalypse."

I checked the logs. According to the logs, at precisely 2:00 PM each weekday, our network traffic spiked from a reasonable 45 Mbps to a theoretical impossibility that made our monitoring graphs look like they were having existential crises.

The TTY suggested solar flares. I documented this for their performance review.

The Investigation

Time for reconnaissance. I deployed tcpdump on the core switch—the digital equivalent of wiretapping the entire office, but legal and documented in the security policy nobody reads.

$ sudo tcpdump -i eth0 -n port 6881-6889
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10LINK (Ethernet), capture size 262144 bytes

The results were immediate and damning.

14:00:01.482943 IP 192.0.2.156.6881 > 203.0.113.47.51234: Flags [P.], seq 1:1461, ack 1, win 502
14:00:01.483122 IP 192.0.2.156.6882 > 198.51.100.89.41829: Flags [P.], seq 1:1461, ack 1, win 502
14:00:01.483356 IP 192.0.2.156.6883 > 203.0.113.12.38472: Flags [P.], seq 1:1461, ack 1, win 502
[... 247 more connections ...]

BitTorrent. Port 6881 through 6889. The entire range. From a single IP: 192.0.2.156.

I cross-referenced with DHCP logs. The IP belonged to Derek from Marketing. The same Derek who was suspiciously productive during the 2 PM apocalypse. The same Derek whose workstation was, according to the asset database, a mid-tier laptop that apparently moonlighted as a seedbox.

I dug deeper. The bandwidth graphs were horrifying. Derek wasn't just downloading. He was seeding with the enthusiasm of someone who'd discovered a personal crusade.

OPERATOR: "The TTY needs to see this. It's educational."

I pulled up the connection details. 847 GB transferred in two weeks. During business hours. All of it Linux ISOs—Ubuntu, Fedora, Arch, Debian, every distribution known to open-source humanity, plus several that might have been theoretical.

TTY: "Why would someone need that many Linux ISOs?"

OPERATOR: "They don't. But the ratio must be maintained."

The Elegant Solution

I had options. Block the ports. Throttle the IP. Send a tersely worded email that would be ignored. But where's the lesson in that?

I deployed a more theatrical solution.

First, I implemented Quality of Service rules. Derek's traffic got bottom priority—below DNS, below ICMP, below the networked printer that only works on Tuesdays.

$ sudo tc qdisc add dev eth0 root handle 1: htb default 30
$ sudo tc class add dev eth0 parent 1: classid 1:1 htb rate 1000mbit
$ sudo tc class add dev eth0 parent 1:1 classid 1:10 htb rate 900mbit prio 1
$ sudo tc class add dev eth0 parent 1:1 classid 1:30 htb rate 56kbit prio 10

$ sudo tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip src 192.0.2.156 flowid 1:30

56 kilobits per second. Dial-up speeds. The nostalgia would be character-building.

Then I configured the firewall to log every single connection attempt from Derek's IP with full verbosity. Every torrent handshake. Every peer negotiation. Every byte. The logs would grow. Management would eventually notice the storage consumption. Questions would be asked.

Finally, I set up an automated email. Every time Derek's torrent traffic exceeded 100 MB in an hour, an incident report would automatically generate and send to his manager with full technical details, timestamped logs, and a helpful explanation of company bandwidth policies.

Subject line: "Network Resource Utilization Report - User: Derek, Department: Marketing."

I did not CC Derek. The surprise would be more educational.

The Resolution

The first email triggered at 2:47 PM that same day. By 3:15 PM, Derek's torrenting abruptly ceased. By 3:30 PM, Derek's manager appeared in the datacenter looking like he'd discovered his teenager's browser history.

MANAGEMENT: "Derek says his computer has been hacked and is sending weird traffic."

OPERATOR: "According to the logs, the traffic originated from authenticated sessions using Derek's credentials. I can provide the full forensic report."

I did not mention I'd already compiled said report, complete with charts showing the correlation between Derek's lunch break and the network's descent into dial-up hell. Management requested the report. I provided it. Twenty-three pages. Appendices included.

The network returned to normal the next day. Derek's seeding ratio remained forever incomplete. The TTY learned about traffic shaping, QoS policies, and the importance of the phrase "according to the logs."

Management implemented a new policy about "appropriate bandwidth usage." I implemented automated monitoring. Derek received remedial training about what "unlimited bandwidth" actually means.

Strategic deployment of technical knowledge: effective. Documentation: complete. Educational value: immeasurable.

The Operator's Notes

The moral: "Unlimited" has implied limitations, particularly when your hobby consumes 847 GB and renders 200 colleagues unable to access email. Derek achieved a seedbox ratio of 2.3 before I intervened. Admirable dedication. Terrible judgment. The network now monitors torrent protocols with prejudice. Derek now downloads Linux ISOs at home. Balance is restored. Such is infrastructure.

Documented for posterity. Filed under "Bandwidth Theft via Open Source Enthusiasm." The TTY suggested we block all torrent traffic entirely. I explained that legitimate use cases exist and blanket policies are the tools of lazy administrators. The TTY is learning. Slowly.

Uptime: 47 days. Coffee consumed: uncountable. Linux ISOs torrented during business hours: zero.