Deploy large files over LAN with torrent

Occasionally tech sites mention that other than pirating movies (statement, accompanied by guilty smile) torrent is actually legit and useful protocol. That some giant company uses it to deploy really large files with their awesome solution (no details on solution given).

What they aren’t writing is that you can easily put together your own torrent deployment system with several small, free and readily available apps. How cool is that?

Why bother

Torrent is most adopted and mainstream system of distributed downloads. It proved practically to be:

• reliable (integrity checks);
• efficient (peer to peer);
• fast (overall implementation).

Direct downloads may not seem such a complex deal in local network… That is until you have to distribute dozens and hundreds of gigabytes over links of variable quality and bandwidth.

This post will cover practical setup of torrent file distribution for local network with minimal amount of time and scripting required.

Software to use

• uTorrent is one of the most popular torrent clients. We will use it to:
  • create torrent;
  • act as tracker;
  • seed file;
  • be client at target PCs.
• HFS is HTTP file server, that is easy to set up and provides excellent performance. We will use it to:
  • host torrent file;
  • web-seed file.
• psexec is awesome utility to run commands on remote computers. We will use it to:
  • deploy client;
  • start download.

Setting things up

Not required, but convenient to keep everything related in one separate folder.

HFS

1. switch to Expert mode, by clicking You are in Easy mode;
2. check that there are no limits and leeching protection is disabled in Menu > Limits (otherwise it can start to throttle and torrent clients will ban it as unusable web seed);
3. add computer’s hostname in Menu > IP address > Custom and switch to it (for convenience);
4. add files (or folder) to distribute in HFS root (this will be our web seed source);
5. Menu > Save file system… ;
6. Menu > Save options .

uTorrent

1. choose listening port and make sure it is non-random in Options > Preferences > Connection (required for tracker);
2. enable tracker in Options > Preferences > Advanced > bt.enable_tracker > set true ;
3. create torrent File > Create New Torrent… :

• tracker is http://hostname:port/announce ;
• web seed is http link to file/folder in HFS;
• check private and start seeding ;
• save created torrent file;

4. right-click task > Force Start ;
5. right-click task > Properties > check Initial Seeding ;
6. add torrent file to HFS root.

psexec

1. create batch file with command that will upload and launch clients:

psexec @%1 -c -v -d -s -i utorrent.exe /NOINSTALL /HIDE /DIRECTORY d:\savepath\ http://hostname/file.torrent

This will:

• copy utorrent to remote computer;
• launch it under system account plus hidden;
• instruct it to fetch torrent file and download content to specified path.

Launch process

Script is made to accept list of hostnames. Create file with one hostname per line and launch process (test several times before grand launch of course):

deploy.cmd hostnames.txt

That’s it. Script will upload utorrent to target computer and instruct it to start download. In no time you have your own cloud of peer to peer nodes, working hard at efficiently spreading your file(s) through network.

Possible enhancements

In this scheme I am using throwaway utorrent client without configuration. It would stay to seed file (in line with default settings), but won’t re-launch after reboot. To do this on regular basis it would be more efficient to deploy persistent uTorrent installations with configuration, enabled web interface and running as system service.

Using web seed isn’t really required. But it is quite snappy way to seed (it outperformed uTorrent seed in upload 4:1 on my first run) and you can easily scale whole thing by adding more web seeds (for example in different network segments).

I also need to think if this will work with magnet links, which will be easier to automate and would allow skip torrent file creation.

Things to be aware off

I had a lot of issues with downloads not working when launching uTorrent in non-interactive mode (default for psexec). In the end I used interactive mode -i key for psexec with /HIDE key for uTorrent. It will keep it under wraps but any error will pop uTorrent on screen, which can be unwanted.

Overall

I had tested this today and distributed 300MB file to over hundred of PCs. Over 30GB of total transfer and my initial seed had only uploaded ~5GB.

Speeds on some groups of computers with slow connection to initial seed, but fast to each other had been much higher than for direct copying. I had one computer, that took three days (on and off) to receive <30% of file, fetch rest in half an hour.

This was so awesome to build and make use of, I just spent rest of day happily (and uncharacteristically) grinning.

Related Posts

• Preventing downloading from hindering browsing
• Precisely log network ports used with Wireshark
• NetSetMan – app for quick changes of network settings
• Network Stuff – extensive multi-functional utility
• Measure file read/write speed with LAN Speed Test