Setting Up A Caching Server For Steam

At home I have a custom-built NAS that I use for all of my storage needs. I use it all the time to store files like game client downloads and backups of my games on Steam. It just makes it easier to restore after an install or when a friend needs to copy a game over. Recently however I was referred to LANcache.

The basic premise of LANcache is there is a computer (or virtualized computer) that sits between you and Steam’s content servers. The computer will listen to any computer on the LAN that requests a download from Steam and jumps in and says “Oh, I’ve got that file right here! We’ll just copy it over the local network so you don’t have to go out to the web to do it.” If it doesn’t have the file stored already, it will start downloading the file from Steam to its drive and then send it to your Steam client at the same time.

This is extremely advantageous as you really only have to download a file once. If you have friends over or do an OS reinstall, a game is only downloaded once saving bandwidth and more importantly, time.

It turns out that LANcache is a fairly straightforward combination of a reverse proxy built with nginx and a DNS server configured to emulate Steam’s content servers. This is easy enough to set up. My server runs Ubuntu, but you should be able to do this with pretty much any OS that supports nginx and BIND. Just make sure that the version of nginx your OS installs is >1.6.0. Anything lower will not have the caching module and you will have to find another way to get it installed.

Installing

First we should install nginx and BIND so we can configure it:

sudo apt-get install nginx bind9

Installed! Easy enough…

Configuring

The configuration is pretty straight forward. I will most likely cover how to cache some other platforms (GOG, Origin, Battle.net, etc) in another post, but Steam should be good enough to start. For the configuration we need a few files. I’ll cover each one, their location, and their purpose.

/etc/nginx/sites-available/cache-steam

This is the main nginx config file. Here we will define the directory being used for the cache and will tell nginx that it should act as a reverse proxy and cache when hitting certain Steam domains.

server {
        listen 80;
        server_name *.cs.steampowered.com content1.steampowered.com content2.steampowered.com content3.steampowered.com content4.steampowered.com content5.steampowered.com content6.steampowered.com content7.steampowered.com content8.steampowered.com;

        access_log /data/Downloads/Cache/logs/access.log;
        error_log /data/Downloads/Cache/logs/error.log;

        root /data/Downloads/Cache/Steam;
        index index.html index.htm;

        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
                root /use/share/nginx/html;
        }

        proxy_temp_path /data/Downloads/Cache/tmp/ 1 2;

        resolver 8.8.8.8;

        location /depot/ {
                try_files $uri @mirror;
                access_log /data/Downloads/Cache/logs/access.log;
        }

        location / {
                proxy_next_upstream error timeout http_404;
                proxy_pass http://$host$request_uri;
                proxy_redirect off;

                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

                add_header X-Mirror-Upstream-Status $upstream_status;
                add_header X-Mirror-Upstream-Response-Time $upstream_response_time;
                add_header X-Mirror-Status $upstream_cache_status;

                add_header Host $host;

                access_log /data/Downloads/Cache/logs/access.log;
        }

        location @mirror {
                proxy_store on;
                proxy_store_access user:rw group:rw all:r;
                proxy_next_upstream error timeout http_404;
                proxy_pass http://$host$request_uri;
                proxy_redirect off;

                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

                add_header X-Mirror-Upstream-Status $upstream_status;
                add_header X-Mirror-Upstream-Response-Time $upstream_response_time;
                add_header X-Mirror-Status $upstream_cache_status;

                access_log /data/Downloads/Cache/logs/access.log;
        }
}

 

For your own configuration you’ll want to change /data/Downloads/Cache to your own directory. If the directory doesn’t exist yet, create it and then create a logs directory inside of it. You’ll also want nginx to pick up on the config from a directory it monitors so create a symbolic link:

sudo ln -s /etc/nginx/sites-available/cache-steam /etc/nginx/sites-enabled/cache-steam

Finally run the following to test your nginx configs. If there’s no errors, then continue to the next section.

sudo nginx -t

/etc/bind/db.cache.steam

This file sets up the A records for the DNS server. It’s fairly straight forward. Replace 10.0.1.10 with your server’s local IP (which should be static of course) and replace pathartl-server with the hostname of your server.

; BIND reverse data file
$TTL 86400
@      IN      SOA      pathartl-server. 10.0.1.10 (
                        1               ; Serial
                        604800          ; Refresh
                        86400           ; Retry
                        2419200         ; Expire
                        86400 )         ; Negative Cache TTL
;
@      IN      NS       pathartl-server.

@      IN      A        10.0.1.10
*      IN      A        10.0.1.10

Now all that’s left is the zone file. This tells clients using our DNS server that Steam’s servers exist at whatever we put in the db.cache.steam file.

/etc/bind/named.local.steam

//
// Add include "/etc/bind/named.local.steam"; to your named.conf.local file
//

zone "content1.steampowered.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "content2.steampowered.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "content3.steampowered.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "content4.steampowered.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "content5.steampowered.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "content6.steampowered.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "content7.steampowered.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "content8.steampowered.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "cs.steampowered.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "clientconfig.akamai.steamstatic.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "hsar.steampowered.com.edgesuite.net." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "steampipe.steamcontent.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

zone "steamcontent.com." IN {
    type master;
    file "/etc/bind/db.cache.steam";
};

As the header of that file pointed out, we need to enable this zone file. Add the following line to /etc/bind/named.conf.local

include "/etc/bind/named.local.steam";

Finishing Up

That should be about it for configuration. Restart your DNS and web server:

sudo service bind9 restart
sudo service nginx restart

There’s only one more step to take. You need to get your computer to use your caching server as a DNS server. There’s tons of tutorials out there to do it on any OS, but I would recommend you make the change in your router. That way anyone that connects to your LAN will download files from your cache. Make sure to set up alternate DNS servers also! Our server will only handle Steam content servers. Other domains (google.com, reddit.com, youtube.com, etc.) will not resolve.

Once you’ve changed your DNS server, go ahead and install a game off Steam. I recommend one <1GB for the initial test. Once it is done downloading, right click the game and hit “Delete Local Content”. This will delete the game off your disk. Next install the game again and watch your download speeds skyrocket! I was able to get some pretty sick speeds when it was all said and done:

2016-07-22 23_35_08-Steam

Linux Can Get Viruses Too (Apparently) or: How I Learned To Clean An Infected WordPress Install

I’ve been having a hell of a time combating my web server that this site runs on. It was first hit by some DDOSing, then someone exploited Revolution Slider which I had on another domain hosted on the same box. I ended up going through line by line in my themes and custom plugins looking for anything that could mess with the site. Continue reading Linux Can Get Viruses Too (Apparently) or: How I Learned To Clean An Infected WordPress Install

Visual Voicemail for iPhone with Google Voice

I got an iPhone recently and I noticed a lot of users raving about the visual voicemail functionality. If you’re on a prepaid carrier like I am (net10/Straight Talk), you don’t get this functionality. Luckily we can fake it with Google Voice and conditional call forwarding.

I used to use Google Voice back when I had my Nexus S on Sprint and could do carrier integration. Sadly those days are passed, but I still kept my Google Voice number. I think I can still port my phone’s number over, but I’m not wildly excited about putting all my calling eggs in one basket, especially since Google Voice has been pretty abandoned for a while. It works great for voicemail though!

Continue reading Visual Voicemail for iPhone with Google Voice

Add a toggle for Philips Hue lights in iOS (Jailbreak required)

I recently got an iPhone 6. I’ve been a die hard Android fan for 5+ years and decided to try out iOS finally. One thing that bothered me is that Android is pretty open and widgets for stuff like Philips Hue are readily available. After jailbreaking my iPhone and doing some research, I found out I could add a toggle to iOS’s control center fairly easily.

Continue reading Add a toggle for Philips Hue lights in iOS (Jailbreak required)