Hotlinking – Why Popularity Can Be Bad

One of the sites Darkside runs is called eSport – It’s a New Zealand orientated, light hearted news service and gaming community. In the past 3 months the page has generated on average 200 unique visitors a day, with a maximum of 1197 users in a single day. Those really aren’t very big numbers in the overall scheme of things, not tiny – but far from impressive.

I had a look at the bandwidth monitoring (purely on a whim) on the machine eSport lives on, and to my surprise the page had gone from pushing 150MB a day, to suddenly pushing 1.5GB a day. That’s a ten fold increase in bandwidth being burnt.

eSport has a gallery used by its members that allows image uploads, and a lot of those images rank very highly in Google Image Search. One of those images in the gallery was of a video game character from a game called Team Fortress 2.

Here he is:

A shady looking fellow indeed. This image comes in at about 60k, which is by no means large. This is a local copy of the image, not the copy on eSport.

Yet this little guy burnt 1.2GB on his very ownsome in a single day. He was downloaded approximately 20,480 times.

A Google Image Search for ‘tf2 spy’ (the character is called a spy) returns a page full of images, the 3rd and best image of the spy as it turns out is hosted in the eSport Gallery.

It’s the 22nd of October, we’re about a week from halloween, and that means dress-ups. I checked out the eSport Gallery access logs and discovered this little image was linked to from dozens of high traffic geek orientated forums and webpages in relation to Halloween, and what folks would be dressing up as.

I felt bad about having to do something about it, but as this box is meant to be high performance and really only for DSM’s NZ and AUS clients it’s located on the Auckland CBD, on fiber, APE and WIX peered and very, very fast. Unfortunately that speed comes at a cost, and international bandwidth from New Zealand is extremely expensive.

A little bit of begrudging .htaccess magic and the problem was solved.

RewriteCond %{HTTP_REFERER} !^http://(.+\.)?esport.co.nz/ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteRule .*\.(jpe?g|gif|bmp|png|jpg)$ /nohotlink.jpe [L]

What does that do? well, this:

I’ve linked to an image in the eSport gallery, and the .htaccess rule has switched it out for this placeholder image. (which is thankfully only 1k)

Be on the lookout for Spies this festive season, and have a great time!