Insane Bandwidth.

For the first time in several months I checked my Image serving bandwidth. Its disturbing to see it crossed 1 TB/day serving 171 million images. This isn’t even all my image traffic as its only the stuff I’m sending through Akamai.

In order to meet all my image requirements I installed a 6TB Storage array today that will handle the millions of full sized images being uploaded every month to the site. I’ve been tossing the originals till now, but i figure i should keep them. I’m doing a total around 2- 3TB a day now, If I didn’t have compression (GZIP) turned on for pages I’d be serving at least another 2 TB extra per day.   All combined with images, page requests and IM status polling there are about  600 to 700 million “hits” per day.

Image traffic only

12 Responses to “Insane Bandwidth.”

  1. Joshua Says:

    Damn. You must sit back and at laugh at and guys like that when you look at this stuff.

    It’s a beautiful day, you’re looking across to the north shore mountains and those guys are running around like a bunch of chickens with their heads cut off wondering how they’re going to save their ship from sinking.

  2. robert klein Says:


    i saw you walk down yaletown tonight with 2 girls…I yelled “you rock Markus”

    rock on

  3. bisi Says:

    How much does using that amount of bandwidth cost ?

  4. jimbo Says:

    Markus, have you ever thought about trying out Amazon s3? I wonder if it would save you a bit of money?

  5. Markus Says:

    I definately want to use something like S3 when they get it geo load balanced.

    I didn’t hear you Rob, was to busy with the girls lol

  6. Fabulous Entertainment Blog » Blog Archive » Scaling static content Says:

    […] read a post today in the Plenty of Fish Blog about his Content Delivery Network (CDN) for his static content. He […]

  7. Rich Says:

    Impressive stats! Why don’t you limit the size of the images during upload?
    Users uploading massive .JPG images will kill the site.

    Good luck! the free alternative to LOL🙂

  8. Markus Says:

    They are on average 6kb in size. You can’t get much smaller than that. The bandwidth numbers don’t mean a thing, its the sheer number of files that are painful. Youtube was only doing 100 million videos a day when it got sold, i’m doing 200 million images a day. It creates all kinds of performance issues because the disks can’t spin fast enough.

  9. Steve Algieri Says:

    Markus, this is amazing.

    Do you have a geo graph showing where the connections are coming from (i.e. which Akamai node they’re hitting)?

  10. -gary Says:

    Any info on the percentage of repeat to unique images? Not that it matters if you are CDNing everything (let them worry about it), but I wonder if a large caching system is a worthwhile investment for those that aren’t.

    There was a discussion somewhere about entertaining the possibility of a 64GB machine to cache all of their small thumbnails.

  11. Rich Says:

    I just looked at 2 images on one profile 10kb and 12kb, sure they are 6kb on average?

    It all adds up dude😛

    Still if you complicate pic uploading the site will suffer!

  12. partition is the key Says:

    You will need much more disk spindles to handle the peak traffic. Partitioning the data would be the way to scale.

    Regarding to the bandwidth cost, it is probably around 20K/month, just a wild estimate🙂

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: