SSD Image Server Array.

On the weekend I went and deployed my first Sold State Drive Array.     15  Mtron 32GB  SSD drives in a MD1000  enclosure.    Plentyoffish has a lot of images on every single page, unlike most dating sites.    As a result i’m serving  5,500 images per second at peak and  50-60,000 new images being uploaded per day.    Anyone else doing anything fancy with image serving ?   Aside from the database,  images are the biggest pain in the ass when it comes to running the site.

27 Responses to “SSD Image Server Array.”

  1. Gopi Says:

    Did you tried something less sophisticated like RAM drive?…Its easy to do in linux, no idea about windows though!

  2. Jeff Lewis Says:

    Mogilefs

  3. Aris Says:

    Content delivery network perhaps? Or maybe something like Amazon’s S3 storage. A good CDN will serve the images from the closest/fastest server to the client I’m not sure if Amazon do this, but with them you don’t have to worry about backups – just pay storage and bandwith costs.

  4. David Says:

    i have to second the comment about a CDN. at my last company, we used both Akamai and Cachefly with great results. and they aren’t as expensive as one might think.

  5. Markus Says:

    I am using a CDN, but even with a CDN You get a massive hit to the image servers. Lots of time for no reason I see double to triple the traffic as the CDN gets overloaded or what not.

  6. Adam Says:

    I’m totally curious about the performance of your SSD array. Can you share some preliminary stats? Are you seeing a speedup? What did your old image server setup look like?

  7. Aris Says:

    Sounds like you need to investigate that with your CDN provider – the whole point of them is that they take load off you, not the other way around.

    Alternatively look at some sort of front-end cache/proxy and make sure your cache-control headers are setting cache expiry times to take full advantage of these caches for static content such as your images. One unit which is real nice for this (but real expensive too) is the F5 WebAccelerator though if you are using a CDN – they should be doing all this for you.

  8. todd Says:

    Take a look at Kevin Burton’s blog. He’s done a lot of good work evaluating SSDs.

  9. Mayo Says:

    Markus are you using Peer1 CDN now instead of Akamai? I would like to hear your experience because i am putting my servers on Peer1 network.🙂

  10. Mayo Says:

    Actually i had leased ServerBeach servers but if they proove quality there is no reason to go shopping elsewhere.😉

  11. Zak Kinion Says:

    There’s always good ole Amazon Web Services ec2/s3. Its gotten a lot better. They let people use static ip’s now.

  12. dmose Says:

    what advantage does a static ip give? skip the DNS lookup on each call?

  13. Zak Kinion Says:

    It used to be that when an AWS instance went down, you would lose the IP address and once it came back up, would take time to propagate. Plus you have to update the ip address in the dns records as well. Its just a pain.

    On my setup, its even more severe, because I use a socket server that is licensed, which takes 24-48 hours to renew the license with a different ip address, resulting in massive down time.

  14. Nick Bolibruch Says:

    Yep, this blog is on my regular site list now. I know large scale image storage will be a long term problem for shit I’m working on. Plus PoF got me laid more than once, in addition to making some good friends.

    Those SSD drives must be pretty sweet.

  15. Avi Marcus Says:

    Hey. Just one note – I remember that people put their pictures on a different sub domain or something to make sure they have more control on making the content not expire / remain cached.

  16. Paul Pattison Says:

    Consider limiting the number of http requests or sub domains being served. There are more tips in this must read for any developer:
    http://www.slideshare.net/stoyan/high-performance-web-pages-20-new-best-practices

  17. Mayo Says:

    “Plus PoF got me laid more than once, in addition to making some good friends.”

    LOL, subscribing on that too!😉

  18. Jenny Kim Says:

    Hi Marcus,

    I’m sure you’ve heard this many times, but I deeply admire your success and seek to emulate it. I have a website in the works, concentrated in the wedding industry. I heard you say that your site plentyoffish.com reached its phenomenal level of success without any marketing on your part. Could you please elaborate on that? So you just made sure you had the proper keywords and people were able to find your site? Is that all it took for you, besides search engine submissions?

    Thanks,
    Jenny

  19. Val Says:

    Hey Markus, we use caching servers. But we’ve got several 1Us doing this. And we can get by without the need of a CDN, and we push about 7K images a second as well. Email me if you want to discuss details.

  20. Marc Says:

    Mayo,

    I’m sure we’re all very impressed and you’ve done Markus proud;)

  21. Mayo Says:

    LOL!

    Markus had done a great job! He might even be partially responsible for getting me married! ;D

  22. Chris Says:

    Markus,

    How do you think about the investment should be on SSD or add proxy server? I know SSD is decent on random access, but how do you compare with adding fleet of proxy server like what wikipedia does ? Cost-wise / Management-wise ?

  23. Free Dating - egoDating.com » Blog Archive » PlentyOfFish Image Serving Says:

    […] Image Serving Posted by admin In Dating 7Апр 08 PlentyOfFish is now serving images off solid state drives, which employs a technology similar to what is found in the new MacBook Air- no moving parts. […]

  24. Free Dating - egoDating.com » Blog Archive » PlentyOfFish Image Serving Says:

    […] Image Serving Posted by admin In Dating 12Апр 08 PlentyOfFish is now serving images off solid state drives, which employs a technology similar to what is found in the new MacBook Air- no moving parts. […]

  25. Virtual Pet Lover Says:

    I was looking at getting one of these for hositing my MySQL database and using memcahce and proxy servers to scale the images and videos.

    I am interested to hear if you think it is worth the expense (as these drives are not cheap)

  26. Paul Says:

    Markus-

    How did these drives end up working out for you? It’s been some time now and I am in a similar situation where SSDs, look like a viable option for us as well. I would appreciate any feedback you have on your experience with SSDs in an enterprise environment.

    Paul

  27. website server hosting Says:

    Attractive part of content. I just stumbled upon your website and in accession capital to say that I acquire actually loved account your weblog posts. Any way I’ll be subscribing on your feeds or even I achievement you access consistently fast.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: