SSD Image Server Array.
On the weekend I went and deployed my first Sold State Drive Array. 15 Mtron 32GB SSD drives in a MD1000 enclosure. Plentyoffish has a lot of images on every single page, unlike most dating sites. As a result i’m serving 5,500 images per second at peak and 50-60,000 new images being uploaded per day. Anyone else doing anything fancy with image serving ? Aside from the database, images are the biggest pain in the ass when it comes to running the site.
March 27, 2008 at 7:07 pm
Did you tried something less sophisticated like RAM drive?…Its easy to do in linux, no idea about windows though!
March 27, 2008 at 7:39 pm
Mogilefs
March 27, 2008 at 7:41 pm
Content delivery network perhaps? Or maybe something like Amazon’s S3 storage. A good CDN will serve the images from the closest/fastest server to the client I’m not sure if Amazon do this, but with them you don’t have to worry about backups - just pay storage and bandwith costs.
March 27, 2008 at 8:09 pm
i have to second the comment about a CDN. at my last company, we used both Akamai and Cachefly with great results. and they aren’t as expensive as one might think.
March 27, 2008 at 8:35 pm
I am using a CDN, but even with a CDN You get a massive hit to the image servers. Lots of time for no reason I see double to triple the traffic as the CDN gets overloaded or what not.
March 27, 2008 at 8:45 pm
I’m totally curious about the performance of your SSD array. Can you share some preliminary stats? Are you seeing a speedup? What did your old image server setup look like?
March 27, 2008 at 9:17 pm
Sounds like you need to investigate that with your CDN provider - the whole point of them is that they take load off you, not the other way around.
Alternatively look at some sort of front-end cache/proxy and make sure your cache-control headers are setting cache expiry times to take full advantage of these caches for static content such as your images. One unit which is real nice for this (but real expensive too) is the F5 WebAccelerator though if you are using a CDN - they should be doing all this for you.
March 27, 2008 at 11:27 pm
Take a look at Kevin Burton’s blog. He’s done a lot of good work evaluating SSDs.
March 28, 2008 at 7:00 pm
Markus are you using Peer1 CDN now instead of Akamai? I would like to hear your experience because i am putting my servers on Peer1 network.
March 28, 2008 at 7:02 pm
Actually i had leased ServerBeach servers but if they proove quality there is no reason to go shopping elsewhere.
March 28, 2008 at 10:32 pm
There’s always good ole Amazon Web Services ec2/s3. Its gotten a lot better. They let people use static ip’s now.
March 29, 2008 at 9:28 pm
what advantage does a static ip give? skip the DNS lookup on each call?
March 29, 2008 at 10:00 pm
It used to be that when an AWS instance went down, you would lose the IP address and once it came back up, would take time to propagate. Plus you have to update the ip address in the dns records as well. Its just a pain.
On my setup, its even more severe, because I use a socket server that is licensed, which takes 24-48 hours to renew the license with a different ip address, resulting in massive down time.
March 30, 2008 at 2:48 am
Yep, this blog is on my regular site list now. I know large scale image storage will be a long term problem for shit I’m working on. Plus PoF got me laid more than once, in addition to making some good friends.
Those SSD drives must be pretty sweet.
March 30, 2008 at 10:55 am
Hey. Just one note - I remember that people put their pictures on a different sub domain or something to make sure they have more control on making the content not expire / remain cached.
March 30, 2008 at 4:32 pm
Consider limiting the number of http requests or sub domains being served. There are more tips in this must read for any developer:
http://www.slideshare.net/stoyan/high-performance-web-pages-20-new-best-practices
March 30, 2008 at 5:19 pm
“Plus PoF got me laid more than once, in addition to making some good friends.”
LOL, subscribing on that too!
March 30, 2008 at 6:44 pm
Hi Marcus,
I’m sure you’ve heard this many times, but I deeply admire your success and seek to emulate it. I have a website in the works, concentrated in the wedding industry. I heard you say that your site plentyoffish.com reached its phenomenal level of success without any marketing on your part. Could you please elaborate on that? So you just made sure you had the proper keywords and people were able to find your site? Is that all it took for you, besides search engine submissions?
Thanks,
Jenny
March 30, 2008 at 7:45 pm
Hey Markus, we use caching servers. But we’ve got several 1Us doing this. And we can get by without the need of a CDN, and we push about 7K images a second as well. Email me if you want to discuss details.
April 1, 2008 at 5:18 am
Mayo,
I’m sure we’re all very impressed and you’ve done Markus proud
April 1, 2008 at 12:18 pm
LOL!
Markus had done a great job! He might even be partially responsible for getting me married! ;D
April 3, 2008 at 4:45 am
Markus,
How do you think about the investment should be on SSD or add proxy server? I know SSD is decent on random access, but how do you compare with adding fleet of proxy server like what wikipedia does ? Cost-wise / Management-wise ?
April 7, 2008 at 9:37 pm
[...] Image Serving Posted by admin In Dating 7Апр 08 PlentyOfFish is now serving images off solid state drives, which employs a technology similar to what is found in the new MacBook Air- no moving parts. [...]
April 12, 2008 at 10:49 pm
[...] Image Serving Posted by admin In Dating 12Апр 08 PlentyOfFish is now serving images off solid state drives, which employs a technology similar to what is found in the new MacBook Air- no moving parts. [...]
May 9, 2008 at 3:02 pm
I was looking at getting one of these for hositing my MySQL database and using memcahce and proxy servers to scale the images and videos.
I am interested to hear if you think it is worth the expense (as these drives are not cheap)