I came across this article today about digg.com's server setup. Digg.com is doing 200 million pageviews a month and has 3 webservers and 8 database servers.
Dating for the most part has been a brutal industry in terms of hardware required. AmericanSingles Quarter filing shows just how bad it is. 34 person development staff, and another 30 person team to maintain hardware and software infrastructure. In total americansingles.com has 202 full time employees, in another filing it states they have around 200 servers. This results in over 1 million a month in tech/software expenses according to the SEC filing, not bad for a site that has less then 200 million pageviews/month, note that lavalife, match.com and anothers have similar numbers.
I didn't know any of this when i created my site. When I created my site I started it in Asp.net 1.0 because I needed to learn it if i ever wanted to work at a decent job. Asp.net 1.0 worked fine until I got around 12,000 concurrent connections.
At this point there were a lot of concurrency issues, the site would slooow down and just keep getting slower as more people came on. After switching to asp.net 2.0 all those problems went away. Now when there are over 32,000 concurrent connections at peak the site does not slow down. There are however still problems. I find that once you pass 100 Pageviews/second threads timers start to not get called for no reason.
I've attached some stats that show plentyoffish at average usage. The site(plentyoffish.com and Forums.plentyoffish.com) serves ~500 million pageviews a month and does so using 1 DB server and 1 Web Server which is a far cry from the industry standard of 300+ servers for a site of this size. The system peaks out at 32,000 concurrent users and 440+ pageviews/second sustained. Pretty much all my performance problems come down to the fact I do radius based searches on nearly every pageview and match people based on their preferences and the preferences of the users being searched on. These screen shots show average usage of the site, about 50% from peak. I am using dynamic GZIP compression, every outgoing page is compressed on the FLY using the built in IIS 6.0 compression, its one of those features you turn on by messing around with the metabase. This reduces my outbound traffic by about 60mb/sec. I also have 1 mail server and 1 Image server.
I find that asp.net gets looked down on sometimes, But for me it more then holds its own and there are no real show stopper issues. I am sure that Robert Scoble will love this post, as it gives real world performance stats, or maybe not now that he has left Microsoft. For me I'm just happy that asp.net works and It doesn't give me problems so i can focus on important things like increasing traffic.
June 10, 2006 at 8:03 pm |
Do you mind sharing some of you ways to increasing traffic?
Many thanks
June 10, 2006 at 8:09 pm |
Really nice posting, and outstanding performance from the hardware resources you’re deployong! You should post this at http://www.dzone.com – a digg-like site specifically for develoeprs. I’m sure the audience there would like this.
Regards,
Rick
June 10, 2006 at 8:12 pm |
Gret article; are you using Kernel Mode Caching with IIS ?
And for anyone interested in ASP.NET performance, here’s great MSDN article addressing performance –
http://msdn.microsoft.com/msdnmag/issues/05/01/ASPNETPerformance/default.aspx
June 10, 2006 at 8:37 pm |
Where do you get your bandwidth? Did you have to scale that as well as you got bigger?
June 10, 2006 at 10:32 pm |
Isn’t Kernel mode caching automatic in IIS 6? As for caching stuff, I use no cache what so ever, every pageview is a hit to the db.
Bandwidth is just a network cable your ISP gives you, I just went and got a extra one to my cage as it went over 100 mb/sec
June 11, 2006 at 5:51 am |
Markus,
Thank you for great posting. If you do not mind, would you share a little bit more about architecture of your site?
June 11, 2006 at 12:13 pm |
[...] Markus Frind has posted his experience of using ASP.NET 2.0 for one of largest dating site PlentyOfFish.com in Canada. In his post, Markus is telling how he has addressed concurrency issues by switching to ASP.NET 2.0: At this point there were a lot of concurrency issues, the site would slooow down and just keep getting slower as more people came on. After switching to asp.net 2.0 all those problems went away. Now when there are over 32,000 concurrent connections at peak the site does not slow down. There are however still problems. I find that once you pass 100 Pageviews/second threads timers start to not get called for no reason. [...]
June 11, 2006 at 12:39 pm |
Note: I posted a duplicate of this on the wrong blog entry. Sorry about that.
Markus, you can’t simply debunk Digg for having more servers than you.
What matters is the total cost of the servers.
From what i guess, to handle so much pageviews with so few servers, i guess you scale up instead of scale out i.e. each of your server has lotsa RAM, FAST CPUs etc.
Digg, i think, chose the scale out approach i.e. many servers with average amount of RAM and average CPUs.
So as you can see, Digg’s approach is definitely more economical and scalable.
Your total server cost is about $5400 right?
assuming Digg uses the AVERAGE server which costs about $200 each with 2000GB of bandwidth thrown in, your $5400 can afford Digg 27 servers.
So i think that Digg’s server costs is lower than yours.
To conclude, Digg’s approach is better.
June 12, 2006 at 11:03 am |
Maintaining 1 server is easier than maintaining 27 servers.
June 14, 2006 at 3:50 pm |
[...] Take a look at the check by clicking here. Apparently the plentyoffish.com site serves 500 million page views per month, but only earns $1 effective earnings per 1,000 impressions through Adsense. With that very low eCPM, it’s a good job he’s getting huge volumes of traffic consistently… the site is just outside the top 500 most visited online according to Alexa… and has an incredible 2.6 million pages indexed in Google… and has 12,000+ links to it according to MSN. [...]
June 15, 2006 at 9:17 am |
Markus would you care to post the hardware specs for your servers? I really never thought asp.net could ever handle this load. I have much to learn I guess. I am going to take the plunge and get going with asp.net now after you have showed what it can really do. Very impressive indeed.
June 15, 2006 at 2:04 pm |
[...] Ein Liedchen von Performance und hohen Traffic-Lasten kann Markus Frind von Plentyoffish.com singen: Microsoft Asp.net 2.0 Performance Wenn dir der Artikel gefallen hat Bookmarke ihn bei del.icio.us Ähnliche Artikel: [...]
June 15, 2006 at 9:31 pm |
Last month I listened to an iPod interview with you and a couple clowns. They kept bashing ASP.NET and telling you that ASP.NET was junk and that you would have more scalability if you went with Linux. I kept thinking, “Who are these guys to give him advice about scalability? They probably never built anything with more traffic than a few hits per minute.”
I was really hoping that you would set them straight during the interview, but you were much too diplomatic for that. I’m glad to see this post to show that ASP.NET can handle serious traffic.
June 23, 2006 at 2:45 pm |
I’m suprised that ASP.net can be so useful as well. And I thought the world today is more crazy about the LAMP (Linux, Apache, MySQL, PHP) since eveything are open source and free.
I bet the open source commnunity will have some say when they get to see this. I hope they could at least find some real life examples to counter ASP and IIS (which are products of M$). But, yeah, thanks for opening up my eyes on how ASP could things nicely.
June 24, 2006 at 8:10 pm |
I find it very hard to believe 1 web and database server can handle
32,000 concurrent connections, regardless what hardware platform and OS you’re running. I work for a software company that deals with loads like this and we have many web servers behind content switches and local directors to handle these types of loads. Please enlighten us with what hardware your running, OS, and DB configuration specifics that allow you this kind of performance.
June 24, 2006 at 10:54 pm |
Lasher, that is my only real competitive advantage.
No other dating site can do what I can do on the technical side without spending over 1 million a month.
July 12, 2006 at 6:42 pm |
Hi Marcus.
I remember in your interview with shoemoney that you said you had an AI for filtering Adult content photos.
I’d really like to know where you found the code or the algo to do that.
Is there a package you can by for that.
Do you check skin tones?
Does it work for all races?
How many images can you check per second?
Also do you code in C# or VB.net?
July 18, 2006 at 9:17 pm |
Hi,
are you pretty sure that with 1 webserver and 1 database server you can handle 32000 concurrent connections?
July 23, 2006 at 9:29 am |
[...] PlentyOfFish started as a way for Markus Frind to teach himself ASP, but quickly marched itself up to become a very large dating site, with nearly 600,000 unique monthly U.S. visitors (Comscore) and claims of 500 million monthly page views (Comscore says only 118 million, in the U.S.). Its look and feel is rough around the edges, but it gives users what they want: the ability to browse personal profiles free of charge. PlentyOfFish also crafted a simple bulletin board system that allows users to freely chat, vent frustrations, and offer up dating advice. The site is doing very well financially. [...]
July 23, 2006 at 9:08 pm |
[...] PlentyOfFishは創設者のMarkus FrindがASPを勉強するために始めたプロジェクトだが、たちまち非常に大規模な出会い系サイトに成長した。米国のユニーク訪問者数は月間60万人に近づいており(Comscore調べ)、運営者側では月間5億ページビューがあると主張している(Comscoreでは米国で1億1800万ページのみと集計)。デザインや使い勝手にはぎごちなさも目につくが、ユーザーがもっとも必要としている機能―個人プロファイルを無料でブラウズできる―はきちんと備えている。 PlentyOfFishは簡単な掲示板システムを用意しており、ユーザーは自由におしゃべりしたり、グチをこぼしたり、デートにあたってのアドバイスをしたりできる。このサイトは経営上たいへんうまくいっている。 [...]
August 23, 2006 at 1:50 am |
Hi, Let me take a guess what about your techniqes Markus, you use some sort of sql optimization, and cache the hell out of it. This way, the content is served straight from memery instead of database querry every time the user ask for it.
Let me know if this is the case
October 4, 2006 at 12:33 pm |
[...] PlentyOfFish started as a way for Markus Frind to teach himself ASP, but quickly marched itself up to become a very large dating site, with nearly 600,000 unique monthly U.S. visitors (Comscore) and claims of 500 million monthly page views (Comscore says only 118 million, in the U.S.). Its look and feel is rough around the edges, but it gives users what they want: the ability to browse personal profiles free of charge. PlentyOfFish also crafted a simple bulletin board system that allows users to freely chat, vent frustrations, and offer up dating advice. The site is doing very well financially. [...]
August 24, 2007 at 2:20 pm |
[...] a lot, isn’t it? But the ratios are even worse. According to the owner himself, he has 500 million pageviews a month which is [...]
December 21, 2007 at 3:50 am |
Hey Marcus,
I think I accidentally made a few duplicate entries so sorry for that but I”m actually brand new to ASP.net and trying to understand better how your sequences of prime numbers helped your efficiency in relation to processing data on your dating site?
Could you explain that a little better to a novice?
Congrats by the way!
January 27, 2008 at 9:24 am |
Hey Markus,
Whats the best way to get in touch with you? I had more than one question for you and didn’t want to gum up your blog
July 25, 2008 at 6:10 am |
im in confused state
October 3, 2008 at 11:22 am |
Hi Marcus!
What is your servers configuration? Handling so much traffic with so few servers seems unreal to me.
January 22, 2009 at 2:26 am |
Bull-manure!