A reader pointed out that digg.com is doing 7 million pageviews and has 75 servers and not 25, a number which I thought was really bloated. Read about digg.com’s server setup here.
7 million pageviews / 24 hours / 60 minutes / 60 seconds = 81 Pageviews/second on average.
On average digg.com is serving 1 pageview per server. Given that most of the pageviews are hitting the homepage and are CACHED, the servers are probably only handling sub 50 pageviews a second on average.
Using myspaces most recent numbers of 1.5 billion pageviews a day they would be processing 17361 pages per second on average. If their infrastructure was as bad as digg.com’s they would need over 18,750 servers!!! I think digg.com wins the worst infrustructure/setup award of any major site hands down.
Now most of my readers are people running small startups or 1 person companies and many of them are looking at scaling or wondering what are good numbers. From what I remember reading anything below 30 pageviews a second on average is probably a bad thing. If you have a great setup you can push 400+ pageviews a second but be happy with 30+/sec.
What kinds of problem does low pageviews/server create?
1. Digg would be making a huge profit off the 200k they are supposedly making a month, not barely breaking even.
2. The company is virtually crippled when it comes to innovation. They have a extremely hard time creating something like a personalized digg. Ie showing only stories that cater to your taste. Anything that is computationally expensive is pretty much out of their reach.
3. They needed to take VC money because of all the bloat, instead of being profitable when making 20k/month they are barely break even at 200k/month. If your a Owner or CEO the last thing you want to do is give away your company.
4. You need to hire people just to manage the bloat. Managing 75 servers is no small feat, setting up a network, repairing servers, monitering, patching servers etc all makes for a LOT of over head. The hosting costs must be crazy to.
What can competitors do?
1. If i can run a mega site out of my aparment several times digg.com’s size and far far more computationally difficult someone else could do the same thing with a digg like competitor. I would find it hard to believe that a proper digg.com clone would take more then 5-25 grand a month to run, given 7 million pageviews a day.
2. Companies can buy their way into the market and innovate. The one that comes to mind is netscape, innovate, add personalization! All Netscape has to do is allow people to filter out results they don’t want. Copy google adwords negative keyword filter. If word ABC appears in the title or description don’t show me the article. ie filter out photoshop, bittorent etc
3. If you don’t know what your doing and your infrustructure is as bad as this hire someone who does. You may need to go through a few employees till you find one that knows what they are doing but this is by far the most important part of your company if you are bootstrapping.
I think we are at a point now where social networking was a few years ago. Friendster came first and was crippled. Digg.com is in the same position right now but we can’t see how crippled it is because the site is so trivial from a programming stand point. If a competetor comes along and innovates adding a lot more computationally expensive features they will pull a myspace on Digg.