We are on the verge of what is and will probably be the greatest technical revolution since before the creation of the internet. Today nearly all major sites performances are limited by the speed of the hard drives used in servers. Most dating sites and companies like Myspace.com are using hundreds and in some cases thousands of servers in order to keep serving up pages. Hard drives are brutally slow several thousand times slower then attempting to access RAM.
This is all about to change, in the last few years Solid State Drives which are essentially RAM modules that retain information when power is turned off have been falling dramatically. Most of the projections out there say that the industry will go from 50 million in sales last year to $5 -$10 billion in sales by 2010 pretty much completely replacing the current hard drive industry.
So what does all this mean? A solid state hard drive is currently about 250 times faster at accessing data than standard hard drives. Right now my database server peaks out when trying to serve 20 million pageviews a day and I need a second server to take over some of the load and all the issues are disk/IO bound. If my database server was using solid state drives I wouldn’t have any disk issues even if I served a billion pageviews a day.
Now of course CPU and bandwidth issues would prevent me from serving 1 billion pageviews a day off a single DB server. But if I were to scale my infrastructure to handle the load of myspace using Solid State Drives I could probably do it by only adding only 5-10 new database servers and 20-25 web servers. This is a far cry from the thousands of servers myspace is currently using and I certainly wouldn’t need the 50+ tech staff that myspace has.
In summary, Solid State Drives are going to dramatically change the way the internet works. With these drives extremely low cost/ free services are possible on a truly massive scale. More importantly sites like facebook and myspace.com are going to go from having tech budgets that run into the tens of millions a year to low digit millions making them wildly profitable business.
October 3, 2006 at 8:04 am |
Well this seems to be great news, with these new solid state drives, would that also mean that pc’s will now boot up into windows a lot quicker, im so sick of waiting for it to load
yes i am very lazy
October 3, 2006 at 8:56 am |
I think a solution is allready here, but it will work mostly on *nix based servers.
1) There are SCSI controllers that have RAM cache of more than 1GB, plus the OS can cache to the RAM
2) Database servers like mysql can be “locked” in memory giving you the same result that you would have with a solid state drive.
Of course this technology will help, and propably will make the smaller companies more capable to compete with the giants…
October 3, 2006 at 9:51 am |
Pstt… who told you about google’s secret
October 3, 2006 at 10:39 am |
Solid state is not going to catch up with magnetic any time soon, price/MByte at least. And if price is no object, then RAM based RAID is an order of magnitude faster than any flash drive.
That is why big companies just throw more machines at the problem, its just cheaper and easier when things are designed to scale in clusters of PCs anyway.
October 3, 2006 at 11:25 am |
revolution? more like evolution.
October 3, 2006 at 2:11 pm |
Its an evolution in hardware but a revolution in what is possible.
Google stores stuff in ram, but the problem is you can’t store a database in ram. A single glitch and everything is gone. As for booting your computer it is instant as you just leave where you left off.
October 3, 2006 at 2:16 pm |
Once again your analysis is perfectly correct. You would be surprised if you knew what was currently being developed by some small teams in stealth mode to take advantage of this.
October 3, 2006 at 2:23 pm |
I’d agree with Rush, this is hardly evolutionary. “In the Beginning”. long before the Internet was even a dream _all_ computer memory was solid state. Magnetic media evolved not because it was better (faster) but because it was cheaper … and engineers knew a lot about recording magnetically (analog on wire, later tape) than they did any other media.
Pundits like Jerry Pournelle in tech journals like Byte (RIP) have predicted the death of “mechanical monster” disk drives for many years based on the principle that sand (silicon) is much cheaper than meticulously machined metal. That their predictions have not yet come to fruition is an anomaly of economics. not technology.
I did have to laugh a lot at Nick’s “*Nix uber alles” view … operating systems do not make up for faulty mechanical structure even if they are “open source”.
If you’ll excuse me I have to get back to my project of converting my 500 cubic inch Cadillac to a full Linux OS … it’ll get 400 mpg at 120 mph and jam all speed radars within 5 miles too, just as soon as I can get the darn kernel to compile
October 3, 2006 at 2:27 pm |
Faster hardware means you can scale easily, but it doesn’t solve the bigger and more important, getting to the stage where you need to scale.
What this means is just that the costs of google and other big players will reduce. It doesn’t sound that revolutionary to me, unless you also magically get 100 million users with the faster RAM to serve it.
October 3, 2006 at 3:49 pm |
Store a copy of the DB in RAM, not the DB itself. Then most visitors use that copy instead of the disk copy.
ie. DB writes should go straight to disk, but most reads come off the RAM based copy.
October 3, 2006 at 4:20 pm |
Regarding reliability of RAM based drives, the ones we use an onboard battery backup. So you have at least 8 hours to get the latest updates off if things really go tits up.
October 3, 2006 at 5:28 pm |
Vast savings possibly, but you still have to pay for bandwidth.
What is does do is that it allows you to simplify storage logic. 2gb solid state change log and 4gb read cache on the application server, and it doesn’t matter much how fast the backend to that is.
October 3, 2006 at 6:59 pm |
Ali and Henrik,
Good points both, but you have to think about what kinds of text-based services are made possible by this technology, which could both attract millions of users and not require a lot of bandwidth at the distribution end. Where this technology leads is towards ‘good enough for the consumer’ solutions to problems currently addressed by some of the largest (and most legacy-constrained) IT applications on the planet.
October 3, 2006 at 9:01 pm |
Hmm… I see many of the comments reference RAM as the storage medium.
Now this is NOT my area of expertise, but I believe the that “revolution” is in FLASH, which is, of course, writeable and non-volatile. Think the cards in digital cameras, cell phones etc., not SDRAM (which does need battery backup).
I read and recommend http://storagemojo.com/ where stuff like this gets written about.
October 3, 2006 at 10:49 pm |
[...] I admittedly am no expert in computing hardware, enterprise server management, or solid state disk drives, but if Markus Frind is correct about coming advances in hardware it may make a lot of the newer web businesses profitable in the next few years, or at least move them from losing money into profitability. [...]
October 3, 2006 at 11:24 pm |
Marcus I agree that solid state drives for primary storage and cache, will provide a huge leap in performance as soon as this is affordable. MRAM replacing RAM could even provide the ability to instantly boot/reboot a server.
I/O performance as well as ability of systems to recover extremely quickly both stand to gain dramatically, so I would agree with the revolutionary nature of these technologies – operating systems will be adapted or new ones will arise to take advantage of the new physical contrainsts on performance.
Now imagine combining these technologies with storage and OS virtualization and perhaps centralizing your solid state storage as a SAN (with ultra high speed connections to your MRAM and 64+ core CPUs). Data centers in 10 years will start to change shape noticeably.
Vista is taking a more evolutionary step including a feature to take advantage of a flash RAM cache to aid in performance providing a tier between RAM and Hard Disk. See here:
http://www.extremetech.com/article2/0,1697,2017818,00.asp
October 4, 2006 at 5:55 am |
Flash-based SSD have great performance – all the benchmarks show that – but they don’t have the durability to handle frequent updates, such as database indices, for very long. That makes them very costly.
There are technologies on the horizon that promise flash speed with disk durability, but they aren’t here yet. The multi-copy strategy – RAM and disk, or on multiple machines as in the Google File System – is likely to be the most efficient solution for some time. For more discussion check out http://storagemojo.com/?p=209 “Architecting the Internet Data Center” and other articles at StorageMojo.com.
October 5, 2006 at 12:05 am |
Is anyone aware of who the leading players are for Solid State Drive? I am interested in finding out more about investment opportunities here.
If Markus is right, going from $0 million to $5-10 billion in sales in four years… well if you bought some stock in those companies, you may do as well as plenty of fish. Okay, not that well but you get the idea.
October 5, 2006 at 12:07 am |
Typo in my last post, meant going from $50 million to 5-10 billion in my last post.
October 5, 2006 at 7:08 pm |
SSD’s have been around for decades and done little. Flash drives, like the thumb drives we all have now, are the hot ticket.
Samsung and Sandisk are big players. There is also a growing software market that provides unique software designed to be used on flash drives. Ceedo and U3 are two examples and there are probably others. I also expect Apple to ride this wave pretty hard, since they have a great flash supply chain.
October 5, 2006 at 11:34 pm |
Thanks Robin.
October 6, 2006 at 2:49 am |
I talked to some suppliers, Myspace.com and other big sites are already using SSD’s. Prices are $200 to $300/gig down from $600 at the start of the year.
October 11, 2006 at 12:54 am |
very nice article, i also think that flash drives will replace HD, more speed and low power consumption, a big evolution is coming
July 8, 2010 at 6:02 am |
Thanks for share. This is the best site i’ve read today. I love reading the informative articles about summer sale discount deals and couponss..And your is really helpful and easy to understand.. I will also try to do.
most up-to-date Coupons