From: Jay Levitt Date: 2007-10-04T06:40:06+09:00 Subject: Re: Recent Criticism about Ruby (Scalability, etc.) On Wed, 3 Oct 2007 22:19:57 +0900, Charles Oliver Nutter wrote: > I find this perspective puzzling. In most large datacenters, the big > cost of operation is neither the cost of the servers nor the cost of the > development time to put code on them, it's the peripheral electricity, > administration and cooling costs once the application written must be > deployed to thousands of users. Speaking from my AOL experiences (which are admittedly getting a bit long in the tooth compared to today's apps and hardware): Yeah, but. On the one hand, our hardware costs certainly dwarfed our development costs. (And by "certainly", I mean "I certainly imagine so, having absolutely no memory of what the actual numbers were.") And our operating costs were a big part of those. What's more, cooling and power do NOT scale linearly; at some point, the incremental cost of one more CPU is a new data center or an extra utility trench. Plus, at the time, anyway, there weren't big enough power/cooling products on the market to handle our needs. I imagine that's changed now. On the other hand, that wasn't our biggest problem. If handling ten million users costs $X/user, and makes $Y/user profit, and doubling to 20 million users still costs roughly $X/user and thus makes $Y/user profit, then doubling capacity to double the user base is really easy. Hardware/operating costs do impact whether you can make a per-user profit in the first place, but they don't impact scaling costs. Whether Ruby takes up too much server power is a base hardware cost, not a scaling cost. If it's too heavyweight, it's too heavyweight at 1000 users, just like it will be at 100 million. But our biggest problem was development - not even development costs, but development *ability*. Software development does not scale anywhere NEAR linearly, as Frederick Brooks well knows. Double the number of programmers, and you've exponentially increased the number of interpersonal communications and meetings. That needs more project managers and technical managers, which again increases the number of communications channels. And that needs more support staff (HR, legal, facilities, etc.) More people need more office space and parking, and the bigger your campus, the more time is wasted travelling, and the more space is "wasted" on non-productive square footage like kitchens and hallways. And so forth. I haven't even mentioned the sheer difficulty of hiring that many developers. I imagine many here have had the problem of "If I could hire someone else, I'd have less to do; but I have too much to do to find time to hire someone." That doesn't scale either. Then there's saturation. At one point, there were nearly as many technical job openings in the Washington Post as there were total unemployed people in the Washington area. Bad news. Then there's the sad fact that software simply does not scale cleanly. Even if the core functions are dollar-scalable - and I'll touch on that in another post - things like maintenance, operability, reporting, metering, debuggability, etc. are not. So as your business scales, you'll spend more and more time doing infrastructure development instead of feature development. There comes a point where there's a negative economy of scale; you're too big to actually do any real innovation, because all your resources are sucked up keeping the beast alive. The point of all this is that anything - ANYTHING - that increases developer productivity is a huge, huge win. Sure, developer salaries may not have been over 50% of our expenditures. But without developer productivity, you stagnate, and someone else zooms right past you. -- Jay Levitt | Boston, MA | My character doesn't like it when they Faster: jay at jay dot fm | cry or shout or hit. http://www.jay.fm | - Kristoffer