sorted by: recent | see : popular

Content Tagged with Technology-News + infrastructure

Updated with Statement from Amazon: Amazon’s S3 cloud storage service went offline this morning for an extended period of time — the second big outage at the service this year. In February, Amazon suffered a major outage that knocked many of its customers offline.

It was no different this time around. I first learned about today’s outage when avatars and photos (stored on S3) used by Twinkle, a Twitter-client for iPhone, vanished.

My big hope was that it would come back soon, but popular S3 clients such as SmugMug were offline for more than eight hours — an awfully long time for Amazon’s Web Services division to bring back the service. As our sister blog, WebWorkerDaily, points out:

With two relatively serious outages in the space of 6 months, some will be asking the question of why depend on S3? The answer is simple: the rates are hard to beat, especially for service that doesn’t require any sysadmin budget.

That said, the outage shows that cloud computing still has a long road ahead when it comes to reliability. NASDAQ, Activision, Business Objects and Hasbro are some of the large companies using Amazon’s S3 Web Services. But even as cloud computing starts to gain traction with companies like these and most of our business and communication activities are shifting online, web services are still fragile, in part because we are still using technologies built for a much less strenuous web.

Update: Antonio Rodrigez, founder of Tabblo, now part of HP, on his blog asks the $64,000 pertinent question:

…if AWS is using Amazon.com’s excess capacity, why has S3 been down for most of the day, rendering most of the profile images and other assets of Web 2.0 tapestry completely inaccessible while at the same time I can’t manage to find even a single 404 on Amazon.com? Wouldn’t they be using the same infrastructure for their store that they sell to the rest of us?

Update #2: Building an offline redundancy for Amazon S3 could be big opportunity, Dave Winer says.

Update #3: A reader sent me an email and asked these two questions

• Is the system designed to be fault tolerant? If yes, then how did it go down? After all they must have massive arrays and mirrors of their storage infrastructure.
• Is this a hardware failure or a software/design problem?

Random Thought: The S3 outage points to a bigger (and a larger) issue: the cloud has many points of failure - routers crashing, cable getting accidentally cut, load balancers getting misconfigured, or simply bad code.

Update/Statement from Amazon in response to our questions:

As a distributed system, the different components of S3 need to be aware of the state of each other.  For example, this awareness makes it possible for the system to decide which redundant physical storage server to route a request to.

We experienced a problem with those internal system communications, leaving the components unable to interact properly, and customers unable to successfully process requests.  After exploring several alternatives, the team determined it had to take the service offline to restore proper communication and then bring service online again.

These are sophisticated systems and it generally takes a while to get to root cause in such a situation—we will be providing our customers with more information when we’ve fully investigated the incident.  We’re proud of our operational performance in operating S3 for almost 2.5 years, and our customers have generally been pleased with the reliability and performance of the service. But any downtime is unacceptable and we won’t be satisfied until it is perfect.

Amazon S3 is used heavily by a number of services behind Amazon’s retail websites.  Those services were impacted, but the retail website did not show noticeable problems because it mostly uses cached data.

You wouldn’t think that next year’s DreamWorks movie, “Aliens vs. Monsters” and the search for more crude would be connected, but they are — in that they both take advantage of parallel programming for multicore chips. And when it comes to multicore chips, big bucks are on the line as the chip firm or software company that figures out how to write code to take advantage of them stands to make boatloads of money.

DreamWorks signed a deal with Intel this week aimed at parallelizing some of its code running on multicore chips to enable 3-D imaging for the 2009 animated movie. It’s not the first company to work with Intel to get more out of the multiple cores now embedded in servers, but it’s a nice example of how Intel is pushing its multicore efforts beyond simply throwing a bunch of chips at a computing bottleneck.

Like other chip firms, Intel knows that to keep compute power on the rise (and customers happy) it has to not only make the hardware more powerful with multicore chips, but also teach programmers how to use them. Otherwise, multicore chips don’t reach their full potential. James Reinders, director of marketing for Intel’s developer products division, pointed out that much of the work Intel was doing with regard to multicore, including investing in software research, selling tools to make parallel programming less cumbersome and participating in standards bodies, was done to deliver more computing power — something that can no longer be done efficiently by increasing clock speeds or adding even more cores.

“Every generation of hardware offers new capabilities, and we have rewritten our software to take advantage of it over time,” Reinders said. “Multicore will inspire us to do the same thing, but it won’t be overnight.”

It’s possible that the chip companies will be the vanguards of a new style of programming, much like programmers had to learn how to program for the web, graphical user interfaces or even e-commerce applications. Paula Richards, director of IBM’s Cell systems business thinks so. The Cell processor, designed for the Playstation 3, contains nine cores and also performs better if you adapt the code to take advantage of it.

So far IBM has focused on selling the Cell processor into financial firms, hospitals, and oil companies like Spain’s Reposal Repsol, which it inked a deal with last week. Richards said IBM doesn’t just dump that hardware and run — it spends time working with clients in each vertical to build software development kits the customer can use to get the most out of the processor. Those kits work with Intel and AMD multicore chips as well, although Richards says a user won’t see the same level of improvement they would using Cell processors.

“We knew multicore was a major inflection point in the industry,” Richards told me. “Everybody realized this and the company that addresses the [ease of programming] for this technology will win.” In some ways it’s not only about making it easy, it’s about attracting the hearts and minds of developers to a certain way of coding. That’s why IBM is offering SDKs to students who want to write parallel code on their PlayStations and Intel is pushing an undergraduate curriculum for parallel programming. This is a hardware battle fought using software.

image courtesy of DreamWorks

In planning for last Wednesday’s Structure 08 conference, we at GigaOM had our heads in the cloud. We aimed to draw attention to the resurgence of hardware underlying the various software and web services that consumers and businesses now use, and hoped to define the emerging set of offerings that comprise cloud computing.

That definition is important. But not as important, I realized, as figuring out which business models will win out. Because while everyone wants to push their own definition of cloud computing, at its heart, cloud computing is about moving, storing and delivering data on demand.

After moderating two panels, watching almost all of the speakers and having numerous conversations, I came away with the belief that most people view cloud computing not as access to computing resources, but access to services ranging from application-specific offerings such as Salesforce.com to compute grids like that of GoGrid or EC2. And when it comes to buying into such data services (be they software, a platform, storage or compute time), there are certain questions that need to asked, among them:

How do I get my data into the cloud? Maybe it’s as simple as calling up Salesforce.com, or a bit more complicated, like using an API to tap into EC2, but to use a cloud you’re going to need bandwidth. Whether it’s figuring out how to measure and appropriately charge people for bandwidth as Google is attempting to do with their structured meta data, or contracting with a CDN to lower latency, the delivery of data in and around the cloud represents one of its biggest costs and is subsequently one of the areas that’s ripest for innovation.

What format does that data need to be in? Different clouds work with different software. Some clouds work with Windows and others are only friendly to the LAMP stack. Various people expressed the idea that the industry would divide along the lines of low-margin, general purpose clouds like Amazon Web Services, and high-margin, special-purpose clouds such as Heroku’s Ruby on Rails testing environment (which is built on AWS). The key is to know what you need from a cloud before investing.

How can I change and move that data? The differing programming languages or operating systems accepted by various clouds are only part of the issue. The still undecided fight will be between proprietary formats such as BigTable and open standards that are truly standard, as in used by many, many developers. It’s a young effort, so there are no set standards yet, but until there are, transferring data kept in the cloud will never be as seamless as the bank analogy pushed by Sun CTO Greg Papadopoulos.

So while I spent most of my time trying to figure out which areas of compute infrastructure can be offered as a service while providing the highest margins or most defensible market share, I should have been keeping my eye on the data, because how providers treat the data will determine how their business models evolve.

The debate around cloud computing and software-as-a-service (SaaS) has energized industry conversations on the future of software. But in fact what we are witnessing in the software industry today is not a revolution, but an evolution. Customers are most concerned with how to use software to sustain competitive advantage, align IT with the business and deliver the best experience for users without compromise — regardless of delivery option — whether that is SaaS, on-premise software or a combination of the two. That’s why this evolution of software in a services world is so important for the industry to broadly support, and why customers deserve more than all-or-nothing ultimatums. For more, see Refresh the Net.

Other infrastructure-themed stories that may be of interest:

The Long Tail of IT
Subscription Services: The Future of Our Entire Economy
Architecting for Failure
Five Nines is Still Not Enough
Do You Know What Kind of Cloud You’re Using?
Defogging Cloud Computing: A Taxonomy
The Craft: Automation and Scaling Infrastructure
Is Infrastructure the New Marketing Medium?
Achieving Equality is Critical to the Future of the Internet
Why Google Needs its Own Nuclear Plant
The Geography of Internet Infrastructure

It’s time to exhale: Our first Structure 08 conference, slated to be held on June 25th in San Francisco, is almost sold out. With a week to go we have just 25 tickets left, and I have to tell you that despite the long sleepless nights ahead, I’m pretty jazzed about the event.

Amongst ticket buyers, we’ve seen a lot of venture capitalists that are clearly looking at the infrastructure space with some interest, possibly for investments. Also attending will be a lot of people from large corporations as they try to make sense of this whole evolution. Despite how central this issue has become to my writing lately, the mix of people is surprising even to me.

And yes, I’m totally biased, but the line-up of speakers and panelists for the conference would give anyone a serious IQ turbo boost. I recently spent some one-on-one time with speakers such as Salesforce co-founder Parker Harris and Aster Data Systems’ Mayank Bawa, and was amazed by their depth of knowledge. Here’s a quick rundown of the keynotes, panels and workshops:

Keynotes: Werner Vogels (CTO, Amazon.com), Jim Crowe (CEO & President, Level 3 Communications), Greg Papadopoulous (CTO, Sun Microsystems)

Fireside Chats: Parker Harris (Co-founder, Salesforce.com), Dr. Mendel Rosenblum (Co-founder, VMware)

Mini-Notes: Debra Chrapaty (Microsoft), Dr. Larry Roberts (Founder, Anagran), Jonathan Yarmis (AMR Research), Zach Nelson (NetSuite), & Dr. Jonathan Koomey (Lawrence Berkeley National Labs) Drew Perkins (Co-Founder, Infinera)

Panels:

• Working the Clouds: NextGen Infrastructure for New Entrepreneurs
• The New New Stack: Re-examining the Internet Platform Chain
• The Race to the Next Database: Overclocking and Analytics Augment Your Data Layer
• Harnessing Explosive Growth: Infrastructure Strategies and Tactics
• Infrastructure On Demand: Should You Build Your Future on Platform-as-a-Service (PaaS)?
• Eyes Wide Shut: Why is the VC Community Keeping Its Wallet Shut to Infrastructure?

Workshops:

• Google App Engine: Learn how to harness the power of Google’s global infrastructure for your idea
• Microsoft Data Center Secrets: Get insights from Microsoft’s infrastructure team on building out large deployments.
• Fenwick & West Venture Capital and Seed Financing: Interact with entrepreneurs and prominent angels focused on raising money for infrastructure ideas.

I want to take a moment to thank our media partners, who have helped spread the word. And of course, none of this would be possible without the generous support of our sponsors. There are many of them, and we thank them all:

If this story interests you then you should definitely check out our upcoming conference, Structure 08.

Linux was just deemed greener than Windows Server 2008 when running as the operating system for servers — a good 12 percent more efficient. Now that might not sound like much, but the same research also drove home just how difficult it is to get servers to run efficiently. So don’t scoff, and all hail the green penguin…read the full story here on Earth2Tech.

If this story interests you then you should definitely check out our upcoming conference, Structure 08.

From the company that spent $4.1 billion buying a tape company comes some cutting-edge storage news: Sun Microsystems said today that it will put solid-state Flash drives into a line of servers and other storage products, making access to stored data faster and more energy efficient. EMC made a similar announcement earlier this year.

The big vendors aren’t alone in their focus on speed. We’ve covered startups in the past whose entire existence is based on figuring out how to get to existing data faster, either through appliances or compression. With users storing more data and expecting continual access to that data, storage is no longer just about cramming as many bits and bytes in an archive as possible; it’s also about getting to them faster.

Flash, however, is a rather expensive way of solving the problem. Prices should drop as larger solid-state drives using Flash begin appearing in more consumer devices such as laptops, and the use of SSDs in the server world will only help prices fall, but it won’t be mainstream in the data center within the next year or two. Even as Flash gains ground, it will still be just one aspect of storage, for years — if not decades — to come.

While it may make sense for some companies to buy servers integrated with Flash, existing technologies will continue to flourish. Even tape is still in use today.

If this story interests you then you should definitely check out our upcoming conference, Structure 08.

Updated: With the Microsoft-Yahoo battle fading from the dynamic random memories of our over stimulated brains, it is time to turn our attention to Hewlett-Packard’s $12 billion $13.9 billion deal to acquire EDS, a services giant in its own right. The news was announced this morning. HP will purchase EDS at a price of $25 per share.

This indeed is the real thing: both companies have confirmed their talks and perhaps their seriousness. HP-EDS pairing will go down as one of the more significant developments of 2008, and its impact will be felt for years to come.

“I see it as an attempt by HP to really go head-to-head with IBM in a much more meaningful way, especially in technology services and IT outsourcing,” Dana Stiffler, research director at AMR Research told Computerworld. I think there is more to this deal than just old-fashioned outsourcing, and competing with IBM.

Typically, such a major deal means two things: Either the buyer has some issues with his current business, or he wants to make a big bet on the future. In case of HP CEO Mark Hurd, it might be a bit of both. There is only so much market share HP can carve out when it comes to printers and computers. More importantly, HP seems to be realizing that the future is about on-demand infrastructure. EDS brings to the table about 100 data centers around the planet.

Not everyone agrees with HP’s decision to buy EDS and get big fast. Forrester analyst Paul Roehrig is in that camp. Vinnie “Deal Architect” Mirchandani is someone I immensely respect and he brings up a very valid point when he writes:

But EDS is not Accenture or PwC (which IBM acquired) or TCS or Infosys. Its major strength is still in infrastructure outsourcing (though it has been growing its application and BPO capabilities nicely). HP’s outsourcing is similarly more skewed towards infrastructure. So, it is a scale play. But the timing is risky because infrastructure outsourcing is being challenged by data center consolidations, a secular decline in processing, storage and network charges and emergence of utility and cloud computing models.

However, I am taking a slightly more optimistic view of this deal, pointing out that this is HP’s bet on those very same trends — utility and cloud computing. HP might have finally realized that the future is about offering hardware as a service. Lets look at some of the recent developments

• HP bought EYP Mission Critical Facilities. John McCain, senior vice president and general manager, HP Services, at the time of the deal remarked: “Acquiring EYP Mission Critical Facilities boosts HP’s ability to help customers transform their data centers and build dynamic computing environments from the ground up.” (via Rich Miller.)
• HP bought Opsware, a data center automation software company started by Marc Andreessen, for $1.6 billion in July 2007.
• In March 2008, HP announced its data center-as-a-service initiative, targeting large companies.
• Yesterday, The Times of London reported that HP is close to buying 24 data centers currently owned by British Telecom in Europe for about $3 billion. The Times report says BT will provide Internet bandwidth, networks and remote access to HP, an area where the Silicon Valley-based giant isn’t that strong.

If you plot the EDS bid against these four recent developments, it is not that difficult to postulate that HP is building its own cloud focused on large global companies. Going further, I would channel something Vinnie says in his post.

HP’s hardware business has seen significant success in a number of emerging economies — running that infrastructure as a service does offer some unique opportunities.

I think this is a good point: Even though it’s growing fast, BRIC Bloc remains reticent to spend big dollars on infrastructure. Offering infrastructure-as-a-service to Indian telecoms or Chinese automakers of Brazilian biofuel companies is a much easier proposition then making them spend millions of dollars on blade servers, storage systems and networking devices.

Update#2: My favorite writer/thinker/troublemaker Nick Carr disagrees with me thesis about HP & the Cloud. He believes that this is a backward looking move:

an acquisition aimed at boosting profitability through consolidation and cost reduction in a mature business. The transition to the cloud will, for big companies, be a slow one, and there will continue to be much money made in running client-server infrastructures for many years.

Nick might be right, but in reality as he argues in his book, Big Switch (have you read it yet?), the world is going the way of the cloud, and even a company as stodgy as HP realizes that it has to transition to the future.

Update #3 Our colleagues over at OStatic, all of whom are open-source experts, have taken a deeper look at the deal as well. Go here for the full analysis.

What do you guys think?

If this story interests you then you should definitely check out our upcoming conference, Structure 08.

HP Labs is asking academic and research institutions for research proposals that focus on cloud computing, sustainable IT, transferring data seamlessly across a variety of media, dealing with the information explosion and dynamic computing. The computer company will offer $50,000 or $75,000 for selected proposals — primarily to pay a graduate student to help. While it’s nice to get a graduate student, an academic who really finds compelling solutions to these infrastructure problems might be able to get more money out of a VC searching for the next hot startup.

If this story interests you then you should definitely check out our upcoming conference, Structure 08.

I have a major problem with many of the Web 2.0 companies that I meet in my job as a venture capitalist: They lack even the most basic understanding of Internet operations.

I realize that the Web 2.0 community generally views Internet operations and network engineering as router-hugging relics of the past century desperately clutching to their cryptic, SSH-enabled command line interfaces, but I have recently been reminded by some of my friends working on Web 2.0 applications that Internet operations can actually have a major impact on this century’s application performance and operating costs.

So all you agile programmers working on Ruby-on-Rails, Python and AJAX, pay attention: If you want more people to think your application loads faster than Google and do not want to pay more to those ancient phone companies providing your connectivity, learn about your host. It’s called the Internet.

As my first case in point, I was recently contacted by a friend working at a Web 2.0 company that just launched their application. They were getting pretty good traction and adoption, adding around a thousand unique users per day, but just as the buzz was starting to build, the distributed denial-of-service (DDOS) attack arrived. The DDOS attack was deliberate, malicious and completely crushed their site. This was not an extortion type of DDOS attack (where the attacker contacts the site and extorts money in exchange for not taking their site offline), it was an extraordinarily harmful site performance attack that rendered that site virtually unusable, taking a non-Google-esque time of about three minutes to load.

No one at my friend’s company had a clue as to how to stop the DDOS attack. The basics of securing the Web 2.0 application against security issues on the host system — the Internet — were completely lacking. With the help of some other friends, ones that combat DDOS attacks on a daily basis, we were able to configure the routers and firewalls at the company to turn off inbound ICMP echo requests, block inbound high port number UDP packets and enable SYN cookies. We also contacted the upstream ISP and enabled some IP address blocking. These steps, along with a few more tricks, were enough to thwart the DDOS attack until my friend’s company could find an Internet operations consultant to come on board and configure their systems with the latest DDOS prevention software and configurations.

Unfortunately, the poor site performance was not missed by the blogosphere. The application has suffered from a stream of bad publicity; it’s also missed a major window of opportunity for user adoption, which has sloped significantly downward since the DDOS attack and shows no sign of recovering. So if the previous paragraph read like alphabet soup to everyone at your Web 2.0 company, it’s high time you start looking for a router-hugger, or soon your site will be loading as slowly as AOL over a 19.2 Kbps modem.

Another friend of mine was helping to run Internet operations for a Web 2.0 company with a sizable amount of traffic — about half a gigabit per second. They were running this traffic over a single gigabit Ethernet link to an upstream ISP run by an ancient phone company providing them connectivity to their host, the Internet. As their traffic steadily increased, they consulted the ISP and ordered a second gigabit Ethernet connection.

Traffic increased steadily and almost linearly until it reached about 800 megabits per second, at which point it peaked, refusing to rise above a gigabit. The Web 2.0 company began to worry that either their application was limited in its performance or that users were suddenly using it differently.

On a hunch, my friend called me up and asked that I take a look at their Internet operations and configurations. Without going into a wealth of detail, the problem was that while my friend’s company had two routers, each with a gigabit Ethernet link to their ISP, the BGP routing configuration was done horribly wrong and resulted in all traffic using a single gigabit Ethernet link, never both at the same time. (For those interested, both gigabit Ethernet links went to the same upstream eBGP router at the ISP, which meant that the exact same AS-Path lengths, MEDs, and local preferences were being sent to my friend’s routers for all prefixes. So BGP picked the eBGP peer with the lowest IP address for all prefixes and traffic). Fortunately, a temporary solution was relatively easy (I configured each router to only take half of the prefixes from each upstream eBGP peer) and worked with the ISP to give my friend some real routing diversity.

The traffic to my friend’s Web 2.0 company is back on a linear climb – in fact it jumped to over a gigabit as soon as I was done configuring the routers. While the company has their redundancy and connectivity worked out, they did pay their ancient phone company ISP for over four months for a second link that was essentially worthless. I will leave that negotiation up to them, but I’m fairly sure the response from the ISP will be something like, “We installed the link and provided connectivity, sorry if you could not use it properly. Please go pound sand and thank you for your business.” Only by using some cryptic command line interface was I able to enable their Internet operations to scale with their application and get the company some value for the money they were spending on connectivity.

Web 2.0 companies need to get a better understanding of the host entity that runs their business, the Internet. If not, they need to need to find someone that does, preferably someone they bring in at inception. Failing to do so will inevitably cost these companies users, performance and money.

If this story interests you then you should definitely check out our upcoming conference, Structure 08.