In the debate around Internet regulation and traffic, it’s important to understand the things that drive how much bandwidth we need. Without fixing the bandwidth shortage on the wire or in the protocols, we make it easy for carriers to claim that they need to regulate bandwidth in order to survive. If we can fix the bandwidth and efficiency problems, we take away one of the main reasons telcos claim they need to shape traffic and interfere with the Internet.To oversimplify somewhat, there are three basic ways to improve the Internet’s capacity:

Regulate traffic

Treat different traffic differently, according to its needs, through tiers of service. Technologies and protocols such as MPLS, RSVP, traffic shaping, queueing and so on can prioritize some applications or sites over others. Unfortunately, this is the zero-sum game that surrounds the Net Neutrality debate: My online video is better, but I can’t use search any more.

There are also significant consequences for users when companies employ some of these technologies. For example, throttling traffic by injecting TCP resets into the traffic stream can interrupt web sessions; similarly, dropping packets midstream messes with voice and video quality.

As a result, many service providers resort to “overlay” networks from content delivery networks (CDNs) like Akamai, BitGravity, and so on. Or they negotiate private peering arrangements. Either way, we’re already overcoming congestion with “premium” delivery services that the hosting provider underwrites.

Throw more bandwidth at it

This is needed both at the edge, in the “last mile,” and in the core. U.S. taxpayers already paid for build-out of broadband at the edge, in 1994, and got nothing for it. The Clinton-Gore administration’s “National Infrastructure Initiative” was supposed to rewire America with fiber optics. The various Bell companies agreed to rewire homes and public buildings in return for financial aid. Millions of households were supposed to get bi-directional, 45 Mbps traffic. But it didn’t happen.

There’s another problem with the edge: It’s asymmetric. The A in ADSL stood for this. Cable networks were designed to push hundreds of channels out to a household, but not to send much back upstream. Today’s symmetric protocols like peer-to-peer and videoconferencing overload the upstream channel. This is gradually changing, but it’s a costly overhaul.

In the core of the network, we’re still finding new ways to split the spectrum and get more bandwidth out of the fiber that’s already deployed. So it’s not a matter of digging trenches as much as it is of swapping out old Wave Division Multiplexing, switches and routers. In many cases, these devices were optimized for relatively short, request-and-respond traffic, so new patterns like P2P and streaming video don’t work as well for them.

Use better protocols

In other words, be more efficient with what’s available. One great example of this is video. Today, the de facto standard for one-way video is a Flash player. But look under the covers, and there’s tons of encapsulation. Video streams are stuffed into HTTP, which in turn goes inside TCP (and sometimes SSL). There’s a lot of overhead in this, not to mention the fact that TCP is a connection-oriented protocol that tries to deliver all bytes and isn’t well-suited to voice and video.

Protocol inefficiencies are one reason companies like Netli (acquired by Akamai), Riverbed and Peribit (acquired by Juniper) are able to squeeze huge performance increases from applications. They simulate inefficient protocols on both sides (to keep client and server happy) and fix them in the middle (where congestion occurs.)

But we rely on encapsulation and inefficiency because it works. Sure, we could stick video on a UDP datastream and get better, more efficient video with less bandwidth and better loss recovery. But our firewalls wouldn’t accept it. (Tunneling through domestic firewalls for voice traffic is one of the reasons for Skype’s market success where other consumer VoIP products failed; it also makes security types nervous.)

Taking away the carriers’ excuses

So the Internet needs several things:

1. Core routers optimized for today’s two-way, multimedia traffic
2. Domestic broadband similar to what taxpayer dollars already paid the telcos for more than 10 years ago
3. Better protocols and smarter firewalls to squeeze every byte out of what we have

Without these three, carriers (and legislators) will take the easy route — traffic shaping — and claim that the loss of Internet freedom is simply a consequence of heavy traffic.