The developmental biology of infrastructure

Biologically-inspired technologies have been a research topic for years, autonomic computing and epidemic protocols being a couple of examples. These techniques often have very attractive properties to go with their fun names: scaling, self-healing, resilience, blah, blah, blah. I think about infrastructure and have taken some different things from biology.

The unfortunately named Universal Genetic Code (UGC) and the other, extremely similar variants found in mitochondrial DNA and other small organisms, evolved. We're used to DNA being the stuff of evolution (RNA fans save your flame mails), so the code itself evolving is a bit of a brain bender. Think of it as meta-evolution.

The end of evolution for the UGC happened billions of years ago. Experts call this "early fixation.". The code itself is a marvel of efficiency. It resists exactly the sort of errors to which the transcription machinery is prone. When a transcription error gets through, chances are the erroneous amino acid will have properties similar enough to the correct one that there is little functional difference in the geometry of the final protein. We can appreciate just how spectacular it is because we can almost completely quantify its environment, something simply out of the question with complete organisms. Physics, chemistry, geometry.

The high school biology textbook version of how genes evolve goes something like this: mutation and crossover. Mostly crossover. Read chapters 5 through 7 for Monday.

Reality, as expected, is far richer. Enter Systems Biology and Evolutionary Developmental Biology (evodevo). I can't do justice to the topics in a blog post, but the important idea is this: genes change very slowly, while the gene regulation networks that control their expression are incredibly complex and evolve rapidly.

The bottom of the stack is ridiculously efficient, defined almost entirely by physical constraints, and completely static. Up a level we see simple, modular structures that can be assembled in a variety of ways, and that change very slowly. Finally, at the top, we see the incredible complexity and the roiling expression of evolutionary change.

UGC -> genes -> regulation networks
hardware infrastructure -> software infrastructure -> applications

A bit more about demand locality

In my last post I focused on geography and culture/language as critical factors for demand locality. However, there are many other ways that demand locality manifests. One of the more subtle is organizational demand locality.

Organizational demand locality is the relative efficiency of carrying traffic entirely within the infrastructure of a single organization versus having to hand-off traffic to other organizations. Wireless carriers offering unlimited "on-net" airtime are exploiting organizational demand locality. Internet service providers spend a surprising amount of resources worrying about peering with each other (or refusing to).

Once again, the geographic/cultural demand locality plays a key role. In places with very high demand locality (Korea, Japan, etc.), the need for numerous, large links between organizations is minimized or avoided entirely. In places with low demand locality (again, I mean the US), peering becomes a constant source of friction with tier-1 ISPs requiring peering at multiple points across the country or not at all. Merging ISPs to raise organizational demand locality only gets you so far.

The applications have to change, not the infrastructure.

Global broadband

I received an email this morning with a link to this article. There's nothing new or especially interesting in it, but it happened to intersect with some other, related ideas I've been pondering. There is a very wide gap between reality and perception of broadband penetration (the handful of comments under the linked article gives some flavor).

The reason for the disparity is, in part, found in this map. More evidence is here. The story these maps tell is of the business challenges to broadband in the US that are far less severe in other countries, if they exist at all: the physical geography, population distribution, and national or language barriers. The central concept here is demand locality, or how closely geographic locality matches traffic locality. More demand locality means easier deployment of faster and faster services.

Most countries with similar uniformity of language and culture are much smaller than the US and have many fewer population centers. Physical proximity and language/geographic barriers in such countries drive demand locality up and drive infrastructure costs down.

In the US we have dozens of high-density population centers spread across thousands of miles. Despite its wide variety of cultures and languages, the US is dominated by native English speakers who speak no other languages. This implies very high connectivity demands between widely dispersed points in the country. Demand locality is low, so infrastructure costs are high. Not surprisingly, the US has an average broadband speed of 1.9Mb/s.

South Korea is a great example of the rest of the world because it is geographically small, has one major population center, and there are no other Korean speaking countries with network infrastructure. Most sites of interest to South Koreans are inside South Korea, so most traffic in South Korea stays in South Korea. Delivering broadband to most of the population of South Korea means delivering it to Seoul and not worrying much about intra-country long-haul or inter-country connectivity. The result: average broadband speed in South Korea of 45Mb/s. Japan has similar advantages and averages 61Mb/s.

This seems to leave the US stuck in the broadband doldrums. Returning to the article at the top, no broadband initiative is going to change the demand locality. Changing the demand locality requires getting data and applications closer to users. That means building a lot of data centers and properly architecting applications to run in a lot of data centers spread far apart from each other. At that point, individual metro areas can be upgraded with extremely high-speed broadband without needing nearly as much long-haul capacity or worries about transcontinental latency.