1: EMBRACE THE SWARM

The Power of Decentralization

The atom is the icon of the 20th century. The atom whirls alone. It is the metaphor for individuality. But the atom is the past. The symbol for the next century is the net. The net has no center, no orbits, no certainty. It is an indefinite web of causes. The net is the archetype displayed to represent all circuits, all intelligence, all interdependence, all things economic, social, or ecological, all communications, all democracy, all families, all large systems, almost all that we find interesting and important. Whereas the atom represents clean simplicity, the net channels messy complexity.

 

The net is our future.

Of all the endeavors we humans are now engaged in, perhaps the grandest of them all is the steady weaving together of our lives, minds, and artifacts into a global scale network. This great work has been going on for decades, but recently our ability to connect has accelerated. Two brand-new technological achievements--the silicon chip and the silicate glass fiber--have rammed together with incredible speed. Like nuclear particles crashing together in a cyclotron, the intersection of these two innovations has unleashed a never-before-seen force: the power of a pervasive net. As this grand net spreads, an animated swarm is reticulating the surface of the planet. We are clothing the globe with a network society.

The dynamic of our society, and particularly our new economy, will increasingly obey the logic of networks. Understanding how networks work will be the key to understanding how the economy works.

 

The dynamic of our society,...

...and particularly our new economy, will increasingly obey the logic of networks. Understanding how networks work will be the key to understanding how the economy works.

Any network has two ingredients: nodes and connections. In the grand network we are now assembling, the size of the nodes is collapsing while the quantity and quality of the connections are exploding. These two physical realms, the collapsing microcosm of silicon and the exploding telecosm of connections, form the matrix through which the new economy of ideas flows.

A single silicon transistor today can only be seen in a microscope. In a few years it will take a microscope to see an entire chip of transistors. As the size of silicon chips shrinks to the microscopic, their costs shrink to the microscopic as well. In 1950 a transistor cost five dollars. Today it costs one hundredth of a cent. In 2003 one transistor will cost a microscopic nanocent. A chip with a billion transistors will eventually cost only a few cents.

What this means is that chips are becoming cheap and tiny enough to slip into every object we make. Eventually, every can of soup will have a chip on its lid. Every light switch will contain a chip. Every book will have a chip embedded in its spine. Every shirt will have at least one chip sewn into its hem. Every item on a grocery shelf will have stuck to it, or embedded within itself, a button of silicon. There are 10 trillion objects manufactured in the world each year and the day will come when each one of them will carry a flake of silicon.

This is not crazy, nor distant. Ten years ago the notion that all doors in a building should contain a computer chip seemed ludicrous, but now there is hardly a hotel door in the U.S. without a blinking, beeping chip in its lock. These microscopic chips will be so cheap we'll throw them away. Thin slices of plastic known as smart cards now hold a throwaway chip smart enough to be your banker. If National Semiconductor gets its way, soon every FedEx package will be stamped with a disposable silicon flake that smartly tracks the contents of the package on its journey. And if an ephemeral envelope can have a chip, so can your chair, each bag of candy, a new coat, a basketball. Soon, all manufactured objects, from sneakers to drill presses to lamp shades to cans of soda, will contain a tiny sliver of embedded thought.

And why not?

 

We are connecting everything to everything.

There is something mysterious that happens when we take large numbers of things that are fairly limited and connect them all together. When we take the dumb chip in each cash register in a store and link them into a swarm, we have something more than dumb. We have real-time buying patterns that can manage inventory. If we take the dumb chips that already regulate the guts of an automobile engine, and let them communicate an engine's performance to the mechanic of a trucking firm, those dumb chips can smartly cut expensive road repairs. (Mercedes Benz recently announced it is planning to embed a web server into its top-of-the-line model cars so technicians can spot service problems remotely.) When connected into a swarm, small thoughts become smart.

 

When we permit any object to transmit a small amount of data...

...and to receive input from its neighborhood, we change an inert object into an animated node.

It is not necessary that each connected object transmit much data. A tiny chip plastered inside a water tank on an Australian ranch transmits only the telegraphic 2-bit message of whether the tank is FULL or NOT. A chip attached to the ear of each steer on the same ranch beams out his location in GPS numbers; nothing more. "I'm here, I'm here" it tells the rancher's log book; nothing more. The chip in the gate at the end of the rancher's road communicates only a single word, reporting when it was last opened: "Tuesday."

It does not take sophisticated infrastructure to transmit these dumb bits. Stationary objects--parts of a building, tools on the factory floor, fixed cameras--are wired together. The nonstationary rest--that is, most manufactured objects--are linked by infrared and radio, creating a wireless web vastly larger than the wired web. The same everyday frequencies that run garage door openers and TV remote controls will be multiplied by the millions to carry the dumb messages of connected objects.

The glory of these connected crumbs is that they don't need to be individually sophisticated. They don't need speech recognition, artificial intelligence, or fancy expert systems. Instead, the network economy relies on the dumb power of bits linked together into a swarm.

Our brains tap into dumb power by clumping dumb neurons into consciousness. The internet banks on dumb power by connecting dumb personal computers. A personal computer is like a single brain neuron in a plastic box. When linked by the telecosm into a neural network, these dumb PC nodes create that fabulous intelligence called the World Wide Web.

Again and again we see the same dynamic at work in other domains: Dumb cells in our body work together in a swarm to produce an incredibly smart immune system, a system so sophisticated we still do not fully comprehend it.

 

Dumb parts, properly connected into a swarm, yield smart results.

A trillion dumb chips connected into a hive mind is the hardware. The software that runs through it is the network economy. A planet covered with hyperlinked chips is shrouded with waves of sensibility. Millions of moisture sensors in the fields of farmers shoot up data, hundreds of weather satellites beam down digitized images, thousands of cash registers spit out bit streams, myriad hospital bedside monitors trickle out signals, millions of web sites tally attention, and tens of millions of vehicles transmit their location code; all of this swirls into the web. That matrix of signals is the net.

The net is not just humans typing at one another on AOL, although that is a part of it and will be as long as seduction and flaming are enjoyable. Rather, the net is the total collective interaction of a trillion objects and living beings, linked together through air and glass.

This is the net that begets the network economy. According to MCI, data traffic on the global phone system will soon overtake voice traffic. The current total volume of voice traffic is 1,000 times that of data, but in three years that ratio will flip. ElectronicCast estimates data traffic--the talk of machines--will be ten times voice traffic by 2005. That means that by 2001 most of the signals zipping around the Earth will be machines talking to machines--file transfers, data streams, and the like. The network economy is already expanding to include new participants: agents, bots, objects, and servers, as well as several billion more humans. We won't wait for AI to make intelligent systems; we'll do it with the swarm power of ubiquitous computing and pervasive connections.

 

The surest way to smartness...

... is through massive dumbness.

The surest way to advance massive connectionism is to exploit decentralized forces--to link the distributed bottom. How do you build a better bridge? Let the parts talk to one another. How do you improve lettuce farming? Let the soil speak to the farmer's tractors. How do you make aircraft safe? Let the airplanes communicate among themselves and pick their own flight paths. This decentralized approach, known as "free flight," is a system the FAA is now trying to institute to increase safety and reduce air-traffic bottlenecks at airports.

Mathematical problems which were once intractable for super-computers have been solved by using a swarm of small PCs. A very complex problem is broken up into tiny parts and distributed throughout the network. Likewise, vast research projects that would tax any one institution can be distributed to an ad hoc network. The Tree of Life is a worldwide taxonomic catalog of all living species on Earth administered on the web. Such a project is beyond the capabilities of one person or group. But a decentralized network can produce the necessary intelligence. Each local expert supplies their own data (on finches, or ferns or jellyfish) to fill in some of the blanks. As Larry Keely of the Doblin Group says, "No one is as smart as everyone."

Any process, even the bulkiest, most physical process, can be tackled by bottom-up swarm thinking. Take, for example, the delivery of wet cement in the less-than-digital economy of rural northern Mexico. Here Cemex (Cementos Mexicanos) runs a ready-mix cement business that is overwhelming its competitors and attracting worldwide interest. It used to be that getting a load of cement delivered on time to a construction site in the Guadalajara region was close to a miracle. Traffic delays, poor roads, contractors who weren't ready when they said they would be, all added up to an on-time delivery rate of less than 35%. In response, cement companies tried to enforce rigid advance reservations, which, when things went wrong (as they always did), only made matters worse ("Sorry, we can't reschedule you until next week.").

Cemex transformed the cement business by promising to deliver concrete faster than pizza. Using extensive networking technology--GPS real-time location signals from every truck, massive telecommunications throughout the company, and full information available to drivers and dispatchers, with the authority to act on it--the company was able to promise that if your load was more than 10 minutes late, you got a 20% discount.

Instead of rigidly trying to schedule everything ahead of time in an environment of chaos, Cemex let the drivers themselves schedule deliveries ad hoc and in real time. The drivers formed a flock of trucks crisscrossing the town. If a contractor called in an order for 12 yards of mix, the available truck closest to the site at that time would make the delivery. Dispatchers would ensure customer creditworthiness and guard against omissions, but the agents in the field had permission and the information they needed to schedule orders on the fly. Result: On-time delivery rates reached about 98%, with less wastage of hardened cement, and much happier customers.

 

Complete surrender to the bottom is not what embracing swarm is about.

Let me retell a story that I told in Out of Control, a book that details the advantages, disadvantages, quirks, and consequences of complex systems governed by swarmlike processes. This story illustrates the power of a swarm, but it has a new ending, which shows how dumb power is not always enough.

In 1990 about 5,000 attendees at a computer graphics conference were asked to operate a computer flight simulator devised by Loren Carpenter. Each participant was connected into a network via a virtual joy stick. Each of the 5,000 copilots could move the plane's up/down, left/right controls as they saw fit, but the equipment was rigged so that the jet responded to the average decisions of the swarm of 5,000 participants. The flight took place in a large auditorium, so there was lateral communication (shouting) among the 5,000 copilots as they attempted to steer the plane. Remarkably, 5,000 novices were able to land a jet with almost no direction or coordination from above. One came away, as I did, convinced of the remarkable power of distributed, decentralized, autonomous, dumb control.

About five years after the first show (this is the update), Carpenter returned to the same conference with an improved set of simulations, better audience input controls, and greater expectations. This time, instead of flying a jet, the challenge was to steer a submarine through a 3D under-sea world to capture some sea monster eggs. The same audience now had more choices, more dimensions, and more controls. The sub could go up/down, forward/back, open claws, close claws, and so on, with far more liberty than the jet had. When the audience first took command of the submarine, nothing happened. Audience members wiggled this control and that, shouted and counter-shouted instructions to one another, but nothing moved. Each person's instructions were being canceled by another person's orders. There was no cohesion. The sub didn't budge.

Finally Loren Carpenter's voice boomed from a loudspeaker in the back of the room. "Why don't you guys go to the right?" he hollered. Click! Instantly the sub zipped of to the right. With emergent coordination the audience adjusted the details of sailing and smoothly set off in search of sea monster eggs.

Loren Carpenter's voice was the voice of leadership. His short message carried only a few bits of information, but that tiniest speck of top-down control was enough to unleash the swarm below. He didn't steer the sub. The audience of 5,000 novice cocaptains did that very complicated maneuvering, magically and mysteriously. All Loren did was unlock the swarm's paralysis with a vision of where to aim. The swarm again figured out how to get there in the same marvelous way that they had figured out how to land the jet five years earlier.

 

Without some element of governance from the top...

bottom-up control will freeze when options are many. Without some element of leadership, the many at the bottom will be paralyzed with choices.

Numerous small things connected together into a network generate tremendous power. But this swarm power will need some kind of minimal governance from the top to maximize its usefulness. Appropriate oversight depends on the network. In a firm, leadership is supervision; in social networks, government; in technical networks, standards and codes.

We have spent centuries obsessed with the role of top-down governance. Its importance remains. But the great excitement of the new economy is that we have only now begun to explore the power of the bottom, where peers holds sway. It is a vast mother lode waiting to be tapped. With the invention of a few distributed systems, such as the internet, we have merely probed the potential of what minimally centralized networks can do.

 

At present, there is far more to be gained...

...by pushing the boundaries of what can be done by the bottom than by focusing on what can be done at the top.

When it comes to control, there is plenty of room at the bottom. What we are discovering is that peer-based networks with millions of parts, minimal oversight, and maximum connection among them can do far more than anyone ever expected. We don't yet know what the limits of decentralization are.

 

The great benefits reaped by the new economy...

...in the coming decades will be due in large part to exploring and exploiting the power of decentralized and autonomous networks.

First we make a chip for every object. Then we connect them. We continue to connect all humans. We enlarge our conversation to include the world, and all its artifacts. We let the network of objects govern itself as much as possible; we add government where needed. In this matrix of connections, we interact and create. This is the net that is our future.

The whole process won't be completed by tomorrow, but the destiny is clear. We are connecting all to all, until we encompass the entire human-made world. And in that embrace is a new power.

 

Move technology to invisibility.

As technology becomes ubiquitous it also becomes invisible. The more chips proliferate, the less we will notice them. The more networking succeeds, the less we'll be aware of it.

In the early 1900s, at the heroic stage of the industrial economy, motors were changing the world. Big, heavy motors ran factories and trains and the gears of automation. If big motors changed work, they were sure to change the home, too. So the 1918 edition of the Sears, Roebuck catalog featured the Home Motor--a five-pound electrical beast that would "lighten the burden of the home." This single Home Motor would supply all the power needs of a modern family. Also for sale were plug-ins that attached to the central Home Motor: an egg beater device, a fan, a mixer, a grinder, a buffer. Any job that needed doing, the handy Home Motor could do. Marc Weiser, a scientist at Xerox, points out that the electric motor succeeded so well that it became invisible. Eighty years later nobody owns a Home Motor. We have instead dozens of micro-motors everywhere. They are so small, so embedded, and so common that we are unconscious of their presence. We would have a hard time just listing all the motors whirring in our homes today (fans, clocks, water pumps, video players, watches, etc.). We know the industrial revolution succeeded because we can no longer see its soldiers, the motors.

Computer technology is undergoing the same disappearance. If the information revolution succeeds, the standalone desktop computer will eventually vanish. Its chips, its lines of connection, even its visual interfaces will submerge into our environment until we are no longer conscious of their presence (except when they fail). As the network age matures, we'll know that chips and glass fibers have succeeded only when we forget them. Since the measure of a technology's success is how invisible it becomes, the best long-term strategy is to develop products and services that can be ignored.

 

If it is not animated, animate it.

Just as the technology of writing now covers almost everything we make (not just paper), so too the technologies of interaction will soon cover all that we make (not just computers). No artifact will escape the jelly bean chip; everything can be animated. Yet even before chips reach the penny price, objects can be integrated into a system as if they are animated. Imagine you had a million disposable chips. What would you do with them? It's a good bet that half of the value of those chips could be captured now, with existing technology, by creating a distributed swarmlike intelligence using such dumb power.

 

If it is not connected, connect it.

As a first step, every employee of an institution should have intimate, easy, continuous access to the institution's medium of choice--email, voicemail, radio, whatever. The benefits of communication often don't kick in until ubiquity is approached; aim for ubiquity. Every step that promotes cheap, rampant, and universal connection is a step in the right direction.

 

Distribute knowledge.

Use the minimal amount of data to keep all parts of a system aware of one another. If you operate a parts warehouse, for example, your system needs to be knowledgeable of each part's location every minute. That's done by barcoding everything. But it needs to go further. Those parts need to be aware of what the system knows. The location of parts in a warehouse should shift depending on how well they sell, what kind of backlog a vendor forecasts, how their substitutes are selling. The fastest-moving items (which will be a dynamic list) may want to be positioned for easier picking and shipping. The items move in response to the outside--if there is a system to spread the info.

Get machines to talk to one another directly. Information should flow laterally and not just into a center, but out and between as well. The question to ask is, "How much do our products/services know about our business?" How much current knowledge flows back into the edges? How well do we inform the perimeter, because the perimeter is the center of action.

 

If you are not in real time, you're dead.

Swarms need real-time communication. Living systems don't have the luxury of waiting overnight to process an incoming signal. If they had to sleep on it, they could die in their sleep. With few exceptions, nature reacts in real time. With few exceptions, business must increasingly react in real time. High transaction costs once prohibited the instantaneous completion of thousands of tiny transactions; they were piled up instead and processed in cost-effective batches. But no longer. Why should a phone company get paid only once a month when you use the phone every day? Instead it will eventually bill for every call as the call happens, in real time. The flow of crackers off grocery shelves will be known by the cracker factory in real time. The weather in California will be instantly felt in the assembly lines of Ohio. Of course, not all information should flow everywhere; only the meaningful should be transmitted. But in the network economy only signals in real time (or close to it) are truly meaningful. Examine the speed of knowledge in your system. How can it be brought closer to real time? If this requires the cooperation of subcontractors, distant partners, and far-flung customers, so much the better.

 

Count on more being different.

A handful of sand grains will never form an avalanche no matter how hard one tries to do it. Indeed one could study a single grain of sand for a hundred years and never conclude that sand can avalanche. To form avalanches you need millions of grains. In systems, more is different. A network with a million nodes acts significantly different from one with hundreds. The two networks are like separate species--a whale and an ant, or perhaps more accurately, a hive and an ant. Twenty million steel hammers swinging in unison is still 20 million steel hammers. But 20 million computers in a swarm is much, much more than 20 million individual computers.

Do what you can to make "more." In a network the chicken-and-egg problem can hinder growth at first--there's no audience because there is no content, and there is no content because there is no audience. Thus, the first efforts at connecting everything to everything sometimes yield thin fruit. At first, smart cards look no different from credit cards--just more inconvenient. But more is different; 20 million smart cards is a vastly different beast than 20 million credit cards.

It's the small things that change the most in value as they become "more." A tiny capsule that beeps and displays a number, multiplied by millions: the pager system. What if all the Gameboys or Playstations in the world could talk to one another? What if all the residential electric meters in a city were connected together into a large swarm? If all the outdoor thermometers were connected, we would have a picture of our climate a thousand times better than we have ever had before.

The ants have shown us that there is almost nothing so small in the world that it can't be made larger by embedding a bit of interaction in many copies of it, and then connecting them all together.

The game in the network economy will be to find the overlooked small and figure out the best way to have them embrace the swarm.

 

Today the world is populated by 200 million computers.

Andy Grove of Intel happily estimates that we'll see 500 million computers by 2002. Yet for every expensive chip put into a beige computer box, there are now 30 other cheap processors put into everyday things. The number of noncomputer chips already pulsating in the world is 6 billion--one chip for every human on Earth.


Network organizations experience small gains while their network is being seeded. Once the network is established, explosive growth follows with relatively little additional genius.

You already have a non-PC chip embedded in your car and stereo and rice cooker and phone. These chips are dumb chips, with limited ambitions. A chip in your car's brakes doesn't have to do floating-point math, spreadsheets, or video processing; it only needs to brake like a bulldog.

Because they have limited functions and can be produced in great quantity, these dumb chips are ultracheap to make. One industry observer calculated that an embedded processor chip costs less to manufacture than a ball bearing. Since they can be stamped out as fast and cheap as candy gumdrops, these chips are known in the trade as "jelly beans." Dumb, cheap jelly bean chips are invading the world far faster than PCs did.

This is not surprising. You can only use one or two personal computers at a time, but the number of other objects in your life is almost unlimited. First, we'll put jelly bean chips into high-tech appliances, then later into all tools, and then eventually into all objects. If current rates continue there'll be some 10 billion tiny grains of silicon chips embedded into our environment by 2005.

Putting a dot of intelligence into every object we make at first gives us a billion dimwitted artifacts. But we are also, at the same time, connecting these billion nodes, one by one.

 

Similar thinking has been used...

...in a GM paint plant in Fort Wayne, Indiana. The wonderful choice of colors that customers now enjoy on new vehicles was playing havoc on the paint line. When one car after another is sprayed black, everything is easy. But when one car is red and the next white, the painting process is slowed down as painting equipment is cleansed of one color to make it ready for the next. (The clean-out procedure also wastes paint left in the paint lines.) Why not gang up all the white cars and do them together? Because ganging up slows the line. A car has to be built and completed as it is ordered, as quickly as possible. The solution embraces the swarm.

In the paint factory each robot painter (basically a dimwitted painting arm) is empowered to bid on a paint job. If it is currently painting red and a car slated to be red is coming down the assembly line, it says, "Let me do it," and it beckons the car to its paint station. The robots schedule their own work. They have very tiny brainlets, connected to a server. No central brain coordinates; the schedule comes from the swarm of mini-brains. The result: GM saves $1.5 million a year. The equipment requires less paint (due to less cleaning between cars), and keeps the line moving faster.

Railways are now employing swarm technology. Centralized traffic control doesn't work when the traffic becomes very complex and time cycles are shortened. The Japanese use a bottom-up swarm model to schedule their famous bullet express trains, which boast incredible punctuality. Switching is done locally and autonomously as if the trains were a swarm with one mind. Railway owners in Houston are hoping to get a swarm model running for their rail yards. With their current centrally controlled system, the switching yards are so clogged that there is a permanent train of freight cars circling the greater Houston area as a buffer. It's like a mobile parking lot. When there's an opening in the yard, cars are pulled out of the holding pattern train. But with a system based on the swarm model, local lines can autonomously switch themselves, using minimal intelligence onboard. Such a self-regulating and self-optimizing system would reduce delays.

That's how the internet handles its amazing loads of traffic. Every email message is broken into bits, with each bit addressed in an envelope, and then all the fragmentary envelopes are sent into a global web of pathways. Each envelope seeks the quickest route it can find instant by instant. The email message becomes a swarm of bits that are reassembled at the other end into a unified message. If the message is re-sent to the same destination, the second time it may go by a wholly different route. Often the paths are inefficient. Your email may go to Timbuktu and back on its way across town. A centralized switching system would never direct messages in such a wasteful manner. But the inefficiencies of individual parts is overcome by the incredible reliability of the system as a whole.

The internet model has many lessons for the new economy but perhaps the most important is its embrace of dumb swarm power. The aim of swarm power is superior performance in a turbulent environment. When things happen fast and furious, they tend to route around central control. By interlinking many simple parts into a loose confederation, control devolves from the center to the lowest or outermost points, which collectively keep things on course.

A successful system, though, requires more than simply relinquishing control completely to the networked mob.

 
 

Archives - This site operates under a Creative Commons License.

This is a blog version of a book of mine first published in 1998. I am re-issuing it (two posts per week) unaltered on its 10th anniversary. Comments welcomed. More details here.
-- KK