If you ever learn about optimization, the second technique they teach you after “hill climbing” is something called “simulated annealing“. Forget trying to work through Wikipedia’s definition of it, here’s how I mentally visualize it:

Imagine you your job was to find the highest point on a random Frank Gehry designed building, say the Hotel Marques de Riscal. The problem is, you’re blind so you can only determine the height of one point at a time.

Frank Gehry Hotel

Hotel Marque de Riscal

One way to do it would be to take the entire outside shell of the structure and turn it upside down and then drop a ball bearing in it. When the ball bearing stops moving, you declare that point to be probably the lowest point.

Upside down hotel

Upside Down Hotel Marque de Riscal

In the field of optimization, this is called hill climbing and it sure is fast but the problem with it is obvious: the ball bearing will likely drop into some rut on the side of the building before reaching the bottom (aka: a local minima).

Simulated annealing is like adding a giant paint shaker and imagining the ball bearing getting heavier over time. At first, the ball bearing is light as a feather and every random vibration is going to make it bounce around, sometimes to a lower point but also sometimes to a higher point. Over time, as the ball bearing gets heavier, it becomes progressively harder to jostle out of it’s rut but when it does move, it moves to a lower rut some distance away. Finally, the ball bearing becomes so heavy it’s virtually impossible to budge, at which point, you declare that position to be probably the lowest point. Simulated annealing can still get trapped in local minima but it usually does a hell of a lot better than hill climbing.

Now, imagine that the ball bearing is society and it’s searching for the solution to a problem like, say, attaching sheets of paper together. If you’re living in any time since the 1930′s, it’s overwhelmingly likely that you’ll be reaching for a “Gem” style paperclip.

A Gem style paperclip

What many people don’t realize is that the Gem paperclip wasn’t the only type of paperclip invented. In fact, the 1909 Websters Dictionary entry on paperclips referred to the “Konaclip” version which was considered the prototypical paperclip of the time.

A Konaclip style paperclip

From 1864 when the first paperclip patent was granted, till 1930 when the gem became dominant, there were literally dozens of designs for paperclips (which, thankfully, has been documented on the Paperclip section of the Early Office Museum). But since the 1930′s, technological progress in the paperclip arena has ground to a halt with the entirety of society standardizing on the Gem and every other style dying in obscurity (For a much more detailed look into the paperclip’s fascinating history, I refer you to the book The Evolution of Useful Things).

To explain this history, I refer back to the simulated annealing analogy. In the early days of paperclipdom, society wasn’t particularly attached to any one design so there was very little momentum in the system. Even a tiny vibration was enough to make another design viable. However, as time went on, certain styles of paperclips started to have legacy effects and it became a race between a few, select alternatives, most notably the Konaclip & the Gem. Finally, society had settled so firmly upon the Gem that it it would take the most extraordinary effort to have any other design supplant it which is why it remains the paperclip of choice to this day. In Simulated Annealing terms, the entropy was now so low that it became practically impossible to escape from a local minima.

Indeed, one of the characteristics of simulated annealing is that it goes through 3 distinct phases. A period of fluid, diverse shifting, a long period of stagnation punctuated by occasional radical shifts and then finally stability from which only incremental improvement is possible.

Looking at the pattern of historical developments of other technologies, it’s possible to spot these same shifts in between phases.

Before 1973, there was a diverse ecosystem of physical interaction paradigms for desktop computing. The most notable examples from that era were Ivar Sutherland’s Sketchpad using light pen interaction and Alan Kay’s tablet style Dynabook.

Sketchpad circa 1962

Sketchpad circa 1963

Dynabook circa 1963

Dynabook circa 1972

Then, in 1973, the Xerox Alto was released and virtually halted the progress of computer interfaces from that point forward. Nearly every desktop computer today is a recognizable descendant of the Xerox Alto.

Xerox Alto circa 1973

The list of genuine innovations in physical UI that have been adopted since the Alto are as laughably few in number as they are trivial in scope. In rough order of importance: Speakers, microphones, webcams, the mouse wheel, wireless networking, higher resolution screens, a numpad, flatter screens & the Windows/Command button (this is not a sampling, it’s the complete list).

The list of potential innovations are as staggering as they are futile: Pen computing, Tabletop computing, Tablet computing, Augmented & Virtual Reality, Touch based interaction, Multitouch, MultiMouse, Bimanual Interaction, Single Display Groupware, 3D displays and the list goes on and on. In the last 37 years, billions of dollars have been poured into these alternative technologies in a vain attempt to supplant the Xerox Alto as the dominant paradigm with pretty much nothing to show from all that work except a bunch of pretty pilot projects.

And with each passing year, it’s become more and more difficult to foster a viable alternative paradigm because more and more gets invested into keeping Alto style computing firmly entrenched. Software is designed for a keyboard+mouse+screen, people have invested time learning the intricacies of a WIMP OS and thousands of companies have a vested interest in keeping the system entrenched.

The same pattern can be found in keyboard layouts with QWERTY, programming languages with C, web standards with HTML+CSS+JS, Office Productivity with Microsoft Office, Email with Eudora and so forth. In every instance, a period of rapid innovation was brought abruptly to a close with a dominant technology and, from that point forward, genuine changes in the status quo happen at best every decade and only after extraordinary effort *.

Almost every discussion of a new innovation focuses on the details of the innovation almost exclusively without considering the broader social context. Such discussion manages to miss the vital point that adoption of an innovation depends only a tiny bit on the actual innovation and almost completely on the current progress of society. Once a technology has reached a certain maturity point, whatever local minima that society has currently reached will establish itself be the dominant technology until the end of time regardless of what technological progress has occurred or has yet to occur. Like a game of musical chairs, it’s all about when the music stops. If you’ve managed to grab a seat at the table, then you’ll stay there forever, if you missed your spot, then it’s going to take extraordinary effort to regain it, no matter how much of an improvement you are over the status quo.

* This same model also explains why European & Japanese mobile & broadband technology is so far in advance of the US even thought the US were the pioneers of both technologies. Because the US started early, it’s industry matured around an earlier technological & social standard which caused it to fall behind the less mature, more technologically limber peers.

a numpad,
  • http://gumption.typepad.com Joe McCarthy

    This is one of the most evocative and approachable visualizations and descriptions of simulated annealing I've encountered.

    Your discussion of the costs of [in]vested interests reminded me of an article by John Gourville in the June 2006 issue of HBR, Eager Sellers and Stony Buyers: Understanding the Psychology of New-Product Adoption, in which he describes how “loss aversion”, the “endowment effect” and “status-quo bias” all contribute to our reticence to try (or buy) new things. The upshot: most people have to perceive a potential improvement factor of 3:1 before they are willing to abandon the old for the new.

  • Hang

    Hi Joe,

    Thanks for the compliment, I was always dissatisfied with descriptions of Simulated Annealing as they made so complicated what was, to me, a blindingly simple concept. I'm glad I managed to convey that simplicity across.

    I don't know if I spoke specifically to you about this but, at CHI last year, I made two specific predictions:

    1. in 10 years time, 90%+ of all phones in use in the developed world would look recognizably like an iPhone (in the same way 90%+ of all desktop computers look recognizably like an Alto)
    2. Apple was too smart to be working on a tablet.

    For the last year, I'd been itching to put down on paper what I had been saying out loud but I hadn't been able to find the exact right framing that would make such an argument clear to a mass audience.

    This post is laying the foundation for what is to become my argument for why the iPad could be the largest mistake that Apple ever made.

  • http://astatespacetraveler.com/ Carter

    Hang, this is weird. It seems like our blog posts overlapped again. This is an awesome post. But what's interesting is that about a month ago I wrote my first blog post which argues that the path to true happiness lies in simulated annealing. I also include an explanation of simulated annealing and how it can be applied to achieving happiness. Would be interested to hear your thoughts.

    http://www.astatespacetraveler.com/a-mathematic

  • Mattisong

    I wonder if there is an applicable lesson in trying to shift to renewable energy sources. We seem pretty firmly planted in a local minima of fossil fuel. Does the science reveal any sociological schema that assist in facilitating these technological leaps that transcend our acquiescence?

  • http://profiles.google.com/angeloeno Angelo Eno

    I wrote my first blog post which argues that the path to true happiness
    lies in simulated annealing. I also include an explanation of simulated
    annealing and how it can be applied to achieving happiness. Would be
    interested to hear your thoughts.

    led christmas lights

  • Anonymous

    Our organization is mostly Classic Short UGG Boots running ugg boots sale.We have a assortment of ugg traditional boots.They have several kings of types.For illustration,ugg ugg boots sale 5825,ugg5803 and ugg 5815.Our Classic Suede UGG Boots ugg boots store is the most well-known web store on the internet.

  • Anonymous

    Our firm is mainly running ugg boots sale.We have a variety of ugg classic boots.They have a number of Kids Infants Erin UGG Boots kings of models.For illustration,ugg ugg Delaine Style UGG Boots boots sale 5825,ugg5803 and ugg 5815.Our ugg boots store is the most common internet shop on the net.

  • Sunlimei1989

    In essence, these Australian sheepskin boots are soothing and made to pamper a lady’s feet.It’s no wonder then that most women have since determined on giving these tall boots, made especially for them, a try. And it’s a good thing, because it turns out that most women have for some time now been looking for boots that would be comfortable and could endure any weather conditions and would invariably cool their feet.
    Previously distrustful about the boots, most eventually realized that Ugg boots for women were stiff competition for big brand names.These super fashionable tall boots for women are available in a range of tinges and shapes. If you’re for a boot that will not swell your feet up or act as a steamer as you go about your daily tasks, then you may want to put away everything you’ve ever been told about flat boots; your search ends here.
    Ugg boots for women come in both leather and suede, with the insole of the boot manufactured from pelt, so that the boots feel soothing to your feet.Designed from pure Australian sheepskin, the Australians have made boot making a fine science.
    Today, those in the know can be found walking the malls and other markets in these very comfortable shoes. These Ugg boots for women are highly luxurious and exceedingly comfortable. Incidentally, these quality boots are adorned with accentuated shapes that would fit a lady’s foot like a mitt.
    The heels are extravagant and can be suited for walking to the subway as well as anywhere your heart desires. For women, these Australian sheepskin discount ugg boots for women will keep you warm under any conditions as they come with a sheepskin pelt with a fold-down cuff.The majority of women wearing these boots have found that, as the weather got colder, their feet became warmer, causing these ladies to neglect their other footwear!
    Where discount ugg bootsfor women come to mind, for the most part people tend to think along the lines of leather boots or stilettos with sharp pointed heels! Of late however, this perception seems to be waning as people begin appreciating Ugg boots all the more!In the event the past 10 years have passed you by and you’ve never heard of Ugg boots, I don’t blame you, it’s not your fault! For years now, Ugg boots for women such as these tall boots, have been a bastion of the well to do of society.
    This Australian footwear became popular locally through the creative imagination of an Aussie surfer somewhere in California where he stupefied the surfing community by wearing this apparently very warm looking footwear in the Californian heat!Upon closer analysis however, it was discovered that discount ugg boots, for women and men alike, are actually very cool on your feet – warm in winter and cool in summer, as the proverb goes.