VMware - A new kind of simple machine

What they taught us in grade school about "simple machines" needs updating, as shown by the wildly successful launch of a new company, VMware. (Priced at $29 it closed the first day at $51 and is now $117 a share, two months later, and growing.)

Why is that so successful? What energy source is that tapping into? Let's look at that.

You remember "simple machines", of course, right? (No, not the rock group.) Our class "went over" this standard list, I think:

• the lever
• the wheel and axle
• the inclined plane
• the wedge
• the screw
• the pulley

Actually, our textbook had this great picture of a steam shovel with transparent overlays that you could flip one by one and peel down the layers and see all the parts inside. I liked that the best and spent more time playing with that than listening, I confess.

So, if you're hyper-active, interrupt yourself and flip to "how stuff works" and see how those huge T-shaped construction cranes get the cab and T-part up on top. Hint - it's not with a bigger crane -- they build themselves up by adding pieces just below the top.

But, back to "simple machines".

First - why did they bother mentioning that? Why are these in the school curriculum at all? A hundred years ago, most people had to solve problems that involved moving heavy objects around the farm, and this was a list of helpful answers to how to do that.

These days, I think most grade-school students in the urban USA have never seen a pulley, "wheels" means a car or bike, "screw" doesn't refer to a machine, and, never having split wood, they don't recognize a "wedge" or why that might be useful to them.

It's a shame, because the whole point was supposed to be a set of really helpful ideas you could go home and use right away to make your life easier, and that got lost somewhere between prep for the SAT and classes that don't seem related to real life.

The whole concept of rigging up something outside you to amplify your own efforts so you could move things you thought were beyond you seems to have been lost.

So, these are now almost unknown, along with other basic skills everyone once knew, like how to make a campfire from scratch in the woods, even with a box of matches, or why washing hands is important so you and your friends don't get sick or die from some disease.

Besides, there are other "simple machines" that I find useful to know too. Siphons have come in handy.

Pushing "sideways" on a taut rope can move heavy objects a few inches at a time, which is often all you need, etc. (Yes, Martha, you CAN push on a rope.)
So, if you don't have a mental toolbox of possibly useful tools, you should consider starting one.
If you have one, here's another one - a "virtual machine."

In the immediate sense, what VM sells is software that lets you make something that looks from the outside just like a file to one computer (Mac, PC, or Linux), but that seems from the inside as if it is an entire computer of one of those types. The result is that you can effectively run a PC on your Mac, or Linux on your PC, etc. That's helpful and cool by itself. In fact, you could have many different virtual computers running at the same time inside one real piece of hardware.

This is an improvement over IBM's mainframe "VM/370" operating system introduced back in 1972, that let each user have a "virtual machine" of their own inside the IBM mainframe, but they were all just clones of the mainframe. (Today, you could have a VM inside your VM, etc.) That was very helpful because it totally insulated users from each other, so that if one person totally messed up their program, it wouldn't take down the whole mainframe, just their own virtual machine, and everyone else would be unaffected.

Some of my friends found out that

if you put a virtual PC inside your PC, it actually runs a lot faster than the real PC.

The reason is that the real PC keeps doing really stupid things like checking many times a second to see if you just put in a CD, and these really slow it down. The virtual PC only has to check a virtual CD drive, which is much faster because it's all in memory, not hardware. So VMware is helpful in making an old machine run faster doing the same thing.

But there's a different side to it, which is the other way around. VMware also sells "virtual servers", which let you take multiple real computers and connect them and create a single, seamless, virtual mega-computer out of the pieces. In this case, you are making "unity" out of "multiplicity" or even "diversity". Or, you can make MANY different virtual servers, of any type, out of many different pieces of hardware you have kicking around.

So, hardware and software become completely decoupled.

The great thing is that if one of the actual physical computers goes down, it doesn't break anything, it just slows everything down a little. Conversely, if your virtual server isn't fast enough, or doesn't have enough RAM, just scrounge up some more discarded PC's and hook them in and make your single virtual server run faster.

This is a machine solution to the class of problem I spend a lot of time on, namely, how to get committees to work. How can we co-operate so that adding another person always improves our thinking power, so we can tackle hard problems by just adding enough people until the problem budges.

This is the new kind of "simple machine" that we need to build, socially.

Since everyone seems to have forgotten the whole point of the "machine" concept, which was to make our chores easier, people aren't even looking for some new "machine" that will make our current jobs easier. (Although some thought that "computers" would help.)

The "magic" is the same as with a simple machine, like a big pry bar. Instead of pushing on the boulder with our hands, we go get some tool instead, and make a two-stage solution, where we push on the tool, the tool amplifies our push, and the tool pushes on the boulder, moving it.

We need to get back to that concept, socially. We need to stop trying to "push on" social problems one person at a time, looking for a "stronger" person if one person can't move the boulder - and think about ways to get a middle step -- building a virtual strong person out of many actual people, where the virtual strong person can lift the boulder easily.
That would say we need to put energy and resources and effort into doing for people what VMware did for computers -- solving the problems of how to make them work together as if they were one, seamlessly.

Like VMware, when we finally get it cracked, it will be wildly, amazingly helpful, successful, and popular and we'll be able to do things easily that we had given up on doing before.

It's a cool problem, and we really need more people working on it.

(Crowbar photo from flickr by jwhairybob)
(Juscelino Kubitschek Bridge in Brazil by by chris.diewald )

Decentralized sense-making in a cluttered world

If central planning isn't helpful for sense-making in a complex and cluttered world, what is?

The world of "image-processing" in computing has come up with some techniques that seem interesting models for action. I want to describe one that I've used in the past. You don't need any math for this. I tried to make it easy to follow.

The problem we had involved finding the edges of a brain tumor on a 3-dimensional Magnetic Resonance Imaging image. This is actually a set of "slices", stacked like a deck of cards, across a section of the brain.

Each slice looks something like this picture, which is a cross-section image I pulled off the web from the NIH Image database of public sample images. That's a vertical "slice" through someone's head, facing to the left. (Note - the person isn't actually sliced or injured - the computer just makes it look that way!)

Maybe if you think of baking an orange into a loaf of bread, and then running it through a bread slicer -- you get the image of a stack of slices, starting with all bread and no orange, then really small circles of orange, then slices with larger circles, then smaller again, and finally bread slices with no orange at all. Our job is to find the orange in the pictures of the slices of bread and reconstruct what it looks like in 3-D.

If there is or might be a tumor, it's important to find the edges as accurately as possible, based on these kinds of images. That's not as easy as you might think, because when you zoom to the high magnification, the images are actually pretty blurry and "noisy" and hard to read as to where, exactly, an "edge" is.

Here's some structure in a brain, probably not a tumor, for illustration. If you click on the image, you can zoom it up and see some sort of black dot with a white border fairly easily in the upper right, "Slice #19". But if you look at the previous slice, the next "card in the deck of cards",
"Slice #18", the edges of this are less distinct and this slice of the orange is smaller.


Similarly around slice 20, maybe we can still be fairly sure we "see" the edges of the white structure, but by slice 21 it's not clear what's that structure, and what's just normal tissue.

And, we're using the magic of human eyes. We want some way the computer can do a better job than people at finding the edges of a structure, once a trained radiologist points it out. (This was all done over a decade ago and I suspect they have way better tools today, by the way.)

Anyway, let me describe how the edges can be found. Look at it on one slice first. Imagine surrounding the tumor with a line of people attached by stretchable elastic cords or "slinkies" or springs. In this picture I just drew a red dot instead of person, but you get the idea. Pretend that's the view from above of many people with red hats connected to each other with adjustable bungee cords.

Then, you ask each person, when he gets to a place where he looks down and sees dark changing to light rapidly, that might be the edge of the tumor, so he should dig in his heels and try to stay there. But, at the same time, you start making the springs stronger, pulling people towards each other.

As you do that, initially with the springs fairly stretchy and loose, the circle starts being pulled smaller and smaller, like a drawstring tightening on a purse. When each person gets to what seems like it might be an edge, they try to drag their heels and stop moving, independently.

After this has gone on a while, you may end up with something like this:


You can see that most of the people have found the edge of the tumor and dug in their heels there. But people #1 and #2 found a bright edge that is probably just "noise". And the people numbered "3" have found something that it's hard to tell if that's tumor or noise.

How should they decide?

In this technique, if you just start tightening the springs, at some point the collective pulling force of the majority will break #1 and #2 loose from the feature they are snagged on, and they'll snap into place around the tumor.

Based on just this slice, the people labelled 3 may not move, because maybe that's actually an edge. (Tumors don't have to be round - they can be irregular.)

Well, how do the people at #3 decide? Here's the trick. While all this is going on on this slice,
the same thing is going on on all the other slices, and springs connect the dots / people across the slices. In other words, we actually have a sphere of dots / people, connected by springs, kind of like an over-inflated balloon, and we let it slowly deflate in three dimensions at once,
around the feature in all the slices.

In other words, there is not enough information in the vicinity of any one person to be able to sort out image from noise with certainty. But, most of them are nearly right. We just don't know which ones that is. So, within each slice (or region) the people consult with each other, while at the same time they are consulting across regions as well, with a mix of believing their own eyes, and enough humility to know, at some point, to let go and go with the crowd.

This seems like a very simple plan, and no computer is required. It turns out to be a very powerful technique ("algorithm") that does a remarkably good job at sorting out "noise" from "signal" in 3-dimensions, with only trivial programming required.

Each person / dot simply has to pay attention to what it sees ("independent investigation"), but balance that with consulting with neighboring people and at some point yielding to peer pressure and moving into line. If the balance of these two competing forces is right, the overall network turns our to be a very powerful analog computer that can solve a problem we have trouble even defining well.

No single person ever needs to "see" everything or see "the big picture" - he just needs to see his neighbors, compare notes, argue for his position, and, if it seems warranted, yield to the majority. If enough different dots do this, coming in from enough different directions at once ("diversity"), and remain independent and yet consulting ("unity in diversity") the algorithm works. The powerful solution "emerges" from each person's behavior.

In image processing this is called an "adaptive contour" technique. It is part of the larger class of techniques called "swarm computing" that is becoming increasingly popular as "the power of crowds" is increasingly being appreciated.

An area this could be used is in any sort of boundary measurement, or in aligning fragments of images to make a coherent overall picture. Examples of these, and my US Patent 5613013 in image alignment using effectively a swarm technique, are described on my web site here.

Wade