Decentralized sense-making in a cluttered world

If central planning isn't helpful for sense-making in a complex and cluttered world, what is?

The world of "image-processing" in computing has come up with some techniques that seem interesting models for action. I want to describe one that I've used in the past. You don't need any math for this. I tried to make it easy to follow.

The problem we had involved finding the edges of a brain tumor on a 3-dimensional Magnetic Resonance Imaging image. This is actually a set of "slices", stacked like a deck of cards, across a section of the brain.

Each slice looks something like this picture, which is a cross-section image I pulled off the web from the NIH Image database of public sample images. That's a vertical "slice" through someone's head, facing to the left. (Note - the person isn't actually sliced or injured - the computer just makes it look that way!)

Maybe if you think of baking an orange into a loaf of bread, and then running it through a bread slicer -- you get the image of a stack of slices, starting with all bread and no orange, then really small circles of orange, then slices with larger circles, then smaller again, and finally bread slices with no orange at all. Our job is to find the orange in the pictures of the slices of bread and reconstruct what it looks like in 3-D.

If there is or might be a tumor, it's important to find the edges as accurately as possible, based on these kinds of images. That's not as easy as you might think, because when you zoom to the high magnification, the images are actually pretty blurry and "noisy" and hard to read as to where, exactly, an "edge" is.

Here's some structure in a brain, probably not a tumor, for illustration. If you click on the image, you can zoom it up and see some sort of black dot with a white border fairly easily in the upper right, "Slice #19". But if you look at the previous slice, the next "card in the deck of cards",
"Slice #18", the edges of this are less distinct and this slice of the orange is smaller.


Similarly around slice 20, maybe we can still be fairly sure we "see" the edges of the white structure, but by slice 21 it's not clear what's that structure, and what's just normal tissue.

And, we're using the magic of human eyes. We want some way the computer can do a better job than people at finding the edges of a structure, once a trained radiologist points it out. (This was all done over a decade ago and I suspect they have way better tools today, by the way.)

Anyway, let me describe how the edges can be found. Look at it on one slice first. Imagine surrounding the tumor with a line of people attached by stretchable elastic cords or "slinkies" or springs. In this picture I just drew a red dot instead of person, but you get the idea. Pretend that's the view from above of many people with red hats connected to each other with adjustable bungee cords.

Then, you ask each person, when he gets to a place where he looks down and sees dark changing to light rapidly, that might be the edge of the tumor, so he should dig in his heels and try to stay there. But, at the same time, you start making the springs stronger, pulling people towards each other.

As you do that, initially with the springs fairly stretchy and loose, the circle starts being pulled smaller and smaller, like a drawstring tightening on a purse. When each person gets to what seems like it might be an edge, they try to drag their heels and stop moving, independently.

After this has gone on a while, you may end up with something like this:


You can see that most of the people have found the edge of the tumor and dug in their heels there. But people #1 and #2 found a bright edge that is probably just "noise". And the people numbered "3" have found something that it's hard to tell if that's tumor or noise.

How should they decide?

In this technique, if you just start tightening the springs, at some point the collective pulling force of the majority will break #1 and #2 loose from the feature they are snagged on, and they'll snap into place around the tumor.

Based on just this slice, the people labelled 3 may not move, because maybe that's actually an edge. (Tumors don't have to be round - they can be irregular.)

Well, how do the people at #3 decide? Here's the trick. While all this is going on on this slice,
the same thing is going on on all the other slices, and springs connect the dots / people across the slices. In other words, we actually have a sphere of dots / people, connected by springs, kind of like an over-inflated balloon, and we let it slowly deflate in three dimensions at once,
around the feature in all the slices.

In other words, there is not enough information in the vicinity of any one person to be able to sort out image from noise with certainty. But, most of them are nearly right. We just don't know which ones that is. So, within each slice (or region) the people consult with each other, while at the same time they are consulting across regions as well, with a mix of believing their own eyes, and enough humility to know, at some point, to let go and go with the crowd.

This seems like a very simple plan, and no computer is required. It turns out to be a very powerful technique ("algorithm") that does a remarkably good job at sorting out "noise" from "signal" in 3-dimensions, with only trivial programming required.

Each person / dot simply has to pay attention to what it sees ("independent investigation"), but balance that with consulting with neighboring people and at some point yielding to peer pressure and moving into line. If the balance of these two competing forces is right, the overall network turns our to be a very powerful analog computer that can solve a problem we have trouble even defining well.

No single person ever needs to "see" everything or see "the big picture" - he just needs to see his neighbors, compare notes, argue for his position, and, if it seems warranted, yield to the majority. If enough different dots do this, coming in from enough different directions at once ("diversity"), and remain independent and yet consulting ("unity in diversity") the algorithm works. The powerful solution "emerges" from each person's behavior.

In image processing this is called an "adaptive contour" technique. It is part of the larger class of techniques called "swarm computing" that is becoming increasingly popular as "the power of crowds" is increasingly being appreciated.

An area this could be used is in any sort of boundary measurement, or in aligning fragments of images to make a coherent overall picture. Examples of these, and my US Patent 5613013 in image alignment using effectively a swarm technique, are described on my web site here.

Wade

When are all of us smarter than some of us?

This mornings New York Times article "A compass that can clash with modern life" discusses the problems Islam is having relating the words of the Prophet Muhammad, written or spoken, to modern life.

Since Islam doesn't have a single, official clergy, anyone can take it upon himself to start issuing "fatwas" or statements or rulings connecting the two. Some of the results are embarrassing and need to be retracted. Sometimes the results is to split the faith into competing sects, rallied around different interpretations of some passage. We see this in Iraq and it is and expensive and deadly problem.

Is there some better way? Does " modern science" offer any insights that could be helpful? Do we, collectively, know anything relevant today, that we didn't know 500 years ago, or 50 years ago, or last week? Or are we spending our society's bank account trying to get to the moon while our own roof still is leaking and our own people are still in need?

One way to approach "this problem" is first to figure out what kind of problem it is. How many other people have "this" problem? And did any of them figure out how to solve "it" yet? Or, if not, could we join forces, pool notes, and work on our common problem together?

As I understand it, the Exam of Life is "open book", "open notes", and it's OK to consult with each other when we don't know the answer. Unfortunately, this is almost exactly the opposite of what is taught in most school systems today, so we're not very good at this.

Well, if we view this as a problem of trying to maintain constancy despite change, it is a familiar problem to any religion, any nation, any organization, and, well, hmm, just about anything that's alive. In fact, this is the core problem of survival of the pattern of "identity" over time, despite decay, changes in the outside world, and developmental growth. This is the "homeostasis" or "allostasis" or "feedback control loop" problem I've been talking about.

So, this is a pretty important problem to solve. Despite that, very few "scientific" resources have been focused on it, explicitly, as a problem.

The same problem is true of any legal system, or for that matter, any "regulatory" system -- to what standard or goal or identity do we return when the outside world has changed, or time has passed and we've "grown up" ?

Another faced of this issue is that we, all Living things, need something like "love" that is "always the same, but always new." Always exactly the same is boring, and gets old very fast, and doesn't seem like a solution. The constancy evolving life needs isn't "constant" but is more "a loop invariant" in terms I used previously.

I think I had used technical mathematics from the theory of General Relativity to support the case that there is machinery to compute what happens when things are "context dependent" and, worse, where they both cause and are caused by context. Maybe one other person followed that, so I need a better example.

An easier example is simply "perspective". If I see you now, and you are facing me, I may see your two eyes. If you turn to the side, suddenly I see only one eye. Or none. Have you suddenly changed and lost your eyes? No - we all know that it's "the same" person, just a different "viewpoint" - and every viewpoint is, like a shadow, a very limited slice of a three dimensional scene -- from one viewpoint, by one observer with biases and preconceptions and limited vocabulary, at one point in time, expressed and conveyed in words that mean different things to different people at different times.

So, we essentially all have "fragments" of wisdom. And, like the three blind men in the Sufi tale, we have felt different parts of the elephant and have very different ideas of what shape the elephant must be - a tree? A rope? A huge leaf? A spear?

The question of assembling fragments into a larger image is, by no coincidence, the subject of my patent in "Image Processing," which I pursued because this is such a core problem to our survival. (United States Patent 5,613,013 - "Glass Patterns in Image Alignment and Analysis.") It is fair to say that I have spent 20 years or so working on this problem, from the perspective of science, from the perspective of business and organization theory, from the perspective of Computer Science algorithms, from the perspective of Public Health, and, lately, from the perspective of "goal-driven, self-aware, self-repairing feedback control loops."

In other words, while I see these other perspectives as helpful, it really seems to me that the self-aware control loop is the best "primitive" element that is common to all manner of organization of effort and people and cells and health care organizations, and solitons, and composite Life in general.

This is why I'm bouncing from foot to foot with frustrated excitement to try to convey to anyone who will listen that,

• We can map almost every major biological, social and business and religious organization into this framework, and
• People known as "control system engineers" already understand that problem and how to measure, analyze, design, and repair such things - but THEY still are looking at building better jet fighters or cruise-control systems for cars - not "social engineering" or worse "Planetary Completion Engineering."

To quote a line from the movie Ghostbusters "Oh, we have to get these two together."

• This is where Science and Religion come together and join forces instead of clashing. This is, in my mind, where they see the differences between them not as "me" and "enemy" but as, "Oh, look, I'm male and you're ... uh ... female and, ... uh... there are other things we could be doing besides trying to kill each other off." We each bring something of great value to the table, that we've each spent a thousand years, at least, trying to preserve and protect and nurture against all enemies without and within. For what? Why did we do all that? Why have a billion people died protecting this precious cargo - one clearly worth dying for?

Is there a way to make this long story short? Looking ahead at where I come out on this.

• Everyone's right and no one's right.
• In complex adaptive systems the answer is usually "AND" not "OR".
• If you want to align images, an area in which I seem to be a documented expert, it really does help if you hit the fragments with a two step approach.

Here's how I aligned microscopy images at Parke-Davis to get both a wide field of view, to see entire blood vessel cross sections, and high resolution of details, using only a single microscpe with a digital camera. I go over this as non-technically as I can, because it's more than an analogy, it's an "algorithm" or a process that can be used in other areas of life. So, bear with me, this is relevant. Extremely relevant, I think.

• First, get a broad overview picture with a low-power lens, a survey picture.
• Second, take a set of high-resolution images, recalling if you can roughly where you were in the broad overview map when you took each one, and recording that. Some of these detailed pictures may be rotated or offset from where we thought the microscope stage was, since it's not very accurate at this resolution -- so take positions as "fuzzy" or approximate data.
• Third, blow up the size of the low-resolution image until it covers the desk at the same scale as the detailed images, but blurry and short on detail.
• Fourth, put each of the small images roughly where it should go, if you remember, and roughly with the North side towards the north if you can. (I used the "Fourier transform" matching to get the best position automatically.)
• Fifth, allowing some flexibility around that estimated position, (attach it to that place with rubber bands, not with steel bolts), search each pair of neighboring images for parts that overlap really, really well. As you find the best place, using something like the easy method in my patent or something harder, lock that pair together into one larger piece of the puzzle.
• Sixth, keep on working on the next best fit, until all the pieces are aligned or irrelevant (like, the center of a blood vessel with nothing in it.)

And, Voila - a beautiful full-size, high-field of view, high-resolution image that can be analyzed by the next process or person in the chain.

That works in practice. I did a lot of those for our investigation of drugs at Parke-Davis in routine daily research.

There are two amazingly powerful and wonderful keys that unlock the process and change it from almost impossible to almost trivial.

1) there is something called the "crumpled paper theorem" that says, basically, if you take two pictures of the same thing, and crumple one up into a ball and throw it on top of the other one anywhere, you can be guaranteed that at least one point in the bottom image will be directly below the corresponding point in the crumpled image. (Actually, it work for stretching as well, or any "affine transformation.")

2) There are things called "Glass patterns" that are formed when two images are on top of each other on transparent media, so you can see through them both, that are sort of like "Moire patterns." These are "higher order" patterns, that are there, but aren't, but are -- very much like the simple moire pattern I showed a few days ago. With overlapping images that are within about 6 degrees of aligned with each other's angle, you'll see large "circles", even if the images are noisy. You'll see them even if there are no edges or "fiduciary points" to align with, like nice obvious corners. You'll see them in sand dunes, or water waves, or grass, or trees, or stars or laser-speckles off a surface, or just random noise. They're always "there" in higher-order vision land. Once you know that, they're easy to find. Then you can complete the "registration" process and get the images perfectly aligned by just watching the big circles instead of the actual images as you slide them around. (or, in my patent, I just find three points that match in each image, connect them with mathematical rubberbands, and "pull" the images into alignment with those.)

OK, enough on the images already. So there is a "massively parallel" method of aligning fragments correctly so we can automate the assembly of the jigsaw puzzle, even if the individual fragments are somewhat rotated and stretched and different scales. It uses two key facts of life to reduce an impossibly difficult algorithm into a very fast algorithm.

Now, back to social organizations. Suppose we assume that everyone at a meeting, or in an organization, or in a society or religion has some wisdom, some fragment of truth, and can see one side of some tiny local part of the world, but really really well. And we would like to "assemble" all these fragments into a larger picture so that we end up with both a huge field of view and huge depth of detail at the same time.

Here I look at the quotation from Jesus in Mathew that I listed a few days ago, that is common to Judaism, Christianity, and Islam. Basically, first of all love God, and second, love your neighbor as yourself. There is, in my mind, more than an analogy here to the image processing solution of -- first get the big picture, even if blurry, approximately right and get your North end pointing North, then, deal with aligning with your nearest neighbors on all sides -- and voila.

I'm not sure what exactly the "voila" picture will be, but I'd love to see it before I die. I think it will be Beautiful beyond our ability to express it.

Except, in the case of living stuff, Life, cells, people, organizations, societies, legal systems, etc., the "pictures" we need to "align" are these basic "self-aware, self-repairing feedback control loops" that I need a shorter name for. Maybe "sloops". Sounds nautical.

There are two dimensions along which these need to be aligned. One is "vertical" along social structure lines of the hierarchy of life. Cells need to align their health with their local tissues which need to align with organs which need to align with the body, which needs to be aligned with the department or work team, which needs to be aligned with the corporation which needs to be aligned with the national good. There are zillions of non-aligned ways to assemble these multi-scale fragments, but probably very few fully-aligned ways, so maybe the "right" way, or rotated and scaled variants of the "right" way, will become obvious. ( I think Toyota's Lean production system is getting close, although that doesn't explicitly consider impact on each employee's physiological health.)

The other dimension is horizontally. This question of "merger and acquisition" on all scales is a harder one, that doesn't happen with"billiard balls" or mechanical models of the world. Almost by definition, two "objects" when put near each other don't attract, rotate like little magnets, and pull each other together. I suspect LIFE is different, and there is a "click" or a "notch" or a resonance or a "snap to grid" option, where two different life-forms suddenly can form such a tight bond that they then operate as one.

There are many examples at the corporate and biological levels, the most startling being "slime mold" where "individuals" flow together, become "one" large thingie to move to a new place, then "dissolve" back into individuals again. How cool is that!

The mathematics of two loops merging into one is undoubtedly tricky. There's been a lot of work done on what happens when two "vortices" collide - and a vortex is getting close to a self-sustaining, energy absorbing control loop. (Think "tornado" or "hurricane".)

There's also some great on-line videos of what happens when two ring-vortices collide. Did you know that whales and dolphins blow "bubble rings" which are like "blowing smoke rings"? And they seem to do it on purpose, just for fun. Here's a link to some amazing on-line dolphin and whale and human videos.

OK, close the loop, Wade. I started with problems with Islamic fatwa's and ended with dophins blowing bubble rings. The theme here is that most of our social problems seem to involve us trying to get a lot of small feedback loops "aligned" or "merged" in a win-win fashion so as to preserve the precious cargo each of them carries in the merger. I suggested that the mathematics involved is the same as that I solved for aligning digital camera images, but instead of pixels, basically, we have these small feedback loops that are central to each "living" thingie at any scale or level -- from cells to nations.

And I suggested that the visual tools for analyzing such interactions are already available in control system engineering, and we should expect the results to be at least as complex as what happens when two "bubble rings" collide and merge. (as in this video.) Except that, bubble rings aren't fully homeostatic, and don't each repair the damage done by the collision and discover they are now linked like links in a chain, or both occupying one larger feedback loop and identity now, or whatever set of operations we find such things tend do do when simulated.

Then I suggested that this massive-parallel solution technique needs a top-down context component and a bottom-up interaction-with-neighbor component to work efficiently. I noted such components in the Toyota Way and in the Abrahamic religions.

All of which continues my brazen advocacy for putting more eyes and research effort and dollars into understanding how this self-assembly process in society could be facilitated, catalyzed, and made 100 times easier.

We've become unbalanced with too much "specialization" and not enough "reintegration" in our social cycles, both in religions and in legal systems and in corporate structure and in health care systems and hospitals.

This "integration" and reaching across silos without destroying the values carried by each silo is, in my mind, THE key social problem right now.

And it's a bootstrap problem, in fact. If scientists or theologians could get better at working with each other ON this problem, then they would make more progress on reaching the solution. Given that some sociologists and behavioral scientists are reading this, I better include them too. The very same equation is at the core of everything from short-term business meetings or town planning meetings that are effective to large scale political and governance processes that are effective, to military operations that draw on the wisdom of every pair of eyes out there.

How do we make a bigger ONE from many? How do we have unity with diversity? How do we comply with both "In God we trust" and "e pluribus unum"?

That's a problem that's common to us. It's common squared, because it also impacts our abilityto even discuss this problem, which needs to co-evolve as wel work on the problem.
It's larger than "interfaith", it's both faith and science and commerce and public health working together on the very same equation, the very same problem.

Well, off to class. I know this should be 12 posts with pictures. Later.

Wade