Thursday, November 22, 2007

Adaptation versus control - and the Army FM 3-24

Too often, the effort to use centralized command to achieve unity of action and speed of response, perversely, fragments action and slows reactions and recognition of problems. The US Army's new FM 3-24 counterinsurgency manual gets part of this right. FEMA still has it wrong, as does the US National Incident Management system. Katrina illustrated what such a system can deliver, regardless who is at the helm. Bad design, not bad people, is the core problem.

Thanks to Anne Applebaum on Slate for posting a link to Fred Kaplan's "Counterinsurgency by the Book" .(Nov 19, 2007), and some trail that led me through Rant Street's "FM 3-24" posting on the subject that kicked off my interest in this.

These immodest conclusions follow from some very basic principles of nature that describe how any system can operate, regardless of design details. I've described these in earlier posts as a need for a "vertical" loop between the front-line staff and headquarters, and a "horizontal" loop between everyone and the outside world.

The classic model of centralized control worked when problems were relatively simple and the purpose of the size of an organization was to expand the muscle power, since the people "at the top" had enough brain power to understand the problems involved. Primarily what was involved was correcting deviations from a known solution to a known problem, and enforcing a status quo that had already proven to work. The strength of this system was stability in the face of external and internal disruptions.

So, regardless how the external world varied, the system could re-establish equilibrium and restore working order back to the state that worked.

Unfortunately, times change, "the cheese moves", and this strength in a new context became a liability not a strength. Rephrasing John Gall, "The old solution became the new problem." Because, when the external world changed permanently, not due to noise, to a new shape with new demands, the internal system still kept on recreating the old way of doing things that "always worked before." The old system could not adapt to a permanently changed external world.

That would be bad enough to wreak havoc, but things got worse. Not only did the external world change once, but now it keeps on changing and changing again. Now it is not enough to have some non-zero learning curve, the speed of learning has to keep up with the speed of change in the world. Old ways become obsolete overnight. This is a whole new ballgame.

And, things got worse than that. Not only did the problems keep changing, but the nature of the problems keeps getting more complicated to even begin to describe, let alone to fully comprehend. So, instead of a change requiring a brief note to headquarters that "they moved from there to here", the changes require a short course in some new subject area to even get a new vocabulary of what "they" are doing now. Lacking that shared vocabulary, a huge amount of information has to stream up to headquarters to be comprehended in real-time.

This is one of those situations described by algorithms in computer science, but the short of it is that it doesn't matter how bright the people are at the top or center of all this. As things continue to get more complex and move faster, at some point the task will always exceed the ability of the center to handle. We've already passed that point in most industries, and in most international situations.

It doesn't help to demand that reports to the center "distill" the problem, or boil it down, or "make it simpler" or focus on the 2-3 key points and "prioritize." These problems don't boil down any more, and they have 20 or maybe 200 points that are simultaneously "key". Like 200 holes in the floor of your boat, "which ones do we block first" doesn't actually reach to cover the task you face. All of them have to be dealt with, and within the next 15 minutes, and then 200 new ones will arise to replace them.

It seems like a losing battle. That's because it is a losing battle, to try to solve that problem using a fixed size group of people in the center "orchestrating" everything base on "their understanding" of "the situation.' That problem requires a different approach, a better algorithm.

The better way is to distribute control away from the center, and empower the front as much as possible. Instead of trying to acquire power, you have to try to delegate it across the whole organization. Then you have a chance of scaling up to the rapid fire changes of situation that are happening. Even then it's not clear there is a single variable called "winning" that you could theoretically know and monitor to see if you are doing it or not. Too much is in flight, in flux, and hasn't yet come home to roost, let alone had time for you to hear about it and understand the full implications of what you heard.

Finally, as I discussed in my white paper on disaster preparedness competencies, when things start going wrong, and stress levels increase, there is an unstoppable tendency at the top to circle the wagons, to close ranks, and to focus on trying to see that "orders" are carried out. This is not a good time to suggest learning a new set of skills and new mental model. It is not a good time to suggest that the assumptions behind the old plan bear no resemblance to the actual situation on the ground. It is a terrible time to suggest that public statements about things "working" are actually incorrect.

Again, if the world simply replayed over and over the prior battle scenes, this might work. The problem is, that's not what's happening. Entirely new situations are happening, that never happened before, and aren't in the play-book. Things we never thought could happen just did. All bets are off. All the practice scenarios are irrelevant.

So precisely when the center needs to hear the most surprising news, that their mental model is wrong about what's out there, is when the center shuts down all incoming traffic that bears such news.

So, it's critical that organizations be capable of learning very rapidly, when more than just the facts are changing, but the whole model of what's happening is wrong. This requires a learning organization that has practiced that scenario over and over before this point -- it's not a new way of doing things you can suddenly switch to in mid-stream.

It requires that the eyes and brainpower of the entire organization be mobilized, not just the muscles. And it requires distributing control as far down the chain of command, as close to the front as possible, so that responses can be as fast and instantaneous as possible.

Our bodies learned this and when our hand touches an unexpectedly hot stove, the arm doesn't wait for the news to get up to the brain and be understood -- it jerks the hand off the stove on its own authority. There is no other way to do this.

The problem looks at first glance like simple signal processing. There isn't time for an impulse to travel up to the brain and back, maybe 150 milliseconds, when a 50 millisecond response is required. As with fluent keyboarding, the fingers have to work faster than the brain can follow.

But for rapid-fire corporate battles or unfolding military situations, it's not an impulse that has to travel to headquarters -- it's a totally new CONTEXT that needs to be conveyed before the CONTENT of the message makes any sense at all. Now multiply this times 1000, speed it up, and realize that any human being can only have one context at a time, and it takes a while to change contexts.

You see the problem. Contexts cannot be processed at that speed, even if contents could be. Contents can often be distilled, by reducing them to words that are simple -- in a given shared context. If the context isn't shared, distillation is not possible, period.

Period.

This is why neurosurgeons can talk to each other, and internal medicine doctors can talk to each other, but when they try to talk across specialties, things break down. And none of them can talk to "computer people" who use entirely different language and concepts to make sense of large scale problems. ( I mean, designing a medical records system, not fixing someone's PC login.) You need the secondary and tertiary and now quaternary experts to begin to understand the problem -- but they can only talk to each other even more with each step of sub-specialization. Conclusions don't "carry across specialties."

So, largely, the only people who can understand much of the local and possibly critically important details of what's going on at front number 3,187 are people who are embedded there who have spent months coming up to speed on the situation. And there is no way they can reduce that context to a 3-page report. They can discuss it with each other in a 3-page report, but the words mean something special that would be missed if you just sent that to headquarters. "Distilling" only works in shared contexts. Period.

Statements or facts taken out of context are worse than misleading.

So, this is already a very long post, but maybe I can write a shorter version tomorrow. But let me complete the thought, with apologies.

So, to be agile and adaptive, decision-making in some ways has to get pushed to the front.
But not in every way. And life operates on multiple levels simultaneously, and we have to win on every one of those levels to win the war. None can be neglected.

So, a fragmented world of every front-line person doing his or her own thing is great for some problems, but terrible for others.

We need the diversity to get depth and resolution of crucial details. But, now, on top of it, we need a kind of unity that is not homogeneity.

We need, as I always come back to, "unity over diversity" or "unity in diversity".

Command and control still has to operate or the large-scale, high-order problems cannot be dealt with. Once you distribute decision-making, how do you ever regain coherent control over the emergent outcome, for which you are responsible "at the top" ?

This is a problem of "emergence." It's not possible to "have" control, but it is possible to ensure that control and coherence are "had."

Huh? This is a tricky nuance so let me hit it from several directions. You can design processes that your are confident will result in emergent coherence and sensible action based on the best understanding of the most data, but you cannot "run" the processes in real-time -- in real-time you can mostly just watch.

If all of the parts have practiced working together and responding to each other's needs, the whole thing can click and adapt. You can get a sort of "phase lock loop" holding the parts together, like the low pressure at the center of a tornado or hurricane. If you've made sure that the vertical and horizontal loops are in place and work better and better each day, that's the best you can prepare for times of stress.

Here's a few quotes from the US Army's new Field Manual FM3-24 on "Counterinsurgency" that illustrate the thinking I'm talking about. This is an unclassified document, available on the web, by the way, and says: "Approved for public release; distribution is unlimited."

This publication is available at Army Knowledge Online (www.us.army.mil) and
General Dennis J. Reimer Training and Doctrine Digital Library at (www.train.army.mil).

"This is a game of wits and will. You’ve got to be learning and adapting constantly to survive."
General Peter J. Schoomaker, USA, 2004. The manual's introduction continues:

Perhaps more importantly, it provides techniques for generating and incorporating lessons learned during those operations—an essential requirement for success against today’s adaptive foes. Using these techniques and processes can keep U.S. forces more agile and adaptive than
their irregular enemies.

One common feature of insurgencies is that the government that is being targeted generally takes awhile to recognize that an insurgency is occurring.

In COIN, the side that learns faster and adapts more rapidly—the better learning organization—usually wins. Counterinsurgencies have been called learning competitions. Thus, this publication identifies “Learn and Adapt” as a modern COIN imperative for U.S. forces. However, Soldiers and Marines cannot wait until they are alerted to deploy to prepare for a COIN mission.

Adapting occurs as Soldiers and Marines apply what they have learned through study and experience, assess the results of their actions, and continue to learn during operations.

As learning organizations, the Army and Marine Corps encourage Soldiers and Marines to pay attention to the rapidly changing situations that characterize COIN operations. Current tactics, techniques, and procedures sometimes do not achieve the desired results. When that happens, successful leaders engage in a directed search for better ways to defeat the enemy. To win, the Army and Marine Corps must rapidly develop an institutional consensus on new doctrine, publish it, and carefully observe its impact on mission accomplishment. This learning cycle should repeat continuously as U.S. counterinsurgents seek to learn faster than the insurgent enemy.

The side that learns faster and adapts more rapidly wins.
I don't quote the military because I think life should be waged as a war, but because I think they are forced to face consequences of their actions and forced to learn how to learn in ways that any organization can profit from studying.

And, the military is directly confronted with the tension between a need for agile, dynamic response from the field, and a need for overall coherence and control of the whole enterprise.

From the smallest robot or bacterium to the entire nation, the very same rules of adaptation apply. Internal coherence, internal order based on constantly updated news, and external awareness of changed circumstances are crucial. Mental models that don't work have to be let go of rapidly, by a constant effort to assume nothing and watch to see what is actually happening and if it matches what you expected to happen.

This is, no big surprise, a key part of the Toyota Way, the philosophy and strategy that let Toyota overtake America's car companies, even on their own ground.

The core lessons of a need for specialized diversity, and a need for an overarching unity on top of that, are the single most important lesson that the military, government, and industry need to focus on.

Again, no big surprise, the very same issues arise in all major religions -- there has to be adaptation to fit local cultures, an overarching unity, and a way to adapt and recognize when some mental model doesn't work.

As the old comment at medical school graduation goes, "Half of what we just taught you is wrong -- the problem is, we don't know which half. "

Every person, every nation, every company, every culture, every religion has to face this cybernetic core problem to survive over time. Every community and organization has this problem to solve.

It is one context that we can all relate to and connect to. We all need internal coherence and external adaptation, and a way to keep those operational in a rapidly changing world.

"Unity in diversity" is worth more study, and worth learning how to see this very same template in every organization around us.

We should pool our notes and learn from each other what works and what doesn't. Most people's frustrations with their boss, or their company, or larger organizations have a great deal to do with breakdowns of exactly these two functions - learning within the organization, failure to hear and be heard, failure to make use of existing people and eyes, and adapting to changing situations outside the organization without spilling the soup or losing coherence, order, and enough control to keep the thing on the road.

I wish 1 percent of media time and space was devoted to comparing notes on what we know about that issue.

=============
Related posts:
Second half of this post at "Unity and Adaptation Continued"
Example: Google Maps lets users update the data
What's wrong with decision making at the top (in general)

Further reading:
US Army Leadership Field Manual FM 22-100 (now replaced) as a model for US Business.
Rant Street's links to yet another web copy of FM 3-24.

No comments: