Hypnotized in high places - Northwest Flight 188

( picture of vulcan cockpit from u07ch on flickr -- Click for larger view.)

So, yesteday, it seems that a Northwest flight #188 overflew its destination city as the FAA attempted desperately to reach it. According to the NY Daily News,

Crew members aboard Northwest Airlines Flight 188 told the Federal Aviation Administration they were distracted during an intense discussion over airline policy and lost track of their location in the bizarre Wednesday night error.

I wrote a comment to ABC News, after reading the other 200 or so comments, as follows (spacing put back in for clarity and ease of reading).

====

There are always multiple levels of contributing factors, from personal to procedural to crew interaction to cockpit design to job design to corporate policy to FAA policy.

For safety, versus lawsuits, it's worth looking at each level of that nested hierarchy of contexts to seek ways to reduce the odds of this type of thing recurring. If we do that, we don't need to know for sure what happened -- only what might have happened that we are now aware of is a gap in our current system that is relatively easy to fix without side effects.

I think the context of the discussion could be expanded in two ways, both of which involve asking "What other events is this event like?" in a much larger framework.For example, I note an uncanny resemblance between this situation and the behavior of CEO's and government regulators as the nation financial situation flew up to the red line, and past it, while thousands of people screamed and called for attention below, and those above seemed to be ... asleep? ... arguing?... out to lunch?

This is not just situational unawareness, it is unawareness or a shared-delusional-mesmerized state that cannot be broken into by repeated efforts from outside and below, in the corporate and governmental boardrooms.

I'm not saying that just to b####, although b####ing can be fun -- I'm saying that human beings, even those with superb qualifications in isolation, can manage, collectively, to get themselves set up so that those "above" are completely and thoroughly "cut off" from input and flying blind or simply not flying at all anymore.

Again, not as legal blame for this accident, but as a route to understanding "what goes wrong with human interactions", this event could spur us to look at that much larger question, asking seriously, "No, seriously, how could THIS KIND OF THING ever actually happen?"

The truth is, socially, it happens A LOT.

There is something structurally seriously wrong with our mental model of how a hierarchical command structure ACTUALLY functions versus how we IMAGINE it to function.The lives destroyed and lost on a corporate and national level from THIS KIND of error are far more than the lives lost in this latest incident (zero).

Wade_AA

====
(picture by aeneastudio on Flickr)

Other observations I've made about structural blindness and delusional-mesmerism in high places:

Why we have so much trouble seeing

Why are so many flights delayed? The circle of blame


Model induced blindness and FEMA

It's a year since Katrina made it obvious that people watching CNN knew more about what was going on top government officials.

We have to ask how that is even possible. It defies our intuition, although not our experience, which is interesting.

While the "blame-game" remains in high-gear, Systems Thinking leads us to discount the obvious "bad people" and look for deeper root-causes in the social structure. FEMA Director Brown has been replaced, but the systems problems are harder to see and may still be there....

The power of delusion

It is an astonishing fact of life, which the Times article reveals, that the desire for life to be simpler is so powerful that it can cause 10,000 "trained" scientists, with PhD's, to take 30 years to finally collectively observe what others outside their mutual-blindness-field already knew.

As I've said, textbooks such as "Feedback Control of Dynamic Systems" are in their 5th editions in Control System Engineering, but biologists, and much of public health's biomedical research community, discount that literature to the point of invisibility and effectively treat it with contempt. To them, this literature does not exist. When seen, it "comes as news to them", and is promptly forgotten, because it conflicts with the shared myth of their culture, and cultural myths always win out over boring contrary evidence.

The Way Things Are (The "Yarn Harlot" tells it like it is, beverage alert!)

There are some truths. Things that just are the way they are, and no amount of desperate human optimism will change them. Allow me to demonstrate.
The guys showed up with the new stove. I went out front to meet them....

OK, Seriously... WHY didn't we see it coming?

High-reliability human systems have been studied extensively, from Chernobyl to The Bay of Pigs to Challenger to aircraft cockpit teams to hospital surgical teams to the US Army Leadership Field Manual. The answer always comes down to the same thing -- dissenting views need to be heard, and dissenters need what Harvard Professor Amy Edmondson calls "Psychological Safety" or they will wilt and become ineffective. This is how humans always behave and unless steps are taken it always breaks along this fault line.

The right question then should be, who is going to take charge of seeing that those steps are taken and that level of social literacy achieved?

I can't emphasize enough how much more important this is than more math and science, in the absence of this. As T.S. Eliot said, we repeatedly get burned "dreaming of systems so perfect that no one will need to be good, but the man that is will shadow the man that pretends to be."

Much of my weblog is about what we really need to do to avoid such errors in judgment.

====
Various related posts:

My 40 page multilevel structural analysis of
the Crash of Comair 5191 crash in Lexington KY. August 2006
with extensive links to source materials

related webpost with links to Comair 5191 cockpit voice recorder transcripts.


Information on the investigation of the crash of Continental flight 3407 in Buffalo, NY Feb 13, 2009, from the Buffalo News.

On Thursday, Feb. 12, 2009 at 10:20 p.m., Continental flight 3407, en route from Newark, N.J., spun from the sky and crashed into a home as it made its approach to Buffalo Niagara International Airport. All 49 people on board the plane were killed, as was one man in the house in Clarence Center. It was the worst aviation accident in Western New York history.

...A moment later, the co-pilot, Rebecca Lynn Shaw, complained of her own inexperience.

"I've never seen icing conditions," she said. "I've never de-iced. I've never seen any. I've never experienced any of that. I don't want to have to experience that and make those kinds of calls. You know I'd 've freaked out. I'd have like seen this much ice and thought oh my gosh we were going to crash."

Moments later, the crew lowered the plane's flaps and landing gear, and the plane quickly encountered trouble.

Things we have to believe to see

Why men don't ask for directions

Pisa/OECD - Why our education stresses the wrong way of seeing

Failure is perhaps our most taboo subject (link to John Gall Systemantics)

Active strength through emergent synthesis

US - Economy of arrogance (and blindness)

Virtue drives the bottom line - secrets of high-reliability systems

High-Relability Organizations and asking for help

Secrets of High-Reliability Organizations (in depth, academic paper)

High-Reliability.org web site

Threat and Error Management - aviation and hospital safety

Failure is perhaps our most taboo subject (link to John Gall Systemantics)

The importance of social relationships.

Houston - we have another problem (on complexity and limits of one person's mind)

Institute of Medicine - Crossing the Quality Chasm and microsystems (small group teamwork)

Here's a few quotations from MIT Professor John Sterman's textbook "Business Dynamics".

Many advocate the development of systems thinking - the ability to see the world as a complex system, in which we understand that "you can't just do one thing" and that "everything is connected to everything else." (p4)

Such learning is difficult and rare because a variety of structural impediments thwart the feedback processes required for learning to be successful. (p5)

Quoting Lewis Thomas (1974):

When you are confronted by any complex social system, such as an urban center or a hamster, with things about it that you're dissatisfied with and anxious to fix, you cannot just step in and set about fixing things with much hope of helping. This realization is one of the sore discouragements of our century.... You cannot meddle with one part of a complex system from the outside without the almost certain risk of setting off disastrous events that you hadn't counted on in other, remote parts. If you want to fix something you are first obligated to understand ... the whole system ... Intervening is a way of causing trouble.

IN reality there are no side effects, there are just effects.

Unanticipated side effects arise because we too often act as if cause and effect were always closely linked in time and space. (p 11)

And, this crucial comment by Sterman, reflecting the same observation by persons such as John Maynard Keynes.

Most of us do not appreciate the ubiquity and invisibility of mental models, instead believing naively that our senses reveal the world as it is (p16).

My additional note on this crucial insight. The reality is that the world, as it shows up on the mental TV screen we watch, is NOT the world that is actually out there. It has been more than rose-tinted by our brains. It has had entire chunks of the scene edited out entirely, and other chunks that "should go there " put in their place. A whole set of things that have given us pain or conflict in the past have been summarily removed, without so much as a place-holder left where they were. A set of things we hope might be true have been "helpfully" added to the scene. People's behavior, where it deviated from what we expected, has been "corrected" to show us them acting "the way we KNOW the are", not the way they actually are.

We are, in other words, flying almost entirely blind. We have papered over the front and side windows of our cars with pictures of the way we WANT the road to be, and are driving and turning the steering wheel based on those internal delusions.

Throughout evolution, this has been useful to reduce the immense fire-hose of data to a smaller set we can live with -- and, if we do a bad job of managing it, heck, we just die off and don't reproduce and others who do better jobs have children and go on. No big deal.

The problem comes when those living in such delusional and self-confirming, often self-congratulory worlds are given the power to rule our communities, our corporations, or our governments and they continue onwards believing that what shows up on their mental TV screens IS in fact what is going on out there, and believing, therefore, that those voices of dissenting views are, in fact, some kind of misguided or enemy action that should best be suppressed, shut out by locked cockpit doors or isolated fortress war-rooms, etc.

I'm not saying that solving this problem of filtering the fire-hose of complexity down to a size we can comprehend and use as a guide for steering is an easy one -- but I am saying that it is the kind of hard, complex problem that can yield its secrets to methodical research and study, and it is THAT research we desperately need at this time in our lives on Earth.

This is where it is breaking.

This is where we need to fix it.

Well, at least, that's what MY internal mental TV is showing me right now as the "obvious truth".

Wade

Why do smart people do dumb things - Countrywide

Countrywide Financial Corporation is the latest case study of how smart people do dumb things. What is surprising, in retrospect, is that failures of this magnitude are not studied with the zeal that commercial airline accidents are studied. After all, probably as many people are hurt or killed by the indirect effects.

Still, are we learning from experience? Or simply repeating the same mistake over and over? Airliners use cockpit simulators to train pilots to avoid errors -- what do we need for CEO's and average home buyers?

What comes to mind is a short poem by Shel Silverstein " The Slithagadee. I can find many variations of it on-line, so I'll just quote it as I remember it:

The Slithagadee.

Oh, the Slithergadee
Came out of the sea.
He caught all the others,
But he won’t catch me.

No, you won’t catch me,
You old Slithergadee.
You caught the others,
But you wo...

—Shel Silverstein

Anyway, here's a few facts on Countrywide Financial from today's LA Times. My excerpts focus on two things:

1. the psychology of feeding frenzy and how, as with "only a foot" of water on the road, you can find your car swept away, and
2. How "systems" effects mean everything affects everything else -- there is no immunity.

Credit crunch imperils lender

Worries grow about Countrywide's ability to borrow -- and even a possible bankruptcy.

By E. Scott Reckard and Annette Haddad
Los Angeles Times Writers

August 16, 2007

[Excerpts]

Angelo Mozilo, chief executive of Countrywide Financial Corp., has been fond of saying that the company became America'sNo. 1 mortgage lender by being smarter than the competition.

In a harangue to Wall Street analysts early last year, the combative Mozilo denounced upstarts for shoveling out too many loans, too easily, to too many people with bad credit, heavy debt and skimpy income.

"I've been doing this for 53 years, and I've never seen that situation sustained," said Mozilo, who co-founded Calabasas-based Countrywide in 1969. "Eventually they gag on it."

Dozens of home lenders have indeed collapsed as defaults have surged on loans made to people with poor credit during the housing boom and as Wall Street has turned off the money tap that funded many of those sub-prime mortgages.

But rather than emerging bigger and stronger as Mozilo predicted, Countrywide -- which made 1 of every 6 home loans in the U.S. in the first half of this year -- now finds itself battling not just its own growing defaults but also a widening credit crunch stemming from the nationwide sub-prime mortgage meltdown.

On Wednesday, the company was said to be having trouble borrowing money on a short-term basis, securities analysts discussed the possibility of a Countrywide bankruptcy and the firm's stock price tumbled 13%, bringing its loss for the year to 50%.

An insolvent Countrywide could also do more damage to the country's already weakened housing market, said Guy Cecala, publisher of Inside Mortgage Finance, a trade publication based in Bethesda, Md.

"It would be a huge shock to the U.S. housing system and the mortgage system as perceived around the world -- and make an already bad situation terrible," Cecala said....

"The problems Countrywide is experiencing has nothing to do with its mortgage business," Cecala said. "By all measures, Countrywide is a well-run, profitable company. What they're finding out is that although they thought they had diversified funding sources, nothing is diversified in a worldwide credit crunch like we're in now."

In 2003, old-fashioned 30-year loans with fixed interest rates and substantial down payments made up about two-thirds of Countrywide's loans, said mortgage executive Bill Dallas, whose Agoura Hills-based sub-prime lender Ownit Mortgage shut down early this year.

By last year, only one-third of Countrywide's loans were of the traditional type, with the rest spread among the more exotic loan variety, Dallas said. He said Mozilo, who had often been quick to criticize rivals for being overly aggressive, had found himself immersed in the same businesses as his competitors.

"Every section of the business that has failed, they're in there big time," Dallas said.

At some branches, managers would buy lunch every day for their staff to keep them at their desks working.

At the height of the boom in 2004 and 2005, it wasn't uncommon for a typical Countrywide loan officer to sell 20 sub-prime loans a week. "It was a feeding frenzy," said one former Countrywide employee who said he joined the company in 2004 and, after six weeks of training, made $6,000 to $8,000 a month. As fast as loans could be signed, they could be sold to investors, according to the former employee, who declined to be identified....

Over the years, Mozilo's pay packages -- $48.1 million in 2006 alone -- ballooned along with his company's fortunes.

Countrywide also bills and collects payments on $1.4 trillion in mortgages for itself and other lenders. What's more, Countrywide is the largest customer for Fannie Mae, the big government-sponsored mortgage buyer. More than one-third of all mortgages sold to Fannie Mae comes from Countrywide.

"The question is, is Countrywide too large to fail? Will the Fed allow it or will it need to step in and bail it out?" Cecala said.

A bailout of Countrywide would make the government's efforts to save automaker Chrysler in the 1970s look puny.

"Countrywide is more important than Chrysler was back then, particularly given the fragile state of the economy and so much tied to housing," Cecala said.

Of course, what's not mentioned is why a healthy company needs to borrow money in the short term to survive. Where did they put all their rainy-day savings from prior profits? Hmm. IT does sound like the "hedge funds" who were so sure of profits that they borrowed against the value of everything they owned and bet it on this "sure thing" -- except the nag broke a leg on the far turn of this track.

If so, this is really no different than the employee who "borrows" $10,000 from the bank till at lunchtime to go bet it on a sure thing and return it by the end of the day and no one will know.

Maybe we need to learn to recognize the typical signs and symptoms of unjustified reliance on ... hot air ... as a basis for business decisions or governmental policy decisions.

What is particularly dangerous, as I've pointed out before, are positive feedback loops, where the "reason" something is becoming more attractive is that "it's becoming more attractive." This phenomenon accounts for some Hollywood fame, where the only reason one can see for why this unremarkable person is famous is that they are famous and it feeds on itself.

That may even be a great strategy to make a buck on as it rises, as long as you stay aware that on any morning the wind may shift and suddenly it is falling just because it is falling, and the spiral up becomes a death spiral down.

Also, "amplifiers" are wonderful things, used correctly. For financial investments, "leveraging" you funds so that you only have to actually shell out 5% of the cost of something (right now) to "buy" it can make sense if the "it" will make you piles of money rapidly. Again, the problem comes in feedback, where you start amplifying the amplifier, etc.

Before long this vortex "takes on a life and momentum of its own", for exactly the same mathematical reasons that a hurricane or tornado starts sucking in surrounding air and "feeding on" the energy in it to become even larger, which lets it feed more, etc.

But, all such "fools gold" will have to turn around some day. This sort of thing is like "silly putty" or cornstarch fluids, that act "solid" if you keep moving fast, but turn instantly to "liquid" if you ever stop moving. (YouTube video worth watching of college students dared to run across such a vat of "liquid". This is great! )

In other words, this is a "good idea" only if you are a heart-beat away from an exit strategy. Otherwise, it is a pure pyramid or "Ponzi scheme" and the company "built on" it is a "house of cards."

There are many examples of such spirals that appeared as magical money machines and seemed from within that they would never end, like a credit card on a company that never remembered to bill you. Then, one day, the accumulated bill arrives. And you better have saved up enough "profit" to be able to bail out, and not sent it all to stockholders on re-invested it.

Tulips were the international craze in the 1700's, I think. Dot.com's. Pet Rocks. The Hula Hoop, 5 cents worth of plastic that sold for $2.00. Actually, the company that made Hula Hoops lost money, because they put everything they had earned into a brand new factory for making even more Hula Hoops, just as the craze ended.

So, again, the "hazards" of this strategy are well known in the business literature. The "right" way to win money in the long run is known. And the reality, that frail humans succumb to the "momentum" is also well documented.

But human ego is a fickle friend. "Just one drink." "Just one more investment and then we'll get out." "Just wait until it comes back up to where we bought it, and we'll sell it without recognizing a loss. "

Oh, the Slithergadee
Came out of the sea.
He caught all the others,
But he won’t catch me.

No, you won’t catch me,
You old Slithergadee.
You caught the others,
But you wo...

This is why a long solid history of "consultation" with a very diversified group of friends, preferably of different cultures and located in different counties, is a good idea. In fact, it's the only strategy I'm aware of that can prevent this kind of localized shared blindness, group-think, from taking the reins away from even very strong, very solid thinkers.

And, even then, that won't work if you reserve the right to just ditch you best friends and redefine them as "enemies" with "negative thinking" when they tell you that you are wrong and making a terrible mistake.

Of course, it never looks like a mistake from inside to the person making it. Think about it. That's the subtlety our school system doesn't train us for. Humility, or just experience ( which is wisdom acquired just after it was needed. ) Maybe a simulator game.

Here's the key lesson: the mind is a fickle friend. Your "obviously" creating sense has blind spots that can ruin your whole day.

Or, as Dennis the Menace said, sitting in the corner being punished - "How come dumb stuff seems so smart when you're doing it?"

Karl Weick teaches a need for "mindfulness" that maybe our "mental model" is outdated, but thats a fancy way of saying the same thing. Neither IQ nor will power nor genetics nor training can overcome the ability of people to fool themselves.

Only a wide grid of interlocking people is robust against such vortices of "easy money". And "wide" has to mean diversified, over a wide area of the planet, where some are not within the range of the "obviously" perception-distorting perceptual vortex.

This is a very common phenomenon, but we, as humans, remain in denial or think we are somehow different and immune to it.

No, you won’t catch me,
You old Slithergadee.
You caught the others,
But you wo...

Sources for the Slithagadee.

"The Norton Book of Light
Verse" (page 314)
source.