I think you (Yangbo) touched a key issue, and a
systems thinking one, in the comment that "Of course, no system of
governance can be perfect but can nonetheless be amended through
continuous improvement. "
The twin questions of what a "more perfect" (!) system of governance
would look like, at all, let alone how to get from here to there, are
central. In general I'm biased towards trajectories that involve
evolution, not revolution, given the long and inglorious track-record
of revolutions being extremely expensive in lives and wealth, becoming
co-opted, going bad, and ending up being as bad or worse than what they
replaced.
More precisely, ARE there systems of governance which can, in fact, be
amended through continuous improvement, and which do not, ultimately
become corrupted, self-serving, and viewing efforts at improvement as
some type of "enemy action"?
If we don't understand the mechanisms by which systems "go bad", it's
hard to imagine we can make one that won't. This is indeed a systems
thinking and modeling question, I think, and this group is a great one
to reflect on that.
Much more specifically, all personalities, morality, and hidden motives
and agendas of individuals aside, what are the STRUCTURAL feedback
loops which comprise "governance"? Which structural loops are
crucial to an on-going process of incremental improvement? And in what
way have these loops failed to be load-bearing when ramping up in scale?
I'll assert without proof that all large organizations face this same question.
I think a wonderfully profound and delightful place to start that
discussion is with John Gall's book "Systemantic... How Sysems Really
Work and How They Fail", with a summary here:
http://en.wikipedia.org/wiki/Systemantics
A quote from the introduction of that book captures the thought:
"Reformers blame everything on 'the system' and propose new systems that
would - they assert - guaranteed a brave new world of justice, peace,
and abundance. Everyone, it seems has his own idea of what the problem
is and how it can be corrected. But all agree on one point - that their
own System would work very well if only it were universally adopted.
The point of view espoused in this essay is more radical and at the
same time more pessimistic. Stated as succinctly as possible: the
fundamental problem does not lie in any particular System, but rather in
Systems As Such."
Of course, Gall goes on to point out that we are surrounded with
previous efforts to build what TS Eliot would call "systems so perfect
that no one will need to be good", and that our landscape is now
dominated with the still-living artifacts of those "solutions" which
have now, in fact, become "the problem". Rather than repeating that
mistake yet one more time, hoping for different results, Gall suggests
maybe we should understand better what exactly it is about systems we've
misunderstood.
Tuesday, November 08, 2011
Wednesday, October 26, 2011
Pathways to a sustainable future
(From a discussion in a Systems Thinking forum in LinkedIn this morning.)
I think there is great hope for the future already visible amid the catastrophic failure all around us of the old ways that make it harder and harder to believe that "This will work, just keep doing more of it."
The paradigm needs to change. The pathways we need to take will be revealed through encouraging feedback to those who take action, again mixed with hostile feedback from those vested in the old ways.
In systems terms, our solutions lie hididen from us in the closed-loop IIR-world and cannot even be seen or imagined in the open-loop FIR-world. http://en.wikipedia.org/wiki/Infinite_impulse_response
To ask "what path leads there" is like asking, "If you start at the Empire State Building in New York City, which straight road leads you directly to Los Angeles?" The answer is "None" but that is not to say you can't get there from here. No single linear "path" goes from here to there, but big deal. The path hasn't even been created yet so of course it's not visible.
As Einstein said, “We can't solve problems by using the same kind of thinking we used when we created them.”
While we may appear to have a shortage of food, water, and power, one thing we do NOT have a shortage of is people. So the question is, are people a resource or a "sink". If they're viewed as a consumption "sink", yes, then, we are basically screwed.
On the other hand, if people comprise an untapped resource, then we are saved. I fully believe in the "saved" model.
The legacy "competition" model between people, and ever larger-scale groups of people, is what is bankrupt -- not the people themselves. That model does not, empirically, either lead to good solutions, or, in application, lead to very efficient utilization of available resources and opportunities. Easily 95% of all effort in most people-structures at any scale is wasted on internal squabbling.
The larger the structure, the more pronounced this system-effect is.
(An excellent and hilarious, but profound, discussion of the failure modes of large systems is Systemantics, by John Gall.
http://en.wikipedia.org/wiki/Systemantics
)
At the same time, very significant research and practice in the creation of high-performance, high-quality, ultra-high-reliability systems in aircraft cockpits, intensive care units, fire-fighting, and military situations have shown, repeatedly, that CORRECTLY organized teams based on COOPERATION and mutual-respect can and DO function spectacularly better than those based on the legacy model of competition.
In management terms, "Theory Y" organizations are profoundly more robust and sustainable than "Theory X" organizations, make better decisions by far, and are far more fun to work for.
http://en.wikipedia.org/wiki/Theory_X_and_theory_Y
I'd suggest focusing on the key elements of what is it that causes large systems to "go wrong" and become, de facto, corrupt, out of touch, dysfunctional and blind? And, can that be changed by a STRUCTURAL DESIGN change, such that the VERY SAME ELEMENTS (people) doing what appear to be the very same activities, now suddenly WORK instead of failing.
As systems thinkers, we know perfectly well that outcomes are as much functions of the STRUCTURE as the COMPONENTS. We know, from empirical examples, that high-reliability teams CAN be made out of low-reliability people, in the right context.
Are humans "incorrigibly selfish and mean"? Only in structural contexts that use feedback to push them into those behaviors. In the correct structural contexts, people are "incredibly strong, supportive, and capable of heroic and inspired action."
We, in systems thinking worlds, hold the keys to unlocking that buried power that is so vehemently and emotionally denied and attacked by proponents of "theory X".
===
Let me addend to my prior comment that there is an illusion that very rich people will oppose cooperation among underlings or people in general because they fear that, united, the oppressed will arise and overthrow them.
To some extent this is incomplete thinking and needs to be exposed. It is akin to saying "I'm glad I don't like ice cream, because if I liked it, I might eat it, and I hate it!"
If you take an incomplete model of a cooperative enterprise and treat it as "cooperation internally in order to COMPETE better externally", then, yes, be concerned, very concerned, if the staff compare notes and unify.
BUT, if you take a complete model, of "cooperation internally in order to be strong enough to cooperate externally as well", then there is no threat to the existing very rich.
Once people realize that we live in a sea of WEALTH, not a sea of shortage, then there is no motivation to organize in order to redistribute the existing wealth (i.e., revolt and take it away from someone else so that we have it, or to "distribute" it more "fairly" or "evenly.")
A revolution in deed, attempting to redistribute wealth, will certainly be met with hostility. This is not surprising.
A revolution in mind, realizing that more wealth can easily be created so there is neither need nor motivation to threaten someone else's existing wealth, can be met with at least neutrality.
In fact, creating wealth-creating structures of people could be WELCOME by the wealthy, who, frankly, are very hard pressed to know where on earth to INVEST a trillion dollars with any hope of getting a positive gain.
New human structures, a systems-approach to altering the outcomes of human behavior, does not require humans to become something they are not. There is no magic, other than the incredible power of structural feedback.
But I think it can help, from day one, if it is made clear that this is NOT a "revolution" with all that implies. It is an evolution, and a win-win approach to the future.
Wade
I think there is great hope for the future already visible amid the catastrophic failure all around us of the old ways that make it harder and harder to believe that "This will work, just keep doing more of it."
The paradigm needs to change. The pathways we need to take will be revealed through encouraging feedback to those who take action, again mixed with hostile feedback from those vested in the old ways.
In systems terms, our solutions lie hididen from us in the closed-loop IIR-world and cannot even be seen or imagined in the open-loop FIR-world. http://en.wikipedia.org/wiki/Infinite_impulse_response
To ask "what path leads there" is like asking, "If you start at the Empire State Building in New York City, which straight road leads you directly to Los Angeles?" The answer is "None" but that is not to say you can't get there from here. No single linear "path" goes from here to there, but big deal. The path hasn't even been created yet so of course it's not visible.
As Einstein said, “We can't solve problems by using the same kind of thinking we used when we created them.”
While we may appear to have a shortage of food, water, and power, one thing we do NOT have a shortage of is people. So the question is, are people a resource or a "sink". If they're viewed as a consumption "sink", yes, then, we are basically screwed.
On the other hand, if people comprise an untapped resource, then we are saved. I fully believe in the "saved" model.
The legacy "competition" model between people, and ever larger-scale groups of people, is what is bankrupt -- not the people themselves. That model does not, empirically, either lead to good solutions, or, in application, lead to very efficient utilization of available resources and opportunities. Easily 95% of all effort in most people-structures at any scale is wasted on internal squabbling.
The larger the structure, the more pronounced this system-effect is.
(An excellent and hilarious, but profound, discussion of the failure modes of large systems is Systemantics, by John Gall.
http://en.wikipedia.org/wiki/Systemantics
)
At the same time, very significant research and practice in the creation of high-performance, high-quality, ultra-high-reliability systems in aircraft cockpits, intensive care units, fire-fighting, and military situations have shown, repeatedly, that CORRECTLY organized teams based on COOPERATION and mutual-respect can and DO function spectacularly better than those based on the legacy model of competition.
In management terms, "Theory Y" organizations are profoundly more robust and sustainable than "Theory X" organizations, make better decisions by far, and are far more fun to work for.
http://en.wikipedia.org/wiki/Theory_X_and_theory_Y
I'd suggest focusing on the key elements of what is it that causes large systems to "go wrong" and become, de facto, corrupt, out of touch, dysfunctional and blind? And, can that be changed by a STRUCTURAL DESIGN change, such that the VERY SAME ELEMENTS (people) doing what appear to be the very same activities, now suddenly WORK instead of failing.
As systems thinkers, we know perfectly well that outcomes are as much functions of the STRUCTURE as the COMPONENTS. We know, from empirical examples, that high-reliability teams CAN be made out of low-reliability people, in the right context.
Are humans "incorrigibly selfish and mean"? Only in structural contexts that use feedback to push them into those behaviors. In the correct structural contexts, people are "incredibly strong, supportive, and capable of heroic and inspired action."
We, in systems thinking worlds, hold the keys to unlocking that buried power that is so vehemently and emotionally denied and attacked by proponents of "theory X".
===
Let me addend to my prior comment that there is an illusion that very rich people will oppose cooperation among underlings or people in general because they fear that, united, the oppressed will arise and overthrow them.
To some extent this is incomplete thinking and needs to be exposed. It is akin to saying "I'm glad I don't like ice cream, because if I liked it, I might eat it, and I hate it!"
If you take an incomplete model of a cooperative enterprise and treat it as "cooperation internally in order to COMPETE better externally", then, yes, be concerned, very concerned, if the staff compare notes and unify.
BUT, if you take a complete model, of "cooperation internally in order to be strong enough to cooperate externally as well", then there is no threat to the existing very rich.
Once people realize that we live in a sea of WEALTH, not a sea of shortage, then there is no motivation to organize in order to redistribute the existing wealth (i.e., revolt and take it away from someone else so that we have it, or to "distribute" it more "fairly" or "evenly.")
A revolution in deed, attempting to redistribute wealth, will certainly be met with hostility. This is not surprising.
A revolution in mind, realizing that more wealth can easily be created so there is neither need nor motivation to threaten someone else's existing wealth, can be met with at least neutrality.
In fact, creating wealth-creating structures of people could be WELCOME by the wealthy, who, frankly, are very hard pressed to know where on earth to INVEST a trillion dollars with any hope of getting a positive gain.
New human structures, a systems-approach to altering the outcomes of human behavior, does not require humans to become something they are not. There is no magic, other than the incredible power of structural feedback.
But I think it can help, from day one, if it is made clear that this is NOT a "revolution" with all that implies. It is an evolution, and a win-win approach to the future.
Wade
Friday, September 23, 2011
Zen of agile programming
There's an interesting thing called the "Agile Manifesto" which is a set of guidelines about how to write software that actually works and is actually helpful to customers.
I love the overlap with my favorite guru of systems, John Gall, in his book Systemantics: How Systems Fail, and his principle #15:
This is a good start of an idea, but like "Lean" and Six-Sigma and many other principles of good software design, there is a subtle but fatal flaw in it, IMHO.
With "Lean" process improvement, this flaw is encapsulated in usually a few short sentences that say something like "For this to work, a culture change is required". As if that's a rather trivial thing, like buying a toaster.
As I view it, Agile development, correctly done, works by using the immense power of a closed-feedback loop, giving you, in technical terms, an IIR system not an FIR system, which is, indeed, infinitely more powerful. (For details see my prior post Systems Thinking and the Emergence of New Life and "Gentle introduction to feedback loops" )
Also, it only works by plugging into the power of the living social context and staying very closely connected to it, again, as we would expect any new life form to do, ala a developing fetus in the womb.
Still, the key aspect to making this whole thing function is a "loop invariant" -- something that is destroyed in the middle of each loop, but is restored to its state before the loop exits, so that it appears to be "constant" viewed from outside the loop. In Agile Programming, this loop invariant has to include cleaning up all of the loose ends and "technical debt" -- everything that needs to be done to make the program "simple" again. In other words, after each step of increased complexity, the change has to be digested, and fully incorporated, and the whole shebang re-diagonalized back to "simple" again. This cannot be "left for later", critically.
Sadly, like "a culture change is required", "clean up after each step" is much easier said than done.
Others have seen this problem:
The problem here is that, in the creation of new life, it is absolutely unavoidable to go uphill in a thermodynamic sense, from a disordered state to a more ordered state. And the laws of thermodynamics forbid such a thing from occurring. Closed systems do not move uphill. Period.
Author T.S. Eliot captures the wished for state:
Well, more correctly, there is no way as a "closed system" we can do that. Only an open system, drawing in energy and revitalizing life from outside (as from a womb) can generate actual new life, actual steps up the entropy chain, actual "value" for the customer.
TECHNICAL DISCUSSION OF THE UNDERLYING MATHEMATICS
But ONLY one step can be taken, and then the whole matrix needs to be re-diagonalized again, because the underlying Riemann metric in which you are "hill-climbing" is only approximately flat, not actually flat. The approximation is only valid for one small step.
In less accessible but far more profound accuracy, the tensor you are working with, connected into physical and social reality, must be "rejuvenated", and returned to elegant simplicity by giving up something, by contraction across one covariant and one contravariant index, setting them equal.
The very choice of this word by a mathematician is no accident. A discussion of basic tensor algebra* is required to understand the precise meaning of these terms. ( We need to understand that concept, the step of "rejuvenation", to truly guide our insight into how agile development must occur to succeed.
This ratcheted hill-climbing algorithm will, in fact, work -- but only on its terms, not ours.
At least, that's how I understand it.
-----
* See, for example, as easy a discussion as you can find for non-mathematicians of tensor algebra in Introduction to General Relativity by Adler, Bazin, and Schiffer, chapter 1. (McGraw Hill, 1965.)
A less lucid reference, but one that is at least online, is Mathematics of classical and quantum physics, by Byron and Fuller,
I love the overlap with my favorite guru of systems, John Gall, in his book Systemantics: How Systems Fail, and his principle #15:
A complex system that works is invariably found to have evolved from a simple system that works.
This is a good start of an idea, but like "Lean" and Six-Sigma and many other principles of good software design, there is a subtle but fatal flaw in it, IMHO.
With "Lean" process improvement, this flaw is encapsulated in usually a few short sentences that say something like "For this to work, a culture change is required". As if that's a rather trivial thing, like buying a toaster.
As I view it, Agile development, correctly done, works by using the immense power of a closed-feedback loop, giving you, in technical terms, an IIR system not an FIR system, which is, indeed, infinitely more powerful. (For details see my prior post Systems Thinking and the Emergence of New Life and "Gentle introduction to feedback loops" )
Also, it only works by plugging into the power of the living social context and staying very closely connected to it, again, as we would expect any new life form to do, ala a developing fetus in the womb.
Still, the key aspect to making this whole thing function is a "loop invariant" -- something that is destroyed in the middle of each loop, but is restored to its state before the loop exits, so that it appears to be "constant" viewed from outside the loop. In Agile Programming, this loop invariant has to include cleaning up all of the loose ends and "technical debt" -- everything that needs to be done to make the program "simple" again. In other words, after each step of increased complexity, the change has to be digested, and fully incorporated, and the whole shebang re-diagonalized back to "simple" again. This cannot be "left for later", critically.
Sadly, like "a culture change is required", "clean up after each step" is much easier said than done.
Others have seen this problem:
There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult. (Tony Hoare)
Je n'ai fait celle-ci plus longue que parce que je n'ai pas eu le loisir de la faire plus courte.In my understanding, this issue gets down to a core issue that is the subtle flaw behind most good advice, such as "To keep your expenses under control, develop a budget and just live within it!." The word "just" in such sentences is a red flag that the complexity and difficulty of this step has been grossly understated. Similarly "Why can't you just have a place for everything and keep it in its place!"
- I would have written a shorter letter, but I did not have the time. (Blaise Pascal, circa 1650).
The problem here is that, in the creation of new life, it is absolutely unavoidable to go uphill in a thermodynamic sense, from a disordered state to a more ordered state. And the laws of thermodynamics forbid such a thing from occurring. Closed systems do not move uphill. Period.
Author T.S. Eliot captures the wished for state:
They constantly try to escapeIn short, the work of our hands must always reflect the state of our heart, and if we have a chaotic spirit, there is no way, regardless of tools, that we can build unchaotic systems, let alone give them a breath of life.
From the darkness outside and within
By dreaming of systems so perfect that no one will need to be good.
But the man that is shall shadow
The man that pretends to be.
(T.S. Eliot Choruses from the Rock)
Well, more correctly, there is no way as a "closed system" we can do that. Only an open system, drawing in energy and revitalizing life from outside (as from a womb) can generate actual new life, actual steps up the entropy chain, actual "value" for the customer.
Unless the LORD builds the house, its builders labor in vain. Unless the LORD watches over the city, the watchmen stand guard in vain. (Psalm 127, Solomon Song of Ascent)Summarizing this so far, for agile to realize the potential power, three things must be true:
- Only a truly elegantly "simple" system can evolve in a sustainable way, so that it will not ultimately crash under the weight of unresolved internal complexity or divergence from external life in which it is embedded. (All complex systems will suffer tissue rejection.)
- Each cycle of the loop, true simplicity must be rebuilt, at surprisingly great effort and expense.
- All efforts to rebuild simplicity will fail unless they are spiritually grounded in outside life, drawing heavily on it to move up the entropy curve.
TECHNICAL DISCUSSION OF THE UNDERLYING MATHEMATICS
But ONLY one step can be taken, and then the whole matrix needs to be re-diagonalized again, because the underlying Riemann metric in which you are "hill-climbing" is only approximately flat, not actually flat. The approximation is only valid for one small step.
In less accessible but far more profound accuracy, the tensor you are working with, connected into physical and social reality, must be "rejuvenated", and returned to elegant simplicity by giving up something, by contraction across one covariant and one contravariant index, setting them equal.
The very choice of this word by a mathematician is no accident. A discussion of basic tensor algebra* is required to understand the precise meaning of these terms. ( We need to understand that concept, the step of "rejuvenation", to truly guide our insight into how agile development must occur to succeed.
By repeating the rejuvenation step, the algorithm becomes truly recursive, and at each step a profoundly simple system can take one step, and only one step, safely, before everything must be recomputed and rejuvenated again. Expand and contract. Differentiate and reintegrate. There is no shortcut, no other path, no way around this thermodynamic truth.
But, because the algorithm is, in fact, completely recursive, you can get better and better at it, faster and faster, over time. The algorithm never changes, even though the software it is generating is, in fact, being continually migrated along a path in curved metric space towards some objective.
This ratcheted hill-climbing algorithm will, in fact, work -- but only on its terms, not ours.
At least, that's how I understand it.
-----
* See, for example, as easy a discussion as you can find for non-mathematicians of tensor algebra in Introduction to General Relativity by Adler, Bazin, and Schiffer, chapter 1. (McGraw Hill, 1965.)
A less lucid reference, but one that is at least online, is Mathematics of classical and quantum physics, by Byron and Fuller,
Subscribe to:
Posts (Atom)
And John Sterman in Business Dynamics notes that essentially all humans, even MIT grads (!), have very poor perception of what parts of their environment are due to feedback from their own prior actions.
I'll toss in my own two cents here, and add that the perceptual problems in LARGE organizations will always occur because of a fact left out of most models, namely:
*** Reality is scale-dependent ***
My background is in physics and I side with Einstein, almost always misundertood and misquoted, who affirmed that there WAS INDEED an underlying reality, but that even perfect unbiased observers would perceive it differently due to space-time-curvature, AND SO THEY WOULD NEED TO CORRECT FOR THAT IN ORDER TO COMPARE NOTES AND SEE THAT THEY AGREED ON THE UNDERLYING SINGLE REALITY.
So, assume that all humans have a finite capacity for information, a finite-radius of perception, and even within that must vastly oversimplify the flood of data to come up with one or more mental models they will attempt to "snap observations to" in order to make sense of what is going on and respond to it. (Basic cybernetic behavior.)
One example of scale dependence would be the question of whether an molecule of H20 ("water") is "free" to move or "captive". On the scale of molecules, the molecule is free to move about. On the scale of plumbing, the "water" in the "pipe" is captive to go from the water-tank on the hill OUT the faucet. Both are true, but they are different. It is wrong-headed to ask "WHICH is true?" It is right-headed to ask "How can both of these be true, and what do we need to correct for before we attempt to compare and align them?"
As organizations become larger, EVEN WITH PERFECT NON-DISTORTING communication by levels of "management", the nature of reality at the top must diverge from the nature of reality at the bottom, because they operate on different scales of space and time.
Since this effect isn't recognized, it leads to increasing friction between "management" and "front line staff" in which each group views the other as being increasingly "out of touch with reality". (Which is true, in a sense.)
It has nothing to do with "intelligence". Here's a more specific example - a "hybrid image" which has the property that if you view it from normal screen distance is "clearly" Albert Einstein, yet if you view it from across the room is "clearly" Marilyn Monroe.
http://newbricks.blogspot.com/2007/08/are-you-my-mommy-what-shape-am-i-anyway.html
If we have any hope of building very tall structures in social space (ie, corporations, governments, etc) then we should take into account that this effect will come into play RAPIDLY, and figure out how to make load-bearing feedback loops that take it into account.
Right now, what happens is that as the top and the bottom of the organization diverge in scope, animosity, blame, and finally fracture occurs. Pretty much every time, and pretty much everything we've ever attempted to build to large scale has collapsed along this axis.
Designing social architectures to take this fourth-dimension into account will require computer modeling, because we as humans are not very good at 3-D geometry, let alone 4-D dynamic feedback geometry.
The only clean solution I can see to this type of fractally complex problem is precisely encapsulated in how fractal shapes emerge from very simple recursive rules. Namely, reverse the process. Seek eigenvectors, as it were. Seek simple recursively-stable governance structures such that at each level the problems remain "the same shape." Then we break the height constraint.
Wade
Related links:
More on Hybrid Images (like "Marilyn Einstein")
http://newbricks.blogspot.com/2007/10/discussion-of-hybrid-images.html
"Hypnotized in High Places"
http://newbricks.blogspot.com/2009/10/hypnotic-trance-in-high-places.html
"Why we have so much trouble seeing"
http://newbricks.blogspot.com/2008/10/why-we-have-so-much-trouble-seeing.html
Blindness at the Top (bandwidth issues in central planning for crisis management)
http://newbricks.blogspot.com/2007/09/whats-wrong-with-decision-making-at-top.html
Unity and adaptation (design of control systems)
http://newbricks.blogspot.com/2007/11/adaptation-versus-control-and-army-fm-3.html
Unity and adaptation (part 2)
http://newbricks.blogspot.com/2007/11/unity-and-adaptation-continued.html
Why are so many flights delayed? (system factors leading to blame and conflict)
http://newbricks.blogspot.com/2007/07/why-are-so-many-flights-delayed.html
Baha'i Faith principles for developing a world government that would work as intended
http://www.bahai.com/Bahaullah/principles.htm
Why Blind and Stubborn Management is not a winning hand
http://newbricks.blogspot.com/2010/11/blind-plus-stubborn-is-not-winning-hand.html
Surprising case of Authority with listening - the US Army Leadership Manual
http://newbricks.blogspot.com/2006/12/us-army-leadership-field-manual-fm-22.html