New life forms from Synthetic DNA - Washington Post

The Washington Post today deals with "Synthetic DNA on the brink of Creating New Life Forms." Talk about children playing with matches... Rick Weiss begins " It has been 50 years since scientists first created DNA in a test tube..." I'd add - it has also been 50 years since Jay Forrester's classic piece on "unintended consequences."

Here was my reply:

wade2 wrote:

Bio-error indeed. Maybe error-gance is the bigger threat, and very real. Our social approach to low-odds of very-high-risk accidents, as Carl Sagan pointed out re return of samples from Mars, is completely overwhelmed by our normal intuition. At Los Alamos, the first atomic bomb was tested when only a minority of the scientists on the project (something like 6 of 14) thought it would detonate the earth's crust and explode the entire planet. No one was sure, so they tested it. Hmm.

Good books like "Lethal Arrogance" by Dumas and "Normal Accidents" by Perrow detail hundreds of examples of our tendency to run it till it breaks, and then, only then, stop to think.
The tools to even begin to think about the way coupled feedback-loops get their job done, such as System Dynamics, have languished for 50 years. MIT's John Sterman, in "Business Dynamics - Systems Thinking and Modeling for a Complex World" , details the lack of correct intuition, even for the MIT community, brighter than most. PhD's don't generally help, and most of us have less to work with.

So, at best we can model and simulate, which has been done at the Santa Fe Institute for the last few decades, with "artificial life" - virtual life and virtual DNA, genetic algorithms breeding and evolving, to see what happens. http://www.santafe.edu/ describes the work of many Nobel Prize winners.

In short (1) the little buggers are far smarter than we are and (2) parasitism evolves almost instantly in every case. The lesson of the movie Jurassic Park is a mild taste of the tenet "Life will find a way."

If the rest of our human affairs were measured and mature and stable, this would be a risky business. Having unstable tyrants convinced they must "master" this technology and use it to attack others, or defend from attack (exact same research), leads to the Russian model of stockpiling hundreds of tons of Anthrax or worse, in delusions that bio-warfare would be controllable or could be "won".

There are good odds the viruses and fungi and insects will win, not so good for humans.

Life is built with interactions with emergent properties on multiple levels, and we tend to think of "machines" at one level with only one function. But genes don't work like machines, they work like cooperative swarms.

Bio-warfare research has a "life of its own" that should already put us on alert that it is way easier to create things that "might as well be alive" than we think. Since we cannot stop it, we are committed to trying to get ahead of it and get the reins back, which means we should pour billions into understanding the world that the Santa Fe Institute has pioneered - massive interactions, how they go good, and how they go bad.

It becomes clear very quickly that, with complex systems, by the time you realize you "shouldn't have done that" it's too late. Experience is something that comes just after we need it.
For very high-stakes mistakes, that's too late. If we keep gambling with the whole planet on the table, sooner or later we'll lose one turn.

One is all it takes.

12/17/2007 6:07:22 AM
=========

Actually, all the research on high-reliability systems like nuclear power plant control rooms show that the maturity of the social system is what makes or breaks the technology-based system. Psychologically safe environments are needed for people to raise their hand, without fear of reprisal, and question what the heck is going on.

What we have instead is a whole culture used to using fear as a workplace and political context to "get things done", as described by Harvard Professor Amy Edmondson.

The Shuttle Columbia (picture at left) exploded because of an "o-ring" problem, that all the project engineers knew about, and had in fact gone in that day to tell the boss to tell the White House that it was too cold to launch safely. They all lost their nerve under workplace pressure to "deliver" so the Pres could talk to an orbiting teacher during the State of the Union address. She did, in fact, leave a message for us (picture at left) of what happens when we don't listen -- but, I guess we're still not learning that lesson.

Further reading

The classic paper in this field is Jay Forrester's congressional testimony:
"The Counterintutive Behavior of Social Systems",
https://mail.jhsph.edu/exchweb/bin/redir.asp?URL=http://web.mit.edu/sdg/www/D-4468-2.Counterintuitive.pdf

Quoting the abstract:

Society becomes frustrated as repeated attacks on deficiencies in social systems lead only to worse symptoms. Legislation is debated and passed with great hope, but many programs prove to be ineffective. Results are often far short of expectations Because dynamic behavior of social systems is not understood, government programs often cause exactly the reverse of desired results.

Another quote from the Washington Post article is this:

"We're heading into an era where people will be writing DNA programs like the early days of computer programming, but who will own these programs?" asked Drew Endy, a scientist at the Massachusetts Institute of Technology.

How true that is. I've been programming computers for over 40 years, and agree that the programs they write will be exactly like the "single-threaded" programs that mess up our airline reservations and everything else. In fact, a look inside some place like a hospital reveals the workings of the multiple legacy computer systems cobbled together in absence of any fundamental theory at all of how many interacting things should be structured in order to be reliable. Thirty years of research in computer science on "distributed operating systems" and how to build reliability in has had close to zero impact on the quick and dirty, cut-corners-now-and-we'll-debug-it-later model that vendors find locally profitable, but that always breaks down, producing, ta da!, more profitable rework. As a business model it's very popular; as a way of getting reliability, we all have seen the results. This is the culture we expect to "program" our genes? I'm not rushing to sign up.

The article quotes someone on the "unprecedented degree of control of creation" that the DNA technology gives us. Right. This is about the degree of "control" that a Labrador Retriever on your lap in the car at rush-hour has -- yes, it can turn the steering-wheel, but I wouldn't use the term "control" for what happens next. If you think our economy and business development and health care system are "under control", then maybe you would think genes could be "controlled" the same way - and they can, with about the same results.

Sadly, control requires maturity and depth of understanding, instead of simply strong muscles and a short attention span. I wish it were our strong suit as a nation, but see little evidence that it is, or even that it is valued or desired as a long-term goal.

We have instead young children playing with the cool gun they found in daddy's nightstand.

Oops.

======= Some after-thoughts:

Unlike the video games and computers this generation grew up with, life does not always have an "undo" button.

The core task of a civilization is to capture the wisdom we finally learn too late, and get it into a form that modifies the behavior of the next generation so those same lessons don't have to be learned all over again.

The hardest part of that task is that the next generation typically doesn't want to take advice from old people about situations the village elders seem way too concerned about - like, not going into debt over your head, you know, crazy stuff like that.

George Santayana said "Those who cannot learn from history are doomed to repeat it." I'd modify that slightly and add "Those who cannot learn from near-misses will someday not miss."

Each time we don't learn this, as a society, the costs go up. The biggest unknown in "the Drake Equation" about odds of there being other intelligent life in the galaxy that we could detect with radio is how long a civilization survives after it has gotten to the point where it has that much technology. The complete absence of any detectable signals from 100 trillion worlds "out there" suggests this is a pretty small number of years -- maybe under 200 years.

At the rate we're going, we're heading towards adding one more point to that data set.

Learning how to learn from our mistakes and our own past seems to be as important a problem as global warming, but actually more urgent, because time is running out a little faster on the 400,000 ways, besides global warming, that we can end human life on the planet.

Humans are remarkably inventive, and if every weapon and sharp object on the planet vanished, they'd find ways to attack each other with stones. Instead of tackling each symptom like global warming or genocide or terrorism, it would seem wiser to track further upstream and find the root-cause problem for why people are driven to fight, and fix that.

======================================

More further reading:

On High Reliablity organizations, which are sobering. They try really really hard to not have accidents, and still don't succeed from time to time:

http://www.highreliability.org/

I'm sure the US military tries very hard to keep nuclear weapons under control. Even that intense level of attention isn't enough to do the job 100% of the time, illustrating John Gall's law that "complex systems simply find complex ways of failing."

"Honey, I lost the nuclear weapons"

The US National Institutes of Medicine on how much the social relations of the front-line teams matter when your job is to get reliability in hospital care:

Crossing the Quality Chasm and other links

=========================
Photo credits :
Oops (car) by estherase
US Space Shuttle by Andrew Coulter Enright

Does ill-gotten gain produce wealth?

We know our eyes don't show us everything. The world around us is filled with WiFi radio waves, cosmic rays, infrared rays, all sorts of things we cannot see. We only perceive a very narrow part of the spectrum.

So, for the rest, we need to figure out how to tell what's there, and how to tell whether we are right or not. It's easy to make mistakes when you can't see what you're doing.

If "things" are hard to see and get right, "events" are even harder, and "causation" is harder still.

For "causation" we have to not only see two different things correctly, but also see the relationship between them correctly.

Now, we run into a second limit of human sight -- we can see most clearly when things are near us, and less clearly when things are farther away.

In fact, as the same "thing" gets farther away from us, it appears to get "smaller" to our eyes, in many senses. And, worse, the number of things at that distance just keeps on going up.

So, maybe there are 3 things going on right here, close up, highly visible. There may be 10 things going on over the space of a week with our friends or the stock market, somewhat farther away, and less visible. There may be 6 billion people on this planet, 6 billion lives and God knows how many relationships, farther away yet and so invisible we tend to forget about them. All that distant stuff becomes a big blur.

This kind of perception is great when the problems we face tend to be local as well, such as a snake on the ground or some berries that may be good to eat. But some important things are not local, and, as a species, in general, humans are pretty bad at managing those. "Out of sight, out of mind" is a good summary. First, they are far away, and we're dying close up, so they can be put off -- we figure, if we don't survive the short run, the long-run doesn't matter -- which is true.

Second, perversely, we tend to ignore distant events because there are so many of them. Our little brains get quickly overwhelmed, which is locally fatal. We miss the snake, or drive off the road while worrying about next year.

Still, when planning, we need to consider both local and distant events, because, in the long run, many things that seem far off now have a habit of suddenly being upon us. Suddenly, the final exam is here, or the term paper is due, or our mortgage payment is out of our "grace period" and is going to go up by $500/month. Suddenly the kids are grown and gone.

So, we need some basic simple rules to use here and now that will protect us in the long run, because trying to think about the long-run hurts and isn't easy to do and is often wrong anyway.

Many rules like that were distilled into the Book of Proverbs in the Christian Bible Old Testament, any many of those came from Egypt before that. They are worth reading and considering. This is a case where "the olds" may be more interesting than "the news."

As with all wisdom, some days the same words make more sense than other days, so we need to revisit them frequently and ask - "Now, from here, today, can I see something here that can help me?"

Here's one, from the Book of Proverbs, Chapter 10, verse 2, from a website BibleBrowser.com that has 20 other translations, including the Hebrew. This is part of the Judaic, Christian, and Islamic writings as well.

Ill-gotten gains do not profit, But righteousness delivers from death. (Proverbs 10:2)

For now, let's just look at the first part of that: "Ill-gotten gains do not profit."

This verse says something that is pretty important, if it's true -- First it says that there are two kinds of "gain", those that are well-gotten and those that are ill-gotten. By gain I believe this would include financial return, as well as outcomes of political maneuvers, military maneuvers, office politics, treatment of your neighbors, treatment on a larger scale of other countries, etc.

The verse asserts that the benefit of such "gains" depends on how they are obtained. It asserts that, even if the events that got you this "gain" is far away from here and now, it is still connected and still affects the benefit of having such a "gain."

Is that true?

Well, it certainly flies in the face of what we usually assume today. Let's simply look at cash, money. We assume a dollar is a dollar, and there is no difference between a dollar that was obtained by fraud or a dollar that was earned honestly. We assume that the origin of us having this dollar "goes away", and , being far away, no longer matters.

The verse in Proverbs I quoted differs, and says that assumption is wrong. It says that "blood money", money we got through bad actions such as theft or fraud, will turn out not to profit us after all, even though it "looks the same" as good money and is counted the same by our bank or phone company, that accept it as payment.

Besides, the form of this "gain" has been laundered, changed, altered many times, and it has passed through many hands, so even if the original dollar, say, was contaminated and "had the cooties", the electronic figure in our on-line checking account has surely separated us from such contamination, right?

Well, the verse says no, wrong. The verse doesn't say that changing the format of the "gain" makes this effect go away. It says that bad gains will result in bad things happening to you, and isn't worth doing, or worse, is actually a net loss to you.

Hmm. Well, first, things can't stay "connected" over a long distance, can they? Actually, they can. Physics tells us that a pair of particles can be "quantum entangled" so that, even if they are taken to opposite ends of the universe, they are still "connected" and if you change one by measuring it, say, that it instantly changes the other. So, there's at least one instance where distance in space is not a barrier to connection.

Well, still - we don't have to believe something just because some old book says it, do we? No, we don -- but we might consider thinking about believing it if someone had more current, solid, reliable data and evidence that such a principle is true.

Generally, physics doesn't suggest "physical laws" that relate physical things to human actions or intentions. That doesn't mean there can't be such laws, only that this isn't the sort of thing a physicist would ever be able to get a grant to study. This isn't "physics" but is something else, akin to both physics and commerce.

So, reflecting on this question, is there any other evidence that piling up fraudulent profits ends up being a bad idea? Maybe, rephrased, "Does crime pay, or not?"

It is clear that "ill" actions, can yield "gain", often immense "gain": ranging from theft to fraud to pillaging whole countries to making billions as merchants of death ( tobacco, heroin, etc.)

The question on the table, suggested but not proven by this old text, is whether such "gain" behaves differently over time than "gain" resulting from some more honest, socially beneficial wealth-building activity.

Are fortunes made through criminal activity less "stable", or less "valuable" to their possessors than fortunes made through hard honest work? Or not only less valuable, but of zero value, or, worse, of negative value?

Should we envy those those who pull off amazing con-jobs and sell drugs and "get rich"? Obviously, some get very rich. The question is, do they simultaneously get poor in some other way, so that, overall, net, it was a lousy life-choice and a poor business decision?

Given that many people and corporations and even nations are attempting to amass fortunes in this way, by ripping off other people, this is a big question. If nothing else, maybe we want to avoid owning stock in such companies, or not stand too near such people, waiting for lightning to strike or something? Hmm.

Oh, notice by the way that this verse I'm thinking about doesn't say money is bad, or wealth is bad, or profit is bad, or commerce is bad. It implies that profit is good, actually, and is a warning that someone long ago put there, trying to share what they saw as hard-won wisdom, telling us who seek profit that we should stay away from "ill-gotten gain", which , by implication, may look very attractive but turn out not to be in the end.

Reasoning about "impact on your soul" or "Heaven or Hell" is not very popular today, so we'll skip that line of thought. So, it comes down to this question:

Aside from such things that cannot be measured by science or commerce, are there things that can be measured by science or commerce that tell us "ill gotten gains " behave differently than "well-gotten gains?'

What exactly would we look for?

Here I want to bring in a board game called "Go", although the same thing is true for other games such as chess. At one time I was the organizer of the Cornell Go Club, which had several hundred members, about 30 very active, who played this game, and I learned a lot about it. Go was required knowledge of all Samuai soldiers in ancient Japan, and I actually advocated that Cornell's Johnson School of Management, where I was teaching, should include it as part of their MBA training. (That's a Go board from Wikipedia pictured at the top of this post.)

The game is one important way to gain insight into thinking and strategy that we can expect Asians, particularly Japanese and Chinese to use to compete with the US. There aren't very many such ways, so this is important.

One thing I found is that it was essentially always possible to defeat MBA students with a very simple strategy -- keep on trading them visible material short-run gain for longer-term much more valuable "position on the board." So, the MBA's would happily capture army after army, winning battle after battle, until suddenly they realized that they had lost the war. This seemed to be bait that such fish would always take.

The game Go teaches one patience and a long-range strategy. The aim of a good player is to "win by one point". Trying to win by more is overly-adventuresome, and requires taking risks that are not warranted. A balance is required between short-range, short-term tactics and long-range, long-term strategy -- but in the end, assuming one survives the middle, it is only the long-term value of each move, seen in hindsight from there, that matters.

So that may be an example where "foolishly-gotten short-range gain that you had to trade position for doesn't profit you in the end."

This is still not yet "ill-gotten gain", the topic of this post, but is getting closer, and the reflection on the proverb is uncovering some useful wisdom, whether the proverb is "right" or not.

So, let's remove that component and try to be more explicit and narrow in the hypothesis we're trying to test. Rephrased, we have two quesitons:

• Is there something hidden set into motion by socially-destructive gain that comes back to haunt the person or company that carried out that destructive action?
• If so, is it something that would also haunt other people downstream who got paid this money for goods or services or as a stock dividend?

There's no point in looking at the second part of that if the first isn't true, so let's focus on the first part. It seems safe to assume that such an effect was considered "hidden" by the writer of the proverb, because if it was obvious, they wouldn't need to go out of their way to write it down for their children to learn, nor preserve the thought over 3000 years.

Well, we know human vision is flimsy and has trouble seeing hidden connections between things, so we can't rule this out just because it is hidden and not obvious. Doesn't prove it, just doesn't rule it out.

And questions of "mechanisms" in my book can come later. First, blind to mechanisms, we can look and see if there is some effect. Then we can spend the energy to worry about mechanisms or pathways, and before then it's not justified. This seems scientifically valid to me. Besides, "That can't be true because I can't think of how it could happen!" has been used against every breakthrough in science, and doesn't prove anything.

First, we need careful, empirical, observational field-work to simply go see what is out there that needs explaining. In Toyota's "Lean" lexicon, this is Genchi Genbutsu -- actually going down and seeing for oneself instead of assuming you can see well from the executive suite or analyst's windowless cubicle.

I guess this comes down to the core question of theology and of this post:

Do your socially-destructive actions have bad consequences for you, even if you pull off a "clean getaway" and no one sees what you did?

People today seem think that, since the "God did it" mechanism is "dead" that all the social phenomena that mechanism explained can also be thrown away. "Allie Allie in-free!" as the child's game goes.

Two notes - first, I'm looking, as always, at "scale-invariant" rules , so the "you" in that question could be a person, or any larger entity that has "intent" such as a corporation or nation. Maybe this is relevant for cells and atoms, but that's a little obscure from here. (And I'm assuming intent matters, not just inadvertent (oopsie!) accidental, incidental, or "collateral damage", although that's not proven either.)

Do destructive actions have hidden bad consequences that make the visible benefits of them irrelevant or misleading?

OK, how would we test that hypothesis? Well, even though we have to look for cases that disprove it, we can start by asking if there's any case that comes to mind that agrees with it -- or else we should abandon this effort right here. The writer had something in mind, probably a lot of somethings distilled into this advice, so we should be able to think of something.

(to be continued...)