One of the things I am trying to teach my 8th graders is that some apparent cause and effect links are not real, because we mistake proximity for causality. That is, sometimes the observed cause that immediately precedes the effect has nothing to do with it, while the actual cause, because it occurred a long time ago or far away (was not proximate in time or space), is invisible on a human scale. I assure you I do not use the words proximity and causality when I try to teach the concept.
Anyway, a few things stuck with me over the years since I graduated from MIT with a B.S. in Management. One was the idea that human beings (including managers) are really pretty awful at predicting the consequences of their actions in complex systems.
Another was the law of unintended consequences--your action may or may not have the effect you intend, but it will certainly have effects you didn't intend or foresee. Frequently, actions result in effects that are the opposite of those intended. Often, this is because people fail to recognize that systems are complex
Finally, optimization of individual decisions does not necessarily produce a globally optimized result. The most famous example is commuting to work by car. For every individual commuter, driving a car instead of riding mass transit is an optimized decision (in terms of convenience, and, theoretically, time). But the global effect is sub-optimization, because the roads become so crowded that the time convenience and time advantages are wiped out, or even become negative. The same thing happens in industry; each department optimizes its own operations, but sub-optimizes corporate performance (read Dilbert for a month to see what I mean).
Jay W. Forrester, Professor of Management, Sloan School, Massachusetts Institute of Technology, recently shared his ideas on these topics with a mailing list of which I am a member. I am not in the habit of running lengthy expositions on management, but I was struck by the clarity, and clarity of expression, evinced in this essay, and Prof. Forrester asked me to print it in full, rather than excerpting it. He concludes with a highly accurate speculation on the No Child Left Behind Act.
There have recently been several communications related to the objective of a system. I think most of the discussion is not anchored in the structural nature of systems. A company may have an "objective" of increasing profit, or a school system may have an objective of providing better education, but these so-called objectives are not connected to the system behavior.
In a model of one of these systems, every flow, of which there may be large numbers has imbedded in it an objective. The person ordering replacement parts has an objective of maintaining a certain level of inventory. The school superintendent has various objectives arising from the various flows that are being controlled--hiring teachers to fill vacancies, communicating with the public to limit criticism, putting pressure on teachers when discipline falters. Every flow has some kind of explicit or implicit goal embedded in the statement that describes the control of the flow.
Super goals of better performance, happier students, or higher profitability, have no systems meaning or significance except to the extent that they lead to altering local goals at specific flow-controlling points. Often the super, or grandiose, goals have no effect. Frequently, such general goals have the opposite of the desired result because they create pressures on the operating flows that are counterproductive.
The global goals that the discussions have cited are often meaningless or produce the opposite of the desired result because there is usually no grasp of where the high-leverage flows are located. It is only through actual model simulation that one discovers how the multitude of local flows, and their separate goals, cause the total system behavior.
When the behavior of a system is not what is desired, one should start by assuming that the deficiency is because of the policies being followed and often because of policies that people are actively pursuing in the mistaken belief that those policies lead to improvement. One sees corporations, and school systems, in which the known policies that are thought to be a solution are, without people being aware of the consequences, causing the problem. This is a treacherous situation because the favored policies cause increasing difficulty and the greater difficulty provides more incentive for doing the very things that are causing trouble. Is "No Child Left Behind" one of these?
And from a second message to the same group:
I believe we are dealing here with a fundamental nature of systems and how they interact with people. A person's intuition and judgement are conditioned by experiences with simple systems. Simple systems are the only ones that we clearly understand. Unfortunately, the lessons learned from simple systems are contrary to the behavior of complex systems and those inappropriate lessons lead people in the wrong direction when faced with the complexity of almost all of the important systems in our lives.
As one of several classes of misleading teachings of simple systems we learn that cause and effect are closely related in time and in space. If one touches a hot stove, the pain is now and the location is here; cause and effect are obvious and close together in both time and space. But this lesson is wrong in more complex systems. In larger real-life systems, the cause of a symptom may be far back in time and may come from an entirely different part of the system. However, to make matters more misleading, the complex system will present what appears to be a cause that is close in time and space, but that apparent cause in only a coincident symptom of the real cause. People are then led to deal with what they see as the apparent cause rather than finding the true cause.
To suggest, without being sure because I have done no modeling in this area, examine the national preoccupation with low standards of performance in our K-12 schools, especially in mathematics and writing. The solution appears to be more pressure on teaching the material. But is the teaching the cause, or is the material the cause. Several writers have commented on how the problem in education arises from the students being bored, but without followup to determine why it is boring and how to eliminate boredom. Does one solve the problem with more pressure to learn what is boring, or is the challenge to have an education process that is exciting and relevant and which students want to penetrate deeply?
In addition to the above matter of causes shifting from immediate in time and space for simple systems to being remote in both time and space for complex systems, there are several other kinds of ways that systems mislead people.