Linear thinking in a non-linear world

 kw: book reviews, nonfiction, prediction, forecasting, disciplines

My colleague at work had a poster: "Life is uncertain. Eat dessert first." We hate uncertainty, which is why we want to find out what is going to happen. We hate lack of control, which is why we try to control the future. Unfortunately, the future is unknowable and beyond our control. We aren't even very good at self-control. How could we hope to control a world composed of billions of others who are just as uncontrollable as ourselves?

What do you see here? Suppose this represents the general state of someone's health, projected over a long lifetime. Where is its end? If this shows the life of someone who is "fated" to live 85 years, they had a near-death experience around age 35.

Sliding toward the bottom of that valley—was it an illness, an accident?—, it may seem like the end is near and that good health is irretrievable. But, hey, things get a little better almost immediately, and later, a lot better. But that's not where these data came from.

Suppose instead it represents some factor of the economy. Is this part of a bigger picture? What comes after, and what came before? Did you notice that the end of the diagram is close to the level of the beginning, though a little bit lower? There are a couple of big, sustained "UPs", and a couple of big, sustained "DOWNs". In the middle of one of those sustained runs, whether up or down if you had to bet on the future of the rest of the diagram, how would you bet? Would you have any idea what the value would be at any point on this chart?

As a matter of fact, this is a financial chart, and adding the axes shows more context:

This is the Google Finance chart of the weekly closing price for Apple, Inc (AAPL), from 1/1/2021 to date (6/3/2021). It looks pretty dramatic! But check the left axis: the low for the year, in early March, was 116, the high in late January was 143, and the June 3 closing was 123.5 (pretty close to the average for the period of 128.6). If you knew just those figures, you could make a "bet with sideboards" that, for the short term, the stock price would be within ±11%. A final chart will show a little more context:

I downloaded the YTD historical data and charted it in Excel. When a stock price chart includes a zero, it looks less dramatic. From this perspective, this stock has been "flat" for five months.

How long will it stay "flat"? I couldn't hazard a guess. Do this: pick a time frame, such as a year, or five years, or (being more canny) the Midterm Election in late 2022 or the Presidential Election in 2024, and call that "the end of AAPL flatness." Write it down (or put a memo in your phone's calendar). Check it on that date and see if AAPL is still flat, or if it entered a period of serious gyration.

Now think about possible "black swans", such as someone buying Apple, or a competing product blowing the iPhone out of the market, or more globally, a new war, a recession, or a suddenly booming economy and the Dow Jones goes to 100,000. I don't have any idea how possible any of these things are, but if the possibility of any of them is more than a few percent, it would call any prediction about this stock or any other into serious question.

Consider this. The moment-by-moment price of a stock is influenced partly by the market as a whole, made up of investors that represent only a couple % of the population, and partly by the emotions of investors in that particular stock, which number a few dozen to a few hundred people. Wouldn't you expect that predicting the outcome of larger questions might be even harder? A year or five years from now, what will be the health of the economy of the nation or of many nations; what is the likelihood of war between two rival nations; or what of the effects of offshoring or onshoring a major segment of the work force? All of these depend on the decisions of thousands to millions of people!

About two months ago I reviewed a book about forecasting and "superforecasters", the kind of people who are better at making predictions that are at least a little more accurate than guesses. The author of that book referred to another, Future Babble: Why Expert Predictions are Next to Worthless, and You Can Do Better, by Dan Gardner. I spent the past two weeks reading this book, with some care. I think it best to begin with a few statements that caught my eye:

On Behavioral Economics, "In the 1950s, Solomon Asch, Richard Crutchfield, and other psychologists conducted … experiments that revealed an unmistakable tendency to abandon our own judgments in the face of a group consensus—even when the consensus is blatantly wrong. …three-quarters of test subjects did this at least once." (p. 99 in Chapter 4)

On "predicting 'more of the same' ", "…[such] predictions are more likely to be right when current trends continue and least likely to be right when there is a drastic change. That's most unfortunate, because when the road is straight, anyone can see where it is going. It's the curves and corners that cause crashes. …predictions are most likely to be right when they are least needed and least likely to be right when they are essential." (p. 105 in Chapter 4)

On Chaos, or the unexpected influence of small differences, "Like most of what's interesting in life, weather is subject to chaos and all sorts of nonlinear weirdness that limits how far we can peer into the future. Those limits will never be eliminated." (p. 105 in Chapter 8)

The first item above refers to "groupthink", which led to the disastrous Bay of Pigs "invasion" of Cuba early in the Kennedy administration, and to the under-planning of a tragic rescue attempt in Iran during the Carter administration. Do note the successful rescue operation in 1979, of two EDS employees being held in Iran, an operation ordered and sponsored by H. Ross Perot: his decision, carried out under the command of one experienced "hired gun". No groupthink.

Allied to this is the need for certainty, which means the most confident (and loud) voice typically carries the day. But turn the last phrase of the around: one-fourth of the experimental subjects did not succumb to groupthink. Their voice may not have prevailed to "turn" the others (who were all confederates of the experimenters), but they did not give up their view. In the terminology used by Gardner, they were probably "foxes", as in the proverb, "The fox knows many things, but the hedgehog knows one important thing." As one might expect, the hedgehog's "important thing" may or may not be correct, but when it is wrong, the hedgehog will not change his mind. Foxes are more flexible. They are willing to consider whether their thinking is wrong. Using the same analogy, the "superforecasters" I wrote about in April are foxes.

Let us consider the idée fixe, or fixed idea: an idea or desire that occupies one's mind, often to the point of obsession. This is the hedgehog's specialty. It takes work to break free from an obsession, but successful forecasters are willing to do the work, and the extra work to gather knowledge from numerous sources (the fox's "many things"). However, foxes have a hard time getting through to the public, which prefers the certainty of hedgehogs, no matter what their track record (which is uniformly abysmal except for the occasional lucky guess).

I recall that the root word for "fool" in the book of Proverbs is "self confident". It is used about 70 times. Not only are hedgehogs such fools, so are all those who listen to them uncritically.

For the second quote, I'd call it self-evident, but we all have "hindsight bias", which is its basis. There is a road I sometimes take to work. It goes over a rather steep hill. Just at the top, the road jogs just a little to the right, which means if you aren't paying close attention, you will suddenly be halfway into the other lane. A great place for head-on collisions. It would be better if the roadway on either side of the hill were curvy, forcing drivers to pay better attention. Even better if the highway department tore out fifty feet on either side of the crest and smoothed that transition.

Life doesn't usually provide smooth transitions. Look again at the stock charts above. Any two points more than a tenth of an inch apart would seem to have nearly no relationship to each other. In a few cases, a tenth of an inch is enough for a rather dramatic swing. The basic rule of thumb we need to guide us is, "The more confident an expert sounds, the less we should trust the prediction." Let me repeat that:

The more confident an expert sounds,
the less we should trust the prediction.

As fun as it would be to belabor the third quote, about Chaos, I'll forbear. I realize that only the "choir" would take such a word, and everyone else would ignore it. Instead let's realize that Chaos is a manifestation of Nonlinearity. We have linear minds. From many years as a working mathematician, I know how hard it is to think nonlinearly. It isn't natural to us. Even when we know something is cyclical, like the march of the seasons each year, the day-to-day variations of sunlight, clouds, rain and high or low temperature are tough to predict. The best weather forecasting computers do OK for a day or two. After that all bets are off.

Mark Twain wrote about a period of around 100 years in which the Mississippi River's length was reduced by dozens of miles, because a few long, loopy bends in the riverbed were cut off when the river flooded and cut shortcuts. He said that if the trend continued, in another couple of centuries St. Louis would be practically a coastal town on the Gulf of Mexico, and that millions of years ago the river must have been sticking out over the Gulf "like a fishing rod." 

We know that is illogical, and we laugh at it. But we neglect to consider that seemingly regular trends don't persist. Any measure we might look at is more like the stock chart above, with ups and downs that nobody could predict. Over the entire five months charted, the stock price went from 129.4 to 123.5. That's about a 5.5% reduction. But there were a couple of spots in there where a day trader could have gained 10% or more in a week's time, and a couple of others where a day trader would have had to ride out a 10% loss.

I apologize if I keep returning to the stock market as an example. It is an area I'm familiar with. It is seldom referred to in Future Babble. The author's interest is in the way we react to predictions and the way we almost always forget that they very rarely pan out. Most pundits are so wrong so frequently, it is amazing that anybody buys their books. So I'll end with a prediction: 

The pundits and "experts" who are the best at sounding confident will get rich on the gullibility of a public that craves certainty where there is none to be had.

Seeing farther, maybe

 kw: book reviews, nonfiction, forecasting, disciplines

Forecasting means different things in different realms. "Prediction is hard", said Yogi Berra, "especially about the future." If you're on a high hillside, looking down at a bend in a river, and a canoeist is paddling industriously downstream, you can see pretty well what he has in store, at least for the next few hundred yards. If there's a waterfall around the bend, it isn't hard to predict that he'll be in trouble if he doesn't soon pull to shore. But there's another factor: You don't know his plan. He may know the waterfall is there, and he is either planning to go ashore and portage around it, or he may have a plan for going over, either because he's really good at riding the rapids, or because he is suicidal. He may not know it is there, and is just exploring, in which case you can hope he has good hearing. Perhaps you can think of other possibilities.

Suppose instead you were asked, "How likely is it that Iran will produce a nuclear weapon in the coming 12 months?" Assuming you can find people without a political investment in that question, and you ask ten of them this question, will you get similar answers, or will they range from, "No way, nohow!" to "I am certain they will."? You may get at least one person who says, "Maybe." I think you are most likely to hear, "How should I know?"…but there I am, making a sort of forecast myself!

Then there is the weather, and there is the climate. When a weather forecaster says, "70% chance of rain tomorrow," what does it mean? Is she talking about your city, your neighborhood, or the whole state (easier to contemplate about Rhode Island than about Texas!)? Let's assume it is a local forecast. Decades ago a weatherman explained on a radio program, "70% chance of rain means that 70% of the area will get rained on." I wonder if that is still true? In these days of supercomputers producing a fresh forecast over the whole earth about every hour, it may mean something else. For example, it may mean that when they run the forecasting software over and over again with tiny adjustments to the initial conditions, 70% of them predict rain in "your area". Or it may mean, of the several hundred supercomputers being used with numerous versions of the modeling software, 70% of them predict rain. Or it may still mean "It will rain over 70% of the several square miles surrounding such-and-such a place."

That last version, akin to the version of half a century ago, is probably still what they mean. The prediction is often based on a squall line passing through. A loose squall line will produce a string of small storms that drop rain for a few miles between formation and dissipation. That might mean 30% to 50% of the area will get rained on. A tighter line will have larger storms, closer together, and we're in the 70% range. A heavy squall line will bring rain almost everywhere, and the forecast is 100% rain.

What about climate? It is the average of "weather" over a period of decades. You can't tell from one year to the next whether changing weather patterns mean the climate is getting warmer or cooler, dryer or wetter. I recently saw an article about cherry blossoms in Kyoto, Japan. They are blooming early this year, during the first week of April (today, the 7th is the expected peak). The article stated that the "usual" peak is between April 10-17. Of course, "global warming" was mentioned. That's funny. Kyoto is south of Tokyo and warmer, and the cherry trees there bloom a few days earlier. We visited the Tokyo area from March 28 to April 11, 1992. The peak of the cherry blossoms was March 31. I presume the peak in Kyoto was closer to March 25. With that in mind, if this year's peak in Kyoto is more than a week later than the peak thirty years ago, neither datum means much about a changing climate.

Back to Iran. There is a reason for stating the question like I did, particularly "in the coming 12 months". The political winds are shifty. If the time frame were 10 years, the answers would, I hope, be different (maybe not!), and much less certain. Too many things can happen in a decade. Lots happens in just one year! But the study and research one would need to do to answer that question are, just barely, possible. Having studied and formed an answer, though, how good is it?

"How good is it?" is the subject of the work of Professor Philip E. Tetlock. He has conducted experiments with numerous forecasters for the past few decades, and has discovered factors that make some people better than others at it. Some are "superforecasters", and with Dan Gardner, he was written Superforecasting: The Art and Science of Prediction.

It so happens that superforecasters have certain characteristics. Super-intelligence is not one of them. Of course, they are intelligent, but the important factors include the humility to question their own assumptions and premises, the willingness to break out of false dichotomies, and the diligence to do lots of study and research. The book has a couple of lists of these factors, which in themselves constitute a good introduction to learning to make better predictions…if you're willing to do the work.

We have to forecast to be able to plan. I have two friends I should mention here. One expects the country to descend into anarchy, and soon. It's not just about President Biden, because he expected the same thing when Donald Trump was President. This conviction colors his thinking. His long-range plans include getting rich enough quickly enough to afford land in some out-of-the way place.

The other friend is older and has a longer view. Remembering the decades of the Vietnam War, and the times that preceded it, and those that came after, he is more optimistic. He also likes suburban living. His long-range plans are very different, including his expectation of a long retirement filled with volunteering and service. If you were to place these two men in almost any situation you choose, unless you know their backgrounds and attitudes rather well, it would make no use to make forecasts about the situation itself. Their own internal landscape would be more important than the external. I dare say, if you were told just one thing about these two men, that one is a paranoid prepper, the other a contented suburbanite, it could greatly improve the accuracy of your forecast. Maybe.

That is a big maybe. Dr. Tetlock found that for most people, if someone had already researched a situation and made a prediction, new information was unlikely to have much effect. Superforecasters, by contrast, would frequently incorporate new information and revise a forecast. Some would do so daily or oftener.

It is helpful to remember that forecasting is like other skills, that practice is needed. To learn to forecast, make lots of forecasts. BUT make them in a particular way: the result needs to be quantifiable. Don't say, "There is a significant chance of X happening." To you, "significant" may mean "a 75% or better likelihood", to someone else, 15%, and to others, as little as 5% or even less. If 75% is what you mean, state it that way. Then keep score. One result isn't too useful. It takes three to see the inkling of a trend, and twelve to begin to do decent statistics. The book contains basic instructions on calculating a Brier Score, a measure of a forecaster's accuracy. It is also useful to learn to use Bayesian methods of revising a prediction. Superforecasters that revised their estimates didn't indulge in large adjustments; they were more likely to change a 40% likelihood to 42% or 37%.

It is also helpful to know that these methods are all about the "what" and sometimes the "how" of something. The "why" of something is outside the realm of science or technology. It is frequently more of a theological matter.

Dr. Tetlock is noble enough to introduce a monkey wrench into this whole matter, by discussing his relationship with Nassim Taleb, author of The Black Swan, a book I reviewed in 2007. The idea of a Black Swan is something nobody could have predicted, but it changes everything. Fourteen years ago I was quite taken with the notion. Now I have a more nuanced view. Dr. Taleb contends that because Black Swans can't be predicted, one cannot account for them, and because they matter more than other events, nothing that can be predicted is going to matter enough.

One of the conceptual ills we encounter is either-or thinking. The idea that Black Swans make forecasting useless is either-or thinking. A superforecaster will weigh the possible influence of something unprecedented throwing everything else out the window, without needing to know just what that might be. This is the realm of the "unknown unknowns" of Donald Rumsfeld. This is also why, as stated in The Art of War (I paraphrase), "No battle plan survives contact with the enemy." Yet a good battle plan puts you in a position to re-evaluate and succeed anyway. I find it fascinating that neither Dr. Tetlock nor Dr. Taleb points out that as historical Black Swan examples accumulate, they allow us to re-adjust our knowledge of the expected range of possibilities, to take a better view of the distribution of possible events, and so make better forecasts and more robust plans. We just need to have or to gain the wisdom to discern whether a particular Black Swan is truly unprecedented, or if it is a case of an extreme event that happens more frequently than we'd been led to believe.

For example, the original Black Swans were discovered in Australia in 1697. They were not new to Australians, of course. They were new to Europeans, who knew only white swans. The Western white swan, Cygnus olor, and the Asian black swan, Cygnus atratus, are related species in the genus Cygnus, but have been separated by half a world for tens of thousands of years. There are four other species in the genus Cygnus, including the black-necked swan, C. melancoryphus. Its existence could have clued the Europeans into the possibility that swans somewhere else on Earth could be black, but it did not.

Other examples used to illustrate Black Swans include stock market booms and busts, and extreme flooding events.  Just last year I discussed a series of mega-floods that caused large boulders to move across a flood plain south of Sturgis, SD. The context, however, was discussing a new way to analyze daily (or weekly or even monthly) changes in the price of stocks. Considering the floods, I realize that the extreme flooding events in the Black Hills may not be extreme examples of the usual flood frequency regime. They are just as likely to be symptoms of extreme weather events that just "work differently" from the ones that occur yearly and decade-to-decade. It wasn't just a thousand-year flood that moved 20-ton boulders half a mile or more. It may have been a 5,000-year flood, except that this happened twenty times in 5,000 years. So something different was happening, and I don't really know what it was. However, we now know that floods of that magnitude occurred about every 250 years in the past, so it's best to take that into account if you want to build anything on the outwash plain east of the Black Hills.

How could you become a superforecaster? An appendix in the book describes the basic skills, and another invites any reader to join the Good Judgment Project, at www.goodjudgment.com. Enjoy!