Friday, June 04, 2021

Linear thinking in a non-linear world

 kw: book reviews, nonfiction, prediction, forecasting, disciplines

My colleague at work had a poster: "Life is uncertain. Eat dessert first." We hate uncertainty, which is why we want to find out what is going to happen. We hate lack of control, which is why we try to control the future. Unfortunately, the future is unknowable and beyond our control. We aren't even very good at self-control. How could we hope to control a world composed of billions of others who are just as uncontrollable as ourselves?

What do you see here? Suppose this represents the general state of someone's health, projected over a long lifetime. Where is its end? If this shows the life of someone who is "fated" to live 85 years, they had a near-death experience around age 35.

Sliding toward the bottom of that valley—was it an illness, an accident?—, it may seem like the end is near and that good health is irretrievable. But, hey, things get a little better almost immediately, and later, a lot better. But that's not where these data came from.

Suppose instead it represents some factor of the economy. Is this part of a bigger picture? What comes after, and what came before? Did you notice that the end of the diagram is close to the level of the beginning, though a little bit lower? There are a couple of big, sustained "UPs", and a couple of big, sustained "DOWNs". In the middle of one of those sustained runs, whether up or down if you had to bet on the future of the rest of the diagram, how would you bet? Would you have any idea what the value would be at any point on this chart?

As a matter of fact, this is a financial chart, and adding the axes shows more context:

This is the Google Finance chart of the weekly closing price for Apple, Inc (AAPL), from 1/1/2021 to date (6/3/2021). It looks pretty dramatic! But check the left axis: the low for the year, in early March, was 116, the high in late January was 143, and the June 3 closing was 123.5 (pretty close to the average for the period of 128.6). If you knew just those figures, you could make a "bet with sideboards" that, for the short term, the stock price would be within ±11%. A final chart will show a little more context:

I downloaded the YTD historical data and charted it in Excel. When a stock price chart includes a zero, it looks less dramatic. From this perspective, this stock has been "flat" for five months.

How long will it stay "flat"? I couldn't hazard a guess. Do this: pick a time frame, such as a year, or five years, or (being more canny) the Midterm Election in late 2022 or the Presidential Election in 2024, and call that "the end of AAPL flatness." Write it down (or put a memo in your phone's calendar). Check it on that date and see if AAPL is still flat, or if it entered a period of serious gyration.

Now think about possible "black swans", such as someone buying Apple, or a competing product blowing the iPhone out of the market, or more globally, a new war, a recession, or a suddenly booming economy and the Dow Jones goes to 100,000. I don't have any idea how possible any of these things are, but if the possibility of any of them is more than a few percent, it would call any prediction about this stock or any other into serious question.

Consider this. The moment-by-moment price of a stock is influenced partly by the market as a whole, made up of investors that represent only a couple % of the population, and partly by the emotions of investors in that particular stock, which number a few dozen to a few hundred people. Wouldn't you expect that predicting the outcome of larger questions might be even harder? A year or five years from now, what will be the health of the economy of the nation or of many nations; what is the likelihood of war between two rival nations; or what of the effects of offshoring or onshoring a major segment of the work force? All of these depend on the decisions of thousands to millions of people!

About two months ago I reviewed a book about forecasting and "superforecasters", the kind of people who are better at making predictions that are at least a little more accurate than guesses. The author of that book referred to another, Future Babble: Why Expert Predictions are Next to Worthless, and You Can Do Better, by Dan Gardner. I spent the past two weeks reading this book, with some care. I think it best to begin with a few statements that caught my eye:

On Behavioral Economics, "In the 1950s, Solomon Asch, Richard Crutchfield, and other psychologists conducted … experiments that revealed an unmistakable tendency to abandon our own judgments in the face of a group consensus—even when the consensus is blatantly wrong. …three-quarters of test subjects did this at least once." (p. 99 in Chapter 4)

On "predicting 'more of the same' ", "…[such] predictions are more likely to be right when current trends continue and least likely to be right when there is a drastic change. That's most unfortunate, because when the road is straight, anyone can see where it is going. It's the curves and corners that cause crashes. …predictions are most likely to be right when they are least needed and least likely to be right when they are essential." (p. 105 in Chapter 4)

On Chaos, or the unexpected influence of small differences, "Like most of what's interesting in life, weather is subject to chaos and all sorts of nonlinear weirdness that limits how far we can peer into the future. Those limits will never be eliminated." (p. 105 in Chapter 8)

The first item above refers to "groupthink", which led to the disastrous Bay of Pigs "invasion" of Cuba early in the Kennedy administration, and to the under-planning of a tragic rescue attempt in Iran during the Carter administration. Do note the successful rescue operation in 1979, of two EDS employees being held in Iran, an operation ordered and sponsored by H. Ross Perot: his decision, carried out under the command of one experienced "hired gun". No groupthink.

Allied to this is the need for certainty, which means the most confident (and loud) voice typically carries the day. But turn the last phrase of the around: one-fourth of the experimental subjects did not succumb to groupthink. Their voice may not have prevailed to "turn" the others (who were all confederates of the experimenters), but they did not give up their view. In the terminology used by Gardner, they were probably "foxes", as in the proverb, "The fox knows many things, but the hedgehog knows one important thing." As one might expect, the hedgehog's "important thing" may or may not be correct, but when it is wrong, the hedgehog will not change his mind. Foxes are more flexible. They are willing to consider whether their thinking is wrong. Using the same analogy, the "superforecasters" I wrote about in April are foxes.

Let us consider the idée fixe, or fixed idea: an idea or desire that occupies one's mind, often to the point of obsession. This is the hedgehog's specialty. It takes work to break free from an obsession, but successful forecasters are willing to do the work, and the extra work to gather knowledge from numerous sources (the fox's "many things"). However, foxes have a hard time getting through to the public, which prefers the certainty of hedgehogs, no matter what their track record (which is uniformly abysmal except for the occasional lucky guess).

I recall that the root word for "fool" in the book of Proverbs is "self confident". It is used about 70 times. Not only are hedgehogs such fools, so are all those who listen to them uncritically.

For the second quote, I'd call it self-evident, but we all have "hindsight bias", which is its basis. There is a road I sometimes take to work. It goes over a rather steep hill. Just at the top, the road jogs just a little to the right, which means if you aren't paying close attention, you will suddenly be halfway into the other lane. A great place for head-on collisions. It would be better if the roadway on either side of the hill were curvy, forcing drivers to pay better attention. Even better if the highway department tore out fifty feet on either side of the crest and smoothed that transition.

Life doesn't usually provide smooth transitions. Look again at the stock charts above. Any two points more than a tenth of an inch apart would seem to have nearly no relationship to each other. In a few cases, a tenth of an inch is enough for a rather dramatic swing. The basic rule of thumb we need to guide us is, "The more confident an expert sounds, the less we should trust the prediction." Let me repeat that:

The more confident an expert sounds,
the less we should trust the prediction.

As fun as it would be to belabor the third quote, about Chaos, I'll forbear. I realize that only the "choir" would take such a word, and everyone else would ignore it. Instead let's realize that Chaos is a manifestation of Nonlinearity. We have linear minds. From many years as a working mathematician, I know how hard it is to think nonlinearly. It isn't natural to us. Even when we know something is cyclical, like the march of the seasons each year, the day-to-day variations of sunlight, clouds, rain and high or low temperature are tough to predict. The best weather forecasting computers do OK for a day or two. After that all bets are off.

Mark Twain wrote about a period of around 100 years in which the Mississippi River's length was reduced by dozens of miles, because a few long, loopy bends in the riverbed were cut off when the river flooded and cut shortcuts. He said that if the trend continued, in another couple of centuries St. Louis would be practically a coastal town on the Gulf of Mexico, and that millions of years ago the river must have been sticking out over the Gulf "like a fishing rod." 

We know that is illogical, and we laugh at it. But we neglect to consider that seemingly regular trends don't persist. Any measure we might look at is more like the stock chart above, with ups and downs that nobody could predict. Over the entire five months charted, the stock price went from 129.4 to 123.5. That's about a 5.5% reduction. But there were a couple of spots in there where a day trader could have gained 10% or more in a week's time, and a couple of others where a day trader would have had to ride out a 10% loss.

I apologize if I keep returning to the stock market as an example. It is an area I'm familiar with. It is seldom referred to in Future Babble. The author's interest is in the way we react to predictions and the way we almost always forget that they very rarely pan out. Most pundits are so wrong so frequently, it is amazing that anybody buys their books. So I'll end with a prediction: 

The pundits and "experts" who are the best at sounding confident will get rich on the gullibility of a public that craves certainty where there is none to be had.

No comments: