Creeping toward the Matrix

 kw: book reviews, nonfiction, cosmology, simulation, modeling

I was for several years, in my career of writing scientific software, the leader of a "Modeling and Simulation Group". One of our products had three sections, simulating first the geochemistry of crude oil generation from organic matter in deep rocks (up to a few kilometers), then the upward migration of the petroleum liquids through porous rocks, and finally their entrapment against nonporous, or less porous, rock layers to form oil and gas reservoirs.

I was sent to a few exploration offices to show off the software. In one instance, after the geologist set up access to a set of grids based on seismic data, I ran the software, which displayed the progress through time of oil and natural gas collecting under the trapping layer some half-kilometer beneath our feet. At the end of the run, he pointed to one green blob on the map, saying, "This is X field," and to another, "This is Y field." Then he pointed to a third one between them, asking, "But what is that?" I answered, "That could represent a lot of money." As it happened, the company had decided to sell that property to another company. That oil company made the money! But the software found the oil before the property was drilled. Later that year I spoke about the experience at a Research Review. My talk was titled, "Finding Oil in a Computer."

It was with great relish that I read The Universe in a Box: Simulations and the Quest to Code the Cosmos by Andrew Pontzen. If you buy the book feel free to download this image to print a bookplate; it's 1024x1024 px. Use Upscayl or something similar if you want it rendered at higher resolution. I produced it using Playground AI; the only prompt was "Cosmology". I tinkered with Samplers and other parameters, looking for something else. Getting this image was a side benefit.

The author could have delved deeply into sundry technical issues—there are many! Instead, he has skirted these, providing just a taste of some of them, in favor of the philosophy and motivations for making computer simulations of natural phenomena.

The terms "simulation" and "modeling" have overlapping meanings. In principle, a Model is the framework and the sets of parameters that define the physical structure and the physics rules to be followed, while a Simulation is the operation of the Model over a chosen span of time, producing a series of output data sets that describe the expected physical state of the modeled "thing" at one or more points in time, whether past or future. Note that a simulation can be done on equipment other than a computer. One story of the book is about a galaxy simulation done with light bulbs and photocells and a glorious tangle of wires.

Weather forecasting is one very visible result of computer simulation, seen daily (or hourly) on newscasts and in the various weather apps on our devices. There are a couple of dozen important models used by weather agencies the world over. One expression of these is the Spaghetti Plot of a hurricane's forecasted track, as produced by several models. The models differ in the importance they place on various aspects of the modeled system, including whether it represents the whole Earth or a hemisphere, or a couple of continents.

All weather models are based on a global General Circulation Model, in which the atmosphere and the land and sea surfaces in contact with it (and sometimes a surface layer of the ocean) are divided up into roughly ¼ to ½ million quasi-rectangular portions. Half a million to a million "cells" is about the most that modern supercomputers can handle. In general, spatial resolution is likely to be as large as 200x200 km! The Earth's surface area is about 127 million km², and the models have between 20 and 40 vertical layers (at present). A 40-layer model would have more than five billion cells of 1x1x0.5 km, so to get the count below one million requires using cells with an area of more than 5,000 km², which is 71x71x0.5 km, and most models are set up for grid squares of about 100x100 to 200x200 (and half a km thick). The physics rules, primarily those relating to pressure and temperature relationships, are applied at the boundaries between grid cells.

To get a "future radar rain" map with finer detail requires using "sub grid" rules, and partial simulations over short times and restricted areas, a subject that Dr. Pontzen discusses. Compared to Earth, the Universe is immensely more complex, and the problems of building an appropriate model and running simulations that may span billions of years, but don't take billions of years of computer time, are truly tough!

For example, consider a galaxy. On the scale of the while Universe, galaxies are tiny and far apart.

This is the Hubble Ultra-Deep Field image, which shows about 10,000 galaxies (only one star is in the field, the really bright point with spikes caused by diffraction). The area of this image on the sky is 0.038°x0.038°, or about 0.15% of a square degree. It is about the size of the smallest thing you can see with your eye.

To a very rough estimate, although galaxies vary a lot in size, the really big ones seen here are a lot closer than the really small ones. The six or eight largest ones in this image are seen to be far from one another. If their intrinsic size is a little smaller than the size of "our" galaxy, the Milky Way, they are about 50,000 light-years across, and the average spacing between them is one or two million light-years. But larger-scale observations reveal that nearly all galaxies are strung out along strands in an immense web, with voids that contain no galaxies at all but span hundreds of millions of light-years.

One problem of computational cosmology that the author dwells on is that it is really hard to produce a cosmological simulation that doesn't result in a much larger number of galaxies. According to most models, this image "should" contain so many galaxies that there would be very little black space seen between them! A conundrum of computational cosmology is, "Why is space so empty?" I suppose all I can say is, "Stay tuned." I await a follow-on book on the subject as more is learned!

The smallness of galaxies compared to the intergalactic web, and the incredible smallness of the stars that make up the galaxies, and even more amazing smallness of planets, moons, and everything "solid" that we are familiar with, produce a huge problem of "stiffness" in any kind of simulation that seeks to span the entire range of sizes. Mathematical equations that drive simulations are called differential equations (DE's). By their nature, DE's produce one or more side effects, which mathematicians deal with using various schemes, and such schemes are embodied in the computer codes that run simulations. However, these schemes are seldom perfect, and runaway effects can swamp the simulation if it is run outside of a carefully chosen range. If a simple simulation includes two processes, and one runs 100 times as fast as the other, it is necessary to cater to the faster process or the results blow up. This time-scale contrast is called "stiffness". One must use time steps shorter than the time scale of the faster process, even though during such short steps, the slower process doesn't do much. Now consider what happens if the time scale varies over a range, not of 100 to one, but millions to one, with numerous processes all across the time spectrum. Not only that, if 99% of the volume is empty, and the remaining 1% has similar ranges of "spatial stiffness", the problem compounds dramatically. A lot of the book deals with such things, but using more accessible language.

The author also discusses dark matter and dark energy. Dark matter is probably quite real. It is needed to keep the stars in their orbits about their galactic centers, because the visible mass is not sufficient. This is not a small effect: the "extra gravity" needed is about five times what would be exerted by all the visible stuff we see. The current theory is that 70+% of the matter in the Universe isn't affected by electromagnetic radiation, so we can't see it. Scientists are working hard to find out what kind of stuff could be so invisible but so heavy.

Side question for the author or other cosmologists who may come across this review: Do black holes consume dark matter that encounters them?

Anyway, dark matter and the properties we infer for it must be included in cosmological models for their simulations to make any sense.

Dark energy is the term applied to an odd effect seen when very distant supernovae are studied. They seem too dim. Their distances are determined from the redshift calculated from their spectrum and, if possible, the redshift of their host galaxies. There are distinct "lines" in the spectrum of any astronomical body that allow us to determine its composition and the speed with which it is moving, radially at least. The Hubble Constant (named for Edwin Hubble, not the space telescope which was also named for him) characterizes the velocity-distance relationship.

Determining the actual brightness of a distant object is not straightforward. Dust and gas in and between galaxies absorbs some light. The relationship between distance and "intergalactic extinction" ("extinction" to an astronomer means light is being absorbed) is thought to be well understood. When such calculations are applied to certain supernovae, a discrepancy is found between how bright they are and how bright they "should" be. The farther away they are, the greater the discrepancy. This indicates that they might be farther away than their redshift would indicate; the "Hubble Constant" would then be not so constant! This implies that cosmological expansion is speeding up, not slowing down as we would expect.

I personally look at two matters that need more study before I will seriously consider that dark energy is real. 

Firstly, it is not mentioned in the book that the kind of supernovae one must study to discern dark energy are Type 1a. They are produced by a special mechanism. Most supernovae result when a large star (8-20x the mass of the Sun) runs out of fuel and its core collapses. About a quarter of supernovae result from a white dwarf star being loaded up with matter from a nearby red giant that is shedding mass. The maximum mass of a white dwarf is 1.44 solar masses; at this point it collapses and erupts as a Type 1a supernova. Because of these mechanics, Type 1a supernovae have very similar maximum brightness, making them a "standard candle". However, I have looked in the literature for an indication that the composition of the white dwarf and/or its red giant companion might affect the brightness of a Type 1a supernova. In the very early Universe there was hardly anything except hydrogen and helium. The first supernovae were all Type 2, when large stars, that had been forging hydrogen into more helium, and then forging helium into heavier elements, up to iron, exploded. Over time, the abundance of heavier elements in the Universe increased. To astronomers, all elements from lithium on up are called "metals" for convenience. Metallicity is a measure of the percent of "metals" in a star or galaxy. Our Sun's metallicity, at its visible surface, is 1.3%. Its age is 4.5 billion years, and it has not undergone fusion reactions that could change its metallicity, but an unknown amount of interstellar "stuff" has fallen into it; this is probably quite small in proportion to its total mass. Thus, a little over 1% probably represents the metallicity of this part of the Universe 4.5 billion years ago. The metallicity of the stars in a galaxy varies with distance from the center also, but not over a huge range. The bigger difference is seen between "Population I" stars, that are younger and have higher metallicity, and "Population II" stars, that are older and have something more like the metallicity of the Milky Way when it first formed, perhaps 10-12 billion years ago. This is roughly 1/10 or less of our Sun's metallicity, or less than 0.1%.

Very early galaxies and their stars had very small metallicities, ranging from 0.001% down to nearly zero. Therefore, so do the earliest Type 1a supernovae. A question I have not seen answered:

We know that white dwarf stars are composed primarily of carbon and oxygen. They are known to have some metals, because they are diagnosed by lines of silicon. BUT: Is the peak brightness of a Type 1a supernova significantly affected by the proportion of elements heavier than oxygen?

Secondly, is it possible that dark matter interacts very slightly with electromagnetic radiation? Simply put, the Universe's age is considered to be 13.8 billion years. At an age of 1.38 billion years, its "size" was 1/10 of its present "size", and the concentration of both ordinary matter and dark matter would have been, on average, 1,000 times greater. Somewhere along midway, say at an age of 4.4 billion years (the square root of 1/10 times 13.8), the "size" would have been about 0.32 of the current size, and the concentration of both ordinary matter and dark matter would have been about 32 times greater than at present. If there is even a slight interaction, "dark matter luminous extinction" could be a genuine effect, yet we would be very hard put to determine whether the dark matter that must be all around us has a measurable influence on light.

For the time being I consider that it is much, much more likely that "dark energy" is a phantom, and will eventually be found not to exist.

That is a significant digression from the discussion of the book. The author discusses the utility of cosmological simulations of various kinds. They aren't just a way for us to have a "pocket Universe" to play with, but they help us understand what might have occurred at various stages of the evolution of the Universe, or of groups of galaxies, or of stars and star clusters. Unlike weather forecasting, Universe simulation focuses on retro-casting, trying to reproduce how things worked out over some interesting span of past time, whether measured in centuries, millennia, or billions of years. To know where we really are we need to know what came before. Looking at distant things, as the Ultra Deep Field does, lets us look back in time. Things were different way back then, and computational cosmology is a powerful tool to help us understand it all. We've made a bit of a start; we're just getting going!

The author also asks whether it is plausible that we are living in an über-simulation inside some super-Matrix run by super-beings. He gets into that because he gets asked about it frequently. I'll mention one thing that he does not: one human brain has complexity of the same scale as a good chunk of the non-human Universe, and all of us together are more complex than the whole rest of the Universe (unless there are lots and lots of alien species!). In the Cosmos series by Carl Sagan, decades ago, it was stated that there are probably 100 billion galaxies in the observable Universe, with an average population of 100 billion stars each. The number of galaxies is probably more like a trillion. The number of stars is thus a number with 23 digits.

What's in a brain? The cortex has 16 billion neurons and the cerebellum has 70 billion. Each neuron has about 5,000 connections to other neurons. The 100 billion smaller "glial cells" also contact numerous neurons and large numbers of each other. The number of connections is thus a number with 15 digits. The number of humans is about 8 billion, a 10-digit number. So the "total human connectome" is about 100 times as great as the number of stars in the Universe. Another number of similar size is the number of molecules in 18 grams of water (a quantity known to chemists as a "mole"), which is a 24-digit number starting with the digit 6. If one could somehow use each water molecule in a tablespoon of water as a computer bit, it would take ten tablespoons to have enough molecules to devote just one "bit" to each connection in the sum total of all human brains. That's the bare bones of what's needed to produce The Matrix. And that's just one intelligent species on one planet. I'd say that if Moore's Law gallops along unimpeded long enough (but it won't, it's already faltering), it would take hundreds of doublings, or at least 1,000 years, for a big enough, fast enough computer to be produced (by Someone) that could simulate the entire Universe in real time. Of course, by making the computer's time steps for each second of real time actually take, say, a century, a much smaller computing system could to the work. How could we tell? Dr. Pontzen doesn't know, and neither do I.

A very enjoyable book. You don't have to be a computer geek like me to understand it.

Mathematical models, useful and otherwise

 kw: book reviews, nonfiction, mathematics, modeling, simulation, cautions, analysis

For a significant part of my career I worked with a group of talented computer programmers in a "skunk works" at an oil company. A colleague and I made up the Modeling and Simulation sub-group among the 20 members of the group. He and I developed software that simulated the production of crude oil and natural gas from kerogen, their migration upward through rock layers, and their accumulation against a trapping layer. No model is useful until it is checked against the real world, what we called "getting ground truth". I visited several exploration offices to show off the software and to use it with data those offices had on hand.

One memorable day in Louisiana, an explorer showed me the 3D seismic survey of one area. He pointed out the most likely source rock and explained the character of other layers, so we entered the appropriate setup parameters, "pointed" the software at the survey data, and let 'er rip. It showed progress over time, of the filling of trapped pools, as growing green blobs on a series of maps. He said, "OK, that blob is 'X' field, that one is 'Y' field,…but what is that?", pointing to a third blob between the other two. I answered, "I don't know, but I suspect it represents a lot of money." As it happened, the company had leases that covered most of the "what is that" area, but a deal had already been made to sell the leases to another company. That company made the money!

I had less exciting encounters with exploration geologists in Europe. The result was the validation of a useful model. Getting "ground truth" turned a simulation program into a tool the geologists could use to rank prospects.

Let me say right now that this tool is a million times less complex than the "general circulation models" (GCMs) used to forecast weather. Crude oil is gummy and moves slowly; air masses in the atmosphere, which are the elements of weather, move rapidly and swirl around on all scales. Oil forecasting is hard, but not as incredibly difficult as weather forecasting. So I was never faced with an irate caller complaining about "shoveling a foot of 'partly cloudy' from [his] @#&% driveway!"

Furthermore, I had the great good fortune to decide early in my career to "let the singers sing and the dancers dance": to turn over to the computer those tasks that are hardest for humans, while retaining tasks for the humans that we do better than computers. This led to very productive synergies. Far too many programmers spend years beating their heads against the wall trying to replace the human element. Futility personified.

I was delighted to read Escape From Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It by Erica Thompson. She sets the tone early on by quoting statistician George Box: "All models are wrong, but some are useful." Those who forget to think this way, or never heard this aphorism, get stuck in Model Land.

The author continues with the observation by President Dwight Eisenhower, that "Plans are useless, but planning is indispensable." The thinking behind the model, or the plan, is the great value of the exercise. I also recall what Sun Tzu wrote in The Art of War, "No battle plan survives contact with the enemy." In more peaceable pursuits, contact with "ground truth" exposes the errors of every model. It is our task to determine the tolerable level of error, for we must typically carry on anyway.

A model is a tool. It can help us understand a process, and perhaps inform the solution to a problem. BUT no model solves any problem all by itself. Even better than one model, a suite of models, built with various assumptions and focusing on different sets of driving parameters, can help us set boundaries on the range of outcomes.

It doesn't seem so long ago that the fastest supercomputer needed to run for half a day to produce a 2- or 3-day forecast for a continent-sized area. Now numerous GCMs exist, and the weather forecasters collect the output from all of them. One result is a spaghetti plots of hurricane tracks. In this image, the letter codes such as COTC represent the names of the models used. The characteristics of a spaghetti plot are used to produce the "cone of likelihood" that is often shown.

The situation with climate modeling is far different. Escape focuses on two areas, because of current events. One is climate "change" (spoiler: it is always changing, but on a slow time scale) which is all based on modeling because we can't perform physical experiments. The second is the epidemiology of COVID-19, modeled numerous ways, and almost never properly! The disease fooled the "experts" almost daily, and the societal flailing around that resulted seems to have caused more harm than doing nothing. I called the CDC the "strategy of the week club."

Both phenomena became so intensely politicized that no actual science has been possible. A certain spokesman whose name I hate to utter said, "I am the science." Tantamount to blasphemy. Another group of mostly pundits and a few scientists bludgeoned the public with the notion of "settled science." There is no such thing, except perhaps certain portions of mathematical physics. Neither climate and weather, nor epidemiology, are amenable to mathematico-physics treatment.

I am an educated layman. I went to school in an era in which we were taught critical thinking, and learned to identify bias. Putting on those hats, I can say the following, first about climate, and then about the pandemic.

1) I learned to apply the mathematics used by Arrhenius to study the Greenhouse Effect before I was in high school. The simplest model of the atmospheric response to sunlight with respect to CO2 has four spectral regions:

1. The Ultraviolet-Visible-Near Infrared region: wavelengths that are not affected by CO2.
2. Three narrow bands of Medium Infrared in the range 2.5µ-4.5µ, one of which is fully lapped over by an absorption band of water vapor. These have little warming effect, but they are well positioned for optical CO2 detectors.
3. A moderately wide band of Longwave Infrared centered on about 14µ, of absorption by CO2; the amount of absorption depends on the concentration. This is the "thermal IR" band of interest.
4. The rest of the Infrared spectrum, Far-Infrared and so on; it is not affected by CO2.

Within region #3 there is a variable level of absorptivity, but once the concentration of CO2 reaches 0.2% (2,000 ppm, comparable to the level during the age of the dinosaurs), the "carbon dioxide window" is effectively "closed". At that point, within that wavelength region, about half of the infrared radiation from the warm ground is absorbed by the atmosphere and is reradiated, half to outer space, and half back down. At a specific temperature a balance is achieved. That temperature is 4°C warmer than the average global temperature in the year 1900. Today, with 400 ppm, we're at 2°C, and it will take much more than another 400 ppm to push into that +4°C region. The relationship is not linear.

What detailed computer modeling can do is to show where the warming is greater, and where it is less. We've been hearing for years that the warming is greater in the polar regions and less in the tropics. The general picture is 6°-7°C warming around the poles and less than 3°C warming in the tropics, when CO2 concentration exceeds 1,000 ppm. At this point the "window" is mostly closed already; extra warming greater than 1°C is unlikely.

The above discussion means that warnings about deadly heat waves in the tropics are overblown. On the other hand, we can expect some frozen polar areas to thaw. One effect I haven't heard the slightest discussion about is that Siberia, northern Canada, and the southern part of South America could be the next breadbaskets. Will the Sahara and Mojave/Sonoran deserts, in Africa and North America, respectively, get even drier and hotter? The computer models are inconsistent. Fretful silence on these questions reflects the uncertainty.

2) The situation of the COVID-19 pandemic, and the incredible array of opinion/ideology presented as "science" is a stunning spectacle. Roughly half the adults in the U.S. think that the crisis was exploited to the hilt for political purposes, partly to remove Donald Trump from office and even more to increase the scope of totalitarian control on the part of the Left. Meanwhile, the other half are thrilled that Trump is out of office, but ambivalent about J.R. Biden's performance.

If there has been any serious modeling of the epidemiology of the C19 virus, I haven't seen it. I've seen numerous toy models presented, followed by lots of screaming to "follow the science". When it became evident that actual science contradicts what the screamers are saying, they took up new mantras about "protecting Democracy" (which really means protecting political power for Democrats). Sadly, I still see people walking alone in near-isolation, wearing a bandanna or cheap mask or, if they have an actual N95 or KN95 mask, wearing it below the nose. Firstly, they are insane to wear the mask at all, and secondly, the "face covering" they are using is not effective. Close to 0%. Nearly everyone who caught C19 after mid-2020 was wearing a mask when they caught it.

There is one and only one valid reason to wear a mask outside, anywhere there is no crowd: To keep the sun off one's face. My wife does this. She wears a mask to keep her cheeks from getting burnt when doing yard work. Never any other time!

There are three simple models that can be used to understand the risks of contracting the COVID-19 virus, SARS-COV2, when outside, with or without a mask. Firstly, except in very humid weather, the virus aerosolizes rapidly. The tiny droplets that a mask would stop evaporate completely in just a few minutes. You can look up the formula (the first "model") to calculate how long a droplet of size 1µ or 5µ will evaporate, at different levels of humidity. That means that the virus particles, which have a diameter of about 120nm (0.12µ), are what your mask has to stop. This introduces the second model.

A N-95 mask is called that because it catches 95% of particles (virus or otherwise) in the size range near 300nm, where the mask is least effective. It is very nearly 100% effective for larger particles (which are caught mechanically) and smaller particles (which are caught electrostatically). Particles in the 120nm range are caught electrostatically with an efficiency near 97%. Think a moment. If there are few viruses about, only 3% of them will get through the mask, if you wear it correctly. But suppose you enter a very crowded area that includes perhaps half a dozen folks who are coughing out C19 particles. Then, 3% of that viral load may well be enough for you to be infected. It is a numbers game.

Thirdly, during the daytime the C19 virus is about twice as susceptible to being disabled by solar ultraviolet as the Ebola virus. I worked out the numbers: Between 10 AM and 2 PM solar time, 90% of virus particles exposed to sunlight are inactivated within about 45 minutes. During the next 3/4 hour, 90% of whatever is left is inactivated, and so forth. It's a statistical function, now long it takes before a UV photon strikes a particular virus particle in a vulnerable spot.

Now we can pull back from my cogitations and look at the book's conclusions. The main problem with any model is the person who uses it. A model will give definite results, but it is easy to forget that those results pertain to the model, not to the system being modeled. They may be close, or they may not. But properly used, a model helps you think about a system of interest. It can't decide for you! Letting models do the deciding is always, always, a travesty.

What does the author suggest? How can we escape from Model Land? Five points form the meat of the last chapter:

1. Define the purpose. That purpose better not be "decide for me." The purpose has to include capturing the relationship between all the relevant parameters and the results. All...but learn what to neglect.
2. Don't say, "I don't know". Ask, "What do I know now that I didn't know before?"
3. Make value judgments. Every model reflects someone's values. Make sure the values behind the structure of a model are the right ones. Remember: genuine science is value-free. YOU supply the values.
4. Write about the real world. Bring the model's conclusions into the real world, by getting "ground truth", for example.
5. Use many models. Consider the spaghetti plot of the hurricane, shown above. If the system is "too simple" for multiple models to be generated, perhaps it is simple enough to comprehend without mathematical modeling.

It takes work just to understand this list. It is worth it! It takes much more work to carry out a modeling exercise that doesn't trap you in Model Land without an exit. I used a Jumping Ship metaphor at the start of this piece to show that sometimes we need to get far away, to seek a really different perspective. This is an extension of Principle #5 above.

Finally, consider this: You have two eyes for a reason. It is not just for parallax, to find out the 3D aspect of the situation. It is for completeness. Except in very simple views, your right eye will see something your left eye cannot, and vice versa. Even more, if you have someone standing near you, each of you will see things the other does not, particularly if you're looking in different directions. Using a numerical or mathematical model in too simple a manner, and yielding too much authority to it, is like viewing a complex scene using one eye from one viewpoint. I leave you with this proverb: "If two people have exactly the same opinion about everything, one of them is redundant."

This book is a great read!

How Apollo's arrow misses

kw: book reviews, nonfiction, mathematics, modeling, philosophy

There is a simple, repetitive calculation called a Shift Map, based on kneading dough, that illustrates the problems caused by repeated calculations that lose precision as rounding errors accumulate. While we knead dough by repeatedly folding and stretching, in this model, the lump is stretched to twice its length, cut in half and the two halves set atop one another before the next stretch. Now imagine two points in the lump, say two yeast cells, that were once close to one another. With each cycle, they get twice as far apart, until one of them is moved beyond the cut point. At the next cycle, it starts somewhere near the opposite end, then begins to move across again. From that point, the distance between the two cells is not a simple doubling of distance any more. Also after the first time it crosses the cut, there is no simple way to calculate where the cell will be after N cycles.

The mathematical model of this situation is very simple. The position of a cell can be denoted x, and its position after the next cut is then x_n+1 = 2x_n mod 1. The mod function divides and leaves a remainder; in this case it removes any portion larger than 1, so 1.1938 becomes 0.1938. A cell beginning at 0.34 moves to positions, 0.68, 0.36 (1 was removed), 0.72, 0.44, 0.88, 0.76, and so forth.

In a continuous system, most positions for a cell never return to a former position, and the cell's motion soon becomes unpredictable from first principles. However, when we model this action with a computer, the whole system becomes totally predictable! The reason is the finite size of a "number" inside a computer, and the fact that only rational numbers can be computed. As a matter of fact, not all rational numbers can be stored accurately in computer memory, only those that represent a finite sum of powers of 1/2. For example, any multiple of 1/3 is stored in binary form as 0.010101010101… and so forth, but eventually you have to stop. So what is stored is not exactly one-third. In fact, 1/10 is also stored as a repeating binary decimal and cannot be exactly represented in binary form!

In most computers, a "word" is a 64-bit quantity. 8 bits are used to store an exponent, two bits for the number's sign and the exponent's sign, and the remaining 54 bits are used for a normalized binary value. Thus 1/3 is not actually stored as 0.01010101 and so forth, but as

01|0000001|101010101010101010101010101010101010101010101010101010

I put pipe signs (|) between the sections to emphasize their use. The first 01 means "positive number, negative exponent", the 00000001 is the exponent 1 (so because it is negative, it means "divide by two"), and the rest of the alternating 1's and 0's store the value 2/3, as accurately as 54 bits allows. The whole thing means 1/3, with a minor error after the 54th binary digit. That is the 16th decimal digit.

What happens, then, when we run this through the dough-stretch-and-cut routine? Doubling the number above is accomplished first by removing the negative exponent. So the first ten bits are now 00|00000000, and the whole thing means 2/3. Double it again, and the number stored begins 00|00000001, and the whole thing means 4/3. But we have to subtract one from this to get the remainder for the mod function. The "add" unit in the computer does this by lining up the internal number for -1.0, which is 10|00000001|1000000 and a whole lot more zeroes, with 00|00000001|10101010 and a whole lot more alternating ones and zeroes., then subtracting the value sections from each other (you don't subtract the exponents, but the unit does determine the final sign by checking which sign is negative).

The result is 00|00000001|001010101010 and a lot more alternating ones and zeroes, but notice that the start of the value section is two zeroes. This is "normalized" by shifting two positions to the left, and subtracting two from the exponent. It has been a 1, so now it is -1. And the resulting number becomes 01|00000001|101010 and so forth. This is just 1/3, however, the next-to-last binary digit is no longer a one but a zero. Precision has been lost.

This kind of lost precision continues to occur with each cycle. Rather than spend a lot more words, let's look at the first fifteen cycles as shown in Excel:


Binary rounding errors don't become visible as decimal rounding errors until the sixth cycle, but they compound thereafter. The number is stored with 54-bit precision, so we ought to expect it to take 54 cycles to exhaust the bits…and that is what we find!


The value at the 52d cycle, 0.625, is stored as 00|00000001|1010000000 and more zeroes. 0.25 has just one "useful" bit in the value section, as does 0.5, and the next doubling produced 1.0, which the mod function returned as zero. From that point, it is zero all the way. The model has totally succumbed to rounding error. A final bit before going on: you can look at the value section of the initial number and tell the total history for the next 54 cycles. For most starting values, after 54 cycles you'll get a zero! If you use a number like a trillionth or so, it'll take longer, but no matter what, this calculation is destined to reach zero in a relatively small number of iterations (a couple hundred or fewer). Finally, you don't gain much by using a longer computer "word" to store a number. Suppose the value section had 120 bits rather than 54. Nice idea. But it would "go to zero" after 100 iterations (or at most, 200). Small help that.

OK, all this is background for The Future of Everything: The Science of Prediction by David Orrell, PhD. Dr. Orrell is currently a controversial figure for his assertion that the forecasting of weather, economics, and public health are not a result primarily of mathematical chaos, such as the "butterfly effect", but of model error. The simple system modeled above is one example of model error making a model useless for long-term calculations. Although chaos effects make the path of the yeast cell hard to predict after several cycles, we can run the model to do so. But once it all "goes to zero", prediction is over.

Complex systems have more than one equation, and the equations are larger. A weather prediction model, such as the Global Circulation Models used by NCAR and others, may have millions of geometric elements, and a few dozen equations representing the state of the modeled weather in each element, that must be balanced across all those millions of elements. The starting point is the values of the equations for each element at some point of time. It may take hundreds of cycles to model the weather for one day, and several thousand for a ten-day forecast.

Fortunately, the kind of rounding errors I emphasized in the Shift Map model tend to compensate for one another, otherwise no model could run for more than a hundred steps or so. But there are other kinds of error. The most significant is sampling error: the "weather" in most of those millions of elements must be estimated from readings taken at a few tens of thousands of locations around the planet. Our coverage over the oceans is particularly spotty.

Also, the equations used are mostly empirical approximations of what an air mass the size of New York City, and a km or so deep, will do given the temperature, pressure, wind motion and cloud cover of all its neighboring elements, plus its own. Orrell likens these approximations to epicycles, those little circles used before Kepler to "correct" the motions of the planets and make predictions of eclipses and other syzygies. Epicycles worked well, but on p.41 the author states,

"The fact that it worked quite well as a model of the universe is a poignant reminder that a model that can be made to fit the data isn't necessarily an accurate representation of reality."

The map isn't the landscape, and the model isn't the system modeled. There is lots of room for the unexpected.

Weather isn't the only complex system that people tackle with mathematical models. The economy is a big one, but is considered less complex. For example, there are less than 300 nations, and a much smaller number of stock and commodity markets, so macroeconomic models don't have millions of "elements" to work with, just a few hundred simulated national markets. These markets are based on the "average man", which is considered a soulless bag of reactions to stimuli such as unemployment figures and corporate profit announcements. But once you realize that the behavior of a few million "average men" must be empirically modeled by some statistical beast, you find epicycles again. Both weather/climate and economic models use Ordinary Differential Equations (ODEs) in abundance as the foundation of their modeled structures. These may not be the best in all cases, but "they can be solved using mathematics" (p.115).

The trouble with both is, there are too many parameters that we don't know very well. I once knew a meteorologist (now deceased) who was brought to a campus to add the simulation of lightning to a model of thunderstorms. I asked him about it, and he said, "We don't know where, exactly, a new stroke will start. We have areas of greater likelihood, and have to pick a random spot in one of them." With some tuning, the model was a pretty good simulation of thunderstorm dynamics. But the point was that it was overdetermined: you could get it to simulate a lot of very unrealistic thunderstorms also. About this phenomenon, the Dr. Orrell writes,

"The models suffer from the same problem [that] the Greek Circle Model did: they are too flexible. As Will Keepin put it, modelers can pull the levers and make the model do whatever they want…It is the signature of uncomputability." (p.205)

Ah, uncomputability. It is his term for a phenomenon that cannot—and maybe never can—be simulated from first principles. This is not because of mathematical chaos. It is because, like Conway's game of Life (a seemingly simple cellular automaton with complex behavior), most of what happens is emergent behavior, not strictly implied in the "rules of the game."

One method used by modelers, particularly weather modelers, is to run an ensemble: to run several simulations with slightly different starting conditions, or to use a number of programs that run in slightly different ways or with different size elements. But they are all based on a similar set of simplifications. The author writes,

"[An] ensemble of wrong models does not make a right model, and the spread between the results is not an accurate measure of uncertainty." (p.301)

This is because of something Don Rumsfeld is famous for calling "unknown unknowns". There are things you know you don't know well enough, the "known unknowns". The real stinkers are the things you don't know at all, the things you don't know you don't know. Considering that, in the metabolism of a yeast cell too small to see, there are parameters we know at best only within a factor of ten, and other parameters we've never thought of, it is no surprise we don't have good models of the dynamics of cellular metabolism.

We need to be a little humble. There are things we can't hope to know well enough to simulate them. It would take a computer bigger than the weather to simulate the weather with any accuracy, assuming we got all the parameters and starting values right in the first place. There are billions of people whose emotional state and level of blood caffeine determine the motion of stock markets. And evolution is going on all the time as bacteria and viruses that don't even know we are there strive to make a living that just might come at our expense. Bohr said it best: "Prediction is hard, especially about the future".