Creeping toward the Matrix

 kw: book reviews, nonfiction, cosmology, simulation, modeling

I was for several years, in my career of writing scientific software, the leader of a "Modeling and Simulation Group". One of our products had three sections, simulating first the geochemistry of crude oil generation from organic matter in deep rocks (up to a few kilometers), then the upward migration of the petroleum liquids through porous rocks, and finally their entrapment against nonporous, or less porous, rock layers to form oil and gas reservoirs.

I was sent to a few exploration offices to show off the software. In one instance, after the geologist set up access to a set of grids based on seismic data, I ran the software, which displayed the progress through time of oil and natural gas collecting under the trapping layer some half-kilometer beneath our feet. At the end of the run, he pointed to one green blob on the map, saying, "This is X field," and to another, "This is Y field." Then he pointed to a third one between them, asking, "But what is that?" I answered, "That could represent a lot of money." As it happened, the company had decided to sell that property to another company. That oil company made the money! But the software found the oil before the property was drilled. Later that year I spoke about the experience at a Research Review. My talk was titled, "Finding Oil in a Computer."

It was with great relish that I read The Universe in a Box: Simulations and the Quest to Code the Cosmos by Andrew Pontzen. If you buy the book feel free to download this image to print a bookplate; it's 1024x1024 px. Use Upscayl or something similar if you want it rendered at higher resolution. I produced it using Playground AI; the only prompt was "Cosmology". I tinkered with Samplers and other parameters, looking for something else. Getting this image was a side benefit.

The author could have delved deeply into sundry technical issues—there are many! Instead, he has skirted these, providing just a taste of some of them, in favor of the philosophy and motivations for making computer simulations of natural phenomena.

The terms "simulation" and "modeling" have overlapping meanings. In principle, a Model is the framework and the sets of parameters that define the physical structure and the physics rules to be followed, while a Simulation is the operation of the Model over a chosen span of time, producing a series of output data sets that describe the expected physical state of the modeled "thing" at one or more points in time, whether past or future. Note that a simulation can be done on equipment other than a computer. One story of the book is about a galaxy simulation done with light bulbs and photocells and a glorious tangle of wires.

Weather forecasting is one very visible result of computer simulation, seen daily (or hourly) on newscasts and in the various weather apps on our devices. There are a couple of dozen important models used by weather agencies the world over. One expression of these is the Spaghetti Plot of a hurricane's forecasted track, as produced by several models. The models differ in the importance they place on various aspects of the modeled system, including whether it represents the whole Earth or a hemisphere, or a couple of continents.

All weather models are based on a global General Circulation Model, in which the atmosphere and the land and sea surfaces in contact with it (and sometimes a surface layer of the ocean) are divided up into roughly ¼ to ½ million quasi-rectangular portions. Half a million to a million "cells" is about the most that modern supercomputers can handle. In general, spatial resolution is likely to be as large as 200x200 km! The Earth's surface area is about 127 million km², and the models have between 20 and 40 vertical layers (at present). A 40-layer model would have more than five billion cells of 1x1x0.5 km, so to get the count below one million requires using cells with an area of more than 5,000 km², which is 71x71x0.5 km, and most models are set up for grid squares of about 100x100 to 200x200 (and half a km thick). The physics rules, primarily those relating to pressure and temperature relationships, are applied at the boundaries between grid cells.

To get a "future radar rain" map with finer detail requires using "sub grid" rules, and partial simulations over short times and restricted areas, a subject that Dr. Pontzen discusses. Compared to Earth, the Universe is immensely more complex, and the problems of building an appropriate model and running simulations that may span billions of years, but don't take billions of years of computer time, are truly tough!

For example, consider a galaxy. On the scale of the while Universe, galaxies are tiny and far apart.

This is the Hubble Ultra-Deep Field image, which shows about 10,000 galaxies (only one star is in the field, the really bright point with spikes caused by diffraction). The area of this image on the sky is 0.038°x0.038°, or about 0.15% of a square degree. It is about the size of the smallest thing you can see with your eye.

To a very rough estimate, although galaxies vary a lot in size, the really big ones seen here are a lot closer than the really small ones. The six or eight largest ones in this image are seen to be far from one another. If their intrinsic size is a little smaller than the size of "our" galaxy, the Milky Way, they are about 50,000 light-years across, and the average spacing between them is one or two million light-years. But larger-scale observations reveal that nearly all galaxies are strung out along strands in an immense web, with voids that contain no galaxies at all but span hundreds of millions of light-years.

One problem of computational cosmology that the author dwells on is that it is really hard to produce a cosmological simulation that doesn't result in a much larger number of galaxies. According to most models, this image "should" contain so many galaxies that there would be very little black space seen between them! A conundrum of computational cosmology is, "Why is space so empty?" I suppose all I can say is, "Stay tuned." I await a follow-on book on the subject as more is learned!

The smallness of galaxies compared to the intergalactic web, and the incredible smallness of the stars that make up the galaxies, and even more amazing smallness of planets, moons, and everything "solid" that we are familiar with, produce a huge problem of "stiffness" in any kind of simulation that seeks to span the entire range of sizes. Mathematical equations that drive simulations are called differential equations (DE's). By their nature, DE's produce one or more side effects, which mathematicians deal with using various schemes, and such schemes are embodied in the computer codes that run simulations. However, these schemes are seldom perfect, and runaway effects can swamp the simulation if it is run outside of a carefully chosen range. If a simple simulation includes two processes, and one runs 100 times as fast as the other, it is necessary to cater to the faster process or the results blow up. This time-scale contrast is called "stiffness". One must use time steps shorter than the time scale of the faster process, even though during such short steps, the slower process doesn't do much. Now consider what happens if the time scale varies over a range, not of 100 to one, but millions to one, with numerous processes all across the time spectrum. Not only that, if 99% of the volume is empty, and the remaining 1% has similar ranges of "spatial stiffness", the problem compounds dramatically. A lot of the book deals with such things, but using more accessible language.

The author also discusses dark matter and dark energy. Dark matter is probably quite real. It is needed to keep the stars in their orbits about their galactic centers, because the visible mass is not sufficient. This is not a small effect: the "extra gravity" needed is about five times what would be exerted by all the visible stuff we see. The current theory is that 70+% of the matter in the Universe isn't affected by electromagnetic radiation, so we can't see it. Scientists are working hard to find out what kind of stuff could be so invisible but so heavy.

Side question for the author or other cosmologists who may come across this review: Do black holes consume dark matter that encounters them?

Anyway, dark matter and the properties we infer for it must be included in cosmological models for their simulations to make any sense.

Dark energy is the term applied to an odd effect seen when very distant supernovae are studied. They seem too dim. Their distances are determined from the redshift calculated from their spectrum and, if possible, the redshift of their host galaxies. There are distinct "lines" in the spectrum of any astronomical body that allow us to determine its composition and the speed with which it is moving, radially at least. The Hubble Constant (named for Edwin Hubble, not the space telescope which was also named for him) characterizes the velocity-distance relationship.

Determining the actual brightness of a distant object is not straightforward. Dust and gas in and between galaxies absorbs some light. The relationship between distance and "intergalactic extinction" ("extinction" to an astronomer means light is being absorbed) is thought to be well understood. When such calculations are applied to certain supernovae, a discrepancy is found between how bright they are and how bright they "should" be. The farther away they are, the greater the discrepancy. This indicates that they might be farther away than their redshift would indicate; the "Hubble Constant" would then be not so constant! This implies that cosmological expansion is speeding up, not slowing down as we would expect.

I personally look at two matters that need more study before I will seriously consider that dark energy is real. 

Firstly, it is not mentioned in the book that the kind of supernovae one must study to discern dark energy are Type 1a. They are produced by a special mechanism. Most supernovae result when a large star (8-20x the mass of the Sun) runs out of fuel and its core collapses. About a quarter of supernovae result from a white dwarf star being loaded up with matter from a nearby red giant that is shedding mass. The maximum mass of a white dwarf is 1.44 solar masses; at this point it collapses and erupts as a Type 1a supernova. Because of these mechanics, Type 1a supernovae have very similar maximum brightness, making them a "standard candle". However, I have looked in the literature for an indication that the composition of the white dwarf and/or its red giant companion might affect the brightness of a Type 1a supernova. In the very early Universe there was hardly anything except hydrogen and helium. The first supernovae were all Type 2, when large stars, that had been forging hydrogen into more helium, and then forging helium into heavier elements, up to iron, exploded. Over time, the abundance of heavier elements in the Universe increased. To astronomers, all elements from lithium on up are called "metals" for convenience. Metallicity is a measure of the percent of "metals" in a star or galaxy. Our Sun's metallicity, at its visible surface, is 1.3%. Its age is 4.5 billion years, and it has not undergone fusion reactions that could change its metallicity, but an unknown amount of interstellar "stuff" has fallen into it; this is probably quite small in proportion to its total mass. Thus, a little over 1% probably represents the metallicity of this part of the Universe 4.5 billion years ago. The metallicity of the stars in a galaxy varies with distance from the center also, but not over a huge range. The bigger difference is seen between "Population I" stars, that are younger and have higher metallicity, and "Population II" stars, that are older and have something more like the metallicity of the Milky Way when it first formed, perhaps 10-12 billion years ago. This is roughly 1/10 or less of our Sun's metallicity, or less than 0.1%.

Very early galaxies and their stars had very small metallicities, ranging from 0.001% down to nearly zero. Therefore, so do the earliest Type 1a supernovae. A question I have not seen answered:

We know that white dwarf stars are composed primarily of carbon and oxygen. They are known to have some metals, because they are diagnosed by lines of silicon. BUT: Is the peak brightness of a Type 1a supernova significantly affected by the proportion of elements heavier than oxygen?

Secondly, is it possible that dark matter interacts very slightly with electromagnetic radiation? Simply put, the Universe's age is considered to be 13.8 billion years. At an age of 1.38 billion years, its "size" was 1/10 of its present "size", and the concentration of both ordinary matter and dark matter would have been, on average, 1,000 times greater. Somewhere along midway, say at an age of 4.4 billion years (the square root of 1/10 times 13.8), the "size" would have been about 0.32 of the current size, and the concentration of both ordinary matter and dark matter would have been about 32 times greater than at present. If there is even a slight interaction, "dark matter luminous extinction" could be a genuine effect, yet we would be very hard put to determine whether the dark matter that must be all around us has a measurable influence on light.

For the time being I consider that it is much, much more likely that "dark energy" is a phantom, and will eventually be found not to exist.

That is a significant digression from the discussion of the book. The author discusses the utility of cosmological simulations of various kinds. They aren't just a way for us to have a "pocket Universe" to play with, but they help us understand what might have occurred at various stages of the evolution of the Universe, or of groups of galaxies, or of stars and star clusters. Unlike weather forecasting, Universe simulation focuses on retro-casting, trying to reproduce how things worked out over some interesting span of past time, whether measured in centuries, millennia, or billions of years. To know where we really are we need to know what came before. Looking at distant things, as the Ultra Deep Field does, lets us look back in time. Things were different way back then, and computational cosmology is a powerful tool to help us understand it all. We've made a bit of a start; we're just getting going!

The author also asks whether it is plausible that we are living in an über-simulation inside some super-Matrix run by super-beings. He gets into that because he gets asked about it frequently. I'll mention one thing that he does not: one human brain has complexity of the same scale as a good chunk of the non-human Universe, and all of us together are more complex than the whole rest of the Universe (unless there are lots and lots of alien species!). In the Cosmos series by Carl Sagan, decades ago, it was stated that there are probably 100 billion galaxies in the observable Universe, with an average population of 100 billion stars each. The number of galaxies is probably more like a trillion. The number of stars is thus a number with 23 digits.

What's in a brain? The cortex has 16 billion neurons and the cerebellum has 70 billion. Each neuron has about 5,000 connections to other neurons. The 100 billion smaller "glial cells" also contact numerous neurons and large numbers of each other. The number of connections is thus a number with 15 digits. The number of humans is about 8 billion, a 10-digit number. So the "total human connectome" is about 100 times as great as the number of stars in the Universe. Another number of similar size is the number of molecules in 18 grams of water (a quantity known to chemists as a "mole"), which is a 24-digit number starting with the digit 6. If one could somehow use each water molecule in a tablespoon of water as a computer bit, it would take ten tablespoons to have enough molecules to devote just one "bit" to each connection in the sum total of all human brains. That's the bare bones of what's needed to produce The Matrix. And that's just one intelligent species on one planet. I'd say that if Moore's Law gallops along unimpeded long enough (but it won't, it's already faltering), it would take hundreds of doublings, or at least 1,000 years, for a big enough, fast enough computer to be produced (by Someone) that could simulate the entire Universe in real time. Of course, by making the computer's time steps for each second of real time actually take, say, a century, a much smaller computing system could to the work. How could we tell? Dr. Pontzen doesn't know, and neither do I.

A very enjoyable book. You don't have to be a computer geek like me to understand it.

Mathematical models, useful and otherwise

 kw: book reviews, nonfiction, mathematics, modeling, simulation, cautions, analysis

For a significant part of my career I worked with a group of talented computer programmers in a "skunk works" at an oil company. A colleague and I made up the Modeling and Simulation sub-group among the 20 members of the group. He and I developed software that simulated the production of crude oil and natural gas from kerogen, their migration upward through rock layers, and their accumulation against a trapping layer. No model is useful until it is checked against the real world, what we called "getting ground truth". I visited several exploration offices to show off the software and to use it with data those offices had on hand.

One memorable day in Louisiana, an explorer showed me the 3D seismic survey of one area. He pointed out the most likely source rock and explained the character of other layers, so we entered the appropriate setup parameters, "pointed" the software at the survey data, and let 'er rip. It showed progress over time, of the filling of trapped pools, as growing green blobs on a series of maps. He said, "OK, that blob is 'X' field, that one is 'Y' field,…but what is that?", pointing to a third blob between the other two. I answered, "I don't know, but I suspect it represents a lot of money." As it happened, the company had leases that covered most of the "what is that" area, but a deal had already been made to sell the leases to another company. That company made the money!

I had less exciting encounters with exploration geologists in Europe. The result was the validation of a useful model. Getting "ground truth" turned a simulation program into a tool the geologists could use to rank prospects.

Let me say right now that this tool is a million times less complex than the "general circulation models" (GCMs) used to forecast weather. Crude oil is gummy and moves slowly; air masses in the atmosphere, which are the elements of weather, move rapidly and swirl around on all scales. Oil forecasting is hard, but not as incredibly difficult as weather forecasting. So I was never faced with an irate caller complaining about "shoveling a foot of 'partly cloudy' from [his] @#&% driveway!"

Furthermore, I had the great good fortune to decide early in my career to "let the singers sing and the dancers dance": to turn over to the computer those tasks that are hardest for humans, while retaining tasks for the humans that we do better than computers. This led to very productive synergies. Far too many programmers spend years beating their heads against the wall trying to replace the human element. Futility personified.

I was delighted to read Escape From Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It by Erica Thompson. She sets the tone early on by quoting statistician George Box: "All models are wrong, but some are useful." Those who forget to think this way, or never heard this aphorism, get stuck in Model Land.

The author continues with the observation by President Dwight Eisenhower, that "Plans are useless, but planning is indispensable." The thinking behind the model, or the plan, is the great value of the exercise. I also recall what Sun Tzu wrote in The Art of War, "No battle plan survives contact with the enemy." In more peaceable pursuits, contact with "ground truth" exposes the errors of every model. It is our task to determine the tolerable level of error, for we must typically carry on anyway.

A model is a tool. It can help us understand a process, and perhaps inform the solution to a problem. BUT no model solves any problem all by itself. Even better than one model, a suite of models, built with various assumptions and focusing on different sets of driving parameters, can help us set boundaries on the range of outcomes.

It doesn't seem so long ago that the fastest supercomputer needed to run for half a day to produce a 2- or 3-day forecast for a continent-sized area. Now numerous GCMs exist, and the weather forecasters collect the output from all of them. One result is a spaghetti plots of hurricane tracks. In this image, the letter codes such as COTC represent the names of the models used. The characteristics of a spaghetti plot are used to produce the "cone of likelihood" that is often shown.

The situation with climate modeling is far different. Escape focuses on two areas, because of current events. One is climate "change" (spoiler: it is always changing, but on a slow time scale) which is all based on modeling because we can't perform physical experiments. The second is the epidemiology of COVID-19, modeled numerous ways, and almost never properly! The disease fooled the "experts" almost daily, and the societal flailing around that resulted seems to have caused more harm than doing nothing. I called the CDC the "strategy of the week club."

Both phenomena became so intensely politicized that no actual science has been possible. A certain spokesman whose name I hate to utter said, "I am the science." Tantamount to blasphemy. Another group of mostly pundits and a few scientists bludgeoned the public with the notion of "settled science." There is no such thing, except perhaps certain portions of mathematical physics. Neither climate and weather, nor epidemiology, are amenable to mathematico-physics treatment.

I am an educated layman. I went to school in an era in which we were taught critical thinking, and learned to identify bias. Putting on those hats, I can say the following, first about climate, and then about the pandemic.

1) I learned to apply the mathematics used by Arrhenius to study the Greenhouse Effect before I was in high school. The simplest model of the atmospheric response to sunlight with respect to CO2 has four spectral regions:

1. The Ultraviolet-Visible-Near Infrared region: wavelengths that are not affected by CO2.
2. Three narrow bands of Medium Infrared in the range 2.5µ-4.5µ, one of which is fully lapped over by an absorption band of water vapor. These have little warming effect, but they are well positioned for optical CO2 detectors.
3. A moderately wide band of Longwave Infrared centered on about 14µ, of absorption by CO2; the amount of absorption depends on the concentration. This is the "thermal IR" band of interest.
4. The rest of the Infrared spectrum, Far-Infrared and so on; it is not affected by CO2.

Within region #3 there is a variable level of absorptivity, but once the concentration of CO2 reaches 0.2% (2,000 ppm, comparable to the level during the age of the dinosaurs), the "carbon dioxide window" is effectively "closed". At that point, within that wavelength region, about half of the infrared radiation from the warm ground is absorbed by the atmosphere and is reradiated, half to outer space, and half back down. At a specific temperature a balance is achieved. That temperature is 4°C warmer than the average global temperature in the year 1900. Today, with 400 ppm, we're at 2°C, and it will take much more than another 400 ppm to push into that +4°C region. The relationship is not linear.

What detailed computer modeling can do is to show where the warming is greater, and where it is less. We've been hearing for years that the warming is greater in the polar regions and less in the tropics. The general picture is 6°-7°C warming around the poles and less than 3°C warming in the tropics, when CO2 concentration exceeds 1,000 ppm. At this point the "window" is mostly closed already; extra warming greater than 1°C is unlikely.

The above discussion means that warnings about deadly heat waves in the tropics are overblown. On the other hand, we can expect some frozen polar areas to thaw. One effect I haven't heard the slightest discussion about is that Siberia, northern Canada, and the southern part of South America could be the next breadbaskets. Will the Sahara and Mojave/Sonoran deserts, in Africa and North America, respectively, get even drier and hotter? The computer models are inconsistent. Fretful silence on these questions reflects the uncertainty.

2) The situation of the COVID-19 pandemic, and the incredible array of opinion/ideology presented as "science" is a stunning spectacle. Roughly half the adults in the U.S. think that the crisis was exploited to the hilt for political purposes, partly to remove Donald Trump from office and even more to increase the scope of totalitarian control on the part of the Left. Meanwhile, the other half are thrilled that Trump is out of office, but ambivalent about J.R. Biden's performance.

If there has been any serious modeling of the epidemiology of the C19 virus, I haven't seen it. I've seen numerous toy models presented, followed by lots of screaming to "follow the science". When it became evident that actual science contradicts what the screamers are saying, they took up new mantras about "protecting Democracy" (which really means protecting political power for Democrats). Sadly, I still see people walking alone in near-isolation, wearing a bandanna or cheap mask or, if they have an actual N95 or KN95 mask, wearing it below the nose. Firstly, they are insane to wear the mask at all, and secondly, the "face covering" they are using is not effective. Close to 0%. Nearly everyone who caught C19 after mid-2020 was wearing a mask when they caught it.

There is one and only one valid reason to wear a mask outside, anywhere there is no crowd: To keep the sun off one's face. My wife does this. She wears a mask to keep her cheeks from getting burnt when doing yard work. Never any other time!

There are three simple models that can be used to understand the risks of contracting the COVID-19 virus, SARS-COV2, when outside, with or without a mask. Firstly, except in very humid weather, the virus aerosolizes rapidly. The tiny droplets that a mask would stop evaporate completely in just a few minutes. You can look up the formula (the first "model") to calculate how long a droplet of size 1µ or 5µ will evaporate, at different levels of humidity. That means that the virus particles, which have a diameter of about 120nm (0.12µ), are what your mask has to stop. This introduces the second model.

A N-95 mask is called that because it catches 95% of particles (virus or otherwise) in the size range near 300nm, where the mask is least effective. It is very nearly 100% effective for larger particles (which are caught mechanically) and smaller particles (which are caught electrostatically). Particles in the 120nm range are caught electrostatically with an efficiency near 97%. Think a moment. If there are few viruses about, only 3% of them will get through the mask, if you wear it correctly. But suppose you enter a very crowded area that includes perhaps half a dozen folks who are coughing out C19 particles. Then, 3% of that viral load may well be enough for you to be infected. It is a numbers game.

Thirdly, during the daytime the C19 virus is about twice as susceptible to being disabled by solar ultraviolet as the Ebola virus. I worked out the numbers: Between 10 AM and 2 PM solar time, 90% of virus particles exposed to sunlight are inactivated within about 45 minutes. During the next 3/4 hour, 90% of whatever is left is inactivated, and so forth. It's a statistical function, now long it takes before a UV photon strikes a particular virus particle in a vulnerable spot.

Now we can pull back from my cogitations and look at the book's conclusions. The main problem with any model is the person who uses it. A model will give definite results, but it is easy to forget that those results pertain to the model, not to the system being modeled. They may be close, or they may not. But properly used, a model helps you think about a system of interest. It can't decide for you! Letting models do the deciding is always, always, a travesty.

What does the author suggest? How can we escape from Model Land? Five points form the meat of the last chapter:

1. Define the purpose. That purpose better not be "decide for me." The purpose has to include capturing the relationship between all the relevant parameters and the results. All...but learn what to neglect.
2. Don't say, "I don't know". Ask, "What do I know now that I didn't know before?"
3. Make value judgments. Every model reflects someone's values. Make sure the values behind the structure of a model are the right ones. Remember: genuine science is value-free. YOU supply the values.
4. Write about the real world. Bring the model's conclusions into the real world, by getting "ground truth", for example.
5. Use many models. Consider the spaghetti plot of the hurricane, shown above. If the system is "too simple" for multiple models to be generated, perhaps it is simple enough to comprehend without mathematical modeling.

It takes work just to understand this list. It is worth it! It takes much more work to carry out a modeling exercise that doesn't trap you in Model Land without an exit. I used a Jumping Ship metaphor at the start of this piece to show that sometimes we need to get far away, to seek a really different perspective. This is an extension of Principle #5 above.

Finally, consider this: You have two eyes for a reason. It is not just for parallax, to find out the 3D aspect of the situation. It is for completeness. Except in very simple views, your right eye will see something your left eye cannot, and vice versa. Even more, if you have someone standing near you, each of you will see things the other does not, particularly if you're looking in different directions. Using a numerical or mathematical model in too simple a manner, and yielding too much authority to it, is like viewing a complex scene using one eye from one viewpoint. I leave you with this proverb: "If two people have exactly the same opinion about everything, one of them is redundant."

This book is a great read!