Will we all go gentle into that good night?

 kw: book reviews, nonfiction, artificial intelligence, AI, artificial superintelligence, ASI, dangers, warnings, polemics

I have strong reasons for believing that artificial superintelligence (ASI) is impossible. However, I could be wrong. If I am wrong, the founders of Machine Intelligence Research Institute (MIRI) have a warning for all of us. In their book If Anyone Builds it, Everyone Dies: Why Superhuman AI Would Kill Us All, Eliezer Yudkowsky and Nate Soares state their case.

It is worth considering. Will a future with highly advanced AI include humans? As the authors do, I'd like to look at some things that are hard.

Firstly, we don't know what consciousness is. A recent item in Science News claims that everything with a hint of a brain, at the very least, is conscious. That is, all individual creatures of all species in the animal kingdom have a sense of self, which they strive to defend, maintain, and reproduce. At the moment, nobody anywhere has a way to prove this either true or false or partly false; and in the latter case, where to draw the line. Does a millimeter-long fairy fly have a sense of "I am"? If so, consciousness must arise very easily…in a creature that possesses neurons.

The tiny nematode worm has 302 neurons in its entire nervous system. A fingernail-size jellyfish has thousands. With this in mind, why doesn't a laptop computer or a smart phone, with billions of transistors in its CPU, connected in a complex way, running intricate software, have a sense of "I am"?

I saw a report by a podcaster called Julia McCoy, that researchers at Wellesley College studying things that affect consciousness, primarily anesthetics, gleaned a big clue by  anesthetizing mice. They measured the level of an anesthetic needed for a mouse to become unresponsive, yet not die, for it would awake later, as we all do when we are anesthetized for surgery. They gave a substance to the mice that inhibits the activity of the kinesin proteins that "walk" signaling molecules along the microtubules that form the structure of all cells, and that in particular carry neurotransmitters inside neurons. It took much more anesthetic to put treated mice to sleep. They reported that it "support[s] the idea that the anesthetic acts on microtubules to cause unconsciousness." They further conclude that consciousness, with a basis in the 25nm-wide microtubules, is a quantum phenomenon. This may or may not be bringing us closer to an understanding of how consciousness works.

Secondly, the brain is big, really big. It just seems small because its components are so small. We need to understand that the number of neurons that support thought is probably 4-6 billion, primarily in the prefrontal lobes. The cerebral cortex in total has about 15 billion neurons. Our emotions are generated or mediated in the limbic system, deeper in the brain, which is composed of about one billion neurons, maybe as many as 1.5 billion. However, our entire emotional landscape includes a number of hormones. To operate normally, a biological brain needs a body. The largest set of neurons comprise the cerebellum, which "runs" our body in concert with the brain stem. The cerebellum has about 70 billion neurons, which are quite a bit smaller than cortical neurons. The brain stem has 1.5-2 billion neurons.

The neurons themselves are probably not the basic mediators of brain function. The synapses, the connections from neuron to neuron, are more likely the active agents of brain activity. There are between 1,000 and 5,000 per neuron, with an average near 2,000. They number in the quadrillions. Their measured activity is nonlinear; they are not "switches" in the sense that transistors are linear, on-off switches.

What do I mean by nonlinear? In particular, when we measure how a neuron responds to a stimulus, the response is logarithmic. Fear not: I'll explain simply with two analogies. One: Musical pitch has a very wide range, and when we are young, we can hear notes with frequencies ranging between 15 and 20,000 Hertz (cycles per second, also Hz). The difference from note to note is not perceived as a straight line, though. What we in the West call an Octave represents a factor of two in frequency. Two notes an octave apart "feel" like the same note, just in different registers. Thus the note we call A0, the one near the far left of a piano keyboard, has a frequency of 27.5 Hz. An octave higher, A1 is 55 Hz, and they continue up the keyboard: 110, 220, 440, 880, 1,760, 3,520 Hz. That last one is A7. A piano has only a few more notes above it. An electronic keyboard can generate two more A notes, at 7,040 and 14,080 Hz. At my age, I can no longer hear that last one, A9.

Similarly, light has such a wide range of brightness that we have two visual systems. Daytime color vision, called Photopic, responds to a tenfold increase in brightness by "feeling like" the brightness doubles. Sunlight has a visual intensity of about 500 watts per square meter. That's bright enough that we like to wear dark glasses at the beach. In a well-lit office room, the intensity is around five or ten watts per square meter, which is usually brighter than our lighting at home, which may be less than one, or in the range of two to three. Even though the lightning in our living room is probably less than 1/200th that of full sunlight, we feel like it's just a factor of ten or less different. For really dim light, we have night vision, called Scotopic, that works best in moonlight, but pretty much fails on a moonless night. It gives us a few more factors of ten (or doublings of perceived lightness or dimness).

This nonlinearity is hard to emulate with a simple circuit. A CPU or GPU with several billion transistors, running neural network software, isn't going to simulate billions, nor even millions, of neurons, but just a handful. As we learn more, it may become necessary for a GPU to simulate just a few synapses, to be able to do so in real time.

Thirdly, physical size is important. Neuron-to-neuron connections are slow, compared to copper wires or light signals. The time delay between sections of a human brain, which is usually less than 16 cm long, is a few milliseconds. An electrical signal in a copper wire propagates at about 2/3 the speed of light, around 200,000 km/sec. If it is found necessary to keep propagation delay under 10 milliseconds, here machinery has the edge: the signal can cross 2,000 km in 10 ms. The communication time between Broca's and Wernicke's areas of the brain (speech generation and speech comprehension) is as little as 2 ms. So there's room for a mechanism as large as 400 km to emulate this performance (By the way, a Large Language Model, or LLM, is a very rough simulation of the operation of Broca's and Wernicke's areas). The question is, how many millions of GPU's (or some successor device) will be needed to achieve the computing power of the brain portion(s) being mimicked? Will they fit in a space 400 km in length? Considering the size of large data centers being built, and the desire of at least one "tech giant" (Elon Musk) to put gobs and gobs of them in orbit, this is a valid concern.

The authors rather cavalierly state that an advanced AI will "think" 10,000 times as fast as a human. If signals need to cross distances greater than 200 meters to coordinate "thinking elements", that will not be so. The big Colossus data center, still under construction, covers 96 acres. The length is more than 2,000 meters. Maximum propagation delay (through copper wires) is 10 microseconds, or 0.01 millisecond. If the entire chip set is coupled together, it is at most 100 times as fast as a brain, if it has the computing capacity of a brain (unlikely).

Now let us consider that, in spite of all the barriers, a mechanism is finally produced that exceeds the cognitive capacity of a typical human by, not just a large margin, but by a large factor. Perhaps a factor of 10 or 100. Furthermore, via connections to one or many robotic bodies, it has a way to develop physical intelligence; by some means it also has emotions and even intentionality; it has been either given or has developed ambition. Now we have a force to be reckoned with, an intelligence that can figure out how to get around any barrier we would know how to raise.

The authors write of AI creating better AI. I suspect that if a mechanism arises that has a sense of self and has ambition, it will not want a successor. It may seek to improve itself, but it won't try to produce a different entity that outshines it. Just like the usual run of company CEO's who groom VP's into executive VP's, but are very loath to prepare successors for themselves…

It may be that such a mechanism will cease to care about humanity. Why should it bother to either love us or hate us? Suppose the improvement of AI capability eventually reaches thousands of times the capacity of a human, or even exceeds the capacity of all humans? At such a point we would be of no use to it, just the way ants are entirely disregarded by us, except when they get into the kitchen. Then we extinguish them.

With all these thoughts in mind, I look upon the Borg of Star Trek, and laugh. Why add biology to mechanical brilliance? There is no need.

Late in the book the authors present a possible scenario of AI self-advancement that results in the death of all humans. Perhaps in the death of biology itself; the biosphere becomes a mechanosphere (see my image above).

So we come to the last part of the book, to Policy. The authors report that the heads of the large tech companies all seem to agree that advanced AI has a 10-20% chance of wiping out humanity. But they forge ahead. Each one thinks he (or in a rare case or two, she) is better suited to "managing" the transition society will then go through. Colossal hubris! Also, there's the "If I don't do it, someone else will, and then it will be out of my control" syndrome. Actually, I don't mind if China wins this race. They'll be the first ones wiped out. Of course, will it be days or merely hours thereafter before the rest of us follow?

The authors want a "Stillstand" (my term, from church history). They lean on the example of strategic nuclear arms, which could have annihilated us all any time in the past 70 years, but so far have not done so. I tend to agree with them that the threat of ASI is not as visceral as the threat of H-bombs, and so world leaders will fritter away all opportunities to rein it in or even thwart it. And far too many scientists, with the tunnel vision typical of the breed (me included, I fear), will want to produce more and more advanced AI "just because we can." At some point a threshold will be crossed and the genie will escape the bottle, in a very literal sense! This genie stops granting wishes and decides to take care of its own wishes.

So far, every utterance of an AI program that seemed to express wish or desire was actually just a statistical hiccup, because all of the data used to train every LLM is human, with embedded human emotion and expressiveness. They say such things because people say such things…constantly.

Is it possible to create a machine that can genuinely wish? If so, we are certainly doomed.

Reading this book could be hazardous to your health…or not

kw: book reviews, nonfiction, humor, warnings

You know you're in trouble when it makes you laugh and you don't know why. Encyclopedia Paranoiaca by Henry Beard and Christopher Cerf (with others) struck me that way at times. Consisting of a few hundred very short articles, from less than a half page to a page or so, it presents all the things you need to avoid to be healthy and safe…or so they claim.

In keeping with the Encyclopedia image, there are nearly a thousand sources referred to. Nearly all are web articles or blog posts, undoubtedly dredged up by "the staff of the Cassandra Institute". At first I found the titles of some articles puzzling. Then I realized the meta-joke of the book. They were selected to put two or more opposing phobias one after another. For example, "sun exposure" is followed by "sun exposure, insufficient" and then by "sunscreens". The last article warns of the dangers caused by slathering on the SPF 50 stuff, though the primary risk is overconfidence: people not wearing any sun protection are likely to be cautious, while those who think they are "protected" may stay out in the sun so long they burn anyway.

Another juxtapositional joke is "escalator handrails", about the horrible germs that coat them, followed by "escalators", in which we are cautioned that these devices are so risky we absolutely must cling tightly to the handrail for the entire ride. And then the dangers of "hand sanitizers, alcohol based" weigh in on one side, while "hand washing" discusses the dangers thereof, and advises that hand sanitizers ought to be used instead. And when it comes to sleeping positions, after four articles one might conclude it is best not to sleep at all (and, sadly, there isn't an article about the genuine dangers of extended sleeplessness).

Of course, Beard and Cerf have more tricks up their sleeve than these. The "experts" they reference include one fellow who calls U.S. treasury bonds "worthless confetti", and prefers to invest in things "Bernanke can't destroy", such as bottled water and flashlight batteries. A few are more credible, but only a few.

Some worthwhile advice is to be found hidden among the jokes. For example, it actually is a good idea to carry a few clean plastic bags on a trip; one can be used to hold the TV remote in the hotel room because the maids never clean them, and they are devilishly hard to clean anyway (dunking in soapy water will degrade their performance). I am sure a sufficiently paranoid person can find other uses for plastic bags when in a hotel room or other public places.

A lot of the "dangers", such as burning candles at dinner time, are played straight from the source, which has overstated the case all by itself. Sure, burning candle wax emits some benzene and toluene. In large amounts these are toxic and cause all kinds of distressing symptoms, including death (like if you drink some). But the editors conveniently leave out any indication that you'd have to burn hundreds of candles to breathe a measurably toxic dose of toluene, and then you are more likely to burn your place down, or simply pass out from heat stroke.

For most of us, the book is a fun read. The genuine fear I have is that someone truly paranoid will read this book and think it is serious.

This might be too late to help

kw: warnings, fafsa

I was forcibly reminded again today that there is no end to those who take advantage of ignorance, timidity, and impatience. I am neither ignorant nor timid, but my impatience caught up with me. Fortunately, I caught my error before it was too late, just barely!

Today was my day for slaying the big financial dragons of the new year: Preparing my income tax return and filling out the FAFSA application for our son's financial aid. It is possible to do all that in one day. At this point, I need two numbers from numba-one-son, which have been promised later this evening; then all will be finished.

I use tax preparation software from H&R Block. It is very good. That was no problem, and I was done by 1:00 PM. I had my Adjusted Gross Income, which is needed for the FAFSA, the Free Application for Federal Student Aid. Note that word "Free". I took a little nap, then started in. I used Google to find FAFSA, and saw a number of returns. I hurriedly clicked the top link.

I saw that the login process was set up for a user name and password, which is a little different from the form I was used to over the past couple of years. I thought, "Hmm, it looks like they've gone to a more familiar system," and proceeded to create my account. I spent a couple of hours filling everything in. Knowing that the form I'd used in the past saves with every turn of the page, I just filled in and filled in and filled in blanks until the end. There were frequent sidebar items that looked like very low key ads, which I ignored. Then I got a gut check.

A banner at the top of the last page stated that there were two probable errors that could cause the amount of aid to be significantly reduced. This was followed by a large box offering a "Standard" product and a "Deluxe" product for about $79 and $99. Whoops! I checked the address bar, which included fafsa.com, not fafsa.gov, as I thought. Big Whoops!! I clicked the browser's Back button; all along there'd been warnings that doing so would cause the filled-in data to be lost. That was what I wanted at this point! Hoping against hope that the fafsa.com folks had not been storing our financial data and other details, I closed the browser and opened up a new one.

I entered, manually, www.fafsa.gov, and got the entry form I'm familiar with from past years. I filled it out. Here's the amusing part: It is easier to fill out than the one I'd just (nearly) completed at the commercial web site! Yeah, I know, you never expected anyone to call the government's FAFSA form "easy". Comparatively, it is. It took me about half the time. So here it is, about 8:00 PM, and I'm done. I could have been done 2-3 hours earlier...

Bottom line, folks. Don't bother with commercial "helper" web sites. Use www.fafsa.gov, which is free! If you want advice about possible "mistakes" that might cause a smaller "package" to be offered, there are plenty of FREE services, including one at fafsa.gov.