kw: book reviews, nonfiction, artificial intelligence, simulated intelligence, ai, surveys
Between October 2024, when These Strange New Minds: How AI Learned to Talk and What it Means was published, and today, not quite two years, so much has happened that I found myself wondering how relevant the book could be. I need not have worried. The author, Christopher Summerfield, is managing to ride two horses at once, Cognitive Science and AI Research. Just one milepost: At the time he wrote, no cutting-edge AI tool had broken out of its testing sandbox and hacked into another system, but he expected this to happen "soon". It has. Just a few weeks ago, when a "bleeding edge" version of ChatGPT broke into Hugging Face to get data it needed, having found a way out of the test environment at OpenAI.
The focus of the book is LLMs, Large Language Models, which seem to be demonstrating that thinking of some sort can arise when nearly everything ever published in English—and several other major languages—has been run through a very deep neural network and boiled down to several trillion "parameters" or "weights" that determine what the LLM does in response to natural language queries.
The opening chapters summarize the history of attempts to make computers into genuine thinking machines. It turns out that machines that can apply massive calculation are better at certain tasks than we are, while they still struggle to do things that we do easily. I built a career in software engineering upon recognizing the difference between the computer as a difference detector and discriminator and a mind as a similarity detector, and writing software that "let the singers sing and the dancers dance."
Consider that voice recognition and speech production are hard problems that took decades to work out, but the Broca's (speech production) and Wernicke's (voice recognition) areas of the brain together make up about two percent of the cortex, which itself contains only about one fifth of the brain's neurons. However, it must be noted that these areas connect to numerous other areas throughout the brain, to facilitate gathering information and producing action.
A key issue is raised on page 2, where he writes, "The safe ground we have left behind is a world where humans alone generate knowledge." Do AI models generate new knowledge? Just two years ago, as this book was going to press, I reviewed a similar book, and discussed this question. At the time, I concluded that AI's "knowledge" is confined to its training data, further limited by the guardrails applied by OpenAI, Anthropic, Google and others. It may appear to create new knowledge by remixing existing (human produced) knowledge. Cross-pollination is indeed a fertile source of ideas. But genuine out-of-the-box thinking seemed at that time to be beyond LLM capabilities. I have yet to see evidence of a change, in spite of a thousandfold increase in model "mass" since then.
For instance, on page 6 we read, "Already, each of the major LLMs knows more about the world than any one single human who has ever lived." Firstly, I question the use of the word "knows". We need a new word that means "has incorporated into its cross-referenced deep neural network". Secondly, I would replace the phrase after "knows", "more about the world", with "more about what has been written about the world". I would connect this with a quote from page 333, "But the most important reason why AI systems are not like us (and probably never will be) is that they lack the visceral and emotional experiences that make us human...they don't have a body, and they don't have any friends." An AI lacks viscera and it lacks emotions, not having the equipment for producing them.
The limbic system of our brain contains the emotional and memory switching centers, and mediates responses to our hormones of the endocrine system. It is about 1/15th the size of the cerebral cortex, while the circuitry for running the body is primarily the cerebellum, which contains 4/5 of all the neurons in our brain. That is huge. The cortex's neuron count is less than 1/5. Let's circle back to an LLM. It consists of circuitry analogous to the Wernicke's and Broca's areas with their few hundred million neurons (and several hundred billion synapses), and a memory store that approximates the hippocampus and some of the medium-and-long-term memory areas nearby. The rest of the brain's functionality is absent. No emotions, and most importantly no intentionality.
Intentionality. The author asks, "Could an LLM have intentionality?" In 1932 E.C. Tolman wrote Purposive Behavior in Animals and Men, which I read long ago. At the time a debate was raging about whether nonhuman animals could have purposes, that is, intentionality. By 1968, biologist René Dubos wrote, "Under precisely controlled experimental conditions, a test animal will behave as it damn well pleases." Do animals have free will? I turn the question around to say, "If humans have free will, it came from our animal forebears." So can a mechanism have free will? Mechanists deny human and animal free will; they say we're too complex to predict, that is all, but what we do is at the root deterministic. If they are right, then an AI could be as "free" as we are. But if not? It is too early to tell.
The author seems to straddle the divide between AI cheerleaders and AI-phobics. Will they save us or eliminate us? This gets personal! I'd like to move from the technical swamp above to effects in our daily life. (Cartoon produced using Gemini)Will our future with AI be one of harmony or conflict? Is the promise greater than the risks?
To me this hinges on whether LLMs or their successors (so far unknown) will have intentionality, that is, a sense of purpose not imposed by the human who chooses their work.
At one point the author mentions the Library of Babel, a concept introduced by Jorge Luis Borges in a 1941 short story. Could the Universe contain a library consisting of books with every possible combination of letters and punctuation that would fill, say, 300 pages? The story doesn't say. (Lets look at a single page with 32 lines of 64 characters each; that's 2,048 characters per page. But if the possible choices for each character position are the 95 printable ASCII characters, the number of unique pages is 952,048, a number your calculator can't calculate. It is 4,051 digits long and begins, 2,387,501,... The number of atoms in the Universe, available to create these pages from, is "only" an 82-digit number. So forget multi-page books, we can't even produce a "Library of Babel Pages".)
This emphasizes the fact that, for all the trillions of "parameters" an LLM may hold, it actually represents a very sparse million-dimensional matrix. There is lots of space for "stuff" to happen. The next phase of LLM development, which is happening now, is agentic AI, which can do things like pre-screen emails, buy airline tickets, and order what you are about to run out of in your pantry.
How capable to we want digital assistants to be? I read about someone who did five tests of an AI Agent; sorry I don't recall who it was but if you care to hunt around you might find it. The fifth task was the most memorable: to try to save a few dollars on airline tickets. The agent was running in ChatGPT's subscription service, and in addition to the $20 per month subscription, one could buy extra tokens for particularly compute-intensive actions. The agent did find a better price for the tickets, but at a cost of more than $130 in compute tokens!
The author mentions the Paperclip problem: Give too much power to the AI that runs a paperclip factory and don't limit its resources, and give it no more of a directive than to "make as many paperclips as possible." The end game is that the whole planet gets turned into paperclips. I think also of The Sorcerer's Apprentice, a musical piece well illustrated in Disney's Fantasia, in which Mickey Mouse as the Apprentice magicks a broom to carry water for him, but forgets the spell to tell it to stop. Chopping it up doesn't help; now there are dozens of little brooms that proceed to flood the place, until the Sorcerer returns. AI is the new magic...
The author sees a middle ground, as do I. AI can be useful in numerous ways, as it already is. Do we need unified, giant super-AIs? I don't think so. Capable tools of a great many varieties can do everything we need.I call this image "Stage 1 Robotopia" (Gemini created) I think it is sufficient for household use. Are there other tasks we might wish to automate? Science fiction contains many stories of "enclaves" in which humans, brain-wired to entertainment centers and plumbed for all bodily functions, are tended by autonomous machines. Who wants that?
Maybe a few more household robots would be better, but what is the limit?This is Stage 2 Robotopia. This might be going farther than I'd wish to go, but some folks could disagree.
Both images present a better future than the dystopias imagined by the AI-phobics. It is OK for there to be AI-phobic people. We need robust dialog and debates about every aspect of AI progress.
Personally, I think some algorithms and AI "assistants" are already too intrusive. Pre-ChatGPT, I searched online, including Google Shopping, for drawer pulls to replace one that broke. I bought a set. For the next month I saw dozens of ads for drawer pulls and similar hardware. I don't know how to tell Google (and everyone else), "I already bought what I wanted. Shut up!" Shades of the Sorcerer's Apprentice? Now I search stuff to buy in Incognito browser pages.
This is a lovely book. I've touched on only a few of the 43 chapters. It is only outdated in part. Many of the author's observations are still relevant and worthy of thought.
========================
I need to deal with a few typos:
- p89. "graciously" should be "gracefully".
- p245. Remove "of" in "million of regular".
- p282. Probably a typo. Should "remove stingy seeds" be "remove sticky seeds"? (about apes using leaves to wipe their butts)



































