kw: personal notes
This will have to be short. I've finished a couple of books, but the reviews will have to wait a bit longer. I was injured a week ago and have been unable to type effectively. Eight sutures near the tip of one finger. I just took out the sutures last evening, so maybe in a few more days I can type without risking a pop-open. To the hundred or so people who have been following along, sorry for the delay and thanks for reading.
Sunday, March 30, 2014
Wednesday, March 19, 2014
This fellow REALLY knows how to chase a rabbit
kw: book reviews, nonfiction, words, wordplay, etymology
Herewith, my own definition:
To chase a rabbit - to follow a series of associations, particularly as they lead in unexpected directions. Particularly applies to rambling discussions.
The above is a pretty good metaphor for a bull session. Bull, of course, being short for bullshit, and both parts of that word have their roots and uses, both together and separate, and so forth. However, The Etymologicon: A Circular Stroll Through the Hidden Connections of the English Language by Mark Forsyth (AKA The Inky Fool) is a far deeper chase through brambles and weeds than I am competent to pursue. As the subtitle says, the work is circular, beginning and ending with the question of whether one can turn up a book (you can't).
Looking back, I find that the author could have closed the loop in a number of places. He is one who prefers to keep the rabbit running as long as possible, or at least until a specified word count is achieved. The 113 chapters cover 259 pages, plus another 19 of quizzes and other arcana. I think it most useful to pick 3 spots at random (and an easy cop out):
Herewith, my own definition:
To chase a rabbit - to follow a series of associations, particularly as they lead in unexpected directions. Particularly applies to rambling discussions.
The above is a pretty good metaphor for a bull session. Bull, of course, being short for bullshit, and both parts of that word have their roots and uses, both together and separate, and so forth. However, The Etymologicon: A Circular Stroll Through the Hidden Connections of the English Language by Mark Forsyth (AKA The Inky Fool) is a far deeper chase through brambles and weeds than I am competent to pursue. As the subtitle says, the work is circular, beginning and ending with the question of whether one can turn up a book (you can't).
Looking back, I find that the author could have closed the loop in a number of places. He is one who prefers to keep the rabbit running as long as possible, or at least until a specified word count is achieved. The 113 chapters cover 259 pages, plus another 19 of quizzes and other arcana. I think it most useful to pick 3 spots at random (and an easy cop out):
- p 27 – The phrase we find is letting the cat out of the bag, which actually has to do with someone trying to sell a pig in a poke (sack). You avoid being scammed by peeking inside, and seeing there is no pig, you let the cat out. But this was a side path in a discussion of archery, and the term point blank. No one should be surprised to know that blank in this case derives from blanc, or white in French. White is the color of the bullseye. If you are standing close enough to the target, you can hit the center without aiming high to account for gravity: you point right at it because you are at point blank range. The term bullseye is not followed, but could lead along an interesting path.
- p 58 – This begins a discussion of three names related to "justice", loosely construed, and their relation in particular to capital punishment. The names are Guillotin, Derrick, and Jack Robinson. We find that Thomas Derrick was a thoroughly bad man whose death sentence was commuted on the condition that he become the executioner for the Earl of Essex. The gallows is also called a derrick in his "honor", and the word is now applied to various kinds of lifting structures. Jack Robinson has several variously plausible sources, but the most likely is that Sir John Robinson, constable of the Tower of London, carried out executions with great efficiency, getting them over with so quickly that it was said you couldn't say "Jack Robinson" between the time you were called out and your head rolled. The final part of executions got even quicker when the mild Dr. Guillotin was placed on a commission to reform executions. Having seen a German head-chopping blade, he told his colleagues that if executions had to be done, that machine would do it best. Now even the Germans call it the guillotine.
- p 247 – The word amateur, which means lover, and why love is zero in tennis. If the gentleman who created lawn tennis had had his way, it would be distinguished from hard-court tennis by being called sphairistike, which rhymes with "very sticky". We've been spared, at least in part because tennis players didn't know enough Greek to figure out the pronunciation. But then, love? Because if you do something for love, you'll do it for nothing. I still don't like it (and the entire love-15-30-40 sequence, which makes no numeric sense, because you have to win on an advantage anyway, after winning at least 3 serves).
Now if you can't imagine how the Inky Fool gets from any of these points to the others, do just read the book. Great fun!
Saturday, March 15, 2014
Mary's monster also is a product of his times
kw: book reviews, nonfiction, literature, history, biographies, gothic horror
A favorite recreation during the Age of Enlightenment was attending scientific and philosophical lectures and demonstrations. Late in the period, popular fancy shifted towards anatomy and dissections. Then, as Volta and Faraday and others were elucidating the nature of electricity, the experiments of Galvani on dismembered frogs led to a huge interest in using electricity to resuscitate the dead. It wasn't long before dissections were accompanied by electrical demonstrations upon corpses, which temporarily restored certain kinds of activity, and seemed on the verge of restoring life.
The great demand for corpses led to multitudes of "resurrection men", grave robbers who sold corpses to doctors and demonstrators, even as morticians employed by the better-off struggled to invent devices to prevent the extraction of bodies from coffins and mausoleums. It was only in the 1830s, after some such as William Burke and William Hare in Edinborough and John Bishop and Thomas Williams in London were convicted of murdering indigents and prostitutes to feed the trade in corpses, that an Anatomy Act was passed in Parliament, and later in other nations. Under such an atmosphere, and with many stories on such themes already in print, it was simply a matter of time until someone would write the ultimate horror story of obsession, grave robbing, galvanism and scientific hubris.
Also in such an atmosphere, the gloomiest of the gloomy seemed to gather themselves around one Percy Shelley, such that on a stormy night in a poorly-kept inn near Lake Geneva, he and others, including the supremely dissolute Lord Byron and Shelley's mistress Mary Godwin, passed the time by embarking on a challenge to write the best "ghost story". Mary began to write a short story, but then spent a number of weeks writing a book, which after a bit of struggle was published as Frankenstein, or, The Modern Prometheus.
The tenor of the early 1800s and the attitudes and personae of the Shelley/Godwin circle are ably captured in Roseanne Montillo's biography of Mary, The Lady and Her Monsters: A Tale of Dissections, Real-Life Dr. Frankensteins, and the Creation of Mary Shelley's Masterpiece. The book is based on epic amounts of research into the lives, not only of the Shelleys and Byron, but everyone associated with them and those who formed the milieu in which they moved, the atmosphere of their minds.
Mary and Shelley married a few months after returning from Geneva to London, after Shelley's wife killed herself. Six years later he died in a boating accident, aged not quite 30. Their association lasted but 8 years. Yet it was a productive period for them both. Percy Bysshe Shelley is still considered the finest lyric poet of the English language. Pity that lyric poetry is about a century out of style; he is nearly unknown today.
I cannot recall anyone mentioned in the book who lived anything like a usual life span, as we consider it today. With child and infant mortality in the 50% range, and adult longevity somewhere in the 50s or less, people seemed to pack more into the years that they had.
If you combine everything from all the films with "Frankenstein" in their title, you'll have less substance than you'd find by reading Mary Shelley's novel. Far, far from the grunting horror of the original film, Mary's creature, produced and vivified by Victor Frankenstein, became literate and well spoken, but he was so badly made (V.F. was no artist!) that he was horrible to look upon. Driven out by all who saw him, his only friend was blind. I, and many of my contemporaries have read Frankenstein, or, The Modern Prometheus. I wonder if anyone under about 40 years of age has done so. If you haven't read Frankenstein, read that first. Then come back and read The Lady and Her Monsters. It will mean a great deal more. Oh, and if you don't know why Prometheus is important, at least a nodding acquaintance with Greek mythology is in order.
A favorite recreation during the Age of Enlightenment was attending scientific and philosophical lectures and demonstrations. Late in the period, popular fancy shifted towards anatomy and dissections. Then, as Volta and Faraday and others were elucidating the nature of electricity, the experiments of Galvani on dismembered frogs led to a huge interest in using electricity to resuscitate the dead. It wasn't long before dissections were accompanied by electrical demonstrations upon corpses, which temporarily restored certain kinds of activity, and seemed on the verge of restoring life.
The great demand for corpses led to multitudes of "resurrection men", grave robbers who sold corpses to doctors and demonstrators, even as morticians employed by the better-off struggled to invent devices to prevent the extraction of bodies from coffins and mausoleums. It was only in the 1830s, after some such as William Burke and William Hare in Edinborough and John Bishop and Thomas Williams in London were convicted of murdering indigents and prostitutes to feed the trade in corpses, that an Anatomy Act was passed in Parliament, and later in other nations. Under such an atmosphere, and with many stories on such themes already in print, it was simply a matter of time until someone would write the ultimate horror story of obsession, grave robbing, galvanism and scientific hubris.
Also in such an atmosphere, the gloomiest of the gloomy seemed to gather themselves around one Percy Shelley, such that on a stormy night in a poorly-kept inn near Lake Geneva, he and others, including the supremely dissolute Lord Byron and Shelley's mistress Mary Godwin, passed the time by embarking on a challenge to write the best "ghost story". Mary began to write a short story, but then spent a number of weeks writing a book, which after a bit of struggle was published as Frankenstein, or, The Modern Prometheus.
The tenor of the early 1800s and the attitudes and personae of the Shelley/Godwin circle are ably captured in Roseanne Montillo's biography of Mary, The Lady and Her Monsters: A Tale of Dissections, Real-Life Dr. Frankensteins, and the Creation of Mary Shelley's Masterpiece. The book is based on epic amounts of research into the lives, not only of the Shelleys and Byron, but everyone associated with them and those who formed the milieu in which they moved, the atmosphere of their minds.
Mary and Shelley married a few months after returning from Geneva to London, after Shelley's wife killed herself. Six years later he died in a boating accident, aged not quite 30. Their association lasted but 8 years. Yet it was a productive period for them both. Percy Bysshe Shelley is still considered the finest lyric poet of the English language. Pity that lyric poetry is about a century out of style; he is nearly unknown today.
I cannot recall anyone mentioned in the book who lived anything like a usual life span, as we consider it today. With child and infant mortality in the 50% range, and adult longevity somewhere in the 50s or less, people seemed to pack more into the years that they had.
If you combine everything from all the films with "Frankenstein" in their title, you'll have less substance than you'd find by reading Mary Shelley's novel. Far, far from the grunting horror of the original film, Mary's creature, produced and vivified by Victor Frankenstein, became literate and well spoken, but he was so badly made (V.F. was no artist!) that he was horrible to look upon. Driven out by all who saw him, his only friend was blind. I, and many of my contemporaries have read Frankenstein, or, The Modern Prometheus. I wonder if anyone under about 40 years of age has done so. If you haven't read Frankenstein, read that first. Then come back and read The Lady and Her Monsters. It will mean a great deal more. Oh, and if you don't know why Prometheus is important, at least a nodding acquaintance with Greek mythology is in order.
Tuesday, March 11, 2014
War from the inside
kw: book reviews, nonfiction, soldiers, war, war stories, memoirs
CJ stands for Crazy Jay, whose civilian name is Dillard Johnson. With the help of James Tarr, he has produced a memoir: Carnivore: A Memoir by One of the Deadliest American Soldiers of All Time. According to an official U.S. Army study titled On Point, his team of about six scored about 3,000 KIA (killed in action) against Iraqi military and paramilitary forces, just in the first weeks of the Iraq war beginning in March, 2003, operation Iraqi Freedom. The facts, corroborated by numerous interviews, emphasize the combination of superior armor and weapons, fighting skill and training, and a lot of great good luck. "Carnivore" is the name Johnson's team gave to their Bradley Fighting Vehicle, a 25-ton mini-tank.
I seldom read war memoirs. Most are simply chest-beating bragging. Somehow, when I saw this book, I couldn't pass it up, and I am glad I read it. War truly is hell, the scenes after battles are truly hellish, and it is amazing that men can enter hell and return with any semblance of sanity. I am not disturbed by reading that about 1/3 of combat veterans have symptoms of PTSD (we used to call it "shell shock"). I am instead astounded that 2/3 of them can return home and get on with their lives, not unscathed, of course, but not disablingly damaged.
Early in the book, Johnson describes getting into Iraq, which was defended by berms along the border with Kuwait. He says you can see them on Google Earth, and indeed you can. The only set of imagery GEarth has is from July 2006, as seen here. This is south of Basrah. For scale, the larger bomb crater is 17.5m across. The yellow line is the border. Note a berm on the Kuwait side also, nearer the road. A third berm, ~350m to the south, is the edge of the UN Peacekeeping Zone prior to 2003. The Iraqi berm has been pierced in many locations, to let through American armor, which at the time consisted primarily of Abrams tanks and Bradley fighting vehicles. The other two berms also had to be pierced, but not in as many places, as they weren't being defended.
I was tickled by a statement about Johnson's sniping activities later in his career (121 kills). He writes, "In war, if you find yourself in a fair fight, your tactics suck." This goes along with my own attitude, that there is no such thing as a fair fight: Don't get into a fight until you can assure you have the advantage. That is just good tactics.
So I'll leave it at that. Read Carnivore for yourself. You'll be glad you did.
CJ stands for Crazy Jay, whose civilian name is Dillard Johnson. With the help of James Tarr, he has produced a memoir: Carnivore: A Memoir by One of the Deadliest American Soldiers of All Time. According to an official U.S. Army study titled On Point, his team of about six scored about 3,000 KIA (killed in action) against Iraqi military and paramilitary forces, just in the first weeks of the Iraq war beginning in March, 2003, operation Iraqi Freedom. The facts, corroborated by numerous interviews, emphasize the combination of superior armor and weapons, fighting skill and training, and a lot of great good luck. "Carnivore" is the name Johnson's team gave to their Bradley Fighting Vehicle, a 25-ton mini-tank.
I seldom read war memoirs. Most are simply chest-beating bragging. Somehow, when I saw this book, I couldn't pass it up, and I am glad I read it. War truly is hell, the scenes after battles are truly hellish, and it is amazing that men can enter hell and return with any semblance of sanity. I am not disturbed by reading that about 1/3 of combat veterans have symptoms of PTSD (we used to call it "shell shock"). I am instead astounded that 2/3 of them can return home and get on with their lives, not unscathed, of course, but not disablingly damaged.
I was tickled by a statement about Johnson's sniping activities later in his career (121 kills). He writes, "In war, if you find yourself in a fair fight, your tactics suck." This goes along with my own attitude, that there is no such thing as a fair fight: Don't get into a fight until you can assure you have the advantage. That is just good tactics.
So I'll leave it at that. Read Carnivore for yourself. You'll be glad you did.
Sunday, March 09, 2014
Space Opera redivivus
kw: book reviews, science fiction, space opera, space aliens
Stephen King calls Jack McDevitt "The logical heir to Isaac Asimov and Arthur C. Clarke." Kind words indeed, but just a bit inflated. Nobody has yet come close to the more than 400 books written by Asimov, nor the breadth of vision and versatility of Clarke. However, McDevitt's 21 novels are nothing to sneeze at, and I greatly appreciate his fresh insights into alien species.
Starhawk is the 7th of the Academy series, also called the Priscilla Hutchins series after the protagonist. I read (and reviewed) the 6th volume, Cauldron, in 2008, but don't recall Ms Hutchins being particularly prominent.
In the Galaxy of the Academy series, alien life of any kind is rare, and aliens—so far located—bright enough to have language can be counted on one's thumbs and big toes. The brightest alien species is stunningly boring, but a series of a dozen or so "great monuments" shows that the Universe can produce beings of amazing power, but, apparently, less-than-amazing longevity.
In Starhawk the author explores two ideas of alien contact, one with an automated device that tells them they "just missed us", and another with an apparently planet-wide being that is helpful but indifferent: not hostile, but not interested in being visited long-term either.
The core of the action surrounds Ms Hutchins, who eventually earns the nickname Hutch. A newly-licensed starship pilot, she trying to start a career in a shrinking economy (2007-2012 writ large), in an era of increasing pressure to cut back the space program. Hard choices and a few untimely deaths provide coming-of-age experiences for her. Early on, I had a certain fellow pilot pegged as her eventual mate, but McDevitt outwitted me; the pilot dies attempting to divert a terrorist-controlled starship being used as an impact weapon.
The mix of hard science with just enough blue-sky stuff (like FTL) seems just right to me, and makes for compelling reading. It provides a canvas on which the alien contact stories, the exoarchaeology in which McDevitt likes to indulge, can play across the imagination. I wonder if I can find all five of the Academy books I've missed…
Stephen King calls Jack McDevitt "The logical heir to Isaac Asimov and Arthur C. Clarke." Kind words indeed, but just a bit inflated. Nobody has yet come close to the more than 400 books written by Asimov, nor the breadth of vision and versatility of Clarke. However, McDevitt's 21 novels are nothing to sneeze at, and I greatly appreciate his fresh insights into alien species.
Starhawk is the 7th of the Academy series, also called the Priscilla Hutchins series after the protagonist. I read (and reviewed) the 6th volume, Cauldron, in 2008, but don't recall Ms Hutchins being particularly prominent.
In the Galaxy of the Academy series, alien life of any kind is rare, and aliens—so far located—bright enough to have language can be counted on one's thumbs and big toes. The brightest alien species is stunningly boring, but a series of a dozen or so "great monuments" shows that the Universe can produce beings of amazing power, but, apparently, less-than-amazing longevity.
In Starhawk the author explores two ideas of alien contact, one with an automated device that tells them they "just missed us", and another with an apparently planet-wide being that is helpful but indifferent: not hostile, but not interested in being visited long-term either.
The core of the action surrounds Ms Hutchins, who eventually earns the nickname Hutch. A newly-licensed starship pilot, she trying to start a career in a shrinking economy (2007-2012 writ large), in an era of increasing pressure to cut back the space program. Hard choices and a few untimely deaths provide coming-of-age experiences for her. Early on, I had a certain fellow pilot pegged as her eventual mate, but McDevitt outwitted me; the pilot dies attempting to divert a terrorist-controlled starship being used as an impact weapon.
The mix of hard science with just enough blue-sky stuff (like FTL) seems just right to me, and makes for compelling reading. It provides a canvas on which the alien contact stories, the exoarchaeology in which McDevitt likes to indulge, can play across the imagination. I wonder if I can find all five of the Academy books I've missed…
Wednesday, March 05, 2014
The Market is People
kw: book reviews, nonfiction, statistics, physics, stock markets
A financial market behaves like a small collection of quantum particles. This is my conclusion after decades of investing (sometimes lucky, sometimes not), and reading about them, from The Emergence of Probability and The Taming of Chance by Ian Hacking, to The Black Swan by Nassim Taleb and Beat the Market by Ed Thorp and Sheen Kassouf, and now The Physics of Wall Street: A Brief History of Predicting the Unpredictable by James Owen Weatherall. The shine isn't quite off Dr. Weatherall's first PhD yet—it is but half a decade—but already he exhibits a breadth of vision that sets him apart. He actually has two doctorates, in physics and in philosophy, so he has the kind of mind I like, not just thinking outside the box, but leaving all the boxes behind.
So why would he be interested in market analysis? For the same reasons a ton of physicists have had already: that is where the money is. Plus it has the un-ignorable allure of a challenge that is almost impossible, yet not quite. Given that many thousands of smart people have been trying to "beat the market" for, oh, half a millennium at least, a few have gotten rich, at least by chance, but rare indeed are those persons or funds who managed to stay ahead of the pack and get rich by actually betting on predictions that panned out, again and again. A physicist-run hedge fund called Renaissance is claimed to be one of them.
The bulk of the book is a history of statistical thought, as it developed over the past few hundred years, frequently in response to the desire to understand price fluctuations in markets for currency, commodities, or stocks and options of various kinds. The tools used for this began with the Normal (AKA Gaussian) distribution, the familiar Bell Curve. All kinds of additive phenomena obey Gaussian statistics, such as average height for men or women of a given ethnicity, or most famously, IQ. A particular Normally distributed population is completely described by a Mean (µ) and a Standard Deviation (σ). The shape is scalable, wider for large σ and narrow for small σ, but is otherwise fixed, so that 68% of the population is found within the range µ-σ to µ+σ, called the 1-sigma range; and the 2-sigma range encompasses 95% of the population. So, for IQ, at least among Euro-Americans, µ is standardized at 100 and σ at 15. Thus the range [70-130] includes 95% of these folks, and 68% are found in [85-115]. Public education was originally aimed at the 1-sigma group, and the rest were left to fend for themselves, until Special Education and Gifted Education movements arose to help out those in the "tails", whether duller or brighter.
Is the Normal distribution a good model of market fluctuations? Not at all. First, we must realize that human perception is involved. A $1 change in a $10 stock feels just as large as a $5 change in a $50 stock, particularly if you have 500 shares of the first one or 100 shares of the other. Both changes are 10% of your $5,000 investment. The chart below shows the day-to-day change of closing price for Coca-Cola common stock, since the beginning of 1986, expressed as a % of the prior day's closing price.
If we sort these numbers and plot them against a "Probability Ordinate", really an inverse Normal ordinate (I use the NORM.S.INV function in Excel 2010), we would get a scatter plot that closely follows a straight line if the distribution were Normal. But here is what we get instead:
If we extend a line tangent to the central part of the distribution, to -4σ or +4σ, it strikes at a 5% change, indicating that variations greater than this ought to be rare indeed (there are 7,101 daily changes plotted here). But what do we see instead? Going back to the original data sheet, I find 42 days on which the stock increased by 5% or more, up to nearly +20%, and 33 days on which it fell 5% or more, to nearly -25%. How'd you like to own a million shares of this stock and have it lose 1/4 of its value on a single day? So early on, the Normal distribution was found wanting.
Normal analysis was based on the concept of a random walk, also called the drunkard's walk. Its additive nature will always result in a distribution of final locations, say after ten staggers, that is Normal. So a different distribution with extra-wide excursions is needed. In an entertaining section, Dr. Weatherall describes a drunken firing squad. They have a target upon a very long wall, but being too drunk to point well, might shoot in any direction at all. Give them lots of ammunition (and hide somewhere until they run out), and the pattern of bullet holes will follow a Cauchy distribution. It looks a little like the Normal distribution, but has a pointier top, and most importantly, "fat tails"; that is, many points that are farther—or much farther—from the middle than a Normal distribution would predict. The distribution above is also fat-tailed, having lots of numbers outside the range we'd expect from a Normal distribution. To test a distribution for Cauchy behavior, plot it against a Tangent function evenly distributed in the range -π/2 to +π/2. For my 7,101 points, the Tangent function ranges from nearly -5,000 to +5,000, so the chart is thus:
The Cauchy distribution is clearly a bit too much, its tails are "too fat", compared to the tails of Coca-Cola daily price fluctuations. This kind of conundrum was tackled by many bright people, from Fischer Black to Benoit Mandelbrot. Mandelbrot probably came closest with fractal analysis, which wasn't wedded to integer exponents. But I got another thought as I read along.
The Normal and Cauchy distributions are related, being examples of Stable distributions. In one formulation, a parameter called α has a value of 2.0 for a Normal distribution, and a value of 1.0 for a Cauchy distribution. The fattest tails possible are at α=0, the Uniform distribution of infinite width. Mandelbrot had used fractal analysis to calculate a distribution with α of 1.7, closer to Normal but still with a fat tail. I realized that the most familiar distribution with at least one long tail is Lognormal, but it is confined to positive only values. Does it have a complex square root, perhaps? I sorted the squares of the KO daily changes and charted them against a Normal ordinate on a logarithmic scale. But there was a problem. On 245 days there was no change in price. Prior to the 1970s stocks were valued in 8ths of a dollar (12.5¢), and in pennies thereafter, though dividend allocations can be calculated to 0.0001¢ increments. Trades are reported to the nearest cent. Anyway, you can't take the logarithm of zero, so in my spreadsheet I used a value a little smaller than the smallest calculated nonzero value for those 245. They form the line at bottom left on this chart:
If trades were made with a continuous range of values, not limited by the minimum value, I would expect the left portion of the chart to be as linear as the rightmost. Quantization errors have artificially depressed the daily motion for about 15% of the trades. In the other charts, the two extreme values, -25% and +20%, seemed like outliers. Here, as the two rightmost data points, they are seen to be at most slightly larger than one might expect.
So, all you quants out there, working out the best formula for calculating risk. Give a little attention to the square root of the Lognormal distribution! Now, back to the book.
A key theme of the book is both the value and the danger of numerical models. A physicist understands that a model is always simplified, and cannot be appropriately used outside its range of application. When the people using models of financial systems, to set option prices and other instruments, are physicists, they will know this and avoid over-extending the model. People without physics education will not. When you have a black box program that seems to work magic, it is easy to use it everywhere (the parable of the man whose only tool was a hammer comes to mind).
There have been several major crashes in the past century, and only the one in 1929 was free of the influence of sophisticated statistical modeling tools. I say "sophisticated" because there were statistical tools in use a century earlier, but they were back-of-the envelope estimates at best. All of the more recent ones show at least traces of "broken model" influence, but the October 1987 crash was an overt "robo-trading" crash. This brings up another principle that physicists, at least, ought to keep in mind: the observer effect.
I am not just talking about Heisenberg Uncertainty. Rather, most observations of physical phenomena disturb the system being measured. I remember my father telling me not to check the air pressure in my bike tires so often, because each measurement caused some air to be lost. Later, working in electronics (in a time when the components were visible and manipulable by hand) I learned how to use a Wheatstone Bridge to measure DC voltage the most accurately, because it uses a counter-voltage to keep from bleeding extra current from the circuit. It is only good for very steady DC, of course. Thermometers change the temperature of the pot roast, but only a tiny bit; still the effect is not zero. But now imagine that you have half a million people whose livelihood depends on knowing the temperature in your pot roast, and they all insist on using their own thermometer. There won't be much left of the roast! THAT's what happened in October 1987.
The use of new tools changes the way markets work. What worked in September 1987 doesn't work today; what worked in 1997 or 2007 doesn't work now, and so forth. This pretty much negates the notion of an efficient market. It can only be efficient under two conditions:
The "efficient market" works like this: In comparatively quiet times, the asking price of a stock or whatever incorporates all the current knowledge about things that might affect its value in the future. To profit from trading that instrument, you either guess it might be underpriced, because of unknown or little-known information, or you try to learn something nobody else knows. The most common source of such knowledge is cadging or coercing it out of an insider, which happens a lot even though it is illegal. Quantitative analysis attempts to find patterns in price fluctuations that signal a change you can profit from. When someone finds a useful pattern, he or his company will profit from it for a while, until others catch on, then pretty soon everyone can do it, and the market is "efficient" again. So quants' work is a continual arms race. Thus, the tools used to test the market change the market.
But the markets are not that efficient. In the medium term they might be, but the momentary trading picture is much more emotional, and tiny bits of information or rumor disguised as information can sway a trader's estimate of value. If that trader is influential, and others see him (usually male) make a move they didn't contemplate before, some will follow. It can cascade into a large market move, that might last a matter of an hour or less, but might last a day or more, and then there is the potential for quite a swing, either towards a bubble or a crash.
The fragility of any market lies in the tendency for all the quantitative trading firms to use the same models, or models based on the same math, with the same or very similar trigger points. Certain rules instituted after 1987 can calm the flurry to some extent, but the events of 2007 to early 2009 present a case in which the agony was simply drawn out over the space of more than a year, rather than taking place in a month or less.
With the contents of this book under my belt, I ask myself, "What is the ordinary investor to do?" We don't have supercomputers and armies of physics PhD's running sophisticated options evaluation software, trading 10-a-minute on our behalf. Dr. Weatherall doesn't tell us what to do. It isn't his business to do so. He is instead advocating for a kind of financial Manhattan Project to set an appropriate, physics-based replacement for the Consumer Price Index, whose flaws are politically grounded, very much on purpose (Oh, you thought it was objective?). As I said, his PhD's are still shiny and new. His next PhD needs to be in human nature, particularly the nature of the political human.
In the meantime, if you dare to invest in stocks, the advice of Will Rogers is still the best:
A financial market behaves like a small collection of quantum particles. This is my conclusion after decades of investing (sometimes lucky, sometimes not), and reading about them, from The Emergence of Probability and The Taming of Chance by Ian Hacking, to The Black Swan by Nassim Taleb and Beat the Market by Ed Thorp and Sheen Kassouf, and now The Physics of Wall Street: A Brief History of Predicting the Unpredictable by James Owen Weatherall. The shine isn't quite off Dr. Weatherall's first PhD yet—it is but half a decade—but already he exhibits a breadth of vision that sets him apart. He actually has two doctorates, in physics and in philosophy, so he has the kind of mind I like, not just thinking outside the box, but leaving all the boxes behind.
So why would he be interested in market analysis? For the same reasons a ton of physicists have had already: that is where the money is. Plus it has the un-ignorable allure of a challenge that is almost impossible, yet not quite. Given that many thousands of smart people have been trying to "beat the market" for, oh, half a millennium at least, a few have gotten rich, at least by chance, but rare indeed are those persons or funds who managed to stay ahead of the pack and get rich by actually betting on predictions that panned out, again and again. A physicist-run hedge fund called Renaissance is claimed to be one of them.
The bulk of the book is a history of statistical thought, as it developed over the past few hundred years, frequently in response to the desire to understand price fluctuations in markets for currency, commodities, or stocks and options of various kinds. The tools used for this began with the Normal (AKA Gaussian) distribution, the familiar Bell Curve. All kinds of additive phenomena obey Gaussian statistics, such as average height for men or women of a given ethnicity, or most famously, IQ. A particular Normally distributed population is completely described by a Mean (µ) and a Standard Deviation (σ). The shape is scalable, wider for large σ and narrow for small σ, but is otherwise fixed, so that 68% of the population is found within the range µ-σ to µ+σ, called the 1-sigma range; and the 2-sigma range encompasses 95% of the population. So, for IQ, at least among Euro-Americans, µ is standardized at 100 and σ at 15. Thus the range [70-130] includes 95% of these folks, and 68% are found in [85-115]. Public education was originally aimed at the 1-sigma group, and the rest were left to fend for themselves, until Special Education and Gifted Education movements arose to help out those in the "tails", whether duller or brighter.
Is the Normal distribution a good model of market fluctuations? Not at all. First, we must realize that human perception is involved. A $1 change in a $10 stock feels just as large as a $5 change in a $50 stock, particularly if you have 500 shares of the first one or 100 shares of the other. Both changes are 10% of your $5,000 investment. The chart below shows the day-to-day change of closing price for Coca-Cola common stock, since the beginning of 1986, expressed as a % of the prior day's closing price.
If we sort these numbers and plot them against a "Probability Ordinate", really an inverse Normal ordinate (I use the NORM.S.INV function in Excel 2010), we would get a scatter plot that closely follows a straight line if the distribution were Normal. But here is what we get instead:
If we extend a line tangent to the central part of the distribution, to -4σ or +4σ, it strikes at a 5% change, indicating that variations greater than this ought to be rare indeed (there are 7,101 daily changes plotted here). But what do we see instead? Going back to the original data sheet, I find 42 days on which the stock increased by 5% or more, up to nearly +20%, and 33 days on which it fell 5% or more, to nearly -25%. How'd you like to own a million shares of this stock and have it lose 1/4 of its value on a single day? So early on, the Normal distribution was found wanting.
Normal analysis was based on the concept of a random walk, also called the drunkard's walk. Its additive nature will always result in a distribution of final locations, say after ten staggers, that is Normal. So a different distribution with extra-wide excursions is needed. In an entertaining section, Dr. Weatherall describes a drunken firing squad. They have a target upon a very long wall, but being too drunk to point well, might shoot in any direction at all. Give them lots of ammunition (and hide somewhere until they run out), and the pattern of bullet holes will follow a Cauchy distribution. It looks a little like the Normal distribution, but has a pointier top, and most importantly, "fat tails"; that is, many points that are farther—or much farther—from the middle than a Normal distribution would predict. The distribution above is also fat-tailed, having lots of numbers outside the range we'd expect from a Normal distribution. To test a distribution for Cauchy behavior, plot it against a Tangent function evenly distributed in the range -π/2 to +π/2. For my 7,101 points, the Tangent function ranges from nearly -5,000 to +5,000, so the chart is thus:
The Cauchy distribution is clearly a bit too much, its tails are "too fat", compared to the tails of Coca-Cola daily price fluctuations. This kind of conundrum was tackled by many bright people, from Fischer Black to Benoit Mandelbrot. Mandelbrot probably came closest with fractal analysis, which wasn't wedded to integer exponents. But I got another thought as I read along.
The Normal and Cauchy distributions are related, being examples of Stable distributions. In one formulation, a parameter called α has a value of 2.0 for a Normal distribution, and a value of 1.0 for a Cauchy distribution. The fattest tails possible are at α=0, the Uniform distribution of infinite width. Mandelbrot had used fractal analysis to calculate a distribution with α of 1.7, closer to Normal but still with a fat tail. I realized that the most familiar distribution with at least one long tail is Lognormal, but it is confined to positive only values. Does it have a complex square root, perhaps? I sorted the squares of the KO daily changes and charted them against a Normal ordinate on a logarithmic scale. But there was a problem. On 245 days there was no change in price. Prior to the 1970s stocks were valued in 8ths of a dollar (12.5¢), and in pennies thereafter, though dividend allocations can be calculated to 0.0001¢ increments. Trades are reported to the nearest cent. Anyway, you can't take the logarithm of zero, so in my spreadsheet I used a value a little smaller than the smallest calculated nonzero value for those 245. They form the line at bottom left on this chart:
If trades were made with a continuous range of values, not limited by the minimum value, I would expect the left portion of the chart to be as linear as the rightmost. Quantization errors have artificially depressed the daily motion for about 15% of the trades. In the other charts, the two extreme values, -25% and +20%, seemed like outliers. Here, as the two rightmost data points, they are seen to be at most slightly larger than one might expect.
So, all you quants out there, working out the best formula for calculating risk. Give a little attention to the square root of the Lognormal distribution! Now, back to the book.
A key theme of the book is both the value and the danger of numerical models. A physicist understands that a model is always simplified, and cannot be appropriately used outside its range of application. When the people using models of financial systems, to set option prices and other instruments, are physicists, they will know this and avoid over-extending the model. People without physics education will not. When you have a black box program that seems to work magic, it is easy to use it everywhere (the parable of the man whose only tool was a hammer comes to mind).
There have been several major crashes in the past century, and only the one in 1929 was free of the influence of sophisticated statistical modeling tools. I say "sophisticated" because there were statistical tools in use a century earlier, but they were back-of-the envelope estimates at best. All of the more recent ones show at least traces of "broken model" influence, but the October 1987 crash was an overt "robo-trading" crash. This brings up another principle that physicists, at least, ought to keep in mind: the observer effect.
I am not just talking about Heisenberg Uncertainty. Rather, most observations of physical phenomena disturb the system being measured. I remember my father telling me not to check the air pressure in my bike tires so often, because each measurement caused some air to be lost. Later, working in electronics (in a time when the components were visible and manipulable by hand) I learned how to use a Wheatstone Bridge to measure DC voltage the most accurately, because it uses a counter-voltage to keep from bleeding extra current from the circuit. It is only good for very steady DC, of course. Thermometers change the temperature of the pot roast, but only a tiny bit; still the effect is not zero. But now imagine that you have half a million people whose livelihood depends on knowing the temperature in your pot roast, and they all insist on using their own thermometer. There won't be much left of the roast! THAT's what happened in October 1987.
The use of new tools changes the way markets work. What worked in September 1987 doesn't work today; what worked in 1997 or 2007 doesn't work now, and so forth. This pretty much negates the notion of an efficient market. It can only be efficient under two conditions:
- The traders have no supercomputers available.
- All traders are coldly rational.
The "efficient market" works like this: In comparatively quiet times, the asking price of a stock or whatever incorporates all the current knowledge about things that might affect its value in the future. To profit from trading that instrument, you either guess it might be underpriced, because of unknown or little-known information, or you try to learn something nobody else knows. The most common source of such knowledge is cadging or coercing it out of an insider, which happens a lot even though it is illegal. Quantitative analysis attempts to find patterns in price fluctuations that signal a change you can profit from. When someone finds a useful pattern, he or his company will profit from it for a while, until others catch on, then pretty soon everyone can do it, and the market is "efficient" again. So quants' work is a continual arms race. Thus, the tools used to test the market change the market.
But the markets are not that efficient. In the medium term they might be, but the momentary trading picture is much more emotional, and tiny bits of information or rumor disguised as information can sway a trader's estimate of value. If that trader is influential, and others see him (usually male) make a move they didn't contemplate before, some will follow. It can cascade into a large market move, that might last a matter of an hour or less, but might last a day or more, and then there is the potential for quite a swing, either towards a bubble or a crash.
The fragility of any market lies in the tendency for all the quantitative trading firms to use the same models, or models based on the same math, with the same or very similar trigger points. Certain rules instituted after 1987 can calm the flurry to some extent, but the events of 2007 to early 2009 present a case in which the agony was simply drawn out over the space of more than a year, rather than taking place in a month or less.
With the contents of this book under my belt, I ask myself, "What is the ordinary investor to do?" We don't have supercomputers and armies of physics PhD's running sophisticated options evaluation software, trading 10-a-minute on our behalf. Dr. Weatherall doesn't tell us what to do. It isn't his business to do so. He is instead advocating for a kind of financial Manhattan Project to set an appropriate, physics-based replacement for the Consumer Price Index, whose flaws are politically grounded, very much on purpose (Oh, you thought it was objective?). As I said, his PhD's are still shiny and new. His next PhD needs to be in human nature, particularly the nature of the political human.
In the meantime, if you dare to invest in stocks, the advice of Will Rogers is still the best:
- Buy a stock.
- When it goes up, sell it.
- If it isn't going to go up, don't buy it.
Sunday, March 02, 2014
Going beyond the gene
kw: book reviews, nonfiction, genetics, epigenetics, twin studies
You meet a pair of identical twins. They seem really, really identical: they may be dressed the same, they have the same mannerisms, hair style, voice and speech pattern, and so forth. Give it time. Over many experiences you'll learn to tell them apart. One has a mole on the cheek the other doesn't have, or they like different bands or kinds of music or subjects at school. The time will come that you wonder why you ever confused them.
They are a lot alike, truly. But why aren't they more identical? Identical twins arise when a single egg, or an embryo at a very early stage, splits in half. The two individuals that are born are thus a clone (did you know "clone" is a collective noun?)—their genetics are identical. Now, in their lifetimes, each will pick up random mutations, and as many as 100 of these will affect their sex cells and be passed on to their offspring. That's two or three heritable mutations per year of life. Such a mutation may explain the mole on one cheek. But some differences that occur cannot be so simply explained. Are they purely due to random environmental differences (this is the nurture hypothesis)?
Tim Spector researches twins. In his book Identically Different: Why We Can Change Our Genes, he discusses many interesting cases that show just how different the members of a twin clone can be, in spite of having "exactly the same genes." I have a quibble about the book's subtitle. In one way, it is quite accurate, but in another it is very misleading. That is, this is a book about epigenetics, and the mechanisms of epigenetics do make changes to genes, and such changes are often heritable. However, the ACGT sequences are not changed, and epigenetic changes are reversible. A more accurate (but clumsier) subtitle would be Why We Can Change Which of Our Genes are Used.
In all the hoopla surrounding the completion of the Human Genome Project a decade ago, it was not mentioned that we knew nothing about why cells differentiated into tissues. Every cell in your body has the same DNA, the same genes. What makes the cells in the cornea of the eye transparent and silent, while muscle cells are dark brownish red and can quickly change shape? The difference is in which portions of DNA, that is, which genes (and, it is quite certain, which sections of "noncoding DNA" that some folks used to call "junk DNA") are actually used and which are not, by that cell. Tissues differentiate by epigenetically silencing many genes.
Two mechanisms make temporary modifications to DNA to prevent or allow certain genes to operate: methylation and histone modification. Look them up for more detail. In short, methylation can happen quickly, as can demethylation. In methylation a number of -CH3 groups are attached to a section of DNA, which jams the DNA-to-RNA copying mechanism so that gene is not expressed (i.e., not used). In demethylation, they are removed and the gene is active again. In histone modification, larger chemical groups get attached to the "spool" proteins that help DNA wind up so it will fit in the cell nucleus. Some modifications slow down the unspooling needed so the gene can be used, or stop it entirely. This is a slower process. But both kinds of modifications can last a long time, and both kinds can affect sex cells and be passed on to offspring. That means that some changes in DNA expression that occur in your lifetime can be inherited by your children.
This means that Larmarck was partly right! He is widely lampooned for having a "wrong" hypothesis for evolution. Yet we find he was not entirely wrong. In one "natural experiment" described in the book, some of the Dutch people were systematically starved during World War II, during a period of a few months. During that time and for some months afterward, babies were born. Babies of starving mothers were small and grew up smaller than ordinary. What is of interest here is, their own children were also smaller. The effect could be traced several generations in some families. Epigenetic changes that occurred during development in the womb affected the size of children and grandchildren, taking 3-4 generations to be reversed.
In areas that two twins are quite ordinary, they are likely to be very similar. In other areas, not so. For example, a number of twins are discussed in which one member is homosexual and the other is not. In some of these cases, the non-gay twin had tried out homo-sex and didn't like it, and remained heterosexual thereafter. The author considers that the tendency to be attracted to one's own sex is under genetic influence, but that a minor epigenetic difference can push a person one way or the other, in how they are going to live as a result. It is well known that in a prison situation about half of male inmates practice homo-sex (not always so willingly), but there is the other half that do not, and apparently put up sufficient resistance when pressed to avoid it altogether. Most of the half who do so practice, never do "on the outside". They prefer female partners, and resort to male partners only when that's all that is available.
Among "identical" twins, then, there is surprising variability. Sometimes they have different eye colors, more frequently they pursue different careers and may seek out different kinds of marriage partners. One may be musical and the other not. In a late chapter, the author discusses an apparent tendency in nature to maximize diversity. It is a kind of hedge against an uncertain future. This is seen in inbred laboratory mice. They have been inbred on purpose, so as to be a near-clone with many members. Yet in any litter, their personalities differ: one is the boldest, another the shyest, and others more or less active or greedy when eating. Natural epigenetic changes seem to assure that as conditions change, one or a few will be better able to take advantage, or to survive.
Now, can we really, on purpose, make epigenetic changes that will affect our children? One way is to make a radical change in your habits of eating and exercise. If you boost your exercise a great deal, your kids may not be a great deal more muscular, but they might find it easier to "get muscles". For more subtle matters, drug companies are very interested in compounds that will promote either methylation or demethylation of target genes that affect the outcome of a disease or birth defect. It may even be possible to modify our alimentary flora (the bacteria that live inside us, some 10 to 100 trillion of them). The internal flora of a slender person differ from those of an obese person. One doctor is experimenting with "fecal transplants" (an enema of the germs from someone else's bowel), which have cured some people of irritable bowel syndrome, and may be able to change how efficiently food is used. That could make a dent in someone's obesity, if their digestion became much less efficient! It is hard to say at this point, though, if this is an epigenetic change, or if having a different internal population just makes things work differently without epigenetics being involved.
The book is a portent of future things. There may one day be ways to tinker with our epigenetics to make us healthier and happier. Just how much of this we want to take advantage of, is anyone's guess. What seems strange or even threatening to this generation will be commonplace to the next.
You meet a pair of identical twins. They seem really, really identical: they may be dressed the same, they have the same mannerisms, hair style, voice and speech pattern, and so forth. Give it time. Over many experiences you'll learn to tell them apart. One has a mole on the cheek the other doesn't have, or they like different bands or kinds of music or subjects at school. The time will come that you wonder why you ever confused them.
They are a lot alike, truly. But why aren't they more identical? Identical twins arise when a single egg, or an embryo at a very early stage, splits in half. The two individuals that are born are thus a clone (did you know "clone" is a collective noun?)—their genetics are identical. Now, in their lifetimes, each will pick up random mutations, and as many as 100 of these will affect their sex cells and be passed on to their offspring. That's two or three heritable mutations per year of life. Such a mutation may explain the mole on one cheek. But some differences that occur cannot be so simply explained. Are they purely due to random environmental differences (this is the nurture hypothesis)?
Tim Spector researches twins. In his book Identically Different: Why We Can Change Our Genes, he discusses many interesting cases that show just how different the members of a twin clone can be, in spite of having "exactly the same genes." I have a quibble about the book's subtitle. In one way, it is quite accurate, but in another it is very misleading. That is, this is a book about epigenetics, and the mechanisms of epigenetics do make changes to genes, and such changes are often heritable. However, the ACGT sequences are not changed, and epigenetic changes are reversible. A more accurate (but clumsier) subtitle would be Why We Can Change Which of Our Genes are Used.
In all the hoopla surrounding the completion of the Human Genome Project a decade ago, it was not mentioned that we knew nothing about why cells differentiated into tissues. Every cell in your body has the same DNA, the same genes. What makes the cells in the cornea of the eye transparent and silent, while muscle cells are dark brownish red and can quickly change shape? The difference is in which portions of DNA, that is, which genes (and, it is quite certain, which sections of "noncoding DNA" that some folks used to call "junk DNA") are actually used and which are not, by that cell. Tissues differentiate by epigenetically silencing many genes.
Two mechanisms make temporary modifications to DNA to prevent or allow certain genes to operate: methylation and histone modification. Look them up for more detail. In short, methylation can happen quickly, as can demethylation. In methylation a number of -CH3 groups are attached to a section of DNA, which jams the DNA-to-RNA copying mechanism so that gene is not expressed (i.e., not used). In demethylation, they are removed and the gene is active again. In histone modification, larger chemical groups get attached to the "spool" proteins that help DNA wind up so it will fit in the cell nucleus. Some modifications slow down the unspooling needed so the gene can be used, or stop it entirely. This is a slower process. But both kinds of modifications can last a long time, and both kinds can affect sex cells and be passed on to offspring. That means that some changes in DNA expression that occur in your lifetime can be inherited by your children.
This means that Larmarck was partly right! He is widely lampooned for having a "wrong" hypothesis for evolution. Yet we find he was not entirely wrong. In one "natural experiment" described in the book, some of the Dutch people were systematically starved during World War II, during a period of a few months. During that time and for some months afterward, babies were born. Babies of starving mothers were small and grew up smaller than ordinary. What is of interest here is, their own children were also smaller. The effect could be traced several generations in some families. Epigenetic changes that occurred during development in the womb affected the size of children and grandchildren, taking 3-4 generations to be reversed.
In areas that two twins are quite ordinary, they are likely to be very similar. In other areas, not so. For example, a number of twins are discussed in which one member is homosexual and the other is not. In some of these cases, the non-gay twin had tried out homo-sex and didn't like it, and remained heterosexual thereafter. The author considers that the tendency to be attracted to one's own sex is under genetic influence, but that a minor epigenetic difference can push a person one way or the other, in how they are going to live as a result. It is well known that in a prison situation about half of male inmates practice homo-sex (not always so willingly), but there is the other half that do not, and apparently put up sufficient resistance when pressed to avoid it altogether. Most of the half who do so practice, never do "on the outside". They prefer female partners, and resort to male partners only when that's all that is available.
Among "identical" twins, then, there is surprising variability. Sometimes they have different eye colors, more frequently they pursue different careers and may seek out different kinds of marriage partners. One may be musical and the other not. In a late chapter, the author discusses an apparent tendency in nature to maximize diversity. It is a kind of hedge against an uncertain future. This is seen in inbred laboratory mice. They have been inbred on purpose, so as to be a near-clone with many members. Yet in any litter, their personalities differ: one is the boldest, another the shyest, and others more or less active or greedy when eating. Natural epigenetic changes seem to assure that as conditions change, one or a few will be better able to take advantage, or to survive.
Now, can we really, on purpose, make epigenetic changes that will affect our children? One way is to make a radical change in your habits of eating and exercise. If you boost your exercise a great deal, your kids may not be a great deal more muscular, but they might find it easier to "get muscles". For more subtle matters, drug companies are very interested in compounds that will promote either methylation or demethylation of target genes that affect the outcome of a disease or birth defect. It may even be possible to modify our alimentary flora (the bacteria that live inside us, some 10 to 100 trillion of them). The internal flora of a slender person differ from those of an obese person. One doctor is experimenting with "fecal transplants" (an enema of the germs from someone else's bowel), which have cured some people of irritable bowel syndrome, and may be able to change how efficiently food is used. That could make a dent in someone's obesity, if their digestion became much less efficient! It is hard to say at this point, though, if this is an epigenetic change, or if having a different internal population just makes things work differently without epigenetics being involved.
The book is a portent of future things. There may one day be ways to tinker with our epigenetics to make us healthier and happier. Just how much of this we want to take advantage of, is anyone's guess. What seems strange or even threatening to this generation will be commonplace to the next.