AutoArt folder distribution

 kw: analytical projects, art generation, ai art, statistics, statistical distributions, lognormal, scale free

I began using art generating software in November 2022, when DALL-E2 became available. Since then, I've enjoyed having a series of art generating "engines" available, including numerous engines (called "models") in the aggregators Leonardo AI and OpenArt. As often as I can, I generate images for this blog; in some cases, I download images I find on the Internet. However, my primary artistic pastime is creating images of things and scenes I imagine.

Just in the past few days I was inspired by a heavy snowfall to find short poems about snow, and use them to create wintry images. This image was drawn by Nano Banana Pro, under the Leonardo AI umbrella with "None" as the style; that is, native NB Pro. The aspect ratio was set to 16:9. It displays the entire poem, something NB Pro can do better than any other art engine I have found. The prompt was "Watercolor painting evoked by a poem:", followed by the text of the poem "The First Snow" by Charlotte Zolotow.

The image is particularly evocative in shifting to an exterior view as the window dissolves. I suspect there are a number of images that use this device in the training material for NB Pro.

When I made signed versions of this and several others that were generated in the same session, to be included in a folder for a "screen saver" slide show, I began thinking about the various numbers of different image types I've created in the past three-plus years. Last year I went through my (poorly organized) folder stack of "AutoArt" and reorganized it into 35 categories, each in its own folder. To date, there are 1,472 signed images in 35 folders containing between two and 405 images. My inner statistician began to stir…

The image below shows two analyses of the statistical distribution of the numbers of files in these folders.

Charts like these make it quite evident which statistical treatment is appropriate to a particular set of data. I'll explain what these charts mean and how they were created.

"Scale Free" is a type of power law distribution related to the Pareto distribution. It is easy to analyze, which makes it popular. To analyze a series of numbers graphically in Microsoft Excel:

• Enter the numbers in column B, starting in cell B2.
• Put an appropriate header in cell B1
• Highlight these data (B1:B36 in this case)
• Sort from largest to smallest, using the Sort & Filter section under Editing in the Ribbon.
• Enter 1 in cell A2 and 2 in A3.
• Put a header in cell A1; I usually put "N".
• Highlight cells A2 and A3.
• Double-click the fill handle at the lower right of A3. This will fill the rest of the column with numbers in order, as far as the data goes in column B. In this case, we get numbers from 1 to 35.
• Highlight these two columns to the end of data. In this case, from A1 to B36.
• In the Ribbon, use Insert and in the Charts section, select the icon showing scattered dots with axes; this is X-Y Chart.
• The title of the chart is whatever the header text is in B1. Edit as you wish.
• Double-click one of the axes to open the Format dialog.
• Click Logarithmic Scale near the bottom of the menu.
• Click the other axis and also click Logarithmic Scale. This is now a log-log chart.

The result will be similar to the upper chart. Now for the lognormal analysis, beginning with these two columns of numbers:

• Insert a new column between A and B; this is the new column B.
• In cell B1 enter a header such as "Prob.". You are going to create a probability axis.
• In cell B2 enter this formula (where the largest number in column A is 35):

=NORM.S.INV((A2-0.5)/35)

• Double-click the fill handle at the lower right of A2 to fill the column with the formula.
• Highlight the data in B and C (B1 to C36 in this case).
• Use Insert as before to create an X-Y Chart.
• Edit the chart title.
• Note that the vertical axis is now centered above the zero. 
• Assuming the Format dialog is still open, click the horizontal axis.
• In the middle of the menu in the section "Vertical Axis Crosses", click the bubble at "Axis Value".
• Enter "-3".
• Click the vertical axis and click Logarithmic Scale. This is now a log-probability chart.
• If you want the markers to be a different color, click one of them. The Format Data Series menu appears at the right.
• Select the icon of a paint bucket pouring paint.
• Click the Marker tab
• For both Fill and Border, select the color you want.

This will be similar to the lower chart. For the data I used, the chart shows the points scattered approximately along a straight line. By contrast, in the upper chart there is a definite downward bend. In a log-log chart such a shape is diagnostic that the distribution is not scale free, but is more likely to be lognormal, or even normal (Gaussian). In this case, the second chart shows that lognormal is a good model of the data distribution.

This is an illustration of the Theory of Breakage, formally described by A.N. Kolmogoroff in 1941. When an area is divided (US state or county areas are good examples), the distribution is lognormal. When a sheet of glass is broken, the weights of the pieces also have a lognormal distribution (I've done this experiment). Some recent publications claim that a theory of breakage produces a power law distribution, but this is false. Certain phenomena in nature tend to be normally distributed. The classic example is the height of adult men, or of women (but not both) in a population, such as the residents of a particular town or county. However, most phenomena produce groups of measurements that are lognormally distributed, in which the logarithm of the quantity being measured is distributed as a normal, or Gaussian, curve.

I could go further into this, but this is enough for the purpose of this post.

Studying tartan designs

 kw: analytical projects, plaids, tartans, statistical distributions, scale free, lognormal

Guess what this is? It isn't quite what it looks like. It's a printed plaid, a plaid-like pattern printed on white flannel, the backing for a comforter we made many years ago. Until I looked at it closely (microscopically), I thought it was a woven plaid.

Close inspection also reveals that the weave is single-over-under, rather than the over-2-under-2 of most plaid fabrics. Nonetheless, it is an attractive pattern, one of my favorites!

Some time ago I began to wonder about the distribution of stripe widths on plaids. Long ago I wrote, in GWBASIC, a "screen saver" program that produced plaid patterns on the screen. I used a scale free distribution because it is easy to program. It would generate a bunch of width values and then scramble them by sorting against a set of random numbers; it would assign colors and generate a plaid pattern.

I don't know how plaids are designed. The Scottish tartans such as Black Watch or Douglas can be centuries old, and were selected with aesthetics in mind, and an eye for being imposing because they were worn into battle. Today I suppose artistic designers pick the colors and stripe widths in a purely aesthetic way.

I decided to study the statistical distributions found in my own shirts and other fabrics. I figured out how to wrap a shirt around a dictionary to hold it on a scanner, and did so for 17 flannel shirts and two plaid jackets, plus the pattern above which I photographed because the comforter is large and very thick. I have a number of plaid summer shirts, which I may analyze in the future, but they are not included here.

The large variation in stripe widths led me to consider three model distributions: Normal, Lognormal and Scale Free or Log-Log. When graphed with appropriate coordinates, each of these is a straight line, but, for example, a Normal distribution will graph as a curved line on either Log-Log or Lognormal coordinates. First, we need to see the shapes of these distributions:

The Normal distribution is frequently called the Gaussian distribution, because it was first proposed by the mathematician Carl F. Gauss in the early 1800's. When several random variables are added and measured repeatedly, the distribution of the sum tends toward the center-weighted shape shown in orange. A mathematical proof of this additive tendency is called the Central Limit Theorem.

The Lognormal distribution results when an exponential function is taken for a set of values that have a Normal distribution. The Lognormal shape is shown in green. Also, when several random variables are multiplied and measured repeatedly, the distribution of the sum tends toward a Lognormal distribution. The logarithmic form of the Central Limit Theorem describes this tendency. Furthermore, when an area or extended object is fractured or divided into many pieces via a random process (such as dropping a pane of glass), the areas or weights of the pieces closely approximate a Lognormal distribution. I verified this once in the laboratory using a small piece of glass I broke with a light blow of a hammer, and then weighed a couple hundred pieces. The mathematical proof of this is called the Theory of Breakage, which was propounded by A.N. Kolmogoroff in 1941.

The Scale Free distribution results when a series of measurements are taken of the reciprocals of a uniform random distribution. This is also called a Fractal distribution, based on the work of Benoit Mandelbrot in the 1980's. A theoretical continuous Scale Free distribution has no limit in either direction; no largest or smallest member being predicted. Discrete sets of values that have a Scale Free distribution, however, do have a largest and smallest member. While the theoretical, continuous Normal and Lognormal distributions also have no limits, the probabilities of extreme values are vanishingly small (for a Lognormal distribution, "extreme" means either a very large positive value, or a value that is positive, but very, very close to zero).

Each distribution can be rectified (made to approximate a straight line) by sorting all the values and graphing them in order in an appropriate coordinate system. Idealized examples of these three distributions are all shown together in the three coordinate systems that are relevant to this discussion:

These charts each rectify one of the distributions. Firstly, for "Probability Coordinates", the horizontal axis has units of standard deviation and the vertical axis is linear. The sorted values in a Normal distribution (orange) follow a straight line here. Secondly, for "Log-Probability Coordinates", the horizontal axis is the same, while the vertical axis is the logarithm of the values, which straightens out the Lognormal distribution (green). Thirdly, for "Log-Log Coordinates", the horizontal axis is the logarithm of the ordinal number of the sorted values and the vertical axis is the logarithm of the values. This rectifies the Scale Free distribution (blue). Note that in each case, the "other two" distributions display a distinct curvature.

Now, for sets of more realistic distributions, created by appropriate random processes, we see the same three graphs:

The three coordinate systems are the same as those above. A straight line has been added to each graph to emphasize which set of values has been rectified.

How does all this apply to a study of plaids? I gathered data from the scans of the 20 plaids, measuring each one in both directions. This is because the warp and woof of the weave have different pitches, so the plaid designers adjust the number of threads of each color so the resulting plaid will not look distorted. Here is an example of a set of data for one of the plaids. I used rather generic color names, because the widths of the stripes were the meaningful parameter, not the color pattern.

Note that, while the order of the colors is the same in both directions, the number of threads is seldom the same in direction 2 as compared to direction 1. This enlargement of the pattern shows the threads; it takes a careful look to see that the spacing is different between horizontal and vertical. Look at the white square. It has 9 horizontal threads but 6 vertical threads, yet the "square" appears pretty close to a square.

One benefit of the over-2-under-2 weave is that it makes counting threads in wider bands easier, because I could count by 4.

This is a more overall view of the pattern. Although each "unit" of the pattern contains 5 white stripes, 4 black stripes, 2 navy stripes and only 1 gray stripe, gray dominates because its stripe is so wide, with navy blue running a close second.

What did I do with all these numbers? There are a lot of them. A few patterns had 38-40 stripes, and many had quantities in the 20's. Some plaids have mirror symmetry, a smaller number don't.

I copied all the data, sorted each set (each direction for each plaid), and set up both ordinal and probability axes for them all. I charted them in groups to see how they looked. I was looking for rectified distributions. As we see below, with a few of them as an example, the results are not clear-cut. I had been hoping to see a clear indication that the distributions were primarily either Scale Free or (my preference) Lognormal. The reality is a little of both. The graphs that follow pertain to six non-symmetrical patterns.

The overall view is that many of the lines have a downward curvature at the right, but not all. In particular, the yellow line and the gray line mostly hidden behind it (#16), and the lighter blue and lighter green lines in the midst of the scrum (#10), don't curve down.

The downward curvature indicates that most of these are better modeled as Lognormal. The next graph shows that presentation.

Here many of the lines appear straighter, while some either flatten out or curve oppositely (not really "upward"). We also see that the dark red line and the dark blue that accompanies it also flatten out, even though they have a bit of downward curvature in the other graph.

None of the patterns showed a hint of being closer to Normal than to Lognormal or Scale Free, so I didn't pursue that any further.

"Eyeballing" the charts proved unsatisfactory, so I used a mathematical measure of linearity, relevant to either Log-Log or Lognormal coordinates, to more clearly discern the trends.

I saw from this that some of the patterns were more Lognormal in one direction and more Scale Free in the other. I found the following:

• 7 patterns were Lognormal in both directions.
• 4 patterns were mixed, but leaned Lognormal more than Scale Free.
• 2 patterns were mixed, but leaned Scale Free.
• 7 patterns were Log-Log in both directions.

Here we have, from left to right, #3, which is the most Lognormal of them all, #8, which is the most ambiguous, and #6, which is the most Scale Free of them all.

As it happens, #3 and #8 are favorites of mine, and if the red plaid from our comforter were made into a shirt, as a pattern, it would also be a favorite (although my wife doesn't like me to wear red shirts); it is also a mixed-distribution pattern. I care less for #6; I consider it almost ugly. Just to show that Scale Free patterns are also attractive, another of my favorites is shown here, #10, which is more Scale Free in both directions:

A characteristic of Scale Free distributions is a greater number of narrower stripes, and this one shows that. It illustrates that what we like doesn't have a very strong mathematical basis. I had been thinking just the opposite, but I don't mind being proven wrong.

In the future I may scan my plaid summer shirts and analyze them, to see if these tendencies hold up. This has been an enlightening exercise.

The usual is less usual than we think

kw: analysis, statistics, black swans, statistical distributions

Alternative Title: Thousand-year Floods Occur Too Often

Just to set expectations ahead of time: The aim of this article is to analyze activity for a stock in the U.S. stock market. However, the phenomena of interest here apply in numerous areas, including flood control, a fun place to begin.

This picture shows the Sorlie Bridge linking Grand Forks, ND with East Grand Forks, MN, during the flood of 1997. It's what happens when you design for a 100-year flood, and it has been 170 years since the last big flood. The peak flow was only 5% greater than the design criterion.

What would a 1,000-year flood look like? Sooner or later it'll happen. Some interesting statistics are involved.

Here we have a log-Pearson plot of gauged "floods" on a river in Australia. AEP is "Annual Exceedance Probability". I was taught how to make such plots during a summer project in graduate school. The data are the highest flow recorded every time the river overflows its inner banks. There are 46 such events charted here. I infer from the rightmost datum that the record goes back about 80 years. A "flood" as thus defined will happen every year or two.

Extending the coordinates of this chart I estimate that a 1,000-year flood would have a peak flow of between 350 and 400 on the scale shown (I infer m³/s; in ft³/s: 12,000-14,000), not quite twice the flow of a 100-year flood.

Quantitative hydrology doesn't extend back even 200 years. How can we validate the use of log-Pearson analysis in extrapolating for 1,000-year floods? There are also at least two other analysis methods in use. How could we validate any of them?

Nearly forty years ago, a fellow student of mine in graduate school worked out such a validation, for at least one "creek" in the Black Hills area. In this view from Google Maps, the creek mouth is off the left. The image width is about a quarter mile.

The boulders seen here are not glacial erratics. The student, Bill, measured lichen patches on hundreds of the boulders, choosing only those larger than he was. He was able to discern, by the size of the largest patch of a certain kind of lichen on each boulder, when they were last tumbled down the creek and washed out onto the plain, sometimes as far as a couple of miles. He determined the outwash area for twenty large floods from that creek. I saw a map he made of those outwash areas.

References on lichenometry helped him obtain approximate dates for the floods. All had occurred in the past 5,000 years. As a student of hydrology, he was able to determine the flow volume needed to move a car-size boulder a half mile to a mile from the creek mouth. Here's the kicker: a chart of historical hydrology measurements on that creek indicate that every one of the twenty floods was at least a 1,000-year flood. But they were actually something like 250-year floods! (or, actually, scattered out from 250 years and upward.) Such data didn't fit into the historical analysis. To be clear about what follows, the log-Pearson analysis is not related to the Normal Distribution, nor to Lognormal statistics.

Statistical analysis in many areas depends heavily on the Normal Distribution, AKA Gaussian. As it happens, this is frequently a very good model. However, even the textbook example, human height, isn't as "normal" as it might seem. In this blog post by John D. Cook, he shows that extremes of height and of shortness are not well predicted by a normal distribution with a standard deviation of 2.5 inches, the criterion that fits well for about 95% of the data for men and for women (which must be analyzed separately). By that measure, there should be no more than one man taller than seven feet, or shorter than 4'-5", nor one woman taller than 6'-9", or shorter than 4'-1", among our entire world population of more than 7 billion. But the NBA is full of seven-footers (more than 40, as counted in 2018), and there are numerous "little people", particularly in entertainment, ranging from four feet down to just above two feet tall. The shortest adult woman on record measured nineteen inches. Data such as these fit better a "fat tailed" distribution, for which more extreme values are more frequent than a Normal analysis would predict.

For events such as floods, it is likely that the excess frequency of extra-large floods is due to a different climate regime from the usual. For height measurements on people, different populations with distributions account for some of the "excess" variation, while medical issues such as glandular gigantism or dwarfism also account for some. These kinds of considerations indicate that we need to study a broader scope of phenomena. If an analysis doesn't include everything relevant, our model is incomplete. But what of "surprises" in a data set that is thought to be well-behaved?

These are examples of black swans. I reviewed The Black Swan by N.N. Taleb thirteen years ago. The book's theme is that "uncommon" events aren't so uncommon. He applied findings such as these to investing. You may recall there had been a significant bear market a few years earlier, starting in 2000. The one that followed the book by a year (2008-09) made Taleb look like a prophet. Now we are in the middle of another one.

One claim in The Black Swan is that daily moves on stock issues follow a Cauchy distribution. It is described as the ratio of the sine of one uniform random variable divided by the cosine of another. Mathematically, that is the tangent of a uniform random variable over the interval [-π/2, π/2], but not including the end points, for which its value is infinite. The Cauchy distribution is typically described as a tangent function over the half circle.

That was something I could test. The Cauchy distribution is very fat-tailed or, viewed another way, it middle part very skinny compared to a normal distribution. It looks like this. We'll see how a normal curve looks in a moment.

Does the distribution of daily closing values of stocks in the market look like this? I chose the most stable stock in America, AT and T, to test this. I used data for the past 37 years (everything available through Yahoo! Finance). I used what they call "Adjusted Close", which factors in the values of stock splits and dividends. I analyzed the entire period, and two sub-periods, the ten years from 1/1/1984 to 12/31/1993 and the two years from 1/1/2018 to 12/31/2019. Note: I'll call this company "ATT" to avoid the ampersand, which is not well behaved in HTML text.

This is the shape of the distribution of daily moves, expressed in percent, and two normal distribution curves. As the legend indicates, the blue curve shows the recent two years, the green curve shows the early ten years (which includes the crash of 1987), and the red curve shows all 37 years. The dashed magenta line is a normal curve with the same standard deviation as the 37-year curve, and the thin black line is a normal curve that fits the bulk of the data, but not the tails. I hope you can see that a normal curve looks broader than the Cauchy curve in the chart just above. The range -10% to +10% doesn't quite encompass all the data; there are three data points further to the right and one further to the left…out of nearly 9,200 daily motions.

It may take some peering, but it is also visible that the black curve outside the range [-2%,+2%] lies below the red, green, and blue curves. Those small amounts are the "fat" in the fat tails of those three distributions. Their very similar shapes also illustrate why I call ATT so stable. Through thick and thin it preserves a certain character of daily responses to market pressures. I do not intend to compare ATT to another stock here; that is for another day, another post.

It will perhaps be easier to see the implications of these data by looking at cumulative distribution functions (CDF's).

The x-axis of this chart is in units of standard deviation. The straight, black dashed line shows how a normal distribution would plot as a straight line. The ATT data going above at the left and below at the right are the fat tails of their distribution.

Looking at the two charts above, one may suspect that the ATT CDF is not as fat-tailed as a Cauchy distribution. The next chart shows this:

The x- and y- axes are in different units because I didn't normalize them. However, the very flat shape of the curve between -2 and +2 standard deviation units shows that the tails are much fatter than those for the ATT data.

Stock market motions, at least for this stock, are thus not as extremely variable as a Cauchy-distributed function.

I must at this point make an aside regarding lognormal distributions. My geologic background as an undergraduate emphasized sedimentology. One tool used to study sediments such as a sandy soil is to shake a sample through a stack of sieves and make a frequency distribution of the result, by weight. The sieves are carefully crafted to separate particles, at each step, that are 1.414 (i.e., √2) smaller in diameter. The frequency distribution so produced is a lognormal analysis. A frequency plot for a well-developed sediment from a single source will look like a normal curve, except that the x-axis is the logarithm of particle diameter. Over the years I found that many natural phenomena that have a wide range are lognormally distributed. We can check for lognormality by charting on log-probability coordinates.

I decided to see if the distribution of stock price daily moves might be related to a lognormal distribution. I squared the ATT data; thus all the results are positive, no longer both up and down. This is a CDF of the result, with the vertical axis logarithmic.

If the data closely followed the black dashed line, they would be lognormally distributed. The data charted at 1E-08 were actually zero, but that cannot be shown on a log-probability chart. What is the big dip leading up to them? This is a consequence of the size of a penny! I call it the "penny effect". During part of the interval stocks were traded only in increments of 1/16 of a dollar, then in increments of certain fractions of a penny, and since then in whole cents.

If the stock is trading for $10, and the next day the closing price is $10.01, the difference between the two is 0.1%, or 0.001 (10.01/10.00 - 1). Squaring this produces 0.000001. When the adjusted closing price is closer to $5, as it was in the early 1980's, a penny difference is 0.002, which squares to 0.000004. This, and a psychological effect that many people avoid prices so close to a prior day's price all led to the drop-off seen here. Since this is a pretty good fit to lognormality, other than the "penny effect", I call the distribution of stock price moves "Square Root of Lognormal."

Let's look at the squared Cauchy data. It is a continuous distribution with no pennies to worry about, so there is no drop-off. But the data are not lognormal either. The black dashed line is lognormal. The tails of this distribution are fatter than the tails of a lognormal distribution, much fatter. From experience I can tell you, that's pretty hard to do!

It so happens that this distribution is ill-behaved no matter how you transform it. In terms only a statistician could love, the Cauchy distribution has no "moments". That means that trying to measure anything other than its average value is meaningless. You can calculate a standard deviation for it, for example, but it has no meaning.

How does this relate to black swans? In stock market terms, if the daily moves were truly distributed according to Cauchy statistics, huge moves would be even more common than they are. However, the Square Root of Lognormal function that daily prices do seem to follow is not nearly so extreme, but it does have many more large moves, up or down, than a normal model would lead someone to believe. The standard deviation of the ATT data is about 1.5%. That means that, as seen in the chart above titled "Daily Change, all", none of the daily price moves should have been outside the bounds of [-4%, +4%]. However, on 250 occasions, a daily move was outside this range. That's only 2.7% of all the data. However, 250 times in 37 years means that six or seven times yearly, ATT's stock price moved by more than a normal model would predict is even possible during a run of nearly 9,200 market days.

Conclusion? To better predict the range of variation for a particular stock, list the daily moves over a period of a few years, square them, and plot as a CDF. From this you can determine a log-standard deviation, and its square root will yield a much better parameter of variation for the stock's behavior.

This last chart is a frequency diagram of the squared data for ATT, and the horizontal axis is logarithmic. The blip at the left is the "penny effect". Other than that, and a bit of skew, it looks a lot like a normal distribution; the choice of axis shows its affinity to a lognormal distribution.

Whether this will lead to a more robust way to set an investment strategy is anyone's guess!

More on Theory of Breakage

kw: analysis, lognormal, power law, statistical distributions

I wrote about the statistical distribution of stellar masses nearly four years ago. The Theory of Breakage, as it is classically derived, indicates that random processes which divide an object or area or substance into many parts will tend to produce "pieces" that have a lognormal distribution of the "size" parameter, whether length or volume or mass. Some scholars, following Benoit Mandelbrot, instead posit that random breakage will produce a distribution of "size" that obeys a power law. Here I apply it to a large collection of computer files.

As I close out a 44-year career in the computer sciences, I need to determine which computer files to pass on to my colleagues, and which can be allowed to vanish. I have had some kind of personal computer on my desk for 32 years, and I have certain files from my mainframe days going back another 12 years. As a long-time member of the Elephant Club (Motto: Do not trust a computer you can see over), I share the computer geek's vice of never throwing anything away. I long ago learned that magnetic disk technology was producing file space much faster than anyone could fill it. So I am a kind of hoarder. My house is not cluttered, but my disk drive certainly is!

I used a DOS command (how few folks even know how to do so!) to gather a complete file and folder listing for the Work disk. The entire corpus came to 35,309 files with an aggregate size of 18 Gbytes. Of this, the Project data consists of 8,297 files that total 8 Gbytes. This Project data are those files directly related to paid work. The rest is support materials and other files kept for historical reasons, such as a great many FORTRAN, Pascal and Perl program source code files that I call my "algorithm collection", presentation files in PowerPoint and older formats such as Framework (by Lotus), spreadsheets (Excel, 1-2-3 and Quattro), flow charts and other drawings in Visio and older tools, plus images, videos and sound files.

With a little bit of fiddling around, I produced two lists of file sizes, and analyzed them two ways. This first chart is a Power Law analysis:

Keep in mind that the "Proj. Files" are included in "All Files". The strongly curved shape of these distributions is diagnostic that they do not follow a Power Law, but are more likely to follow a Lognormal distribution. If you simply project the slope of the upper quarter or third, to the "1" line, you see that it would require trillions of files to extend the line (I estimate 20 trillion for the blue line). By contrast, here is a Lognormal analysis:

In this presentation, it is evident that the distribution is very nearly lognormal. I have no explanation for the few departures from linearity, and they don't really need "explaining", anyway. I simply find it fascinating that these thousands of items, some written, others downloaded or generated by many methods and processes over four decades, should result in such a distribution.

Real Estate market distribution analysis

kw: real estate, analysis, statistical distributions, market distributions

Real estate values at "best places to live" kinds of web sites are typically stated only as median values. It is of great value to know the distribution of the real estate market in an area, particularly to determine if the market is distorted by excess or insufficient valuation in a portion of the market range. While I sometimes perform a detailed analysis for a small market, we'll also look at a powerful tool for quickly analyzing any size market.

1. Detailed Analysis

Keep in mind what any Realtor will tell you: "Location, Location, Location," but also note that every location has a context. This analysis is appropriate in a small market, such as a single Zip Code or small city, that contains no more than 150 homes for sale.

I pre-draw a grid such as that shown here. Number of Bedrooms runs vertically, and number of Bathrooms runs horizontally. At Realtor.com a bath-and-a-quarter or bath-and-powder room is counted as 2. When you actually look at individual homes, you'll pay attention to quarter and half baths, but for this analysis, a room is a room is a room.

These figures are for Zip 74075, the northern half of Stillwater, OK, an area in which I lived for a number of years so I am familiar with the market. The numbers are in thousands, rounded ($198,500 to $199,499 become 199). You can see that 3BR 2Ba is by far the most popular, and I ran out of room and stole some space from the 3+3 category. This is an economical market, with a total range in value of single-family homes from $46,000 to $499,000. I gathered the data October 25, 2011.

This Zip Code contained 125 single-family homes for sale. Just the 3+2 category contained 76, or 60% of the entire market. That indicates a homogeneous market. These two data sets are charted along with analyses of two markets closer to home, Ridley Park, PA (54 homes) and the Zip Code 19803 in northern Delaware, where I have friends.

I used Minitab to analyze these on Lognormal coordinates. The straight line for 19803 indicates it is a very homogeneous market, a set of similar bedroom communities. The two plot-lines for 74075 have a zig and a zag at the ends, indicating the market is not as homogeneous; there is at least one "ritzy" area that breaks the trend, and I suspect the low-end zag represents trailer houses on land on the edge of town.

Ridley Park shows up as quite homogeneous, but there is a wiggle in mid-line that indicates a depressed market for the middle-of-the-road homes there, a depressed median. This illustrates why you have to know more than the median to understand a market; if you wanted a home in the top or bottom of the range, you'd find yourself spending more than you expected.

This is a closer look at the four lines. Recall: 74075 = blue triangles, the same Zip with only 3BR+2Ba homes included = black circles, Ridley Park = red squares, and 19803 = green diamonds.

Recall from the handwritten grid that the three highest-priced homes in 74075 are 495, 495 and 499. They are the three triangles surrounding a black dot at the top. The black dot is the single 3+2 house among them, in its own distribution.

The tendency of home prices in a market to approximately follow a lognormal trend indicates that near the median is where you will find the largest number of homes on the market. As you move far from the median, there is not going to be much for sale.

2. Quick Analysis (7-Point Charting)

To understand this method, look at the image above. See how the 1 and 99 points are rather far from the 10 and 90 points, and that these distances are quite similar to the distance between 10 and 25 or 75 and 90. Of course, the 50 point is the exact center, the median value. So we want to pick the values from the home listings that represent 1%, 10%, 25%, 50%, 75%, 90%, and 99%. For a 100-home distribution, this is just the first, tenth, 25^th and so forth. If a distribution has fewer than 50 homes, don't bother with the 1 and 99 points.

This table shows just these seven values for the three areas outlined above; I didn't bother with the 3+2 homes in 74075. To figure which homes to use, multiply the total number by the seven numbers, add ½ to each, and round the result.

For a 125-home market, the key numbers are 1.25, 12.5, 31.25, 62.5, 93.75, 112.5 and 123.75. Add ½: 1.75, 13, 31.75, 63, 94.25, 113 and 124.25. These round to 2, 13, 32, 63, 94, 113 and 124. We will discuss in a moment how to locate these items.

The "50" points are in slant type; these are the medians in each market. I also calculate the geometric midpoint, or virtual median, based on the 25 and 75 values, and 10 and 90 values. The calculation is, for example, SQRT(260*400) = 322.49, which rounds to 322, the 25-75 Median for Zip 19803. If both of these are close to the median, the market is balanced. If there is a strong trend, it reveals imbalance. Of course, the charts above show the imbalance in Ridley Park by the bent line, but here the numbers reveal it just as clearly. And, these can be plotted, as shown next.

There is a very subtle scoop in the 19803 line, and the numbers 315, 322, 331 show the same thing. 331/315 = 1.051. A 5% shift is insignificant. More significant is the very visible dip in the middle of Ridley Park, and the numbers show it: 150, 172, 187 and 187/150 = 1.247. A 25% shift is quite large. It indicates that the middle-priced houses are underpriced. A buyer's market!

The Stillwater Zip Code shows very good flatness in the middle range, with a zig and a zag, as mentioned, at the extremes of the range. The middle 80% of this market is balanced and homogeneous.

Now let us look at larger markets. This is useful to determine a price range that is appropriate for most "middle class" folks to buy homes, and how balanced the regional prices are. For three of the four analyses below, I selected only single-family homes; for Philadelphia I added condo/townhomes, which comprise 80% of the total market.

This shows a calculation table selecting the numbers for each market from the total number of homes on sale. Now, how do you find home #38 or #3,242?

By default, at the Realtor.com web site, there are ten houses listed at a time. At the bottom of the page you can click the "Next" button to get the next ten. When you do so, you'll see in the URL field (which starts "http"), near the end, "pg-2" (you may have to scan in the field). Change the "2" to the number of the page you want to go to. Now think about this; home #40 is at the end of page 4, so home #38 is the eighth listing on page 4. Similarly, to find listing #3,242, go to page 325, where it is the second listing (the page ends in #3,250). You have to count, because the web page doesn't put numbers on the listings.

Here are the 7 numbers collected for each of the four markets. I picked OK City because of familiarity, Columbus at random, San Jose because I knew it would be high, and Philadelphia because that's home these days (actually, I live in a suburb outside city limits).

A quick look shows that all these markets are well balanced. Dividing 517/490 = 1.055 indicates that San Jose may be slightly out of balance. By the way, prior to the 2008 crash, when I found an unbalanced region, it was more likely that the 10-90 median was lower than the actual median; high priced homes were undervalued, or conversely, the middle class, fueled by over-liberal lending policies, was overpricing mid-range homes. Let's look at the chart.

As expected, San Jose is high priced. Surprisingly, though Philadelphia is in the "rich" Northeast, its values are only slightly higher than the West and Midwest.

I was, frankly, quite surprised to find the distribution for Columbus as low as it is, lower than OKC. A number of years ago this was not so.

Seven-point analysis is the quickest way I know to learn so many things about markets of any size. It is the precursor to more specific analyses, helping you narrow your interest to price ranges which will have large numbers of listings.

Note: While I used Minitab for the first chart, there is a way to get a very similar chart using Excel or the Excel clone in the OpenOffice suite (it is called Calc). If a few folks rattle my cage about it, I might post a tutorial on making normal and lognormal plots using Excel. (Just to set expectations, there is no way to get the axis labeling the same. My method just produces a line with the right scaling and labels the probability axis in Standard Deviation units.)

A final tip. Choosing 3+ for the Bathrooms in Realtor.com nearly guarantees a recently built home, at least in the Northeast. In most areas of the country, only the largest houses had more than two bathrooms until the building boom of the 1980s and 1990s.

2010 Census and lognormal statistics

kw: census records, statistics, statistical distributions, analysis

Some time ago I wrote briefly about the US state population distribution in the 2010 Census, but the County data were not yet released. County data were released a couple months ago, and I just had the chance to review them.

This shows the county populations analyzed in Lognormal coordinates. While the fit is not perfect, a lognormal fit is better than any other that Minitab can analyze.

The lognormal distribution is appropriate for collections of parameters that divide up a fixed quantity using several criteria. The more "reasons" there are for division, the closer to a lognormal distribution the data will be, according to the logarithmic form of the Central Limit Theorem (CLT).

This repeats the US state information I plotted before, showing the goodness of fit within a lognormal model.

As I was thinking about this, I began to wonder why people choose to live in one place or another. In my own case, I made one choice based on the school I wanted for my graduate studies, and a few other moves based on job opportunities. One move was arranged to avoid a certain climate.

There are so many reasons, it is likely the CLT is well-satisfied. This does appear to be the case for these state and county data. I sought another level of analysis in order to see if I could tease out a few of the big factors.

This chart shows one factor and gives hints at one or two more. For each state, it plots average yearly rainfall horizontally and population density vertically. The state abbreviations help us discern part of what is going on.

With the exception of Alaska, there appears to be an upward-trending lower bound, anchored on the left by the cold, dry north-central states and on the lower right by the hot Gulf Coast states. The odd combination of NE, OR and ME fills in the middle. Moving upward from this line, I think I see two trends. One is that the states that were the original thirteen Colonies have high population densities, capped by New Jersey. Three states, CA, FL and HI, are "great climate" states. The remaining states tend to line up with warmth being an upward indicator, though there is lots of scatter.

This just scratches the surface of such an analysis. I suspect parameters such as the unemployment index and state taxation laws have their own influence; for example, I would expect Oregon to have a density closer to that of Delaware, as both have no sales tax, and they have similarly mild climates. Clearly, other factors come into play (like half of Oregon being mountainous and thus rather unlivable unless you are Jeremiah Johnson). For the moment, I've gone as far as I can without dragging together lots more data.

Imperfect Spider wins and losses

kw: games, statistical distributions

During the last year that I had a computer running Windows XP I kept statistics for 670 Spider Cell games. I won 169 of the games, or 25.2%. I have not yet kept any statistics for the Windows 7 version, but the program reports that I have won 29%. Perhaps this version is easier, or perhaps I am just temporarily ahead of the curve. Also, it may be that keeping the statistics interrupted the flow of play enough that I didn't play as well.

I gathered these statistics to see what the probabilities are for games of various length. The shortest possible winning game is 96 moves. Though there is no longest possible game, because you can use useless moves to inflate the numbers, I used rational rules of play to avoid making extra moves.

Of the games I won, the final tally ranged from 112 to 165, with a mean value of 140. This chart shows that the tallies are normally distributed, with a standard deviation of 11. That means that the intercept at a tally of 96 is at a standard deviation of -4.0. Thus I would expect a game in the 96 range about once per 31,600 winning games. At the rate I win, I might see such a score if I played about 120,000 games. The lowest tally I've seen is 108. That is at 2.9 sigmas, or once per 536 wins.

The statistics on losing are equally interesting. The most likely circumstance is just moving cards about and getting no suits to "complete". Generally, if you can get four suits completed, you will win, and I consider getting five completed means winning is assured, but I've had two games that had five completed suits, yet no win was possible. A statistical chart of losing games, charted by completed suits, is no surprise:

The more suits you complete, the more likely you are to be able to play longer, because typically more cards get uncovered. Note that my shortest game tallies 25 moves. There were two deals of six for which no move was possible. The shortest possible game is 0 moves, but that would require all the deals to be stonewalls, with no possible moves.

Though there is a little curvature to these distributions in line-normal space, we can estimate how likely such a situation is. Zero-deck games are nearly normally distributed with an average of 54 and a standard deviation of 13. This intersects zero at -4.15, meaning once each 60,000 zero-deck games. Such games make up about 45% of all games, so again, it would take playing about 120,000 games to have much chance of seeing a total tally near zero. The negative curvature of the line hints that this estimate may be very optimistic!

Well, I've certainly spent a lot of time gathering these data. Analyzing them has been fun. At the moment I don't expect to gather more statistics. I got different kinds of irons in the fire at present.

Trying to use statistics well

kw: observations, musings, statistical distributions

A simple illustration is more enlightening than a page of derivation, at least to me. I was thinking recently about the law of small numbers and its inverse, the law of large numbers. Though they are related, they have quite different emphases.

The law of small numbers embodies the observation that a sample of a few items taken from a large population, such as four balls from an urn containing an unknown number of black and white balls, is quite likely to give you a very inaccurate impression of the relative distribution in the larger population. Suppose all four balls drawn are white. How likely is it that the actual population contains equal numbers of black and white balls? Not as unlikely as you might imagine. What would you guess? One chance in ten, or fifty, or 1,000?

Suppose there are ten white and ten black balls. As you extract ball after ball, each being white, the odds for each draw are:

• 10/20 = 0.5000
• 9/19 = 0.4737
• 8/18 = 0.4444
• 7/17 = 0.4118

Multiplying these four together, we find the total probability of drawing four balls to be 0.04334, or about one in 23. If the number of balls is much larger, the probability approaches 0.5⁴, or 0.0625, one chance in 16. How does that square with your guess above?

Now, suppose you decide to draw twenty balls, hoping to get a better estimate of the distribution. If you don't know there are only twenty balls, you don't know, without trying to draw a 21^st ball, that you've taken the entire population! But if there are a great many balls, and you draw twenty white balls, you have a much better idea that there must be very few black balls, because 0.5²⁰ = 0.00000095, about one chance in a million. On the other hand, you can state that, if only one-tenth of the balls are black, 0.9²⁰ = 0.1216, or one chance in about eight. But this is not the law of large numbers. It is still making inferences from a single sample.

The law of large numbers expresses the surprise we feel when a "one in a million" event occurs, until we realize that there are billions of events occurring every day among the seven billion people on this planet: Given a large number of events, some of them are bound to be very unlikely. For this, a different kind of illustration is in order. The following four charts show Gaussian, or Normal, distributions containing one hundred, one thousand, ten thousand, and 100 thousand observations. I have them all here in a bunch, as Minitab "statistical summary" charts. Click on each to see it in more detail.


The overall impression is what is important at first. The first chart shows how a relatively small number of observations of a truly Gaussian variable add up to a rather poor fit to the normal bell curve. The fourth chart shows how a large number of observations produces a much better fit.

The standard Gaussian, or standard Normal, variable has mean of zero, standard deviation of one, and zero skewness and kurtosis. I'll explain the latter two terms in case they are new to someone.

• Skewness represents an imbalance between the right and left halves of a distribution. Positive skew means the distribution is "right-heavy". Natural phenomena that exhibit positive skew often follow a lognormal distribution. The power law distribution, beloved of fractal enthusiasts, has the greatest positive skew of any naturally-produced variable.
• Kurtosis refers to an imbalance between the "pointiness" of the distribution and the heavy or light tails that result. Positive kurtosis yields a super-Gaussian or Leptokurtic distribution. "Lepto" means "skinny" or "small" in Greek (the smallest Greek coin is the Lepton). Negative kurtosis yields a flattened or Platykurtic distribution (think Plate). Positive kurtosis means the tails are heavier than "normal", and extreme events are more likely than you might expect. Negative kurtosis means the tails are light and extreme events are scarcer than expected. Significantly negative kurtosis is rare in nature.
  

I planned to present a table of the key parameters of the four distributions, but the Blogger interface does not support HTML tables properly. So I'll finesse it this way. Each statement below gives the values of a parameter for the four data sets in order, for 100, 1,000, 10,000, and 100,000:

Mean: 0.0032, -0.0276, 0.0136, 0.0010
CI Mean: ±0.1831, ±0.0625, ±0.0195, ±0.0062

CI Mean refers to the confidence interval for the mean, as calculated by Minitab. Note how rapidly they shrink, reducing as the square root of the number of observations.

St Dev: 0.9231, 1.0070, 0.9941, 0.9975
Skewness: 0.0865, 0.0483, 0.0062, 0.0002
Kurtosis: -0.581, -0.169, 0.019, -0.033
Extremum: 2.2, 3.2, 4.0, 4.7

I know these are harder to read than a properly formatted table. The last line illustrates the law of large numbers. While a value such as 4.7 is possible for a 100-observation sample, it is very unlikely. Such a value occurs about once per 770,000 observations, so it is actually rather unlikely even for the sample of 100,000. The expected extremum for each sample is 2.33, 3.09, 3.73, and 4.27.

OK, how is this practical in my daily life? Let us consider height. The standard deviation for the height of American males is three inches (7.6 cm), and the mean value is currently seventy inches (178 cm). Out of 100,000 American men, what is the expected extremum? Take 4.27x3 to get 12.8, nearly thirteen inches or 32.5 cm. In such a sample, then, you can reasonably expect there will be one man nearly 83 inches tall (6-ft 11 or 2.1 m) and one man as short as 57 inches (4-ft 9 or 145 cm). What does it take to get someone eight feet tall (I think there is currently one Chinese man this tall)? That is an excess of 26 inches, or 8.67 standard deviations. Minitab tells me that the expected proportion is one chance in 4.6×10¹⁷. The tallest expected man out of 3.5 billion males on earth today is at 6.175 standard deviations, which is an extra 18.5 inches, or a total of 88.5 inches (7-ft 4.5 or 2.25 m).

Finally, can a study of statistics help us out in the stock market? Can a fellow get rich? See the following chart (click on it to see more clearly).

This is the daily variation for AT&T stock (symbol T) since 1984, nearly 6,600 observations. Note that the two or three central bars push way above the fitted curve, and that the tails also rise above the curve. This is classic positive kurtosis. It means that unexpectedly large deviations from the normally-fitted curve are very likely. In fact, although the standard deviation of the fitted curve is 0.76, the extremum is 7, more than nine standard deviations away!

These data are far from "normal", which gives us our first hint that the stock market is a great place to lose your shirt, as millions of investors already know. Of course, a plus extremum is as likely is a minus. The problem is, by the time one happens, you've missed it anyway, and because the stock market has a history (investors remember), regression toward the mean is more common than for a truly random variable. Study the curve above to understand the shape of investor emotion, which is what drives it away from a Gaussian normal curve: when the daily motion is small, the next day's motion is likely to be small also. A big motion might precipitate an even bigger motion the next day, but is more likely to be followed by a much smaller change, closer to the mean. Lots of "technical analysts" stay up late at night trying to fit time series estimators to stock market motions. None of them gets rich. D'you know what my richest friend invests in? Real estate. As Will Rogers said, "Buy land. Nobody is making any more of it."

Visual analysis of star mass distribution

kw: analysis, statistical distributions, stars, lognormal, power law

Below I analyze the mass distribution of a couple thousand of the nearest stars. I must first explain what is going on here. It has to do with finding a way to chart a series of data so that they approximate a straight line. This is called linearizing the geodesic.

One of my nerdy obsessions is gathering examples that illustrate the Theory of Breakage. The premise was proven mathematically by Kolmogoroff in 1941 (published in German) and offered to the English-reading public in The Lognormal Distribution by J. Aitchison and J.A.C. Brown in 1957. If you drop a brittle object onto a hard surface so it shatters, then weigh every piece, an analysis would show that the weights of the pieces are distributed lognormally. The analysis of clastic sediments follows lognormal reasoning.

Rather than foist a mathematical proof on you here, I'll support the idea conceptually. The lognormal distribution is based on the normal, or Gaussian, distribution, the famous "bell curve." The relation is thus: If you take the logarithm of a set of items which are distributed lognormally, the resulting distribution will be normal. These six numbers make up a very small normal sequence: 7, 8.4, 9.5, 10.5, 11.6, 13. The mean (and median) value for this distribution is 10. If you add the first and last numbers you get 20, twice the median; so if you add any pair of numbers equally distant from the "gap" between the 9.5 and the 10.5, you get twice the median.

If we take an exponential function of these numbers, we get a lognormal distribution. In this case, let us take 2 to the power of each of the numbers, and divide the resulting series by 100. Then these six numbers form a very small lognormal sequence: 1.29, 3.4, 7.2, 14.5, 30.7, 81.3. As these are lognormally distributed (for they were constructed so), the median is a logmedian, formed by multiplying the extrema and taking the square root: SQRT(1.29*81.3) = 10.2. Again, this works with any pair of numbers equidistant from the "gap" between the 7.2 and the 14.5.

Where a normal distribution is characterized by values clustered about a median value, a lognormal distribution has a cluster on the small side of the median, and a scattering of the largest values, in other words, a heavy tail. In many cases, the largest member of a lognormal sequence is larger than the sum of the rest of the terms.

A bunch of random items that are related by a similar additive process tend to be normally distributed. The Central Limit Theorem, upon which Kolmogoroff's proof is based, states that repeatedly taking the sum of several random numbers produces a normal distribution. Exponentiation turns addition into multiplication, so by the same reasoning, repeatedly taking the product (or the quotient) of several random numbers produces a lognormal distribution. Thus, in nature, additive (and subtractive) processes produce normally distributed values, while multiplicative or divisive processes (exemplified by breakage) produce lognormally distributed values.

I got interested in this subject when I read (I don't recall where) of an alternative hypothesis of breakage, that the resulting pieces might constitute a "scale free" or "power law" distribution. A power law distribution somewhat resembles a lognormal distribution, in that there are many small members and few large ones. However, the quantity of small members is quite a bit greater. A power law sequence is most simply produced by dividing a series of numbers into a constant, but all practical sequences are produced by dividing some power of each member of the series into a constant.

A very small power law sequence, produced by dividing 30 by 6, 5, 4, 3, 2 & 1, is
5, 6, 7.5, 10, 15, 30. Using the generating law to determine the median, we find it is 30/3.5 = 8.57 (The procedure used for a lognormal sequence yields 8.66). The drawback to continuing this series is that it has no finite sum. A power law sequence from a distribution that will converge to a finite sum is found by dividing the 3/2 power of the first six numbers into 30: 2.04, 2.7, 3.8, 5.8, 10.6, 30. This has median 4.6.

While there are software routines that can distinguish whether a sequence is distributed according to a power law or lognormally (and a great many others), there is a simple visual test that I like to administer. The illustrations that follow have larger-sized versions "behind" them, available by clicking. I prepared these using Microsoft Excel, which unfortunately doesn't have a probability axis option for its charting tools, so I use a transformation to linearize a probability axis.

I illustrate the technique here; I generated six sample distributions of 25 members each. Three are power law sequences and three are lognormal sequences. The latter were scaled to have a largest member equal to 10, so we can see their shapes in this chart and the one that follows.

This chart presentation is log-log. On such a chart a power law sequence plots as a straight line. The three blue lines are power law sequences, and the three dark red lines are lognormal sequences with different breadths of distribution (different values of the logvariance). The strong curvature of the red lines indicates that they are far from linearized in this type of chart.

This chart takes a standard normal sequence as the vertical axis; the value "1" means one Standard Deviation from the mean (1σ or "one sigma"). 25 values have probabilities ranging from 0.04 to 0.96, which are from -1.75σ to +1.75σ. The horizontal axis plots the values of the members of the six distributions, and is shown in logarithmic transformation. As before, the blue lines are the power law sequences and the red lines are the lognormal sequences.

Now it is the latter that plot in a straight line on these axes, while the power law sequences plot as distinct curves. This is diagnostic for each type of data; they plot as a straight line on appropriate axes, and as a curve under any other transformation.

Now we can plot the stars' masses. I gathered stellar data from many sources, but I'd like to acknowledge the Nearby Star Observers for their work and for a great collection of links that allowed me to find some of the sources of data I used.

I gathered information on mass where I could, but for most of the stars I used databases that had the stars' spectral and luminosity types, distances and magnitudes, and applied stellar evolution theory to approximate the masses. I eventually had a list of the masses of 2,200 stars out to a distance of 100 light-years. I plotted these, the 688 stars closer than 50 light years, and the 51 stars closer than 25 light years, in log-log coordinates for this chart. Of the 2,200 stars, 1,876 are on the Main Sequence (Class V), and are distributed spectrally as follows:

• B - 2
• A - 38
• F - 50
  
• G - 379
• K - 578
• M - 829

These accord well with other analyses of the relative number of each type of star. The total list also includes some giants and close to 100 white dwarfs, plus a very few brown dwarfs. For the white dwarfs I did my best to infer the mass of the star when it was younger, because the aim here is to investigate the breakup of the gas-and-dust clouds that formed these stars.

The clear conclusion from this plot is that the data I have are not distributed according to a power law. There are two possible reasons: a great many K- and M-type stars may yet be discovered within 100 light years of Earth, or the actual distribution is not power law.

A lognormal analysis yields much straighter lines. Let us focus on the pinkish line. This set of 688 stars is probably very nearly complete.

A sphere twice the radius of another ought to have eight times as many stars within it, compared to the other. But 2,200/688 = 3.2, so many stars in the 50-100 light-year range are probably not discovered yet, and most of them will be small, dim stars (K and M) and brown dwarfs. On the other hand, 688/51 = 13.5, so there is a dearth of stars in the solar neighborhood. This is known from the literature about the "solar bubble".

What does this chart show us? It indicates to me that the distribution of star mass is most likely lognormal, and if so we can infer a few things. Projecting a straight line through the pink sequence to the 4σ lines indicates that in a complete sample of about 32,000 stars there ought to be a smattering of giants with up to 9 or 10 solar masses, and similar smattering of M9 types and brown dwarfs as small as 0.02 solar masses, about 20 Jupiter masses. Extrapolating wildly to 7σ (almost a trillion stars, close to the probable number in the Galaxy), puts us in a realm in which the heaviest star approaches 100 solar masses and the lightest brown dwarfs are no more than twice the mass of Jupiter. These conclusions seem plausible, so I have considerable confidence in a lognormal distribution of stellar mass.