Another spider breakout

 kw: blogs, blogging, spider scanning, ai training

For many months the traffic to this blog has been ten to a hundred to a thousand times the level that it was for many prior years. I attribute the recent onslaught to the entire blog being snarfed up by online spiders for adding to training data for large language models (LLMs). The past couple of days the traffic has set a new record.

One one prior occasion this year it peaked at about 60,000 daily views, or more than forty per minute. Yesterday there were 61,894 and so far today there have been 62,417. For most of the past month traffic has been rocking along in the range of 3,000 daily views, just over two per minute. Now it is approaching one per second, as this chart of the prior two hours shows:

For the first hour, the average appears to have been at or over 60 per minute, or one per second, or 86,400 per day. The second hour sits near 50 per minute, or 72,000 per day. The sharp drop at the very end may portend a return to "only" a few thousand daily views. I'll know in a few hours.

This time, more than half the traffic originates (or passes through VPNs) in the US. China is not seen at all. I wonder if blogs that are a lot more popular—with tens of thousands to millions of "honest" daily views—are getting hit this hard, or perhaps much harder. Is anybody looking?