Spiders getting down to business

 kw: blogs, blogging, spider scanning, AI training

What I call spider scanning software seems bent on setting new records. The spiders have buckled down to work! For the past week the average hit rate has been quite steady near one per second.

What do you make of these numbers?

Compare this month (so far) with the prior month. Nearly 7 times the traffic. A full month at one hit per second would come to about 2.6 million. Note that the "All Time" figure goes back to the date that Google Analytics were applied to Blogger, around a decade ago. For more context, here are the charts for the past 30 days, 7 days, 24 hours, and 2 hours.

In the upper two charts, a one-pre-second rate would be 86,400. We see that wasn't quite achieved. In the third chart, it would be 3,600 per hour, and in the last, 60 per minute. The last couple of hours before I obtained these charts, we see that a sustained rate of 80 per minute is possible; that's 1.33 per second, 4,800 per hour, 115,200 per day, and 3,456,000 per 30 days (or a bit more than 3.5 million per average month). In a few days it will be more clear how this is progressing.

Meanwhile, I'm interested in "where?". Here are the 7 day and one day top countries:

For the most recent full day, USA is running at about half its rate for the prior six days. Otherwise, these top 9 actors are the same for both periods. It is an odd mix. Although I mostly attribute this level of activity to a combination of AI training and AI agent lookups, I really can't be sure.

By looking at a list of the most "popular" posts in the different time periods, it is still true, as I reported earlier, that page-by-page views only constitute a few percent. Most of these views are front page only, with the understanding that the most recent ten to fifteen posts are presented in their entirety, and a "Load Older Posts" button brings up the next group, and so forth. That means the entire blog can be reached with fewer than 200 clicks, clicks which do not add to the totals for individual posts.

I'm curious: to any folks who read this, who use Blogger, please check in your stats to see if there are patterns similar to what I am seeing, and leave comments. Thanks!