Task: Analyze website visits

Google Analytics, Yandex.Metrica, and similar tools provide a lot of useful information, but: 1. It is retrospective. That is, you can find out what happened, but you don't really see what is happening right now. 2. They do not fully tell you how often and which bots visit your site, which pages they view, etc.

Now I am going to add new sections and publish a lot of new information. I need to understand how well search engines consume this (how often they visit and where).

I recently added logging for 404 and other errors. Now I need to enable logging for all visits in general, so that I can monitor what is happening in real time.