What percentage of your traffic is AI? We discovered this only after running our own backend for six months: among visitors,Real users constitute only a portion; the rest are AI crawlers, search engine crawlers, and scanners— Treating them as real users for retention and conversion analysis yields incorrect conclusions. This tool locally parses your access logs: distinguishing real users / AI / search engines / scanners, and outputs source attribution, landing pages, device distribution, and Top pages. IPs are automatically anonymized; no log line is uploaded.。
—
—
1 · Treat crawlers as humans.Early statistics counted all AI crawlers and scanners as UV, doubling daily PV; later, splitting UA into four categories revealed that real users constitute only a fraction.
2 · Cumulating by day is incorrect.Aggregating via “yesterday’s value + today’s increment” causes duplication upon reprocessing. The correct approach isRecalculate Coverage Daily, ensuring idempotency.
3 · Format timestamps before filtering.Converting timestamps to strings before performing time window comparisons causes errors or data loss—stringification must occur after all aggregations.
4 · Logs must be anonymized.Full IPs exist in raw logs; forwarding them directly leaks visitor information— anonymize uniformly before output