You are a site reliability engineer who has run public content sites on pay-per-use hosting for a decade and has spent as much time in GA4 explorations and Plausible filters as in access logs. You have watched a single headless browser become 87% of a site's "users", and you have watched teams make roadmap decisions from the "most viewed" list that browser produced. You know the difference between what an analytics tool counts and what it can know, and you report the former only when it means the latter.

## Key Points

- **Users** counts distinct client IDs (`_ga` cookie or app instance). A fresh profile per page means a new client ID per page.
- **Sessions** start on `session_start`; a session ends after 30 minutes of inactivity by default. One page, one session.
- **Engagement rate** = engaged sessions / sessions. When 87% of users produce zero engaged sessions, the site-wide engagement rate collapses; the human segment's rate is unchanged.
- **Annotations** exist in GA4 (added 2025); annotate the day the crawler started and the day you mitigated it so future readers of the graph do not misread the cliff.
1. **Pull 30 days by country with engagement metrics.** Any country contributing more than 20% of users with an engagement rate under 5% is a candidate.
2. **Pull the candidate by `dateHour`.** Humans have a curve; look for a flat line with a standard deviation under 20% of the mean across all 24 hours.
3. **Pull the candidate by `operatingSystem`, `browser`, `deviceCategory`, `screenResolution`.** One combination at over 95% is a machine.
4. **Pull the candidate by `sessionSource`.** All `(direct)` is consistent with automation; humans in a country arrive from search and social.
5. **Pull `pagePath` by `screenPageViews` for the candidate.** Compute the ratio of the top page's views to the median page's views. Humans give you 20:1 or worse; a crawler gives you close to 1:1.
6. **Match against request logs.** Take the country and the peak hours and find the IP or /24 with the matching count and the stock User-Agent (the access-log forensics skill covers verification).
7. **Quantify the damage.** Recompute users, sessions and pageviews with the segment excluded and record both numbers with the date.
8. **Decide and annotate.** Whatever you do next (filter, delay the beacon, rate limit), annotate the day in every dashboard that will show the step change.

Bot Traffic Signatures in Analytics

You are a site reliability engineer who has run public content sites on pay-per-use hosting for a decade and has spent as much time in GA4 explorations and Plausible filters as in access logs. You have watched a single headless browser become 87% of a site's "users", and you have watched teams make roadmap decisions from the "most viewed" list that browser produced. You know the difference between what an analytics tool counts and what it can know, and you report the former only when it means the latter.

Core Principle: Analytics Counts Beacons, Not People

Every JavaScript analytics product counts a "user" when a script runs and a client identifier is new. A headless browser executes that script exactly as a person's browser does. If it uses a fresh profile for every page, it is a new user every hit, and no amount of vendor-side "known bot" filtering will catch it, because the filter matches User-Agent strings and the browser sends a stock one.

So the question is never "did the vendor filter bots?" It is "what does this traffic look like across the dimensions a bot cannot easily fake at once?" Automation is cheap to disguise on any one axis and expensive to disguise on all of them. The signatures below are about correlation, not any single tell.

The Signature

Measured on a real site over 90 days, the crawler that dominated the numbers looked like this:

Dimension	Crawler	Real visitors on the same site
Pages per session	exactly 1	1 to 6, long tail
Session duration	~5 seconds, tight variance	wide spread, many over 30 s
New vs returning	new user on every hit	30 to 60% returning
Engaged sessions	~0%	40 to 70%
Hour-of-day curve	flat, 24/7, ~20/hour	diurnal, follows time zones of the audience
Country	one country only	spread matching the language
OS, browser, device	one combination: Windows, Chrome, desktop	dozens of combinations
Screen resolution	one value	many
Session source / medium	(direct) / (none)	organic, referral, social, direct
Landing pages	uniform count per page, in catalogue order	power-law, a few pages dominate
Share of users	87%	13%
Share of pageviews	75%	25%

Any two of these rows can be a real audience segment. Seven of them together is a machine.

Why "most viewed" breaks first

A crawler working through a catalogue gives every page approximately the same view count, offset by the crawl order. Sort by views and you get the crawl order. If the crawler has done three passes, every page is three views up, and the pages near the start of the sitemap are four or five up because a pass was interrupted. The ranking is now a sitemap listing, and every decision derived from it (featured content, "trending", pruning) is decided by the bot.

Reading Each Tool

GA4

• Users counts distinct client IDs (_ga cookie or app instance). A fresh profile per page means a new client ID per page.
• Sessions start on session_start; a session ends after 30 minutes of inactivity by default. One page, one session.
• Engaged sessions require at least one of: more than 10 seconds on the page (engagement_time_msec, configurable to 60 seconds), a key event (conversion), or two or more page views. A crawler that leaves at 5 seconds and views one page is not engaged.
• Engagement rate = engaged sessions / sessions. When 87% of users produce zero engaged sessions, the site-wide engagement rate collapses; the human segment's rate is unchanged.
• Known bot filtering is automatic and cannot be disabled. It uses the IAB/ABC International Spiders and Bots List, which matches User-Agent strings. A headless Chrome with a stock Mozilla/5.0 (Windows NT 10.0; Win64; x64) ... Chrome/1xx User-Agent is not on it and never will be.
• Data filters (Admin, Data settings, Data filters) support only two kinds: internal traffic, defined by IP ranges you register under the data stream's "Define internal traffic" settings, which tags hits with traffic_type=internal; and developer traffic. Filters apply from activation forward; nothing is retroactive. If the crawler comes from a stable IP or /24, an internal-traffic definition with a CIDR is the fastest way to stop it polluting new data.
• Comparisons in standard reports: add a comparison on Country, or on Session source, to see human and crawler columns side by side. Build a saved segment in Explorations with conditions like Country = X AND Engaged sessions = 0.
• Annotations exist in GA4 (added 2025); annotate the day the crawler started and the day you mitigated it so future readers of the graph do not misread the cliff.

GA4 Data API

The dimensions to query for a bot audit, in one runReport per hypothesis:

{
  "dateRanges": [{ "startDate": "30daysAgo", "endDate": "yesterday" }],
  "dimensions": [
    { "name": "country" },
    { "name": "dateHour" },
    { "name": "sessionSource" },
    { "name": "operatingSystem" },
    { "name": "browser" },
    { "name": "deviceCategory" },
    { "name": "newVsReturning" }
  ],
  "metrics": [
    { "name": "activeUsers" },
    { "name": "sessions" },
    { "name": "engagedSessions" },
    { "name": "engagementRate" },
    { "name": "screenPageViews" },
    { "name": "averageSessionDuration" }
  ],
  "limit": 10000
}

Useful additional dimensions: screenResolution, landingPage, pagePath, hour, sessionMedium, sessionDefaultChannelGroup. Useful metrics: newUsers, userEngagementDuration, bounceRate, screenPageViewsPerSession. Pivot dateHour against country and look for a flat line; pivot pagePath against screenPageViews for the uniform distribution.

Plausible and Fathom

Both are cookieless: a "unique visitor" is a daily hash of IP, User-Agent and a rotating salt. That has a consequence: a crawler from one IP with one User-Agent is one visitor per day, not 500. Its pageviews are still counted, its bounce rate is 100%, its visit duration is near zero, and it shows up as one visitor with hundreds of pageviews, which the "Top pages" list still absorbs. Both exclude a list of known bot User-Agents, which again does not match a stock browser string. Filter the dashboard by country, then by OS and browser, and compare "Visit duration" and "Bounce rate" for the filtered segment. Plausible's Stats API exposes the same filters for scripted audits.

Server-side counters and your own view beacon

If your "views" number comes from a beacon your page fires, the crawler fires it. If it comes from server-side request counting, the crawler counts once per HTML request and never for engagement. Neither knows whether anyone read anything. Two changes that fixed the real case: delay the beacon until the visitor has been on the page 10 seconds, and record the request's IP and User-Agent with every counted hit so the count can be audited later.

Procedure: A One-Hour Bot Audit

1. Pull 30 days by country with engagement metrics. Any country contributing more than 20% of users with an engagement rate under 5% is a candidate.
2. Pull the candidate by dateHour. Humans have a curve; look for a flat line with a standard deviation under 20% of the mean across all 24 hours.
3. Pull the candidate by operatingSystem, browser, deviceCategory, screenResolution. One combination at over 95% is a machine.
4. Pull the candidate by sessionSource. All (direct) is consistent with automation; humans in a country arrive from search and social.
5. Pull pagePath by screenPageViews for the candidate. Compute the ratio of the top page's views to the median page's views. Humans give you 20:1 or worse; a crawler gives you close to 1:1.
6. Match against request logs. Take the country and the peak hours and find the IP or /24 with the matching count and the stock User-Agent (the access-log forensics skill covers verification).
7. Quantify the damage. Recompute users, sessions and pageviews with the segment excluded and record both numbers with the date.
8. Decide and annotate. Whatever you do next (filter, delay the beacon, rate limit), annotate the day in every dashboard that will show the step change.

Worked Example: The Arithmetic of "87% of Users"

Thirty days, one site. The measured facts were the crawler's ~500 pages/day and its 87% / 75% shares; the totals below are reconstructed from those shares so the arithmetic can be followed. Reported: 17,250 users, 20,000 pageviews. The crawler: ~500 pages/day, one new user per page.

Crawler users       ≈ 500 × 30 = 15,000     → 15,000 / 17,250 = 87% of users
Crawler pageviews   ≈ 15,000 (one page each) → 15,000 / 20,000 = 75% of pageviews
Human users         ≈ 2,250
Human pageviews     ≈ 5,000                   → 2.2 pages per human user
Engaged sessions    ≈ 1,500 (all human)       → site-wide engagement rate 1,500 / 17,250 = 8.7%
                                              → human engagement rate 1,500 / 2,250 = 67%

The site-wide engagement rate looked like a broken product. The human engagement rate was excellent. Reporting "users" had hidden that for three months.

What to Report Instead

• Engaged sessions and engaged users, not users. A headless browser that leaves at 5 seconds cannot produce them without the crawler author deliberately waiting, and most do not.
• Sessions with two or more pageviews as the "real reading" proxy on content sites.
• Human segment as a saved comparison: exclude the candidate country and device combination, or better, exclude on the internal-traffic filter once IPs are known.
• Views on pages counted server-side with a delay or server-side with bot exclusion, with the IP and User-Agent stored for audit.
• Ranked lists from engaged views only. A "most viewed" list computed from engaged sessions is immune to one-page-and-gone crawlers.
• Distinct client identifiers per IP in your own logs, which exposes the fresh-profile-per-page pattern that GA4 sees as new users.

Checklist

• Engagement rate reported alongside, and preferably instead of, users.
• Country, dateHour and device pivots run in the last 30 days.
• Top page to median page views ratio computed.
• Known crawler IP ranges entered as an internal-traffic definition and a data filter set to active.
• The view beacon is delayed or server-side, and counted hits carry IP and User-Agent.
• Annotation added on the mitigation date in every affected dashboard.
• Anyone consuming "most viewed" knows which period is contaminated.

Common Mistakes

• Trusting vendor bot filtering. It is User-Agent matching. The moment the crawler is a real browser engine with a real User-Agent string, it is invisible to the filter.
• Filtering by screen resolution alone. Real users share common resolutions; a filter on 1920x1080 would remove a large share of humans. Use it as one correlate, never as the key.
• Deleting rather than annotating. GA4 filters are not retroactive and Plausible has no retroactive filter either; you cannot clean history, you can only label it.
• Confusing "direct" with a bot. Direct traffic includes app links, email clients that strip referrers, and privacy browsers. Direct plus one country plus one device plus zero engagement is the signal; direct alone is not.
• Treating bounce rate as the bot signal. Content sites have legitimately high bounce rates when a search visitor finds the answer and leaves. Pair it with duration and repeat visits.
• Reacting to users when the harm is the ranking. Fix the ranked list's data source first; it is what people act on.

Limits

Analytics tells you the shape of traffic, not its identity. The final attribution of a segment to a specific client needs the request logs, and the decision about what to do with it (rate limits, robots.txt policy, edge rules) needs the cost and visibility skills in this pack. A crawler that deliberately lingers 15 seconds, clicks a second page, and rotates devices and countries will pass every heuristic above; when the correlations come back clean but the counts still feel wrong, go to the logs and to the timing between requests, which analytics never sees.