Search & SEO

AI visibility scores need a behaviour check before budget

Four datasets show AI assistants send little traffic next to Google. How to test an AI visibility score so it holds up when finance asks questions.

Short answer

An analysis in Search Engine Journal argues that AI visibility dashboard scores have not been validated against real customer behaviour. Traffic from assistants is small next to Google, and the effect on branded search could not be isolated. The author says the problem is not visibility itself but measuring it like a traffic channel.

Highlights

  • In a 100-site dataset, ChatGPT sent 0.62% of Google organic's volume, though its visitors converted at a higher rate.
  • In Kevin Indig's 632 tests, no reliable link appeared between AI visibility and later direct traffic or branded search.
  • The advice: treat AI referrals as diagnostic data, test the score against behaviour and re-run a fixed prompt library on a schedule.
Data analysis with laptops, charts and reports on a desk
Photo: Nataliya Vaitkevich / Pexels

2 min readAuthor: UNIT Journal EditorEditor-in-chief: Uğur Deniz İlhan

If an AI visibility dashboard helped you win budget this year, there is a roundup worth reading before your next quarterly review. In an analysis on Search Engine Journal, Greg Jarboe lines up four datasets and argues that most AI visibility scores would not pass a basic data quality test.

How much traffic do AI assistants send?

Very little next to Google. In the Marie Haynes study the analysis cites, 12 months of GA4 data from 100 sites showed ChatGPT sending 0.62% of Google organic's volume. ChatGPT visitors converted at 13.8%, against 3.02% for Google organic. Even so, Google drove 96.5% of total conversions.

Not yet in a way anyone can isolate. The usual argument is that people see a brand in an answer, do not click, and search for it later. Kevin Indig tested this with 632 tests, comparing weekly AI visibility against direct traffic and branded search at different lags. When he shifted the dates deliberately wrong, some mismatched alignments correlated as strongly as the real ones. His conclusion: an effect could exist, but he could not isolate it. Much of the rise in direct traffic was also made up of non-engaged sessions, most likely bots.

What should brands and agencies do?

The author's point is that AI visibility is not the problem; the score that grades it with traffic-channel maths is. Practical steps for your own dashboard:

  • Keep reporting AI referrals but do not set targets on them; use them as diagnostic data.
  • Filter direct traffic in GA4 by engaged sessions; bot sessions distort any validation of the score.
  • Compare the score with weekly direct traffic, branded search and lead counts at a lag, and repeat with wrong dates to see whether the match is real.
  • Pick prompts from real buyer questions, run them as a fixed library across several tools on a schedule, and count mentions, citations and recommendations separately.
  • Add a 'How did you hear about us?' field to forms and demo requests, with AI assistants as an option.

You can carry the same framework into GEO (generative engine optimisation) work, which manages visibility in search and AI answers with one measurement. Most of the data comes from US-focused sites; Turkish-language queries may look different, so test it in your own GA4 first.

Frequently asked

Should I stop measuring AI visibility?
No. The author criticises the unvalidated score, not visibility itself. He recommends testing the score against behaviour data and tracking it like a brand channel, through exposure and recall.

Sources

  1. Search Engine Journal ·

Follow UNIT Journal

What's new in search, AI and technology, in your feed every day.

Related articles

← Back to UNIT Journal

Let us measure
your visibility today.

We map your current search visibility and your standing inside generative engines. Free, one page, real data.

Request an Analysis
Get a Quote