API and Google results miss most of what ChatGPT cites

According to Search Engine Journal, researchers from the National University of Singapore and Xi'an Jiaotong University tested whether Google results and OpenAI's Responses API can stand in for ChatGPT's citations. The work is an arXiv preprint that has not been peer reviewed, so the findings are a first signal rather than a final verdict.

How much of ChatGPT's citations do Google and the API capture?

Only a part. The researchers recorded the data flow of ChatGPT's web app in their own sessions and checked which pages it chose as sources. They then compared this with Google's organic results and with the API with web search enabled. In the GPT-5.5 Instant setting, the two together found 26.2% of the cited pages and 48.7% of the sites behind those citations.

The test used 80 questions, each run 20 times. A tool or a single manual check may therefore see less than half of the sources ChatGPT actually shows.

Does ChatGPT search the question as written?

No. The researchers report that in the GPT-5.3 Instant and GPT-5.5 Instant runs where searches were visible, ChatGPT never searched the exact wording of the question. For Chinese questions, 69.6% of the runs with visible searches used only English for GPT-5.5 Instant. This does not mean the same holds for Turkish; the study did not test Turkish.

Sources also change between runs: 44.5% of the distinct pages returned for a question appeared in only one of the 20 GPT-5.5 Instant runs. Overlap is higher at site level than at page level.

What should brands and agencies do?

The report states that it did not test commercial tracking tools, so it does not say your tool is wrong. Still, reporting a tool's output as a fixed number is risky. Suggested steps:

• Report citation tracking at site level as well as page level; page-level numbers are far more volatile.
• Run the same prompt several times and report how often you appear rather than a single result.
• Ask whether your tool collects data from the ChatGPT web app or from the API; the two should not be treated as equivalent.
• If your audience asks in Turkish, remember that AI tools may also search in English and monitor your English content too.
• Do not compare data directly with earlier months in weeks when the model changes.

Work on GEO (generative engine optimisation) should account for this uncertainty from the start. The data covers models from March to July 2026, and the picture may change in later versions.