Short answer
Google's John Mueller said a popup that visitors are required to see, such as an age verification, can be hidden from Googlebot with robots.txt. There are two ways: block the external domain that serves the popup, or block the script file that loads it on your own site. The page must still load properly without the blocked resources.
Highlights
- Mueller said blocking an age-verification popup with robots.txt would generally work.
- The first way is to disallow crawling in the robots.txt of the external domain that serves the popup, a method common with ad services.
- The second is to block the script file that loads the popup in your own robots.txt, though Mueller does not advise it if that file does many things at once.
2 min readAuthor: UNIT Journal EditorEditor-in-chief: Uğur Deniz İlhan
As Search Engine Journal reports, Google's John Mueller answered whether an age-verification popup that visitors must see for legal reasons can be hidden from Googlebot with robots.txt. The answer clarifies which resources can be blocked on purpose.
Is it safe to block a JavaScript resource with robots.txt?
According to Mueller, in this specific case yes: blocking Googlebot from rendering the popup with robots.txt would generally work. The report recalls that Google once cost sites rankings when they blocked JavaScript and CSS files, so the general rule has not changed; this is an explanation about one kind of resource. What matters is that the page still loads properly when the blocked resources are not fetched.
What are the two ways to block it?
The first is to disallow crawling in the robots.txt of the external domain the popup comes from. Mueller says that if the domain serving the embedded content or script disallows all crawling, crawlers that follow robots.txt will not fetch it or use it in rendering; many ad services work this way.
The second, if you cannot influence the external domain's robots.txt, is to block the script file that loads the popup in your own robots.txt. Mueller notes this is less common and advises against it if a single script file does many things at once.
What should websites in Turkey do?
Sites that use third-party components such as age verification, cookie consent or chat windows can follow these steps in their SEO checks:
- Use developer tools to identify which script file loads the component you want to block, and target it only if it does one job.
- Compare the rendered page in Search Console's URL inspection before and after blocking, and confirm the main content and links stay visible.
- Do not block scripts that load the main content or navigation; consider only components that visitors must see but that add nothing to search.
- Try the change on a few pages before rolling it out site-wide, and monitor rankings.
Frequently asked
- Does this mean blocking any JavaScript file is now allowed?
- No. Mueller addressed only a specific age-verification popup. Blocking resources that load page content or navigation can hurt rankings.
Sources
- Search Engine Journal ·
Follow UNIT Journal
What's new in search, AI and technology, in your feed every day.
