Stop AI training on your website: what Google's plan means
Google plans to back a single robots.txt setting that tells AI providers whether they can train on your content. Here's what to decide now and what to check this week.
Google has announced plans to support one shared setting in a website's robots.txt file. It would let you tell AI companies whether they can use your content to train their AI. It isn't live yet. Google says it aims to support it, along with other AI providers, in 2027.
This matters because the choice is yours, and it's better made calmly than in a rush when the setting arrives. Here's the short version, and what to do about it.
What Google announced
In a post on Google Search Central, Google's John Mueller said the company plans to support a shared AI training control. At the moment, each AI developer can have its own instructions. The new approach uses a single "policy token" in robots.txt to set a broad preference across participating AI providers.
The idea comes from the AIPREF working group at the Internet Engineering Task Force (IETF), a body that develops internet standards. Google describes it as a draft proposal. It says it will provide updates on progress.
What robots.txt is
robots.txt is a small text file on your website. It gives automated visitors, such as search engines, instructions on what they may and may not use. Most small business owners have never opened it. Many don't know whether theirs says anything useful.
Can you stop AI training on your website now?
For Google, yes, in part. Google has offered a control called Google-Extended since 2023. It is Google's AI training control.
The new shared setting adds to that rather than replacing it. Google says existing company-specific controls will keep working alongside the new signal. Here is how the two fit together:
- If your robots.txt uses the new token to opt out of training, and Google-Extended isn't mentioned, Google plans to treat that as if you had blocked Google-Extended.
- If you do mention Google-Extended, that instruction is still there for you to tailor.
- Other AI providers can handle the token similarly, so you could opt out across all of them or pick and choose.
The catch: this only works for providers who adopt it. Google says broad support across major AI providers is what will make a lasting standard possible. Don't assume every AI company will honour it.
The source doesn't say how the new setting will work specifically for UK sites. It does say the proposal is consistent with the EU Code of Practice for general-purpose AI, which recognises robots.txt as a main way for publishers to state their preferences.
The trade-off to think about
There's a reason not to opt out of everything without thinking. Google says websites that specifically allow Google-Extended are eligible to appear in responses and get clicks through grounding in Gemini Apps. Grounding is where Gemini uses web content to back up its answers.
So you have a real decision:
- Opt out of training. Your content isn't used to train AI. You may also give up that route to appearing in Gemini responses.
- Allow it. Your content may be used for training, but you stay eligible to appear and get clicks through Gemini Apps.
- Mix it. Use the shared signal for a broad stance, then tailor individual providers.
There's no right answer for everyone. A trades firm with a handful of service pages has different concerns from a consultancy publishing original guides, or a shop with its own product photography and descriptions.
What to do this week
None of this needs a rushed change. It needs a decision and a check.
- Decide your stance. Ask: is our content valuable and original enough that we'd rather not have it used for training? Would we like to turn up in AI answers?
- Find out who controls your robots.txt. It might be your web developer, your hosting company or a plugin. Ask them.
- Ask what it currently says. Does it mention Google-Extended? Has anyone ever set it deliberately?
- Write your decision down. Then when the shared setting arrives, whoever looks after your site knows what you want.
- Run a quick check. Our free website check looks at security, speed, SEO and AI-readiness, and it's a good starting point.
If you're thinking about how AI fits into your business more widely, our page on getting a business AI-ready covers it.
What not to do
Don't panic, and don't block things you don't understand. A wrong line in robots.txt can cause problems that are hard to spot. Nothing about the new setting changes until 2027, so you have time to get this right.
Need a hand?
If you'd like someone to look at your robots.txt and talk through the options, book a free 30-minute call. We'll tell you what we see and what we'd suggest.
Questions
Is the new shared AI training setting available now?
No. Google has announced plans and says it aims to support it, along with other AI providers, in 2027. It describes the proposal as a draft.
Does it replace Google-Extended?
No. Google says existing company-specific controls, including Google-Extended, will continue to work alongside the new shared signal.
Will it stop every AI company training on my content?
Only providers that adopt it. Google says other AI providers can handle the setting similarly, but support across the industry is still to come.
Should I opt out of AI training?
It depends on your business. Google says sites that specifically allow Google-Extended are eligible to appear in responses and get clicks through grounding in Gemini Apps, so weigh that against your concerns about your content.