问题原文
原帖链接 · ardacarofficial · CC BY-SA 4.0
I have 1 own website and 1 CDN server. I have CSS, JS and images on my CDN server. All the CSS, JS and images on my own website come from the CDN server.
If I block my CDN server with a robots.txt file, will images on my own website but whose URLs are on the CDN be indexed? Also, does Google Bot ignore CSS and JS files?
已采纳解答
原帖链接 · Maximillian Laumeister · CC BY-SA 4.0
The robots.txt standard is per domain/subdomain. So the file at cdn.example/robots.txt will control access to resources on cdn.example, and the file at yoursite.example/robots.txt will control access to resources on yoursite.example.
What this means is that for example if cdn.example/robots.txt has a User-agent: * ⏎ Disallow: / directive, all resources on cdn.example including images will be blocked. So in that case if you have a web page on yoursite.example that tries to load those CDN images as part of the page, those images will be blocked from being accessed by the crawler, so the crawler will render your page without the images.
Google bot uses CSS and JS files to render and rank your page properly. It will complain if you don't allow it access, and it may demote your site somewhat due to not being able to check it for mobile friendliness, page load time (part of "page experience"), etc. Definitely do not block CSS or JS files from being crawled by Google.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
