问题原文
原帖链接 · maro_vargovcik · CC BY-SA 3.0
i tried searching on internet about this question and how Firebase handle indexing and how owner of the content running on Firebase hosting can influence crawling by Google bot.
Personally, i am running website built with Polymer + Firebase as backend (so basically SPA) with like 150 visitors per week and my website has common navigation with 5 links:
- /price-list
- /contact
- /projects
- /about-us
- /home
Also projects page has nested
- /projects/interior
- /projects/architecture
Google indexed my website far more than it should and currently results like this appear when googling
/projects/interior/[[id]]/[[name-of-interior-project]]
So back to my question: How can i influence Google bot to forbid crawling specific sites? It is enough to just upload robots.txt or sitemap file along with other files like i would normally do with common hosting?
Thanks!
已采纳解答
原帖链接 · Maadinsh · CC BY-SA 3.0
Yes, you can block access to specific urls with robots.txt rule. If the page has real, unique url and google is indexing it, it doesn't matter that it's a SPA and by default links are loaded without page refresh.
Disallow: /projects/interior/
should match /projects/interior/[id]/[name] but not /projects/interior
You can test robots.txt rules with Google Search Console to be sure.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
