问题原文
原帖链接 · cbdeveloper · CC BY-SA 4.0
I have this web app that is accessible through those different URLs.
URL_1: www.mycustomdomain.com // THIS IS MY CUSTOM DOMAIN (PUBLIC TO THE USERS)
URL_2: www.my-project.firebaseapp.com // THIS IS A DEFAULT FIREBASE URL
URL_3: my-project.web.app // THIS IS A DEFAULT FIREBASE URL
Obviously I want Google to index my pages using my custom domain, and not the Firebase default URLs.
I don't publicize the default URLs anywhere, but what if Google discovers them at some point? What is the proper way of letting Google know that those pages/URLs should NOT be indexed?
OPTION #1
Should I always add a <link rel="canonical" href={"https://www.mycustomdomain.com/some-page"}/> for each page?
This way, if Google happens to crawl the default URLs, it should see that canonical pointing to the custom domain, right? Is this the correct approach?
OPTION #2
Since I'm dynamically generating the robots.txt, I could detect that the request is coming from URL_2 or 3 and simply block the entire website from being crawled, by adding Disallow: /.
How would you solve this?
已采纳解答
原帖链接 · Itai · CC BY-SA 4.0
It happen that Googlebot discovers the alternate URL. The official way if you want your application to be accessible through all those URLs for internal purposes, is to use canonical links as you suggest.
The other way is to simply issue 301 Permanent Redirect so that all traffic uses the official URLs. This will also allow you to enforce the use of HTTPS as it also happens that Googlebot indexes both the HTTP and HTTPS version when they exist and there is no redirect between them.
Given that you control the site, you are even free to not issue the redirect for internal traffic. This is common if the internal server does not have an SSL certificate. You can redirect when the IP address is internal to your network, for example.
Robots is generally not a good option because bots can choose to ignore it. Googlebot will respect it but you may have to cover for other bots.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
