问题原文
原帖链接 · ZeSoft · CC BY-SA 3.0
I have a php website(symfony) with dynamic content, some is published by the admin and some by the loggedin users. I have created a dynamic sitemap that gets updated whenever a content get added, edited or deleted.
Is this a good and optimized way to do it?
I added the URL of the sitemap to the robots.txt file and I should as well submit it to search engines(google).
I am wondering if I submit it and it's dynamic then google will have only that version I submitted at that time?
While the sitemap is dynamically updated, will it automatically read each time the new version or should I resubmit it after some time which I wont be able to specify because users posts also update the sitemap?
已采纳解答
原帖链接 · Goyllo · CC BY-SA 3.0
Is this a good and optimized way to do it?
I would say no, because first time when Google see some URL, then they will index it, and later if that URL is removed then index page will not going to remove immidietly, Google assign there might be some problem and hence return a 404 error in search console, and Google will try to re-crawl same URL again even it is not linked anywhere. Yes Google re-crawl some pages again to check weather the content is updated or not.
And when the URL is changed, then new URL will going to index, here you should do 301 redirection for better visibility, it means remove the old one and index the new one.
I added the URL of the sitemap to the robots.txt file and I should as well submit it to search engines(google).
What you want to opimize? Google, Bing or Yandex? When you add your sitemap into robots.txt then all search engine follow your sitemap in regular days, even dead search engine like ask.com says, we don't accept new site submission, but if there is sitemap link then we will crawl it. But having a sitemap link into robots.txt means it is easy to scrape your all content by bad bot. But only popular site need to worry about it like Stackoverflow hide their sitemap from all users execept search engine bot.
I am wondering if I submit it and it's dynamic then google will have only that version I submitted at that time?
Yes Google don't understand dynamic thing in sitemap. They take action at a time.
While the sitemap is dynamically updated, will it automatically read each time the new version or should I resubmit it after some time which I wont be able to specify because users posts also update the sitemap?
Sitemap crawler set a time based on frequency of updated content, but sitemap is not only one crawler which index pages, Google follow a link as well to index content. Google crawl frequency is based on many things.
The old crawler(Since 1998) is still used heavily by Google.
- The site having a good amount of Pagerank crawl often, example Wikipedia.
- The site having a good amount of Backlinks crawl often, example Stackoverflow.
The new sitemap crawler.
- Sitemap crawler.
- Google submit Tool or Fetch/Render tool.
- Refresh crawler (I don't know what google call it) but they re-crawl indexed pages, even when remove all pointing links.
- Google now use many other sources as well to find popular link like Google DNS service or may be Social media trends. I don't have more info in this.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
