大网数据广告活动
网站运营与故障问答

Redirect Errors in Google Search Console when trying to index site

· 52CE 我爱测

I've built a small site for a local trader which is not a complicated site in any way. In fact it is 100% static. I have a few years experience and in that time have never encountered an error with Google Search Console where my …

问题原文

原帖链接 · Cutey from Cute Code · CC BY-SA 4.0

I've built a small site for a local trader which is not a complicated site in any way. In fact it is 100% static. I have a few years experience and in that time have never encountered an error with Google Search Console where my robots.txt and sitemap.xml have always been accepted and the pages properly indexed.

On this most recent site, the robots.txt and sitemap.xml are both fully verified and accepted but I am receiving redirect errors on some of the most important pages and I don't understand why. I have not set up any intentional redirects. Importantly, the main index page has been indexed at Google with no problems, the difficulty only seems to be with pages that are inside folders.

The error (appears when inspecting url's in Google Search Console and it reads as follows)

- Headline
URL is not on Google : Redirect Errors
It cannot appear in Google search results until it's indexed.

- Further details
Crawl Allowed
Page Fetch : Redirect Error

My guess work

Maybe on this occassion I have entirely screwed up the robots and sitemap but I do not know for sure so here they are for you to check.

I do also suspect that the SSL security certificate may have something to do with this issue but I am struggling to make sense of it as by all accounts the following files are acceptable with zero errors or warnings. There has never been a http version of the site, the errors are appearing even though it's a new upload with https installed prior to dealing with Google.

Please note that in the actual files a real URL is used. I have ommitted it from this post though so as not to spam inadvertently.

Robots.txt (somebody has pointed out that the URLS should not be detailed this way, this is just the latest thing I have tried, more details below).

Sitemap: https://-----.co.uk/sitemap.xml
User-agent: *
Allow: https://-----.co.uk/fitting/
Allow: https://-----.co.uk/carpentry/
Allow: https://-----.co.uk/utilities/
Allow: https://-----.co.uk/contact/
Disallow: https://-----.co.uk/assets/
Disallow: https://-----.co.uk/charity/
Disallow: https://-----.co.uk/coronavirus/
Disallow: https://-----.co.uk/privacy/
Disallow: https://-----.co.uk/dev/
Disallow: https://-----.co.uk/contact/PHPMailer-Master/
Disallow: https://-----.co.uk/contact/contact.php/

sitemap.xml (edited to exclude disallowed folders)

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
        <url>
        <loc>https://-----.co.uk/</loc>
        <lastmod>2022-03-25T12:39:57+00:00</lastmod>
    </url>
    <url>
        <loc>https://-----.co.uk/carpentry</loc>
        <lastmod>2022-03-25T12:39:57+00:00</lastmod>
    </url>
    <url>
        <loc>https://-----.co.uk/contact</loc>
        <lastmod>2022-03-25T12:39:57+00:00</lastmod>
    </url>
    <url>
        <loc>https://-----.co.uk/fitting</loc>
        <lastmod>2022-03-25T12:39:57+00:00</lastmod>
    </url>
    <url>
        <loc>https://-----.co.uk/utilities</loc>
        <lastmod>2022-03-25T12:39:57+00:00</lastmod>
    </url>
</urlset>

So, initially, I looked around Stack Overflow. I found some posts but none seemed relevant.

To keep trying for myself....

Every folder on the site was detailed in the sitemap.xml and the robots.txt but naturally in the latter some folders are detailed as Disallowed. Initially I thought that was how the problem was arising so I removed the disallowed folders from the sitemap leaving only the allowed folders. This has made no difference.

I have also tried detailing pages themselves in both files as opposed to just a folder location. EG, I have tried ----.co.uk/carpentry and I have tried -----.co.uk/carpentry/index.php.

On the topic of being canonical, As you can see full URL's have been used but I have also tried the alternative. EG just detailing the location of the sitemap and using /carpentry and /fitting etc.

With regard to my expected result. Ideally I need Google to actually see there are no redirects. I simply cannot understand why these two files are verfied and the main index page is properly indexed but Google can't find its way into any of my folders.

Completely stuck, no idea what's going wrong. Could really do with moving on.

已采纳解答

原帖链接 · MrWhite · CC BY-SA 4.0

<loc>https://-----.co.uk/contact</loc>

By the look of your robots.txt file, it would seem /contact (and I assume the other "folders") are physical directories on the filesystem. In which case the canonical URL must end in a trailing slash (unless you take additional steps to workaround this). So, the URLs in your sitemap and all associated internal links need to end in a trailing slash.

If you don't include a trailing slash then mod_dir will issue a 301 external redirect to append the trailing slash (to "fix" the URL). This might be the erroneous "redirect" that Google is seeing.

Also make sure you are looking at (and submitting HTTPS sitemaps to) the HTTPS property in GSC, not the HTTP property (which is presumably redirected to HTTPS).


Allow: https://-----.co.uk/fitting/
Allow: https://-----.co.uk/carpentry/
Allow: https://-----.co.uk/utilities/
Allow: https://-----.co.uk/contact/
Disallow: https://-----.co.uk/assets/
Disallow: https://-----.co.uk/charity/
Disallow: https://-----.co.uk/coronavirus/
Disallow: https://-----.co.uk/privacy/
Disallow: https://-----.co.uk/dev/
Disallow: https://-----.co.uk/contact/PHPMailer-Master/
Disallow: https://-----.co.uk/contact/contact.php/

Yes, as you stated, the "the URLs should not be detailed this way". The Allow and Disallow directives take a root-relative URL, starting with a slash. So, in it's current state, your robots.txt file is not doing anything since none of the rules will match (so is not the cause of your immediate problem).

But your 4 Allow directives do not seem to be required here anyway. The default is to allow crawling and there is no conflict with the Disallow directives in what you have posted (which would warrant a need for a more specific Allow directive).

So your robots.txt file should look more like this:

User-agent: *
Disallow: /assets/
Disallow: /charity/
Disallow: /coronavirus/
Disallow: /privacy/
Disallow: /dev/
Disallow: /contact/PHPMailer-Master/
Disallow: /contact/contact.php/

Sitemap: https://example.co.uk/sitemap.xml

Note that the Sitemap directive does take an absolute URL.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2022-03-30)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。