Learn about Centmin Mod LEMP Stack today
Become a Member

Google Search Console reporting 5xx error in /wp-includes folder

Discussion in 'Blogs & CMS usage' started by Keroteta, Aug 8, 2019.

  1. Keroteta

    Keroteta New Member

    12
    0
    1
    Aug 2, 2019
    Ratings:
    +0
    Local Time:
    10:20 PM
    nginx/1.17.2 (280719-043529-centos7-kvm)
    10.3.17-MariaDB
    Hi,

    I just received an email from Google Search Console reporting that there is a lot of crawling errors on my Wordpress site. When I go to Google Search Console's Coverage section I find that there is a lot of urls that are throwing a 5xx error. All of those urls are like:

    https://www.domain.com/wp-includes/...../somefilename.php

    i don´t know why the Google mobile crawler needs to crawl this folder, but it wants to do it because I tried to Disallow in robots.txt just to see if it fixed the 5xx error but then I receive messages saying that I don´t have to Disallow because it's crawler needs to see those files...

    So, Why in first place the Google crawler is viewing those 5xx error and how can I fix it so it can access to them??


    Note: The website is working perfectly is just the Google Crawler who protests!
     
  2. eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    6:20 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    That is strange, how was wordpress site installed ? via centmin.sh menu option 22 wordpress installer or via menu option 2 for regular nginx site setup and then installed wordpress manually ?

    Or was it a wordpress site migrated/moved from existing site to a centmin mod created wordpress site install ?

    Are you using wordpress plugins for SEO ? make sure they're properly configured or you maybe telling search engines to crawl urls which are not intended to be crawled

    what is the exact full path and file name of the filenames minus your domain name for privacy ?

    you can use CODE bbcode tags for easier formatting for the list if you want
     
  3. Keroteta

    Keroteta New Member

    12
    0
    1
    Aug 2, 2019
    Ratings:
    +0
    Local Time:
    10:20 PM
    nginx/1.17.2 (280719-043529-centos7-kvm)
    10.3.17-MariaDB
    It was moved from existing site located on a LAMP VPS to a centmin mod server. To do so I use Duplicator Wordpress plugin to create a backup of the original site and then I used centmin.sh option 22 to create a brand new Wordpress installation on the new server. After that I just removed the files on the public folder of the new site and restored the original site there with Duplicator's files. Basically it puts there the original files and recreates the database. The site is working fine from a user perspective, can´t find any problem, but Google crawler is throwing those warnings. Also mention that I have moved other 6 sites and this is the only one with this problem.

    I'm using YOAST Seo plugin, but I can´t see any misconfiguration on it. In fact is exactly configured as in the other sites where there is not this crawling issue.

    Actually they are these 59 paths...
    Code:
    https://www.domain.com/wp-includes/Requests/Exception/HTTP.php
    https://www.domain.com/wp-includes/Requests/Exception/Transport.php
    https://www.domain.com/wp-includes/Requests/Hooks.php
    https://www.domain.com/wp-includes/Requests/Response/Headers.php
    https://www.domain.com/wp-includes/IXR/class-IXR-clientmulticall.php
    https://www.domain.com/wp-includes/IXR/class-IXR-introspectionserver.php
    https://www.domain.com/wp-includes/SimplePie/Core.php
    https://www.domain.com/wp-includes/Requests/Exception/Transport/cURL.php
    https://www.domain.com/wp-includes/SimplePie/Cache/MySQL.php
    https://www.domain.com/wp-includes/SimplePie/Cache/File.php
    https://www.domain.com/wp-includes/SimplePie/Cache/DB.php
    https://www.domain.com/wp-includes/SimplePie/Cache/Memcache.php
    https://www.domain.com/wp-includes/ID3/module.audio-video.riff.php
    https://www.domain.com/wp-includes/ID3/module.audio-video.flv.php
    https://www.domain.com/wp-includes/ID3/module.tag.id3v2.php
    https://www.domain.com/wp-includes/ID3/module.audio-video.matroska.php
    https://www.domain.com/wp-includes/ID3/module.tag.id3v1.php
    https://www.domain.com/wp-includes/ID3/module.tag.lyrics3.php
    https://www.domain.com/wp-includes/ID3/module.tag.apetag.php
    https://www.domain.com/wp-includes/ID3/module.audio-video.quicktime.php
    https://www.domain.com/wp-includes/ID3/module.audio-video.asf.php
    https://www.domain.com/wp-includes/ID3/module.audio.mp3.php
    https://www.domain.com/wp-includes/ID3/module.audio.ac3.php
    https://www.domain.com/wp-includes/ID3/module.audio.dts.php
    https://www.domain.com/wp-includes/ID3/module.audio.flac.php,
    https://www.domain.com/wp-includes/ID3/module.audio.ogg.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/504.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/500.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/414.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/Unknown.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/401.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/501.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/404.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/413.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/415.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/410.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/503.php,
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/306.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/405.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/304.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/418.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/416.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/511.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/407.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/412.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/402.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/417.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/409.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/408.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/411.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/305.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/428.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/403.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/505.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/502.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/400.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/429.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/431.php
    https://www.domain.com/wp-includes/Requests/Exception/HTTP/406.php
     
  4. eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    6:20 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    are those valid path/files that exist ? those don't look like wordpress default files ? and certainly not at wp-includes directory location. If they're files that exist but you didn't put them there, then it could be your wordpress site was compromised on previous server prior to migration to centmin mod - malware or some kind ?

    edit: looks like SOME of those are wordpress files while others aren't - just checked so no idea why search engines crawling them - you can use google crawl site manually or url inspection of the url to see what google reports
     
  5. Keroteta

    Keroteta New Member

    12
    0
    1
    Aug 2, 2019
    Ratings:
    +0
    Local Time:
    10:20 PM
    nginx/1.17.2 (280719-043529-centos7-kvm)
    10.3.17-MariaDB
    I just downloaded latest version of Wordpress from https://wordpress.org/ and checked the paths and files and they are all identical to mine in /wp-includes/ . Also Sucuri plugin and Sucuri site check here https://sitecheck.sucuri.net doesnt finde any malware or security issue. So I'm scratching my head to guess why Google is trying to crawl those files in this site and not in the others.

    I'm attaching a file with the output I get when I try to crawl any of those urls from Google Search Console.
    GSearchConsoleOutput.jpg
    For some reason Google knows those files exists even when that folder is protected and forbidden and tries to crawl them. I don´t know if it is really important o I should leave it as is and forget about it.
     
  6. eva2000

    eva2000 Administrator Staff Member

    53,142
    12,108
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,643
    Local Time:
    6:20 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    weird then, no idea