Google Crawl stats to low

#1
I switched ubuntu from using apache + php 7.4 to OpenLiteSpeed 1.7.14 + lsphp73
I have a website with approximately 4-5 million url (wordpress cms)
Before I switched to openlitespeed , google sent over 300k requests every day to crawl my site, and now it sends a maximum 10k-15k requests!
I have multiple sitemaps
example.com/old1/sitemap.xml
example.com/old2/sitemap.xml
example.com/old9/sitemap.xml
example.com/sport/sitemap.xml
example.com/archaeology/sitemap.xml
etc etc ..
I did not make any changes other than transfer from apache to openlitespeed.
 

Attachments

Last edited:
#2
That’s a huge drop , definitely not normal if you only switched the web server. A few things are worth checking:
  1. robots.txt or redirects – Make sure OpenLiteSpeed isn’t serving a cached or incorrect robots.txt file that’s blocking Googlebot. Also confirm there are no misconfigured redirects (like to HTTPS or trailing slashes).
  2. Response codes – Check your access logs for Googlebot. If you see lots of 403, 404, or 500 responses, that’s likely the reason.
  3. Performance settings – OpenLiteSpeed caching or rewrite rules might be interfering with sitemap or page access. Verify your .htaccess rules transferred properly to vhconf.conf.
  4. HTTP headers – Ensure your pages and sitemaps return 200 OK and not noindex or similar directives.
  5. Search Console – Re-submit your sitemaps and check for crawl errors or coverage drops.
 
Top