New and improved crawl stats are now available for your site.

November 24, 2020

To help website owners better understand how Googlebot crawls their sites, Google is launching a brand new version of the Crawl stats report in Search Console.

The new Crawl Stats report brings the following exciting new features:

  1. Total number of requests grouped by response code, crawled file type, crawl purpose, and Googlebot type.
  2. Detailed information on host status
  3. URL examples to show where in your site requests occurred
  4. Comprehensive summary for properties with multiple hosts and support for domain properties

Over-time charts

The Crawl Stats report enables website owners to see Google crawl data totals and overtime charts for: total requests, total download size and average response time.

Search Console Crawl Stats over-time charts

Grouped crawl data

The new version of the report also provides data on crawl requests broken down by response, file type of the fetched URL, purpose of the crawl request, and Googlebot agent. See example URLs of each type by clicking on a row in the grouping table:

The Crawl stats report provides high level and detailed information on host status issues to help users understand how Googlebot crawls their site. The host status details let users check the site's general availability to Google in the last 90 days, while the file types and file sizes returned by the site are also available to view. In addition, users can track their site's availability issues in the host status view. The new data is designed to be useful and actionable for users, and questions or comments on the report can be directed to the Search Central help community or mentioned on Twitter. The Crawl stats report documentation has more detailed information on what users can do with the new report.

Hillel Maoz, Engineering lead for Search Console, recently posted about some new features coming to the Google Search Console. This includes an updated version of the URL inspection tool, as well as a new Rich Results testing tool.

The updated URL inspection tool will now show whether a particular page is eligible for rich results. This includes information on structured data, AMP pages, and whether the page has any errors that might prevent it from being eligible for rich results.

The new Rich Results testing tool allows you to test how your pages will appear in search results, including rich results. This can be helpful in troubleshooting why your pages are not appearing as you expect them to.

These new features are expected to launch in the next few weeks.

Google Webmasters is no more, hello Google Search Central
According to Merriam-Webster, the first known use of the word "webmaster" was in 1993. However, the term is becoming archaic, and according to data found in books, its use is in sharp declin...
Read More
Updates to Search Console’s API
A few months ago, Google announced an API infrastructure upgrade to improve the performance of the Search Console API as demand grows. Today, Google is announcing some more updates to the API, includi...
Read More