Indexing
In order for web pages to be included within search results, they must be in Google’s index. Search engine indexing is a complex topic and is dependent on a number of different factors. Our SEO Office Hours Notes on indexing cover a range of best practices and compile indexability advice Google has released in their Office Hours sessions to help ensure your website’s important pages are indexed by search engines.
Googlebot No Longer Needs to Convert Hashbang URLs into Escaped Fragments
Googlebot no longer converts hashbang URLs into escaped fragments as it is able to render and index them directly rather than using the pre-rendered version specified with the escaped fragment. Therefore, John would recommend moving to something that’s URL-based rather than hashtag-based.
The Indexing Issue From Last Week Has Been Resolved
The indexing issue seen last week related to Google having issues indexing new content has now been resolved. If sites are still experiencing issues with the indexing of their content this won’t be related to the previous issue.
Resource Loading Errors Within Mobile Friendly Test and GSC Don’t Reflect Indexing Errors
Within the Mobile Friendly Test and Inspect URL Tool, Google fetches all of the page content and embedded URLs including images, fonts and JavaScript, which can sometimes cause timeout issues. However, this shouldn’t affect the indexing of the content for search results due to caching and the use of older versions of files.
URL Removal Tool in GSC is Fastest Way to Remove a Test Site From Search Results
While there are several ways to remove a staging site from organic search results, including blocking Googlebot from crawling it or returning 404 or 410 error codes, John recommends using the URL Removal tool in GSC to remove it quickly.
Speed Metrics Important for UX Are Different to Metrics Important for Crawling & Indexing
While there is some overlap, speed metrics that are important for UX are completely different to those used for crawling and indexing. For the latter, Google needs to be able to request the HTML pages as quickly as possible and server response time is also important.
Disrupted Index Coverage Report Data in April Due to Indexing Bug
Google’s indexing issues occurred from 9th-25th April and data in the Index Coverage report could not be shown during this time. John hopes to write a post clarifying how Google handles these types of situations internally.
Google Being as Transparent as Possible About Visible Indexing Bugs
Google is being as transparent as possible about their recent bugs, because they have had visible impacts on indexing. It is important for John and the team to keep everyone updated on these issues so that webmasters don’t take actions that could be unnecessary or detrimental to their websites.
Google Doesn’t Differentiate Between UGC & Other Content
Google doesn’t differentiate between UGC and the other content on a website, so it is important to control how low-quality UGC is dealt with.
Indexing API Currently Limited to Jobs & Live Video Content in Limited Countries
The Indexing API is currently limited to jobs and live video content and has only been rolled out to a limited number of countries.
Sites Should No Longer be Impacted by Google Deindexing Bug
Google’s deindexing issue was fully fixed on 16th April. If pages still aren’t being indexed, then this will have been caused by a different issue.