Deepcrawl is now Lumar. Read more.
DeepcrawlはLumarになりました。 詳細はこちら

Indexing

In order for web pages to be included within search results, they must be in Google’s index. Search engine indexing is a complex topic and is dependent on a number of different factors. Our SEO Office Hours Notes on indexing cover a range of best practices and compile indexability advice Google has released in their Office Hours sessions to help ensure your website’s important pages are indexed by search engines.

Dynamic Rendering Can be Used to Show Googlebot Fully Rendered Pages

You can use dynamic rendering to serve Googlebot with pages that are already fully rendered, meaning there won’t be a gap between the initial indexing and rendering.

4 Sep 2018

There Will Continue to be a Delay Between Indexing & Rendering Due to Resource Issues

John explained that for the foreseeable future there will continue to be a delay between initial indexing of HTML and rendering, because JavaScript requires resource to be rendered and this can’t happen immediately with the current system.

4 Sep 2018

Google Have Removed Their Public URL Submission Feature

Google’s public URL submission tool has been removed but URLs can still be submitted through Search Console and sitemaps.

27 Jul 2018

Robots.txt Files Don’t Need to be Indexed by Google

Robots.txt files need to be machine-readable, but they don’t need to be indexable for Google to be able to process them.

24 Jul 2018

Google Doesn’t Have Separate Way of Prioritising Pages For Rendering

Google doesn’t have a separate way of prioritising the order of pages to be rendered in the second wave of indexing that differs to the way sites are prioritised for regular crawling and indexing.

13 Jul 2018

Google’s Indexing Systems Are More Patient With Rendering Than Live Testing Tools

When using Google’s testing tools for JavaScript rendering issues, you won’t get a truly accurate view of how Googlebot is rendering and indexing as the tools have a much stricter timeout limit to give webmasters quick results.

10 Jul 2018

Google Treats Meta Refresh as Redirect Meaning Wrong Content Might be Indexed

Google treats meta refresh as a redirect, which may mean the wrong page is indexed e.g. a product listing page with a meta refresh to a payment page will mean the latter is indexed rather than the actual content.

29 Jun 2018

The Signals You Provide Via JavaScript Shouldn’t Conflict with the Ones in HTML

The signals you give Google via JavaScript shouldn’t contradict the ones you’ve provided in the HTML. For example, if you add a follow link in the HTML but use JavaScript to inject a nofollow tag, it will be too late as the signals will be passed through this link with the first wave of indexing.

26 Jun 2018

It Will Take Years to Switch All Sites Over to Mobile-first Indexing

John confirmed that it could take years to switch all sites to mobile-first indexing because there are many sites that aren’t ready yet. Google is assessing how to best provide more information on how to help people make their sites mobile-first ready.

26 Jun 2018

Different Signals Determine Google’s Canonical Selection

John confirmed that rel canonical, redirects, internal linking, URL parameters and sitemaps are all signals Google uses to decide which page is the canonical from a group of pages that have been folded together.

26 Jun 2018

Back 8/21 Next