Companies with better websites have a search advantage over brands with dated domains, according to Google’s Matt Cutts. In a recent Webmaster Help Channel video, Cutts said stale sites with older templates can easily be outranked by pages with fancier bells and whistles because they’re more aligned with users’ expectations. However, sites with infinite scroll may not be sending all the right signals to search crawlers.
No crawling = No search presence
A recent Google Webmaster Central Blog explained that pages with infinite scroll (the ones where web content continues to appear as you scroll down) rather than traditional pagination may not consistently be crawled. When Googlebots visit those pages, they can only crawl the links that are already loaded on the screen and will generally only show those in search results. This means brands’ valuable content may be virtually invisible unless a visitor has already landed on the page and loaded all the content.
As the post details, “Crawlers cannot always emulate manual user behavior – like scrolling or clicking a button to load more items – so they don’t always access all individual items in the feed or gallery.”
“Crawlers cannot always emulate manual user behavior – like scrolling or clicking a button to load more items”
Chunk content for easy crawling
To get specific, there should be a “Page 1″ and a “Page 2,” but not a “Page 1.5″ that shows all the content falling between the first and second pages. A crawler might mistake this additional information as duplicated content, the post added.
Confused? Talk to your webmaster
A lot of marketers get lost when it comes to technical website development, and they may need to query their webmasters to make sure infinite scroll pages are optimized for search. If troubleshooting isn’t a problem, they can use the following process to test their website chunks:
There is fine line marketers must walk in today’s search environment between user optimization and designing for search crawlers.
“[Make sure] each component page (the URL) works to take anyone directly to the content and is accessible/referenceable in a browser without the same cookie or user history,” the post advised.
Google points to resources that might help, such as a video by Webmaster Trends Analyst John Mueller. As a whole, this news (combine with Cutts’ recent guidance about updating Web 2.0 designs) implies there is fine line marketers must walk in today’s search environment between user optimization and designing for search crawlers.
SEO isn’t dead, and companies are smart to consider crawlers (and their limitations) when designing pages that will rank.