Conflicting Website Signals & Confused Search Engines - Rachel Costello, Technical SEO at DeepCrawl

CONFLICTING WEBSITE
SIGNALS & CONFUSED
SEARCH ENGINES
Rachel Costello, Technical SEO
SearchLeeds
@rachellcostello SearchLeeds

When it comes to selecting URLs, Google
often assumes you are wrong.

TECHNICAL IMPLEMENTATION
Not all website owners have
knowledge of SEO.
Imagine what their technical
implementation looks like.

Google uses an algorithm for URL selection
to make sense of all the chaos out there.

If you sit back and do nothing, Google will
do its best to figure out canonicals for you.

TAKE BACK CONTROL
BUT the algorithm isn’t perfect,
do you really want to be entirely
at its mercy?

The algorithm’s results will be better if you
do your part and use consistent signals.

Source: https://twitter.com/JohnMu/status/1003631027494014976

WHAT WE’LL COVER
1. How Google handles website signals
and selects URLs
2. How you can test the signals your
website is sending
3. How to see Google’s interpretation of
signals and what it chooses to show

*DISCLAIMER*
To anyone who’s not a fan of
Google’s John Mueller: this talk
might not be for you.
He will be referenced repeatedly.

HOW GOOGLE HANDLES
WEBSITE SIGNALS

301 REDIRECTION DOESN'T ENSURE THE FINAL
DESTINATION URL WILL BE INDEXED & SHOWN
A 301 redirect doesn’t mean that the final destination URL will be the page
that is favoured. You need to make sure all other signals are consistent.
John Mueller, Google Webmaster Trends Analyst
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-january-9th-2018/#link10

The canonical tag is a signal, not a directive.

Things to note about canonicalization...

Any unique content on a page with an
accepted canonical tag will be ignored.

Pages have to be very similar, if not
identical, for canonicalisation to work.

Does Google process canonical tags in the
rendered version or the raw HTML?

Source: https://twitter.com/rebelytics/status/1003951913510014976

If the <head> section prematurely ends,
Googlebot may not read its contents.

For more information on this, read
http://ohgm.co.uk/breaking-head-quietly/

Canonicals
SIGNALSSitemaps Parameter
Handling
Internal
Linking
BacklinksRedirects
The signals
Google uses for
URL selection.

The end goal for all of this is to
avoid content duplication.

MANY TO ONE CANONICAL TAGS MAY BE IGNORED
Google will try to follow canonical directives by default, but they ignore
canonical tags if they think it’s a mistake i.e. if there are significant content
differences, or if a lot of URLs canonicalise to the same page.
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-thursday-17th-may/

UPDATE INTERNAL LINKS TO CANONICAL URLS
Update internal links to canonicals to give Google a clean signal about which URL to index. If the
canonical is accepted, any links to canonicalised URLs will be associated with the canonical URL.
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-october-6th-2017/#link8
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-august-25th-2017/#link8
ALIGN LINKING & REL CANONICAL TO GET A PARTICULAR PAGE INDEXED
Ensure internal links and rel canonical are pointing to the preferred page for indexing to ensure
you aren’t giving Google conflicting signals.

Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-may-15th-2018/#link14
PARAMETER HANDLING SIGNALS ARE STRONGER THAN
THE CANONICAL SIGNAL
Google won’t blindly follow URL parameter handling set in GSC, but John
says that this is a stronger signal than canonicalization.

CANONICALS ARE CHOSEN BY GOOGLE USING XML SITEMAP URLS
XML sitemap URLs are used to help inform Google’s decision on which URL is chosen to be the
canonical.
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-november-14th-2017/#link12
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-july-7th-2017-2/#link11
URLS IN SITEMAPS ARE NOT GUARANTEED TO BE INDEXED
Google may choose not to index URLs in sitemaps that are very similar to ones already indexed or
if they differ to the ones linked internally.
INTERNAL & SITEMAP LINKS MAY OVERRIDE CANONICAL TAGS
Canonicalized pages may still be chosen to be shown by Google if you link to them internally and
in sitemaps.

With website signals, there’s strength in
numbers.

Mixed signals waste Google’s time, your
site’s crawl budget and also delay indexing.

CANONICALIZING TO REDIRECTS CAN INCREASE TIME
TAKEN TO INDEX PREFERRED VERSION
Canonicalizing to redirects can increase the time taken for Google to
decide which version to index, especially if these pages aren’t identical.
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-january-23rd-2018/#link5

GOOGLE & WEBSITE SIGNALS
To summarise:
1. Google uses the following signals: canonicals,
internal linking, parameter handling, backlinks,
redirects and sitemaps
2. These signals are strongest when used in
combination
3. The purpose of these signals is to avoid content
duplication

HOW TO TEST YOUR
WEBSITE’S SIGNALS

KEY ELEMENTS TO TEST
1. Internal linking
2. Redirects
3. Noindex
4. Sitemap inclusion
5. Disallow

Check for inconsistencies. Are there any
signals that don’t quite match up?

Testing website signals with DeepCrawl.

Examine all canonicalized pages

Examine links in to canonical pages

Examine canonical pages with no links in

Examine redirecting internally linked URLs

Examine canonical tags to non-200 URLs

Examine canonical URL sitemap inclusion

Examine non-indexable pages being shown

DISALLOWED URLS MAY SHOW IN SEARCH DUE TO INTERNAL LINKING
If you’re seeing disallowed URLs showing up in search results, it may be because of internal
linking to these pages.
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-april-4th-2017/#link2
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-november-4th-2016/
GOOGLE MAY CHOOSE A REDIRECTED URL INSTEAD OF THE TARGET
Even in the case of a redirect, Google might still choose to index the redirect source instead of the
target if it is judged to be the canonical.

DON'T USE NOINDEX ON CANONICAL PAGES
Don’t use a noindex on canonical pages because if Google accidentally picks a noindexed page
as the canonical, nothing will show up in search results.
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-september-9th-2016/
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-august-11th-2017/#link1
GOOGLE TAKES CANONICAL TO BE A MISTAKE IF PRESENT WITH NOINDEX
If a canonical and noindex are both used, Google judges the rel canonical to be a mistake and the
noindex as being used to force a canonical.

Nothing appearing in search - the biggest
canonicalisation disaster you can get!

Make sure your site’s signals are aligned
and pointing in the same direction.

TESTING YOUR WEBSITE SIGNALS
To summarise:
1. Examine the following areas: internal linking,
redirects, noindex, sitemap inclusion, disallow
2. Check each of these areas for inconsistent URL
focuses
3. Having aligned signals is the key to getting
content indexed by Google

HOW TO TEST GOOGLE’S
URL SELECTION

YOU CAN SEE WHICH CANONICAL GOOGLE HAS
CHOSEN IN THE NEW SEARCH CONSOLE
In the new Search Console you can select individual URLs which takes
you to the info query showing you the canonical that Google has chosen.

To Google Search Console we go!

The key reports in Google Search Console
for canonicalization analysis...

Google thinks these are duplicate pages
despite you marking them as canonical.

GSC REPORT MEANING
Google chose different canonical
than user
“This page is marked as canonical, but
Google thinks another URL makes a
better canonical which has been
indexed instead.”
@rachellcostello SearchLeeds Source: https://support.google.com/webmasters/answer/7440203

Google thinks these are also duplicate
pages.

GSC REPORT MEANING
Alternate page with proper
canonical tag
“This page is a duplicate of a page that
Google recognizes as canonical. This
page correctly points to the canonical
page, so there is nothing for you to do.”

Google thinks another page is a better
choice to canonicalize duplicates to.

GSC REPORT MEANING
Submitted URL not selected as
canonical
“The URL is one of a set of duplicates
without an explicitly marked canonical
page. Google did not index this URL.
Instead, we indexed the canonical that
we selected.”
Source:
https://support.google.com/webmasters/answer/7440203?authuser=0#submitted_url_not_selected_as_c
anonical

Google thinks these are part of duplicate
page sets which don’t have a canonical.

GSC REPORT MEANING
Duplicate page without canonical
tag
“This page has duplicates, none of
which is marked canonical. We think
this page is not the canonical one. You
should explicitly mark the canonical for
this page.”

Go to Google to see the results...

CHECKING RESULTS IN GOOGLE
The best way to see Google’s
URL selection for yourself is
through the
info query and cache.

Info: query
Cache

USE THE INFO QUERY IN GOOGLE TO FIND CANONICAL URLS
Search for a URL with an “info:” query to see if a different URL has been indexed instead.
Source: https://www.deepcrawl.com/blog/news/google-webmaster-hangout-notes-april-4th-2017/#link4
INCORRECT GOOGLE CACHING SUGGESTS A DIFFERENT CANONICAL HAS
BEEN CHOSEN
If the cached content from a different page is displayed, this could be because Google has
determined duplication and has canonicalised one.

TESTING GOOGLE’S SELECTION
To summarise:
1. Despite your website signal implementation,
Google has the final say in what is indexed
2. Check Google Search Console canonicalization
reports
3. See the live results in the SERPs by checking
what appears for “info:” queries and caching

All you can do is give aligned signals to
make Google’s job easier.

The rest is in Google’s hands.

TO SUM UP
Understand Google’s URL selection algorithm.
- Canonicalization, internal linking, parameter
handling, sitemaps, redirects, backlinks
Monitor your site for conflicting signals.
- Internal linking, redirects, noindex, sitemaps, disallow
Analyse Google’s interpretation of signals.
- Google Search Console, SERP “info:” queries and
caching

THANK YOU!
Any questions? Send me a tweet:
Rachel Costello
Technical SEO, DeepCrawl
deepcrawl.com @DeepCrawl

Conflicting Website Signals & Confused Search Engines - Rachel Costello, Technical SEO at DeepCrawl

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Conflicting Website Signals & Confused Search Engines - Rachel Costello, Technical SEO at DeepCrawl

Similar to Conflicting Website Signals & Confused Search Engines - Rachel Costello, Technical SEO at DeepCrawl (20)

Recently uploaded

Recently uploaded (20)

Conflicting Website Signals & Confused Search Engines - Rachel Costello, Technical SEO at DeepCrawl