Today: Duplicate content by Derek Mabie.
Duplicate content is every webmaster's nightmare.
It's exactly what it sounds like: A portion of text that exists on multiple URLs, making it difficult for search engines to determine how to rank them in order of importance.
Google will ultimately choose one URL to show in the results page - but it may not be the page you want to be exposed.
The bigger issue, though, is Google may prematurely give up on crawling your site if it keeps encountering instances of duplicate content. No one likes doing the same job multiple times, and Google is no exception.
How Does Content Get Duplicated?
It seems as though this would be a pretty easy fix. All you have to do is refrain from copying and pasting words into another page or post on your site, right? Not exactly. Many times, duplicate content isn't intended to manipulate the search engines. For example, printer-friendly versions of pages are duplicate content. These cases are not confusing to users who understand the purpose - but to the search bots, you are wreaking havoc!
We recently discovered a client had identical content on two URLs that read like this:
The only difference between the two URLs is one word is capitalized. To search engines, these two pages are completely separate from one another and thus, are competing to rank in the search engine results page (SERPs).
Avoid Duplicate Content
Class, repeat after me: ca-non-ih-cole-eye-zay-shun. Canonicalization, easier done than pronounced, is the act of conveying to search engines which single version of the duplicate content rules supreme. This means one URL will be indexed and the identical versions won't compete to outrank each other in the results pages.
1. 301 Redirect
Sometimes pages serve different purposes, but contain similar chunks of content. An example of this would be inserting an "Our Philosophy" paragraph into every footer of your site's many pages. Other times, multiple pages will serve the exact purpose, the only difference being the web page addresses (URLs). This conundrum is common during website redesigns, but it can be resolved by assigning 301-redirects to the URLs. By doing so, you can guide users and search engines to the page you actually want them to visit.
Home pages often require 301-redirects:
These pages are all identical, but they are seen as completely separate entities to search engines. This can be a mess if inbound links are distributed to each of these versions. The value from the links will be spread thin instead of adding up to boost domain authority. Adding a 301-redirect, thus identifying one URL as the master, will harness the authority and increase the potency of link juice from outside sources.
301-redirects serve two purposes: They can essentially delete the old URL (if the site is being redesigned) or they consolidate multiple versions (such as the home page examples shown above).
Rel=canonical is the name of a link that can be inserted into the header of the non-preferred pages.
The URL you do want Google to use as the preferred version of content will go in place of www.example.com/best-url, ensuring that every duplicate piece of content will then internally link to the correct version. Search engines appreciate this clarity.
What if Other Sites are Using My Content?
Duplicate content is really only an issue when it exists on your own domain. Other websites may use your articles as syndicated content - and that is just fine. In fact, Google has stated that as long as the syndicated material links back to the original source, it will not interfere with your site's ability to rank.
To play extra safe, we recommend asking those who use specific content to include a "noindex" meta tag in the header of that page. This communicates to search engine spiders that the page is not intended to be included in the index of pages.
Hopefully you have a clearer understanding of what duplicate content is, the harm it can cause, and what measures you can take to canonicalize the crap out of it.
The beauty of practicing "white hat" SEO is that your refinements are done according to the search engines' recommendations. As their algorithms improve to better reflect the needs of users, your website will in turn transform into a user-friendly domain.
Derek Mabie is president of his St. Louis-based digital marketing agency, Evolve Digital Labs. His "white hat" SEO efforts allow him to empower brands through search. You can follow him on Twitter at derekmabie. Download Evolve's SEO Guide for Beginners to learn more about the industry.