Duplicate Content – It May Not be so Bad

February 14th, 2008 | 21 comments

Duplicate Content – This is one of those subjects that is the root of thousands of heated discussions. Some say SE’s hate duplicate content, some say SE’s don’t mind it, while others say it all depends.

For the longest time I was completely against duplicate content. I thought that since there is so much scraped content and trash out there in the form of duplicate content it must be bad.

Well, I now have somewhat of a different opinion on the matter.

First, let’s go over a few aspects of Duplicate content.

Duplicate Content that is Absolutely, Positively FINE to Use

Quotes and Snippets: There are some popular quotes that can found on thousands, possibly millions, of pages across the net. Does that mean all of these pages do not deserve exposure? Absolutely not. Quotes and small snippets simply can’t be regarded in a bad way.

Translations: The main reason SE’s don’t like some duplicate content is because they want to return various, UNIQUE pages in their SERPS. If you’re looking for “dog training tips” you want a selection of unique pages to choose from, not a bunch of different pages with the same content. However, since English search queries return English pages (same with all other Languages) there’s no issue with the same version of any pages being translated into other languages.

Google knows this and makes it very clear that both quotes and translations are perfectly OK on their Webmaster Central Blog:

Though we do offer a handy translation utility, our algorithms won’t view the same article written in English and Spanish as duplicate content. Similarly, you shouldn’t worry about occasional snippets (quotes and otherwise) being flagged as duplicate content.

With that being said you’d think there must be some duplicate content that is NOT fine to use, right? Not necessarily!

The Facts on Duplicate Content

As I explained above ,when talking about translated pages, SE’s don’t want to return a bunch of different URL’s for a search query that contains the same exact text because that’s just not good for the user experience. This is true, but that doesn’t mean there are any penalties. As a matter of fact, I’ve heard Vanessa Fox (a former Google Employee,) Matt Cutts (a current Google Employee) and several others clearly state that there is no penalty at all. When they find different URL’s with the same content they just index one version and exclude the other(s.) If you don’t define which page you want indexed (robots.txt and rel=”nofollow”) they’ll do it by looking at the number and quality of links pointing to each page. The version with the most authority will be indexed, while the other(s) won’t.

However, they were referring to the same content appearing on different URL’s of a site and not the same content appearing on different URL’s of DIFFERENT sites!

For this lets take a look at article marketing, since it’s a technique that is driven off of duplicated content. I’m also somewhat of a SMA on the subject who has seen a great deal of success marketing with articles.

So you write an article and publish it to a bunch of different article directories. Now you have alot of links pointing to your site and you’re also getting direct traffic as well. This is true and that’s why so many people use mass submission services to publish their articles to hundreds of different sites.

It’s a good idea to submit these articles to several different sites, but it’s not a good idea to over-do it. Why?

Useless Duplicate Content is Is Naturally Filtered

Search queries are returned based on hundreds, possibly thousands, of different variables within Googles algorithm, but PageRank™ plays the biggest role. (To learn the difference between toolbar PR (useless) and actual Pagerank (Important) see my explanation of Google PageRank and Alexa Traffic Ranking.)

The primary factor behind Pagerank™ is links! Both quantity and quality. The more quality links pointing to any particular page, the more authority that page will aquire and adversely be able to pass. Authority equals higher SERP placement and of course anchor text plays a big role in what search terms pages rank for.

Sites like Digg, EzineArticles etc. are full of duplicate content. But yet they get more traffic than most other sites on the entire Internet. Why? Because they have LOADS of quality links pointing to them.

This is why you’ll notice that an article can be submitted to EzineArticles and 50 other sites, yet 9 times out of 10 the version that appears on EzineArticles will outrank all other versions. Many of the other sites, however, will still get the article indexed on their article directories!

Content that appears on multiple pages within a site is dealt with different that content that appears on multiple URL’s of multiple sites. Many people don’t understand this and that’s why so many people claim that articles that are submitted to multiple sites will only be indexed by one. What it all comes down to it authority!

This is also why, for non-competitive keywords it’s very possible to see search results returning the same content on different URL’s. I’ve seen it several times with my own long-tail articles.

When it comes to article marketing mass submission isn’t going to help you simply because most article directories out there don’t have enough authority to get traffic to your articles.

When it comes to people stealing your content don’t even worry about it. 99% of the time that content will never get traffic because those spammers will never get enough authority to cause that page to get anywhere, while your page has authority. You can’t be punished for idiots stealing your content! However, there are very rare instances of stolen content ranking, although I still don’t believe it hurts the originators rankings.

Wen it comes to your own site content it ultimately doesn’t hurt you too bad. Your “money” pages will most likely be the pages to be indexed while the useless pages will not. To ensure the right pages are indexed be sure to use robots.txt and a good internal linking structure. If you’re running a WordPress blog there are many ways to prevent duplicate pages including redirects and installing plugins.

photo credits: bogdan.glushak

Related Posts

21 comments

Eartha (1 comments.)
14th February, 2008 at 1:26 am

Great post Josh. I wondered how stolen content was dealt with because it happens to me quite a bit. It’s good to know the spammer pages are nothing to really worry about.
Caroline Middlebrook
14th February, 2008 at 10:51 am

This is something that interests me a lot and confused the heck out of me when I worked on my bum marketing project a few months back. So, lets say you are using some article marketing to help with a brand new site that is still unranked, how many directories is it worth submitting to? I mean, if there are a few smaller ones with PR2-3 is it worth submitting to those just for the link juice?

I dont do article marketing for my blog but I’ve considered it but would the strategy be different for that as its a PR5? I’m still quite clueless about the relevance of PageRank.
Josh Spaulding
14th February, 2008 at 1:45 pm

@ Eartha, Thanks and yep, it’s annoying, but it normally isn’t anything to worry about.

@ Caroline, Good question. A couple years ago I started doing alot of testing with articles. What I found to be true then was that mass submission was working.

Then about a year ago I started testing again over the course of a few months. What I found to be true then was that mass submission wasn’t working nearly as well, so I started submitting to only the top 5 or so and I was seeing good results.

Then a few months ago I started testing the mass submission waters again and I noticed that when I submitted articles to 30 or so high-quality article directories they were getting indexed increasing my incoming links, but when submitting to only the top 5 or 10 I was getting more traffic.

So, right now I’m submitting half my articles to the top 5-10 article directories and the other half to about 30-50 quality article directories.

I would say not to even bother submitting articles to promote your blog, as it’s doing very well and growing quickly as it is, but it’s always hard to get links with your chosen keywords in them, so article marketing is a great way to do that and get your blog ranked for the keywords you want it to rank for.

For a new sites it’s definitely worth it!!!
Josh Spaulding
14th February, 2008 at 1:46 pm

@ Caroline, for a PageRank Explination see http://www.ethicalim.com/internet-marketing/google-pagerank-and-alexa-ranking-explained/
Caroline Middlebrook
14th February, 2008 at 1:51 pm

@Josh, yeah that’s pretty much what I was thinking. For the blog I’d much rather invest that kind of time and effort into writing blog posts but for a little niche site with no PR I’d do more.
JoLynn Braley (1 comments.)
15th February, 2008 at 12:31 am

Hi Josh,

Thanks for the details on duplicate content, especially the point you made about why the content thieves won’t outrank your original content. I still don’t agree with sploggers stealing content but you helped me change my attitude a bit about the entire topic, which can only help me in the long run.
Elliott (7 comments.)
15th February, 2008 at 4:43 am

Josh, great article. To clarify however, you are saying that it is duplicate content within the same site that draws the penalty, not duplicate content from seperate sites?

Would this mean that you really need to clamp down on the indexing by Google and such for your archives and tags/categories that show up as “duplicate” content then?
Josh Spaulding
15th February, 2008 at 12:56 pm

@ Caroline – yep, article marketing is great for giving newer sites a jump start, but sites with alot of established authority like yours, it’s not going to do you much good.

@ JoLynn – Sploggers are pathetic, but unfortunately it’s not worth it to stress. Many of them will eventually learn that it’s not a good business model. In the mean time I just ignore them.

@ Elliott – There is no “penalty” either way. The same exact content on two different URLs within the same site could cause only one of those versions to be indexed in Google. I suppose you could call it a penalty, but it’s really not.

A good internal linking structure is very important though.
Best Posts for the Week of February 11th, 2008
17th February, 2008 at 10:18 pm

[...] StumbleUpon Combo to Benefit the Blogosphere. Guide to Link Cloaking, Masking, and Url Redirection. Duplicate Content – It May Not be so Bad. Internet Home Based Business Advice. The SEO Sins No Designer Should Ever [...]
DeathMitten (1 comments.)
20th February, 2008 at 7:39 am

I have to thank you, man.

I’m planning on making a money-making site of my own, soon, and I’m doing research for that.

That post was extremely helpful.

Thank you so much, again.
Stephen Cronin (34 comments.)
20th February, 2008 at 8:12 am

Hi Josh,

This is food for thought indeed!

I’ve got to admit I got a bit caught up in the duplicate content things, just going by what others were saying. Over the last few months I’ve begun to wonder if it’s really such an issue. So this post comes at a good time – what you’re saying makes sense to me.

It’s interesting that some people are still making a big deal about it – I’m thinking of Courtney Tuttle and Vic who together run Blogging Zoom. They’ve made a rule that people can’t copy and paste the description of posts submitted to Blogging Zoom, because they say it will hurt the bloggers with Google’s duplicate content filter. Both those guys seems to know what they’re doing, but I thought it was a little odd.

So, not sure what to think – food for thought as I say – but I’m not going to worry too much about it personally.
Josh Spaulding
20th February, 2008 at 1:57 pm

Hi Death,

Good deal, Best of luck with your new site.

Hi Stephen,

Well, most people go off of what people tell them, so you’re definitely not alone on that one

I guess Court should possibly publish some results of a case study or something to prove his theory? I’d love to see that because there is no duplicate content “filter.”
Stephen Cronin (34 comments.)
21st February, 2008 at 10:07 am

I’d like to see Court publish some results too – the only thing I can think of is that they have a minimum of 350 characters for the description, so maybe that puts it above the limit for snippet rule or something (and “filter” may have been my wording).

Anyway, what you say above makes sense to me and you back it up. I’m happy to go with that!

It doesn’t affect me too much except for the scrapers as I use the more tag (so I don’t have full posts on my category pages etc).
Carla @ WordPlay
22nd February, 2008 at 12:59 am

Thanks so much for this. I’ve been looking for an explanation about the whole article marketing/duplicate content thing for weeks.

I’ve been submitting articles to ezinearticles.com for about a month to promote a new niche blog. I was reticent to branch out to other article directories, because I wasn’t sure about the duplicate content penalty. I had a feeling it wasn’t really a factor, even though some people make a lot of noise about it. After all, why on earth would so many smart people use article marketing to good effect if it was just going to bring grief?

Well, now I know the real scoop and can go on to worrying about something else!

Thanks again.

(BTW, I just read your Easy SEO e-book and got the link to this post from there. The e-book was very helpful.)
Josh Spaulding
22nd February, 2008 at 4:22 pm

Hi Carla,

yep, it’s ok to broaden your submissions, just don’t get too crazy with it.

Thanks for the compliments! I’m glad you enjoyed the report
Mario (1 comments.)
24th February, 2008 at 2:25 am

Nice article Josh,
I have a question you may be able to answer, does it hurt my website to use the ezine articles I published on my website article page? I haven’t done it because I have been afraid of hurting my website using duplicate content, what are your feelings.
Thanks again
Josh Spaulding
24th February, 2008 at 5:48 pm

Hi Mario,

Good question. It doesn’t “hurt” your pages, but why compete against yourself?

Sure, it brings in a few extra links and a little more exposure, but now your stepping on your own feet with pages from article directories competing against your own pages.

If your site has a good amount of authority, then it wouldn’t be a problem. But if your site is relatively new and/or doesn’t have a whole lot of authority there’s a good chance those articles will be competing with your pages and theres a very good chance those article pages will rise above your own in the SERPS.
Batten Down the Hatches | The BookmarkMoney Blog
26th February, 2008 at 3:26 am

[...] BlogsRating.com – Like Hot or Not for Bloggers DotCom Mogul: These WP Plugins are the Shizzle Josh Spaulding: Duplicate Content – It May Not Be So Bad NoFollow Free WP Plugin Updated – About two days after I installed it. 5 Unconventional Ways to [...]
Victor
7th March, 2008 at 3:01 pm

Great post Josh. I was kind of aware of the issue, but you wrote it in a way that was easier to understand and helped clarify some things for me. It still comes down to lots of good backlinks.
Josh Spaulding
7th March, 2008 at 3:12 pm

Glad you enjoyed in, Victor
Karen (2 comments.)
20th June, 2010 at 8:41 pm

The best explanation of duplicate content I’ve come across. Excellent!