A lot of us do a basic form of keyword research of one kind or another, using tools like Google’s to learn what search phrases are relevant to our topic and how often those phrases are searched.
By using the words real people use to search our topic, we have a better chance of getting search engines to send us those visitors. Obvious, right? Keyword research is all about finding a common language with our readers.
Most keyword researchers also know that if you want search traffic, you have to keep competition in mind. That is not the “competition” column listed in Google’s keyword tool, which is really designed for advertisers to bid on certain words and have their ads show up. Google’s keyword tool is talking about competing advertisers.
We don’t care about that. What we do care about is how many other webpages rank for the keywords we’re aiming for. The more competing pages there are, the more likely that their pages appear above ours in search results, burying our page back on page 3 where no one ever looks.
There’s probably tools out there to measure the number of other pages competing for a particular keyword, but as a rule of thumb, I just do a couple Google searches:
- allinurl: keyword phrase
- allintitle: keyword phrase
Those two Google searches pull up a list of pages with those particular keywords in the page title and URL. That gives them a strong chance of ranking well for the keyword. That’s your competition.
There’s tons of discussions about “how much competition is too much?” I’ve never been able to come up with an exact number.
I used to use a neat little trick invented by SEO expert Sumantra Roy came up with an alternate method used by many, KEI, the Keyword Efficiency Index. In his article, he simply Googles “keyword phrase” in quotes to see the number of competing pages. He then has a spreadsheet perform a quick calculation: [Search traffic for that phrase] x [Search traffic for that phrase] ÷ [Competition]. By doing this with a list of phrases you’ve picked out using Google’s Keyword Tool, you can then see which ones you’re most likely to rank for.
Whichever method you use to measure search traffic and competition, there are three caveats. Two you know, and two you may not have considered. Unfortunately, the last factor may be the most important.
You probably know this:
- Using Google’s tools only tells you what Google thinks, not what you can expect from other search engines, and even with Google searches, there’s no guarantees. Google’s keyword tool isn’t plugged into its search ranking algorithm; it works totally independently. So actual results may vary.
- Google’s search results are all personalized, taking into account your browsing and search history. Even if you turn off personalized results by clicking the globe icon at top right on a Google search results page, the results you get are still tailored to your geography and language. So any time you do a Google search, remember the results will be slightly different (or sometimes drastically different) from everyone else. This shouldn’t impact your “competition” score too much, but be wary.
You may not have considered that:
- The “number of competing websites” doesn’t tell you anything about how hard that competition is to beat, so that’s why I’ve stopped using KEI (or take it with heavy grains of salt). I eyeball the ten results on the first page of a web search and ask myself, “is my content better than these results?” Sometimes, you’ll hit a search where nearly all the search results are people asking a question, not providing a useful answer. If you see that, it’s not competition, but an opportunity! It shows that no page out there has answered the question very well.
However, here’s the BIG problem:
- The number of competitors you see now is less than the number of competitors there will be in the future. I posted a webpage on how to build a kite back in 1993. For a year or so, it was the only webpage out there on the topic. Even 14 years later, when I rebuilt the page as a Squidoo lens, it owned the search rankings for various kite-related searches. Over time, though, more people have posted kite craft projects, kitemaking websites, kite videos, etc. Not only is my page competing against thousands, maybe millions more competing pages, but also, some of those new pages are better than the article I slapped together in 1993.
Short-term, you may beat out the competition and earn money. And it might be worth targeting a high-traffic, low-competition phrase for that short-term profit.
However, long-term, you must assume that more competition and better content will eventually compete for every single search phrase you can imagine. The web is growing incredibly fast. Therefore, whatever metric you use to measure competition will quickly become an underestimate.
Your only defense is to write on topics for which you can add something unique, valuable, and useful that other sites can’t replicate.
Do consider competition, yes. If, in checking the competition, you see a lot of excellent pages and websites already addressing the search phrase, chances are, you can’t beat them. But even if competition is currently low, you still need to create content as if you’re in a web wide “Best Page on Topic X” contest — because you are! Even if good competitors haven’t shown up yet, assume that they will.
GreekGeek: Great advice as always. I will add one thing. I’ve found a great keyword competition tool that has really helped me in my quest for Amazon sales. I’ve heard good things about other tools, but I don’t want anything I have to pay a monthly fee on.
The Stealth Keyword Competition Analyzer is a one-off price of I think around $30. It has more than paid for itself by helping me find rank-able keywords with the least amount of aggravation.
I’m not making anything from this recommendation. I just wanted to spread the good word about a tool that is working great for me.
Thanks again, for all of your great information and tips!
Oops. Sorry. I just checked the price…it’s actually $37 for the download. But I have to admit the lenses I’ve made from the keywords this tool has helped me uncover are the ones I’m making sales on now!
About the best article on SEO I’ve seen. I have yet to master SEO and just write from the soul. It seems to be working for me.
As you know, I am a believer in keyword research and I have found this article very interesting. One reason I like Market Samurai to research keywords is that not only does it tell you about the competition, but also analyses the competing sites to see where they may be weak enough to compete with.
I have not used the KEI calculation before but I do divide the number of competing pages by the number of searches to give a ratio. This helps when comparing keywords and the lower the ratio, then the stronger the keyword is likely to be.
However, if the number of searches is very low, then you do have to make a judgement call as to whether any possible returns will be worth the effort in making the page.
The point you make about how there’s more and more competition year on year is a very valid one and I do revisit my keyword research on my best pages from time to time to make sure I can keep them performing.
It is amazing how the language that people use to search for a particular topic can evolve and change with the passage of time.