Return of the Missing Keyword Research Data

If you have been using Google's Keyword Planner for any length of time, you'll know that Google decided to hide all of the demand data and give us ranges for search volume instead.  This article helps you fight back and shows you how to leverage a number of free tools to get more precise supply and demand data.

Two Types of Keyword Research?

When most people talk about keyword research, it's usually a discussion about what keywords to target on their website.  

It usually goes like this:

1. Blue widgets has high demand and low competition, so I'll target that on my site.

2. I'll use the term “blue widgets” in the H1 header, page title, meta keywords, meta description, opening paragraph twice, sprinkled into other paragraphs, ALT tag, and ensure “blue widgets” appears with a density of 5% on the page as a whole.

That is how we all did keyword research a decade ago, and it worked.  Today, that way of thinking is more likely to get your web page (and possibly site) penalized for keyword spamming.  You just cannot optimize around a single keyword phrase and more.

Step 1 is still correct.  Finding keywords with potential to generate traffic on your site is a great start, but what you do with those keywords is the big question.  Step 2 as outlined above will cause you problems, so we need another plan. 

This is where my idea of two types of keyword research comes in.

The first type of keyword research is done to determine which keywords you should target on your web pages.  The second type of keyword research is done to find out how you can safely target those phrases.

This  ultimate guide to keyword research will cover both of these methods.


Google Keyword Planner?

The Google Keyword Planner is a great place to start when you need to do some keyword research.  There are one or two limitations and annoyances, but this ultimate guide to keyword research will show you workarounds for those.

The first problem is that the Google tries really hard to get you to commit to creating an advertising campaign.  For those of us that don't want to advertise with Google Adwords, this is a nuisance.  But there is a solution.  In the following video, I'll show you how to set up your account so that you can use the Keyword Planner without having to set up an Ad Campaign.

Let's take Keyword Planner for a Spin

Load up the Keyword Planner, and let's see what it can (and cannot) do.  On the opening screen, click the Find new keywords button:

Enter a keyword that you want to explore, and click the button to get started. 

Let's assume that I have a technology site and I want to write some articles about the new Samsung Note 9.  That is the term I entered as my search term.  

Google Keyword Planner found 988 keywords related to this term.

You can click any column header to order your results by that column.  Since we are looking for high demand keywords, let's order our keywords by average monthly searches (click that column header).


But Those Demand Figures...

This brings us to the second major issue with the Google Keyword Planner.  Average monthly searches is a range, and a wide range at that.

But I have a fix for that too.  Users of Google Chrome and Firefox can install an extension called Keyword Everywhere.  Not only does this fix the Keyword Planner range issue, but it gives us some very powerful tools.  

Head on over to Keyword Everywhere and install the extension for your web browser of choice.  You'll need to enter an email address during the install, and Keywords Everywhere will send you an email link to get your API key.  You'll need the API key to use their software.

Once installed, you'll have a new button in the toolbar of your browser.   That button opens up a menu, and an on/off switch so you can turn the extension off when not being used.

Click on the Update Settings menu item.

In the API key box, paste in the API key they gave you and click on the Validate button.

Once done, you can close the Keywords Everywhere settings and head back to the Google Keyword Planner.   Log out and then back in again. 

Repeat the search you did earlier.  Do you notice the difference?

You've still got the same columns you had before, but you also have three new ones added by Keywords Everywhere.  

Vol – The number of monthly searches made for that keyword.

CPC – The cost per click of that keyword.

Comp – A number between 0 (easy) and 1 (hard) for the competition you will face if you want to rank for that keyword.

Keywords Everywhere has added back the detail that Google stripped out in previous updates to the keyword planner.

A great option you have with Keywords Everywhere is to save all of the keyword data to your “favourites”.  Look in the lower right of the screen:

You'll get this button on all screens that show Keywords Everywhere data (we'll come on to these in a moment).  Notice also that you can save the data as a CSV file, meaning easy import into Excel or similar spreadsheet.

If you do click Add All Keywords, you'll only get those keywords that are currently visible, so scroll to the bottom of the page and select 500 from the Show Rows option.  You can then scroll through fewer pages of keywords to add them all to your favourites.

To view your favourites, select My favourite keywords from the menu:

The Keywords Everywhere favourites table will open, showing you all of the keywords you have added.  The columns can be ordered by clicking the column title.  So, for example,  you can see which keywords cost the most for advertisers:

The Keyword Planner is Only the Beginning

I can see from the data on Google's Keyword Planner that there is a huge demand for content on the Samsung Note 9.  However, I want to get a better idea of what type of content might be in demand.  Keywords Everywhere can come to the rescue again.  It doesn't just work with Keyword Planner.  It also adds keyword data to a lot of other tools, like Google Search:

You can see a complete list of sites that get the Keywords Everywhere treatment in the settings screen:

As you can see, this plugin will add keyword data into your Google Analytics and Google Console screens.  However, there are two that I really like when doing keyword research.   Keyword Shitter and Answer the Public.

The first of those keyword tools spits out keywords at a rapid pace, and Keywords Everywhere provides the useful data about each phrase.  Go and try it.  

Here is a snapshot of the phrases returned from that tool:

I actually stopped tool after a minute or so as it keeps on finding new words.  I also filtered out only those phrases that contain the word Samsung.  So that's 970 keyword phrases that might be useful.  

Keywords Everywhere works hard on that page too, finding all the supply and demand data.  You can see that as you scroll down the page:

Commercial Keywords?

One thing I like to do with keyword lists like the one from Keyword Shitter is to collect them all in a text file and use Google Keyword Planner to analyze them for CPC, impressions and potential clicks.  In other words, get Google to tell me which of the keywords are the most commercial in nature.

Let's see how this works.  To demonstrate, I'll grab those 970 keywords from the Keyword Shitter.

On the Keyword Planner site, go up to the Tools menu and select Keyword Planner to start a new session.  From the opening screen, select Get search volume forecasts:

Now paste in the keywords from Keyword Shitter and click Get Started:

Google Keyword Planner will now go away and find data about your keywords:

The data in the table is designed for Adwords advertisers, so provides data about the potential cost per click, CTR, Impressions, etc of these keywords.  This can be very interesting data to look through as it highlights the more commercial keywords (the ones advertisers are already bidding on). 

Finding Content Ideas

The tool I find the most powerful to find in-demand content I can provide on my site for my readers, is Answer the Public.  A quick search for Samsung Note 9 returned keywords in three categories: Questions, Prepositions, and Comparisons. 

This tool found 50 questions that people ask about the Samsung Note 9:

I love these visual diagrams, but you can see the data as plain text if you prefer by switching from Visualisation to  Data:

Some of these questions provide excellent ideas for articles on the phone and represent real questions asked by real searchers.

The prepositions section provide further content ideas.  These are search phrases about the Samsung Note 9 that include words like can, to, without, near, with, for, and is.

The final section is the comparisons.  This again can be pure gold if you are looking for content ideas.  These are search phrases that include words like versus, and, like, vs, and or:

So where does Keywords Everywhere come in to all this?
Well keep scrolling down and you'll see a list of all keywords at the bottom of the page:

And don't forget that Add All Keywords button at the bottom of the page if you want to save all of this data.

So finding keyword ideas is actually quite easy.  But what happens once you have completed the first phase of research and you want to create a web page about one of the terms you've found?  Well that is where the second type of keyword research comes in.  

For example, if you have chosen the phrase “how to root Samsung note 9” as one you want to target, you've now got to identify the words and phrases that SHOULD also be found in a high quality article about rooting this phone.  

As you know from earlier, you cannot just stuff how to root Samsung note 9 into the H1 header, page title, description, etc.  Google will penalize you for that.  So how can you make sure Google understands that your page is about rooting this phone?

What you need to do is write the article using words and phrases that convince Google that the topic of your page is about rooting the Samsung Note 9 phone.  For any article you want to write, there are words and phrases that will be essential.  In this example, I'd imagine words and phrases like these would be pretty important:

android, device, phone, galaxy, samsung note 9, TWRP recovery mode, access, factory reset, install TWRP, root access, USB, computer, flash, warranty, and so on…

I could go on, but for those of you not familiar with rooting an Android device, it's probably a little pointless.  What I am trying to get across is that these words and phrases are probably essential in any good article on rooting the Note 9 phone.  In fact, given these words and phrases, Google can probably guess what the page is about, even if I didn't use the original phrase “How to root Samsung Note 9”.

How to Find These Theme Words & Phrases

When you do a search on Google, the pages that are returned in the top 10 are there because Google thinks these are the most relevant to the search term.

That means these pages should contain all of the relevant theme words and phrases for your term.

Keywords Everywhere can pull out the words and phrases from any page.   

Let's try it.  

Search Google for a term you are interested in targeting on your site.

Visit one of the top 10 pages.

Once it is loaded, select Analyze this page from the Keywords Everywhere menu.


Keywords Everywhere will show you a table of the keyword data.  You'll see words and phrases listed, with the locations these words and phrases were found.

There are 4 theme words identified by Keywords Everywhere.  The second column shows where these words were found.  e.g. the word root was found in the title, H1, H2 & H3 of the page.  That third column is density, so root appeared at a 1.13% density, which is about 11 times in a 1000 word article.

You can see that how to also appeared in the title of the page.  In fact, the title of the page is “How to root Samsung Galaxy Note 9” which is very similar to the phrase we identified as our target.

To identify the important theme words and phrases, you  can visit several top 10 pages and run Keywords Everywhere on these pages.  Keep notes on which words and phrases appear on most of those top 10 pages and you'll have your list of theme words.

Of course, if you are an expert on this topic, you won't need to research which theme words and phrases you need to include.  As you write, you'll include them naturally.  That's the advantage of being an expert on the topics covered on your site.

Is there a shortcut?

I have been creating my own content this way since around 2006.  I knew back then that I wanted to be able to write content that made me look like an expert.  And I knew that to do that, I'd need to be able to identify the important words and phrases for any article I wrote.  These important words and phrases would be the same ones an expert would use if they wrote this article for me.

To identify these words and phrases for any article is a huge time suck.  My solution was to create some computer software that would do it for me, reliably, repeatedly, and without me needing to know much about the topic.

Let's take the same “How to root samsung note 9” phrase and run it through the software to see what information we can get about theme words and phrases, but also idea on what we need to cover in our article.  

Here is a video I recorded to show the process:

So there we have it.  My content keyword strategy for any website I am working on.

Step 1.  Find the keywords that are in demand and can bring traffic to my site.

Step 2. Find the theme words and phrases that I need to use in the article.

Whether you use a tool like Web Content Studio, or do it all manually is your choice.  However, I cannot emphasize how important it is to look at the top 10 ranked pages.  By looking at these pages, you'll know the 10 pages you have to beat and can plot your own strategy to make your content the best out there.  

If you are using other free keyword research tools that you think are worth including in this guide, please leave a comment below and let me know.  

Want to Learn WordPress?

WordPress is an amazing platform for building any type of website.   It’s used by large corporations and small mom & pop sites.

You may also like


7 thoughts on “Return of the Missing Keyword Research Data”

  1. Hi Andy,
    I enjoyed your keyword research article above. Questions:
    What software do you recommend for doing keyword research for an Amazon store or a Shopify store?
    Please advise. Thanks.

  2. Dear Doctor Andy
    You rat! About half way thorough….and I must admit I was beginning to fall asleep, it dawned on me that you were being sneaky.

    I was getting sleepy because, naturally, I own Web Content Studio. What need I of these keyword things? (That’s what came to mind halfway through your epistle.) Now, if I did not have Web Content Studio, and had no plans to acquire Web Content Studio, I would NOT have gotten sleepy bekez what you teach and profess is dynamite for those new to the business.

    Were I still in the IM game big time I would be quite happy if I were one of the few with Web Content Studio because it gives a totally unfair advantage over the hit and miss knucklebuster keyword finding crowd. (I was one of those for many years so I know!)

    Tell you something new. My father was a Master Electrician. He could fix anything electrical. Things he had never seen before. Toasters to trains. He was intuitive about all things electron powered. I was a journeyman and I was pretty good too…but it came hard for me because I was not intuitive. I had to knucklebust to get my electrical smarts.

    You, my friend are intuitive about Google. You take the paltry lemons they grudgingly give us and make the most delicious lemonade out of them. Like my father you have forgotten more about your trade than most of us will ever learn.

    Fact is Dr. Andy that compared side by side you give away far far more than you will ever get back. You could give away much less….stretch out your tips and tricks, and so increase your ROI. Others always do that. Or bait and switch….all the usual behaviors we have seen over the years.

    But you DON’T do that. You put it all out here, trusting in humanity that you’ll increase ten fold.

    I kinda doubt the equation ever balances for you using this method. But on the other hand you have ALL THE FUN doing it your way. And in the end, that means more than gold.

    Since I’m into book writing these days… (My latest fun project is turning a certain book of the Bible into a stage play) I’m quite happy to urge the readers to just do it. Get Web Content Studio. Best bucks you’ll ever spend if you’re serious about making your mark marketing on the net.

    Your constant fan in Alabam,

  3. As always Andy very useful and interesting thanks, But! there are a lot of business who gladly pay for google ads to sell their products online. They don’t get a good return, mainy its because of a lack of knowledge on how to pick good keywords and how to use negative keywords on their campaigns (They rely on google for advice ). So if someone was to do a course on this it would be very helpful.
    Regards Alan

  4. Thanks Andy for this informative post. I quit using Keyword Planner after they changed the search results to a range. After following your instructions, I can now see the extra columns provided by Keywords Everywhere. Thanks for this information.

  5. Hello Andy,
    I would like to thank you for introducing us to the Keywords Everywhere browser add-on. It seems to disclose a ton of helpful information. However, a couple of things were not clear to me.
    a) What are the ‘winnable’ search volume/month and competition numbers? Do you have any specific guideline as to which keywords we should focus on based on these two parameters? Especially the competition – should we target (and hope to rank on google’s first page) keywords with competition less than 0.5/0.2/0.1 or any other number?
    b) When I analyzed one of the top pages of a certain google search result using the “Analyze this page” feature, I found that several of the keywords the page was using was showing zero monthly search volume and competition. It didn’t seem obvious to me why someone would write about phrases that are not searched at all! Is this a reason for us to be suspicious about the genuineness of the numbers Keywords Everywhere is showing?
    Can’t wait to hear from you.
    P.S. I am one of your customers on Udemy.

Comments are closed.

Hello, I'm
Andy Williams!

You can get up to 90% off my Online Courses for webmasters, marketers & affiliates (plus a free course on Gutenberg).

Create your own WordPress Theme

It's built in to WordPress using Gutenberg, and my new course shows you how.