SEM Punch

Arguably, one of the most labor intensive part of search engine marketing is generating keywords. We’ve all spent countless hours digging around on the web, have tried various software tools, walked in circles around our room at 4 am thinking it would help our creative side come out, or clicked apart our competitors’ websites.

Aaron from SEO Book wrote an excellent overview of some of the existing keyword research tools out there - I am not going to cover the same stuff again and hence I suggest you read up on it. However, his post falls short of investigating automated keyword generation, which will be the focus of mine.

One thing I would like to mention first…I never understood marketers’ obsession with scanning competitors’ websites for keywords. As you will find, coming up with keywords is not an art - the challenges in the process are not getting keywords, but getting the right keywords. Save your energy and focus on your own process; anything else is just a testament of ignorance and you potentially make yourself vulnerable to poor competitor judgement. I know I know, I have strong opinions on a few things…they come from a good place

Here are my recommendations for researching keywords:

Use your internal data!

1. Log file
   Your own log files offer a wealth of information; keywords are of high quality, because they provide insights into how users have actually arrived at seeing your ad and clicked on it. Whenever a user performs a search on a search engine and clicks on your ad, the user’s raw search term will be passed to your webserver as part of the referrer URL. Every decently functioning analytics package will already parse the original search term for you and expose them under the organic search traffic section. Unfortunately it is hard to seperate your true organic search terms from the raw search terms thet resulted in paid search clicks. To get arround this you can also write a custom script that parses the raw search terms out of referrer URLs in your log files.
2. Internal search function
   An internal site search provides information on user behavior and search patterns once the user is on your site. That means, in the context of your site and landing pages users will perform very specific searches that are of high value.Why not log every search term that is being punched into a search box in a database? Even better, first parse the referrer URL for the user’s raw search (as passed from the search engine). Add this raw search term into a database tabel in column 1; add a subsequent search on your site in column 2. What you end up with is a relationship between the user’s initial search on a search engine and your own site. Often, it just takes quickly scanning over this dataset to identify a user’s intent when coming to your site and to identify potentially bad landing pages.
3. Data feed
   Another high quality source for keywords is your own product catalog. Your product title usually contains sufficient keywords to describe a product - use combinations of the main words in your title. Note, however, that if you automate the process, depending on the size of your feed, you might end up with a lot of junk keywords. But hey, you could be pumpin’ them out like crazy! Seriously though, the challenge in automated keyword generation from a product feed is filtering the newly created terms. Doing proper keyword research in your industry is crucial do define proper, automated filter rules.

External sources

1. Yahoo (Overture) keyword tool
   We all know it, the much loved Yahoo (Overture) keyword inventory tool - you type in a keyword and get a list of up to 200 “related” keywords. Yahoo exposes this tool as an API, which is a seperate API from their normal search engine marketing API (DTC-XML API) - creatively, it is called Research API. But sign-up is locked at the moment in anticipation of Yahoo’s new Ad Platform. Generally, not every advertiser can afford the Yahoo APIs, but is there a different way to somewhat automate the keyword generation process using the inventory tool? Yes, there is! Stay tuned…a post on this, which involves a tool I will make available as a free download, is in the works.
2. Google keyword tool
   Google also offers a keyword expansion service, which is much like Yahoo’s. I personally don’t like it as much, because the search volume trends Google associates with a keyword are only on a scale from 0-10. Yahoo actually gives you a monthly search count value. I know I know, it is not accurate. But who cares? All I’m interested in is how one keyword compares relatively in search volume to another - as long as you consistently compare apples to apples using Yahoo’s count works just great.However, Google’s expansion service is publicly available through their AdWords API as part of the KeywordToolService. Using the expansion is fairly inexpensive, but can explode into high costs depending on how exactly you expand. My advice is to expand a small list of root terms in your industry. Once you have the results, expand those. Get results and expand them. You get the point…a leaf-like expansion.
3. Semantic expansion services
   There are actually companies out there that offer services on semantic keyword generation, such as Relekey. The service is very cheap and keywords can be generated en masse using a simple API. I have tested the tool a bit and it seems to work fairly well. Semantic expansion is a bit tricky and most people perform it manually by trying to come up with synonyms or related keywords to their root terms.
4. Software tools
   There are tons of cheap software programs out there. Using those tools still involves a fair amout of manual work. Again, please check up on Aaron’s suggestion of tools.
5. Use excel
   At this point in your keyword generation process, you probably have come up with some major word themes. For example, if you are a travel site you would have major words (or tokens) such as “hotel” “lodging” or “vacation.” Just generate concatenations of your root terms & other iinformation such as all the cities in the world. Search for a database of all cities in the world? Generate your concatenated keywords and start the automated expansion process on them.

These are my tips on the keyword research if you don’t want to spend hours and hours generating keywords using some desktop tool. Did I forget anything important?

Bookmark to:
         

Here the continuation to the common AdWords mistakes made by beginners from part 1…

1. Bids are too low at the beginning
   If the bids are too low, keywords will hardly receive impressions due to low positions. If you start out with a new account, this can be particularly bad because Google attempts to evaluate your “quality” relative to that of your competitors. That means, if your bids are too low, you will likely have a fair amount of inactive keywords, since as a new advertiser you are at a disadvantage - your minimum CPC set by Google is relatively high. In order to gain impressions and for Google to quickly assess your “quality,” I would set the bids high for about a week, sacrificing some dough for the greater good of the campaign.
2. Keywords are too broad
   Often new advertisers start out shooting for the big gun keywords - the high volume drivers that are ultra competitive. Sure, buying “mortgage” may sound appealing, but the secret of this keyword’s traffic showers have already been discovered by the LendingTrees, eLoans & even Ditechs of this world. Unless you truly believe your backend economics are stronger than those of the big spenders, leave this keyword (at least at the beginning) and try finding some hidden gems. Too many advertisers, large or small, are focused on the volume drivers, but they lose out on a whole universe of keywords. But keyword generation is a topic for a whole new post…
3. No ROI tracking
   Newbies sometimes seem to underestimate the value of proper tracking. In the past this was mainly caused due to high prices of tracking programs. But there are plenty of companies that offer decent low-cost conversion tracking, although some are also of poor quality…but free You want to make sure you track revenue & cost properly because almost every keyword portfolio has a few black sheep - keywords that lose money, sometimes a lot of money. Don’t assume that cost & revenue are equally distributed across your portfolio. But optimization is also a stuff for independent posts…
4. No action
   Some marketers throw up their keywords and then don’t really keep an eye on them. If you setup your conversion tracking, you have no excuse not to watch and take action when you see bad keywords!

Alrighty, I hope this helps - go forth and prosper!

Bookmark to:
         

I just read a pretty interesting post, originally raised by a member on the Search Engine Watch Forums:

Does Google AdWords PPC Prices Affect Yahoo! Search Marketing’s Prices?

I would like to chip in on this a bit. But first I want to state that I assume search marketers are rational players, meaning they will do what is in their best interest and make rational decisions. I think it is a fair assumption, given that we are in the most metrics-driven industry there is.

There are two logical arguments why prices on Google influcene bid prices on Yahoo:

Google is profitable - just replicate what works!

Growing competition generally occurs in industries with a) relatively low barriers of entry or b) high profits. For example, it is extremely easy to sign up for tons of travel affiliate programs and start driving traffic to an affiliate site via AdWords. Whenever money is easy to make people want to participate - it gets crowded. At the same time, high-profit industries with high barriers to entry like mortage loans or online education degree programs provide incentive enough to push up the prices. It just happens to be that Google is the largest search engine and easy to use, so starting out marketing on Google instead of Yahoo come natural to marketers constrained by a marketing budget.

Given my assumption that marketers are rational, they will generally only bid so much that they don’t lose money. Or even better, they bid just enough to still make a profit. Once time passes and data accummulates, bad keywords are being bid down or deleted. At this point, one is left with the good keywords, the gold nuggets. So what do with them? Throw them up on Yahoo, MSN & Ask. Chances are they bring in some bacon as well. And instead of waiting to get some good data, just submit the same bids you have on Google.

To shorten my point, competition on Google is likely going to spill over to Yahoo, because the bid prices and distribution of compeitors on Google are a reflection of profitable business.

Budget constraints vanish

Quite simply, if Google is profitable then over time a marketer’s cash stash gets bigger, freeing up additional funds for marketing initiatives. Sure, those could be invested in Google. But why experiment around with a new keyword portfolio and risk losing money? Now that your hard work has paid off on Google, leverage it and throw stuff up on Yahoo & MSN - chances are very good that the same things that worked on Google will work on Yahoo. So as you enter Yahoo, like other rational marketers as well, competition drives up the bids.

However, I think that the spill-over effects are not one-directional, going from Google to Yahoo. They are bi-directional - if some keywords or copies are perform well on Yahoo, they will receive the marketer’s proper attention on Google.

Bookmark to:
         

Just found this interesting verdict of a court ruling in Isreal that legalizes bidding on competitor names on AdWords there. Quoting the news source:

In it’s verdict, Judge Magen Altubia states that a search engine is an ad based media, and the fact that competitors use each other names is just natural.

I wonder how long it will take until verdicts like this pop up in the US & Europe. It makes sense, especially judging by “relevance” of the search performed by the user. For example, if a user searches for “Nike”, why shouldn’t Adidas be allowed to buy the name? After all, the user is clearly interested in sports merchandise, of which Adidas has to offer a few things here and there as well. Also, ss far as I know, nothing would prevent Adidas issuing a press statement saying their sneakers are better than “Nike’s,” right?

Bookmark to: