Quality Score Explained by former Google Employee

With the continuing expansion of ad space at the top of the page (from ad extensions like sitelinks with descriptions), it’s more important than ever to make sure your ads have a great ad rank. But CPCs are on the rise – so unless you can improve your conversion rate so you can increase bids, Quality Score (QS) optimization may be your only way to maintain a high rank without breaking the bank.
While I’ve been a panelist representing Google in Quality Score sessions at more conferences than I can remember, I’ve been meaning to put together some of my insights on the topic for a broader audience. So, here goes….

The Evolution Of Quality Score

Even if you don’t care much for a history lesson, it may help paint a picture of why QS exists in the first place, and the evolution of factors may give you some insight into what Google cares about.
When I started working at Google in 2002, Quality Score didn’t exist. But what set Google apart from the other PPC ad systems at the time was how they determined if ads were relevant enough to warrant an impression. In addition to having a team of humans reviewing all ads, the system was also monitoring the click-through rate (CTR) of every keyword — if a keyword’s CTR fell below 0.5%, it would become disabled for poor relevance. Google was using the wisdom of the crowds to let its users determine which ads should show and which shouldn’t.
The problem with disabling keywords at an arbitrary CTR level was that advertisers were unhappy to see some of their keywords disabled forever (unless, of course, they knew of the workaround to resubmit the same keyword with different capitalization). So, we started to evolve the system. Instead of disabling keywords, we came up with new statuses like “slowed,” “in trial” and “on hold.”
The idea was to give advertisers a way to fix low-relevance keywords by giving them a small allowance to run ads that were considered poor quality so that they might prove otherwise. We also gave the system more flexibility so there wasn’t a hard cutoff at the 0.5% CTR level. Ultimately, though, most advertisers still saw keywords they really wanted to advertise on get disabled, and they remained displeased.
Google addressed this with the introduction of the minimum bid. Rather than flat out disabling keywords, they asked advertisers to pay more for keywords that had low relevance — the idea being that eventually, it would make no sense for an advertiser to keep paying the high cost-per-click (CPC), and they’d either optimize the relevance or delete the keyword.
In today’s system, the minimum bid has been engulfed by the first page bid, which also takes into account competition. It’s a little harder to see the direct correlation between what you pay and the quality score, but the connection’s definitely there.
Below is an example of where we see the correlation between the average CPC and the QS in our Quality Score Tracker tool.

As the Quality Score starts to increase, the average CPC in this campaign starts to drop.

Quality Score’s Impact On Ad Rank

When Google launched AdWords Select and started to shift from CPM-based pricing to CPC-based pricing, they couldn’t afford to lose all the revenue from their CPM program (which was called AdWords at the time). They were still a pretty small company, and Yahoo/Overture was a formidable competitor. So, to ensure revenue was maximized, they ranked ads in the CPC program according to a very simple rule:
Ad Rank = Max CPC * CTR
If you take a minute to look at this more closely, you’ll quickly see that ad rank is, in fact, equal to CPM or dollars-per-impression. This was the simple but brilliant insight that made AdWords so powerful — advertisers only had to pay when they got clicks, users would see more relevant ads because ads with high CTRs were more likely to appear high in the results, and Google was making as much money as possible from these ads.
The equation for ad rank is a whole lot more complicated these days and now includes thresholds for appearing at the top of the page, landing page factors, and more. But at the heart of it, the original principle still applies: if Google can show more relevant ads, they will get more clicks, have happier users and make more money. And, the key component for achieving this is CTR.
The importance of CTR to Quality Score is a bit like the importance of TF-IDF to SEO. While there are hundreds of factors that go into ranking in paid or organic search, these long-established principles are still some of the most important ones. In the 80/20 rule, these are your 80 percent factors and the first ones you should pay attention to.

Factors Of QS

While I’ve explained that CTR is a main driver in QS, it’s useful to understand how Google thinks about CTR. After all, there are many things that influence the CTR you see in your account such as the device, the network, or the position of your ad on the page. Thus, the average CTR you see in your account is not the CTR Google uses to determine Quality Score.
To make sure advertisers have a level playing field, they evaluate small slices of CTR.
For example, they look at different CTRs by device type so that your performance on mobile won’t affect your performance on desktops. They also have a different CTR they look at for the Display Network and Google Search — a good thing, since CTR on Display is usually much lower, and you wouldn’t want that to hurt your QS for search.
Where possible, they also favor looking at the CTR when the keyword in your account matches the search query exactly (don’t confuse this with the “exact match” keyword match type), and they normalize the CTR based on the number of ads on the page and your ad’s position amongst them.
Furthermore, Google has to make some guesses before they have a statistically significant amount of CTR data for new accounts, new keywords and new ads, and they do this by evaluating the CTR at different levels as explained in the diagram below.

The various levels at which Google evaluates CTR to determine Quality Score.

As you can see, there are 3 levels of CTR evaluation: the account, the keyword and the ad. These CTR elements are all combined into a secret formula and out comes your keyword-level Quality Score and the corresponding number between 1 and 10 that you can see in your account.

How Quality Score Is Set For New Keywords

When a keyword is new in an account, there is not a strong historical element for how the keyword performs with its ad text (factor 3), so the QS is mostly based on system-wide data for that keyword in all other accounts. That gets combined with data for how this particular account and its ads have performed historically. If these elements have good QS, the new keyword is likely to also start off with a better QS.
To give an example, if you have 2 accounts, you should see a lower starting min bid in the account with the better account-level QS. If you have 2 domains, you will see a lower starting min bid when using the domain that has a better QS.
After the system gets enough data about more specific things, like how the keyword performs with the ad you wrote for it, it will rely much more on this to determine the Quality Score. This is why it’s so important to have great account structure and split up your ad groups in a way that allows you to create great performing ad texts for each grouping of tightly related keywords.

Other Relevance Factors

According to Hal Varian, Google’s Chief Economist, QS also considers “relevance” in addition to CTR. But what does that mean? The easiest way to think about this is once again based on click-through rate — but, rather than using CTR to generate the QS number between 1 and 10 that you see for each keyword in your account, it’s used at the time a user does a search to determine if there are any correlations between that user’s search and your ad that could predict the CTR (Google’s Quality Score is a predictive system that tries to predict CTR for each ad and each query). Some examples:

1. Did the user’s search include some additional words, and do those correlate with your ad’s expected chance of getting clicked? E.g., if you have a job website and want to advertise on the keyword [jobs], your ad is probably not relevant when someone searches for “Steve Jobs.”
2. Does the location of the user have any correlation to your predicted CTR? E.g., if your business is in the US and the searcher is in Belgium, perhaps it’s less likely your ad will get the click because the users may prefer a business closer to them.
3. Does the time or day of week influence your predicted CTR? E.g., Google may know that users are less likely to click on your ad on a Tuesday.

These factors let Google assign you a real-time quality score which they can use to better rank your ad for that particular query. There could be many other “relevance factors,” but just know they’re all based on the same principle of trying to predict the likelihood of your ad getting a click based on something Google knows about that specific query.
While the lack of transparency into the factors may be annoying, this relevance component has helped advertisers by automatically giving them more good clicks and fewer bad ones.

Landing Page Quality Score

The landing page is one of the newer factors used for QS. Landing Page Quality (LPQ) started as a way to counteract bad sites that duped users into clicking their ads and hence had a good CTR but a lousy user experience. Now that LPQ can also improve your QS, it’s getting a lot more attention from advertisers (probably more than it warrants).
Remember, the CTR of your ads is still the bigger QS factor and probably the better thing for most advertisers to focus on optimizing. Every now and then, I hear of advertisers who are spending a ton of time creating one landing page per keyword so that the keyword will appear on the page and score a better LPQ. That’s probably overkill — Google is very good at understanding how words are related, so it’s unnecessary to include every variation on the page.
My personal recommendation is to also keep a close eye on bounce rates and time on site, two metrics you can see directly in AdWords when you link it with Analytics. A high bounce rate or very short time on site both provide a great way for you to find keywords that are not relevant in the minds of users.

Optimizing The Right Elements Of Your Account

If you haven’t guessed by now, optimizing Quality Score is really all about optimizing for CTR. The challenge is to optimize for the right CTR. For example, because Google uses position normalization when determining how your CTR impacts your QS, it could very well be that your ad with a 15% CTR in the top position on Google is actually worse than your 3% CTR ad in the last place on the right side of the page.
You should also look at the impression-weighted Quality Score to determine which keywords and ad groups are most in need of an optimization. I shared a script for automating the calculation with AdWords Scripts.

Google Adds Enhanced Campaigns Bid Adjustment Reporting To Google Analytics

Google announced today that Bid Adjustments reports for AdWords enhanced campaigns are now included in Google Analytics.
The reporting, found in the Advertising section under Traffic Sources in Analytics, are designed to help advertisers analyze the performance of each bid adjustment within a campaign — by device, location and time of day.
You’ll notice in the Google-provided screenshot example above, there are columns for Revenue and Ecommerce Conversion Rate in the Summary view. These metrics are available when Ecommerce tracking is enabled in Google Analtyics, allowing you to analyze bid adjustment performance by ROI. However, when you look at your own reporting, you may see that the Summary view only shows goal results and that the columns for Revenue and Ecommerce Conversion Rate appear under the Ecommerce view instead.

The Most Important KPI For A Performance Marketer

Many performance marketers continue to consider click-through rate (CTR) as a key performance indicator of their search campaigns’ effectiveness and evolve their PPC optimization strategy around that.
At the end of the day, what matters most is achieving the best ROI given your business objectives and budget, and you might optimize directly to CTR or ROI or a combination of success metrics to achieve that.
In order to have the best optimization strategy for your SEM campaigns, it is important to understand and quantify the influencers of ROI.

The Two Extreme Optimization Strategies

There are two types of strategies performance marketers consistently use as their campaign optimization strategy:
1)    Optimizing To A CTR Goal
One of the main factors influencing Quality Score (QS) is CTR, which affects your cost-per-click (CPC) and in turn affects your ROI. An increase in QS due to a boost in CTR would lower CPC and improve ROI.

2)  Optimizing To An ROI Goal (Revenue-Per-Click & Cost-Per-Click)
Direct optimization to revenue or a conversion metric is a common strategy amongst performance marketers. Making sure an intelligent bid management is in use will be crucial to your campaign’s success.

While perhaps no marketer purely optimizes to CTR or ROI, they tend to skew towards one of these camps. Each method has its pros and cons. A CTR strategy will get you more clicks but does not guarantee the highest ROI. A purely ROI approach will get you the highest ROI but you potentially lose out on customers early in the sales funnel who might eventually convert.

ROI Breakdown

ROI equals Revenue-Per-Click (RPC) over Cost-Per-Click (CPC). Data analyzed from over two dozen advertisers using econometric methods (a simplified version of the equation is shown below) shows that 34% of ROI is influenced by RPC and 66% by CPC.
Bid management is by far the most important influencer of ROI. Forty-nine percent (49%) of ROI is influenced by bid management, 13% by other factors (i.e., marketplace, seasonality, etc.), and 4% by CTR. The data show the importance of having an intelligent bid management strategy in place for your SEM campaigns. But, does this mean a CTR maximizing strategy is a wasted effort?

A Deeper Dive Into The Relationship Between ROI & CTR

Previous studies have looked at the relationship between CTR & ROI by purely relying on correlations. A correlation analysis alone cannot determine the effects of CTR on ROI, and a more robust statistical technique is required to answer that question. These techniques enable us to control for all the factors that can potentially influence ROI.
From the chart below, we do see a relationship between the CTR & ROI — but not a very strong one.


Applying statistical modeling techniques will allow us to quantify any statistically significant relationship between the two if it exists.
In this model, I control for position, CPC, industry, and bid management differences across the different advertisers in the data in addition to CTR.
The results show that there is a statistically significant relationship between CTR and ROI; but in terms of impact, it’s quite small. For a 10% increase in CTR, expect to see a 1.2% increase in ROI. This means that if you increase your CTR from 10% to 11% for a campaign with an average ROI of $5, the ROI will increase to $5.06 due to the improvements made in CTR.

Key Takeaways For Performance Marketers

• Campaign managers should utilize both strategies above in optimizing their campaigns; main focus should be on ROI but do not completely ignore CTR
• 49% of ROI is influenced by bid management; intelligent bidding is integral to a campaign’s success
• CTR does have a small but statistically significant impact on ROI; a 10% change in CTR affects ROI by 1.2%

In Summary

Focus on optimizing your SEM campaigns for ROI but keep an eye on CTR. There is no need to purely optimize to CTR as it influences only 4% of ROI; but, it is important to account for it in your longer term strategy and make sure healthy CTR rates are met and maintained.
Intelligent bid management heavily influences ROI and is absolutely necessary to ensure your ROI goals are met.

Quality Score vs Page Rank

Is Quality Score in PPC Adwords (or Yahoo Index in Yahoo Search) simillar to Page Rank in SEO… Lets find out!
The article includes points of similarity on two important Algorithm metrics in Search Engine Marketing World (PPC + SEO). The list is not final as there is more, which needs to be included, but I’ve tried to highlight dots that could be a part of our initial point of understanding / learning / conversation.

Google Tablet CPCs Rise 1.7% Above Desktop For First Time [The Search Agency Report]

The Search Agency issued its Q2 State of Paid Search report today, finding that overall impressions were up 19.2 percent across all search engines quarter-over-quarter (QoQ), while overall clicks declined 7.8 percent. Cost-per-click (CPC) rose across all devices.
This marks the first quarter The Search Agency has seen tablet CPCs outpace those of desktops on Google. Tablet CPCs rose 26 percent QoQ, coming in 1.7 percent higher than desktops.

Overall tablet clicks dropped 8 percent QoQ. Year-over-year, however, tablet clicks were up 62 percent. Bing continued it growth on tablets, increasing its share of clicks from 8.7 percent in Q1 to 10.9 percent in Q2.
Smartphone were the only device segment to see an increase in clicks QoQ, with a slight uptick of 1.6%. Smartphone ad spend leaped 25 percent for the quarter.
CPCs Rise Across Search Engines
Overall cost-per-click increased 17.3 percent QoQ and 10 percent YoY.
On Google, CPCs showed an increase across every type of device QoQ,  with the largest increase coming on tablets at 26 percent. Bing’s average CPC increased 18.9 percent YoY and remained relatively flat QoQ.

The Impact Of Enhanced Campaigns 

While reports from RKG and Covario concluded that enhanced campaigns have had little impact so far on CPCs, The Search Agency finds otherwise. The report says, “The rise in marketplace competition caused a spike in CPCs during Q2″.
The report also suggests that enhanced campaigns played a role in the relative decline in clicks QoQ compared to impressions: Declining CTR “may indicate a shift in matching or query mapping changes at the search engine level.” Google impressions rose 21.4 percent while clicks fell off 7.1 percent QoQ.
CTR dropped sharply in Google from last quarter — from 3.46 percent to 2.66 percent. However, it looks more like there was a spike in Q1.

Though less dramatic, Bing also saw click declines. Bing impressions rose 15.6 percent while clicks dropped 4.1 percent QoQ. Bing also saw an increase in CTR in Q1 and dropped back closer to Q4 levels last quarter.
Overall CTRs fell across devices in Q2. Desktop was off 22.6 percent QoQ, though just 3.9 percent YoY. Smartphone CTR fell 18.6 percent QoQ and 16.5 percent YoY. And Tablets saw the biggest QoQ drop, falling 21.1 percent, and 15.1 percent YoY.
Desktop Losing Share of Spend 
The Search Agency found desktop share of spend continued to fall, coming under 75 percent for the first time this quarter. Desktop 74.5 percent share of spend marks a 12.5 percent drop YoY and 2.5 percent dip QoQ.
Smartphone share of spend has risen 70.4 percent YoY and 12 percent QoQ. While tablets have increased their share of spend by 74.7 percent YOY and 4.2 percent QoQ.

Mobile Efficiencies Still High On Bing

While Google CPC efficiencies compared to desktop are falling for both smartphones (11.3 percent down from 14.1 percent in Q1) and tablets (now 1.7 percent higher than desktops), there are still sizeable discounts on Bing. Bing smartphone CPCs were 28.3 percent lower than desktop in Q2 (way up from 16 percent in Q1), and Bing tablet CPCs are off 7.7 percent compared to desktops. That’s up from 5.2 percent in Q1.
The Search Agency sees device adoption on Bing continuing to grow and suggests there is a particularly strong opportunity to add tablet campaign on Bing. Overall they suggest adopting a consistent mobile strategy to better understand consumer search behavior and increasing clickshare on tablets and smartphones.

Are Google’s Results Getting Too Ad-Heavy & Self-Promotional?

Are Google’s search results getting too ad-heavy and leading back to Google’s own content too much? A new blog post suggesting that Google’s non-paid listings make up only a tiny fraction of the entire search results page has sparked some discussion, though the exact percentage actually varies on how you count what’s on the page and from query-to-query.
According to the blog post by Aaron Harris, co-founder and CEO of Tutorspree, organic search results — listings that are not paid ads but ranked highly because Google thinks they are the best answer to a query — made up from 0-to-13% of a Google search results page.
However, if you measure the page not by pixel count but by actual listings, the situation is brighter than some of the “death of organic listings” proponents might think. Also, some things considered to be “Google” listings might not make sense to count that way.
Here’s what Harris found, along with some further analysis.

Auto Mechanic: Only 13% Unpaid?

When performing a Google search for “auto mechanic” using his Macbook Air with a 13-inch screen, Harris discovered that AdWords paid listings took up 29% of the page (12% at the top and another 17% to the side). The Google navigation bar took up 14% of the page. Unpaid “organic” listings got 13%, with the Google map plotted with local results having 7%, as illustrated below:

Vignesh Ramachandran at Mashable experienced similar results when he performed his own test on Google. Using the same search term as Harris — “auto mechanic” — Ramachandran found that organic search results only accounted for an estimated 13.5 percent of screen real estate on his 15-inch MacBook Pro in a Firefox browser:

Caveats & Counting Issues

Two sources coming away with a 13% figure for unpaid listings can sound pretty low. But that’s not the same as being able to declare that all searches are this way. Each search may have a different mixture of paid and unpaid results, based on the ads competing to appear, the location of the searcher and whether Google itself determines if a particular query deserves to be ad-heavy or not.

Beyond the variation from search-to-search, how you measure the percentage of a page that’s deemed “unpaid” isn’t as clear cut as it may seem.

One immediate caveat is the assumption that the map is somehow not worth counting in as “organic” listings. Clicking on the map leads to a page that will have both organic listings and paid listings plotted on a map. There’s a usefulness for search engines to show local information on a map. Arguably, some of the map “percentage” should count into the organic listings.

Another issue is the idea that the search box and navigation links should be somehow counted against Google as some type of new attempt to drive more traffic to Google products. Google’s long had navigation links. In fact, at some points, the navigation and search box unit might have been larger than it is now.

Beyond that is the idea that measuring in pixels is somehow the correct way to go. It assumes that the entire page is seen and interacted with in the same way. It also, oddly, counts the ads on the side as having nearly twice as much space as they actually take-up, because the box around them includes a bunch of white space.

Traditionally, what has concerned search marketers worried about Google (or any) search engine encroaching on the organic space has been to count the actual number of listings, especially those in the middle of the page, where people typically focus their attention and clicks.

By those measures, the example above works out to have 10 listings that are fully visible, with percentages like this:

• Total paid versus unpaid: 70% to 30%
• Paid versus unpaid, middle column: 50% to 50%

Those percentages are much better than the “13%” you might come away with from the original blog post. But then again, they still feel pretty low.

Meanwhile, Over At Bing…

For comparison sake, I ran the same search on my own MacBook Air with a 13-inch screen at Bing:

I didn’t try to measure the pixel count, because as explained, I’m not sure that’s the right approach for various reasons. But at-a-glance, it’s pretty clear that an ad-heavy page isn’t just a Google problem. In this example, Bing manages to push all but a single organic listing to the bottom of the page — and only the title of that one shows.
Like Google, the map leads you to a mixture of paid and unpaid listings. Unlike Google, selecting the local listings that appear next to the map sends you back into Bing Maps rather than to the actual business. Google used to do the same but changed this practice about a year or two ago, after criticisms.

Italian Restaurant: Only 7% Unpaid?

For another search, organic listings won even less screen real estate. When Harris looked for “italian restaurant,” the newly introduced Google Local Carousel located at the top of the page garnered a full 30% of the screen real estate.
With the navigation bar still taking 14%, AdWords at 9%, the Google map with 15%, a Google-owned Zagat listings (outside of the carousel) at 4%, organic search results for Harris’s search made up for only 7% of the page:

Again, that’s a shockingly low number at first glance. But, it gets better when you understand more about how the search page actually works.
The carousel links aren’t paid listings. Clicking on them leads to a fresh search results pages for the particular restaurants that are listed. The downside is that, as with Bing, this drives people back into Google rather than over to the restaurants themselves.
That’s disappointing. Google’s goal here is that the carousel is part of its Knowledge Graph, where it’s trying to share answers and information about things, including restaurants. But if someone clicks on the name of the restaurant, there’s a good assumption they just want to go to that restaurant’s page, not get stuck in an endless loop of Google search results.
As said, Google changed how its map results did this in the past; hopefully it will reconsider how the carousel works.
Another issue is counting the Zagat listing separately from the overall “organic” figure just because it’s a Google-owned property. Google asserts those properties are competing with all other pages and only appear if its algorithm believes they are relevant. There’s no programmatic command to always show a Zagat page at the top. Some won’t believe this, of course. But still, that’s far different than the assumption that a Zagat page might always show.
Indeed, here’s what I see in my location for the same search:

In this case, I don’t get any ads at all. I also don’t get any Zagat listings. Instead, I get the Olive Garden, an LA Times article and two listings from Google-competitor Yelp.

Meanwhile, Over At Bing…

On the one hand, I’m much happier with what I get from Bing:

There are ads, but they are over in the middle column, under the map. Clicking on the main listings takes you to the restaurants, unlike how Google’s carousel works. Organic search is far from “dead” here.

Then again, Google’s giving me a more colorful display that actually lists more restaurants than Bing does. If the carousel took me directly to those restaurants, rather than back into the search results (where you can then get to them), I think it would be a much better winner.

A final caveat in all this. Looking at a page doesn’t help you understand some of the interactivity that goes on. With Bing, selecting the “Reviews” link changes the middle column into showing more information from Bing’s own sources. How that gets measured is another complication, if we’re counting pixel space versus listings.

0% On Mobile?

Organic search results fared even worse in mobile searches, claiming 0% of the initial screens. When Harris searched “Italian Restaurants” from his iPhone, he had to scroll through four full screens before seeing any organic listings, which appeared after ad units, Google-owned Zagat listings, and a Google map followed by Google local listings:

Again, it sounds terrible until you get into the caveats. First, there’s the counting of Zagat in the first example as not being an organic search results, when it is. That really makes the results on that page 50/50 paid versus organic, unless you assume based on one search that Zagat will always occupy the top spot and that there will always be an ad. That’s not the case for me, in my location. In my area, Yelp has the entire page first screen.
After that, I get the same type of Google local listings as shown as “2″ in Harris’s example. I suppose that’s bad news for the Yelps of the world who want even more of that page, and it really does illustrate what Harris said, something the Yelps and others have already themselves said many times before:

“If you compete with Google in any way, you’re in its crosshairs. Your chances of ranking high enough to garner traffic are virtually nil and getting smaller.”

Life Harder For Competing Search Engines; Not Necessarily For Web Sites

Indeed. While the future may not be as bad as Harris paints it for other local search providers, in getting traffic from Google, the trend is pretty clear. Where Google can provide answers, it’s going to do that more and more directly, rather than feeding out to competing search engines.
That, however, isn’t necessarily bad for the user. If I’m on my phone, and I want to learn about a restaurant, the Google local results in the second screenshot are extremely useful, offering to let me call the restaurant or get directions to it. If I drill in, I’d even get an Urbanspoon menu. I’d also likely get Yelp reviews along with Zagat reviews, if that’s what Yelp wanted. But that’s not what Yelp wants, because Yelp blocks those reviews from appearing in Google.

But Google Owning Content Is Troubling

For me, the concern isn’t that Google doesn’t show enough listings of other search engines, any more than I’m not concerned that the New York Times doesn’t run enough Wall Street Journal articles.

To me, the real concern has been the transformation of Google from being a search engine that pointed out to destination sites (like those restaurants) to wanting to be a destination of its own.

SEO and organic search is far from dead, and anyone who runs a site can look at their traffic logs to know how much traffic Google sends them every day, for free. But selling movies, offering restaurant reviews, hosting video, hosting book content and more does further pollute the clarity we used to have about what Google’s role was as a search engine, and whether it pushes its own content above others.