Search terms that are (not provided) and what it means for your website SEO

Maybe you take a close interest in your website’s analytics.  Or perhaps you just get reports on how your business is doing online – that information comes from your web analytics.

In either event, you may have recently become aware that there’s something missing in your website data.

That’s because in many cases you no longer know the actual search terms people used to find your website. It’s now quite possible that your most popular organic search term is simply shown as (not provided).

Above: Does Google Analytics show your most popular organic search term to be (not provided)?

Why (not provided) is an issue for search marketers

Use any search engine and, when you go to a site in the search results, your search query is passed as part of the URL referral string (the “referrer header”) to the site you’re visiting. The search query is then parsed and stored in server log files and website analytics.

That, at least, is the way it has always been.

In aggregate, search query information is useful to online marketers who want to understand what is driving visitors to their websites, the relative value of these referrals, and whether or not these visits result in conversions (however “conversions” might be measured).

For search engine optimisation (SEO), in particular, measuring and understanding the value of these search queries is critical. After all, it’s important that a website be optimised to the right keywords – those that are popular, not too competitive, and convert site visitors into clients and customers.

Who is no longer providing my keywords (and why not)?

Since November 2011, Google has stopped providing the organic search terms used by people who are currently logged in to their Google account (whether they’re using Gmail, Google Calendar, Google Docs, Google Plus, YouTube or another Google property). When a logged-in search visitor comes to your website, the referring search query is now shown as (not provided) in Google Analytics and other analytics software.

Google has cited privacy concerns as the reason for this change. However, search query data for paid (AdWords) visits is still provided.

Matt Cutts of Google initially stated that this change would affect a single digit percentage of all search queries but in fact the effect has been far greater.

Is this something I should be worried about?

Analytics offers various metrics to help determine whether or not your search visitors are finding what they want on your website (or, to put it another way, whether or not your site provides answers to their search query). Not all metrics are perfect – bounce rate and time on site, for instance, are flawed indicators if, in fact, your visitors can quickly and easily find what they want, and if all they need can be found on the landing page.

If you see that organic search visitors have a high propensity to complete conversions – online purchases, phone calls (you are tracking those too, aren’t you?), downloads and the like – you can reasonably assume that for the most part your site is performing to expectations. But without knowing what they were looking for in the first place, it’s hard to identify the opportunities you are missing, and what improvements might mean even better results.

We’ve reviewed what effect (not provided) has had on a sample set of our clients’ sites over the past six months. We selected 10 sites from different verticals and all with distinctly different demographic appeal; among them some of New Zealand’s largest.

The results were surprising – a steady and rapid increase in the percentage of organic search queries now being reported as (not provided). For the 10 clients we reviewed, (not provided) search terms now account for 18.3% of all search visits on average, with the relative percentage increasing month-by-month (the average was 1.9% in February).

Should the Google+ social network grow in popularity (and you can believe that Google is doing all it can to make it a success) then Google users will have more reasons to be logged in. And, with the latest version of Mozilla Firefox (used by around 20%-25% of all Internet users) now defaulting to secure (HTTPS) Google search results, we expect the relative percentage of (not provided) searches to continue to grow. We would be surprised, too, if Google’s Chrome browser (19%-34% of users, depending on whose numbers you use) does not soon follow Firefox 14’s lead.

Above: Growth in the percentage of organic search terms shown as (not provided) over the past six months. For the 10 sites we reviewed, the (not provided) search terms were on average below 2% of all queries in February, but have grown to over 18% in July.

What can we do?

There may well be legitimate privacy concerns that prompted Google to stop sending logged in users’ search terms to destination websites. After all, apart from the websites that the search user visits, unencrypted search query information can also go to any Internet service providers and governments who may be monitoring web users or censoring and filtering search results.

As less query data becomes available – and since it is likely that over time the (not provided) search terms will increase – SEO practitioners have to make do without a significant percentage of search query data.

Naturally enough, various workarounds have come to light which purport to give some insight into (not provided) search terms. However, they often assume commonality between paid and organic search visits or (not provided) and other organic search visits. Alternatively, they may deduce the likely search terms based on landing pages, or historical data. These methods, however, are based on assumptions that cannot be verified nor relied upon.

In the end, all we can do is make the best use of the data we do have, remembering that we still have much more information to work with than most other marketing channels.