IDC Tech Marketing Benchmark: Behind the Scenes

This week the IDC CMO Advisory Service will start revealing results from the 2012 Tech Marketing Benchmark. In this 10th annual study we found some surprises – as you might expect in this era of marketing transformation. In anticipation of the results, I thought I would share a bit of what goes on behind the scenes in the benchmark.

First – what is a benchmark? The term was first used by early land surveyors to describe the fixed point against which all others were compared. Today, benchmarking means the systematic practice of comparing your business processes to what others are doing in order to achieve superior performance. Companies benchmark against peers to learn how they compare with similar companies and best-in-class to compare with those that achieve optimal results.

Why do companies benchmark? A benchmark provides context for decision-making. You spend a million dollars a year on social marketing. So what? If your CEO asks you this question, what will you say? Tech marketers tell us that they like to benchmark for the following reasons:

• Improve the quality of annual planning: Last year’s program budget and gut feelings are no longer sufficient input
• Gain insight into critical trends: Learn what industry leaders and competitors are doing – and how you stack up
• Reallocate costs: Identify areas of overspending and opportunities for better value
• Transform with confidence: Answer questions such as how much to invest in new areas like social marketing or how should I re-organize my department?
• Drive with data: C-level executives increasingly expect marketing leaders to manage their business with the same level of operational excellence as other corporate functions.
• Get an independent view: Benchmark data provides IDC analysts with a wealth of information that make guidance to clients personalized and accurate guidance

How does benchmarking work? At IDC, we use a six-step method.

1. Participants are given a standard taxonomy. This is SUPER important.  IDC requires that participants bucket responses in accordance with rigorous activity-based costing methods and a marketing taxonomy based on 10 years of experience so that we're truly comparing apples to apples. We start agonizing over the taxonomy early in the year. It must evolve with changing times but maintain enough consistency for trending.  This year, we carved out marketing automation as a new category and adjusted definitions to accommodate new media and practices.
2. Participants bucket their marketing investments into categories.  We start participant recruitment in the spring. Fortunately, IDC has a large constituency of companies that participate annually, but we always conduct outreach to get new blood. 
3. IDC collects the data. For IDC's benchmark, the tech company participants are primarily mid-sized and large companies and we have a 95% B2B focus.
4. IDC creates a database of normalized data. This is our secret sauce and takes a ton of work.  Every survey gets scrutiny. Anomalies get investigated. We use statistical methods, sophisticated tools, and marketing experience to work the data so that it really means something.
5. IDC analyzes the database for benchmarks and trends. We conduct analysis of various kinds – comparing years, industry sectors, and program and people data. We also conduct interviews with CMO's to lend color to what we are seeing (although we are constantly out talking to practitioners and marketing leaders during the year).
6. IDC reports.  All participants are invited to a webcast and get a free report that includes a large amount of data and IDC insights.  Over 100 tech companies each year contribute to the database and get this free report.  For participants who desire a more personalized view, IDC offers a custom service that compares their data with a "market basket" of appropriate peers. IDC conducts an analysis of this custom benchmark and then works with companies to provide guidance decision-making and for instigating change.

Watch this space as well as the press for this year's findings!