ContributorsEnabling Technology

Lies, Damned Lies, and Statistics: How Research is Manipulated

Trust but verify HRM BPO market research data. The devil, as the saying goes, is in the details of the study.

by Naomi Lee Bloom

As HRM BPO heated up as a concept and a market, there’s been a rush of market research activity. In addition to a growing pool of buyers trying to figure if, when, how, and with whom to expand their use of HRM BPO and beleaguered providers trying to figure out how to make money and deliver excellent service in this business, a whole ecosystem of influencers, suppliers, and advisors also trying to figure out how to make money without either buying or selling HRM BPO has emerged, many pedaling market research data.

While high-quality market research does contribute to the vitality of the HRM BPO industry, too much of it or research of poor quality can also create enough noise to drown out the important, direct dialogue between the buyers and sellers. And the real problems begin when the noise is being generated for the sake of noise rather than for the sake of improving the industry.

Not unlike our need to discriminate among the wide range of news media, it’s important for buyers and sellers of HRM BPO to know how market research outlets are slanted, which prognostications are bought and paid for and by whom with what intent, which are more objective, and which influencers and advisors have street credibility. This column has advocated using rigorous techniques for proving or disproving the capabilities of HRM software and BPO services, so you will not be surprised that I am also recommending that we use all of our critical thinking and interview skills to separate fact from fig newton in the marketing research.

Consider the grid charts that show the relative market positioning, by various criteria, of HRM BPO providers (or even HRM software vendors). The most obvious questions are who paid for this research and where did the data come from? Did the chart creators have access to every relevant deal? Did they base charts on the press release trail of such deals, when we all know that most deals are never publicized, let alone provide accurate details of scope and pricing?

If done on the basis of serviced lives, what kinds of lives were included (employees, contingent workers, beneficiaries and designees, qualified applicants, unsolicited applicants, etc.) and how were those lives counted (via estimates from the providers/vendors? via interviews with all of the relevant customers? via metrics produced automatically as a byproduct of the provider’s HRM delivery system)?

If done on the basis of estimated contract values, were those estimates obtained by an independent reading of the underlying contract, or via press releases or interviews with the relevant providers and their customers? I’ve even seen charts that are based on the interviewed perceptions of the providers by a less than scientific sampling of HR leaders. While all of this information may be interesting and useful, it’s important to get the story behind the story—and reputable market researchers will be glad you asked before you rely on these charts for guidance to the likely future performance of the providers you’re considering.

There’s some terrific market research data available on the HRM BPO industry that helps us understand deal trends (scope, pricing, terms, etc.), provider strengths and weakness, overall market size and growth rates, etc. Surveys of end-users, providers, sourcing advisors, and others can offer important insights into what has already happened and what may happen in the future. The best work is carefully presented as to definitions, data collection methodology, sample sizes and makeup, etc., and the best analysts are happy to answer your questions about how their data was obtained and compiled. But when these questions are side-stepped, dismissed, disparaged, or met with veiled hostility by the study authors, look elsewhere for quality research. And when sensationalist headlines are used to introduce the so-called research, I, for one, do not read much further.

Tags: Contributors, Enabling Technology

Related Articles