Rating outsourcing providers is a great idea. Except when it isn’t.
By Elliot Clark
The mission of HRO Today magazine is to educate the HRO buying community about what does and does not work in HRO. We have deep questions about the credibility of the methodology of the Black Book of Outsourcing, the latest edition of which has just been published by Brown-Wilson Group. We are warning our readers not to use it.
This is not a competitive issue, although it’s true that their book attempts ratings similar to our RPO Baker’s Dozen. Instead, the critique is based on the described methodology itself.
First, I want to agree that some of the firms on their list are high-quality firms. The Black Book rated Adecco RPO number one. We feel Adecco RPO is a great company run by great executives. Mike Beygelman is our RPO contributing editor. There are other great companies among the rankings, but some listings expose deeply rooted flaws for which there can be no plausible explanation.
No Proof in the Pudding
According to their announcement, the survey was based on more than 26,000 responses. From whom? I do not believe there are 26,000 hiring managers to be surveyed. And, do hiring managers know whether the source of hire is a contingent labor provider, contingent recruiting provider, or RPO solution provider?
By comparison, our RPO Bakers Dozen surveyed the HR senior leadership of the RPO buying community. Surveys on HR services should be the province of HR leaders. HR leaders conduct internal surveys of the hiring managers and can accurately match solution and benefit. Bigger numbers, even if plausible, will still not make this list reliable.
So let’s look at the list. We already mentioned Adecco. Also on the list were Spherion, PeopleScout, Alexander Mann, Accolo, FutureStep, Bee Line RPO, Novotus, Select International, The Right Thing, RPO Worldwide, Talent2, Manpower, Kelly HRFirst, Yoh HR, Volt Services, Ancons International, Kenexa, Pinstripe, and Hyrian. Some of these listings require attention.
Spherion (which changed its name to SourceRight Solutions before the Black Book release), The Right Thing, Futurestep, Manpower, Kelly OCG (oops they changed their RPO brand too), Kenexa, Pinstripe, and Talent2 are all major players. Accolo is an excellent mid-market player, but as you get to Novotus, Select International, and Beeline you are looking at good companies in a different category of scale from the first group.
After that, you fall off the cliff, unfortunately, and here is where the real problem exists.
Hyrian has been out of business nearly a year. Its talented sales leadership moved to SourceRight. How are they on the list? Volt Momentum is practically closed as well. Their website lists a 2006 press release as recent news along with their attendance at Taleo World 2007. The website might still exist as it is part of the Volt Temp Staffing, but it has been dormant as a market force. RPO Worldwide was part of iGate, ran out of steam, was pulled out of dormancy by the ITO player MasTech, and might have some project business. Yoh HR could not produce enough client references to qualify for our survey. They have one big client and some project work. They made our survey last year but this year could not. Ancons International is a small offshoot of a company that helps place foreign graduates in U.S. university programs (www.ancons.com). It might be a good fledgling operation but it is hardly a market mover.
To be fair, I am not the first person to call out Brown-Wilson Group. According to Business Week July 2008, “Claes G. Fornell, a professor at the University of Michigan, Ross School of Business who specializes in customer satisfaction surveying, says Brown & Wilson’s methods aren’t sound. First, he says, the firm can’t be sure all the people who respond are qualified.”
The publisher defended itself to Business Week: “Brown and Wilson argue that their results are fair and valid because they check the credentials of many of the participants.” (Italics added). This is vague assertion, not methodological transparency. Concluded Dr. Fornell: “You’d be better off not doing anything than doing a survey like this.” The article also states, among other troubling anecdotes, that the publisher’s office is a UPS store lockbox. Please read the article at http://www.businessweek.com/magazine/content/08_28/b4092084064809.htm?chan=top+news_top+news+index_news+%2B+analysis.
Brown-Wilson’s process is incomprehensible. The numbers are not credible, and the list composition is suspect. I have never said our methodology is the only viable one, but buyers can rely on our research team doing a quality study. I feel the need to warn the RPO buying community about reliance on this survey. As always, we invite Brown-Wilson’s response and your comments, and we stand by our mission to provide readers with the best information about HRO.
