IBM uses data from web analytics to improve internal training practices.
By Lee Wasson and Mike Ticknor
How can we enhance measurements to improve the way we design, develop, and deliver learning?
With 85 percent of IBM’s training online, much can be learned from the way successful e-commerce sites—think Amazon and eBay—measure their online activities. A user-centric approach observes their customers’ online behavior. What are their customers searching for? What product descriptions do they read? They use metrics from advanced web analytics to improve what they sell, how they deliver, and what they market.
Web analytics is the measurement, collection, analysis, and reporting of internet traffic in order to understand and optimize web usage. After acquiring Unica in 2010, the IBM Center for Learning and Development started using its NetInsight to collect and analyze learner demand. From that data, IBM outlined its best use in four core areas, including improving skills development; emphasizing web design decisions based on metrics data rather than personal preferences; tapping into what leaners want; and tracking our communication campaigns.
How Does It Work?
Using Unica web analytics, IBM applied a simple JavaScript tag to varying IBM learning pages and sites. Once applied, an employee visiting a web page triggers the tag to send key information about the web page and the visitor. Out of the box, things like unique visitors, visits and page views, browsers used, time on page, and geography are captured.
The tags are customized to capture summary statistics on users’ IBM background, including their business units, their job role, and the markets they support. (To comply with privacy regulations, we do not capture specific emails or names.) During the first quarter of 2010, these tags were applied to key learning sites and delivery systems including Saba LMS, expertise management, Weblecture services and IBM’s career development site.
This process helped in four core areas.
1. Focus on program adoption to move skills development faster. With Unica NetInsight, IBM can now quickly do a side-by-side comparison of its different sites and programs. This allows leaders to explore conversations with program owners about sunsetting older sites and/or improving the communication cadence on sites that need more awareness. Learning professionals can also review the reports, creating a level of transparency across the organization on learning projects.
2. Emphasize design decisions based on web metrics data rather than personal preferences. New metrics help shift the focus to facts while neutralizing uninformed opinions. For example, in a global design review, an HR leader reported that IBM employees in Europe rarely use web offerings for learning. Therefore, “you cannot design web-based programs for us; they will not work.” However web analytics showed the team how past online web deliverables indicated strong usage in Europe. So the data helped move the conversation forward.
The analytics allow for deeper dives into ways to improve designs. For example, two weeks after the launch of the new IBM Competencies website, 6,000 IBM employees had explored the site. The most viewed competency was “embracing challenge,” with peak volume at 4 a.m. Eastern time, which suggested high usage in a growth market. The growth market visitors used collaborative sections more than major market IBM employees. The “embracing challenge” competency was first on the list and it got more visits than the other competencies. With this insight, IBM was able to build more usage in the other competencies by rotating the competency list order.
3. Discover insights on what learners want—what is hot, what is not, and what is missing. Unica shows what IBM employees are searching for, broken down by job role, location, business unit, or combinations of all three. This allows IBM to address areas with content gaps. For example, a popular search term did not have any relevant learning associated with it. So, a subject-matter expert suggested a match for this search term and it was added. “Search information” helps ensure that adequate learning is available for popular topics and helps IBM to focus resources on learning topics relevant to practitioners.
4. Track email campaigns on key learning programs. During the year,
managers are asked to work on development improvements based on feedback from their employees. Unica proved if our reminders were effective. The first reminder engaged 1,000 managers within a week.
Reporting Into the Future
Reports include breakdown by site comparison, business units, growth markets, key learning initiatives, and projects. Reports are also customized based on need. For example, a learning leader requested specific visits to the career site by the countries she oversaw. The analytics toolset is so it eliminated the need to build custom reporting within specific learning applications, which helps reduce costs.
Moving forward, there’s an opportunity to gauge informal learning by visitors to internal social media tools such as blogs, communities, and wikis. IBM is also building a learning demand dashboard that integrates these measurements with traditional metrics such as enrollments, completion, portfolio management, user feedback, and evaluation. It gives learning professionals a one-stop place to review demand for all learning activities.
Lee Wasson is learning technologies manager for IBM Learning and Mike Ticknor ia a learning developer for IBM Learning.
