Employee EngagementLearning & Development

Lesson on Measurement

A new report offers tips on tracking the impact of informal training. 

By Debbie Bolla 
Long gone are the days when a company’s learning platform solely consisted of instructor-led, classroom-based programs. New informal methods have popped up over the last few years—virtual classrooms, videos, smart phone apps, social media activities—forming one diverse arena for training.
“With the explosion of access to the Internet through social networks in the last five years, knowledge management has lead the industry to think of ways to redesign how we train,” explains Josh Bersin, CEO and president of the research firm Bersin & Associates. “We can set up ways to access experts, share information, and contribute information so others can learn from it.”
However, some of the technology- and media-driven platforms have been criticized because it is difficult to measure their return on investment (ROI). Some executives are left wondering how effective the methodologies are. Formal training programs are typically measured via postmortem surveys or testing, while informal learning environments are not conducive to tracking impact. Or are they?
Bersin’s firm has explored executive strategies to track informal training effectiveness in a new study, The Impact Measurement Framework: An Update for the Measurement of Informal Learning. The firm defines informal learning as the combination of formally designed programs with on-demand, social, and embedded learning activities, each described below.
Learning On-Demand: a wide variety of references, job aids, online videos, podcasts, books, articles, and other types of supplementary materials that aid in an employee’s learning or job performance. They might or might not be in electronic form.
Social Learning: online and in-person social experiences, including the use of collaboration systems, blogs, wikis, social networks, question-and-answer databases, group projects and discussions, and coaching programs.
Embedded Learning: institutionalized processes that drive learning in a tacit way, including customer advisory boards, rotational assignments, development planning conversations, and employee feedback.
Bersin & Associates’ research has shown that today many companies still don’t understand the power of informal learning. “You can unlock why the successful employees are so successful by sharing what the top people are doing, how they use their skills, and what they know,” says Bersin.
Even still, organizations strive to justify the programs’ costs. While the firm reports employee knowledge-sharing and collaboration as two of the most important elements of organizational performance, it understands the real need to present and track its ROI.
Model for Informal Learning
The research firm incorporated three elements into its framework that can be used to track impact—contribution, feedback, and activity. 
Contribution. The first and perhaps most important measure of the effectiveness of informal learning is how widely and deeply people are contributing content. Take, for example, a knowledge-sharing portal that supplements a new-hire training program. Employees are asked to contribute their own experiences and findings about how to best become integrated into the company. One of the most important measures of success in this situation would be level of contribution. The report offers these considerations for management:
• How many people are contributing content?
• What percent of the total audience is contributing content?
• How frequently are people adding content?
• What are the demographics of the people who are adding content?
These types of measures can show how well the system is bringing value. If content is being added, then employees feel empowered and enabled to share information. The system must be easy to use, and there must be a comfort level with exchanging ideas among peers.
If the results show that few people are adding content, then the informal learning environment is failing. Some possibilities as described in the report:
• There aren’t clear guidelines on what can be contributed.
• The system is hard to use.
• The management culture doesn’t promote sharing and feedback.
• The program is intimidating or is misaligned.
• Management hasn’t marketed the need for contributions by learners.
Finding out who is contributing content and why will provide insight into why the informal learning program is working as well as how to improve it. It will also show the types of employees in the organization who are engaged in the program—and those who are not.
Feedback. The report also suggests enabling a five-star or other rating system to measure informal learning elements. As companies such as Amazon.com, eBay, YouTube, and others have found, success or popularity of an initiative can be determined by letting employees rate it. Make it easy for people to provide feedback, and make it public. Let employees add open comments to materials so that management and peers alike can understand why something is useful and why it is not.
Unlike the public Internet, internal informal learning environments should not let employees remain anonymous. So when employees provide feedback, they are reflecting not only their own opinions, but their values too. This prevents employees from randomly rating things high or low, and it encourages employees to think carefully as they offer feedback. As crowd sourcing has proven, a feedback loop can deliver a critical learning experience. Management can get in the middle of the workforce learning environment, and employees can teach each other.
Ways to measure feedback include:
• Volume of feedback provided
• Percent of positive and negative feedback
• Percent of feedback distributed by audience type
• Areas in which feedback is high or low
• Types of feedback
Case in Point
According to the study, companies such as Intel, NetworkAppliance, and The Cheesecake Factory offer employees open, easy-to-use sharing portals through which they can upload videos, audios, and documents that show peers how they have solved various problems. Such environments have proven to be successful in developing new content, empowering employees to help each other, and creating a feedback-rich and dynamic learning experience.

British TeleComm uses information portals to helps its employees—and its bottom line. It was impossible for the firm’s tech employees to have knowledge on how to fix every piece of equipment they distribute. So the company was losing money on return trips to jobsites. To curb these costs, the company started a knowledge-sharing portal of YouTube-like videos. Technicians would upload videos illustrating how to repair certain equipment that others could refer to in the future. This informal learning environment saved the company between $2 million and $3 million just in the elimination of return trips to jobsites.
Activity. The study notes that the frequency of use of informal learning elements is another avenue for measuring effectiveness. How many people downloaded a certain PDF and how frequently? How often does an individual use or view a certain tool or assessment?
Bersin & Associates applies this approach to its own research library. The firm carefully tracks the download history of all research members across all reports. Activity is tracked by time of day, by topic, and by research type. This shows what research is most interesting and useful—and what research was hard to find or of lesser value.
The study explains that while the activity will not necessarily tell you the ROI of an informal learning element, it will implicitly tell you how useful, valuable, and convenient it is. So in the informal learning area, while activity might seem tactical, it is one of the most important measures. Organizations should put in place software and systems to help track activity on a regular and granular basis.
Measuring the ROI
Is it possible to find the cost justification of an informal training environment? The firm’s research suggests the following options:
1. Find places in which informal learning is already taking place. Before creating a major project to compute the financial return on a set of job aids for collaborative environments, look around for places in which it is already happening. In nearly every company there are work groups, which have already adopted sharing, job aids, knowledge databases, lists of experts, and other forms of informal learning. Take some time to seek out these job aids and listen to the people who are using them. They will provide highly valuable anecdotal information about how much time, money, and effort these have saved. This information can be used to create a model for other informal learning efforts and future implementation of others.
2. Understand why you are being asked for ROI. Who is asking for ROI? In many cases, the underlying reason is fear of the unknown. The cost of social and informal learning is often near zero—there are a wide variety of free and public domain knowledge-sharing environments—so the real cost that management worries about is the loss of control by the legal affairs, management, or the learning and development (L&D) department.
Bersin says, “When you work in a company that has a lot of online communities, knowledge sharing, and a culture of learning, the employees know that it’s valuable, and the cost is next to nothing.”
3. Look at existing measures in your workplace that are hidden costs today. Informal learning is not usually cost-justified by helping people learn better, but more often by helping employees save time, reduce errors, improve teamwork, and increase employee engagement and retention. There are a variety of well-known problems that can’t be solved by formal training. They are often informational in nature and require access to expertise in order for a solution.
The Bottom Line
Bersin & Associates’ research has shown that in today’s highly connected environment, employees of all demographics understand the power of informal networking and learning on-demand. While some organizations might have legal or compliance challenges preventing open sharing of information, there are parameters than can be followed to make it possible. All learning practices have been traditionally difficult to measure, but enlisting those suggested is a step in the right direction.
Building an Informal Learning Culture
With today’s fast-paced business environment, traditional learning structures have started to become unrealistic for on-the-go employees. By delivering training directly to staff and allowing them to learn at the time and place that suit them best, informal or just-in-time learning has become an effective training technique. As a result, organizations are searching for ways to support this type of learning in tandem with more formal, blended training programs. SkillSoft, an elearning solution provider, suggests these tips on how organizations can build an informal learning culture:

• Know your audience. Build your solution around your employees’ needs, working conditions, familiarity with technology, learning styles, and how they use information. This will encourage informal learning during daily work life.
• Rethink the roles in L&D. As leading analyst Josh Bersin states in his blog, “We-learning will require a focus on facilitation, information architecture, and audience analysis—not just learning design and development.”
• Leverage your existing processes and systems. “Start by identifying where employees already naturally congregate (in the virtual world) and recognize them as ripe for igniting a community with social tools,” suggests John Ambrose, SkillSoft’s senior vice president, strategy, corporate development and emerging business.
• Good things come in small packages. Provide information in portable, bite-sized chunks, such as an audio book summary that an executive can listen to on an mp3 player or a five-minute video to watch on a tablet.
• Promote, promote, promote. Become an informal learning champion and encourage the use of your resources among employees.
• Keep an eye on things. Monitor and evaluate your informal learning program to see which elements are working effectively and which may need some adjustment.
Success and a Social Learning Program
In the last few years, the elearning industry has experienced a shift in its learning leaders’ focus toward integrating collaborative and social learning into traditional learning models. Below are tips and tricks from elearning solution provider SkillSoft on how to avoid common pitfalls when implementing a social learning platform.

• Recognize that social learning is a three-legged stool. Consisting of a technology platform, vibrant community, and great content, your program needs to include a plan for all three components.
• Create a clear picture of available resources. Offer guidelines that outline appropriate use of your company’s social learning tools.
• Make sure your social learning platform includes a way to identify, search, and find your internal experts. The knowledge captured in your system will be of little use if knowledge leaders can’t be pinpointed at the moment of need.
• Cater to multiple generations. While your millennials probably see social learning as second nature, your gen-xers, boomers, and traditionalists might need some guidance to get started.
• Understand that a top-down approach won’t work. Social learning needs to come from the bottom up.
• Emphasize current learning programs. Build on the sharing that’s already happening within your organization. Be sure to recognize and encourage those who are already working collaboratively.
• Keep yourself informed. Like all technology, social learning programs are always changing. It’s important to keep yourself up to date on what experts are saying on today’s social learning strategies.
 Learn more about SkillSoft’s free We-Learning workshop series here.


Tags: Engaged Workforce, HRO Today Global, Learning

Recent Articles