Making Sense Out of Learning Data
Why do we want to collect learning data?
Data without purposes is meaningless, what should the data strategy be?
We want to understand how learning impacts business results.
We want to know the correlation between L&D efforts and organizational key indexes.
We want to know how well the training improve employers’ performance in workspace.
We want to know more about learner’s behaviors in response to training courses so that we can improve the learning design.
We want to know more about learner’s profiles so that learning experience can be customized better, and learner’s learning journeys can be personalized.
Who has achieved mastery? Who is in danger of failing? What’s the best next step for the learner? How do we help people learn?
The Experience API gives us the opportunity to skip on measuring learning outcomes, and focus instead on performance outcomes. Isn’t that what we wanted to do with learning outcomes anyway?
If we can correlate the actual job performance data and the training data, we can determine the effectiveness of our training programs and measure ROI.
As we start to aggregate these activity streams across an enterprise, or even across an industry, we can start to identify the training paths that lead to the most successful outcomes. Or, conversely, we can identify the training paths that are leading to problematic outcomes.
Or, think outside the training box… What are the habits of really effective employees?
From Saltbox (creator of Wax LRS):
“The exciting things are going to be around seeing actions and behaviors happening at work in the systems in their workflow, and be able to track results,” “You can finally glimpse into how [employees] are performing on the job.” One of the biggest assets of Wax LRS is the technology’s near-real time APIs that can be exported to a company’s business intelligence visualization tool. Learning data can now be paired closely against job performance.”
“The xAPI captures many more learning experiences for analysis & visualization and provides the tools for improved learning experience design! Linearly it looks like this:
Current Learning State > Organizational Readiness Assessment > Business Outcome Measurement > Systems Readiness Evaluation > Resource Planning > Implementation > Iteration
Assisting people through the process clarified the path ahead and provided tangible next steps for moving forward.”
“Analysis answers questions, and while analytics questions are interesting, in being easily specified they skirt the Big questions, the ones we yearn to tackle, though they often play an important part in the fullness of a Big answer. “Who has achieved mastery?” “Which students are in danger of failing without a course correction?” “What steps will help prevent the loss of institutional capacity?” “Is my organization’s job preparedness improved by this training program?””
What are some Big questions you think you might be able to start answering with xAPI data?
Read more : Question Analysis in WaxLRS
The Turn : by Aaron E. Silvers
“no good will come of tracking everything”
This isn’t a one-and-done process… it has to spiral — it must be tweaked and iterated. As a designer and as an analyst, we’re interested in the truth: are our assumptions about what’s supposed to happen correct? If so, if we track some more things, will those also prove correct? If what we expect to happen doesn’t happen, what does that mean? Perhaps we need to look at other information; perhaps we need to design a different set of activities (or redesign the existing activities differently) to create the desired experience.
From Rustici Software (Co-builder of xAPI standard)
“I think what’s going to come from [LRS] data is the ability to really personalize learning more and more with intervention engines,” Delano said. Picture Netflix recommendations, only instead of “You May Also Enjoy Watching,” it’s “You May Also Enjoy Learning.” Furthermore, LRSs can certify where course learnings take place (whether on Lynda.com or in a MOOC). If teachings were regulated by the proverbial badge that everyone loves talking about, we could be on our way to a more systemized version of L&D across the industry.
Should learners own their own data? They may wish to protect data that they don’t wish to share: a fumbled test score or embarrassing seminar absences.
These concerns are loftier ones, and at times seem a bit more sci-fi than reality until the LRS gains a stronger foothold over the next decade. But at this very moment, the LRS is a valuable contribution to our increasingly knowledge-based economy. “If we can start to quantify an employee’s ability to learn, and capture that,” Rustici said, “I think that has massive potential to change how we look for jobs and how we hire people.”
“As an employer, I don’t really care about people’s resumes. I care about their ability to learn,” he said. And with the LRS, the data behind learners has never been more free.
Do data tell the whole story? Remember the movie “Moneyball”? In Moneyball, Brad Pitt plays Oakland A’s general manager Billy Beane, who turned his back on a long tradition of building baseball teams based solely on talent scout opinions. Instead, Beane built a winning team based entirely on statistics. Having less money than just about every other team in the major leagues, Beane led Oakland to win 103 games that year, including an amazing 20-game winning streak.
What’s the data model you are using in your field that xAPI data should make sense and connect with? (for example, HR XML’s schema for HR system interoperability.) How about K-12 area?