Read Your xAPI Data in 5 Minutes by Leveraging Dataviz
Experience API (xAPI) is the latest learning standard built by the Advanced Distributed Learning (ADL) team (under the Department of Defense) and open community. It is an improved version of the Activity Streams standard for social media. A lot of properties are defined and added into the xAPI standard for learning experiences, which allows us to record rich and granular information about objects, results, or context (including content structure, time stamp, location, platform etc.). If necessary, Community of Practice (CoP) can create new vocabulary and extensions keys to record any relevant data needed for analysis. All these possibilities are powerful, but also create a problem — information overload. It is impossible to read xAPI statements line by line.
Dashboard Builds Situation Awareness
We advocate for xAPI because of the promise that it can carry rich and contextual data for us. But how can we make sense of the data efficiently? Raw data has no value in itself, only the extracted information does, but it is time-consuming to deal with huge volumes of data. Business intelligence (BI) dashboards are not a new concept: through well-designed data visualization as a cognitive agent, stakeholders or learners can “read” high-volume data within minutes. According to data visualization expert Stephen Few:
A dashboard is a visual display of the most important information needed to achieve one or more objectives that has been consolidated on a single computer screen so it can be monitored at a glance.
A dashboard can build high level situation awareness, and enable users to identify and focus on particular items that need attention. When necessary, users can update awareness of this item in greater detail, and determine whether action is required. If action is required, they can access additional information that is needed to determine an appropriate response; thus, they can respond very quickly.
K–12 schools and districts are also adopting such institution-level analyses for detecting areas for instructional improvement, setting policies, and measuring results. Making students’ learning and assessment activities visible opens up the possibility for students to develop skills in monitoring their own learning and to see directly how their effort improves their success. Teachers gain insights into their students’ performance, which can help them adapt their teaching or initiate tutoring, tailored assignments, and the like.
Fast Answers, Better Questions
Start your xAPI project by asking questions. An effective chart can answer many questions, but it can never answer all questions. A curious audience will read a chart and come up with new questions. These in turn create new answers and, over time, the community of these progressive processes becomes wiser.
When an answer comes back in seconds versus hours later, or overnight or next week, people will react differently. When stakeholders can get answers to questions in seconds, they can ask more questions or different types of questions. xAPI shouldn’t just be only about offering answers, it should be about creating better questions.
Data visualization pioneer Edward Tufte, referred to as “The da Vinci of Data”, said a well-designed graphical display should:
- avoid distorting what the data have to say,
- present many numbers in a small space, (to increase “Data-Ink Ratio,” considering our limited working memory)
- make large data sets coherent
- encourage the eye to compare different pieces of data,
- reveal the data at several levels of detail, from a broad overview to the fine structure,
- serve a reasonably clear purpose: description, exploration, tabulation or decoration
- be closely integrated with the statistical and verbal descriptions of a data set
Data visualization is a professional domain with lots of scientific research and experiments on how our perception, iconic memory, and working memory work to identify best practices. For example, great design leverages a person’s preattentive processing to transfer messages efficiently by using preattentive attributes (such as colors and shapes encoding). And the designs for dashboard layout and effective comparison is decided by how our memory works. There are many details to consider; a reading list is added at the end of this article for interested readers.
Read Your Huge xAPI Data in 5 Minutes
Visca, which means visual catch, is designed to visualize xAPI data. Before we built Visca, we’ve tried using BI tools, like Tableau, to display xAPI data. However, we found we cannot present information according to the xAPI statement structure. If we let users build or customize their charts, since they aren’t dataviz or xAPI experts and they may not be familiar with designing statements, it’s very possible the charts will be primitive types such as number counting on completion or other verbs.
Here is a short tour to show the design thinking behind Visca. Only sampled charts are included in the Visca demo version(enter through “Demo”), and there is no filter setting, role-based dashboard, or correlation setting. True optimized dataviz shall be designed according to use cases and user roles. (Check out full features list here.)
The Methodology for Processing xAPI Data
Our methodology starts with a hierarchy view of data metrics. The collection of low-level metrics such as time spent, time stamp, all possible interactions with UI, and how they are linked with context, content, or event parents through statement design should be standardized. (More explanation here) Therefore, we provide standard recipes for basic content types, as well as standardized visualizations. These are basic building blocks for customized dashboards or interlinked drillable charts. In xAPI statements, we can use several keys to indicate the structure relation, so it is leveraged in Visca to enable drilling down data and to make exploring data very intuitive. Case-specific vocabulary and recipe design can be added when designing the deployment for a project.
We makes it easy to implement xAPI for technology providers. Just follow our recipes to send xAPI statements according to use cases (activity types), or use Visca wrappers — in this case you just need to “fill in blanks” by following instructions and the wrappers can validate statements automatically. Wrappers reduce communication efforts and error rate. The vocabulary in Visca recipes follows the xAPI Vocabulary Spec. released by ADL this February. (Thanks to help from ADL)
Whether you send data to Visca or keep your xAPI data in your own LRS (inside your own firewall), you can call Visca dataviz APIs to embed the visuals inside your LRS or the BI dashboard you’re using. Of course dataviz can be used by themselves in Visca. Users should monitor their dashboards regularly, just like drivers need a dashboard to be in control of their journey. A well-designed dashboard will start from an overview of what is important and urgent to a user; then they can zoom in to linked details as needed and zoom out to stay aware of a whole picture.
Aggregating and mapping many xAPI statements of low-level experiences into higher-level experiences or patterns are necessary to let users stay on top, and to aid users as quickly as possible. (Recipe design is critical.) For example:
- When a learning goal is achieved with qualified performance metrics extracted from xAPI statements, then a badge is triggered.
- When the analytic engine detects a learner failed a test, it can send him personalized recommended activities according to his mistakes in the test. (Use Hook.)
Dataviz should be designed according to learning design patterns and structures, so the feedback to learning designers is efficient. A designer of performance support tool will want to know how the tool is used in all different of cases by different users. If complex learning model like 4C/ID is deployed, hierarchical dashboard design according to the framework is crucial for diagnosing problems (about design or learners) in each component or coordination between components.
Dataviz Enables Humans and Machine to Work Together
Machine Learning (ML) is a sub-domain of Artificial Intelligence (AI). In recent years the advance of technology has enabled AI and ML to make more significant impacts in real applications. They are already in our daily life, for example, blocking spam emails, detecting credit fraud, and calculating the best prices for retail stores. Salesforce just announced an intelligent assistant called Einstein in its CRM to help its users close deals smarter.
ML learns from data to become smarter without explicit programming, so ML algorithms are often called “learners.” ML can learn from all actions that humans take to respond to real-time dataviz, and can be trained to help humans (instructors or learners). We can embed recommendations from ML in dashboards, and collect feedback to improve the recommendation engine as well. Check out an example here.
In short, dataviz enables humans and machine to work together!
Reading list for data visualization design:
- Information Dashboard Design – Displaying Data for at-a-Glance Monitoring, by Stephen Few
- The Wall Street Journal Guide to Information Graphics, by Dona M. Wong
- Does Data Display Impact Instructional Decisions? from Pearson Research Network