How #xAPI Enables Analyzing Connected Learning Beyond the LMS
Connected Learning is a modern pedagogical approach holding that knowledge and learning is distributed across a social, conceptual network. A LMS might provide discussion forum, wikis, file sharing services, but our students are often not even there. They tend to favor social media tools that they are using already. This affects what Learning Analytics (LA) we can carry out, we only see a tiny portion of their behavior. How can we handle the educational paradigm shift?
The Connected Learning Analytics toolkit aims to enable teachers to provide learning experiences “in the wild”. Rather than offering learning in a LMS, we obtain data about student engagement in carefully designed learning activities by interfacing with standard APIs for a range of online tools. The xAPI allows us to unify different data streams into one format and a controlled vocabulary across all platforms. Learner oriented learning analytics then helps us to encourage metacognition and reflection in our students about their own learning and social interactions. This talk provides an overview of the architecture, data structures and analytics that we currently use to leverage xAPI as a tool for enabling connected learning beyond the LMS.
Speaker: Dr. Kirsty Kitto
Here is the briefing of the talk.
Why Teaching in the Wild?
What motivates this project? Well, because learners are all there — social media platforms and blogs. If educators aren’t there, they might miss some important opportunities to understand and help their students. For example, at QUT we use BlackBoard, there are wiki and group discussion forums and more, but nobody is using them. Students are using Facebook, Dropbox, Google Drive etc.. If the group works are not working well, instructors won’t know about it.
Also I believe learning is heading toward a wide open space, really what we need is some loosely jointed solutions teachers can put together as they need instead of relying on a system providing all functionalities. And, life long learning can’t happen in a safe platform disconnected from the real world, we need to learn new skills through out our life. The project is really to help us push into a real world learning experience.
Take an example, in a capstone project, different students with different talents are responsible for different jobs – organizing, coding, documenting etc., how to know who is contributing what to the team? How to help them improve their group work skills in the right time? It’s all about a solution to help learners understand their own behaviors across multiple platforms. To do that, we use Learning Analytics (LA) – it’s to use data to improve learning outcomes. But the problem is that a lot of LA’s works are to understand the learners, but not for the learners. Actually very little data are returned to the learners at present. So this project aims to teach learners about their own behaviors and process, rather than writing reports to the institutions.
Connected Learning Analytics Toolkit based on xAPI
We use xAPI as a core component in this project. We build a toolkit to interface with standard social media APIs to pull data from several platforms, and make sure the data is stored using a standard format and vocabulary set. Once we’ve done that, we can use semantic technology and a number of different analytics techniques to that aggregated data. We can analyze data at a high level or zoom in to a granular level. And we can develop different reports for different roles.
Students sign up for data collection voluntarily, because we need to be respectful for students in social media spaces. Then data about a specific event (for example, a hashtag) will be pulled into LRS. Because the data follows a standard recipe we create, visualizations can be generated immediately, for example, a learner’s cognitive presence indicators in an event.
How Can We Get from xAPI to Learning Analytics?
One of the key questions we have to deal with this project is controlled vocabularies (recipes, profiles, Linked Data). Since we are trying to aggregate data from multiple sources. We have to put a lot of work into this. Another consideration is data portability across different LMS, institutions and other standards (e.g. IMS Caliper). For example, a learner’s participation in the elementary school can be put together with data from his high school activities, which can be carried into his college or even professional careers, and all the data still make sense.
Also we need much more sophisticated data extraction capabilities, so far we see a lot of counting and bar charts in xAPI world, but more complex analytics have been developed in Learning Analytics research field about discourse analysis, metacognition, reflection, group dynamics. Analyzing these higher level cognitive processes is possible only after we put a lot of efforts in data standardization, otherwise it will be a big mess. So at the beginning we spent a long time to create the recipe for social media(we are still expanding it). Maybe more than 6 months were spent to build the recipe, after doing that we are able to generate reports within a couple of weeks.
We Have Data Stored in a General format – Now What?
Step1: Pull data out of LRS
- xAPI specification does not provide a RESTful interface to perform aggregate queries (e.g., counts of verbs and object) against the statements in an LRS.
- LRS is really just a giant log file.
- Specification provides a URL to access queried statements (i.e. end-point) for 24 hours.
- Some commercial vendors have extended the standard with analytics enabled LRSs… but we are developing new LA solutions.
We are using PostgreSQL for easy querying over JSON.
Step2: Construct analytics and dashboards fit to your purpose
(Dr Kitto did a demo on the CLAtoolkit dashboards – video timeline 19:00 ~ 27:30)
Returning Data to The Human in The Loop
If you are using Machine Learning to classify learners you’ll find it’s very difficult, because learning is a highly contextualized activity. With different contexts, for example different classes or different languages, you are going to find very different behaviors. It’s quite hard to find a data set that you can train the machine to perform robustly in different scenarios. For example, cognitive presences is to classify how people are participating in a community of inquiry which is highly relevant to connected learning, there are presences of triggering, exploration, integration, and resolution. We found using a training data set that’s from Moodle forum(happened in Canada) to classify the real data set from Youtube(happened in Australia) didn’t perform well. So we build a “Active Learning Squared” tool, it allows the students to learn while the machine is learning based on the student’s interaction with the classifiers. In the process we can train students to think about their participation in the community, as well as generate a more accurate classification specifically for the context. In this way we can return data to learners themselves immediately to help them reflect and change their behaviors instead of saying we are not happy with their behaviors later.
Discourse Analytics – Analyzing The Nature of Interaction
We also try to look at the content and the nature of student’s writing. It can help students see what’s reflective and different levels of reflective nature can be highlighted. This tool is freely available (http://alasi.nlytx.io/), we plan to integrate it into our dashboard.
Start Going beyond LMS
If you like to know more about the whole project, please visit www.beyondlms.org, related information will be updated as the project proceeds.
About Dr Kirsty Kitto
Dr Kitto is the Lead Investigator for a project sponsored by the Australian Government’s office for Learning and Teaching Enabling Connected Learning via open source analytics in the wild, ID14-3821. This project is developing a Connected Learning Analytics toolkit, which uses xAPI to store data about student participation in learning activities designed using standard social media tools. More generally, Kirsty is a Senior Research Fellow in the Information Systems School at Queensland University of Technology and models the ways in which humans interact with complex information environments, paying special attention to the interdependencies between language, attitudes, memory and learning.