OPM Viz: Visualization Systems for Student Reflection
Designing and evaluating a system to help students reflect on their gameplay and learn from peers in the educational game 'Parallel'.
TL;DR
The Problem
Educational games generate data, but it's hard for students to visualize their own data to support reflection.
- Existing tools lack features for self-analysis and peer comparison
- Gap in bridging raw gameplay data with deep learning
- Students need better ways to understand their strategies
My Approach
Design-Based Research combined with UX methods.
- Instructor focus groups and student playtesting
- Thematic analysis to identify core user needs
- Iterative design and development process
The Solution
OPM Viz: An interactive visualization system for the 'Parallel' game.
- Enables students to compare gameplay with anonymized peers
- Provides metrics, synchronized replays, and reflection prompts
- Integrates seamlessly with educational game environment
Key Results
Successfully facilitated student comparison and learning.
- Students identified inefficiencies in their approaches
- Sparked curiosity and supported self-assessment
- Published at ACM CHI 2024, contributing to learning sciences
The Challenge: Bridging Gameplay and Learning
Educational games like 'Parallel', which teaches parallel programming concepts, generate vast amounts of player data. However, a significant gap exists: how can this data be effectively visualized to help students reflect on their own learning process and understand the strategies used by their peers?
Existing tools often lack features specifically designed to foster reflective learning. Our core challenge was to design a visualization system that moves beyond simple data presentation to actively guide students in analyzing gameplay, comparing strategies, and ultimately deepening their understanding of complex concepts taught through the game.
Understanding the Users: Instructors & Students
To ensure our solution was grounded in real needs, we employed a Design-Based Research approach, incorporating UX methods to understand both instructor goals and student expectations.
 (1).png)
Instructor Focus Groups: How Experts Facilitate Reflection
We conducted workshops with two parallel programming instructors. They analyzed anonymized student gameplay recordings from 'Parallel', identifying different strategies, ranking performance, and pinpointing moments ripe for student reflection.
Key Insights from Instructors:
- Chunking Gameplay: Instructors analyzed playtraces in segments to understand student actions (e.g., "Steps 2 and Step 3, is where the student is trying to figure out the Switch...")
- Spatial Awareness: They connected gameplay actions in specific game areas to programming concepts
- Comparative Analysis: Instructors frequently compared different students' strategies to highlight efficiency or alternative approaches
- Performance Metrics: Ranking was based on core parallel programming concepts like identifying critical sections and minimizing their size
Student Playtesting & Personas: What Learners Need
We observed 10 students playing 'Parallel' and followed up with semi-structured interviews to understand their struggles, successes, and desires for a support system. Thematic analysis led to key insights about student needs.
Key Insights from Students:
- Desire for Efficiency Improvement: Players want to understand how to make their solutions better and faster. They need ways to identify and learn from more efficient peer strategies.
- Need for Easy Gameplay Comparison: Players want to compare their approach with others from different viewpoints (high-level overview vs. detailed step-by-step).
- Seeking Guidance and Suggestions: Players appreciate hints or alternative solutions, especially when stuck. They also value exploring different valid approaches after completing a level.
Designing the Visualization System
Guided by insights from instructors and students, and utilizing Shneiderman's Visual Information-Seeking Mantra ("Overview first, zoom and filter, then details-on-demand") as a framework, we designed and developed the OPM Viz system.
Key design goals included:
- Providing an overview comparing a student's performance against the community
- Allowing users to filter and select specific peer playtraces for deeper comparison
- Offering a synchronized, detailed side-by-side view of playtraces
- Incorporating metrics relevant to parallel programming efficiency
- Suggesting moments for reflection based on common patterns or deviations
.png)
The Solution: OPM Viz
OPM Viz is an interactive visualization system designed to integrate seamlessly with the 'Parallel' serious game. It empowers students to reflect on their gameplay, compare strategies with peers, and gain deeper insights into parallel programming concepts.
The system allows students to:
- View their own performance metrics
- Compare their solution path against anonymized peers
- Filter peers based on performance or specific strategies
- Replay their own and selected peers' games side-by-side
- Identify key differences and potential areas for improvement
Evaluation: Does it Foster Reflection?
To assess the effectiveness of OPM Viz, we conducted an evaluative study with 8 student participants. They used the system after playing 'Parallel' while thinking aloud, followed by semi-structured interviews.
Our analysis focused on how and why students used the system and whether it prompted the desired reflection and learning.
.png)
Key Findings from Evaluation:
- Facilitated Comparison: Participants actively used the system to compare their strategies with peers, often expressing surprise or gaining new ideas
- Identified Inefficiencies: The visualizations helped students pinpoint specific parts of their gameplay that were less efficient than others'
- Sparked Curiosity: Exploring different solutions through the system encouraged students to think more deeply about the underlying programming concepts
- Supported Self-Assessment: Participants used the comparative data to gauge their own understanding and performance relative to others
- Highlighted Trade-offs: Observing different strategies allowed students to understand the trade-offs between various parallel programming approaches
Overall, the evaluation indicated that OPM Viz successfully prompted reflection and supported learning by making gameplay data accessible, comparable, and insightful.
Impact and Contributions
This project demonstrates the potential of integrating player-facing visualization systems, informed by Open Learner Model principles, into serious games to enhance learning.
Key Contributions:
- Bridging OLMs and Games: Introduced an Open Player Model approach, adapting OLM principles for the dynamic context of serious games
- Learning-Focused Visualization: Designed and validated a player-facing visualization system prioritizing reflection and learning over purely aesthetic goals
- Enhancing 'Parallel': Provided a novel tool for the established 'Parallel' research platform, enabling new avenues for studying game-based learning
- Published Research: Shared findings and system design with the academic community via publication at ACM CHI 2024
The positive evaluation results and academic recognition underscore the value of this user-centered approach to designing learning support tools within educational games.
Conclusion
The OPM Viz project successfully demonstrated how thoughtful visualization design can bridge the gap between gameplay data and meaningful learning reflection. By centering the design process around both instructor expertise and student needs, we created a system that not only presents data but actively facilitates the kind of comparative analysis and reflection that leads to deeper understanding.
This work contributes to the growing field of learning analytics and educational technology, showing how UX research methods can inform the design of tools that truly support learning rather than simply displaying information.