Xbox Research conducts player research to understand behavior, preferences, and experiences across gaming ecosystems. But historical focus on Seattle-based players created a fundamental limitation: as Xbox expands into global markets like Southeast Asia, Latin America, and Eastern Europe, traditional research methods break down.
The organization faced a choice: accept the cost and time of hiring language-fluent researchers across time zones, or miss critical player insights in growth markets that represent billions in potential revenue.
This case study documents AURA Toolkit (AI User Research Assistants Toolkit)—a comprehensive suite of AI-powered tools I designed and developed to transform how Xbox Research operates globally. Through rigorous co-design with researchers, I built four integrated LLM-powered applications that eliminate language barriers, overcome timezone constraints, and dramatically accelerate the research-to-insight pipeline.
AURA Toolkit demonstration showing the four integrated tools in action.
The Problem
"I collect the data, send it to vendors, wait weeks, then finally get back something I can analyze. It feels like I'm constantly waiting."— Xbox Research Participant
Xbox Research historically focused on Seattle players. While comprehensive and high-quality, this geographic limitation meant insights primarily reflected US cultural contexts and gaming behaviors. As Xbox expanded into global markets—Southeast Asia, Latin America, Eastern Europe—traditional research methods hit fundamental barriers.
Language barriers created expensive bottlenecks. Conducting UX studies in markets like Thailand, Indonesia, or Colombia required hiring researchers fluent in local languages and dialects. Finding qualified UX researchers who spoke specific languages and understood gaming cultures often meant expensive vendor contracts or hiring delays of weeks or months. Translation costs added another layer: surveys, interview transcripts, and playtest feedback all required professional translation services.
Timezone constraints limited scheduling flexibility. Traditional moderated sessions demanded real-time coordination between Xbox researchers in Seattle and participants in, for example, Vietnam (15-hour time difference) or Brazil (varies by 4-8 hours). This meant limited scheduling windows, higher no-show rates, and reduced ability to conduct iterative studies where follow-up questions could be explored immediately.
Vendor dependency slowed research velocity. Researchers reported spending weeks waiting for vendors to clean interview data, identify key clips from video sessions, and arrange findings into analyzable formats. This created a bottleneck: researchers couldn't progress from data collection to insight synthesis quickly enough to inform product decisions at the speed Xbox's competitive landscape demanded.
Opportunity cost in growth markets. Without scalable solutions to conduct research across languages and time zones, Xbox risked missing player insights in high-growth international markets. Understanding how players in different cultural contexts experience games isn't just nice-to-have—it's essential for product decisions that could impact millions in revenue.
My Approach
Phase 1 — Discovery Research
8 contextual inquiries and 5 interviews with researchers to understand current workflows, pain points, and opportunities for AI integration across all stages of the research process.
Phase 2 — Synthesis and Design
Analyzed qualitative data to identify core problems, then designed 4 integrated AI-powered tools addressing language, timezone, data processing, and collaboration challenges.
Phase 3 — Full-Stack Development
Built production-ready applications integrating LLM APIs (GPT-4, Claude) with intuitive interfaces designed specifically for UX researchers' mental models.
Phase 4 — Rigorous Evaluation
Conducted mixed-methods studies including within-subjects comparisons, statistical analysis, and qualitative interviews to validate effectiveness with 35+ participants.
Phase 5 — Leadership Presentation
Demonstrated business impact (50% reduced vendor costs) and strong adoption (4 teams) to secure organizational commitment across Xbox Research.
Key Outcomes
AURA Toolkit delivered measurable improvements across efficiency, cost savings, and research quality.
Co-Designing with Researchers
While AI offers tremendous potential, I recognized that understanding exactly how AI could support UX researchers required deep engagement with researchers themselves—not assumptions from outside the discipline. To ground the AURA Toolkit in real researcher needs, I conducted 8 contextual inquiries (observing researchers in their actual workspace doing actual work) and 5 semi-structured interviews with researchers across Xbox Research.
Time Gap Problem
Researchers spent weeks waiting for vendor services to clean data, arrange findings, and identify important clips—constantly waiting for the next phase of their own work.
Geographic Constraints
Finding qualified UX researchers fluent in Thai, Indonesian, or Tagalog meant expensive vendor contracts or research gaps in critical growth markets.
Collaboration Friction
Distributed teams needed real-time discussion of findings, shared note-taking during analysis, and collaborative refinement—traditional tools fragmented this work.
AURA Toolkit: Four Integrated Tools
Based on the pain points discovered through co-design, I conceptualized and built four integrated AI-powered tools that address specific bottlenecks in the research lifecycle:
AURA Interviews
AI-assisted interview platform that conducts and analyzes user interviews language-agnostically and timezone-flexibly. Uses conversational AI to conduct adaptive interviews that follow up on participant responses with contextual questions. Handles translation automatically, allowing researchers to analyze interviews conducted in Thai, Indonesian, or any language without manual translation overhead.
AURA Surveys
Intelligent survey platform that adapts questions based on participant responses in real-time. Instead of presenting static questions to all participants, uses LLM-based logic to ask follow-up questions based on previous answers—creating personalized survey experiences that generate deeper insights than traditional one-size-fits-all surveys.
AURA Canvas
Visual collaboration tool that helps researchers organize and synthesize qualitative data in real-time using a "Human in the Loop" grounded theory approach. Researchers can arrange transcripts, survey responses, and behavioral observations spatially while LLM assistance suggests themes, connections, and patterns.
AURA Coder
Qualitative coding tool for collaborative analysis that allows researchers to develop and apply codebooks systematically. By integrating Microsoft Copilot, assists researchers in identifying codes from data, applying codes consistently across large datasets, and refining codebooks iteratively.
For detailed demonstrations of these tools in action, I recommend watching the video at the top of this page.
Evaluation
To evaluate the effectiveness of various tools across the AURA Toolkit, I conducted mixed-methods experiments. Below I outline the methods used to evaluate, and the key findings associated with experiments for each tool.
AURA Interviews Evaluation
Within-subjects experiment with 12 participants. Each completed two playtests—one using traditional unmoderated interviews, the other using AURA Interviews. Order counterbalanced to control for effects.
- Participants reported AURA Interviews were more efficient in providing playtest feedback compared to traditional unmoderated methods
- The adaptive questioning approach led to more in-depth responses from participants
- Overall preference for AURA Interviews due to their streamlined nature and improved user experience
AURA Surveys Evaluation
Within-subjects experiment with 43 participants. Each completed two surveys about their gaming experience—one traditional static survey, the other using AURA Surveys. Order counterbalanced.
- Participants found AURA Surveys more engaging and interactive compared to traditional static surveys
- Dynamic question generation led to more relevant and personalized questions for participants
- Preference for AURA Surveys due to improved user experience and ability to capture nuanced feedback
AURA Canvas Evaluation
Between-subjects experiment with 4 researchers from Xbox Research. Each completed a playtest analysis using either traditional methods or AURA Canvas.
- AURA Canvas facilitated better organization and synthesis of data compared to traditional methods
- Real-time collaboration features led to more dynamic discussions among researchers
- Preference for AURA Canvas due to its innovative approach to qualitative analysis
AURA Coder Evaluation
Between-subjects experiment with 6 researchers from Xbox Research. Each completed a qualitative coding task using either traditional methods or AURA Coder.
- AURA Coder streamlined the coding process compared to traditional methods
- AI-assisted coding features led to more consistent application of codes among researchers
- Preference for AURA Coder due to its efficiency and effectiveness in qualitative analysis
Organizational Impact
AURA Toolkit has been adopted by multiple teams across Xbox Research, fundamentally transforming how the organization conducts player research. The adoption didn't happen through top-down mandate—it emerged organically as researchers discovered the tools solved real problems in their daily workflows.
Operational Impact: Teams report completing studies in half the time previously required. Xbox can now conduct player research in markets previously considered infeasible due to language or timezone constraints. The adaptive questioning in AURA Interviews and Surveys generates more nuanced insights than static methods.
More Inclusive Research
Language-agnostic capabilities mean Xbox can understand player needs across diverse global markets, not just English-speaking regions.
Faster Decision Cycles
Reduced time-to-insight allows research to inform product decisions at the speed the competitive gaming landscape demands.
Innovation Catalyst
AURA Toolkit has become a catalyst for rethinking research processes more broadly across Xbox Research.
Conclusion
AURA Toolkit demonstrates that transformative impact in enterprise research doesn't require massive teams or years of development—it requires deep understanding of researcher needs, thoughtful application of AI capabilities, and rigorous validation through mixed-methods evaluation.
Beyond individual tools: The success of AURA Toolkit extends beyond the four specific applications—it represents a proof-of-concept for how AI can augment qualitative research expertise at scale. The 50% reduction in vendor costs and adoption across 4 teams don't just validate the tools' effectiveness; they demonstrate that AI integration in research organizations can create business value while improving researcher experience and research quality.
Future directions: The AURA Toolkit framework—co-designing with domain experts, integrating AI capabilities thoughtfully, and validating through rigorous evaluation—could extend to other research contexts beyond gaming. The approach demonstrated here offers a template for building AI-assisted research tools in any domain where qualitative expertise requires augmentation rather than replacement.
Skills & Methods Demonstrated
Contextual Inquiry, Semi-Structured Interviews, Mixed-Methods Research, Data Synthesis, Usability Testing
UX Design, Full-Stack Development, LLM Integration, Python/JavaScript, Prototyping
Product Strategy, Stakeholder Management, Change Management, Presentation Skills