A leading global technology company wanted to ensure that the onboarding flow for its gaming device delivered a cohesive, easy-to-follow experience that prepared users for early success. Through 12 in-depth interviews with participants across diverse demographics, we evaluated how well the NUX taught essential concepts, built user confidence, and met expectations for ongoing education. Findings revealed opportunities to improve the narrative and visual coherence of the onboarding, strengthen skill-building and comprehension, and provide continuing, contextual education through smart tutorials, pop-ups, and in-the-moment support.
NUX Evaluative & Usability Testing
IN-PERSON TESTING
Research Objectives
The client was looking to deliver a visually and narratively coherent end-to-end onboarding flow that prepares users for their first gaming device experience. The onboarding should:
Teach users about the basics of how to use the gaming device.
Introduce users to how it works.
The goal was to evaluate how well current designs of the NUX prepare users for their initial experiences.
Key research questions included:
How cohesive does the end-to-end onboarding experience feel overall (visually, narratively, etc.)?
How prepared do users feel to get started with their device and to have their first experiences?
What are participants’ expectations in terms of continuing education (beyond an upfront tutorial)? What do they expect to learn more about, when, and how?
Following the device NUX flow, how well can users comprehend key concepts that are needed to understand how to navigate the system?
How well does device NUX teach users the skills that they need to perform usability tasks?
Research Approach
Participants
n=12 participants
mix of intended use cases for and experience of the gaming device
distribution across age, gender, ethnicity
Timeline
fielding over 1 week
deliverables after 2 weeks once fielding ended
Methodology
12, 2 hr In-depth Interviews (IDIs)
My Services
Project Set-up
Established product knowledge through exploration and research to better design and interpret study findings
Developed a session guide and usability tasks tailored to the project’s objectives
Research execution
Moderated IDIs in-person, allowing for:
Gaming devices to be provided for non-users
Tracking of eye and hand movements
Analysis & Reporting
Comprehensive qualitative and usability data analysis
Synthesized findings into clear deliverables with actionable insights and recommendations based on objectives
Analysis & Deliverables
The IDIs provided insights that informed:
Short-term changes to improve the usability and coherence of the NUX flow
Longer-term changes for the client's end-to-end NUX strategy
One example insight & a few recommendations were:
Almost all participants wanted continuing education available that would supplement upfront learning. Participants expressed wanting some combination of: contextual pop-ups (ideally “smart” ones), on-demand tutorials & tips, and an AI assistant.
Continue to pursue the direction of providing contextual, just-in-time education to teach users skills when they need them most.
Explore creating “smart” contextual education that appears when a user is detected to be struggling and/or has not used a feature in a long time.
Improve the discoverability of Help & Tips by providing a shortcut to it.
Explore incorporating an AI assistant into the OS that can provide in-the-moment support.