Voice of Customer Overview
Electronic Arts was trying to mature their process of collecting customer feedback for their Game Creation products.
I took a macro-to-micro approach to satisfy short-term business needs e.g. capturing baseline metrics via surveying then interviews before diving into the nuances with 1:1 interviews.
My role was to design the strategy and surveys, align with the myriad of product managers, set a timeline and manage the project.
Problem Statement
Currently, Electronic Art’s IT does not not have a systematic process for capturing customer feedback thus causing data gaps across the Game Creation portfolio prevents a cohesive view the portfolio’s performance.
Likewise, the ad hoc research efforts including survey and interview activities shine light into small pockets of insight but do not illuminate the entire picture.
A Voice of Customer program provides product teams with a continuous stream of feedback that is non-intrusive, partners with customers for feature development and meet business objectives with a data first approach.
Summary
A VoC program captures, benchmarks and reports customer feedback (willingness to recommend, likes, and dislikes) about a customer journey. Two business objectives are achieved by capturing customer feedback; benchmarking and continuous improvement.
Our Voice of Customer roadmap began modestly at a benchmarking phase to produce a scorecard for products within the Game Creation portfolio.
Our goal is to achieve continuous top-line visibility, such as trending data for NPS, systemic problems, or initiative-specific metrics to address real business challenges with a more mature program.
Purpose & Goal
By the end Q1 the goal is to meet business objectives with a continuous flow of data that is used for the creation of new features, new products and better product experiences.
Approach
Start small to go big with a straightforward baseline survey including the Net Promoter Score question and Usability Metric of User Experience-Lite two part questions.
Methodology
Send employees who regularly use Game Creation products an invitation to take a survey and/or be part of a 1:1 interview. Our goal is to reach a statistically significant sample size based on usage per products: surveys to reach a 95% confidence level and interviews reach a saturation point.
Survey Strategy
Gain a better understanding of how demographics can influence preferences.
Capture “in-the-moment” responses to design iterations.
Trigger surveys after specific behaviors to gain contextual data.
Track trends and patterns in the data so you can see the big picture as well as the specifics.
Understand causality and what impacts loss and cost savings.
Game Creation Portfolio
Jukebox
1,000 users
Game creation application available to game creators with development workstations, can access remotely.
Glaas
4,000 users
Providing wiki, issue-tracking and continuous integration and deployment pipeline features, available to you through Jukebox without VPN
JaaS
11,000 users
JaaS is a combo of Jira itself (data center addition hosted in cloud [AWS]) and separated delegated administration service.
PerforceOne
5,000 users
Built out an enterprise platform around Perforce, a third party tool used by AAA titles (games) by most development teams who are accessing code.
Artifactory
800 users
How EA shifts code and data in small packets of functionality.
GameVault
100 users
Like YouTube or Netflix, browse content, consume content, submit content. Content = game development artifacts.
Shift
17,000 users
A cloud based storage for all games during release cycle that can enable game in any EA location on PCs, consoles, mobiles, cloud gaming. Develop any title, any time, anywhere.
3 Phased Approach
Survey development for each product, working with each product team to ask questions for benchmarking + getting behavioral data.
Interviews with customers to dive into the themes surfaced in surveys.
Experience management with listening posts stood up in MVP.
SURVEYS
Surveying users of Game Creation products help us achieve a baseline benchmark.
Example Questions
Frequency of use
In a given month, how often do you use X?
NPS
On a scale from 0 to 10, how likely are you to recommend X to a colleague?
UMUX-LITE
X capabilities meets my requirements
X is easy to use
Open response
INTERVIEWS
Using survey data to create interview guides allows us to delve into nuances and understand scoring.
Example questions:
What about the onboarding made it difficult to get up and running in the tool?
Do you have issues understanding all of the features?
Have you used comparable tools in the past?
Listening Posts
These different types of listening posts were built to capture customer feedback. By collecting active and passive surveys in one place and being able to trigger ticketing, product is able to quickly see the whole picture and prioritize work efforts.
Survey
Interviews
Help Channels
Feedback Loop
A customer feedback loop creates a way to bring user needs into product development. By using a passive survey embedded in the experience or triggering a survey model, we were able to gain contextual feedback. As a feedback loop implies, closing it is essential so that users feel heard. An email or a Slack message let us close the loop with customers so they understood that their time to deliver the feedback is valued.
Feedback pop up
It is non-intrusive to the experience
Allows users to add their feedback for product feature ideas and experience updates
Survey Modal
Allows users to provide customer satisfaction feedback plus open responses
data = power
Outcome
Budget for a consumer analytics tool
Created a matrix to help identify investment opportunities
Helped identify new features
Leveraged findings for product roadmapping
Saved IT hundreds of thousand of dollars from leveraging outside agencies to conduct research activities
Team
Faith McGee (Manager), Katie Domines (Research), YiJu Chang (Research), Michael Edwards (Design)