Persona Development
CloudHQ, a cloud provisioning solution, offers teams the convenience of purchasing necessary resources and monitoring their expenses. The product team observed user complaints about the onboarding process within Slack. They engaged my expertise to formulate a research strategy aimed at determining the most effective approach for prioritizing the development of onboarding materials.
Problem Statement
CloudHQ users complained of UX issues in Slack channels. Adoption of this tool is key, because users will provision cloud services from unauthorized websites. Without a clear onboarding strategy, it was evident that this product would not make adoption goals.
Summary
The CloudHQ Usability Study leveraged a mixed methods approach (qualitative and quantitative) for measuring usability, user satisfaction and efficiency for the customer lifecycle - onboarding, product experience and engagement.
Purpose & Goal
Capture performance metrics such as time from launch of homepage to up and running service, Usability of User Experience (UMUX-Lite) and Net Promoter score (NPS) to judge product’s performance and user satisfaction
Find out how product delivers on it’s time-saving promise
Document pain points and moments of delight
Recommend experience improvements to meet best practices and increase user satisfaction
Methodology
Targeted CloudHQ users via Slack and email invites. Sent 518 invitations and achieved 90 survey responses.
Interviewed 4 CloudHQ users who were asked to recall their experience and answer survey questions verbally
Approach
Identify onboarding behavioral tendencies using Mindset Segmentation through surveying and interviews.
Outcome
Developed a UX Scorecard of the two most common journeys.
Created user mindsets for content creation of onboarding materials.
Identified quick wins for removing friction from the experience.
Usability Testing Findings
We’re able to correlate high usability scores to high adoption rates, Net Promoter Scores and customer satisfaction, which is why we examine these tenets of usability to ensure that the product meets usability performance goals.
Intuitive Design
3 out of 4 users stated navigation issues.
Efficiency of User
4 out 4 users stated that it was faster to use than other cloud provisioning services.
And all 4 stated that it took 15 min from launch of website to getting up and running on a service.
Ease of Learning
Learnability was mixed; 2 out of 4 stated that they have navigation and comprehension issues each time they used CloudHQ.
Subjective Satisfaction
All 4 interviewees stated that satisfaction with and preference for with CloudHQ.
Error Frequency
Poor quality documentation and lack of onboarding were cited as issues or all 4 interviewees.
Insight Themes
Onboarding
Findability issues with materials
"I stumbled upon the documentation."
Users are skeptical about the quality of the documentation.
"I don't trust our documentation or tutorials. They tend to be hit or miss."
Users gravitate to positive onboarding experiences for other products when asked to compare.
"I needed a QuickStart guide."
Guidance
The homepage is met with confusion
"(Homepage) is not super intuitive.There's way too many of them. It's like they shotgun them out. It's very busy. Almost too many options. If you go with a tile based solution, then go with tiles and have filters from a dropbox."
Users spend time trying to figure out how to complete tasks.
"It (managing account) took a little bit of time to figure out."
Engaged
The product’s benefits roadmap is not reaching users outside of a grassroots effort by product team and GIS meetings
"More communication on what the roadmap looks like and when new things are rolling out."
Users want a feedback loop to communicate bugs and feature ideas.
”What happened with Modi, is that it came out and everyone was like rah rah rah. But when they submitted bugs it went to never never land.” -
4 Different Modes for Onboarding
We documented four distinctive behaviors when it came to onboarding:
Jump In
Learning by doing. Just jumping into tasks without instruction.
"It took me about probably five minutes to get orientated because I just started clicking on stuff."
Toe In
Needing some instruction while testing out tasks.
"I needed a QuickStart guide."
Skim
Wanting some quick tips and brief instructions to skim before diving into tasks.
"Using gifs is really common on GitHub."
Navigate
Dive into instruction before setting out to complete tasks.
"Taking a sample application and how to move it to the cloud would be helpful (for on- boarding)."
These modes can help us understand what onboarding materials to build to meet these different learning needs.
Mindsets
We saw that educational needs and product values described by these participants fall under four mindsets and become solidified as our truths.
The goal for CloudHQ is to offer pathways for all four mindsets to be successful by developing up-to-date onboarding documentation, UI guidance and ways to self-serve.
Additionally, it is important to “do a little marketing” that clearly communicates what the product is and what it isn’t as some participants are unclear how comparable the product is to Modi.
Plus, we suggest having an ongoing open dialogue about the long term vision and how users can plug in with their feature ideas as the product matures.
Top Mindset
The Communicator
Communicators like different types of multimedia for understanding new technology. They spend 0 to 10 min onboarding before jumping in to perform a task.
This behavior is generally caused by their expectation for clear UI help within the product. When they do have to read long documentation, they generally scan for information. These people will go to Slack to get product questions answered first before self-serving with FAQs.
The Goal Setter
Goal setters learn by orienting themselves quickly in materials before jumping into performing tasks. They like materials that are scannable and easy to follow.
“I just need to glance at a QuickStart guide to get a feel before starting. I get a lot of information from a roadmap about what the product can do and can’t do.”
Example: Google Cloud Quickstart Guide
Example: Yotpo product walkthrough
The Gambler
Gamblers are willing to struggle if the payoff is accomplishing tasks without having to review documentation. They appreciate easy to follow workflows and interfaces that match their mental models and guide them through the experience so they don’t have to leave the channel to get instructions.
“I just jump in and struggle. I’m not concerned about what the product can do in the future as much as I need to know it can meet my needs today.”
The Planner
Planners like to become immersed in documentation. They prefer to understand the future vision before becoming true product advocates. Delivering varying content to them gives them different entry angles for understanding the product.
“I want to know that I am reading up-to-date documentation and not wasting my time. Before I will recommend to the time, I want to know what this product is and where it’s headed.”
Example: authO blog
Example: Monday.com YouTube guide
The Communicator
Communicators like different types of multimedia for understanding technology. They benefit from a wide array of different onboarding materials.
“Documentation is great to have but seeing someone perform actions is more beneficial for my learning. I’m always trying to figure out things from gifs on GitHub.”
Homescreen
The use of tiles to organize the blueprints were mentioned by 3 out of 4 people in the study as feeling “confusing” and “chaotic.” The filtering system on the right is viewed as not following the mental model of a tile design.
QUESTIONS
Are these all available to me?
Is EA Private Cloud, Modi?
Can I get my machines from Modi into this?
NOTES
Tiles are used to make features findable and provide contextual information, which is why they are used for many types of interfaces to improve scanning. The use of tiles and filtering systems in play have been described as impeding focus and increasing cognitive load so are not performing as intended.
Naming convention context (EA Private Cloud) with descriptions would aid in product comprehension.
Ideas for improvement have included a dashboard view homepage of active accounts.
"It's a little bit confusing at the beginning because there's a whole bunch of cookie boxes and filters and tiles and everything else."
Form Fields
While the learning curve (as admitted by some) is steep, 3 out of 4 of the interview participants stated that they had enough information to complete the form fields for provisioning a cloud service.
QUESTIONS
What do you put in these boxes?
What happens if I put something in that is inaccurate?
Why do I need to rename an order after I submit?
NOTES
Overall 3 out of 4 interviewees had no issues with filling out the form fields.
A follow up to this study could include a user research study which focuses on form field accuracy. This data could be used to inform UI guidance and onboarding materials.
The process still for requesting a Cloud account is like needlessly tedious and has some pieces in it where like you have to fill it a value, and you're like, I don't know what it means...like exist dev test or existing."
Priority Matrix
“Create a direct way of getting back in the workflow so it is one step instead of five or six. Put everything on one provision page.” - Chris Lessmann
"I think the home screen, in my opinion, should be the dashboard which you get by clicking on the little cloud HQ logo or some variation of the dashboard, it has some information about the servers or services that you have running has some information about your environments. But it also has information, the ability to click into your top use blueprints."
Team
Faith McGee (Research Manager), YiJu Chang (Researcher) and Ed Monterrubio (Designer)