Pop social
The Challenge
Pop Social Inc founded by a student at UT Austin was in the process of releasing their app into the market and wanted us to run usability tests and provide feedback before it went live.
Client
Pop Social Inc
My Team
Mavis Klemcke, Ryan Wolter, Christian Onuogu, Shravya Kolavara
My Responsibilities
Heuristic Evaluation, Moderated 3 tests, Notetaker for 2 tests, Secondary Research
My Role
UX Researcher
Our Advisor
Eric Nordquist
Timeline
10 weeks
.png)
Let's get some context
The application aims to change the paradigm of people socializing in college. The startup reported that 64% of students feel lonely during their college years. This app wants to provide a platform for students to connect with each other based on shared interests alone. This is achieved by locking the persons name and picture until a certain number of interactions between the users, thus creating more compatible meetups.
Secondary research
The scope of the research was broad and delved into the problem space the app was trying to solve. The research proved to validate some of the research provided by the company and looked beyond to find solutions that would help improve the application.
40% of Students Report that they feel very lonely in college.
This can lead to an increased likelihood of depressive symptoms among other negative effects
Go to Source
A good way to make new friends is by finding shared interests.
Go to Source
Many students cite the relative difficulty of making connections on campuses of hundreds or thousands
Go to Source
Subject Matter expert
Before diving into the usability test, we wanted to get some advice and feedback from experts in order to better understand the problem space and some of the content the app uses. We spoke to a counselor from the Center for Mental Health at UT Austin.
These were some of her insights into the application:
Though the app can help with depression symptoms, it should be framed as a social communication app rather than a mental health app.
The app can't replace actual treatments for depression such as therapy, so at best it can serve as an aid
A mental health based framing might also turn-off potential users who would believe that this app isn't targeted at them
A potential compromise could be to display the number of a mental health crisis line to potentially help users suffering from these issues
Heuristic Evaluation
Evaluation Measurements
I've used the 10 usability heuristics by Nielsen Norman Group to conduct the evaluation .
The evaluation provides a severity level to each of the problems to indicate which problems need to be tackled first and in what order might they want to proceed with to make changes.
Crucial : Cannot function unless Fixed.
Major: Fix Now
Minor: Fix Soon
Trivial: Fix when Possible
Task #1 Onboarding
1
7
18
Issues Found
3
Crucial
Cannot function unless Fixed.
7
Major
Fix Now
4
Minor
Fix Soon
2
Trivial
Fix when Possible
Task #2 Connect With People
5
6
5
Issues Found
3
Crucial
Cannot function unless Fixed.
2
Major
Fix Now
1
Minor
Fix Soon
0
Trivial
Fix when Possible
Task #2 Edit Your Profile
Issues Found
0
Crucial
Cannot function unless Fixed.
2
Major
Fix Now
4
Minor
Fix Soon
0
Trivial
Fix when Possible
who is the competition?
In our competitive analysis we compared 4 apps to the Pop Social app. These were the key insights and recommendations.
Making Connections
Starting conversations can be difficult for users who want to meet new people.
Other apps encourage friendship with ice breakers, questions, or tasks
Personality Sliders
Personality sliders are subjective and difficult to comprehend. Other apps have users answer questions. It is more effective method for users to determine their personality.
Interests and Communities
Other apps provide more flexibility to users by letting them choose as many interests as they would like and make connections with more specific choices for smaller communities.
Editing Profile
Other apps allowed users to update their profile interests easily. This allows users to change their profiles as and when they interests and goals evolve in real life.
Usability test
User Interview Process
Survey
Surveyed students using Google Forms to identify candidates for interviewing
Technology
Using Zoom, we performed remote testing by having users share their screen.
Post Task Interview
After all the tasks were completed, we walked users back through the tasks with us driving, asking specific questions.
Selection
8 participants were selected to be interviewed producing a mix of demographics.
Tasks
Each User performed 5 tasks.
- Creating an account
-
Reading new messages sent from other users
-
Connecting with a user by sending them message
-
Accepting a new request to connect from another user
-
Updating user profile to add new interests
Qualitative Results
General confusion on the "home screen"
-
Half of users were confused and thought the first profile they encountered was actually their profile
-
“I’m not sure if this is my profile or not”
-
“A short tutorial process could help.”
Several confusing symbols with unknown purpose.
The halo is...
-
"how much you have talked”...
-
“how compatible you are”...
-
“an expiration timer before a match runs out”
The gold star is ...
-
“Not sure” x5 (five of eight users)
The phrase "Pop your Bubble" has mixed reactions:
-
“I like the phrase, it’s cute!”
-
“I think that the phrase is cool, but it sounds promiscuous”
-
“I didn’t like the phrase”
-
"Avoid the phrase ‘burst your bubble’, It’s too aggressive
Lack of feedback after performing actions
-
Actions such as saving your interests, or sending a message to a user which affect change in the system failed to provide feedback.
-
7 of 8 users reported frustration trying to save their profile, or after sending a message
Quantitative Results
We used the System Usability Scale (SUS) and Difficulty Rating per Task to quantify and measure the usability of the application.
.png)
Difficulty Rating per Task
Users indicated they believed the tasks were easy to complete, rating them on average as a difficulty of 1.3 / 5, with 1 being very easy.
85.9
System Usability Scale
Users rated the app highly on the SUS.
92%
Task Completion Rate
Users had very little difficulty completing the task
2.9/5 !
App Usage
Users rated the app lowest on the category " I would use this app frequently"
Key Feedback
Usability Related
-
Users need an onboarding tutorial for “unique” features such as pick a challenge
-
Ensure users can learn or see exactly what particular symbols mean.
-
Be more consistent and use standard design practices for certain aspects of the app, such as navigation and notification feedback.
-
Change the way language is used in certain places to ensure users understand the meaning as intended.
Non-Usability Related
-
Users are in general ok with idea of meetups, but would feel safer if the app was sponsored by the University.
-
This app requires a lively user base to succeed. Marketing to just international students may limit that user base. Also, users reported only a moderate interest in if they would use the app.
Some of the lessons learned during this project were
-
You have to encourage and push users to share their thoughts and feelings as they go through the testing process. It was not natural for users to share their opinion.
-
Remote interviews are less than ideal, but you can still gain tons of useful information from them.
-
Asking users to take their time on each page and reminding them to think aloud especially during remote interviews. Users sometimes rushed through pages ( since it was on Figma) instead of thinking loudly through each process.
-
The questions you ask have a big difference on the quality of data you get back. Many things we wanted users to noticed were skipped over because the questions weren’t specific enough.