Pop social

The Challenge

Pop Social Inc founded by a student at UT Austin was in the process of releasing their app into the market and wanted us to run usability tests and provide feedback before it went live.

Client

Pop Social Inc

My Team

Mavis Klemcke, Ryan Wolter, Christian Onuogu, Shravya Kolavara

My Responsibilities

Heuristic Evaluation, Moderated 3 tests, Notetaker for 2 tests, Secondary Research

My Role

UX Researcher

Our Advisor 

Eric Nordquist

Timeline

10 weeks

Let's get some context 

The application aims to change the paradigm of people socializing in college. The startup reported that 64% of students feel lonely during their college years. This app wants to provide a platform for students to connect with each other based on shared interests alone. This is achieved by locking the persons name and picture until a certain number of interactions between the users, thus creating more compatible meetups. 

Secondary research

The scope of the research was broad and delved into the problem space the app was trying to solve. The research proved to validate some of the research provided by the company and looked beyond to find solutions that would help improve the application.

40% of Students Report that they feel very lonely in college.

This can lead to an increased likelihood of depressive symptoms among other negative effects

Go to Source

A good way to make new friends is by finding shared interests.

Go to Source

Many students cite the relative difficulty of making connections on campuses of hundreds or thousands

Go to Source

Subject Matter expert

Before diving into the usability test, we wanted to get some advice and feedback from experts in order to better understand the problem space and some of the content the app uses. We spoke to a counselor from the Center for Mental Health at UT Austin. 

These were some of her insights into the application:

Though the app can help with depression symptoms, it should be framed as a social communication app rather than a mental health app.

The app can't replace actual treatments for depression such as therapy, so at best it can serve as an aid

A mental health based framing might also turn-off potential users who would believe that this app isn't targeted at them

A potential compromise could be to display the number of a mental health crisis line to potentially help users suffering from these issues



Heuristic Evaluation 

Evaluation Measurements

I've used the 10 usability heuristics by Nielsen Norman Group to conduct the evaluation . 

The evaluation provides a severity level to each of the problems to indicate which problems need to be tackled first and in what order might they want to proceed with to make changes. 

Crucial : Cannot function unless Fixed.

Major: Fix Now

Minor: Fix Soon

Trivial: Fix when Possible

Task #1 Onboarding

Onboarding 1.jpg

1

7

Onboarding 3.jpg

18

Issues Found

3

Crucial 

Cannot function unless Fixed.

7

Major

Fix Now

4

Minor

Fix Soon

2

Trivial

Fix when Possible

Task #2 Connect With People

5

6

5

CONNECTING – 1.jpg

Issues Found

3

Crucial 

Cannot function unless Fixed.

2

Major

Fix Now

1

Minor

Fix Soon

0

Trivial

Fix when Possible

Task #2 Edit Your Profile

Issues Found

0

Crucial 

Cannot function unless Fixed.

2

Major

Fix Now

4

Minor

Fix Soon

0

Trivial

Fix when Possible

who is the competition?

In our competitive analysis we compared 4 apps to the Pop Social app. These were the key insights and recommendations.

Making Connections

Starting conversations can be difficult for users who want to meet new people.

Other apps encourage friendship with ice breakers, questions, or tasks

Personality Sliders

Personality sliders are subjective and difficult to comprehend. Other apps have users answer questions. It is more effective method for users to determine their personality.

Interests and Communities

Other apps provide more flexibility to users by letting them choose as many interests as they would like and make connections with more specific choices for smaller communities.

Editing Profile

Other apps allowed users to update their profile interests easily. This allows users to change their profiles as and when they interests and goals evolve in real life.

Usability test

User Interview Process

Survey

Surveyed students using Google Forms to identify candidates for interviewing

Technology

Using Zoom, we performed remote testing by having users share their screen.

Post Task Interview

After all the tasks were completed, we walked users back through the tasks with us driving, asking specific questions.​​

Selection

8 participants were selected to be interviewed producing a mix of demographics.

Tasks

Each User performed 5 tasks.

  • Creating an account 
  • Reading new messages sent from other users

  • Connecting with a user by sending them message

  • Accepting a new request to connect from another user

  • Updating user profile to add new interests

Qualitative Results

General confusion on the "home screen"

  • Half of users were confused and thought the first profile they encountered was actually their profile

  • “I’m not sure if this is my profile or not”

  • “A short tutorial process could help.”

Several confusing symbols with unknown purpose.

The halo is...

  • "how much you have talked”...

  • “how compatible you are”...

  • “an expiration timer before a match runs out”

The gold star is ...

  • “Not sure” x5 (five of eight users)

The phrase "Pop your Bubble" has mixed reactions​:

  • “I like the phrase, it’s cute!”

  • “I think that the phrase is cool, but it sounds promiscuous”

  • “I didn’t like the phrase”

  • "Avoid the phrase ‘burst your bubble’, It’s too aggressive

Lack of feedback after performing actions

  • Actions such as saving your interests, or sending a message to a user which affect change in the system failed to provide feedback.

  • 7 of 8 users reported frustration trying to save their profile, or after sending a message

Quantitative Results

We used the System Usability Scale (SUS) and Difficulty Rating per Task to quantify and measure the usability of the application. 

Group 132 (1).png

Difficulty Rating per Task

Users indicated they believed the tasks were easy to complete, rating them on average as a difficulty of 1.3 / 5, with 1 being very easy.

85.9

System Usability Scale

Users rated the app highly on the SUS.

92%

Task Completion Rate

Users had very little difficulty  completing the task

2.9/5 !

App Usage

Users rated the app lowest on the category " I would use this app frequently"

Key Feedback

Usability Related

  1. Users need an onboarding tutorial for “unique” features such as pick a challenge

  2. Ensure users can learn or see exactly what particular symbols mean.

  3. Be more consistent and use standard design practices for certain aspects of the app, such as navigation and notification feedback.

  4. Change the way language is used in certain places to ensure users understand the meaning as intended.

Non-Usability Related

  1. Users are in general ok with idea of meetups, but would feel safer if the app was sponsored by the University.

  2. This app requires a lively user base to succeed. Marketing to just international students may limit that user base. Also, users reported only a moderate interest in if they would use the app.

Some of the lessons learned during this project were

  1. You have to encourage and push users to share their thoughts and feelings as they go through the testing process. It was not natural for users to share their opinion.

  2. Remote interviews are less than ideal, but you can still gain tons of useful information from them.

  3. Asking users to take their time on each page and reminding them to think aloud especially during remote interviews. Users sometimes rushed through pages ( since it was on Figma) instead of thinking loudly through each process.

  4. The questions you ask have a big difference on the quality of data you get back. Many things we wanted users to noticed were skipped over because the questions weren’t specific enough.