Explicit Signals

Wouldn’t it be nice?

What if your streaming service actually recommended content you enjoyed? How might we allow users to express feedback and share their preferences in order to deliver a more personalized experience?

The Team
I lead the design for this project. I was
assigned one additional designer to assist me.
The overall team was robust. I was able to
partner with a lead PM, include machine learning
engineers, front end engineers and the UX
research team along the way. The concept
was approved by executive leadership and is
set for final production and release end of 2024.

The Problem
How might we allow users to express their feedback and share their preferences in order to curate a more personalized experience?

Devices
Designing across devices considering mobile, TV, tablet and web.

Research Review
What have users already been telling us directly or indirectly about how they want their streaming service to work? Reviewing research reports from previous projects and collating customer feedback from regular reports led me to the following areas of focus.

Comparative Analysis
I delved into the explicit feedback mechanisms of 13 streaming services and 10 streaming-adjacent services. We identified positives, negatives, and any unique or delightful experiences therein. I was also able to better understand common entry points & successful design patterns.

Types of Explicit Signals
The comparative analysis was fruitful. I was able to break down explicit signals into forms and functions. This gave the team a common language and jumping point for ideation.

User Journey Map
It was important to prioritize a user journey map for this experience. A part of the success of explicit signals will be how it relates to the user over time. How might we grow and change with our customer’s tastes?

User Research & Concept Testing
With this being such a high value feature we wanted to curb our assumptions up front and better understand our customers motivations and expectations. We performed a mixed method research study including
12 user interviews and a card sort with 438 participants.

1. Why do they want to provide feedback?

2. What are their expectations from a feature like this?

3. When and where and how do they want to share their opinions?

4. What feedback features are preferred over others and why?

5. How much control over and engagement with the system are they looking for?

Viewing history is expected to inform recommendations
Participants assumed that their viewing history, should influence the content they see recommended on their homepage. Even with out sharing explicit feedback for that content.

Direct feedback is expected inform recommendations and improve them over time
Participants indicated that the ability to provide feedback that directly improves their personalization should be the top priority of a streaming service.

More nuanced ratings are desired to better convey sentiment
Many participants are hesitant to provide overtly negative or positive feed back due to the uncertainty of it’s impact. They crave a more granular way to convey nuanced feelings about content in order to improve recommendations.

Design Principles
Using research as our foundation I developed these four guiding principles.

Transparent | Clarifies what user feedback is doing to the system, its value, and its scope.

Empowering| Builds trust by giving users agency over their data while balancing control and automation.

Voluntary | Never forces engagement. Allow opt-out or undo on interactions.

Clear | Uses simple, accurate language and iconography to describe action and result.

Brainstorming 🔁
With this project I really had the chance to set the team up well before engaging in brainstorming. I held 3 rounds of brainstorming sessions including product management, engineering, machine learning, & editorial team. I split the brainstorms into three areas.

Ideation
A lot of time was spent deciding just what kind of feedback we would be asking from users and how that would appear to them.

Wireframing

Entry Points
A huge part of this project is deciding the right place and the right time to request explicit feedback from users. It’s important we don’t get in the way or waste our users time and attention.

After a cost-benefit analysis I decided to direct the team’s attention to onboarding, inline, on asset and profile opportunities to engage users in feedback. Pop-ups and requesting feedback outside of the app is time intensive, disruptive and gathers less impactful information for the platforms recommender system.

Wireframing & User Flows
After doing a cost benefit analysis I narrowed down our ideas and create low-fi iterations in order to better understand where they fit within the system.

Browsing
Users can give feedback on content directly through interacting with that asset, on the content details page or as prompted by the system.

End Card
Finally users should be able to give feedback during their watch experience, from the content details page, on pause or before abandoning the content and at the end of their watching session.

Final Designs
Doubling down on our entry points and solutions I created final design concepts which were signed off by the executive leadership team. This project was put on hold during the HBO Max merger with Discovery. Onboarding preference selection in slates to release Q2 2024 followed by end card and finally on asset explicit feedback before 2025.

Preference Selection
Gathering preferences across the user experience.

Explicit Signals on Content
Requesting feedback on content at the right time and place.​

User Agency and Identity
We aim to empower the user by giving them agency over their own recommender system. Offering transparency into personalization and allowing them to tell us how they have changed and how we should respond over time.