Yahoo – Stream Personalization

Client: Yahoo | Year: 2019-2020 | Role: Lead Designer, User Research

The Problem:

Yahoo’s DAU’s and LIDAU’s were steadily declining. Our technology and feature set were falling behind some of the competitors. One missing feature was the ability to personalize your own “stream”. The stories we showed were based on our ML algorithm but the user had little control.

The Challenge:

Create an experience where a user could give us explicit signals to personalize what is shown in their stream.  

My Role:

I was the sole designer on this project that ran weekly status meetings, interacted with engineers, and product managers, while also running user testing sessions alongside our user researchers, and prototype demonstrations to our CEO, board, and investor panel.


Narrative:

Our users come to Yahoo Home to stay up to date on what matters to them most. The key here is ‘what matters to them’. This makes our job to ‘Sift through the noise and only surface relevant and personalized content for each user. The pace and precision with which we can get to a state where most of what a user sees on the homepage is based on their interests, the more likely we are to retain the user.

Phase 1 – Show more / show less

To improve our understanding of our users and their interests, we will be asking them to provide us with overall feedback about content in the stream. This feedback will be a general/broad-spectrum on the entire content piece and allow us to reinforce our inferred signals to understand the user better. From a user standpoint, this feature will function as a Show More / Fewer tool. The feedback will Not be editable or exposed via any feedback management tools. The user will not be required to sign in to start with but will be requested to do so and will have an artificial limit post which it will be required.


User Journey 

I created a simple user journey flow to show proof of concept of how the icons may look, location, and actions a user can take. Straight forward with only two main actions, show more and show less. With Show more, we invite them to log in to save their preference, and with show less we ask for additional feedback.

 

Prototype:

Click to view

User testing:

Key findings:

  • P2 felt it was difficult to respond with a Thumbs Up or Down without reading the article and was concerned it would contribute to people reading fewer articles.
  • Most participants (P1, P3, P4, P5) thought that they would respond based on whether they liked the article. Others (P2) thought they would respond whether they would recommend the article.
  • All participants understood they would see more basketball, ESPN, or sports stories if they clicked Thumbs Up
  • 4 participants saw that the message changed from “Show me more stories like this” to “Great, we will show you more stories like this” when they clicked Thumbs Up. (P4 clicked too quickly to see.)
  • Participants weren’t sure at what level these settings were. Overall, all participants thought they should manage content at the Topic level


Mobile Web:

For mWeb, I wanted to have a very similar experience, but instead of traditional icons, I wanted to try to use native gestures to simplify the process for users. This included challenges for discoverability, but I remedied those by adding hint animations to educate the user.

Prototype:

Click to view

User Testing: We first tested a mobile web version of the feature, with multiple variations tested in random sequence to reduce bias. These variations included multi-sided swipe, single-sided swipe, variations of language, and variations of icons.

Some key insights:

  • P1 and P4 were able to discover Show More/Less when prompted to manage their feed. Other participants guessed it would be in settings, profile, or under the ellipses (in two-sided Swipe)
  • Between the one-sided Swipe and two-Sided Swipe, all participants preferred the one-sided swipe. They were mainly worried about errors and felt it was easier to understand.
  • All participants understood Show More/Less would increase/decrease the amount of similar content in their feed. Some participants thought it would be instantaneous with articles being added/removed immediately, while others thought they would need to refresh the page. Most participants thought it would manage content at a topic or category level, like sports, entertainment, baseball, or Cardinals.
  • Participants expected a wide variety of options under the Ellipses/More Options button including hiding articles, saving/sharing/downloading articles, and refining the Show More/Less setting to a specific topic or category. 
  • A few participants misread “Hide posts from Yahoo Lifestyle” as “Hide post from Yahoo Lifestyle” and thought it would hide the article with no other effects. Participants also misunderstood Yahoo Lifestyle as managing the category of Lifestyle and not the publisher Yahoo Lifestyle.
  • All participants felt positively towards the language used. P2 and P3 thought it could be more specific. P5 felt the color was not intuitive.
  • Participants were asked how they felt about the heart and crossed heart icons and whether they would prefer a thumbs up and down. Overall, participants did not have strong opinions about the icons used. 

Style Guide:

I created a style guide (the desktop example is shown) for the developers to quickly see what new components need to be made, each variation and its implementation. I would of course QA these before release.

 

Phase 2- Follow

The more granular the feedback we get from the user the better equipped we become to provide them with a robust personalized experience that starts to move beyond just content. By allowing our users to follow or block granular topics/entities we will be able to not only personalize the content but also extend this to the utility modules such as Sports and Finance, and provide the user with targeted notifications.

From a user standpoint, this feature will provide them with the following functionality as a ‘Follow’, ‘Unfollow’ & ‘Not Interested’. All the user interests will be captured in a topic management page.

Mocks:


Follow in stream:

User Journey (Desktop):

User Journey (Mobile):

Topic Management Page:

Topic Management screenshot
Style Guide: 

Follow Style Guide – Click here to view

 

Learnings and results:

What did we learn?

  • Mobile gestures didn’t feel native
  • UI was seen as crowded
  • Users had trouble understanding why they should personalize or how it made an impact
  • How did users know their feedback mattered?
  • We could do a better job of educating the user, better value props. 
    • Maybe gamify it?

Changes:

  • Cleaned up desktop UI, showed icon on hover only
  • Mobile gestures were changed to be more in line with a recent native change (iOS)
  • The stream was changed to show more immediate feedback when action was performed (hiding that story for example)

Results (after 90 days):

  • 7 day retention went from 2.4 -> 4.6
  • Slowed our ad-revenue decline
  •  Interactions per user per session 6.3 (well above the target of 3)