Personalization & Machine Learning for News Feeds and Social Networks

Winds is an open source RSS reader is powered by React, Redux, Sails and Stream. This tutorial explains how we’ve built personalization for Winds, as an example of how using Stream makes it easy to build personalized feeds.

About Personalization

Personalization is a very broad concept. In this case, personalization equates to leveraging engagement data to build an interest profile for a user. Many of the apps you use every day leverage machine learning to personalize your feeds. Here are 4 examples:

1. Instagram created an “Explore” section that shows you pictures you’re likely to be interested in, based on your past clicks, likes and engagement.
2. Quora uses your engagement data, like the posts that you search for and comment on, to personalize your feed experience and emails.
3. Facebook uses advanced machine learning to prioritize posts from the friends you engage with most, after starting out with just a basic edge rank algorithm.
4. Etsy analyzes your search and shopping behavior to optimize your newsfeed, email and e-commerce conversion.

Personalization in Action: Instagram’s Explore, Quora’s weekly email, Etsy’s news feed.

Building a Personalized News Feed

Now that we’ve covered the basics of personalization, let’s explain the technical details of how it works in Winds. (If you’re not a developer you can learn more about personalization here.) Winds’ personalization is accomplished via 3 basic steps:

1. Follow
2. Engage
3. Learn

Whether your app is content related or an RSS reader, or whether you are building a social network or an ecommerce community, the underlying tech is all very similar. Let’s get started by explaining how Follows work for Winds.

Step 1 - Follow

1A. Follow Relationships

The following example shows you how to create follow relationships between your activity streams:

    // jack's 'timeline' feed follows chris' 'user' feed:
    var jack = client.feed('timeline', 'jack');
    jack.follow('user', 'chris');
    // jack's 'timeline' feed follows an rss feed
    var jack = client.feed('timeline', 'jack');
    jack.follow(‘rss_feed’, 'cnn-rss-feed-id');

1B. Adding Activities to Your Feeds

Here’s an example of adding an activity to your feed:

    var chris = client.feed('user', 'chris');
    // Add activity; message is a custom field - tip: you can add unlimited custom fields!
    chris.addActivity({
        actor: 'chris',
        verb: 'add',
        object: 'picture:10',
        foreign_id: 'picture:10',
        message: 'Beautiful bird. Absolutely beautiful. Phenomenal bird. The best bird.'
    });

In the RSS app, this is done in models/article.js. The toActivity method shows how an article is translated into an activity. The fields used in the activity such as “actor”, “verb” and “object” are part of the open activity stream spec. Now that we have the basic follows and activities in place, we can move on to tracking engagements. If you want to learn more about adding activities and creating follow relationships, try our 5 minute interactive tutorial.

Step 2 - Engage

Follow relationships provide the first component of your personalized feed. The second component is tracking which content users see (impressions) and what they engage with (clicks, shares, likes, etc.)

2A. Initialize the Stream Analytics Client

As the first step, you’ll want to initialize your Stream Analytics Client. Visit the dashboard (getstream.io/dashboard/) to get your analytics token. In layout.ejs you will see how we initialize the Stream client:

    var client = new StreamAnalytics({
        apiKey: '<%- sails.config.stream.streamApiKey %>',
        token: '<%- sails.config.stream.streamAnalyticsToken %>'
    });

2B. Track Engagements

The next step is to track engagement events. We want to track the actions that indicate that a user is interested in a certain piece of content. For Winds our users express interest via clicking, reading articles or opening a specific RSS feed. If you’re building a social app, you’ll want to track clicks, likes, comments, visits to a friend’s profile, etc. For e-commerce apps, you will typically track clicks, add to cart actions, and search events. Let’s get started by tracking clicks. The code below shows how we track clicks for Winds:

    client.trackEngagement({
        label: 'click',
        content: {
            foreign_id: `articles:${e.currentTarget.getAttribute('data-id')}`
        },
        position: e.currentTarget.getAttribute('data-position'),
        location: 'subscriptions',
    })

The “track engagement" logic is implemented in Subscription/index.js

2C. Track Impressions

The next bits of information you need for personalization are the impressions. Tracking impressions allows us to understand which content users are not interested in. If you always ignore posts from a certain site, our machine learning models will start to hide it from your feed. This is how we track Impressions for Winds:

    client.trackImpression({
        content_list: [`articles:${id}`],
        feed_id: `timeline:${localStorage.getItem('id')}`,
        location: 'subscriptions',
    })

The “track impression” logic is implemented in Subscription/index.js

Step 3 - Learn

Now that we have both the Follow and Engagement data in place, we can start to show the user’s personalized feeds and their interest profile.

3A. Reading the Personalized Feed

To read the personalized feed we’re sending a GET request to this url = https://reader.getstream.io/reader/personalized/${userId} The full code is visible in StreamController.js;have a look at the function “personalized”. This personalized endpoint uses a basic version of Stream’s personalization and machine learning algorithms that is tailored to the Winds RSS app. The following is the end result of our hard work: \ (If you need personalization for your own app contact our data science team to learn about the possibilities and how we can tailor the machine learning algorithms to your app.)

3B. Reading the Interest Profile

Sometimes you’ll want to get a text-based description of what the user is interested in. The underlying machine learning models are too complex to represent in a text format, though. This endpoint will give you a simplified approximation of what the user is interested: To read the interest profile we request this URL: https://reader.getstream.io/reader/profile/${userId}/.

3C. Behind the Scenes

Every time a user engages with your app, Stream’s personalization endpoint will update the machine learning models to better understand what the user is interested in. Over time it learns an increasing amount about your users’ interests. You can use this to power a discovery style feed, as we’ve done for Winds. Other common use cases include feed/edge ranking (similar to Facebook), follow suggestions and email personalization.

Conclusion

In the past you needed a large team and an even larger budget to build personalization into your app. Only a select few, like Instagram, YouTube, Etsy, Quora and DeviantArt could afford to make this type of investment. Now, Stream’s API makes it simple to add personalization to your app. This blogpost showed you one example of how to implement personalization for an open source RSS reader (Winds). (P.S. We’re actively working on Winds and contributions are much appreciated!) If you want to learn more about personalization or brainstorm approaches with our data science team feel free to contact us.