•June 16th 2016
This is the 4th post in the 8 part tutorial series created by getstream.io. The final result is your own feature-rich, scalable social network app built with React and Redux! Visit getstream.io/cabin for an overview of all 8 tutorials and a live demo.
Visit getstream.io/cabin for an overview of all the tutorials, as well as a live demo. The source code can be found on the Stream GitHub repository for Cabin, and all blog posts can be found at their respective links below:
We’re using Stream to power the feeds in Cabin. Stream allows you to build newsfeeds and activity streams without worrying about the scalability, reliability, or maintenance of your feeds. Many companies have invested years in building their feed tech. (If you’re interested in learning more about this problem set, check out the papers mentioned on the stream-framework github repo.) For a more in-depth look at the features behind Stream, click here, or try the five minute tutorial.
A Little Background
By the end of this tutorial, we will have built a fully-fledged social application. Cabin relies on three specific feeds that power the application. Learn about these three feed types:
The timeline is the main page of Cabin and shows you the uploads from the people you follow.
The second feed is the Notification Feed. If someone comments on your picture, likes it, or follows you it shows up in this feed.
Lastly, we have our Incoming Activity Feed. The Incoming Activity Feed shows the likes, comments and follow activities from the people you follow.
Now that we have an understanding of the feeds that power Cabin, let’s take a look at an example activity:
The actor, verb and object fields are a standard way to represent activities. It follows the official activity stream spec. You might display this in your app as: “Jack added a photo of The Hill to Great Hill Photos – with @Jill.” The “to” field offers us the ability to essentially CC someone in a notification. For more information about this functionality, check out our info on “targeting” in the Stream documentation.
Setting Up Stream
- user_posts (flat)
- user (flat)
- timeline_flat (flat)
- timeline_aggregated (aggregated)
- notification (notification)
Note: You’ll want to make sure to turn on notifications on all of your feed groups as we’ll be utilizing realtime functionality in this application. Now that we’ve setup our feed groups, let’s collect the following information, as we’ll need it to initialize the application on both the client and server sides:
- App ID
- API Key
- API Secret
Next up, we’ll need to clone the repository from GitHub:
Then, add your credentials to the env.sh file located in the root directory by modifying the value and sourcing the file:
Okay, nicely executed. Now your environment is setup to run with your Stream credentials! The last thing we’ll need to do is install Stream from npm. Run the following command from both the
Note: You’ll also want to make sure that wherever you’re using Stream, you require it at the top of your file:
Adding Activities to Feeds
There are two sections where we utilize Stream in the Cabin application, the first being the API, the second being the client (a.k.a. app). In our case, the API handles the heavy lifting – it adds activities to Stream, in addition to enriching the data when the client makes a request. “Enriching data” refers to the to task of taking an object like the one we referenced in our example above, looking up the associated data, and returning a complete object to be shown to the user. Take a look at an example of the client side view for uploading a photo: Now, take a look at the server side (API) code that powers an “upload” activity to the feed:
Breaking it down:
- Instantiate a new API client using our Stream API key and secret.
- Then instantiate a new feed class using user_posts and the actor’s user id (from the database).
- Build an activity object:
- “actor” is the user creating the activity.
- “verb” is the action being taken.
- “object” is a descriptor of the action being taken, with the database id of the action (concatenated by a colon).
- “foreign_id” is the database ID for reference during enrichment.
- “time” is a timestamp of when the action occurred (in our case, we’re using “now”).
- Add the activity to the feed.
For more information on adding activities, visit the Stream documentation.
Following other users is an important aspect of any social application, especially Cabin. Luckily, Stream makes it easy for us to follow other users’ feeds in just a few simple steps. Take a look:
Note: The API is passed both a user_id parameter and a follower_id parameter from the data object. The
user_id parameter is the user who is following someone, and the follower_id parameter is the user that is being followed. With that in mind, let’s break things down a bit:
- First, we instantiate the streamClient by passing in our new API credentials
- We then instantiate and follow these feeds:
- Lastly, we build our activity object and add the activity to our Stream feed
Enriching Our Data
Next up is the “enrichment” step of this tutorial. Stream allows you to store as much data in an activity as you’d like – think of this as metadata. However, in many cases it’s best to store a reference. For example, a user profile can change over time, but you wouldn’t want to have to update every activity associated with that user profile. For example, the enrichment step would translate this reference:
Into this enriched object:
Enrichment is extremely important, as this is when we parse activity data for consumption by the client. It can become a bit unwieldy if you’re not careful. Given this, we decided to abstract the code into a small library that can be found here. Take a look at this enrichment code:
Let’s break it down:
- First, we instantiate a new stream client using our Stream API key and secret found on our dashboard.
- We then instantiate a new feed (assigned to the uploads variable) using the timeline_flat class and the user ID from our database. This specifies that we want to bind to the timeline associated with the specified user.
- From there, we call the “get()” method to retrieve all activities. Activities are returned as an array, which we then pass into our “utils” library for enrichment.
For Cabin, we’re using a MySQL database to store all of our data; however, it’s possible to do build the same with a NoSQL database such as MongoDB – in which case, you may want to have a look at the stream-node package (it takes care of handling enrichment with Mongoose). At the end of the day, your enrichment steps will likely differ, depending on your database of choice. For this reason, we’re leaving out the queries required to “enrich” data, as it’s different for nearly every use case. If you have questions about your specific use case, feel free to post on our Stackoverflow page and we’ll do our best to help!
Aggregated feeds allow you to specify a rule by which activities should be grouped. For example, in the screenshot below, we are aggregating “like” activities, but not the comments and follows associated with the aggregated images. Setting up an aggregation rule for Stream is rather straightforward.
Follow along to understand how we setup aggregated feeds: You simply head to your application on Stream and create a group with the “feed type” of “aggregated” (in our case, the feed groups are “timeline_aggregated” and “notification”). Then, you can specify the aggregation format that best fits your application. Here’s a screenshot of the aggregation format for Cabin: From the screenshot above, you can see that if the aggregation type is “like”, we concatenate the actor with the user id, giving us the following grouping key:
If we're dealing with a comment or follow, we group based on the activity id. (basically not grouping since activity ids are unique)
Activities are stored in Stream as a reference to user ids and upload ids. With that said, we need the full object, not just the reference for the template. The code below queries the database (using our nifty Stream utility) to translate the data.
Once translated, we end up something like this:
Note: Changes to the aggregation format only affect new activities. This can be a bit confusing when configuring the aggregation format.
One of the many amazing features of Stream is that you can listen to feed changes in real-time. We use this heavily in our notification feed on Cabin – every time that a new photo is uploaded, we receive a notification via the Stream websocket client letting us know that there’s been an update.
Subscribing to Real-Time Changes
Subscribing to real-time changes with Stream, at it's core, is simple. Take a look at the documentation. Essentially, subscribing to real-time requires two steps: 1. Generate your read-only token on the server side. 2. Use the
Cabin Code Examples
In our app, it looks a bit more complex as it is done across files - yet the core principles are the same.
- View code in
api/routes/users.js- Generate the token on the server side:
- View code in
app/modules/App.js- Subscribe to the changes:
Notification Feeds are similar to aggregated feeds. There are two key differences that make them more suited to building notification systems:
- Notifications can be marked as “seen” or “read.”
- You get a real-time count of the number of “unseen” and “unread” notifications.
(If this functionality was not baked into Stream, we would have to spend some serious engineering time setting up and configuring a socket server to handle the heavy lifting of routing updates to our users!) Check out more information on Notification Feeds by visiting the Stream documentation.
Feed Group Structure Recap
Recap the Feed Group Structure:
- The timeline feeds follow user_posts (containing uploads)
- The timeline_aggregated follow user feeds (containing likes, comments, follows)
- notification is populated using the to field
If you write an activity to
user_posts:nick it will automatically show up in the timeline feeds of people who follow Nick. Similarly if you add an activity to
user:nick it will show up in the timeline_aggregated feeds of people who follow Nick.
One thing we didn’t try out are the ranking methods. By default, all feeds are ranked in reverse chronological order. If you want to show popular or promoted content higher in the feed you can use the ranking methods. Read this documentation about ranking methods.
Another powerful feature is personalization. Personalization uses analytics and machine learning to tailor a feed to the user’s experience. A good example is the discovery feed you see on Instagram when you open search. Learn more about personalization.
Using Stream as a building block, we were able to build scalable feeds in just a few hours. That’s quite cool if you compare it to the months/years it took popular apps to build their feeds. In the next post we’ll cover how we’re using imgix to power the real-time image edits for Cabin. Add your email on https://getstream.io/cabin or follow @getstream_io on Twitter to stay up to date about the latest Cabin tutorials.