An activity feed is a list of recent actions happening in realtime. In this article, we’ll build a simple feed app powered by GitHub WebHooks and Stream Feeds in JavaScript.
The app will track the activities of your GitHub organization or personal repositories. So, we’ll see what is happening right there from our app. For example, if someone opens a Pull Request, our app should immediately tell us that they did so.
Here is an example of what we’ll be building. 🙂
You’ll learn how to use Stream and GitHub WebHooks to build activity feeds functionalities into your application or build an application that is heavily dependent on WebHooks and WebSocket's.
Prerequisites
Below are the tools required to follow along with this tutorial. I encourage you to follow along and practice with the code in this tutorial, as practicing is a great way to learn.
Note: Having a basic understanding of javascript will help you follow along in this tutorial
Install the latest NodeJS version from the NodeJS website, if you don't already have it installed.
Create a directory called "activity-feed" (mkdir activity-feed) and navigate to that directory (cd activity-feed).
Initialize a NodeJS project with the command npm init, then follow the instructions to configure your node app.
Once you have finished the setup of the above steps, install Express, Stream, and EJS. EJS is a simple templating engine; I chose it because of its simplicity.
You can install these packages using the following command:
$ npm install express ejs getstreamWe now have all we need to run our application!
Setting Up Your App File Structure
The file structure should look like this:
.
├── controllers
│ └── index.js
├── index.js
├── package-lock.json
├── package.json
├── public
│ ├── dist
│ │ └── stream-js.js
│ ├── index.js
│ └── style.css
├── readme.md
├── routes
│ └── index.js
└── views
└── index.ejs
You should already have package-lock.json and package.json files automatically generated for you if you ran npm init.
StreamJS enables you to quickly create scalable activity feed applications — because you don’t have to worry about the underlying infrastructure or building everything from scratch yourself.
stream-js is the version of Stream we’ll use in the frontend. Download it from here and add it to your /public/dist/ directory.
All of our static files will be organized in the public directory, and the HTML files will be in the /views/ directory.
Add the following code to /views/index.ejs:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" media="screen" href="style.css" id="cm-theme" />
<title>Activity Feed</title>
</head>
<body>
<div class="container">
<h2>Activity Feed</h2>
<ol class="activity-feed" id="activity-feed"></ol>
</div>
<script src="/dist/stream-js.js" type="text/javascript"></script>
<script src="index.js" type="text/javascript"></script>
</body>
</html>
This is where our feeds will be posted!
Note how we included the Javascript files. The order is essential; the stream-js module should come first, since we’ll be using Stream in our custom javascript file (index.js).
<script src="/dist/stream-js.js" type="text/javascript"></script> <script src="index.js" type="text/javascript"></script>
At this point, add this CSS code to the /public/styles.css file:
html {
box-sizing: border-box;
}
*,
*:before,
*:after {
box-sizing: inherit;
}
@font-face {
font-family: 'Open Sans';
font-style: normal;
font-weight: 400;
src: local('Open Sans Regular'), local('OpenSans-Regular'), url(https://fonts.gstatic.com/s/opensans/v17/mem8YaGs126MiZpBA-UFVZ0e.ttf) format('truetype');
}
body {
font-family: 'Open Sans', sans-serif;
color: #4e555f;
font-size: 14px;
}
a {
color: #50bced;
}
.activity-feed {
padding: 15px;
list-style: none;
}
.activity-feed .feed-item {
position: relative;
padding-bottom: 20px;
padding-left: 30px;
border-left: 2px solid #e4e8eb;
}
.activity-feed .feed-item:last-child {
border-color: transparent;
}
.activity-feed .feed-item::after {
content: "";
display: block;
position: absolute;
top: 0;
left: -6px;
width: 10px;
height: 10px;
border-radius: 6px;
background: #fff;
border: 1px solid #f37167;
}
.activity-feed .feed-item .date {
display: block;
position: relative;
top: -5px;
color: #8c96a3;
text-transform: uppercase;
font-size: 13px;
}
.activity-feed .feed-item .text {
position: relative;
top: -3px;
}
.container {
display: flex;
-webkit-display: box;
-moz-display: box;
-ms-display: flexbox;
-webkit-display: flex;
flex-wrap: wrap;
justify-content: center;
align-content: center;
padding: 6%;
margin: 0;
}
At this point, we have our static files all set up, but we can’t view their results, because we are not serving the files yet. So, let’s set up our express server to serve our static files and run our backend code.
Before that, we need to register on Stream, as we are going to need our Stream credentials to continue with this tutorial.
It should take less than 10 seconds to create an account. 🙂
Setting Up StreamJS
Visit the Stream Website and click on the SIGNUP button.
After the registration, you should be redirected to your dashboard where you’ll find your app credentials:
Take note of the following:
- KEY
- SECRET
- APP ID
and copy them to your environment. Create a .env file, where you’ll save all the environment variables, in the root directory.
API_SECRET=<YOUR_APP_SECRET> API_KEY=<YOUR_API_KEY> APP_ID=<YOUR_APP_ID>
We’ll need the dotenv package to access the environment variables in our node application.
After you install the package, with the command npm install dotenv, we’ll be able to access the variables like so: process.env.API_SECRET, etc..
Setting Up the Server, Routes, and Middleware
Next, let’s set up our express server and routes, and plug in some middlewares. Add the following code to the ./index.js file in the root directory of the application to set these up:
const express = require('express')
const ejs = require("ejs")
const path = require('path')
const cors = require('cors')
require('dotenv').config()
const routes =require('./routes')
const app = express()
app.use(express.static(__dirname + '/public'));
app.set('views', path.join(__dirname, 'views'));
//set view engine
app.set('view engine', 'ejs');
app.use(express.json())
app.use(express.urlencoded({ extended: true }))
app.use(cors())
app.use(routes)
var port = process.env.PORT || 3002
app.listen(port, () => console.log('server started', port))
Let me explain:
app.use(express.static(__dirname + '/public'));
app.set('views', path.join(__dirname, 'views'));
Here, we are telling express to look for our static files in the /public directory, and also telling EJS to look for our views (HTML files) in the views directory.
We added a middleware "cors" because we want our routes to be accessed from other domains. Be careful with how you use this in your application, though. You can read more about cors here.
To use CORS in this app, just use npm install cors, and include it the way we’ve done in the sample code above.
We’ve also added our routes as a middleware; that’s because we want our entry file to be clean. So, we have all the routes in a different file:
app.use(routes)
Our routes are in the route directory ./route/index.js:
const express = require("express");
const app = express.Router();
const { getToken, githubHook } = require("../controllers");
app.post("/get-token/", getToken)
app.post("/github", githubHook)
app.get("/", (req, res, next) => {
res.render("index")
});
module.exports = app
The three routes we need are as follows:
/get-token: We’ll use the/get-tokenroute to generate a token to authenticate our users/github: Our GitHub webhook will hit this route. We’ll get to know more details about the functionality of this route when we start looking at the controllers./: This is the home page, where we’ll display all the feeds.
Now that we have our routes, let’s look at the controllers!
Setting Up the Controllers
The controllers are in ./controllers/index.js:
const stream = require('getstream');
const client = stream.connect(
process.env.API_KEY,
process.env.API_SECRET
);
const getToken = (req, res, next)=>{
const token = client.createUserToken(req.body.username)
res.json({token})
next()
}
const githubHook = async (req, res, next) => {
// feedManager(req.body)
const { user, created_at, state, html_url } = req.body.pull_request;
const feedUser = client.feed('notification', "peter");
await feedUser.addActivity({
actor: user.login,
verb: 'add',
object: 'picture:10',
foreign_id: 'picture:10',
created_at,
state,
html_url,
message: `${user.login} Created a Pull Request`
});
const results = await feedUser.get({ limit: 10 });
res.json(results);;
}
module.exports = {
getToken,
githubHook
}
So, let’s see what’s happening in our controller. We’ll go over it together bit-by-bit.
const stream = require('getstream');
const client = stream.connect(
process.env.API_KEY,
process.env.API_SECRET
);
Here we import and initialize Stream. Notice that we are using process.env.API_KEY, as described earlier, to access our APP credentials.
Next, we create a controller to generate tokens. Once a post request is sent to this API with a username, through the route we described earlier (/get-token), the controller will talk to Stream and generate a token with that username:
const getToken = (req, res, next) => {
const token = client.createUserToken(req.body.username);
res.json({ token });
next();
};
Also, let’s look at the githubHook controller:
const githubHook = async (req, res, next) => {
// feedManager(req.body);
const { user, created_at, state, html_url } = req.body.pull_request;
const feedUser = client.feed('notification', "peter");
await feedUser.addActivity({
actor: user.login,
verb: 'add',
object: 'picture:10',
foreign_id: 'picture:10',
created_at,
state,
html_url,
message: `${user.login} Created a Pull Request`
});
const results = await feedUser.get({ limit: 10 });
res.json(results);
}
When the GitHub Webhook is triggered, it hits the /github API endpoint we mentioned earlier, and it then executes this controller.
At this point, let’s create a webhook on GitHub that will be triggered when a pull request is created...
Setting Up your Github Webhook
Log in to your GitHub account and create a repository. Then, go to the settings of this repository.
Once here, click on the Webhooks link to open the "WebHooks" page. From here, click on the Add Webhook button.
Great Job!
Choose the Content-type "application/json", as we'll be expecting a JSON object from GitHub. Finally, choose the event you want to track. You can choose to monitor all activities of the repository, or just select some events like we are doing here:
Scroll down the list and chose the Pull requests event. Finally, hit the Create Webhook button.
You now have a webhook ready! Each time a pull request activity happens, that hook will send us JSON as a POST request to our /github route.
Using the Webhook
Retrieving data from the webhook is extremely easy to do. Simply pull data off of req.body.pull_request, as shown in the following snippet:
const { user, created_at, state, html_url } = req.body.pull_request;
Create a feedUser:
const feedUser = client.feed('notification', "peter");
Then, create an activity for that feed user on Stream like so:
await feedUser.addActivity({
actor: user.login,
verb: 'add',
object: 'picture:10',
foreign_id: 'picture:10',
created_at,
state,
html_url,
message: `${user.login} Created a Pull Request`
});
You can add as many items as you want from the request to the activity, as you receive them all from the GitHub webhook.
When the activity is created, it triggers a feed notification event that we’ll catch in the frontend and show to the user.
Let’s take a look at how we’ll handle the event in the frontend and display it on the interface; add this code to the /public/index.js file:
// Instantiate new client with a user token
const init = (url, username) => {
fetch(url, {
headers: {
'Content-Type': 'application/json'
},
method: "post",
body: JSON.stringify({username})
}).then((data)=>{return data.json()}).then(
(response)=>{
const { token } = response
feedManager(token, username)
}
)
};
const feedManager = async (token, username)=>{
const client = stream.connect('wuf946y45ahq5j', token, "66027");
const notificationFeed = client.feed('notification', username);
const results = await notificationFeed.get({ limit: 10 });
const feedTemplate = (date, message, link, state)=>{
return(
`
<time class="date" datetime="9-17">${date}</time>
<span class="text">Pull Request: <a href="${link}">${message}</a></span>
`
)
}
const singleFeed = (data) =>{
const div = document.createElement('li')
div.className = "feed-item"
div.innerHTML = feedTemplate(data.created_at, data.message, data.html_url, data.state)
document.getElementById('activity-feed').appendChild(div)
}
// get historical feeds
results.results.map((data)=>{
data.activities.map((p)=>{
singleFeed(p)
})
})
const callback = data => {
singleFeed(data.new[0])
};
const successCallback = async () => {
console.log('now listening to changes in realtime');
};
const failCallback = data => {
alert('something went wrong, check the console logs');
console.log(data);
};
notificationFeed.subscribe(callback).then(successCallback, failCallback);
}
init("http://localhost:3002/get-token", "peter");
Let’s break it down to see what’s going on here...
The first thing we need to do is authenticate the user:
const init = (url, username) => {
fetch(url, {
headers: {
'Content-Type': 'application/json'
},
method: "post",
body: JSON.stringify({username})
}).then((data)=>{return data.json()}).then(
(response)=>{
const { token } = response
feedManager(token, username)
}
)
}
Here, we make an API call to the /get-token API endpoint, which returns a token that we’ll use to authenticate the user.
Once we get the token, we use it to connect stream together with the APP_ID and API_KEY:
const client = stream.connect(APP_KEY, token, APP_ID);
const notificationFeed = client.feed('notification', username);
const results = await notificationFeed.get({ limit: 10 });
Remember to use the same username and feed type you used in your backend;
in our case, we are passing the same username parameter in the init function that we had on the backend.
init("http://localhost:3002/get-token", "peter");
Now, let’s subscribe to that feed:
notificationFeed.subscribe(callback).then(successCallback, failCallback);
The callback method is called if everything is okay, else, the failCallback method is called.
Once our call back is called, we push the feed data to the UI:
const callback = data => {
singleFeed(data.new[0])
};
And we can also pull all the feeds for this user, as shown here:
results.results.map((data) => {
data.activities.map((p) => {
singleFeed(p)
});
});
That’s pretty much all there is to do to get our Github webhook to work with Stream!
Wrapping Up
Let’s look at the workflow again in 5 steps:
- Create a webhook on Github.
- Create a feed client.
- Create an activity for that feed client when the webhook hits the backend.
- Connect to Stream on the frontend and subscribe to the same feed you created in the backend.
- Push the data from the call back to the UI.
These are the necessary steps you need to get Stream to work with GitHub Webhook or any other WebHooks!
There is so much more you can do with Stream Feeds; check out the documentation to find out more! You don’t need to spend the time to build the activity feed infrastructure. You don’t need to bother with databases, servers, or the like; Stream provides you with all of the power you need to build amazing and scalable activity feed apps in hours.
You can check out the full and completed project on GitHub
Happy coding!