Activity Feed Powered by GitHub WebHooks

An activity feed is a list of recent actions happening in realtime. In this article, we’ll build a simple feed app powered by GitHub WebHooks and Stream Feeds in JavaScript.

The app will track the activities of your GitHub organization or personal repositories. So, we’ll see what is happening right there from our app. For example, if someone opens a Pull Request, our app should immediately tell us that they did so.

Here is an example of what we’ll be building. 🙂

Activity Feed - Example

You’ll learn how to use Stream and GitHub WebHooks to build activity feeds functionalities into your application or build an application that is heavily dependent on WebHooks and WebSocket's.

Prerequisites

Below are the tools required to follow along with this tutorial. I encourage you to follow along and practice with the code in this tutorial, as practicing is a great way to learn.

Note: Having a basic understanding of javascript will help you follow along in this tutorial

Install the latest NodeJS version from the NodeJS website, if you don't already have it installed.

Create a directory called "activity-feed" (mkdir activity-feed) and navigate to that directory (cd activity-feed).

Initialize a NodeJS project with the command npm init, then follow the instructions to configure your node app.

Once you have finished the setup of the above steps, install Express, Stream, and EJS. EJS is a simple templating engine; I chose it because of its simplicity.

You can install these packages using the following command:

$ npm install express ejs getstream

We now have all we need to run our application!

Setting Up Your App File Structure

The file structure should look like this:

https://gist.github.com/nparsons08/477230890b48f6776669c53215408e3f

You should already have package-lock.json and package.json files automatically generated for you if you ran npm init.

StreamJS enables you to quickly create scalable activity feed applications — because you don’t have to worry about the underlying infrastructure or building everything from scratch yourself.

stream-js is the version of Stream we’ll use in the frontend. Download it from here and add it to your /public/dist/ directory.

All of our static files will be organized in the public directory, and the HTML files will be in the /views/ directory.

Add the following code to /views/index.ejs:

https://gist.github.com/nparsons08/b146c42e557a19396767ab5fe3872a8c

This is where our feeds will be posted!

Note how we included the Javascript files. The order is essential; the stream-js module should come first, since we’ll be using Stream in our custom javascript file (index.js).

https://gist.github.com/nparsons08/0e93f0e6f4c14a313fd0324fe699f25f

At this point, add this CSS code to the /public/styles.css file:

https://gist.github.com/ezesundayeze/ef89ef7739a40243f050f68ac8bda761

At this point, we have our static files all set up, but we can’t view their results, because we are not serving the files yet. So, let’s set up our express server to serve our static files and run our backend code.

Before that, we need to register on Stream, as we are going to need our Stream credentials to continue with this tutorial.

It should take less than 10 seconds to create an account. 🙂

Setting Up StreamJS

Visit the Stream Website and click on the SIGNUP button.

Stream - Sign Up

After the registration, you should be redirected to your dashboard where you’ll find your app credentials:

Take note of the following:

  • KEY
  • SECRET
  • APP ID

and copy them to your environment. Create a .env file, where you’ll save all the environment variables, in the root directory.

https://gist.github.com/nparsons08/02d2b9aa1482b7c57bfe7df8d0f45428

We’ll need the dotenv package to access the environment variables in our node application.

After you install the package, with the command npm install dotenv, we’ll be able to access the variables like so: process.env.API_SECRET, etc..

Setting Up the Server, Routes, and Middleware

Next, let’s set up our express server and routes, and plug in some middlewares. Add the following code to the ./index.js file in the root directory of the application to set these up:

https://gist.github.com/nparsons08/476fc384f1723c46e6746ec06da71614

Let me explain:

https://gist.github.com/nparsons08/51c8eef39921239023b29866e74eead7

Here, we are telling express to look for our static files in the /public directory, and also telling EJS to look for our views (HTML files) in the views directory.

We added a middleware "cors" because we want our routes to be accessed from other domains. Be careful with how you use this in your application, though. You can read more about cors here.

To use CORS in this app, just use npm install cors, and include it the way we’ve done in the sample code above.

We’ve also added our routes as a middleware; that’s because we want our entry file to be clean. So, we have all the routes in a different file:

https://gist.github.com/nparsons08/3a998970849285fecb029cd490af4ff2

Our routes are in the route directory ./route/index.js:

https://gist.github.com/nparsons08/16b92dfbcd573b5ca2d7e91dd1c7090f

The three routes we need are as follows:

  • /get-token: We’ll use the /get-token route to generate a token to authenticate our users
  • /github: Our GitHub webhook will hit this route. We’ll get to know more details about the functionality of this route when we start looking at the controllers.
  • /: This is the home page, where we’ll display all the feeds.

Now that we have our routes, let’s look at the controllers!

Setting Up the Controllers

The controllers are in ./controllers/index.js:

https://gist.github.com/nparsons08/66c8c7e931ff77ddc6b3e3f31d939516

So, let’s see what’s happening in our controller. We’ll go over it together bit-by-bit.

https://gist.github.com/nparsons08/2e5c75c15e7afd9b10bf78b07b8c68b9

Here we import and initialize Stream. Notice that we are using process.env.API_KEY, as described earlier, to access our APP credentials.

Next, we create a controller to generate tokens. Once a post request is sent to this API with a username, through the route we described earlier (/get-token), the controller will talk to Stream and generate a token with that username:

https://gist.github.com/nparsons08/82f4c4afbed28b95644efe0c118fec86

Also, let’s look at the githubHook controller:

https://gist.github.com/nparsons08/6d208e01132308488251ba609c11872c

When the GitHub Webhook is triggered, it hits the /github API endpoint we mentioned earlier, and it then executes this controller.

At this point, let’s create a webhook on GitHub that will be triggered when a pull request is created...

Setting Up your Github Webhook

Log in to your GitHub account and create a repository. Then, go to the settings of this repository.

GitHub - Settings

Once here, click on the Webhooks link to open the "WebHooks" page. From here, click on the Add Webhook button.

GitHub - WebHooks

Great Job! Now, head to the URL we defined in our route for GitHub WebHooks (https://yourdomain.com/github/).

GitHub - Content Type

Choose the Content-type "application/json", as we'll be expecting a JSON object from GitHub. Finally, choose the event you want to track. You can choose to monitor all activities of the repository, or just select some events like we are doing here:

GItHub - Individual Events

Scroll down the list and chose the Pull requests event. Finally, hit the Create Webhook button.

You now have a webhook ready! Each time a pull request activity happens, that hook will send us JSON as a POST request to our /github route.

Using the Webhook

Retrieving data from the webhook is extremely easy to do. Simply pull data off of req.body.pull_request, as shown in the following snippet:

https://gist.github.com/nparsons08/60bfd7374fdb5d73aa60fdd896d96d82

Create a feedUser:

https://gist.github.com/nparsons08/176845bf4936600b4fc0f25dd13485a6

Then, create an activity for that feed user on Stream like so:

https://gist.github.com/nparsons08/cfe00e21439fbad67c4d2c320162d544

You can add as many items as you want from the request to the activity, as you receive them all from the GitHub webhook.

When the activity is created, it triggers a feed notification event that we’ll catch in the frontend and show to the user.

Let’s take a look at how we’ll handle the event in the frontend and display it on the interface; add this code to the /public/index.js file:

https://gist.github.com/nparsons08/ec36910fdc88c49384f5774b22419f95

Let’s break it down to see what’s going on here...

The first thing we need to do is authenticate the user:

https://gist.github.com/nparsons08/6a96004271fd8cc14b747b43c876eae3

Here, we make an API call to the /get-token API endpoint, which returns a token that we’ll use to authenticate the user.

Once we get the token, we use it to connect stream together with the APP_ID and API_KEY:

https://gist.github.com/nparsons08/52c98ea0cfd7f41f29593115ad8c2a65

Remember to use the same username and feed type you used in your backend;
in our case, we are passing the same username parameter in the init function that we had on the backend.

https://gist.github.com/nparsons08/e618affffc0a513f8f36bf8c05f71aef

Now, let’s subscribe to that feed:

https://gist.github.com/nparsons08/856abf5393cfc365d29550b0e280cf45

The callback method is called if everything is okay, else, the failCallback method is called.

Once our call back is called, we push the feed data to the UI:

https://gist.github.com/nparsons08/cb248b42918e749b9f9c9be995961342

And we can also pull all the feeds for this user, as shown here:

https://gist.github.com/nparsons08/906363900cd629d24d1fa4d4818650d0

That’s pretty much all there is to do to get our Github webhook to work with Stream!

Wrapping Up

Let’s look at the workflow again in 5 steps:

  1. Create a webhook on Github.
  2. Create a feed client.
  3. Create an activity for that feed client when the webhook hits the backend.
  4. Connect to Stream on the frontend and subscribe to the same feed you created in the backend.
  5. Push the data from the call back to the UI.

These are the necessary steps you need to get Stream to work with GitHub Webhook or any other WebHooks!

There is so much more you can do with Stream Feeds; check out the documentation to find out more! You don’t need to spend the time to build the activity feed infrastructure. You don’t need to bother with databases, servers, or the like; Stream provides you with all of the power you need to build amazing and scalable activity feed apps in hours.

You can check out the full and completed project on GitHub

Happy coding!

Tutorials

Feeds