All Engineering

Simple Steps to Optimize Your App Performance with MongoDB, Redis, and Node.js

First Things First Here at Stream, we power activity feeds for 300+ million end users and love playing around with ways to utilize our product in all sorts of use cases. Most recently, we built Winds, an open-source RSS and Podcast application with a strong focus on UI and UX. Sh…

Building an End-to-End Deep Learning GitHub Discovery Feed

There’s hardly a developer who doesn’t use GitHub. With all those stars, pulls, pushes and merges, GitHub has a plethora of data available describing the developer universe. As a Data Scientist at Stream, my job is to develop recommender systems for our clients so that they can p…

Testing Node.js in 2018

Stream powers feeds for over 300+ million end users. With all of those users relying on our infrastructure, we’re very good about testing everything that gets pushed into production. Our primary codebase is written in Go, with some remaining bits of Python. Our recent showcase ap…

React Fragments – The End of the Wrapper Div

Hi there! I’m Ken. I’m a Developer Advocate over at GetStream.io, where we build personalized and scalable activity feeds. For the last several months, I’ve been working on Winds 2.0, an open-source RSS reader and podcast listening app. It’s built in Node.js, Electron, Redux and …

Fixing the billion dollar mistake in Go by borrowing from Rust

panic: runtime error: invalid memory address or nil pointer dereference If you ever used Go, you probably saw this error at least once. Somewhere a nil pointer or nil interface was passed to a function that doesn’t handle nil. In all cases this is a programming error, either the …

Go Client Now Available

Go is one of the fastest growing languages around. We ourselves use it extensively inside the Stream API service. Support for Go was added almost two years ago by MrHenry and HyperWorks, which open-sourced a Go client. We are proud to announce that starting today, we are releasin…

Building a Node.js Powered API with Express, Mongoose & MongoDB

One of my favorite parts of my job as a Developer Evangelist at Stream is building sample applications. It is an enthralling way to engage and interact with potential and existing customers, as well as show off the fun technology we use and build with every single day. The applic…

Moving Beyond EdgeRank for Personalized Newsfeeds

This blog post is broken into two parts and harkens back to learnings from a prior post. The sum of all these parts is altogether my best effort to provide you with a framework of how to take the creation of personalized news feeds to the next level. Part 1: Theory behind a very …

Using .IO Domain Names for Production Traffic

We’re currently updating this article. Please check back soon. In the meantime, check out these helpful resources: APIs: Feeds, Messaging UI Kits: Feeds, Messaging Tutorials: Feeds, Messaging

Building Your Own Instagram Discovery Engine: A Step-By-Step Tutorial

Isn’t it great how Instagram’s “Explore” section displays content that matches your interests? When you open the application, the content and recommendations shown are almost always relevant to your specific likes, interests, connections, etc. While it may be fun to think we’re t…

Why we switched from Python to Go

Updated May 14th 2019 to better reflect improvements to Go in the last 2 years (package management, better performance, faster compile times and a more mature ecosystem) Switching to a new language is always a big step, especially when only one of your team members has prior expe…

JavaScript Frontend Framework CLI Showdown

Many modern JavaScript frontend frameworks are complemented by a standalone Command Line Interface (CLI) program that assists with setting up a suitable development environment. Since many developers who work with Stream also use one or more of these frameworks, we like to stay u…

Testing Go at Stream

Stream’s API is used in production by more than 500 companies and 200 million end users. While we like to move fast, we definitely don’t like to break things. An extensive test infrastructure enables us to move quickly and deploy code with confidence. A solid testing workflow is …

Follow Recommendations in Social Networks

Social media is a series of networks connecting individuals, companies, organizations, and groups to one another. These networks can transcend local, national, and international borders connecting people to networks far and wide. With all those connections, how can a user find th…

Adding TypeScript Type Definitions to the Stream JavaScript API Client Library

TypeScript is a language that Stream has been interested in for quite some time. Over the past year, we’ve had an increasing number conversations with TypeScript and Angular developers. So naturally, it’s been our ambition to eventually distribute Type Definitions for our JavaScr…

Examining Decentralized Social Networks

Most companies who create a social media network do so with the end goal of collecting information, interests and habits of their users in order to monetize that data (usually through advertising). They guard this data heavily and many of the largest social networks are trusted e…

Best Practices for Recommendation Engines

In this blogpost I will describe how to implement a feature-rich activity feed that will make relevant and accurate personalization algorithms easier to implement. As we have already explored in previous blog posts, app personalization is linking activity feeds and user engagemen…

Building a Performant API using Go and Cassandra

Introduction Software performance is critical to a SaaS company like Stream and while the majority of our infrastructure is written in Python, we are actively porting portions of our code base to Go. Stream is an API for building scalable feeds and now handles over 20 billion fee…

Factorization Machines for Recommendation Systems

As a Data Scientist that works on Feed Personalization, I find it it important to stay up to date with the current state of Machine Learning and its applications. Most of the time, using some of the better-known recommendation algorithms yields good initial results; however, some…

Example Ranking Methods for Your Feeds

In this short tutorial we will show you how to use Custom Ranking for your activity streams and news feeds. By default all feeds on Stream are ranked chronologically. Custom ranking allows you to take full control over how your feeds are sorted. Some common use cases include: Sho…

In Depth Guide on Building a REST API with Node.js, Restify & MongoDB

Update for August 2017: This guide has been updated for Restify 5 and Mongo 3.4 with modern Javascript practices. Clone the Github repo to get the full working project. Choosing the Technology With the multitude of JavaScript frameworks and databases available nowadays, there are…

Personalization & Machine Learning for News Feeds and Social Networks

Winds is an open source RSS reader is powered by React, Redux, Sails and Stream. This tutorial explains how we’ve built personalization for Winds, as an example of how using Stream makes it easy to build personalized feeds. About Personalization Personalization is a very broad co…

An Introduction to Contextual Bandits

In this post I discuss the Multi Armed Bandit problem and its applications to feed personalization. First, I will use a simple synthetic example to visualize arm selection in with bandit algorithms, I also evaluate the performance of some of the best known algorithms on a dataset…

How to Setup a Highly Available Multi-AZ Cassandra Cluster on AWS EC2

Originally built by Facebook in 2009, Apache Cassandra is a free and open-source distributed database designed to handle large amounts of data across a large number of servers. Cassandra at Stream At Stream, we use Cassandra as the primary data store for our feeds. Cassandra stan…

Fast Recommendations for Activity Streams Using Vowpal Wabbit

The problem of content discovery and recommendation is very common in many machine learning applications: social networks, news aggregators and search engines are constantly updating and tweaking their algorithms to give individual users a unique experience. Personalization engin…

React/Redux – Best Practices & Gotchas

This is a bonus post in the Cabin tutorial series created by getstream.io. Visit getstream.io/cabin for an overview of all the tutorials, as well as a live demo. The source code can be found on the Stream GitHub repository for Cabin, and all blog posts can be found at their respe…

Building News Feeds & Activity Streams With Meteor

We are happy to announce stream-meteor. This integration library makes it easy to build scalable newsfeeds and activity streams for your Meteor app.

Stream JavaScript & Node Client (V3)

Version 3 of the JavaScript & Node library adds support for new Stream API features, reduces the size of the distributable JavaScript file, creates library documentation pages, and improves browser compatibility of our client.

Redis: Reducing Memory Usage

Before switching from Redis, there are quite a few things you can do to reduce memory usage.

Python 3 & Stream Framework

Stream Framework now running on Python 3.4!

5 Reasons to Use Cassandra For Building Your Newsfeed

Users of the open source Stream-Framework often ask us if they should use Redis or Cassandra to power their newsfeed. This article highlights five scenarios in which you are better off going for Cassandra.

Scalability as a Service, Guest Post on HighScalability

Thierry wrote a guest post on HighScalability called Scalability as a Service. It explains the trend how development is moving towards a hosted component based approach.