Architecture & Benchmark

APIs should scale with your growth, both from a tech and pricing perspective. Here we explain the architecture that allows the activity feed API to scale to apps with hundreds of millions of users, while keeping response times below 25ms.

We'll cover 3 categories:

• Tech deep dive / architecture
• Benchmark results
• For you style personalization

Activity Feeds Tech Deep Dive

Push/Pull & Fanout → Materialized Feed

Activity feeds are typically powered using push (fanout on write) or pull (fanout on read). As we started working on feeds we worked through many iterations of pull and push based architectures on various databases. We've found the best results come from using a combination of these approaches and building what we call a Materialized Feed.

The concept of the materialized feed is similar to what a database does when creating a materialized view. If a user follows 100 other users it will take these activities and store it. The next time you open your feed it will fetch the changes/delta of changes by those users. There are a few clever tricks that speed it up:

• In most apps many users/feeds are inactive. The algorithm for updating feeds prioritizes recently active feeds
• As it builds the top X items in the feed in a heap, it keeps track of which feeds don't have recent enough content / can't contribute to this feed
• At first we used RocksDB/raft for this. The most recent iteration uses TiKV, a very low level database
• Column denormalization. The fields used for ranking or aggregation are denormalized. This allows you to rank feeds before loading the full activity.

The end result is a feed that can load in less than 10ms. Our work here was inspired by the Yahoo paper on activity feeds, and LinkedIn's work on activity feeds.

Go

Go is an extremely good fit for activity feeds. It handles aggregation, ranking, push and websockets very well. The websocket implementation uses cuckoo filters, so it has a lightweight way to validate if a user is active on a given server.

Client Side Caching & Cache Stampede Protection

We use Redis client side caching for optimal performance. For high traffic feeds, this client side caching approach saves you a roundtrip to Redis. A combination of ristretto and singleflight help prevent cache stampedes.

Filtering

A common feature request is to filter a feed on a certain tag. For instance only showing images, or only videos, or only posts in a certain language. The filtering system for feeds allows you to look for up to 10 tags. It also limits you to equal or in, not in lookups aren't allowed. This makes it possible to create a materialized feed for the specific filter. If your app commonly shows a filtered view of the feed it will run with a performance that's close to identical to an unfiltered feed.

Aggregation & Ranking

Aggregation keys are precomputed on write. This ensures that opening up an aggregated feed is extremely fast.

Ranked feeds are more dynamic in nature and can't be precomputed. Feeds includes a fast expression parsing library that enables us to rerank feeds quickly.

Hotspot Prevention

Social networks often have hotspots in the data. Everyone is following the same few people, people open a few activities repeatedly etc. So it's important to ensure that updates which happen on the activity and feed level don't break the database by causing hot spots.

Infrastructure & QA

Infra is defined in code and runs on a multi AZ high availability setup on AWS. An extensive test suite, monitoring and production smoke tests ensure a high uptime. (100% over the last 12 months at the time this was written.) We also offer a 99.999% uptime SLA with acceleration.

Benchmark Results

For You Feeds & Personalization

A for you style feed is key for user engagement. Out of the box we ship with the following activity selectors:

• Following
• Popularity
• Location
• Interest
• Activities from people in your follower suggestions
• Query activities (anything you can query on)

You can add up to 10 different ways to select activities and rank the end result with any of our ranking methods.

You can also enrich activities as they are added. The AI/LLM based enriches can add topics for further improvements to interest based ranking.

We're always expanding the list of built-in capabilities for ranking / for you style feeds. Be sure to reach out if you have suggestions.

Transparency & Tech

When you select an API you want to know it can scale and will be reliable. So instead of just claiming that Stream works for large apps, we like to share these details on how we achieve the performance and scalability. If you'd like to learn more about Stream be sure to check out the docs or contact our team.