Building an activity feed at scale is a demanding engineering problem. Real-time updates, high-volume content creation, media handling, personalized ranking, and user safeguarding all depend on backend systems that are reliable, consistent, and globally performant.
The Stream Activity Feeds API provides your application with a solid foundation. Feeds v3 is built on a highly available, replicated architecture using Go, Raft, RocksDB, and a global edge network to deliver predictable performance at enterprise scale. The system is designed to be modular, extensible, and production-ready, enabling you to build complex social and community experiences without having to construct the underlying infrastructure yourself.
The sections below walk through the core design principles, building blocks, and architectural patterns that power the Feeds v3 API. We’ll also cover how those choices support scalability, personalization, security, and real-time performance for modern applications.
Design Philosophy
Stream Feeds incorporates media enrichment, feed aggregation and ranking, follow and unfollow functionality, real-time updates, moderation, and more to support sophisticated online community experiences and services for production and enterprise use cases.
The API’s backend is built on Raft, a robust standard Go library for handling replicated logs. The backend infrastructure integration with Go and Raft enables the development of secure and reliable feed solutions for teams, using RocksDB for high performance and fast storage.
To provide an optimal, low-latency feed delivery experience, Stream maintains a global edge network of servers for the feed API’s foundation to run its entire stack. The feed API is built upon several pillars and design principles, allowing enterprises and developers to create solutions that are easily extendable, customizable, and modular.
The following are some of the key principles behind the Feeds v3 Architecture.
- Enterprise Scalability: Stream has the necessary technologies and API infrastructure to scale any activity feed solution worldwide without hindering the app’s performance or uptime. Activity feeds built using the Stream API have a response time of less than 10 milliseconds. This is possible due to our custom Go and Raft implementation with the backend.
- Ease of Integration: Build fully featured activity feeds in days with client and server-side platforms, including React, iOS, Android, Flutter, Node, Go, PHP, and Python.
- Personalization: Build personalized feeds features into enterprise-grade apps for discovery and interest-based content for users. Developers can use advanced ranking techniques to recommend content to users based on their locations, past interactions with a service, and the people they follow.
- Modularity: Developers can mix and match various components of our client and server-side feed SDKs to build social apps with popular features, such as the For You feed. This can be achieved using built-in activity selectors and a flexible ranking system, along with filtering, which provides an efficient way to display trending, interest-based, or location-aware activities.
- Security: Stream is committed to keeping business and enterprise customers safe and secure, using industry-standard technologies and services to protect users’ data from unauthorized access. We comply with relevant regulations, including SOC 2, ISO 27001, HIPAA, and GDPR.
- Simple Migration Architecture: The backend is designed to make it simple to replicate, import, and export from previous versions to v3 with minimal effort.
Building Blocks
The underlying API structure is built upon the W3C Activity Streams 2.0 specification, and the current iteration focuses on personalization, rich media, and client SDKs. It provides a flexible and straightforward abstraction for implementing feeds, activities, comments, reactions, push notifications, and more.
The following are the essential building blocks of the Feeds API and SDKs.
Activities
The fundamental units that appear in feeds are referred to as activities. An activity can take the form of posts, photos, videos, or polls on popular social and community platforms, such as Instagram, X, and Reddit. The API also provides an easy way to create custom content types for activities.
Adding an activity to feeds requires a User field and optional Type and Content fields.
- User: A string representing the entity posting the activity.
- Type: A string representing a custom type, post, video, attachment, image, etc.
- Content: Specifies the actual information users want to share.
Type has a default value called post. However, it can be a strin
Feeds
A stack of several activities, in the form of a first-in, last-out (FILO) queue, constitutes feeds.
A feed can be categorized as personal, timeline, or notification. Developers can also define custom feed types for specific application areas. A single application can contain several feeds with activities posted to it. A feed can be propagated using Fan-Out and Follow relationships, techniques that show activities to users based on the people they follow.
Stream Feeds enable developers to use templates to define the behavior of various feed types within an application for specific purposes, utilizing techniques such as activity selection, aggregation, ranking, and processors. The API supports feed groups like user, notification, and timeline. A custom feed group can also be created.
Client and Server-Side Calls Separation
The Stream Activity Feeds API is designed to allow API calls to be done on both the client and server sides. The separation of these two layers enables the backend to handle aspects that may have more security concerns, while keeping the front end agile.
- Feed Interactions: Client-side API call can be used for feed interactions using our mobile and web SDKs, such as iOS, Android, React Native, React, and Flutter.
- Security: The API securely implements the user-facing functionality using a token to control what can be viewed.
- Robust Permission System: Client calls validate users with an authentication token using a robust permission system.
The implementation of the server-side requires the use of an API key and secret for elevated access to feed applications built with Stream. Since the backend has full permission to the system, it can be used to sync users, update feed groups and views, and issue user tokens.
In general, the client-side (web and mobile) is used for reading feeds belonging to a particular user, posting, liking, commenting on added activities, managing changes to the feed UI, and receiving updates via WebSockets. The server-side service helps handle feed configurations, including aggregation, ranking, imports, and batch operations.
Realtime Event Updates Architecture
Real-time activity updates are crucial for Stream when building a feed application with our API. The API architecture is designed to enable the low-latency delivery of notifications and updates for new activities, allowing your development teams to focus on the app’s logic.
During activity insertion and Fan Out, we use a globally distributed feed storage to persist all activities. New activities in a user’s notification and timeline feeds are always delivered based on other people following them.
Real-time events are delivered on the client-side via WebSockets. A custom transport can also be implemented for server-side processing. We rewrote our real-time mechanism for Python in Go to enable a more efficient pipeline and deliver low-latency live activity feeds for use cases such as real-time social, live sports updates, and streaming.
Token and Authentication
The Stream Feeds API uses JSON Web Tokens (JWT) with robust logic to separate user authorization from authentication.
During a user sign-in, Stream-powered feed apps must call a server-side token generation and provider for authentication and ensure a clear security boundary. Your app’s backend can access the token provider’s endpoint after implementation to validate users, for example, with a session ID against your backend’s data storage integration. The tokens have a limited lifespan. However, they can be configured with a custom expiration.
Using a server-side JWT helps to prevent token forgery and malicious attacks from bad actors. The JWT architecture also helps to refresh expired user tokens without any friction, ensuring secure sessions and avoiding frequent re-logins.
Activity feeds user management is also unified with our Chat and Video products. This provides a seamless mechanism for using a particular feed user for the other two services. The API’s architecture is designed to eliminate any data synchronization issues.
To comply with global data regulatory requirements, the Activity Feeds API provides endpoints for deleting and exporting user data. The data export integration enables businesses to export all information storage for specific users, including their activities, profile, metadata, and feed items, into a portable JSON format, thereby adhering to users’ right to access.
The API also provides support for erasing user data, like activity records. Our built-in data erasure and export endpoints help developers, businesses, and enterprises comply with the General Data Protection Regulation (GDPR) in specific regions or countries.
Scalable Feeds Architecture
One of the primary challenges of integrating activity feeds with any service or platform is scaling them to support an unlimited number of active users over time. Our API service can be used for building a wide range of scalable applications for social, news, and instant notification (alert) systems. Our platform is used by popular social networks, marketplaces, and content-heavy platforms that require reliable, real-time updates.
To ensure high concurrency and infinite scalability of the Feeds API, we utilize pull-based aggregation, distributed key-value storage, and adaptable data structures, while maintaining a low latency level. To enable teams to build and scale faster, we manage all the intricacies of implementing feeds for millions of users, including filtering, ranking, and sharding.
- Pull-Based Foundation with TiKV: The Stream Feeds API is built upon TiKV’s distributed storage layer for efficient and fast retrieval of feed data. The implementation of this storage engine enables our API to handle massive activity feed datasets by fetching them on demand, thereby minimizing server loads during peak times, rather than using the traditional push-based approach for broadcasting or pushing updates to every follower’s feed. The pull-based approach helps to scale the API architecture horizontally and eliminates the bottlenecks associated with building activity feed applications for millions of followers.
- Materialized View-Like Feed Model: To help easily scale activity feed solutions for large user group use cases, we built the Stream Feeds API with an approach that treats feed as materialized views instead of full feed materialization. This approach helps to prevent the duplication of activity objects across multiple fanouts. Treating each feed as a materialized view at the feed level, rather than the table level, also helps optimize feed construction for efficient and faster reads.
- Gathering Recent Feed Items: To retrieve the newest feed items (posts, shares, etc.), the API uses a heap-based merge approach for priority queueing. Using the heap method allows for the insertion of the newest activities into a max-heap, enabling the efficient extraction of the latest feed entries. This approach results in an early termination during data iteration in user-followed feeds, feed groups, and channels, minimizing the time spent scanning irrelevant data history. To retrieve the latest feed result efficiently, source feeds, such as user-followed feeds and channels, are iterated based on their last-used pointer. A min-heap or max-heap is used to merge the recent items across all sources.
- Radis Client-Side Caching: Caching plays an essential role in scaling any feed solution. Our high-performance client-side caching implementation, utilizing Redis, provides a scalable caching layer to build always-available and fast activity feed solutions. The Redis client-side caching is an excellent solution for building real-time feed applications at an enterprise scale. The implementation enables sub-millisecond latency and high performance for enterprise feed services.
- Granular Activity Targeting: The activity feed architecture utilizes granular and tag-based filtering, enabling content to be fetched through subsets without degrading performance. Activities can be queried using up to 10 distinct tags to build personalized views, such as those highlighting specific content or media only.
- Duckoo Filter for Sockets: For real-time updates to activity feeds, our API’s architecture uses a probabilistic data structure to handle WebSockets-based updates in a fast and space-efficient way. It implements Cuckoo Filter to efficiently identify specific socket connections that require updates, resulting in a minimal memory footprint and ensuring reliable broadcast.
Feeds Moderation
Moderating and safeguarding user activities, content creation, and interactions are must-have features of scalable community and social platforms.
When building a feed solution like X or Reddit, user-posted content may include spam, toxicity, policy violations, harassment, and other issues. Our Feeds API comes with a reliable manual and AI moderation solution to prevent app policy violations and unacceptable user behaviors. The moderation platform features a user-friendly dashboard that allows administrators to perform actions, review, and manage system violations. It has built-in user-level tools for banning, flagging, muting, and blocking specific users.
The moderation service integration with any activity feed-powered app enables developers to have full control over the backend, preventing false-positive issues and ensuring reliable auditability of activity content for community-powered feed products.
Push Notifications
In community and social platforms, updating users with timely information is essential for retention and engagement. The Feeds API provides support for pushes through the Apple Push Notification Service (APNs) and Firebase Cloud Messaging (FCM).
The seamless built-in integration enables scalable feed products to share real-time updates, such as new activity posts, reactions, mentions, follow, and unfollow actions to end-user devices. The above actions are default, already-made features of the Feeds API.
Stream Activity Feeds v2 to v3 Migration
Stream Activity Feeds v3 employs an entirely different mental model, resulting in breaking change but making integration with our other products, such as Chat, Moderation, and Video, simple. If your app is powered by v2 of the Feeds API, the following are the core architectural changes and why to migrate to v3.
| Area | Feed v2 | Feed v3 | Migration (Python Example) |
|---|---|---|---|
| Activity uniqueness | \foreign\_id\ + optional time | Custom ID field (UUID or your primary key) | \foreign\_id\ is no longer needed. Use your stable ID (recommended: UUID4). |
| Activity field names | actor, verb, object, to, custom top-level fields | user, type, attachments, feeds, structured content, or custom objects | \actor\ → \user\, \verb\ → \type\, Top-level custom fields like \image\_url\, \title\ → move into \attachments\ array or nested objects, \to\ targets → \feeds\ array |
| Feed creation | Implicit: feed created on first write | Explicit: Feed must exist before adding activities | Call \client.feed\_group('user').create\_feed('123')\ or use the dashboard/API to pre-create. Feeds now have an explicit owner, which simplifies the management of client-side tokens. |
| Aggregated & Notification feeds | Separate feed types (aggregated, notification) | Same feed group, controlled by settings (aggregation and notification flags) | You can now toggle aggregation/notification per feed group or even per feed without creating new groups. |
| Comments | Implemented as reactions with the kind "comment" | Native nested Comment objects | Switch to the new Comments API. Comments support arbitrary nesting depth and separate read/mark-seen semantics. |
| Pagination limit | Hard 1000-activity limit per feed | No limit removed + activity selectors & queries | You can now paginate indefinitely, include activities from other feeds via selectors, or run more complex queries (such as those based on popularity, etc.). |
| User data | Isolated to the Feeds product | Shared across Stream Chat, Video, and Feeds | Existing user profiles automatically become shared. New fields added in one product become visible in others. |
Feeds v3 Performance Improvements
Stream Activity Feed v3 is designed with numerous performance enhancements and new features, including For-You feeds, interest-based ranking, activity selectors, and more. Our engineering team's testing demonstrates a 20-30% improvement in performance and a reduction in latency for feed aggregation and ranking.
| Area | Improvement | Impact |
|---|---|---|
| Flat feeds | 20 – 30% faster than v2 | Faster timeline loading, even at enterprise scale |
| Feeds aggregated/notifications pipeline | Significant speed in grouping + ranking | Sorting in the v3 API feels instantaneous |
| Activity filtering | Nearly zero performance hit | You can now filter by type, content, and custom fields |
| For-You / selector-based feeds | Combined activity selectors + ranking in a single request | Personalized feeds mixing following + popular + interest with one API call |
Next Steps
The Feeds v3 API introduces a modern architecture built for personalization, real-time delivery, and large-scale feed aggregation. Features like For-You selectors and improved ranking pipelines enable more sophisticated recommendation systems, while the underlying storage, fan-out, and caching layers deliver a measurable 20 - 30% performance improvement.
To explore the full capabilities of Feeds v3, check out the v3 Activity Feeds API tour, browse the SDK tutorials, review pricing, or dive into the documentation.