How Do You Efficiently Fan-Out Activities to Large Numbers of Followers?

In January 2019, Japanese billionaire Yusaku Maezawa posted a tweet offering one million yen to 100 random people who retweeted it.

Within 24 hours, the post had been retweeted over 3.8 million times, making it at the time the most retweeted post in the platform's history.

Every one of Maezawa's followers needed that post delivered to their feed. Every retweet triggered another round of deliveries to that user's followers. A single action by a single user cascaded into hundreds of millions of write operations across the system. This is the fan-out problem, and it's the core infrastructure challenge behind every activity feed, notification system, and social timeline.

What Is Activity Fan-Out?

Activity fan-out is the process of distributing a single action (a post, a like, a comment) to every user who should see it. In a social feed, that typically means every follower of the user who performed the action. The activity fans out.

The simplest math is straightforward but unforgiving:

That's a single user. Multiply across every active account on the platform, and you start to see why fan-out is one of the hardest scaling problems in distributed systems.

The challenge breaks down into three tensions:

• Write cost vs. read cost. You can pay upfront by writing the activity to every follower's feed at post time, or you can defer by assembling each user's feed on demand. One of these costs scales with follower count; the other scales with the number of accounts a user follows.
• Latency vs. throughput. Users expect feeds to load in under 200ms. But processing millions of writes for a single post can take seconds or minutes. The system needs to deliver fast reads without blocking on expensive writes.
• Consistency vs. availability. Should every follower see the post at the same instant? Or is it acceptable for some users to see it a few seconds later? Relaxing consistency requirements opens up architectural options that strict consistency does not.

Different fan-out strategies make different tradeoffs across these three axes. The two foundational approaches are fan-out on write and fan-out on read.

How Does Fan-Out on Write Work?

Fan-out on write (also called push-based fan-out) distributes the activity as soon as it's created. When a user publishes a post, the system looks up their follower list and writes a copy of that activity (or a reference to it) into every follower's feed.

The process typically works like this:

1. The user creates a post. The system writes it to a primary activity store.
2. A fan-out worker retrieves the user's follower list.
3. The worker iterates over the list and inserts the activity ID into each follower's prebuilt feed (usually stored in an in-memory datastore such as Redis).
4. When a follower opens their feed, the system reads directly from their pre-built feed. No merging, no sorting, no computation at read time.

This approach makes reads trivially fast. Each user's feed is already assembled and waiting. They open the app, and it’s there. For the vast majority of users on a platform (those with hundreds or low thousands of followers), fan-out-on-write is efficient and simple. The write amplification is manageable, and the read path is a single key lookup.

The problems emerge at the extremes:

• What happens when a high-follower account posts? A user with 10 million followers generates 10 million writes from a single post. Those writes consume memory, network bandwidth, and worker capacity. If that user posts frequently, the fan-out pipeline can fall behind, creating a backlog that delays delivery for everyone else on the platform.
• What about followers who aren't active? Fan-out on write updates every follower's feed, regardless of whether they'll ever see it. If a follower hasn't opened the app in three months, their pre-built feed is being updated for no one. A significant percentage of registered accounts on any platform are inactive at any given time, and every write to an inactive user's feed is wasted work.

A single post can disrupt the entire pipeline. A viral post from a high-follower account can saturate worker queues, causing fan-out delays that cascade to unrelated users. Without isolation between high-follower and normal fan-out jobs, one celebrity post can degrade feed freshness platform-wide.

How Does Fan-Out on Read Work?

Fan-out on read (also called pull-based fan-out) takes the opposite approach. Instead of pre-building each user's feed, the system stores each activity once, on the author's timeline, and assembles the feed at read time.

When a user opens their feed, the system:

1. Looks up the list of accounts the user follows.
2. Fetches recent activities from each followed account's timeline.
3. Merges and sorts those activities (typically by timestamp or relevance score).
4. Returns the assembled feed to the user.

This approach eliminates write amplification entirely. A post by a user with 10 million followers requires exactly one write. The cost shifts to read time, where the system must perform a sorted merge across potentially thousands of source timelines.

The read-time merge is the bottleneck. If a user follows 500 accounts, the system needs to fetch and merge 500 sorted lists on every feed load. Even with efficient merge algorithms, this introduces latency that grows with the number of followed accounts.

Several techniques mitigate the read-time cost:

• Caching. The system caches assembled feeds with a short TTL. Subsequent reads within the TTL window hit the cache instead of re-merging. Invalidation happens when new activities arrive from followed accounts.
• Pagination with cursors. The system only merges enough activities to fill one page of the feed (typically 20-50 items). Cursor-based pagination lets subsequent pages pick up where the previous merge left off.
• Source pruning. Not all followed accounts need to be queried. If an account hasn't posted since the user's last feed load, it can be skipped. The system tracks last-post timestamps to avoid unnecessary fetches.
• Parallel fetches. Timeline fetches for each followed account can run concurrently. With a well-tuned connection pool, the latency for 500 parallel fetches is closer to the slowest single fetch than to the sum of all fetches.

Fan-out on read works well when write amplification would otherwise be prohibitive. Platforms with high-follower accounts and strong caching infrastructure benefit most, since the read-time merge cost can be amortized across cached results.

It breaks down when users follow thousands of accounts, since every feed load requires merging that many timelines. Read-heavy workloads compound the problem: users who refresh frequently multiply the compute cost. Adding ranking or filtering on top of the merge further increases latency.

What Is Hybrid Fan-Out and Why Do Most Production Systems Use It?

Fan-out on write breaks down for high-follower accounts. Fan-out on read breaks down for high-frequency readers. Production systems solve this by using both strategies simultaneously, routing activities through different paths based on the characteristics of the author and the follower.

The most common hybrid approach classifies users into tiers:

At read time, the system assembles the final feed by combining the pre-built feed (populated by fan-out on write from regular accounts) with on-demand fetches from high-follower accounts the user follows. This merge is fast because it typically involves only a handful of high-follower sources rather than hundreds.

The hybrid model introduces additional infrastructure requirements:

• User classification service. The system needs to categorize accounts by follower count and update those classifications as accounts grow. The thresholds between tiers are tunable and platform-specific.
• Activity router. A routing layer inspects each new activity and directs it to either the fan-out-on-write pipeline or the fan-out-on-read storage path. This router must handle the platform's full write throughput with minimal latency.
• Dual-path feed assembly. The read path must merge pre-built feed entries with on-demand fetches from fan-out-on-read sources. This merge needs to handle deduplication (a user might follow both a high-follower account and someone who reshared that account's post), as well as consistent pagination and ranking.

Follower activity also plays a role in routing decisions. A follower who hasn't opened the app in two weeks doesn't need their pre-built feed updated. The system can skip fan-out on write for dormant users and assemble their feed on read if and when they return. This optimization alone can reduce write volume by 30-50% on platforms with typical engagement distributions.

How Do You Implement Fan-Out at Scale?

Implementing fan-out from scratch requires building and operating:

• Classification services. Categorizes accounts by follower count and activity level to determine which fan-out path they use.
• Routing layers. Inspects each new activity and directs it to either the push or pull pipeline at the full write throughput of the platform.
• Fan-out workers. Process the follower list for each push-path activity, writing references into individual feeds in parallel.
• Feed storage. Holds pre-built feeds for push-path users and author timelines for pull-path users across hot and cold tiers.
• Caching layers. Serves recently assembled feeds from memory to avoid redundant merges on repeated reads.
• Merge logic. Combines pre-built feed entries with on-demand pulls from high-follower accounts into a single ranked or chronological feed.

Each component needs to handle failures gracefully: what happens when a fan-out worker crashes mid-delivery? How do you ensure exactly-once semantics when retrying failed writes? How do you handle schema changes to activities that are already sitting in millions of pre-built feeds?

Building all of this in-house is a significant engineering investment, typically requiring a dedicated team and months of iteration to reach production quality. Stream's Activity Feeds provides this infrastructure as a managed service, handling fan-out routing, ranked and chronological feeds, and the underlying storage and caching layers. For teams that need to ship a feed without building the distributed-systems plumbing from scratch, this removes the hardest parts of the problem.