Throttling & Slow mode

Throttling & Slow Mode

Stream’s APIs for activity feeds and chat are use by over a billion end users. From a chat API perspective it’s hard to run into limits. One thing to be careful with is the performance of the chat when many users participate in a single channel. For live events or concerts, you can sometimes have so many users, that the sheer volume of messages overloads the browser or mobile device. This can cause the UI to freeze, high CPU usage, and degraded user experience. Stream offers 3 features to help with this:

1. Channel Slow Mode

2. Automatic feature Throttling

3. Message Throttling

Channel Slow Mode

Slow mode helps reduce noise on a channel by limiting users to a maximum of 1 message per cooldown interval.

The cooldown interval is configurable and can be anything between 1 and 120 seconds. For instance, if you enable slow mode and set the cooldown interval to 30 seconds a user will be able to post at most 1 message every 30 seconds.

Slow mode is disabled by default and can be enabled/disabled via the Dashboard, using the Chat Explorer:

It can also be enabled/disabled by admins and moderators via SDK.

val channelClient = client.channel("messaging", "general")

// Enable slow mode and set cooldown to 1s
channelClient.enableSlowMode(cooldownTimeInSeconds = 1).enqueue { /* Result handling */ }

// Increase cooldown to 30s
channelClient.enableSlowMode(cooldownTimeInSeconds = 30).enqueue { /* Result handling */ }

// Disable slow mode
channelClient.disableSlowMode().enqueue { /* Result handling */ }

// enable slow mode and set cooldown to 1s
await channel.enableSlowMode(1);

// increase cooldown to 30s
await channel.enableSlowMode(30);

// disable slow mode
await channel.disableSlowMode();

let channelController = client.channelController(for: .init(type: .messaging, id: "general"))

channelController.enableSlowMode(cooldownDuration: 10)

channelController.disableSlowMode()

// Enable slow mode and set cooldown to 1s
Channel->EnableSlowMode(FTimespan::FromSeconds(1.));

// Increase cooldown to 30s
Channel->EnableSlowMode(FTimespan::FromSeconds(30.));

// Disable slow mode
Channel->DisableSlowMode();

var update = new ChannelUpdateRequest { Cooldown = 30 }; // 30 sec

await channelClient.UpdateAsync(channel.Type, channel.Id, update);

$update = $channel->update([ 'cooldown' => 30 ]); // 30 sec

channel.update({ "cooldown": 30 }) # 30 sec

channel.update({"cooldown" => 30}) # 30 sec

channel.Update(ctx, map[string]interface{}{"cooldown": 30}, nil) // 30 sec

// Not yet supported in the Unity SDK

// Android SDK
final ChannelClient channelClient = client.channel("messaging", "general");

// Enable slow mode and set cooldown to 1s
channelClient.enableSlowMode(1).enqueue(result -> { /* Result handling */ });

// Increase cooldown to 30s
channelClient.enableSlowMode(30).enqueue(result -> { /* Result handling */ });

// Disable slow mode
channelClient.disableSlowMode().enqueue(result -> { /* Result handling */ });

// Backend SDK

// Enable slow mode and set cooldown to 1s
Channel.update("messaging", "general").cooldown(1).request();

// Increase cooldown to 30s
Channel.update("messaging", "general").cooldown(30).request();

// Disable slow mode
Channel.update("messaging", "general").cooldown(0).request();

When a user posts a message during the cooldown period, the API returns an error message. You can avoid hitting the APIs and instead show such limitation on the send message UI directly. When slow mode is enabled, channels include a cooldown field containing the current cooldown period in seconds.

val channelClient = client.channel("messaging", "general")

// Get the cooldown value
channelClient.query(QueryChannelRequest()).enqueue { result ->
  if (result.isSuccess) {
    val channel = result.data()
    val cooldown = channel.cooldown

    val message = Message(text = "Hello")
    channelClient.sendMessage(message).enqueue {
      // After sending a message, block the UI temporarily
      // The disable/enable UI methods have to be implemented by you
      disableMessageSendingUi()

      Handler(Looper.getMainLooper())
        .postDelayed(::enableMessageSendingUi, cooldown.toLong())
    }
  }
}

const p = channel.sendMessage(msg);

if (channel.data.cooldown != null && channel.data.cooldown > 0) {
  p.then(() => {
    // first lock the UI so that the user is aware of the cooldown 
    disableSendMessageUI();
    // restore the UI after the cooldown is finished
    setTimeout(enableSendMessageUI, channel.data.cooldown);
  });
}

await p;

let channelController = client.channelController(for: .init(type: .messaging, id: "general"))

channelController.synchronize() {
  if error == nil {
    channelController.channel?.cooldownDuration
  }
}

Channel->SendMessage(
  Message,
  [](const bool& bSuccess)
  {
    if (Channel->Properties.Cooldown > 0)
    {
      // first lock the UI so that the user is aware of the cooldown
      DisableSendMessageUI();
      // restore the UI after the cooldown is finished
      GetWorld()->GetTimerManager().SetTimer(
        TimerHandle, 
        [] { EnableSendMessageUI(); }, 
        Channel->Properties.Cooldown.GetTotalSeconds(), false);
    }
  });

// Will be implemented soon, raise a GitHub issue if you need this feature https://github.com/GetStream/stream-chat-unity/issues/

final ChannelClient channelClient = client.channel("messaging", "general");

// Get the cooldown value
channelClient.query(new QueryChannelRequest()).enqueue(channelResult -> {
  if (channelResult.isSuccess()) {
    final Channel channel = channelResult.data();
    int cooldown = channel.getCooldown();

    Message message = new Message();
    message.setText("Hello");
    channelClient.sendMessage(message).enqueue((messageResult) -> {
      // After sending a message, block the UI temporarily
      // The disable/enable UI methods have to be implemented by you
      disableMessageSendingUi();

      new Handler(Looper.getMainLooper())
          .postDelayed(() -> enableMessageSendingUi(), cooldown);
    });
  }
});

Automatic Feature Throttling

When a channel has more than 100 active watchers Stream Chat automatically toggles off some features. This is to avoid performance degradation for end-users. Processing large amount of events can potentially increase CPU and memory usage on mobile and web apps.

1. Read events and typing indicator events are discarded

2. Watcher start/stop events are only sent once every 5 seconds

Message throttling

Message throttling that protects the client from message flooding. Chat clients will receive up to 5 messages per second and the API servers will allow small surges of messages to be delivered even if that means exceeding the 5 msg/s rate.

Here is an example of how message throttling works:

In this example, the client will receive several more messages above the 5/s limit (the yellow bar), and once this burst credit is over, the client will stop receiving more than 5 messages per second. The burst credit is set to 10 messages on an 8 seconds rolling window.