SQS

Stream can send payloads of all events from your application to an Amazon SQS queue you own.

A chat application with a lot of users generates a lots of events. With a standard Webhook configuration, events are posted to your server and can overwhelm unprepared servers during high-use periods. While the server is out, it will not be able to receive Webhooks and will fail to process them. One way to avoid this issue is to use Stream Chat’s support for sending webhooks to Amazon SQS.

SQS removes the chance of losing data for Chat events by providing a large, scalable bucket that holds events generated by Steam Chat in a queue for your server or other.

The complete list of supported events is identical to those sent through webhooks and can be found on the Events page.

Configuration

You can configure your SQS queue through the Stream Dashboard or using the Dashboard or programmatically using the REST API or an SDK with Server Side Authorization.

There are 2 ways to configure authentication on your SQS queue:

By providing a key and secret
Or by having Stream’s AWS account assume a role on your SQS queue. With this option you omit the key and secret, but instead you set up a resource-based policy to grant Stream SendMessage permission on your SQS queue. The following policy needs to be attached to your queue (replace the value of Resource with the fully qualified ARN of your queue):

{
  "Sid": "AllowStreamProdAccount",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::185583345998:root"
  },
  "Action": "SQS:SendMessage",
  "Resource": "arn:aws:sqs:us-west-2:1111111111:customer-sqs-for-stream"
}

Configuring SQS through the Dashboard

Open the Dashboard.
Select the App you want to configure from the App dropdown.
Open the Chat dropdown and select Overview.
Scroll down to the Amazon SQS section and enter the following information.

SQS URL
AWS Key
AWS Secret

At this point, you can click Test SQS to have Stream’s servers test their connection with SQS and report back.

Configure Programmatically

To configure an SQS queue, use a server-side SDK to set the sqs_url, sqs_key & sqs_secret app settings.

// set your SQS queue details
await serverClient.updateAppSettings({
  sqs_url: "https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue",
  sqs_key: "yourkey",
  sqs_secret: "yoursecret",
});

//send a test message
await client.testSQSSettings();

// set your SQS queue details
$client->updateAppSettings([
 "sqs_url" => "https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue",
 "sqs_key" => "yourkey",
 "sqs_secret" => "yoursecret"
]);

// send a test message
$client->checkSqs([
 "sqs_url" => "https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue",
 "sqs_key" => "yourkey",
 "sqs_secret" => "yoursecret"
]);

# set your SQS queue details
client.update_app_settings(
 sqs_url="https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue",
 sqs_key="yourkey",
 sqs_secret="yoursecret"
)

# send a test message
client.check_sqs("yourkey", "yoursecret", "https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue")

# set your SQS queue details
client.update_app_settings(
 sqs_url: 'https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue',
 sqs_key: 'yourkey',
 sqs_secret: 'yoursecret'
)

# send a test message
client.check_sqs('yourkey', 'yoursecret', 'https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue')

// set your SQS queue details
settings := &AppSettings{
	SqsURL:  "https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue",
	SqsKey:  "yourkey",
	SqsSecret: "yoursecret",
}

client.UpdateAppSettings(ctx, settings)

//send a test message
req := &CheckSQSRequest{
	SqsURL:  "https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue",
	SqsKey:  "yourkey",
	SqsSecret: "yoursecret",
}
client.CheckSqs(ctx, req)

// set your SQS queue details
App.
 .update()
 .sqsKey("yourkey")
 .sqsSecret("yoursecret")
 .sqsUrl("https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue")
 .request();

//send a test message
App.checkSqs()
	.sqsKey("yourkey")
	.sqsSecret("yoursecret")
	.sqsUrl("https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue")
	.request();

// set your SQS queue details
await appClient.UpdateAppSettingsAsync(new AppSettingsRequest
{
  SqsKey = "yourkey",
  SqsSecret = "yoursecret",
  SqsUrl = "https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue",
});

//send a test message
await appClient.CheckSqsPushAsync(new AppCheckSqsRequest
{
  SqsKey = "yourkey",
  SqsSecret = "yoursecret",
  SqsUrl = "https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue",
});

SQS Permissions

Stream needs the right permissions on your SQS queue to be able to send events to it. If updates are not showing up in your queue add the following permission policy to the queue:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1459523779000",
      "Effect": "Allow",
      "Action": [
        "sqs:GetQueueUrl",
        "sqs:SendMessage",
        "sqs:SendMessageBatch",
        "sqs:GetQueueAttributes"
      ],
      "Resource": ["arn:aws:sqs:region:acc_id:queue_name"]
    }
  ]
}

Here’s an example list of messages read from your SQS queue:

{
  "type": "message.new",
  "cid": "messaging:fun-d5f396e3-fbaf-469c-9b45-8837b4f75baa",
  "message": {
    "id": "8bffc454-e1da-4d91-8b88-a87853dfb41c",
    "text": "Welcome to the Community!",
    "html": "<p>Welcome to the Community!</p>\n",
    "type": "regular",
    "user": {
      "id": "tommaso-52ec3a5f-e916-469f-bf54-b53b5247a4b0",
      "role": "user",
      "created_at": "2020-03-30T07:54:46.207332Z",
      "updated_at": "2020-03-30T07:54:46.207719Z",
      "banned": false,
      "online": false
    },
    "attachments": [],
    "latest_reactions": [],
    "own_reactions": [],
    "reaction_counts": null,
    "reaction_scores": {},
    "reply_count": 0,
    "created_at": "2020-03-30T07:54:46.277381Z",
    "updated_at": "2020-03-30T07:54:46.277382Z",
    "mentioned_users": []
  },
  "user": {
    "id": "tommaso-52ec3a5f-e916-469f-bf54-b53b5247a4b0",
    "role": "user",
    "created_at": "2020-03-30T07:54:46.207332Z",
    "updated_at": "2020-03-30T07:54:46.207719Z",
    "banned": false,
    "online": false,
    "channel_unread_count": 0,
    "channel_last_read_at": "2020-03-30T07:54:46.270208768Z",
    "total_unread_count": 0,
    "unread_channels": 0,
    "unread_count": 0
  },
  "created_at": "2020-03-30T07:54:46.295138Z",
  "members": [
    {
      "user_id": "thierry-735d0d44-8bf1-40df-81db-fa83363ac790",
      "user": {
        "id": "tommaso-52ec3a5f-e916-469f-bf54-b53b5247a4b0",
        "role": "user",
        "created_at": "2020-03-30T07:54:46.207332Z",
        "updated_at": "2020-03-30T07:54:46.207719Z",
        "banned": false,
        "online": false
      },
      "created_at": "2020-03-30T07:54:46.255628Z",
      "updated_at": "2020-03-30T07:54:46.255628Z"
    }
  ],
  "channel_type": "messaging",
  "channel_id": "fun-d5f396e3-fbaf-469c-9b45-8837b4f75baa"
}

SQS Best practices and Assumptions

Set the maximum message size set to 256 KB.

Messages bigger than the maximum message size will be dropped.

Set up a dead-letter queue for your main queue.

This queue will hold the messages that couldn’t be processed successfully and is useful for debugging your application.

Debugging with Ngrok

SNS