Importing Data
Confused about "Importing Data"?
Let us know how we can improve our documentation:
If you've just started using Stream, you might want to import data from your infrastructure or previous chat provider. User, Channel, and Message data can all be imported through the Stream Dashboard.
Importing From Dashboard
Copied!Confused about "Importing From Dashboard"?
Let us know how we can improve our documentation:
From your dashboard select the app you wish to import chat data for. Then click on 'App Settings' in the top-right corner of your screen, and click 'Import Chat Data'
You will be presented with a modal asking for a JSON file. Please ensure you follow our import format below to format to avoid errors.
Once you have uploaded your file, scroll down on the chat overview page to see a ‘Data Import’ section. Here you can view some details about the file upload:
The person who uploaded the file
The date the file was uploaded
The filename that was uploaded
The current status of the upload
If your import fails validation on our end you can upload a new file by clicking on the ‘New Import’ button. To view more details of your import click on it within the list shown under ‘Data Imports’. This will show a modal where you can see the size of the import and a preview of the JSON that was uploaded.
What To Expect
Copied!Confused about "What To Expect"?
Let us know how we can improve our documentation:
Once the file validation begins, the status of the job will update to analyzing
.
If unsuccessful, the job moves to analyze_failed
status including a json with the results to help correct errors. Once corrected, upload a new file, which will create a new job.
If successfully validated, the job moves to waiting_confirmation
status. We will approve the import on our side, and the import process will startcustomer support to complete the rest of the import process. A completed
status means the data has been imported and will be visible on the dashboard explorer, or through API query.
A file which is error free can be imported quickly, however, it is recommended to allow up to 7 days for this process as it can be difficult to predict errors, especially with large amounts of data.
Import File Format
Copied!Confused about "Import File Format"?
Let us know how we can improve our documentation:
The import file must be structured as a JSON array of objects. Each object is an item to add to your application (eg. a user, a message, etc.).
Here's an example of an import file:
Import Entries Format
Copied!Confused about "Import Entries Format"?
Let us know how we can improve our documentation:
Name | Type | Description |
---|---|---|
type | string | the type of data for this entry. Allowed values are: user, channel, message, reaction, member |
item | object | the data for this entry, see below for the format of each type |
Item Types
Copied!Confused about "Item Types"?
Let us know how we can improve our documentation:
An import file can contain entries with any of these types.
User Type
Copied!Confused about "User Type"?
Let us know how we can improve our documentation:
Any user referenced in the import file needs to be included in its entirety, even if it already exists. The user
type fields are shown below:
name | type | description | default | optional |
---|---|---|---|---|
id | string | the unique id for the user | - | ✓ |
created_at | string | user creation time in RFC3339 format | - | ✓ |
deleted_at | string | user deletion time in RFC3339 format | - | ✓ |
updated_at | string | user update time in RFC3339 format | - | ✓ |
role | string | user role | - | ✓ |
invisible | boolean | user visibility | - | ✓ |
teams | list of string | list of teams the user is part of | - | ✓ |
* | string/list/object | add as many custom fields as needed | - | ✓ |
Channel Type
Copied!Confused about "Channel Type"?
Let us know how we can improve our documentation:
The channel type
fields are shown below:
name | type | description | default | optional |
---|---|---|---|---|
id | string | the unique id for the channel, not required if you provide member_ids | ||
type | string | the type of the channel. ie livestream, messaging etc. | ||
created_by | string | the id of the user that created the message | ||
frozen | boolean | you can't add messages to a frozen channel | false | ✓ |
member_ids | list | can be used to generate the channel lD based on member IDs and create distinct channels | ✓ | |
disabled | boolean | if the channel is disabled | false | ✓ |
created_at | string | channel creation time in RFC3339 format | - | ✓ |
updated_at | string | channel update time in RFC3339 format | - | ✓ |
* | string/list/object | add as many custom fields as needed | - | ✓ |
Distinct Channels
Copied!Distinct channels are channels that are defined by their specific members, not by id. In this case the Channel will be defined as:
Any object referencing this channel (including Member objects), will have to reference this channel similarly:
Member Type
Copied!Confused about "Member Type"?
Let us know how we can improve our documentation:
Channel members store the mapping between users and channels. The fields are shown below:
name | type | description | default | optional |
---|---|---|---|---|
channel_type | string | the type of channel | ||
channel_id | string | the id of the channel | ||
user_id | string | the id of the user | ||
channel_role | string | the role of the member in channel (channel_member, channel_moderator or any custom role) | channel_member | ✓ |
created_at | string | the time the member was created in RFC3339 format | the import time | ✓ |
last_read | string | the time when the member last read the channel in RFC3339 format | ✓ |
Message Type
Copied!Confused about "Message Type"?
Let us know how we can improve our documentation:
name | type | description | default | optional |
---|---|---|---|---|
id | string | the id of the message | A random UUIDv4 | |
channel_type | string | the type of channel for this message | ||
channel_id | string | the id of the channel for this message | ||
type | string | the type of the message, regular, deleted, system or blocked | regular | ✓ |
user | string | the id of the user that posted the message | ||
attachments | list of attachments | list of message attachments, see the attachment section below | [] | ✓ |
parent_id | string | the id of the parent message (if the message is a reply) | null | ✓ |
created_at | string | message creation time in RFC3339 format | ||
updated_at | string | last time the message was updated | created_at | ✓ |
deleted_at | string | message deletion time in RFC3339 format | null | ✓ |
show_in_channel | bool | reply messages are only shown inside the message thread unless show_in_channel is set to true, in that case the message is shown in the channel as well | false | ✓ |
* | string/list/object | Add as many custom fields as needed | ✓ |
Message Attachments
Copied!Confused about "Message Attachments"?
Let us know how we can improve our documentation:
The only required field of an attachment is "type". All other fields are optional, and you can add as many custom fields as you'd like. The attachments are a great way to extend Stream's functionality. If you want to have a custom product attachment, location attachment, checkout, etc., attachments are the way to go. The fields below are automatically picked up and shown by our component libraries.
We support the migration of attachments to our CDN, you should use the the field migrate_resources
and set it to true.
Reaction Type
Copied!Confused about "Reaction Type"?
Let us know how we can improve our documentation:
The reaction type has the following fields:
name | type | description | default | optional |
---|---|---|---|---|
message_id | string | The id of the message | ||
type | string | The type of reaction (up to you to define the types, it's a string) | ||
user_id | string | The ID of the user | ||
created_at | string | The creation time in RFC3339 | ||
* | string/list/object | Add as many custom fields as needed | ✓ |
Error Messages
Copied!Confused about "Error Messages"?
Let us know how we can improve our documentation:
When problems occur during analyzing, they will show up in the dashboard. A list of errors will be shown in JSON format. Where applicable, the offending item will be included, for example:
Error | Description |
Validation error: max "field" length exceeded ("field-length") | Maximum length of field exceeded |
Validation error: either channel.id or channel.member_ids should be provided, but not both | Either define channel as a regular channel or a distinct channel, but not both |
Validation error: channel.id or channel.member_ids required, but not both | At least on of either channel_id member_ids should be provided |
Validation error: "field" required | Missing required field |
Validation error: created_by user "username" doesn't exist (channel "messaging:abc"). please include all users as separate user entries | All users referenced by all objects, for example in channel.created_by_id, should be included in the import file. |
Parse error: invalid item type "foobar" | An item was included with an invalid item type, only: user, channel, member, message & reaction are allowed. |
Parse error: invalid character ',' looking for beginning of value | The import contains malformed json |
Importing With CLI Tool
Copied!Confused about "Importing With CLI Tool"?
Let us know how we can improve our documentation:
Stream CLI allows you to manage your Stream Chat Imports easily and validate your imports quickly before uploading them for processing. Please, refer to our CLI docs to learn more about import.