Importing Data

LAST EDIT Oct 07 2024

If you've just started using Stream, you might want to import data from your old infrastructure. Instead of using the APIs and creating your own import scripts, you can make use of our import feature. If you are on a paid plan you can import data into your app via the dashboard.

The Process

Copied!

The steps for importing data into your app are as follows:

  1. Generate the import file for your data (full file reference below)

  2. Upload the import file via the dashboard. The import file is limited to a maximum size of 300MB. For larger imports please contact us

  3. The file will be validated according to the rules described in this doc

  4. If validation passes, a member of our team will approve and run the import

  5. Once the import is completed (usually a few minutes), you will get a confirmation email

Before uploading the import file make sure that every feed group in your import file is configured in your app.
Importing data on a live app may cause high response times for requests during the import process.

Import File

Copied!

The import file must be in the JSON Lines text format (newline-delimited json). That means that each line must be a valid json. More info here.

Below example snippets are written in multiple lines for clarity but they are wrong formats for JSON Lines because each instruction must occupy a single line. If you're copying these examples directly, get rid of line breaks. Otherwise, import will fail in analyze step.

Instruction Reference

Copied!

Each line consists of an import instruction and must have the following elements:

Name

Type

Description

instruction

string

Instruction name. Must be a valid instruction type.

feedId

string

The feed to import data to, required by add_activities and follow

data

array

The data for that particular instruction, see below.

Instruction Types

Copied!

An import file can contain different types of instructions. This is the list of supported instruction types:

  • add_activities

  • follow

  • reactions

  • users

  • collections

follow

Copied!

An array of feed ids. As you can only follow flat feeds, make sure that all the feeds are flat feeds.

add_activities

Copied!

An array of activities at max 3600 in a line. The rules described in the adding activities doc apply for each activity. In addition, each activity must have the foreign_id and time fields present. The time field must have the RFC3339 format.

collections

Copied!

An array of collection entries in which each line can have up to 10,000 entries. Entries must have the following fields not empty: collectionid and data.

users

Copied!

An array of users in which each line can have up to 10,000 entries. Entries must have the id field.

reactions

Copied!

An array of reactions in which each line can have up to 10,000 entries. Entries must include: id, kinduser_idactivity_idtime. If the related activity is part of the same import, activity_foreign_id and activity_time replace activity_id. Check the REST docs for the complete list of allowed fields.