Collections

Overview

Collections provide a way to attach data to activities that is shared between these activities and can be updated in a single place. A typical use case is to track the status of an activity such as started, completed, or failed. Instead of having to update multiple activities when the status changes you can create a collection and add it to the activities that needs this data. When the status changes you only need to update the collection.

A collection consists of a name, an id and some data. You can think of the name as a namespace for the collection. A collection reference is a string that looks like <name>:<id> and is the unique identifier for the collection.

Info:

Data stored in collections can not be used in custom ranking or aggregation.

Info:

The batch endpoints supports up to 100 collections in a single request. Automatic deduplication will be applied, keeping the last occurrence only.

Creating Collections

The example below shows how to create a collection.

final response = await client.createCollections(
  collections: [
    const CollectionRequest(
      name: 'movies',
      id: 'lord_of_the_rings',
      custom: {
        'title': 'Lord of the Rings',
        'genre': 'fantasy',
        'rating': 9,
      },
    ),
  ],
);

Overview of the collection model

CollectionResponse

Name	Type	Description	Constraints
`created_at`	`number`	When the collection was created	-
`custom`	`object`	Custom data for the collection	-
`id`	`string`	Unique identifier for the collection within its name	Required
`name`	`string`	Name/type of the collection	Required
`updated_at`	`number`	When the collection was last updated	-
`user_id`	`string`	ID of the user who owns this collection	-

Updating Collections

The example below shows how to update a collection.

final response = await client.updateCollections(
  collections: [
    const CollectionRequest(
      name: 'movies',
      id: 'lord_of_the_rings',
      custom: {
        'title': 'Lord of the Rings',
        'genre': 'fantasy',
        'rating': 9.5,
      },
    ),
  ],
);

Upserting Collections

The upsert endpoint creates a collection if it does not exist, or updates it if it already exists. This is useful when you want to ensure a collection is in a specific state without checking for its existence first.

Info:

The upsert collections endpoint is server-side only.

Node.js

serverClient.feeds.upsertCollections({
  collections: [
    {
      name: "movies",
      id: "lord_of_the_rings",
      custom: {
        title: "Lord of the Rings",
        genre: "fantasy",
        rating: 9.5,
      },
    },
  ],
});

Deleting Collections

The example below shows how to delete a collection.

final response = await client.deleteCollections(
  collectionRefs: ['movies:lord_of_the_rings'],
);

Reading Collections

The example below shows how to read a collection.

final response = await client.readCollections(
  collectionRefs: ['movies:lord_of_the_rings'],
);

Query Collections

Retrieve collection entries that match a filter, with sorting and cursor-based pagination. This is useful when you don't know the exact IDs of the entries you want (unlike readCollections, which fetches by reference). For example, listing every entry in a movies collection, or all entries owned by a given user.

queryCollections requires the read collections permission. When called client-side, results are automatically scoped to the calling user: only the user's own entries and global (no-owner) entries are returned, and any user_id filter is ignored.

final query = CollectionsQuery(
  filter: const Filter.equal(CollectionsFilterField.name, 'movies'),
  sort: [CollectionsSort.desc(CollectionsSortField.createdAt)],
  limit: 10,
);
final collectionList = client.collectionList(query);
final collections = await collectionList.get();
// collections -> matching entries
// collectionList.next -> cursor for the next page

Parameters

Name	Type	Description
`filter`	object	Query filter. See the filterable fields below. Omit or pass `{}` to match all entries.
`sort`	array	Sort clauses, applied in order. Each is `{ field, direction }`. Defaults to newest first.
`limit`	int	Maximum number of entries to return per page. Defaults to `10`. Maximum is `100`.
`next`	string	Pagination cursor pointing to the next page (from a previous response's `next`).
`prev`	string	Pagination cursor pointing to the previous page. Cannot be combined with `next`.

Filterable fields

Filters use Stream's standard query syntax. A bare value is an equality match ({ "name": "movies" }); operators are expressed as nested objects ({ "created_at": { "$gte": "2024-01-01T00:00:00Z" } }).

Field	Type	Supported operators
`id`	string	`$eq`, `$in`
`name`	string	`$eq`, `$in`
`user_id`	string	`$eq`, `$in` (ignored on client-side auth)
`created_at`	date	`$eq`, `$gt`, `$gte`, `$lt`, `$lte`
`updated_at`	date	`$eq`, `$gt`, `$gte`, `$lt`, `$lte`

Sorting

sort accepts one or more clauses of the form { field, direction }, where direction is 1 for ascending and -1 for descending. Sortable fields are created_at, updated_at, and id. When sort is omitted, entries are returned by created_at descending (newest first).

Pagination

queryCollections is cursor-paginated. Each response includes a next cursor when more entries are available; pass it back as next to fetch the following page. next is absent on the last page. Keep the filter and sort identical across page requests.

let next;
const all = [];
do {
  const page = await client.queryCollections({
    filter: { name: "movies" },
    limit: 100,
    next,
  });
  all.push(...page.collections);
  next = page.next;
} while (next);

Response

Field	Type	Description
`collections`	array	Matching collection entries.
`next`	string	Cursor for the next page, if any.
`prev`	string	Cursor for the previous page, if any.
`duration`	string	Server-side processing time.

Each entry in collections has the following shape:

Field	Type	Description
`name`	string	Name/type of the collection.
`id`	string	Unique identifier within the collection name.
`custom`	object	Custom data attached to the entry.
`user_id`	string	Owner of the entry (empty for global entries).
`created_at`	date	When the entry was created.
`updated_at`	date	When the entry was last updated.

Activities and Enrichment

Collections are added to activities in the form of collection references. An activity can reference up to 10 collections.

Info:

Support for adding collections directly when creating activities is coming soon.

Adding Collections to an Activity

final response = await feed.addActivity(
  request: const FeedAddActivityRequest(
    type: 'post',
    text: 'I love this movie!',
    collectionRefs: ['movies:lord_of_the_rings'],
  ),
);

Enrichment

When you have added collection references to your activities these will automatically be enriched with the collection data when reading feeds.

// Create a collection
final createResponse = await client.createCollections(
  collections: [
    const CollectionRequest(
      name: 'movies',
      id: 'lord_of_the_rings',
      custom: {
        'title': 'Lord of the Rings',
        'genre': 'fantasy',
        'rating': 9,
      },
    ),
  ],
);

// Add the reference to an activity
final feed = client.feed(group: 'user', id: 'jack');
final activityResponse = await feed.addActivity(
  request: const FeedAddActivityRequest(
    type: 'post',
    text: 'I love this movie!',
    collectionRefs: ['movies:lord_of_the_rings'],
  ),
);

// Read the feed and see the enriched collection data
await feed.getOrCreate();
final activities = feed.state.activities;
// The activities will contain enriched collection data in the collections field
// Example response structure:
// {
//     "activities": [
//         {
//             "type": "post",
//             "text": "I love this movie!",
//             "collections": {
//                 "movies:lord_of_the_rings": {
//                     "name": "movies",
//                     "id": "lord_of_the_rings",
//                     "custom": {
//                         "title": "Lord of the Rings",
//                         "genre": "fantasy",
//                         "rating": 9
//                     },
//                     "user_id": "jack",
//                     "created_at": "2025-01-01T00:00:00.000Z",
//                     "updated_at": "2025-01-01T00:00:00.000Z",
//                     "status": "ok"
//                 }
//             }
//         }
//     ],
//     ...
// }

Info:

The enrichment is a best effort process. Missing or deleted collections will not be enriched but will be retured with a status of notfound