Skip to main content

Collection Tools Component

Common collection operations

Component key: collection-tools

Description#

The collection tools component contains actions that let you perform common tasks on collections of data (objects and arrays). You can do things like concatenate objects, filter items of lists (arrays), map functions on items, and more.

We use the terms list and array interchangably on this page. Both describe a set of objects. Similarly, the terms items and elements of a list are used interchangably.

The functions that you provide these actions for filtering, mapping, etc., should be NodeJS functions. Examples are below.

Actions#

Add Key/Value to Object#

Add a value to an object with the given key | key: addKey

InputNotesExample
Input
Key
string
/ Required
Notes
My Comments
Example
FirstName
Input
Object
code
/ Required
Notes
 
Example
{  "employee": {    "id": "123",    "hired": "2022-03-05T00:08:00",    "contact": {      "phone": "6024441234"    }  }}
Input
Insert Path
string
Notes
Optionally define a path using dot notation of where to add the key/value
Example
employee.contact
Input
Value
string
/ Required
Notes
 
Example
Jake

If you would like to insert a key/value pair into a nested portion of the object, use the Insert Path input to specify where it should be added. For example, if your object reads:

{  "employee": {    "id": "123",    "hired": "2022-03-05T00:08:00",    "contact": {      "phone": "6024441234"    }  }}

You can add an email address to contact by specifying "email" for Key, "example@company.com" for Value, and employee.contact for Insert Path. The result will be:

{  "employee": {    "id": "123",    "hired": "2022-03-05T00:08:00",    "contact": {      "phone": "6024441234",      "email": "example@company.com"    }  }}

Output Example Payload#

{  "data": {    "FirstName": "Foo",    "LastName": "Bar"  }}

Aggregate#

Apply aggregate function to list | key: aggregate

InputDefaultNotesExample
Input
Aggregate Function
string
/ Required
Default
Notes
Aggregate function to apply (choose from AVERAGE, COUNT, MAX, MIN, SUM, PRODUCT, MEDIAN).
Example
SUM
Input
Filter Function
code
Default
(item, index) => { return true; }
Notes
Filter out any elements that do not return true
Example
(item) => item.prop > 17
Input
List
code
/ Required
Default
Notes
Reference to a list of data to operate on
Example
 

This action applys the filter function (if given) first, and then applies the aggregate function to the filtered list. See the Filter action for examples of how to use the filter function.

Output Example Payload#

{  "data": 117}

Append#

Append element to the end of the list | key: append

InputNotes
Input
Element
data
/ Required
Notes
The item to append to the end of the list
Input
List
code
/ Required
Notes
Reference to a list of data to operate on

Output Example Payload#

{  "data": {    "list": [      1,      2,      3,      4    ]  }}

Chunks#

Chunk the list into lists of the specified number of elements | key: chunks

InputDefaultNotesExample
Input
List
code
/ Required
Default
Notes
Reference to a list of data to operate on
Example
 
Input
Number of Elements
string
/ Required
Default
1
Notes
Number of elements to take
Example
3

Output Example Payload#

{  "data": [    [      1,      2,      3    ],    [      4,      5,      6    ],    [      7,      8    ]  ]}

Combine Collection#

[Deprecated - use 'Create Object'] | key: combineCollections

InputNotesExample
Input
Collections
string
/ Required
Key Value List
Notes
Returns an object with the specified key and corresponding collection as the value
Example
[ {"key": "CustomerQueryResult", "value": { Customer: { Name: "Acme Contracting" } }}, {"key": "AccountQueryResult", "value": { Account: { AccountId: "123123123" } }}, ]

Output Example Payload#

{  "data": {    "CustomerQueryResult": {      "Customer": {        "Name": "Acme Contracting"      }    },    "AccountQueryResult": {      "Account": {        "AccountId": "123123123"      }    }  }}

Concatenate#

Concatenate two lists together into a single list | key: concatenate

InputNotesExample
Input
List
code
/ Required
Notes
Reference to a list of data to operate on
Example
[4,5,6]
Input
List
code
/ Required
Notes
Reference to a list of data to operate on
Example
[1,2,3]

Output Example Payload#

{  "data": [    1,    2,    3,    4,    5,    6  ]}

Count#

Count the number of occurrences of element in list | key: count

InputNotes
Input
Element
data
/ Required
Notes
Reference to an element to look for
Input
List
code
/ Required
Notes
Reference to a list of data to operate on

Output Example Payload#

{  "data": 3}

Create List#

Create a new list with the given inputs | key: create

InputNotes
Input
List Items
data
Value List
Notes
 

Output Example Payload#

{  "data": [    1,    2,    3,    4  ]}

Create Object#

Creates a new object from provided key/value pairs | key: createObject

InputNotes
Input
Key & Value Pairs
data
/ Required
Key Value List
Notes
Key and value pairs

Output Example Payload#

{  "data": {    "first": "value",    "second": 17  }}

De-duplicate#

De-duplicate the elements of the list | key: deduplicate

InputNotes
Input
List
code
/ Required
Notes
Reference to a list of data to operate on

Output Example Payload#

{  "data": [    1,    2,    3  ]}

Field Value Mapping#

Maps the values from two different collections and returns a key/value list where the 'key' is the value of the Key Mappings input and the 'value' is the value of the Value Mappings input | key: fieldValueMapping

InputNotesExample
Input
Key Mappings
data
Notes
 
Example
[{"key":"AccountName","value":"Deploy_Time_Specified_Account_Name__c"},{"key":"AccountValue","value":"Deploy_Time_Specified_Account_Value__c"}]
Input
Value Mappings
string
Key Value List
Notes
 
Example
[{"key":"AccountName","value":"bar"},{"key":"AccountValue","value":"baz"}]

Output Example Payload#

{  "data": [    {      "key": "Deploy_Time_Specified_Account_Name__c",      "value": "bar"    },    {      "key": "Deploy_Time_Specified_Account_Value__c",      "value": "baz"    }  ]}

Filter#

Filter elements of a list | key: filter

InputDefaultNotesExample
Input
Filter Function
code
Default
(item, index) => { return true; }
Notes
Filter out any elements that do not return true
Example
(item) => item.prop > 17
Input
List
code
/ Required
Default
Notes
Reference to a list of data to operate on
Example
[  {    "name": "Widget",    "prop": 25  },  {    "name": "Thingy",    "prop": 10  },  {    "name": "Whatsit",    "prop": 19  }]

This action applies a NodeJS filter function to an array of data. The filter function should use arrow notation, and include an input (an element) and a function that evaluates to true or false. Its return value is an array of elements that evaluated to true.

Simple Filter#

For example, if you have an array, ["exuberant", "spray", "limit", "elite", "destruction", "present"], and you would like words with a length greater than 6, your filter function could read:

(word) => word.length > 6;

The result would be ["exuberant", "destruction", "present"] - all words whose length is greater than 6.

Filters on Objects#

If your array is comprised of objects, you can apply a filter using each object's properties.

For example, suppose you have an array of objects like this:

[  {    "name": "Widget",    "cost": 80,    "available": true  },  {    "name": "Whatsits",    "cost": 90,    "available": false  },  {    "name": "Whoseits",    "cost": 120,    "available": true  },  {    "name": "Whysits",    "cost": 75,    "available": true  }]

If you want to find items whose cost is less than 100 that have true for availability, you can write a filter function like this:

(item) => item.cost < 100 && item.available;

The return value of this example would be an array of two objects that passed the filter:

[  {    name: "Widget",    cost: 80,    available: true,  },  {    name: "Whysits",    cost: 75,    available: true,  },];

First#

Get first element from a list | key: first

InputNotes
Input
List
code
/ Required
Notes
Reference to a list of data to operate on

Flatten#

Flatten an array of arrays into a single array | key: flatten

InputNotesExample
Input
List
code
/ Required
Notes
Reference to a list of data to operate on
Example
[  1,  [    2,    [      3,      [        4      ]    ],    5  ]]

Output Example Payload#

{  "data": [    1,    2,    [      3,      [        4      ]    ],    5  ]}

Key Value Pair List to Object#

Convert a Key Value list to an Object | key: toObject

InputNotesExample
Input
Key/Value List
code
/ Required
Notes
 
Example
[  {    "key": "FirstName",    "value": "Foo"  },  {    "key": "LastName",    "value": "Bar"  }]

Output Example Payload#

{  "data": {    "FirstName": "Foo",    "LastName": "Bar"  }}

Last#

Get last element from a list | key: last

InputNotes
Input
List
code
/ Required
Notes
Reference to a list of data to operate on

Length#

Count the number of elements in list | key: length

InputNotes
Input
List
code
/ Required
Notes
Reference to a list of data to operate on

Output Example Payload#

{  "data": 7}

Map#

Transform a list and its elements | key: map

InputDefaultNotesExample
Input
Context Data
data
Default
Notes
Additional contextual data to supply to the Transform Function
Example
 
Input
Filter Function
code
Default
(item, index) => { return true; }
Notes
Filter out any elements that do not return true
Example
(item) => item.prop > 17
Input
List
code
/ Required
Default
Notes
Reference to a list of data to operate on
Example
 
Input
Transform (map) Function
code
Default
(item, index) => { return item; }
Notes
Function to transform each element
Example
(item) => item.prop

This action applies a NodeJS map function to an array of data.

It also has an optional filter function - see the Filter action for examples. The action applies the filter function (if present), and then the map function to your array.

Simple Map Function#

If you have an array of integers that represent pennies, like [1234, 567, 890], and you would like to turn all values in dollar amounts, you could divide all values using a map function like this:

(value) => value / 100;

The result of the step would be an array that reads [12.34, 5.67, 8.9]

Mapping on Objects#

If you have an array of objects, you can reference the object's fields in your map function. Suppose you have an array of "people" objects, and you'd like to concatenate their names and order in the list.

Your data might look like this:

[  { "first": "Bob", "last": "Smith", "middle": "Billy" },  { "first": "John", "last": "Doe" },  { "first": "Lisa", "last": "Nguyen", "middle": "Sue" }]

Your map function could look like this:

(person, index) => {  if (person.middle) {    return `${index} - ${person.last}, ${person.first} ${person.middle[0]}.`;  } else {    return `${index} - ${person.last}, ${person.first}`;  }};

The result of the step would be an array of strings:

["0 - Smith, Bob B.", "1 - Doe, John", "2 - Nguyen, Lisa S."]

Object to Key Value Pair List#

Convert an Object to a Key Value List | key: fromObject

InputNotesExample
Input
Object
data
/ Required
Notes
 
Example
{ FirstName: "Foo", LastName: "Bar" }

Process In Order#

Ensures that payloads are processed in order across executions according to an ordering specified by a payload attribute. Returns the largest possible set of ordered payloads on the Process branch, and otherwise follows the Skip branch and returns the current item. | key: processInOrder

InputNotesExample
Input
Collection ID
string
/ Required
Notes
A value that uniquely identifies the collection that is being processed out of order.
Example
da41e39f-ea4d-435a-b922-c6aae3915ebe
Input
Collection Length
data
/ Required
Notes
The number of items in the collection. When processing is finished the interim data for the collection is removed.
Example
100
Input
Item
data
/ Required
Notes
The current item to consider for processing.
Example
{ Index: 0, Name: "Acme Contracting" }
Input
Item Index
data
/ Required
Notes
The integer value to consider as the index for the current item that specifies intended processing order. 0 is the first index value.
Example
10

The Process In Order action allows you to send several requests to an instance out of order, and helps to ensure that data runs through your integration in order. Requests with data payloads are collected, and when an ordered set of requests have been received this action processes the requests in the order you specify.

In order to use this action, you will need to know how many total items you are sending ahead of time.

Example: suppose we are updating an inventory system with three updates, and order is important. We want to process widgets first, then gadgets, and finally whatsits. We know that we have three items to import, and due to limitations of our third-party system we can't send them all at once. We're not confident that they'll arrive to Prismatic in any particular order, so we'll use this action to help.

Collection Tools - Process in Order flow in Prismatic integration designer

We'll come up with a unique "Collection ID", and begin sending our data in any order:

$ curl 'https://hooks.prismatic.io/trigger/EXAMPLE==' \    --header "Content-Type: application/json" \    --data '{"item": "whatsits", "index": 2}' \    --header 'collectionid: abc-123' \    --header 'collectionlength: 3'
$ curl 'https://hooks.prismatic.io/trigger/EXAMPLE==' \    --header "Content-Type: application/json" \    --data '{"item": "widgets", "index": 0}' \    --header 'collectionid: abc-123' \    --header 'collectionlength: 3'
$ curl 'https://hooks.prismatic.io/trigger/EXAMPLE==' \    --header "Content-Type: application/json" \    --data '{"item": "gadgets", "index": 1}' \    --header 'collectionid: abc-123' \    --header 'collectionlength: 3'

The first time that our integration is invoked, the {"item": "whatsits", "index": 2} payload will be stored for future processing, since items with index 0 and 1 have not yet been processed.

The second time that our integration is invoked, the {"item": "widgets", "index": 0} payload will be processed immediately, since it has index 0, but then our loop will stop since an item with index 1 has not yet been received.

The third time our integration is invoked, the {"item": "gadgets", "index": 1} will be processed right away, since it has index 1 and an item with index 0 has already been processed. The {"item": "whatsits", "index": 2} payload will also be pulled from storage and processed since it is next in line to be processed.

At this point all items will have been processed.

Note: Items must be zero-indexed.

Output Example Payload#

{  "data": [    {      "Index": 0,      "Name": "Acme Contracting"    },    {      "Index": 1,      "Name": "FooBar Consulting"    }  ],  "instanceState": {    "7d577253-3ef0-4a0a-bb7f-8335c2596e70": {      "da41e39f-ea4d-435a-b922-c6aae3915ebe": {        "lastIndex": 1,        "items": []      }    }  },  "branch": "Process"}

Remove#

Remove all occurrences of an element from a list | key: remove

InputNotes
Input
Element
data
/ Required
Notes
Reference to an element to look for
Input
List
code
/ Required
Notes
Reference to a list of data to operate on

Output Example Payload#

{  "data": [    1,    3  ]}

Sort#

Sort elements using a JavaScript comparison function | key: sort

InputDefaultNotes
Input
List
code
/ Required
Default
[  {    "name": "Widget",    "prop": 25  },  {    "name": "Thingy",    "prop": 10  },  {    "name": "Whatsit",    "prop": 19  }]
Notes
Reference to a list of data to operate on
Input
Sort Comparison Function
code
Default
(a, b) => { return a.prop < b.prop ? -1 : 1; }
Notes
Sort elements by the given comparison function. See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort for compare function documentation.

Take First#

Take first number of elements from a list | key: takeFirst

InputDefaultNotesExample
Input
List
code
/ Required
Default
Notes
Reference to a list of data to operate on
Example
 
Input
Number of Elements
string
/ Required
Default
1
Notes
Number of elements to take
Example
3

Output Example Payload#

{  "data": [    1,    2,    3  ]}

Take Last#

Take last number of elements from a list | key: takeLast

InputDefaultNotesExample
Input
List
code
/ Required
Default
Notes
Reference to a list of data to operate on
Example
 
Input
Number of Elements
string
/ Required
Default
1
Notes
Number of elements to take
Example
3

Output Example Payload#

{  "data": [    7,    8,    9  ]}