Skip to main content

Process Data Component

Process data sets

Component key: process-data

Description#

The process data component allows your integrations to "remember" what data they have processed, so they don't process the same data over and over. The deduplicate action tracks the data that is processed each execution, and returns only items that it didn't previously process.

For an example of how to use the deduplicate action, see the building integrations article.

Actions#

Deduplicate#

Takes a sorted descending list of objects and a list of field names to use as identifiers and returns the list of objects that have not been previously processed | key: deDuplicate

InputNotesExample
Input
Data
code
/ Required
Notes
This is the dataset that will be operated on
Example
[  {    "id": "123",    "firstname": "John",    "lastname": "Smith"  },  {    "id": "122",    "firstname": "Jimmy",    "lastname": "Popadopoulos"  },  {    "id": "121",    "firstname": "Sally",    "lastname": "Smith"  }]
Input
Key Field Names
string
/ Required
Value List
Notes
The names of the fields in each item of the collection to use as a unique identifier
Example
lastname

Output Example Payload#

{  "data": [    {      "id": "123",      "firstname": "John",      "lastname": "Smith"    },    {      "id": "122",      "firstname": "Jimmy",      "lastname": "Popadopoulos"    }  ],  "instanceState": {}}