Defining Integrations as Code

Overview

There are many ways to create Prismatic integrations. You can assemble one through the web app, through the Prismatic CLI, through calls to the Prismatic API, or by defining your integration as code in a YAML file.

We recommend that you assemble your first few integrations through the web app so you can acquaint yourself with triggers, action steps, inputs and outputs, configuration variables and credentials, but once you feel comfortable with these concepts we encourage you to explore creating integrations programmatically.

Anything you can add to an integration in the web app, you can add through YAML as well. Using YAML syntax, you can define an entire integration, including its trigger and action steps, and how data flows between the steps.

Advantages of Writing Integrations as Code

If you are a developer, regardless of if you fall on the Vim, Emacs, or VSCode side of the editor war, you are likely more productive in a text editor than you are in a web browser. You can grep over a codebase and make changes to your integrations more quickly in an editor than you can in a browser. In addition to yielding greater productivity, writing integrations as code gives you:

  • A faster feedback loop. Make significant changes to multiple steps of an integration and test the integration from your terminal - no clicking required!
  • The ability to store integrations in your source repository. Your integration can go through whatever spec writing, development, code review, and deployment cycle you currently use. You can develop your integration in tandem with your core product and its APIs.
  • The option to update integrations as part of your CI/CD Pipeline. Is your integration dependent on updates to your APIs? You can plug your YAML-defined integrations into your CI/CD pipeline, so integrations are deployed alongside the rest of your code base.

Integration YAML File Anatomy

A YAML-defined integration contains six sections: name, description, isSynchronous, trigger, requiredConfigVars, and steps.

Integration YAML File Anatomy
---
name: String
description: (optional) String
isSynchronous: Boolean
trigger:
name: String
description: (optional) String
schedule: (optional)
type: [value, configVar]
value: Cron string or required config variable name
requiredConfigVars: (optional)
RequiredConfigVarName: RequiredConfigVarValue
steps:
- name: String
description: (optional) String
action:
key: String
componentKey: String
inputs: (optional)
InputKeyOne:
type: [value, reference, configVar, complex]
value: String
InputKeyTwo:
type: [value, reference, configVar, complex]
value: String
  • name is a brief unique identifier, and the name of the integration. For example, "Queue 3D Print Job".
  • description is an optional longer explanation of what the integration does. Your team members in the web app can search for integrations by name or description. For example, "This integration converts a completed .rocket file to AutoCAD format and queues a 3D print job."
  • isSynchronous is a boolean (true/false) that determines if the integration should be run synchronously or asynchronously
  • trigger defines how the integration should be invoked. It takes a required name, and optional description and schedule, each of which are described in more detail below.
  • requiredConfigVars describes the configuration variable keys and their default values that your integration uses.
  • steps is an ordered list of actions your integration will execute. Each step takes a required name, an action to execute, and optional description and step inputs, described in more detail below.

Integration Triggers in YAML

An integration trigger defines when an instance of an integration should run. The integration can be triggered by a webhook, or on a schedule similar to a cron job.

A trigger can take an optional description, which is a longer, more detailed description of how the trigger is invoked.

If a schedule is specified, the trigger will run on a schedule. If schedule is omitted, a webhook trigger will be created.

For More Information: Integration Triggers

Specifying a Scheduled Trigger in YAML

You can run integrations on a schedule using cron notation. For example, if you want instances of an integration to run at 03:00 AM every Sunday and Wednesday morning, you can configure a trigger to run at that time:

trigger:
name: My Trigger
description: Runs at 03:00 every Monday and Thursday
schedule:
type: value
value: "0 3 * * 1,4"

Like inputs, the schedule can be configured to reference a config var. This is helpful if different customers want to their instances run at different times or intervals:

trigger:
name: My Trigger
schedule:
type: configVar
value: cronSchedule
requiredConfigVars:
cronSchedule: "0 3 * * 1,4"

For More Information: Schedule Triggers

Accessing Webhook Trigger Outputs in YAML

If your trigger is invoked by a webhook, the body and header information of that trigger invocation is accessible using a reference input and referencing triggerName.results.{body,headers}.

Note that trigger names follow step naming conventions, and step names are converted to camelCase when referenced in YAML.

To see what a named trigger's referenceable name is in camel case, type it in here:

Suppose a webhook invocation contained a JSON payload body of {"customer":{"id":123}} and you have a custom component that you would like to pass the 123 value into as an input. You can access that portion of the output of the trigger as input like so:

trigger:
name: My Trigger
description: Kicks off the integration
steps:
- name: Do something with the customer ID
action:
key: my_custom_component
componentKey: myCustomAction
inputs:
myCustomerId:
type: reference
value: myTrigger.results.body.data.customer.id
note

If keys contain spaces or other special characters, you can reference those using ["bracket"] notation. For example,

value: myTrigger.results.body.data["My key!"]

For More Information: Webhook Triggers

Configuration Variables in YAML

Configuration variables are defined in the requiredConfigVars portion of integration YAML files as key-value pairs:

requiredConfigVars:
myEndpoint: https://my.company.com/api
customerId: A99E0DE1-6187-4BF7-A873-288F34E0BB03

Steps can reference these configuration variables as inputs by referencing their required variable names (like myEndpoint or customerId).

For More Information: Config Vars, Config Vars in Integrations

Integration Action Steps in YAML

Integrations are comprised of multiple steps. A step requires a name (referenceable name), and an action to execute. Optionally, a step can take a description, and can accept inputs that might be strings, config vars, or references to outputs from a previous step.

The step's action takes a key of an action, and the action's componentKey. The componentKey is required, since action keys are not necessarily unique (AWS S3 and Azure file might both have a gitObject action).

Action keys and component keys can be found in our component docs or by using the Prismatic CLI components:actions:list subcommand.

For example, if you want to add a step that lists all objects of an S3 bucket, you can run this to get the action's key and componentKey:

$ prism components:actions:list --columns key,componentKey,label,description --filter 'componentkey=aws-s3'
Key Componentkey Label Description
copyObject aws-s3 Copy an object Copy an object in S3 from one location to another
deleteObject aws-s3 Delete an Object in a Bucket Delete an Object within an S3 Bucket
getObject aws-s3 Get an object Get the contents of an object
listObjects aws-s3 List Objects in a Bucket List Objects in a Bucket
putObject aws-s3 Put an object Write an object to S3

You can then use those keys in a step:

steps:
- name: Get rocket schematic files
description: Get a list of all rocket schematics in our S3 bucket
action:
key: listObjects
componentKey: aws-s3
inputs:
awsRegion:
type: value
value: us-east-1
bucket:
type: configVar
value: myBucketName

Referencing Step Outputs as Inputs in YAML

A step can reference another step's output as its own input. For example, you may want to download a file from Dropbox and upload it to S3. Your steps, then, would look like this:

steps:
- name: Download from Dropbox - A PNG
description: Download a .png file from DropBox
action:
key: downloadFile
componentKey: dropbox
inputs:
path:
type: value
value: /path/to/my/file.png
- name: Upload the file to S3
description: Upload the downloaded PNG file to AWS S3
action:
key: putObject
componentKey: aws-s3
inputs:
awsRegion:
type: configVar
value: awsRegion
bucket:
type: configVar
value: bucketName
fileContents:
type: reference
value: downloadFromDropboxAPng.results
objectKey:
type: value
value: path/to/output.png

Note the fileContents reference above. The step "Download from Dropbox - A PNG" was referenced by the value downloadFromDropboxAPng. The step name was converted to camelCase so that sub properties of the output could be referenced if need be.

To convert a step name to a referenceable name for the code component or YAML, remove all special characters and camelCase the remaining string. You can test step name -> reference name conversions here:

Drilling Down Into Trigger and Step Outputs in YAML

It's useful to be able to pass in a portion of a step result rather than the whole thing into an input of a subsequent step.

For example, suppose you've pulled down a list of TODO tasks with an HTTP - GET action named Pull down some JSON, and for whatever reason you need the third item's title for the input of a step.

You can access that value for an input in one of two ways:

inputs:
someInput:
type: reference
value: pullDownSomeJson.results[2].title

or

inputs:
someInput:
type: reference
value: pullDownSomeJson["results"][2]["title"]

Note that the dot .key syntax works well when keys have no spaces or special characters. If your keys have spaces or special characters, you can use bracket ["key"] syntax to drill down into properties of previous steps' results.

Loops in YAML

As noted on the integrations article, you can loop over steps within an integration. This is useful if you want to perform a series of actions on a list of things (a list of files, a list of users, etc).

In YAML, a loop is a component like any other, and the loop component itself takes a series of sub-steps.

For example, this YAML definition loops over users "Bob", "Sally", and "Susan":

steps:
- name: List Users
action:
componentKey: code
key: runCode
inputs:
code:
type: value
value: |-
module.exports = async (context, params) => {
return { data: ["Bob","Sally","Susan"] };
};
- name: Loop Over Users
action:
componentKey: loop
key: loopOverItems
inputs:
items:
type: reference
value: listUsers.results
steps:
- name: Print a greeting
action:
componentKey: code
key: runCode
inputs:
code:
type: value
value: |-
module.exports = async (context, params) => {
return { data: `Hey ${params.loopOverUsers.currentItem}` };
};

Each iteration of the code component within the loop returns "Hey ${NAME}", and the result of the loop component will read:

["Hey, Bob", "Hey, Sally", "Hey, Susan"]

For More Information: Loops in Integrations, Looping Over Files Quickstart

Branching in YAML

An integration can contain logical branches, which are paths the integration can take based on some config variables or step results. More information about branching is on the integrations article.

The branch component contains two actions: branchOnExpression and branchOnValue.

Branching On Value in YAML

In this example, we use the branchOnValue action to follow one of three logical branches based on the value of a config variable:

- name: Branch on Value
action:
componentKey: branch
key: branchOnValue
inputs:
inputValue:
type: configVar
value: notificationMethod
branchValueMappings:
type: complex
value:
- name: Send SMS
type: value
value: sms
- name: Send Email
type: value
value: email
branches:
- name: Send SMS
steps: []
- name: Send Email
steps: []
- name: Else
steps: []

In this example the inputValue pulled in is a config variable named notificationMethod. That inputValue is then checked against two values, sms and email, and if inputValue is equal to sms, the integration follows the Send SMS branch (same for the Send Email branch).

If inputValue is not equal to sms or email, the integration follows the Else branch.

In this example the three branches do not contain any steps. You can add steps to each branch like you would add steps normally: create a list of steps under the steps: section in each branch.

Branching On Expression in YAML

In this example, assume that another step named Get User Data has pulled down data that looks like this:

{
"username": "bob",
"age": 40,
"address": {
"street": "123 Fake St",
"state": "SD",
"city": "Sioux Falls"
}
}

If we pull down data about a user from South Dakota who is over 20 years old, we want our integration to branch one way. Otherwise branch the other way.

We can set up our branch on expression action like this:

- name: Branch on User Data
action:
componentKey: branch
key: branchOnExpression
inputs:
conditions:
type: complex
value:
- name: Interesting User
type: complex
value:
- and
- - greaterThan
- type: reference
value: getUserInfo.results.age
- type: value
value: "20"
- - equal
- type: reference
value: getUserInfo.results.address.state
- type: value
value: SD
branches:
- name: Interesting User
steps:
- action:
componentKey: slack
key: postSlackMessage
inputs:
message:
type: reference
value: getUserInfo.results.username
webhookUrl:
type: value
value: https://hooks.slack.com/services/FOO/BAR/BAZ
name: Slack Message
- name: Else
steps:
- action:
componentKey: stop-execution
key: stopExecution
name: Stop Execution

Here, we used a complex type to do an expression comparison. We use the and operator to check that both the next expression and the expression after it are true. The next two expressions use greaterThan and equal operators to compare the values that they're presented.

The operators that are available are:

OperatorDescriptionArguments
andReturn true if all of the arguments return trueA series of conditionals, that may themselves be and, or, greaterThan, equal, etc.
orReturn true if any of the arguments return trueA series of conditionals, that may themselves be and, or, greaterThan, equal, etc.
equalReturn true if argument one is equal to argument twoTwo values which are static values, config variables, or references to step outputs
notEqualReturn true if argument one is not equal to argument twoTwo values which are static values, config variables, or references to step outputs
greaterThanReturn true if argument one is strictly greater than argument twoTwo values which are static values, config variables, or references to step outputs
greaterThanOrEqualReturn true if argument one is greater than or equal to argument twoTwo values which are static values, config variables, or references to step outputs
lessThanReturn true if argument one is strictly less than argument twoTwo values which are static values, config variables, or references to step outputs
lessThanOrEqualReturn true if argument one is less than or equal to argument twoTwo values which are static values, config variables, or references to step outputs
inReturn true if argument two contains argument oneTwo values which are static values, config variables, or references to step outputs. Value two must be an array.
notInReturn true if argument two does not contain argument oneTwo values which are static values, config variables, or references to step outputs. Value two must be an array.

Multi-line Values in YAML

YAML syntax supports multiline strings, so Prismatic does, too. Multiline strings are especially helpful if you use a code component. Your code component code can be written entirely in-line:

steps:
- name: Return Hello World
description: Return the string "Hello World"
action:
key: runCode
componentKey: code
inputs:
code:
type: value
value: |
module.exports = async (context, params) => {
return {
data: "Hello, World",
};
};

Importing a YAML-Defined Integration

To import a YAML-defined integration, use the Prismatic CLI integrations:import subcommand:

prism integrations:import --path path/to/your-integration.yaml

Exporting an Integration as YAML

An existing integration that was built via YAML or in the integration designer can be exported to a YAML file using the Prismatic CLI integrations:export subcommand:

prism integrations:export SW50ZWdyYXRpb246EXAMPLE > my-integration.yaml

Example Integration in YAML

The integration from the getting started tutorial pulled down "TODO" items from a list, and posted a message to Slack. That entire integration can be defined in a short YAML file:

---
name: My First Integration
description: My first foray into Prismatic
requiredConfigVars:
slackWebhookUrl: https://hooks.slack.com/services/REPLACE/THIS/URL
trigger:
name: Integration Trigger
steps:
- action:
componentKey: http
key: httpGet
name: Get TODO Items
description: Fetch TODO items from an API
inputs:
responseType:
type: value
value: json
url:
type: value
value: "https://jsonplaceholder.typicode.com/todos"
- action:
componentKey: jsonata
key: transform
name: Generate Slack message
description: Count the number of completed TODO tasks and generate a message to send
inputs:
data:
type: reference
value: getTodoItems.results
expression:
type: value
value: >-
$count($[completed]) & " of your " & $count($) & " tasks are marked
complete."
- action:
componentKey: slack
key: postSlackMessage
name: Send Slack message
description: Send the generated message to a Slack channel
inputs:
message:
type: reference
value: generateSlackMessage.results
webhookUrl:
type: configVar
value: slackWebhookUrl

Opening up the web app, we see that this YAML-defined integration created the appropriate trigger, steps, config vars and inputs.

Further Reading

After you feel comfortable defining basic integrations with YAML, we recommend you check out our Writing an Integration in YAML with Trigger Payloads, and Inputs quickstart, where we examine how to handle webhook trigger payloads, code components, and knitting inputs and outputs throughout a YAML-defined integration.