Deep Dive: Building an Integration Using Prismatic

In our previous blog post we examined Progix, a company that develops project management software for rocket manufacturers, and how Progix used Prismatic to integrate their software with their customers' ERP systems. Progix is not a real company, but represents many companies we've worked with that have run into similar integration challenges.

In this post we'll walk through Progix's integration to AcmeERP together, with focus on:

1. Triggering a Prismatic integration via a webhook, and how to get data from your software into a Prismatic integration using a webhook's data payload.
2. Writing custom components that can be reused in multiple integrations.
3. Creating short, integration-specific JavaScript functions using our code component.
4. Letting Prismatic handle OAuth complexities, so you can concentrate on more interesting problems!

If you have not yet done so, I recommend reading through the Building the AcmeERP Integration portion of our previous blog post to get a sense of the steps that made up Progix's AcmeERP integration. TL;DR: Progix wanted to send rocket fuel data from their application to customers' ERP systems. To do that they needed to validate webhook requests, convert fuel from pounds to gallons, transform the fuel data from JSON to XML, and then ship the data to customers' ERPs. Prismatic integrations are self-documenting, so you can see the trigger and four steps that make up this integration here:

Let's look at these steps in more detail...

Triggering an ERP integration from Progix's software with a data payload

After a rocket is launched, we want Progix software to update customers' ERP systems to account for fuel expenditures. Instances of Prismatic integrations can be triggered via a unique webhook, and that webhook can contain a payload body, so it makes sense to transport data into Prismatic through that payload.

Additionally, we want to verify that webhook invocations come from real Progix systems. We'll do that by hashing the webhook body payload with a secret key and including a custom X-Progix-Signature HTTP header with our webhook request that contains that hash.

The resulting webhook trigger invocation, including data payload and signature header, ends up looking something like this:

curl 'https://hooks.prismatic.io/trigger/EXAMPLE1FjMmUtOWNiZS00MmI3LTllYzEtMDQzYjY5ZDkxNThj' \
  --request POST \
  --data '{"fuelUsed":[{"type":"Kerosene","pounds":"1357964"},{"type":"O2","pounds":"3028029"}]}' \
  --header "Content-Type: application/json" \
  --header "X-Progix-Signature: 272779AA79944DB52C65DF13401F0DFF12E770B9"

(curl is used for illustration here; Progix would probably use a language-specific HTTP library.)

While assembling this integration, we can run our test curl invocation and immediately see the results of our test within the integration designer. Here, we can see that the payload body and header arrived as expected:

Examining the trigger output within the integration designer is helpful, because we can see that integrationTrigger.body.data and integrationTrigger.headers.x-progix-signature have the correct values, and can therefore be referenced as inputs in subsequent steps. We'll look at how those are used in a moment.

We now have data streaming from Progix systems into a Prismatic integration. Our next step is to verify that webhook requests are signed correctly.

Verifying payload signatures with a custom component

Once a webhook is triggered, we want to verify that the trigger invocation came from real Progix systems by checking a signature that was passed in through an HTTP header. This "signature verification checker" seems general enough to be useful in multiple integrations, not just our rocket fuel / ERP one, so it makes sense to create a reusable custom component using the custom component library.

Source code for this custom component can be found on GitHub. Let's dive into a couple of interesting pieces of the component:

The verifySignature component action takes three inputs: the payload body, the secret that was used to sign the payload body, and the signature itself. Using the Prismatic custom component library, we can easily manipulate how the action presents itself in the web app UI. We can change the placeholder text, input label, requiredness of an input, etc., so that a section of code that looked like this:

export const signatureInputField = input({
    key: "signature",
    label: "Signature",
    placeholder: "Signature contained in X-Progix-Signature Header",
    type: "string",
    required: true,
    comments:
        "HMAC SHA1 hash constructed from the request body and known secret with hyphens removed",
});

will yield a UI that looks like this:

Using the input fields that we created, we can pass in the signature header, payload body, and secret information into the custom component. The verifySignature action's perform function references the signature, body, and secret as parameters, and throws an error (which stops the integration run) if the request isn't signed correctly:

perform: async (
  { logger },
  { signature: expectedSignature, body, secret }
) => {
  const calculatedSignature = generateSignature(secret, body);

  if (calculatedSignature !== expectedSignature) {
    throw Error(
      "Unexpected signature, request did not originate from a known Progix application."
    );
  }

  logger.info("Verified valid request signature.");
},

We could have used a code component to handle signature verification since the verification code is pretty short and concise, but we want to reuse this code in several other integrations that Progix plans to build in the future, so writing a reusable custom component is the right thing to do in this case.

Integration-specific code components

At this point we have data from Progix flowing into a Prismatic integration via a webhook trigger, and we've verified the payload for authenticity. The next thing we need our integration to do is convert the fuel data from pounds to gallons (Progix measures fuel in pounds, but AcmeERP measures fuel in gallons). None of the built-in Prismatic components convert rocket fuel from pounds to gallons (since that's a very industry-specific function), so we need to write our own code to accomplish the conversion. AcmeERP is the only ERP vendor Progix needs to integrate with that measures fuel in gallons, so the code that we write isn't going to be reusable. We could write a full custom component, but for simplicity's sake it's easier to write a short JavaScript function within a code component:

module.exports = async (context, params) => {
    const gallonsToPoundsConversion = {
        Hydrazine: 8.38,
        Kerosene: 6.68,
        Nitromethane: 9.49,
        O2: 9.52,
    };
    const fuelUsed = params.integrationTrigger.body.data.fuelUsed;
    return {
        data: {
            fuelGallonsUsed: fuelUsed.reduce((obj, item) => {
                return {
                    ...obj,
                    [item.type]:
                        item.pounds / gallonsToPoundsConversion[item.type],
                };
            }, {}),
        },
    };
};

As I noted above, the webhook payload body is accessible as an output of the trigger step. The code component is able to reference the trigger's output through the params.integrationTrigger.body.data parameter. Then, we can convert the Progix-formatted payload of the form {"fuelUsed":[{"type":"Kerosene","pounds":"1357964"},{"type":"O2","pounds":"3028029"}]} with a few lines of JavaScript code and a reduce function to the desired key-value form {"Kerosene": 203288.02395209583, "O2": 318070.2731092437}, where values are in gallons of fuel instead of pounds.

We're able to test our changes as we write them by clicking through the return value of the code component to the left of our code editor. We don't have to wait minutes or hours while code compiles and redeploys with each change we make (who gets reduce syntax correct on the first try, anyways?). Rather, our feedback loop can be measured in seconds, and we're was able to see the results of our changes almost immediately.

After mutating fuel data into a JSON object that's formatted the way we want it, we can throw the code component's output through Prismatic's built-in Change Data Format component, which takes care of converting JSON into XML (the format AcmeERP accepts) for us.

Leaning on built-in components and authorization functionality

We now have properly-formatted inventory data that we're ready to ship to AcmeERP. AcmeERP provides a modern HTTP-based API that uses OAuth 2.0 for authentication.

Anyone who has tried implementing an OAuth client knows how much of a pain authentication to third-party services can be. Even after exchanging credentials for an authorization code and turning that code into an access token, you need to perpetually juggle refresh tokens on a schedule (lest you want to re-enter credentials!). Plus, you need the credentials in the first place. In my experience, that usually translates to customers emailing credentials to integrators in plain text, which defeats the purpose of OAuth entirely!

Luckily, we don't need to worry about any of that to make our REST call to AcmeERP. We can rely on Prismatic's built-in OAuth 2.0 functionality, and don't need to ask customers to email us their credentials if we follow Prismatic's OAuth credential creation flow:

1. Create a new credential of type "OAuth 2.0" within the Prismatic web app for each customer who needs the AcmeERP integration, entering the OAuth client ID, client secret, permission scopes, authorization URL, and access token URL. This information can be found within AcmeERP's docs.
2. The customer is then emailed a link to their AcmeERP instance where they are prompted to enter their AcmeERP credentials. AcmeERP sends an authorization code to Prismatic's OAuth callback URL.
3. Prismatic exchanges the authorization code for an access token that has an expiration date. The customer's Prismatic instance uses this access token to interface with AcmeERP's API.
4. Prismatic refreshes the authorization tokens periodically before expiration, so you and your customers don't need to worry about renewing tokens or re-authenticating.

By doing this, customers are able to securely enter their own credentials without having to resort to risky practices like emailing sensitive information, and the customers' instances are granted access to perform REST calls against AcmeERP's API. With authentication solved, we can now add an HTTP - POST action to our integration that uses an OAuth 2.0 token, and post fuel data to AcmeERP. Integration complete!

Learn More

In this post we covered a variety of topics, from integration triggers to writing custom components and our own code within code components to leaning on our ever-growing built-in component catalog and OAuth 2.0 authorization functionality. We showed how you can incorporate custom code alongside built-in components within Prismatic to quickly create specialized integrations.

For more information on any of these topics and more, check out our docs, or reach out - we'd love to chat about your experience building integrations, and about how Prismatic can help!