Advanced Integration

The guide for Your First Integration walked you through basic concepts in Prismatic, like fetching data from an API, passing data between steps of an integration, and using built-in connectors to send data to a third-party app. In this tutorial, we'll look at a sample integration that expands on those ideas and uses webhooks to sync data between two systems.

The GitHub - Zendesk integration overview

For this example, pretend that we are engineers at Zendesk. Our customers maintain public GitHub repositories, and have asked for an integration that syncs GitHub repository issues with Zendesk tickets.

So, our product team provided us with this spec for an integration:

• When a customer enables this integration, they should use OAuth 2.0 to connect their GitHub account.
• After authenticating, the customer should be able to select their GitHub repository from a dropdown menu.
• If someone creates an issue in their chosen GitHub repository, a corresponding Zendesk ticket should be created.
• If someone comments on the GitHub issue, that comment should be added to the same Zendesk ticket.
• If a support person comments on the Zendesk ticket, that comment should be added to the GitHub issue.
• If someone closes the GitHub issue, the Zendesk ticket be automatically closed.

The desired result - GitHub issues syncing with Zendesk tickets

Setting up

Behind the scenes, integrations are represented in YAML. If you would like to import, test and modify this example integration, download its YAML definition from our GitHub examples repo. Then,

• Create a new integration
• Import the integration's YAML definition

In order to run the integration, you will need to sign up and create OAuth 2.0 apps for both GitHub and Zendesk (developers can sign up for both for free).

Creating the configuration wizard experience

We want our users to have a good experience when they deploy our GitHub integration. Per our product spec:

1. When a customer enables this integration, they should use OAuth 2.0 to connect their GitHub account.
2. After authenticating, the customer should be able to select one of their GitHub repositories from a dropdown menu.

To address the first item, we can configure a GitHub OAuth 2.0 connection for our app using a client ID and client secret that we've generated.

When an end user configures our integration, they'll be presented a simple button that they can click to begin an OAuth 2.0 authentication flow.

For the sake of this example, we set up an OAuth 2.0 connection for Zendesk, as well (though if we were actually Zendesk engineers, we could dynamically set Zendesk credentials for our users so they didn't have to).

To address the second item (a repository dropdown), we can reach for the GitHub component's List Repos data source.

The data source will use the end user's OAuth 2.0 credentials to fetch a list of repositories that the user has access to, and it will present the repositories as a dropdown picklist menu.

In addition, we'll prompt the end user for two string config variables - one to represent a webhook secret (we'll address that later), and one to prompt the user for an email address to attach to GitHub-generated tickets.

Multiple flows

The integration we built in Your First Integration had a single flow. For this example, we'll split our logic into four purpose-built flows:

1. Our first flow will run when an instance is deployed, and will configure webhooks in Zendesk and GitHub.
2. Our second flow will run if the instance is deleted, and will remove the webhooks from Zendesk and GitHub.
3. Our third flow will accept webhook requests from GitHub when an issue is created or commented on, and will forward that issue/comment to Zendesk.
4. Our fourth flow will accept webhook requests from Zendesk when a ticket is commented on, and will forward that comment to GitHub.

Webhooks

We want our integration to be event-driven. That is, when something happens in one app (like a comment being made on a GitHub issue), we want our integration to be alerted to the change so it can make a corresponding change in Zendesk. We want our data sync to be as real-time as possible, so we should avoid checking for new data on a schedule (which could be slow).

Luckily, both Zendesk and GitHub support webhooks.

Creating webhooks on instance deployment

We'll create one flow that is dedicated to configuring webhooks in Zendesk and GitHub when an instance of our integration is deployed. In order to make this flow execute when the instance is deployed, we'll start the flow with an Instance Deploy trigger.

A GitHub repo config variable is presented as a string of {OWNER}/{REPO NAME}, so we'll split the string on the / character, and then feed in the owner and repository name into a GitHub Repos Create Webhook action.

Creating a Zendesk webhook is a two-step process. We need to:

1. Create the webhook with a Create Webhook action
2. Instruct Zendesk to send ticket events via the webhook using a Create Webhook Trigger action

Our completed deploy flow is a 5-step (a trigger and 4 actions) flow.

Removing webhooks on instance removal

We also want to remove webhooks that we've created if someone removes an instance of the integration. To do that, we'll begin another flow with an Instance Remove trigger, which executes when someone deletes an instance.

We can leverage GitHub and Zendesk's respective Delete Instance Webhooks actions. These actions are aware of the current instance's webhook URLs, and remove only webhooks in GitHub and Zendesk that target those URLs. The clean-up flow consists of a trigger and three steps:

Don't delete other apps' webhooks!

It's easy to make the mistake of listing webhooks and removing all of them. Use caution when listing and removing webhooks – some of your customer's webhooks may be configured for other apps and completely unrelated to your integration. Most built-in components that support webhooks have a Delete Instance Webhooks action that removes only webhooks for your instance.

Validating HMAC signatures

We want to ensure that messages received from GitHub and Zendesk originated from GitHub and Zendesk (rather than from some mischievous party). It's common practice with webhooks to sign a message using Hash-Based Message Authentication Codes (HMAC). We have an entire article on HMAC, but the quick summary is that you and the third party (GitHub or Zendesk) know some secret, and the third-party uses that secret to generate a unique hash of the message they sent. By verifying the HMAC signature of a webhook request, you can be sure that the message originated from the correct party, since no one else knows your HMAC signing secret.

Different apps implement HMAC in different ways:

• When Zendesk sends a webhook request, it includes an HMAC signature header and a webhook ID header. The Zendesk trigger, then, uses the Zendesk connection to fetch the webhook's signing secret and verifies the HMAC signature.
• GitHub lets you optionally set an HMAC signing secret when you create the webhook. We use a config variable that users set at deployment time as the HMAC signing secret. The GitHub trigger references that config variable to verify webhook HMAC signature headers.

For both apps, their respective triggers throw an error and immediately stop an execution if HMAC signatures are not correct.

Syncing data from Github to Zendesk

Now, let's process GitHub webhook requests and send data to Zendesk. We'll start with a few checks to see whether or not the data that came in is something we want to process. Then, we'll determine what kind of data came in and process it accordingly.

Avoiding infinite loops with branching

The first thing to notice about the GitHub-to-Zendesk flow is the branching logic at the beginning. It would be very easy to get into an infinite loop with this integration:

• A comment is added to a GitHub issue
  • This causes a GitHub webhook to fire
• A corresponding Zendesk comment is added
  • This causes a Zendesk webhook to fire
• A corresponding GitHub comment is added
• GOTO 10

We avoid this simply by prepending messages from the integration with [From GitHub] or [From Zendesk]. Then, when we get a webhook request from GitHub or Zendesk we check if the message contains those strings. If the message does, we stop and don't process the webhook request. There are likely more elegant ways of tracking messages that have been passed, but for illustration purposes this is sufficient.

Handling different webhook events

GitHub could send one of three events:

• A new issue is created
• A comment is added to an existing issue
• An issue is closed

We leverage a Branch on Expression action to handle these different events.

• If a new issue is created, we create a new Zendesk ticket.
• If a comment is added, we fetch the existing Zendesk ticket and append the comment.
• If an issue is closed, we fetch the existing Zendesk ticket and close the ticket.

Leveraging external IDs

In order to map data from one app to another, it's common to leverage external IDs. In our case, we use GitHub's issue number as a Zendesk ticket's external ID. The Zendesk ticket is assigned an external ID like github:1234.

External IDs allow you to easily look up matching records. When a comment is added to a GitHub issue, we can look up the corresponding Zendesk ticket by fetching github:1234 and append a comment to the ticket we looked up.

Syncing data from Zendesk to GitHub

The Zendesk to GitHub flow works in a very similar way. Webhooks from Zendesk are received, avoiding infinite loops as necessary. This flow is simpler because we only handle comment creation in Zendesk, though it could be extended to handle creating and closing tickets on the Zendesk side.

Once again, external IDs are leveraged, this time to map Zendesk tickets to GitHub issues:

Conclusion

This example integration modeled what a bi-directional, event-driven integration could look like. We used Zendesk and GitHub for illustration purposes, but you can build similar integrations between your app and a variety of other third-party apps.

In the next guide, Your First Custom Component, you'll walk through how to use NodeJS to build a custom component for your app.