Anatomy of an API Connector

We recently covered API connectors from a high-level perspective, what they are and what they do. We noted that they are used with both enterprise iPaaS and embedded iPaaS to expedite the creation of integrations. I want to look at one of the API connectors we built for the Prismatic embedded integration platform – a peek behind the scenes to show you what we've done and give you some ideas for building your own custom connectors.

Our example connector (we also call them components) is one we created to connect with Asana, a project/task management system. I've made the code for this connector available via Prismatic's public GitHub repository.

I wrote the Asana API connector in TypeScript, which then compiles to JavaScript. The API connector talks to Asana through its REST API over HTTP with JSON as the transport language for both requests and responses. The REST API also supports form-data (file) requests when you need to deal with binary files (PDFs, images, etc). In addition to the REST endpoints, Asana provides webhook support for sending event-driven updates to an integration.

API connector code file structure

While you could conceivably place all the code for a very simple connector in a single .ts file, this isn't a simple connector. And best practices require that we organize the code somewhat logically. So, from the top level of the src/ directory, here's what we have:

• actions/ directory: all the actions/functions that the connector supports
• client.ts: an HTTP client to connect to Asana API. I like using the NPM package axios.
• connections.ts: we support both API key and OAuth 2.0
• index.ts: imports all other functions in the connector and the Prismatic component code and exports the Asana component
• inputs.ts: a complete list of the data (field) inputs used with the actions. Inputs could be specified in-line when you define an action, but keeping them all together makes them reusable between actions.
• triggers.ts: webhooks and HMAC validation
• util.ts: various options used throughout the connector

Further, within the actions directory, actions are grouped by resource type (attachment, customFields, items, portfolio, etc.) with each action's name reflecting its function.

What does the Asana API connector do?

In our overview post on API connectors, we noted that connectors include functionality to work with or define webhook behavior, auth, and actions for the corresponding integration. Let's look at each of these for the Asana API connector. In addition, we'll also examine the the built-in Axios HTTP client.

Perform actions

The actions are the bulk of the connector. Currently, the API connector includes 60+ actions, primarily performing CRUD (create, read, update, and delete) functions against Asana resources from projects and tasks to teams and users. Each action is named for the function it serves. So, if we want to retrieve a task from the API, we would call getTasks, but if we need to retrieve a list of the tasks, we would call listTasks instead. These actions wrap Asana's get a task (GET /tasks/{task_gid}) and get multiple tasks (GET /tasks) API endpoints respectively.

Handle webhooks

If we want Asana to push data to us (rather than fetching the data), Asana also supports webhooks. The actions in actions/webhooks.ts allow an integration to configure webhooks in Asana, and the trigger defined in triggers.ts implements HMAC verification so we can be sure that webhook requests we receive are genuine and come from Asana.

Handle auth

Auth is handled via connections.ts. As noted, the API supports auth via either API key or OAuth 2.0. You can use an API key to access the API for testing purposes, but if you want to give your customers the best deployment experience by giving them a single button to click to log in, I would strongly recommend OAuth 2.0.

Run HTTP client

Since we are accessing a REST API, we need some means of sending requests via HTTP. Axios is a common promise-based NodeJS HTTP client library that contains some handy response deserialization and error handling. We create a reusable Axios client in client.ts.

Action(s)!

As noted above, the Asana connector has over 60 actions. I'm not going to go through all of them here (you can do that via GitHub), but I would like to examine the code for a handful of them so you can get an idea of the scope of actions provided by the connector.

We'll look at the following actions in detail:

• getProject
• updatePortfolio
• addUserToTeam
• deleteTag
• attachFileToTask

getProject

This action allows you to retrieve detailed data for an Asana project record. It imports the HTTP client using client.ts and the list of possible inputs from inputs.ts.

The action then creates the HTTP client and uses the GET method to send projectId as a parameter to the API, along with expected field inputs.

The last half of the code for this action is a sample payload, making it very clear to devs who may be troubleshooting an issue what to expect when we run this code.

import { action } from "@prismatic-io/spectral";
import { createAsanaClient } from "../../client";
import { connectionInput, projectId } from "../../inputs";

export const getProject = action({
  display: {
    label: "Get Project",
    description: "Get the information and metadata of a project by Id",
  },
  perform: async (context, params) => {
    const client = await createAsanaClient(params.asanaConnection);
    const { data } = await client.get(`/projects/${params.projectId}`, {
      params: {
        opt_fields:
          "layout,team,workspace,html_notes,notes,color,custom_field_settings,custom_fields,followers,members,public,archived,modified_at,created_at,start_on,due_on,current_status,owner,name,resource_type,gid",
      },
    });
    return { data };
  },
  inputs: { projectId, asanaConnection: connectionInput },
  examplePayload: {
    data: {
      data: {
        gid: "1202461773653662",
        archived: false,
        color: "light-green",
        created_at: "2022-06-16T22:55:11.208Z",
        current_status: null,
        custom_fields: [],
        due_on: null,
        followers: [{ gid: "1202178852626547", resource_type: "user" }],
        html_notes: "<body>My new project notes</body>",
        members: [{ gid: "1202178852626547", resource_type: "user" }],
        modified_at: "2022-06-16T22:55:13.275Z",
        name: "My new project name",
        notes: "My new project notes",
        owner: { gid: "1202178852626547", resource_type: "user" },
        public: true,
        resource_type: "project",
        start_on: null,
        team: { gid: "1202178854270529", resource_type: "team" },
        workspace: {
          gid: "1126509132283071",
          resource_type: "workspace",
        },
      },
    },
  },
});

updatePortfolio

This action allows for a specific Asana portfolio record to be updated (partially changed). It imports the HTTP client using client.ts and the list of possible inputs from inputs.ts.

updatePortfolio action then creates the HTTP client and uses the PUT method to send portfolioId as a parameter to the API, along with expected field inputs.

This code does not define an example payload inline, but does import it from portfolioPayload since the payload data is extensive. It would be easy to lose focus with a wall of example payload text.

import { action } from "@prismatic-io/spectral";
import { createAsanaClient } from "../../client";
import {
  connectionInput,
  portfolioId,
  color,
  portfolioName,
  workspaceId,
  isPublic,
} from "../../inputs";
import { portfolioPayload } from "./portfolioPayload";

export const updatePortfolio = action({
  display: {
    label: "Update Portfolio",
    description: "Update the information and metadata of the given portfolio",
  },
  perform: async (context, params) => {
    const client = await createAsanaClient(params.asanaConnection);
    const { data } = await client.put(`/portfolios/${params.portfolioId}`, {
      data: {
        color: params.color || undefined,
        name: params.portfolioName || undefined,
        public: params.isPublic,
        workspace: params.workspaceId || undefined,
      },
    });
    return { data };
  },
  inputs: {
    asanaConnection: connectionInput,
    portfolioId,
    color: { ...color, required: false },
    portfolioName: { ...portfolioName, required: false },
    workspaceId: { ...workspaceId, required: false },
    isPublic,
  },
  examplePayload: portfolioPayload,
});

addUserToTeam

This action adds a user to a specific Asana team. It imports the HTTP client using client.ts and the list of possible inputs from inputs.ts. It then uses the POST method to send TeamId as a parameter to the API, along with expected field inputs.

This action includes an example payload showing the team and user data.

import { action } from "@prismatic-io/spectral";
import { createAsanaClient } from "../../client";
import { teamId, userId, connectionInput } from "../../inputs";

export const addUserToTeam = action({
  display: {
    label: "Add User To Team",
    description: "Add an existing user to the given team",
  },
  perform: async (context, params) => {
    const client = await createAsanaClient(params.asanaConnection);
    const { data } = await client.post(`/teams/${params.teamId}/addUser`, {
      data: { user: params.userId },
    });
    return { data };
  },
  inputs: { teamId, userId, asanaConnection: connectionInput },
  examplePayload: {
    data: {
      data: {
        gid: "1202178854270530",
        resource_type: "team_membership",
        team: {
          gid: "1202178854270529",
          resource_type: "team",
          name: "Engineering",
        },
        user: {
          gid: "1202178852626547",
          resource_type: "user",
          name: "Example User",
        },
        is_guest: false,
      },
    },
  },
});

deleteTag

This action removes an Asana tag. It imports the HTTP client using client.ts and the list of possible inputs from inputs.ts.

deleteTag then creates the HTTP client and uses the DELETE method to send the tagId as a parameter to the API, along with expected field inputs.

The example payload in this case is empty.

import { action } from "@prismatic-io/spectral";
import { createAsanaClient } from "../../client";
import { limit, offset, connectionInput, tagId } from "../../inputs";

export const deleteTag = action({
  display: {
    label: "Delete Tag",
    description: "Delete the information and metadata of the given tag",
  },
  perform: async (context, params) => {
    const client = await createAsanaClient(params.asanaConnection);
    const { data } = await client.delete(`/tags/${params.tagId}`);
    return { data };
  },
  inputs: { asanaConnection: connectionInput, limit, offset, tagId },
  examplePayload: { data: { data: {} } },
});

attachFileToTask

The actions we've looked at before this one were all focused on creating, reading, updating and deleting records. This one introduced file handling into the mix. This action attaches a file to an Asana task. To do so, it imports form-data (for handling the file), imports the HTTP client using client.ts, and pulls in the list of possible inputs from inputs.ts.

attachFileToTask then creates the HTTP client and the formData object and uses the POST method to send fileName and taskId as parameters to the API, along with expected field inputs. form-data handles the creation of the multi-part upload, and necessary related headers.

The example payload for this is perhaps the simplest of the actions we've looked at, as it includes just a handful of fields for describing the file attachment.

import { action, input, util } from "@prismatic-io/spectral";
import FormData from "form-data";
import { createAsanaClient } from "../../client";
import { connectionInput, taskId } from "../../inputs";

export const attachFileToTask = action({
  display: {
    label: "Attach File to Task",
    description: "Attach a file to a task",
  },
  perform: async (context, params) => {
    const client = await createAsanaClient(params.asanaConnection);
    const formData = new FormData();
    formData.append("file", params.file.data, {
      filename: params.fileName,
    });
    const { data } = await client.post(
      `/tasks/${params.taskId}/attachments/`,
      formData,
      { headers: formData.getHeaders() }
    );
    return { data };
  },
  inputs: {
    asanaConnection: connectionInput,
    file: input({
      label: "File",
      comments: "File to attach. This should be a reference to a previous step",
      type: "data",
      required: true,
      clean: util.types.toBufferDataPayload,
    }),
    fileName: input({
      label: "File Name",
      comments: "Name of the file to attach",
      type: "string",
      required: true,
      example: "my-image.png",
      clean: util.types.toString,
    }),
    taskId,
  },
  examplePayload: {
    data: {
      data: {
        gid: "12345",
        resource_type: "attachment",
        name: "Screenshot.png",
        resource_subtype: "asana",
      },
    },
  },
});

Best practices for API Connectors

Most of the Prismatic iPaaS users I work with use a solid mix of built-in connectors and custom API connectors that allow them to connect with the verticals and niche applications their customers need. I trust that this walkthrough for one of our API connectors has provided some patterns that will be useful as you build your own custom connectors.

That said, here are a few things to keep in mind:

• Wrap everything related to connecting to the API (auth URL, API version, authentication fields, and more) in a connection function.
• Build actions to be reusable across integrations.
• Remember that you don't need a one-to-one correlation between actions and endpoints.
• Use clear examples and comments for input fields and sample payloads.

If you'd like to see more content on API connectors and other integration topics, follow us on LinkedIn or Twitter.