How to Gather Integration Requirements: Questions and Tips

Integrations are hard. By definition, an integration connects two or more systems to enable data exchange. These systems often use disparate databases, auth, and APIs. And it's also common for these systems to be under development, which means you might be trying to hit a moving target.

That's why gathering well-defined requirements to build integrations can speed time-to-market for your product's integrations, reduce development, deployment, and support costs, and ensure that end users are pleased with the results.

What are integration requirements?

Integration requirements describe what an integration does and how it is expected to function. A collection of them is called a specification.

We sometimes talk about high-level versus detailed requirements. Here's an example of a high-level integration requirement: "The integration syncs order data between the CRM and Accounting systems."

If we were to start developing an integration based on this requirement, we would end up in an unknown space very quickly. High-level requirements help us define the box, but we need detailed requirements to fill the box. For the balance of this post, we are talking about detailed requirements.

Why are requirements critical for integrations?

Requirements for integrations are often considered unimportant compared to requirements for other software. This thinking about integrations stems from a simplistic understanding: "It just copies data from one system to another. How hard can it be?"

Integrations are much like icebergs. Most of the integration is not visible. So, if we only create requirements for what we can easily see, we'll build something ill-suited to the underlying business need.

Integration requirements are essential to establish an agreed-upon integration scope for all stakeholders, from engineering to end-users. Yes, solid requirements are crucial even if your devs take an agile approach to integration development.

What happens when you build integrations without requirements?

Attempting to build integrations without well-defined requirements usually leads to various issues, including a poor UX. Without documentation detailing everything from APIs to auth to data transformation, the development process moves from something structured and repeatable to something more chaotic. Devs lack a single source of truth, so any existing requirements are misconstrued or misinterpreted, efforts are duplicated, and everyone hopes to happen upon the magic set of functionality that will make the customer happy.

This approach leads to project delays and a product that, while the customer may accept it, generally only meets that customer's needs and is not built for scalability and further enhancements. Testing is problematic because no one knows what success looks like exactly, and edge cases are hit or miss. The entire process drives up costs and increases frustration for teams and customers.

In short, building integrations is pointless unless you first define the requirements.

Common channels for integration requests

When customers need to move data between their tech stack and your product, they are not shy about letting you know. Here are some common channels for receiving integration requests:

• Direct (phone, chat, email)
• Support ticketing systems
• Customer feedback portals
• Sales and onboarding interactions
• Community forums
• Surveys and market research

Regardless of which path an integration request might take, you should evaluate and prioritize each request before you move on to assembling the requirements into a specification. Check out our post on prioritizing integrations for more details.

Key integration terminology

Language is essential, but when it comes to integrations, it can be challenging to get technical and non-technical users to speak and write using the same terms.

At best, this slows things down; at worst, it creates substantial rework when customer requirements don't match what's been coded for the integration.

Here are a few terms that you should make sure everyone understands as you start the requirements gathering process:

• API – Application programming interface. An API is the interface that enables an integration to connect with an application.
• Connection – A data link between systems or apps. A connection could be several things (including an integration).
• Connector – The piece of an integration that enables communication with an API or other system.
• Endpoint – The specific URL or address in an API to which a connector connects.
• Event – A change or activity in a system that can trigger an integration or workflow.
• Execution/Run – A single instance of a workflow or integration that runs or is executed.
• Flow – A single branch of an integration or workflow.
• Integration or Workflow – The software that connects two or more systems and enables intermittent data transfers with them.
• Transformation – The manipulation of data as it moves through an integration. Includes things like encoding changes and concatenations.
• Trigger – The beginning of a workflow or integration. These can be automated (such as event-driven via webhook), scheduled, or manual.

Common questions for integration requirements gathering

The nice thing about gathering requirements for integrations is that we know, at the highest level, what integrations do. This knowledge allows us to narrow the scope of requirements questions from the beginning.

Here are standard questions to solicit integration requirements:

• What will cause the integration to run, and how often will it run?
• Will this be a one-way or two-way integration (or something else)?
• What auth method will each system use?
• What API will each system use?
• What data is to be transferred?
• What transport languages will be used?
• What transfer protocols will be used?
• Will any data need to be modified within the integration?

Let's look at these questions in detail and see how they help us build a complete picture of integration requirements.

What will cause the integration to run, and how often will it run?

Integrations generally have a trigger that causes them to run. These triggers may be event-based (for example, a change in an order record in the CRM) or scheduled (for example, every Thursday at 7:15 AM local time). They may also have manual triggers (if something needs to be forced to run or if the need for data transfer can't be planned programmatically).

Will this be a one-way or two-way integration (or something else)?

A one-way integration sends data from one system to another. A two-way integration sends and receives data. In addition, a single integration could involve three or more systems, each of which could send and/or receive data.

Since one-way integrations tend to be simpler than N-way integrations, knowing the direction of the integration is a significant factor in determining the overall integration effort.

What auth method will each system use?

Auth (or authentication) is needed to ensure integrations run securely. Knowing the auth method for each system lets us determine a crucial part of what we'll need to configure for the integration. Systems generally use basic auth, API keys, or OAuth (though other auth types exist). Along with the type of auth, there is the question of whether auth is defined per user or system.

What API will each system use?

Not all integrations need APIs, but most of them will. Knowing the types of APIs being used gives developers a good idea of the rules and restrictions they'll be working with. REST APIs are used more often than any other type of API, but webhooks (sometimes called reverse APIs) are increasingly common. And some integrations may have an API on one end and a file import/export or something else on the other.

What data is to be transferred?

Data transfer is the core of an integration, so getting this right is critical. We often speak loosely about data, using terms like "order data" (going back to our high-level requirement example). Does it mean records from the order table in the sending or receiving system's database (or both)? Does the order data include data from any other tables? If there is conflicting data in the two systems, which overrides the other?

In many cases, end users look at data on a screen and don't realize that the screen is displaying data from half a dozen database tables. When you specify the data to be transferred, doing so field by field is essential. If “"order data" is comprised of 35 fields, explicitly list all those fields.'

What transport languages will be used?

People use human languages to communicate. Integrations use transport languages to encode the data that is sent between them. XML and JSON are the most common transport languages (also called data exchange or data interchange formats), but others exist.

Sending and receiving systems may use different transport languages. In that case, transport language decoding and encoding must be part of the integration.

What transfer protocols will be used?

In addition to being encoded in specific languages, integrations also use transfer protocols (a network protocol that supports data transfer between systems). While the APIs often determine these protocols, some systems may provide a choice of protocols.

HTTP (and HTTPS) are the most used transfer protocols for integrations running over the web. Integrations that transfer files might rely on FTP, FTPS, or SFTP.

Will any data need to be modified within the integration?

We touched on translating/mapping between transport languages above, but that's only the start. You may also need to manipulate text strings, perform math calculations, and map data elements from dissimilar schemas. While these are the most common ways to modify data within an integration, this is not an exhaustive list.

To determine the modifications needed for the integration data, you must compare input and output data requirements to see what actions the integration must take to transform the data.

Tips for integration requirements gathering

If you've received detailed answers to all the questions in the preceding section, you may not have a finished integration requirements specification, but you should have a good draft.

In addition to those specific questions, here are some tips to inform your overall process and help you turn that draft into a solid specification:

• Talk to end users.
• Talk to more users.
• Seek out corner cases.
• Engage with your tech partner.

Talk to end users

Every organization handles requirements gathering differently and expects different roles to contribute. That's fine if you can ensure that you are soliciting information on the requirements from everyone. And by everyone, we mean everyone who has something to contribute.

To ensure you capture a sufficiently detailed list of requirements, we recommend talking with end users who work with any part of the systems or processes with which the integration will interact. If there are users who copy data from System A and place it into System B (and the integration is supposed to replace that manual step), talk with them. Get them to show you precisely what they are doing today and share their expectations of how the integration will change their process.

Talk to more users

Remember to gather requirements from multiple customers, ensuring you capture the differences between those customers. 95% of the functionality may be identical between customers. However, the 5% that is different is critical. Some customers may need to include one or more JPG files in the data transfer, while others don't need JPG files at all. Some customers may only need 20 of the 25 available data elements, but others need all 25.

To productize an integration, you'll need to document the differences (the 5%), decide what is essential to all your customers, and then build a single integration. Inputs and other values that should differ per customer can be set as configuration options, allowing you to build once and deploy many times.

Seek out corner cases

As part of fleshing out the high-level requirements to get the level of detail you need for the specification, you'll need to identify the corner cases – scenarios that may not be immediately obvious but can have an oversized impact on the integration.

Examples of corner cases may include things like:

• The middle name is optional in our product, but required in the system to which we send data. What are we planning to do for records with no middle name value?
• The other system includes voided transactions. Can we process those into our product?
• Are there other record types that don't fit the standard pattern?

These questions boil down to not making assumptions but figuring out the possible scenarios and how you will ensure your integration can handle them. Sorting these out in the requirements phase will save your team time and rework during development.

Engage with your tech partner

To integrate your SaaS product with a tech partner's app, confirm that you have access to the corresponding API and app (and all available documentation). If you need an NDA with that partner before you and your team can dig into the details, get that done as quickly as possible.

When working with a tech partner, it is common to find that the integration is not as important to them as it is to you. As a result, they lack a sense of urgency and don't view answering your questions or meeting with your team as priorities. You must determine how to make gathering requirements a priority for them, or at least come to a shared understanding regarding how quickly and thoroughly they will answer questions, so that you can plan accordingly.

Requirements are foundational

There is more to building good integrations than gathering requirements. But you need solid requirements to build first-class integrations.

Asking questions to gather integration requirements can fill that box we mentioned earlier. But what do you do at that point? Do you hand the box over to your development team? Or do you implement an embedded iPaaS to supercharge your integration development, deployment, and support efforts?

An embedded iPaaS provides:

• A means to react quickly to new integration requests from prospects and customers
• A low-code designer with pre-built components
• The ability to write code when needed
• Collaboration with your current dev process and tools
• An embedded workflow builder for end users to build their own integrations with your product
• Infrastructure purpose-built to address security, reliability, and scaling

Check out our post for a full run-down of what an embedded iPaaS is and how it can help drive a scalable integration strategy for you