On-Prem Agent

The On-Prem Agent lets you connect your instances to resources that are not accessible from the public internet. This is helpful if you or your customers have databases, file storage systems, or other services that reside on a private network behind a firewall.

The on-prem agent is a feature that can be added on to an enterprise plan. Contact your account manager to discuss enabling the feature for your organization.

How the on-prem agent works

The on-prem agent is a lightweight Docker container that you or your customer can install on your own infrastructure. When the Docker container is started, it establishes a secure mutual TLS (mTLS) connection to an on-prem service running within the Prismatic platform and thereafter maintains a persistent connection with Prismatic. When an instance of your integration is deployed, your customer can select the OPA as the connection method.

When an on-prem connection is used in the instance, the instance communicates with the OPA on the private network using the established connection, which in turn communicates with your resource on the private network.

Data sent from the instance to the OPA through the on-prem service is encrypted using mTLS, and data is transmitted on OSI Layer 4 (transport layer). This allows you to send both HTTP and non-HTTP traffic through the OPA.

No inbound ports need to be opened

Note that the on-prem agent initiates the connection to the Prismatic platform, so you do not need to open any inbound ports on your firewall. The on-prem agent only needs to be able to make outbound connections to the Prismatic platform on ports 22 and 443.

Setting up the on-prem agent

To set up an on-prem agent, you will need a system on your private network that is capable of running a Docker container. This can be the same server that serves the database, filesystem or other resource you want to connect to, or a separate server on the same network that can access the resource. The on-prem container itself is very light-weight, generally consuming less than 100MB of memory and a small amount of CPU.

Docker support on Windows

While you can run Docker on Windows, and the on-prem agent works well on Windows, keep in mind that:

Docker Desktop for Windows requires that you enable Hyper-V or install WSL 2.
A user must log in to the Windows machine to start the Docker Desktop service. That means that if the machine reboots, the Docker container will not start until a user logs in.

Linux Docker hosts generally do not have these limitations and can be configured to start the Docker service automatically on boot.

Configuring the on-prem Docker container

An on-prem resource is configured for a specific customer. As an organization team member, you can view all on-prem resources by running prism on-prem-resources:list:

prism on-prem-resources:list

 Name            Status      Customer
 ─────────────── ─────────── ────────
 Acme PostgreSQL AVAILABLE   Acme Corp
 Hooli SFTP      UNAVAILABLE Hooli

To create a new on-prem resource, first look up the ID of customer whom the resource is for:

prism customers:list --columns Id,Name

 Id                                                           Name
 ──────────────────────────────────────────────────────────── ─────────────────
 Q3VzdG9tZXI6YjBmZDAyZTItYmE1OC00NzE0LWJhYzgtMDMwNWM5N2JiY2Vj Acme Corp
 Q3VzdG9tZXI6MTE0ODdlYmItNDdlMC00MGFjLWI1NjYtYzBiZWVjNjlkZTMz Initech
 Q3VzdG9tZXI6M2RkMjAwYjAtMjlmYy00MzZjLTk2OWYtMmNkMjUzYWNkYzY1 Stark Enterprises
 Q3VzdG9tZXI6NzFlY2NiYzQtYjc5OC00YzQzLWIzZDAtZjdmYzE5OTEyYzlj Hooli

Next, generate a registration JSON web token (JWT) for your customer:

prism on-prem-resources:registration-jwt \
  --customerId Q3VzdG9tZXI6YjBmZDAyZTItYmE1OC00NzE0LWJhYzgtMDMwNWM5N2JiY2Vj

eyJ0eXAiO....

create org-only resources for testing

To test the on-prem agent in the integration designer, you can create an on-prem resource that is only visible to your organization (and not attached to a particular customer). To do that, run prism on-prem-resources:registration-jwt --orgOnly

Now, with a registration JWT in hand, you can start the on-prem agent Docker. The container takes a set of environment variables to configure the connection to the Prismatic platform:

PRISMATIC_URL is the URL of the Prismatic platform. For the US commercial region, that's https://app.prismatic.io. For other regions, use the appropriate URL.
APP_HOST is the hostname of the service running on the private network. For example, if you're connecting to a database that runs on a host with IP address 10.1.2.3, enter that as the APP_HOST.
Connect to the docker host
If you run the on-prem agent on the same host as the service you're connecting to, you can use the special hostname host.docker.internal to connect to the host. host.docker.internal resolves to the internal IP address of the host running the Docker container.
Note that localhost or 127.0.0.1 does not work in this context, as it refers to the container itself.
APP_PORT is the port on which the service is running (5432 for PostgreSQL, 3306 for MySQL, 22 for SFTP, etc.).
NAME is the name of the on-prem resource that you will see when you run prism on-prem-resources:list.
REGISTRATION_JWT is the JWT you generated for the customer.

Start the on-prem agent Docker container
export REGISTRATION_JWT=$(prism on-prem-resources:registration-jwt --customerId Q3VzdG9tZXI6YjBmZDAyZTItYmE1OC00NzE0LWJhYzgtMDMwNWM5N2JiY2Vj)

docker run \
  --env PRISMATIC_URL=https://app.prismatic.io \
  --env APP_PORT=1433 \
  --env APP_HOST=host.docker.internal \
  --env "NAME=Acme MS SQL" \
  --env REGISTRATION_JWT \
  -t prismaticio/on-prem-agent:latest

Running the on-prem agent using Docker Compose

Docker Compose allows you to define and run multi-container Docker applications, and has some handy features like automatic restart of containers on system reboot. Here's an example docker-compose.yml file that starts the on-prem agent:

On-Prem docker-compose.yml
services:
  on-prem-agent:
    image: prismaticio/on-prem-agent:latest
    environment:
      PRISMATIC_URL: https://app.prismatic.io
      APP_PORT: 1433
      APP_HOST: host.docker.internal # Or specify the IP of the service
      NAME: Acme MS SQL
      REGISTRATION_JWT: ${REGISTRATION_JWT} # Source from host's environment variable
    restart: always # Use "always" to start this service when the Docker engine starts

After creating a docker-compose.yml file you can run docker-compose up from the command line to start the on-prem agent, or docker-compose up -d to start it in the background.

Configuring an instance to use the on-prem agent

Once an on-prem agent is running and has connected to the Prismatic platform, you can configure an instance to use the on-prem agent.

First, you need to update connections on your integration to support an on-prem connection. Open a connection in your config wizard designer, and select Allow On-Prem Connections.

When your customer configures an instance of your integration, they can select an existing on-prem agent to use for the connection by toggling Use On-Prem Connection and selecting a connection to use:

Note that when an on-prem connection is selected, the connection's "Host" and "Port" inputs disappear. That is because the on-prem service is responsible for connecting to the private network service, and the instance communicates with the on-prem service. The on-prem service will provide the instance with a local host and port to connect to when an execution is run.

Regenerating or revoking the registration JWT

If you lose the registration JWT for an on-prem resource, you can regenerate it using the prism on-prem-resources:registration-jwt command. You will need to provide the command with a --customerId and --resourceId of the on-prem resource you want to regenerate the JWT for. Those values can be found by running prism on-prem-resources:list --extended --output json.

If you need to revoke an on-prem resource registration JWT, you revoke all old JWTs and generate a new one by running prism on-prem-resources:registration-jwt --customerId {ID} --resourceId {ID} --rotate.

Components with on-prem support

The following built-in components support on-prem connections:

Supporting on-prem connections in a custom component

Prismatic provides several built-in components that connect to systems that are often on-prem (like PostgreSQL, MySQL, and MS SQL databases, SFTP file systems, SMTP and IMAP email servers, etc.). Those built-in components support on-prem connections out of the box.

If you've built a component that connects to a system that is often hosted on-prem, you can add support for on-prem connections, too. Within your custom component, change your connection() invocation to an onPremConnection invocation. The onPremConnection function takes the same arguments as connection but requires that your connection include inputs named host and port. Additionally, updte your host and port to have the property onPremControlled: true.

For example, here is a basic auth on-prem connection for a custom component:

import { onPremConnection } from "@prismatic-io/spectral";

export const basicAuth = onPremConnection({
  key: "basicAuth",
  label: "Username, password and endpoint",
  comments: "Basic auth username and password and endpoint",
  inputs: {
    username: {
      label: "Username",
      placeholder: "Username",
      type: "string",
      example: "john.doe",
      required: false,
      shown: true,
    },
    password: {
      label: "Password",
      placeholder: "Password",
      type: "password",
      example: "p@s$W0Rd",
      required: false,
      shown: true,
    },
    host: {
      label: "Host",
      placeholder: "Name of the host",
      type: "string",
      required: true,
      comments:
        "The address of the Acme server. This should be an IP address or hostname.",
      example: "server.example.io",
      onPremControlled: true,
    },
    port: {
      label: "Port",
      placeholder: "Port of the host",
      default: "1234",
      required: true,
      comments: "The port of the Acme server.",
      type: "string",
      onPremControlled: true,
    },
  },
});

For a full example of a component that supports on-prem connections, see our SFTP component source code in GitHub. src/connections.ts contains the connection definitions for the component.

SFTP component source code

When an execution is run, the instance will provide the connection with the host and port of the on-prem agent to connect to.

Support connections that don't have host and port inputs

What if your custom component doesn't have host and port inputs? You might have an input called endpoint for example that represents your customer's (generally publically available) app endpoint that is a URL like https://my-customer-id.example.com.

Add host and port inputs but set them to required: false, shown: false. Then, in your component's code, you can check if yourConnection.fields.host has a value. If it does, construct the endpoint from https://${yourConnection.fields.host}:${yourConnection.fields.port}.

Handling servers that use host-based routing with the on-prem agent

Some HTTP servers use host-based routing to determine which site to serve. For example, a server with IP 10.1.2.3 might serve both your app and a different app on port 80, and determine which app to serve based on the Host header in the HTTP request. When using the on-prem agent, the Host header in your HTTP request will default to the IP address of the on-prem service. You can override the Host header in your HTTP client to match the hostname of the service you want to connect to.

For example, you could specify the Host header as the endpoint input of your connection:

const response = client.get(
  `https://${yourConnection.fields.host}:${yourConnection.fields.port}`,
  { headers: { Host: yourConnection.fields.endpoint } }
);

A full example component that supports on-prem connections with host-based routing can be found in the GitHub examples repo.

Example on-prem-compatible component

Handling HTTPS-based connections with the on-prem agent

If the service that you are connecting to uses HTTPS on the private network, you will need to make sure that your HTTP client in your component is configured to trust (or ignore) the SSL certificate of the service. The HTTPS client that the custom component SDK provides is an instance of Axios, which uses the https module from Node.js. You can ignore SSL certificate errors by setting the rejectUnauthorized option to false in the https module's global agent:

const https = require("https");

const agent = new https.Agent({
  rejectUnauthorized: false,
});

const response = client.get(
  `https://${yourConnection.fields.host}:${yourConnection.fields.port}`,
  {
    headers: {
      Host: yourConnection.fields.endpoint,
    },
    httpsAgent: agent,
  }
);

White-labeling the on-prem agent

If you would like to white-label the on-prem agent, so your customers install and run a Docker container from your organization, follow these steps:

Create a Dockerfile that reads:

FROM prismaticio/on-prem-agent:latest
ENV PRISMATIC_URL=https://app.prismatic.io

Build and publish the image with a white-labeled name to Docker Hub:

docker build . -t acme-corp/on-prem-agent:latest
docker push acme-corp/taylor-on-prem-agent:latest

Your customers can then start a Docker container using your white-label name, and can omit the PRISMATIC_URL parameter, since that's hard-coded in your Dockerfile above:

docker run \
 --env APP_PORT=1433 \
 --env APP_HOST=host.docker.internal \
 --env "NAME=Acme MS SQL" \
 --env REGISTRATION_JWT \
 -t acme-corp/taylor-on-prem-agent:latest

How the on-prem agent works​

Setting up the on-prem agent​

Configuring the on-prem Docker container​

Running the on-prem agent using Docker Compose​

Configuring an instance to use the on-prem agent​

Regenerating or revoking the registration JWT​

Components with on-prem support​

Supporting on-prem connections in a custom component​

Handling servers that use host-based routing with the on-prem agent​

Handling HTTPS-based connections with the on-prem agent​

White-labeling the on-prem agent​