Post 4 of the Integration Health series
- A high success rate can hide why executions fail. The remaining ones could be expired credentials, an API outage, rate limiting, or a schema mismatch, and each needs a different fix.
- Category error rate and category share answer different questions. Rate divides a category's executions by total executions and shows portfolio exposure. Share divides by failed executions and shows the failure mix.
- Each failed execution gets exactly one primary category. Retries don’t count as new executions, and a 429 that retries and ultimately fails stays a rate-limit failure. Use unknown for a category and track its rate so the data doesn't look more precise than it is.
- A small taxonomy maps categories to response paths. Start with big buckets (auth, 5xx, etc.) and add timeouts, configuration errors, or dependency failures if they need a distinct response.
- Segment by integration, connector, and customer over time. A portfolio average can hide a category concentrated in one integration or high-value customer.
A dashboard says that 95% of your integration executions succeeded.
That number tells you how much is failing, not why. The remaining 5% could be expired credentials, an API outage, rate limiting, a schema mismatch, or something else. And each of these problems takes a different fix.
Error composition by category groups failed executions by root cause so you can see portfolio exposure, failure mix, and where you need to focus on improvements next.
If your dashboards show high overall execution success rates, but your team still spends hours digging through log data every time an integration breaks, this is for you. Measuring error composition by category is essential as soon as you have multiple customer integrations in production and need a solid process for targeted, root-cause remediation.
Watch the video
Watch the overview, or keep reading for the details.
What is error composition by category?
Error composition by category is the rate and distribution of failed executions by root cause. It is a diagnostic breakdown. Assign each failed execution one category, then report:
Category error rate (%) = Executions assigned to category ÷ Total executions × 100
and
Category share of errors (%) = Executions assigned to category ÷ Total failed executions × 100
Category error rate shows portfolio exposure; category share shows failure mix.
For example, authentication failures could represent 0.8% of all executions but 40% of failed executions: low overall exposure, but the largest failure category.
Category error rate (authentications)
Category share of errors (authentication)
Keep categories mutually exclusive for the primary calculation, count each failed execution once, and preserve raw error details for diagnosis. Include an unknown or unclassified category so the data does not appear more precise than it is.
Why error categories matter
Aggregate error rate tells you how much is failing; categories help explain why. Credential, API, rate-limit, and data errors need different owners and fixes.
Segmenting categories connects operational evidence to the right response without hiding issues in one portfolio trend.
Start with a simple error taxonomy
A useful taxonomy is small enough to apply consistently and specific enough to map to a distinct response path. Start with:
- Authentication and credential failures
- 5xx API errors
- 4xx API errors (excluding 429s)
- Rate-limit responses (429)
- Data and transformation errors
- Unknown or unclassified errors
Add timeouts, configuration errors, or dependency failures when they support a distinct response path. Keep 429 responses separate from other 4xx errors when remediation differs, and avoid categories with overlapping resolutions.
How to measure error composition
1. Define the unit and denominator
Use executions as the unit. Do not mix executions with execution attempts, API calls, or business events without labeling the unit separately.
Use one denominator consistently:
- Total executions for category error rate.
- Total failed executions for category share of errors.
Treat one top-level business execution as the counted item. Do not count each retry as a new execution. If you analyze individual attempts separately, label that as a separate metric and document how it relates to the execution-level measure.
2. Assign one primary category per failed execution
Assign one root cause to each failed execution. A 429 that retries and ultimately fails remains a rate-limit failure – not separate 429, retry, and execution errors.
Preserve raw error context for diagnosis. Keep connection and authentication failures separate from failures that occur after execution begins.
3. Calculate both category measures
Calculate both measures together. A category can have a high share but low overall rate, or a lower share but high exposure in a frequently running integration.
Track the unknown-category rate because it can dilute named-category shares and make the taxonomy less useful.
4. Trend and segment the results
Report category rates and shares over time, segmented by:
- Integration
- Connector
- Customer
- Time period
A portfolio average can hide a category concentrated in one integration or high-value customer. Use the same taxonomy and inclusion rules across comparable periods, and record taxonomy changes so they are not mistaken for reliability changes.
5. Review the categories that need action
Use the results to prioritize investigation, not to prove causation. Review rising categories, high-share or high-impact categories, unknowns, and concentrations in specific integrations, connectors, or customers.
Then inspect the underlying logs and execution context.
What Prismatic provides
Prismatic can provide operational evidence for error categorization, including:
- Searchable execution logs filtered by severity or type
- Execution history, failure context, connection data, and instance-level context, along with the ability to retry or replay failed executions
- External log streaming, where enabled on the plan, for category dashboards and alert thresholds
Your team defines the taxonomy, unit, denominator, inclusion rules, and reporting segments. Execution data will not automatically classify every root cause, so preserve the unknown category until the evidence supports a more specific assignment.
Where to start
If you are beginning to measure error composition from scratch:
- Choose executions as the unit and document how retries are treated.
- Start with a small taxonomy that maps each category to a distinct response path.
- Assign one primary category to each.
Establish a trustworthy baseline, reduce preventable failure categories, and make the remaining failures easier to diagnose, route, and resolve.
An aggregate error rate tells you how much is failing but error composition tells you where you need to spend your time.
Series navigation
Post 3: Mean time to recovery
Post 4: This one.
Post 5: Coming soon
View all the posts in the Integration Health series.
