Webhook processing outage
Incident Report for Myndbend
Postmortem

Root Cause Analysis

On March 23rd at 9:09 am EDT, we were alerted of slow processing time on automated actions performed by the Myndbend Process Manager via the webhook. After further investigation, we identified and resolved an issue with sequenced child ticket processing that exponentially added to the volume of messages in our processing queue.

Resolution

Messages that were in the queue were held while the faulty messages were processed. Once the messages cleared, our service began operating normally at 10:16 EDT.

Remediation

No further actions are required at this time. All delayed messages were eventually processed, and tickets were updated properly.

We apologize for the processing delays and will ensure that future occurrences are prevented.

Posted Apr 04, 2022 - 13:11 EDT

Resolved
This incident has been resolved.
Posted Mar 23, 2022 - 10:16 EDT
Monitoring
We have identified the issue as slow webhook processing due to excessive messages in queue that were not handled normally. We are investigating the root cause.
Posted Mar 23, 2022 - 10:10 EDT
Investigating
We are currently investigating this issue.
Posted Mar 23, 2022 - 09:09 EDT
This incident affected: Myndbend Process Manager.