Intermittent Inbound & Outbound voice
Incident Report for TelcoSwitch Group
Postmortem

The message queuing server process on both servers processing outbound call traffic routing requests failed, which in turn resulted in a high level of errors with the dialplan endpoint.

This, in turn, caused the load balancer for this endpoint to mark both servers as out of service. Although this should not have affected any inbound calls, these servers also handle post-call CDR processing for both inbound and outbound calls, which would have caused an increase in load on inbound call SBCs and some delays or possible call failures.

Posted Sep 23, 2022 - 13:53 BST

Resolved
The message queuing server process on both servers processing outbound call traffic routing requests failed, which in turn resulted in a high level of errors with the dialplan endpoint.

This, in turn, caused the load balancer for this endpoint to mark both servers as out of service. Although this should not have affected any inbound calls, these servers also handle post-call CDR processing for both inbound and outbound calls, which would have caused an increase in load on inbound call SBCs and some delays or possible call failures.
Posted Sep 23, 2022 - 13:53 BST
Monitoring
A fix has been implemented and we are now monitoring the results, there should be no further impact to service.

Postmortem to follow
Posted Sep 23, 2022 - 13:39 BST
Investigating
We are currently investigating reports of intermittent outbound and inbound voice.

We believe the root cause has been identified and resolved but will confirm in a further update.
Posted Sep 23, 2022 - 13:22 BST
This incident affected: CallSwitch (Hosted Telephony Platform), SipSwitch (SIP Platform), and SimSwitch (Mobile Platform).