10:35 - We have received reports that customer's agents were being logged of the system.
10:40 - The platform has returned to normal operation.
Reason for Outage – unable to Login or Disposition calls on Easycontactnow
Date of event: 2015-07-30
On Thursday 2015-07-30 at 10:31-10:36am:
Customers on Easycontactnow were unable to login or disposition calls for five minutes. This was associated with an alarm for the number of sessions in progress on a load balancer which was significantly higher than normal as shown in the 5 minute average sample graphic below.
A query being run manually by an engineer was terminated.
Root Cause and Preventive Measures:
Root cause is human error. The query being run by an engineer was not unusual, but the target machine had not been removed from the database cluster to which it belonged, and this caused writes into the cluster to be stalled.
The Infrastructure Manager has educated the team to ensure this does not happen again.