Latency Issues with App
Incident Report for Chargebee
Postmortem

On 8th October 2018 Between 13:05 and 13:20 UTC, Users accessing Chargebee app would have witnessed delayed responses and request timed out. The issue primarily occurred due to degraded performance in some of our APP servers. On further investigation, We identified that certain pattern of API requests resulted in a lot of objects being created causing performance impacts. We immediately provisioned hot standby servers to reduce the impact and to bring the service back to normalcy.

We have tuned some of the system parameters in app servers for increased stability against these kind of issues . We have also improved the alerting systems and setup additional metrics to detect issues early. For effective communication of such incidents in future, We have enabled subscribe option in our status page as well and would recommend you to subscribe at https://status.chargebee.com for quick notifications.

Posted Oct 26, 2018 - 10:50 UTC

Resolved
This incident has been resolved.
Posted Oct 08, 2018 - 17:02 UTC
Update
We are continuing to monitor for any further issues.
Posted Oct 08, 2018 - 13:47 UTC
Monitoring
Between 13:05 and 13:20 UTC, We experienced increased latencies and errors for the app. The issue has been resolved and the service is operating normally. All components are operational and we are continuously monitoring the system.
Posted Oct 08, 2018 - 13:47 UTC
This incident affected: Chargebee UI (Admin Console (US)), API Endpoints (API (US)), Checkout (Checkout (US)), and Self-Serve Portal (Self-Serve Portal (US)).