Increased Error Rates

Incident Report for Chargebee

Postmortem

On 9th May, Chargebee had an intermittent db connection issue with one of our databases, which affected some of our users between 03:42 UTC and 04:01 UTC.

While performing a database migration we ran into an issue caused by a bug in the db version running on the particular database instance. We immediately diagnosed the issue and resolved it.

We have included additional pre-checks to make sure such incidents are avoided in the future. We’re also analyzing internally about a potential database version upgrade.

Our apologies for the disruption caused.

Posted May 10, 2019 - 16:32 UTC

Resolved

This incident has been resolved.

Posted May 09, 2019 - 05:09 UTC

Update

We are continuing to monitor for any further issues.

Posted May 09, 2019 - 04:58 UTC

Monitoring

A fix has been implemented and we are monitoring the results.

Posted May 09, 2019 - 04:58 UTC

Investigating

We had experienced increased number of error rates between 03:35 UTC to 04:00 UTC for a set of users, we are investigating the issue.

Posted May 09, 2019 - 04:40 UTC

This incident affected: API Endpoints (API (US)) and Checkout (Checkout (US)).