Users unable to log into Vyopta CPM

Incident Report for Vyopta

Postmortem

The failure was a result of a code change that caused a disruption in the authentication process. We apologize for any inconvenience caused by the outage.

Upon discovering the authentication system failure, our engineering team initiated an investigation to identify the cause and restore the system's functionality. To address the issue, we decided to back out the code change, effectively reverting the system to the previous stable version. This rollback allowed us to restore the authentication process and resolve the incident.

We apologize for the inconvenience caused by the authentication system failure and assure you that we are taking this incident seriously. We have taken immediate steps to address the issue and have implemented measures to prevent similar incidents from occurring in the future.

Posted Jul 06, 2023 - 11:59 CDT

Resolved

We have been monitoring the platform since the fix has been implemented and have not seen the issue arise. A RCA will be provided within a week.

Posted Jun 28, 2023 - 09:37 CDT

Monitoring

A fix has been implemented and we are currently monitoring the platform. Users are now able to access CPM Monitoring and Analytics. The alert generator was shutdown to minimize false alerts from being sent out, and will be turned back on in an hour.

Posted Jun 26, 2023 - 20:53 CDT

Identified

The issue has been identified and our engineers are working on a fix.

Posted Jun 26, 2023 - 20:37 CDT

Investigating

We are currently investigating reports of users unable to log in.
Users using my.vyopta.com are getting a "Failed to fetch" message while SSO users are getting a "502 Bad Gateway"
We will provide an update once we have identified the issue.

Posted Jun 26, 2023 - 20:20 CDT

This incident affected: Commercial - my.vyopta.com (CPM Monitoring, CPM Analytics, Vyopta Admin Portal).