We've been continuing to monitor error rates and can confirm that the issue has been fully resolved. The timeline for the final outage was from 3:50pm PT to 4:16pm PT.
We're looking into remediation items that will allow us to switch providers more quickly for similar future outages.
Posted Dec 12, 2024 - 01:40 UTC
Monitoring
We continue to see error rates decline. We're continuing to monitor error rates and request latency.
Posted Dec 12, 2024 - 00:16 UTC
Identified
We've switched over to different service providers and are starting to see error rates decline. We're continuing to monitor error rates and request latency.
Posted Dec 12, 2024 - 00:02 UTC
Investigating
We are currently experiencing intermittent delays and increased error rates in our response generation. Our team is actively monitoring the situation and working to minimize disruptions.
This issue is related to a latency problem with one of our core service providers, which is temporarily impacting our system’s performance. We're actively switching over to another core service provider to reduce error rates.