Previous incidents

The API issues have been resolved. An update to our ingress had caused incorrect SSL certificates to be served, but the correct certificates have now been applied.

The service was temporarily down due to an upstream database provider outage. Everything is back online and operating normally.

The elevated API timeouts have been fully resolved. All services are operating normally. We apologize for the disruption and appreciate your patience.

The elevated API timeouts have been fully resolved. All services are operating normally. We apologize for the disruption and appreciate your patience.

We are seeing complete system recovery. We will continue to monitor the system.

The issue has been resolved.

The issue is now resolved. We are putting in further fixes to prevent the issue from reoccurring.

Customers who were exceeding their concurrency limit during this outage may be encountering strange errors in their crawls.

This is due to the nature of the outage affecting our concurrency limiting system, causing unique jobs to be inserted into a separate per-team concurrency queue multiple times. When these jobs are promoted from the concurrency queue into the main queue, they retain their IDs, which causes a violation of the unique ID constraint.

A fix for this issue is now deployed.

The issue is now resolved. As part of the hotfix, all ongoing concurrency queued jobs had to be discarded, causing ongoing crawls to freeze -- please reach out to support if you are affected. We apologize for the disturbance, and thank you for your patience.

The issue has been resolved. Thank you for your patience.

We've shipped a fix and the incident is now resolved. Thank you for your patience.

The elevated API timeouts have been fully resolved. All services are operating normally. We apologize for the disruption and appreciate your patience.

We have identified and pushed a hotfix for the root cause. We are working on a permanent fix.

The issue is now resolved. We apologize for the inconvenience. The maintenance we had previously scheduled for Sunday will address the root cause of this downtime.