Pingcord - Hinweisverlauf

We have returned service to normal and are currently monitoring. Thank you for your patience.

Maintenance has now completed and service is restored. Thank you for your patience.

Further maintenance will be necessary over the next few days, but this will not have any impact on website availability or service uptime. Notices for those will be published soon.

Pingcord funktioniert wieder normal und Pings sollten wie gewohnt eingehen. Vielen Dank für Ihre Geduld!

Hello everyone,

The Dashboard is offline while we perform urgent maintenance upgrades to protect the security and stability of our platform. Further updates will be provided in the next hour or so. Please note Pingcord core services are still functional. Thank you for your patience.

Pingcord funktioniert wieder normal und Pings sollten wie gewohnt eingehen. Vielen Dank für Ihre Geduld!

This issue has been resolved.

Following a fix by Google Engineering last week, all YouTube trackers have been re-established over the last few days.

We will continue to run the backup system while monitoring traffic to ensure we are seeing a similar level of activity from before the issues started. Once we're happy, we'll switch off the backup system.

Update: We apologise that the backup system had malfunctioned since earlier today. The backup system's issue has now been resolved and pings are recovering.

We are *still* waiting on updates from Google Engineering regarding the main issue that has caused us to rely on the backup system.

Original issue:

We have identified an issue relating to trackers for YouTube channels. The issue is caused by an internal Google service. Google Engineering are aware of the issue and are actively looking into it.

Existing channels tracked by Pingcord have now been moved to a backup system until the issue is resolved. Newly-added channels are also now being tracked by the backup system. Pings may be delayed by up to 10 minutes as a result of this.

Update: We apologise that the backup system had malfunctioned since earlier today. The backup system's issue has now been resolved and pings are recovering.

We are *still* waiting on updates from Google Engineering regarding the main issue that has caused us to rely on the backup system.

Original issue:

We have identified an issue relating to trackers for YouTube channels. The issue is caused by an internal Google service. Google Engineering are aware of the issue and are actively looking into it.

Existing channels tracked by Pingcord have now been moved to a backup system until the issue is resolved. Newly-added channels are also now being tracked by the backup system. Pings may be delayed by up to 10 minutes as a result of this.

We have identified an issue relating to trackers for YouTube channels. The issue is caused by an internal Google service. Google Engineering are aware of the issue and are actively looking into it.

Existing channels tracked by Pingcord have now been moved to a backup system until the issue is resolved. Newly-added channels are also now being tracked by the backup system. Pings may be delayed by up to 10 minutes as a result of this.

We will provide further updates when we receive them from Google.

Current estimated impact: 5% of YouTube channels affected

While we await updates from Google, we are continuing to identify and setup mitigations to reduce any impact to notifications. Any affected existing YouTube channel trackers are being moved to a temporary system. Once this is complete, we will be able to support new YouTube channel registrations.

We are closing this incident but will continue to monitor ongoing. Our incident monitoring stack will also be reconfigured to generate an urgent alert to the team for the specific issue that occurred this evening.

A critical internal service handling communication to Discord restarted due to a fault. The restart was successful but it did not report to our other services that it was back online. An internal incident was raised automatically by our monitoring stack but was misclassified as a minor severity issue.

We have restored service currently. Pingcord is now recovering pings and data from prior to the incident.