WinterNode - Majority of CHI Nodes Unavailable – Incident details

All systems operational

Majority of CHI Nodes Unavailable

Resolved
Major outage
Started 11 months agoLasted about 5 hours

Affected

Chicago - Game Servers

Major outage from 9:47 PM to 2:58 AM

gs.chi2.winternode.com

Major outage from 9:47 PM to 2:58 AM

Chicago - Minecraft

Major outage from 9:47 PM to 2:58 AM

mc.chi2.winternode.com

Major outage from 9:47 PM to 2:58 AM

mc.chi5.winternode.com

Major outage from 9:47 PM to 2:58 AM

mc.chi7.winternode.com

Major outage from 9:47 PM to 2:58 AM

Updates
  • Resolved
    Resolved

    No further issues have been reported from any of the affected nodes.

    This incident was caused by a PDU failure which resulted in the loss of power for the 14 affected nodes.

    We were unable to bring MC.CHI8 up with the rest of the nodes due to a drive failure. We immediately began our restoration process, rolling back game server files to our 2pm Central Time backup. We are happy to report MC.CHI8 is back in service. Please note, at this time MySQL Databases are still not backed up, therefore, data stored in the MC.CHI8 database was lost.

    If you require any further assistance, our Support Team in Discord will be more than happy to help you.

  • Monitoring
    Monitoring

    All affected nodes except MC.CHI8 are online again according to our monitoring software. Clients will need to manually power on their server again.

    We'll be monitoring over the hour and will continue to actively work on restoring access to MC.CHI8.

    Please reach out to our Support Team on Discord if you need any assistance.

  • Investigating
    Update

    We are still investigating this issue. In the meantime, we've re-scoped this incident with the correct nodes affected according to our monitoring software.

  • Investigating
    Investigating

    We have detected a problem related to WinterNode.com services and are actively investigating.

    More details to follow.