Affected
Major outage from 8:06 PM to 3:54 PM
Major outage from 8:06 PM to 3:54 PM
- ResolvedResolved
No further prolonged outages have occurred since our last update. We'll be marking this as resolved for the time being.
As a friendly reminder, issues with our Panel do not impact the accessibility of your game servers as they are hosted on separate infrastructure.
We open similar incidents for all our services as well to keep you informed. You may view them on https://status.winternode.com/history/1 and https://resources.winternode.com for uptime as reported by our monitoring software.
If you are getting the error "We're having some trouble connecting to your server, please wait…", please open a ticket in our Discord Server, as we may need to restart the daemon which interfaces between the Game Panel and your server. The easiest way to check, is to visit the Web File Manager.
- MonitoringMonitoring
We've received acknowledgement of the incident from our Panel Provider at 1:44 AM PT. A temporary fixed has been put into place at 2:30AM PT and an investigation into what occurred is underway by the Panel Team. The Game Panel is now online again.
- InvestigatingInvestigating
We're getting a bit tired of formal messages… We know the Panel is down again 🤷♂️.
Game servers hosted by us are not affected. It's been a rough couple of days for our Panel Provider. 😐 Feel free to take a look at the previous post-mortem provided to learn more about the current infrastructure setup.
- ResolvedResolved
A post-mortem was provided by our Panel Team - https://gamepanel.notion.site/11-10-2023-0d4aaef1626f4494b6238ef7044de522
In summary: Currently all traffic is diverted to a small set of webserver Point of Presence (POPs) (mostly nl-ams-web01), but in the upcoming days 3 new POPs will be introduced and ready to serve you our Game Panel with lower latency and resiliency.
- MonitoringMonitoring
Our Panel Team is actively working on improving our Panel infrastructure to diversity the number of providers used to serve our Game Panel. After everything falls into place, it is expected to be uphill from here and a big reduction in the chances of downtime, as more webservers will be available to takeover any traffic from Points of Presence that go unavailable.
Game Servers themselves are managed separately and are not affected.
More information will be provided soon. We do receive automated alerts for whenever the Game Panel is unavailable.