ATLAS Media Group Status Page

All systems are operational.

ATLAS Media Group Status Page

The latest status updates from the team. Updates provided on known and ongoing incidents.

Planned maintenance

  • Upcoming

    We will be updating MastodonAppUK to the latest version of Mastodon. During this time there will be a short outage while we restart the web server and run the relevant migrations.

    We expect the maintenance overall to take approx 1 hour with an outage of approx 10 mins to complete the switch over.

    Affected Components: MastodonApp.UK Website, MastodonAppUK Advanced / Full Text Search, MastodonAppUK Database Service, MastodonAppUK E-Mail Service, MastodonAppUK Media Storage and MastodonAppUK Queue & Content Processing Service
  • We need to perform routine maintenance to our physical hardware located within the Redditch Data center. As part of this we will need to take all services offline for a short amount of time while we apply system and security updates to our shared storage pool and restart the underlying host. We will also be performing patching and upgrades to our application hosts during this time and will be failing over our applications across the nodes in the environment.

    We expect the maintenance itself to take approx 3 hours with a short outage of up to 30 mins throughout this while we take the environment offline for full system restarts and hardware improvements.

    Affected Components: VPS Control Panel, Universeodon Database, Universeodon Queue & Content Processing Service, Universeodon Website, Universeodon Relay, Billing Panel, Blog, MastodonApp.UK Website, MastodonAppUK Database Service and MastodonAppUK Queue & Content Processing Service

Past incidents

No incidents reported.

No incidents reported.

No incidents reported.

No incidents reported.

Fixed

3 years ago

Maintenance has concluded.

There is an outstanding task to resolve with content processing however at this time the temporary configuration can remain in place to be worked on at another time.

Investigating

3 years ago

We have managed to bring our content processing back online temporarily to restore service. We are currently investigating what exactly went wrong.

Identified

3 years ago

The upgrades to our content processing appear to have resulted in some more significant issues and our content processing is currently unable to start. We are working to resolve the issue ASAP

Identified

3 years ago

All user facing upgrades have now completed. We will now be applying the patches to our content processing servers which may result in small delays with sending / receiving posts from remote servers.

Identified

3 years ago

We have completed the bulk of the Mastodon upgrade and are now running the latest version of Mastodon. We will now be rolling the update out on our second web server before progressing with further OS Patching to the physical servers which have patching outstanding.

Identified

3 years ago

We are comfortable at this point to progress with the upgrade to Mastodon itself. Maintenance should now conclude by 17:00 UK Time.

Identified

3 years ago

We are continuing to work towards progressing out upgrade. At this time we are awaiting database backups which failed to upload due to earlier server restarts. As soon as these are downloaded we will be able to progress with the Mastodon software upgrade.

Identified

3 years ago

We have identified a minor issue with our database server which we will need to resolve before we can fully progress with the upgrade. The maintenance is now likely to continue until 16:00 UK Time.

Identified

3 years ago

Upgrades to the host running the primary database is now well under way and we are performing one of two reboots to the database server. Our next step from here is to apply all of the OS Patches to the physical server that the database VM resides on and reboot everything at that point. From here we should be in a position to start the actual mastodon software update.

Identified

3 years ago

We are finishing the steps to upgrade our "Web02" VM (One of two user facing webservers that host the site) as well as the physical host that runs it and the router firmware on that host. Next we will be moving on to complete the same steps on the Database server before we start the actual Mastodon upgrade. During the database upgrades the site is likely to have intermittent outages.

Identified

3 years ago

We have now completed the most disruptive works to the site. We will now be moving on to a number of underlying OS Patches for the fleet where there will be less disruption.

Identified

3 years ago

Maintenance has now started. We will be starting to upgrade a number of components that are most disruptive first (Load Balancer, Redis Storage) and will then move to reduce resiliency on the site before upgrading the Mastodon software itself.

Reported

3 years ago

[MAINTENANCE UPDATE] This Incident is to provide live updates on ongoing maintenance.

We are planning to update the Mastodon software to the latest version. During this time the site will be unavailable and there may be delays in receiving remote posts and local posts being federated. We expect the maintenance to last no more than 2 hours.

MastodonAppUK - Mastodon Update

3 years ago
Complete

We are planning to update the Mastodon software to the latest version. During this time the site will be unavailable and there may be delays in receiving remote posts and local posts being federated. We expect the maintenance to last no more than 2 hours. Updates will be posted here: https://status.atlas-media.co.uk/incidents/63

No incidents reported.

Fixed

2 years ago

Retroactive Close

Watching

3 years ago

The cause of the outage appears to be a significant DDoS attack against our infrastructure which resulted in a short outage while our systems mitigated the attack. Service is currently restored and support will be actively monitoring for further attempts to take the infrastructure offline.

Reported

3 years ago

We are currently investigating a major outage of the forums and website. Updates to follow.