All systems are operational

About This Site

Real time service status from across the ATLAS Media Group portfolio

Past Incidents

20th November 2022

Queue Service Delay to local content processing

We've identified an issue where our internal content processing for the Mastodon App service appears to be running slow and is currently suffering from around a 30 min queue. We're working to increase our content processing to cope with this increased demand.

  • Service restored.

  • We've reduced back the scale of our queue processing due to overloading the host that we were using. The surge in compute has reduced queue length back to a more manageable duration of around 60 seconds. We'll continue to monitor over the next few hours.

  • We've significantly scaled up our content processing for the queue that is backed up (Currently by around 5 times our previous processing scale) but we're still seeing around 20-25 mins delays on content being published.

    • Queue Service Content Processing Blocked

    We've identified an issue where our content processing has ground to a halt. We've identified the issue being with a full disk due to our system logs filling up. We've cleaned up this and are in the process of restarting services.

  • We've managed to now restore full service

  • We've managed to resolve the disk issues with the content processing and are in the process of re-running a large number of jobs that had been failing. This may result in delays to content being displayed on our site and communication with the wider fediverse.

    • MastodonApp.UK Website Mastodon UK Partial Outage

    We are seeing partial outages of the Mastodon site due to exceptionally high demand. Support are working to increase server capacity to cope with this increase in demand. Some users may be unable to reliably access the site, view messages or post content.

  • We've now restored service to the application

    • 19th November 2022

    No incidents reported

    18th November 2022

    No incidents reported

    17th November 2022

    No incidents reported

    16th November 2022

    No incidents reported

    15th November 2022

    MastodonApp.UK Website Mastodon Site Outage

    The MastodonApp.UK instance is suffering from a major outage, with the entire service being offline. Support is investigating.

  • The issue has been identified and the service is fully operational again.

    • 14th November 2022

    No incidents reported

    13th November 2022

    No incidents reported

    12th November 2022

    MastodonApp.UK Website Mastodon Site Outage

    We are seeing intermittent outages to the MastodonApp UK Website. Support are investigating.

  • Monitoring has confirmed the incident is now resolved

  • Service is fully restored, we have been able to restart the physical host running the web server and restore the server load to normal levels. We're continuing to monitor that server to ensure no further issues.

  • External monitoring currently confirms this is a wider spread outage, support continue to investigate.

    • 11th November 2022

    MastodonApp.UK Website MastodonApp - Increase in content processing delays

    We are starting to see increased delays in the ability for us to process content on the MastodonApp.UK Site, we are currently seeing queue delays of approx 12 mins for local processing including image and video uploads. This may cause delays to the local feed, notifications and your ability to upload files onto the site. Support are working to scale the environment appropriately to cope with this spike in demand.

  • Queue sizes are now back under control and service is fully restored.

  • We have introduced additional capacity to our content processing service which is starting to work through the queue size. Current delay is somewhere just over 15 mins and falling.

    • 10th November 2022

    No incidents reported

    9th November 2022

    MastodonApp.UK Website Intermittent 500 errors

    We are seeing intermittent errors on the Mastodon site. This is the result we believe of scaling up to cope with increased delays to our content processing. We are working to increase our database processing which we believe will resolve the issue.

  • We have been able to make some tweaks to our existing infrastructure which has helped to minimise these issues. At the same time the number of concurrent users on the site we believe has dropped as the night has got later, which has reduced the number of issues we've been seeing. We are planning to increase the site's scale tomorrow.

    • 8th November 2022

    MastodonApp.UK Website Increase Error Codes - 500 and 525

    We are seeing increased errors on the MastodonApp.UK Site. Support believe this is due to the significant increase in traffic and are working to resolve the issue.

  • Issue was resolved yesterday morning and this incident should have been closed.