All systems are operational.
The latest status updates from the team. Updates provided on known and ongoing incidents.
The main website for the ATLAS Media Group Team
The ATLAS Media Group Ltd. Blog Platform
Service which hosts static content across a number of our parked and inactive domains.
The primary billing panel for the Superior Networks clients.
The VPS Control panel for Superior Networks Customers.
The main Superior Networks website.
Our servers in the French location.
Our new London Region for Superior Networks
The mastodonapp.uk site
The database service storing appropriate content for the service
Handles the processing of content both internally to our server as well as communicating with remote servers
Storage for all videos, pictures and other content on the site
Used to handle delivery of e-mails to users
Search capability for MastodonApp.UK
The Universeodon public website
Universeodon's Database
The Universeodon Queue and content processing service.
Universeodon Media Storage
Universeodon's Advanced / full text search service.
Universeodon e-mail service
The ActivityPub relay hosted on relay.universeodon.com
Located in the London Region | Legacy Server
Located in the France Region
Located in the London Region
We need to perform routine maintenance to our physical hardware located within the Redditch Data center. As part of this we will need to take all services offline for a short amount of time while we apply system and security updates to our shared storage pool and restart the underlying host. We will also be performing patching and upgrades to our application hosts during this time and will be failing over our applications across the nodes in the environment.
We expect the maintenance itself to take approx 3 hours with a short outage of up to 30 mins throughout this while we take the environment offline for full system restarts and hardware improvements.
Between 18:00 and 21:00 on Sunday 7th Jan 2024 the MastodonApp.UK site will be offline for scheduled maintenance. This will allow us to upgrade the site to the latest Mastodon software release.
Service was restored.
We are seeing a series of 502 errors when loading MastodonApp.UK, some of the site will work fine whereas some areas appear to have issues. We are investigating and will look to identify which of the web servers is at fault.
We have continued to monitor and the issue has been resolved.
We have managed to bring one of the two servers back online and work through the backlog of tasks to process. We are currently investigating why the other server is not resolving DNS properly.
We have identified an issue with our content processing which has resulted in a large number of images from remote servers not being processed as they should. This is due to the DNS settings on the server being lost. We are working to resolve the issue and put a permanent solution in place.
No incidents reported.
We have continued to see issues with DNS resolution and have now restored full capacity. We are now tracking this under the incident raised on Jan 6th.
Partial service is now restored. We are continuing to investigate and are now running at a reduced capacity.
We are seeing major issues with our content processing services. This started off with DNS resolution issues and at the current time the servers are not coming back online.
The data transfer has completed for our network infrastructure and full service has been restored. We will be looking to perform further maintenance behind the scenes to resolve some minor outstanding points.
We have completed the data transfer and as hoped have a significant amount more disk space available now on the host as a result of this migration back and forth. We will need to perform further emergency maintenance tomorrow to resolve some outstanding issues as well as some urgent maintenance to fully resolve this issue going forward and prevent it from hopefully happening again in the future.
The migration of the data to the new storage environment did succeed however the storage is not as fast or as reliable as it requires. This has however reduced the size of the disk to free-up enough capacity to move the server storage back without causing further outages.
We have provisioned additional infrastructure and started to migrate aspects onto this infrastructure. We are currently working to migrate the storage from our database server onto this infrastructure to enable us to complete a further migration in the coming days to resolve this issue on a longer term basis.
We are currently purchasing and deploying additional infrastructure to resolve this issue in the short term. We will need to work to validate that the approach scales in the longer term.
Our database server host has ran out of disk space causing a major outage to our database server. We are working to bring services back online.
This issue has been resolved.
We are monitoring the database service going forward, we know this issue is being caused by the high processing needs of our content processing service however we're unable to switch the processing onto a more efficient connection type due to a bug we have yet to identify. We hope that once we complete the software update to Mastodon this bug should be resolved.
We are seeing major performance issues with our database service which is causing partial outages to Mastodonapp.uk. We are working to identify and resolve the root cause.
We will be performing some critical maintenance to our production database server. During this time our MastodonApp.UK website and content processing will be offline to support the database maintenance tasks. This is expected to complete by 13:00 UK Time.
09:21 UK Time: Maintenance is now starting.
12:59 UK Time: Maintenance is currently over-running slightly, we are waiting on a final maintenance task to complete before re-activating the website.
16:07 UK Time: Maintenance has now completed and completed at around 15:20 UK time, apologies for the significant over-run.
We are currently updating our billing panel.
No incidents reported.