The important information first: All data stored on Tarsnap is safe, but you can't access it right now. I'm working to bring it back online. The detailed story: At approximately 2023-07-02 13:07 UTC the main Tarsnap server suffered a "failed system status check" in Amazon Web Services vernacular. In plain language, something broke -- either the server hardware, the power, or its connection to the rest of the EC2 network (including network-attached disks). At approximately 2023-07-02 13:52 UTC the server rebooted (probably running on new hardware, considering that it's a virtual machine), but had filesystem corruption affecting cached metadata. This cached metadata essentially tells the Tarsnap server where to find user data in Amazon S3; without it, the Tarsnap server cannot service requests. I *think* that the metadata present is accurate as far as it goes, but it definitely lacks information about archives stored in the last few seconds before the outage started. While I could probably "replay" the most recently stored data from the backing store in S3, the nature of filesystem corruption means that I can't guarantee that this would work; since Tarsnap is a backup service I have to prioritize safety, and so I've started the process of reconstructing the Tarsnap server state from the "known good" data stored in Amazon S3. If anyone needs data urgently, I can bring the Tarsnap server up in read-only mode, subject to the caveat that the most recently stored data may not be available. Please contact me if this would be useful to you. It's too early in the state reconstruction process to provide an accurate ETA; my best guess is somewhere between 12 and 24 hours. As always, feel free to email me if you have any questions. Sorry about the headaches, -- Colin Percival FreeBSD Deputy Release Engineer & EC2 platform maintainer Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
Attachment:
OpenPGP_signature
Description: OpenPGP digital signature