[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Tarsnap down?



Colin Percival wrote:
> Shaneal Manek wrote:
>> I can't seem to restore a tarsnap backup
> 
> That's odd.  I'm investigating.

Ok, I think I know what happened here.  Amazon's status page for the N. Virginia
EC2 cluster (where Tarsnap lives) states:
> 12:30 PM PDT We are investigating connectivity issues to a small set of
> instances in a single Availability Zone.
> 12:51 PM PDT Connectivity is being restored and instances are being brought
> back online.
>  1:08 PM PDT The event began at 12:05 PM PDT. Instances began recovering
> at 12:35 PM PDT and the majority of affected instances have now recovered.

Tarsnap never completely lost connectivity, but between 19:04 and 20:12 UTC it
had a significantly higher error rate; given the timing, I think it is very
likely that these are related.

Because of Tarsnap's fault-tolerant nature, the failed requests were retried;
unfortunately this slowed down some operations enough to make them time out.
Reads (archive listing and extraction) were more affected than writes and
deletes, since they timeout more aggressively (this was useful for performance
reasons, but in light of this glitch I think I'll tune that back a bit).

At the moment everything seems to be back to normal; if you encounter any more
problems, or if you have any questions, please let me know.

-- 
Colin Percival
Security Officer, FreeBSD | freebsd.org | The power to serve
Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid