[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Delete archive time
On 04/18/12 12:13, Michael Stevens wrote:
> The machine I was particularly complaining about has taken almost
> exactly 48 hours to delete around 300 archives (one failed due to
> network problems my end).
Ok, 48 hours is definitely well into the annoying range...
> I had one delete I wanted to do left over, which I've timed:
>
> root@osaka:~# time tarsnap -d -f osaka-home-2011-05-18
> Total size Compressed size
> All archives 926836591828 596496483543
> (unique data) 96754663741 68973013309
> This archive 148593871412 93015468246
> Deleted data 8215346246 6582996263
>
> real 14m37.343s
> user 0m42.731s
> sys 0m4.344s
>
> Network connection is roughly 10mbit down/1mbit up ADSL line.
... on the other hand, you do realize that you're deleting archives at a
rate of almost 170 MB per second, right? :-)
> In my case, the reason I'm deleting so much is that I have no automated
> way of cleaning up old incremental backups, and they're cheap, so I've
> let them build up, and am finding cleaning things up is slower than I
> expected.
Right. And if the bits I'm working on right now turn out well this sort
of "periodic cleanup" should be a good way of doing things, since deleting
many archives at once will end up being faster than deleting them one by
one.
> I'm wondering if a cheap fix for this would be to provide some higher
> level operations (possibly as a wrapper), like "archive daily and keep
> last n days" (<insert attempt to get colin to do all the work here>).
That's not possible -- when you create a new archive, the Tarsnap server
has no way of knowing (nor *should* it be able to know, as a matter of
security) which old blocks your new archive is using.
--
Colin Percival
Security Officer, FreeBSD | freebsd.org | The power to serve
Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid