Re: Delete archive time

[Michael Stevens <mstevens@etla.org>]

On Wed, Apr 18, 2012 at 10:09:48AM -0700, Colin Percival wrote:
> [picking a random email to reply to...]
> 
> On 04/18/12 06:34, Michael Stevens wrote:
> > I'm finding deleting archives is very slow and quite bandwidth intensive
> > as well.
> 
> The bandwidth used should be about 0.1% of the size of the archive you're
> deleting; are you seeing more than this?

It looks like I've done about 10-20gb of bandwidth in the last 48 hours
deleting archives. I'm deleting ~100 archives with content mostly
duplicated between them.

> > I've been trying to clear up old archives and deleting about a year's
> > worth of one a day - it's been going around 36 hours now and using a
> > fair bit of bandwidth.
> 
> Can you send me your --print-stats output?  It's possible that if Tarsnap's
> deduplication worked extremely well when it was creating archives then the
> 0.1% overhead is significant -- I hadn't worried about it because I figured
> it would always be much less than the time taken to originally create the
> archives.

On one machine (but not the one I'm referring to above):
									Total size  Compressed size
All archives                         603859837025     245148994034
  (unique data)                       24571415132      10465650048

I'll see if I can get some more numbers later.

Michael