Re: tarsnap and sparse files

[Colin Percival <cperciva@tarsnap.com>]

[Resending my earlier reply to Eric to the list, since Thunderbird managed
to break "reply all" functionality.]

On 02/09/17 09:59, Eric Brown wrote:
> I am working with some OpenBSD vmd disk images. They are allocated at
> 1T, though they have only a base OS install weighing in at about 6G of
> data.
> 
> I am running tarsnap on this sucker, but it never seems to finish.  I am
> wondering if there is something going on here, such as tarsnap unfurling
> the 1T file (but perhaps compresses it supremely well) or if maybe I
> just need to be more patient.

Sending a SIGINFO should confirm this, but I suspect tarsnap is spending
all of its time reading and deduplicating zeroes from the sparse file.  It's
just possible that the libarchive code is doing something smarter than I
remember, but I don't think so.

The best workaround I can suggest for this is to find a tool which can archive
sparse files -- I think GNU tar has support for this? -- and use that to turn
your 1T sparse file into a ~6G non-sparse file.  It's not ideal, but sparse
files are rare enough that in ten years of tarsnap you're the first person I
remember who needed to handle large sparse files...

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid