Re: Tarsnap deduplication across machines

[Colin Percival <cperciva@tarsnap.com>]

On 01/09/14 09:36, Clayton Davis wrote:
> From the documentation, "if you have multiple machines, you almost certainly
> want to create a separate key file for each machine."  But if I've understood
> correctly, this will make it impossible to deduplicate files which are stored on
> more than one of those machines.

Correct.  This is the "almost" part in "almost certainly". ;-)

> Is it possible to reuse a key file across multiple machines?  Does this
> guarantee deduplication, or would the machine names also need to be the same? 
> And how would this be accomplished?  (Eg. run tarsnap-keygen, followed by
> overwriting the generated key file?)

There's nothing sophisticated about key file handling here -- you would just
take the same key file and copy it onto multiple machines.  The problem is
the tarsnap cache directory, which is used by the deduplication process -- it
tells tarsnap which blocks have previously been uploaded.

If you want to use the same key file on multiple systems, you would need to
(a) make sure that you're only running tarsnap from one machine at once, and
(b) copy the cache directories back and forth so that whenever a machine is
running tarsnap it has an accurate view of the "server state".

If you try to use the same key from multiple machines without doing this, you'll
get an error message -- tarsnap checks that the local and remote states are in
sync before it will start creating or deleting an archive.  (This check is not
comprehensive mind you -- there's ways you can trick it.  But it's good enough
to catch PEBKAC errors.)

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid