Re: Database backup and deduplication question

[Colin Percival <cperciva@tarsnap.com>]

On 01/10/12 11:01, Greg Larkin wrote:
> All machines have been using unique keys up until now.  Following your
> instructions, I swapped keys between the Leopard (Mac Book Pro) and Snow
> Leopard (iMac) machines, and for good measure, I installed tarsnap on a
> Lion machine (Mac mini) and tried both sets of keys there.  At this
> point, it appears to be a machine-dependent problem:
> 
> 
> # iMac using Mac Book Pro keys (tarsnap 1.0.31): [broken]
> # Mac Book Pro using iMac keys (tarsnap 1.0.28): [works]
> # Mac mini using both sets of keys (tarsnap 1.0.28): [works]

And your previous results were:
# Mac Book Pro using Mac Book Pro keys (tarsnap 1.0.28): [works]
# iMac using iMac keys (tarsnap 1.0.31): [broken]
correct?

> You might think that based on the results above, tarsnap 1.0.31 is the
> cause of the problem, since it's the obvious difference between the
> three machines.  In order to remove that variable, I built 1.0.28
> (current version from MacPorts) on the iMac and retested.  That forced a
> rebuild of OpenSSL to bring it up to version 1.0.0f.  Previously, I had
> 1.0.0e installed on the iMac.  The results are interesting:
> 
> sh-3.2# tarsnap --keyfile mbp-keys/tarsnap.key --dry-run -cf testarchive
> file1 file1 part1 2>&1 | grep "^New data"
> New data                                105098054        105606442
> sh-3.2# tarsnap --keyfile mbp-keys/tarsnap.key --dry-run -cf testarchive
> file1 file1 part1 part2 2>&1 | grep "^New data"
> New data                                105244753        105751786
> sh-3.2# tarsnap --keyfile mbp-keys/tarsnap.key --dry-run -cf testarchive
> file1 file1 part1 part2 part3 2>&1 | grep "^New data"
> New data                                105331315        105837316

So that's
# iMac with Mac Book Pro keys (tarsnap 1.0.28 & new OpenSSL): [works]
right?  Can you try using building tarsnap 1.0.31 with the updated OpenSSL
in order to distinguish between the effect of the tarsnap version and the
OpenSSL version?  Also, can you try building tarsnap 1.0.28 the same way as
you built tarsnap 1.0.31 (manually from the source tarball, I'm guessing?)
in case you're getting a different compiler or build environment?

> These numbers are what I expect from tarsnap, so is it possible that
> some bug in OpenSSL triggered the original deduplication problem that I
> noticed? 

I can't see any way that would be possible, but I can't see any way that
this bug is possible either... so it's worth considering.

-- 
Colin Percival
Security Officer, FreeBSD | freebsd.org | The power to serve
Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid