Re: Interpreting "dry-run archival without keys"

[Colin Percival <cperciva@tarsnap.com>]

On 11/05/15 15:55, Quinn Comendant wrote:
> The process took about 8 hours, with fairly intense CPU usage and disk IO. Is this dry-run considered representative for a typical backup session? In other words, will tarsnap take 8 hours (discounting the time to actually do the uploading) to discover what files need to be backed-up, every time? If so, my plan to run it hourly is hopeless.

On subsequent runs tarsnap can recognize unmodified files and short-circuit
the deduplication process.

> The printed results show "new data, compressed size" is "245 GB". Is that
> value before or after the magical de-duplication that is to be expected?
> Will my monthly fee (at least initially) be approximately 245 * $0.25 =
> $60/month?

It looks like your 270 GB was deduplicated to 262 GB, which was then
compressed down to 245 GB.

> When I read that it’s “possible to store ‘archives adding up to several
> terabytes’ while paying less than $10/month” I initially assumed the
> deduplication compares my data with that uploaded by someone else (so my
> "Rembihnútur.mp3" isn't uploaded, if another tarsnap user has already
> uploaded "Rembihnútur.mp3"). But that’s not possible, because
> deduplication is only per-account-key.

Correct.  The "adding up to several terabytes" is because you can have a
large number of backups which deduplicate against each other -- for example
I'm backing up about 50 GB of data from my laptop every hour, but I have
about 20,000 archives; my 1.1 PB of data deduplicates down to 400 GB.

> Then I assume my pricing estimate
> above is accurate, and tarsnap is a high-end service that is not priced for
> consumer use. =(

I think most consumers only use tarsnap to back up their more important
files, not the contents of their MP3 collections. ;-)

Colin Percival

> -------------------
> 
> [quinn@macbook] tarsnap --dry-run --print-stats --humanize-numbers -c --exclude=Films /Users/quinn
> tarsnap: Performing dry-run archival without keys
>          (sizes may be slightly inaccurate)
> tarsnap: Removing leading '/' from member names
> tarsnap: SystemKey: could not open file: Permission denied
>                                        Total size  Compressed size
> All archives                               270 GB           251 GB
>   (unique data)                            262 GB           245 GB
> This archive                               270 GB           251 GB
> New data                                   262 GB           245 GB
> tarsnap: Error exit delayed from previous errors.
> 

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid