Re: Feasability of using tarsnap on crap internet connections

[Colin Percival <cperciva@tarsnap.com>]

On 11/05/15 14:16, Quinn Comendant wrote:
> Another question has come up while evaluating tarsnap. I'll be uploading
> from a MacBook in rural Colombia, with a flapping internet connection. It
> is a 3Mbps radio uplink, which is sufficiently fast to upload my 400GB
> (pre-de-duplication) data in a month or two (initial upload), BUT the
> connection is intermittent: the connection usually drops several times per
> hour, and at worst every few minutes.

How long does it drop for?  The tarsnap client will attempt to reconnect if
its TCP connection drops; network outages of under 5 minutes shouldn't be a
problem at all.  If you use the --retry-forever option, it will do what you
expect, too.

> I've been reading about the
> Interrupted Archives, and how subsequent archive creation (with a new
> archive name) will resume a previously interrupted one.

Not exactly *resume* -- but the new archive will be deduplicated against all
past archives, including the partial one.

> It seems this *might just work*, as long as I'm using a small
> `checkpoint-bytes` value and an archive name with a timestamp, except that
> every month I will have created maybe a few thousand archives. Perhaps
> that's fine, if I never need to think about the archives on a human scale,
> instead letting a script manage their creation and rotation. Obviously if I
> will ever need to use the `--list-archives` or `--recover` commands this
> would be insane.

Nothing wrong with --recover.  It just does something (check for an
interrupted archive creation and finalize it into a partial archive) which
gets done automatically the next time you create or delete an archive.

> How would recoveries work, with so many archives? Let's say I discovered my
> MacBook was compromised a week ago, and I want to do a full system
> restoration to the state of the files uploaded 8 days ago. Would it be as
> simple as running extract on the last archive with the timestamp of "8 days
> ago", i.e., `tarsnap -x -f backup-YYYYMMDD.SSSSSSSSSSS /`, with the most
> complete and consistent archive found by running something like `tarsnap
> --list-archives | grep YYYYMMDD | sort | tail -1`?

I'd go with something like `tarsnap --list-archives | fgrep -v .part | sort |
tail -1` to get your most recent full archive.

But you might not even need that; tarsnap is designed to work on internet
connections which are not entirely reliable.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid