Re: Big initial upload

[Dave Cottlehuber <dch@skunkwerks.at>]

On Wed, 29 Jun 2016, at 09:29, Colin Percival wrote:
> On 06/28/16 23:33, Gregory Orange wrote:
> > The numbers are in the order of 1TB of data over Australian ADSL at a maximum
> > of 1 megabit per second uplink. By my reckoning, that might finish in 36
> > hours, but is more likely to be double that.
> 
> https://www.google.ca/webhp?#q=1+TB+%2F+1+mbps
> 
> 92.59 days

Ouch. As an expat Kiwi I feel your pain. Latency and bandwidth are not
our friends.

This might be an option *waves hands vigorously*

- create your tarsnap keys & server info and do a mini backup to prove
it works
- send an encrypted drive to AWS and have them transfer it into S3/EC2
for you
- spin up a new VM in AWS and extract the encrypted info
- transfer the local server keys to the VM
- run the first backup in AWS directly off your transferred data
- transfer the tarsnap cache info back to Australia via rsync or tar/scp
- start running backups from Oz again
- rotate the tarsnap keys used for backup

There are a fair few details elided but largely this looks to me like a
tarsnap-driven "reverse" server recovery, if that helps make sense of it
- you are transferring credentials and cache between machines, but for
the purposes of backup and not recovery.

My 2 major concerns would be:

- a WAN-based full recovery is going to take a looooooooooooong time be
careful this is what you want
- general risks of hosting sensitive keys and potentially all your data
in AWS even briefly, although GELI and similar encryption layers would
at least reduce the risk somewhat

A+
Dave