[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Big initial upload
On Wed, 29 Jun 2016, at 09:29, Colin Percival wrote:
> On 06/28/16 23:33, Gregory Orange wrote:
> > The numbers are in the order of 1TB of data over Australian ADSL at a maximum
> > of 1 megabit per second uplink. By my reckoning, that might finish in 36
> > hours, but is more likely to be double that.
>
> https://www.google.ca/webhp?#q=1+TB+%2F+1+mbps
>
> 92.59 days
Ouch. As an expat Kiwi I feel your pain. Latency and bandwidth are not
our friends.
This might be an option *waves hands vigorously*
- create your tarsnap keys & server info and do a mini backup to prove
it works
- send an encrypted drive to AWS and have them transfer it into S3/EC2
for you
- spin up a new VM in AWS and extract the encrypted info
- transfer the local server keys to the VM
- run the first backup in AWS directly off your transferred data
- transfer the tarsnap cache info back to Australia via rsync or tar/scp
- start running backups from Oz again
- rotate the tarsnap keys used for backup
There are a fair few details elided but largely this looks to me like a
tarsnap-driven "reverse" server recovery, if that helps make sense of it
- you are transferring credentials and cache between machines, but for
the purposes of backup and not recovery.
My 2 major concerns would be:
- a WAN-based full recovery is going to take a looooooooooooong time be
careful this is what you want
- general risks of hosting sensitive keys and potentially all your data
in AWS even briefly, although GELI and similar encryption layers would
at least reduce the risk somewhat
A+
Dave