Re: Backing up streams (subversion/mysql dump)

[Colin Percival <cperciva@tarsnap.com>]

Somehow I missed the original email...

On 01/08/15 21:32, Daniel Parks wrote:
> I'm afraid I haven't had a chance to test this with tarsnap, but typically the way to do this is with mkfifo, which creates a named pipe (a FIFO: First In First Out). An example from memory:
> 
> $ mkfifo mysqldump.sql
> $ mysqldump mydb  > db.sql &
> $ tarsnap -cf myarchive db.sql
> 
> Tarsnap should read the output of mysqldump and store it as db.sql. The db.sql named pipe on disk doesn't contain the data from mysqldump; if you try to read it again it will hang waiting for input from another process.

This doesn't work; tarsnap doesn't archive fifos.

>> On Jan 3, 2015, at 9:27 AM, Philipp Riegger <lists@anderedomain.de> wrote:
>> I am looking for a backup solution for mysql databases and subversion
>> repositories. I think it would be more interesting to backup the dumps
>> from "svnadmin dump" and the corresponding mysql command.
>>
>> Saving the dumps each time wastes a lot of disk space (and produces a
>> lot of unnecessary disk IO). Is there a way to save streams (or data
>> from stdin)? Since data from stdin has no timestamp and no user/group
>> information, maybe there is a way to create a fake tar file from stdin
>> and pipe that to tarsnap? I did some searching on the internet and read
>> the man page. This does not seem to be possible at the moment. Is that
>> correct?

You can't pipe data into tarsnap.  The reason for this is that tarsnap uses
the tar format internally, and so it needs to know the size of a file before
it starts to read the file data.

You can pipe an archive into tarsnap (e.g., `... | tarsnap -c -f foo @-`) but
this doesn't really solve the problem -- you need to know the amount of data
you're going to stream in before you start streaming the data, in order to
create the necessary archive header for tarsnap.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid