Re: Copy/migrate archives/history with a new key?

[Colin Percival <cperciva@tarsnap.com>]

On 8/22/25 14:07, creed-january-twig@duck.com wrote:
> I *literally* ran this by mistake:
>
> tarsnap -c -f "new-prefix_test" @@Job_most-essential-files_2025-07-08_10-00-08
> 	--print-stats \
> 	/usr/local/etc/ /path/exclude.pattern.file \
> 	-v --dry-run

If you have the latest version of tarsnap, this should have printed a warning
message along the lines of

> tarsnap: List of objects to archive includes '--dry-run'.  This might not 
be what you intended.

Once tarsnap sees a non-flag on the command line, everything else is treated
as a non-flag.  So that's why this wasn't a dry run.

> Changes were made on the server, and a "new-prefix_test" archive
> indeed was created. Verified with 'tarsnap --fsck'.

For future reference, there's no need to run tarsnap --fsck unless you lose
your cache directory or tarsnap asks you to.

> I had saved the full file list for each archive (in separate text files), and
> also output of "tarsnap --print-stats -f '*'".
>
> So I matched them after my mishap:
>
> - Job_most-essential-files_2025-07-08_10-00-08: list of files in the archives
> are exactly same before and after - Job_most-essential-files_2025-07-08_10-00-08
> sizes before:
>    Job_most-essential-files_2025-07-08_10-00-08    841 MB        783 MB
>    (unique data)                                   5.0 MB        1.7 MB
>
> - Job_most-essential-files_2025-07-08_10-00-08 sizes after (unique data
> decreased):
>    Job_most-essential-files_2025-07-08_10-00-0     841 MB        783 MB
>    (unique data)                                   890 kB        288 kB

That makes sense, data was copied into a new archive so it's no longer unique
to that archive.

> - new-prefix_test: its list of files is much smaller than the source archive.
> But when I compared the file list they match exactly until new-prefix_test file ended.
> As you see there is no ".part" in its name.
>    new-prefix_test                                 36 MB         15 MB
>    (unique data)                                   205 kB        37 kB

That's very strange.  Is there anything interesting about the next file in the
old archive after the new archive stopped?  I'm wondering if somehow tarsnap
failed in the middle of copying the archive... if that happened you should
have seen an error message though.

> Will this command work (fixed, afaicu, version of what I actually ran)?:
>
> tarsnap -c -f "new-prefix_test" @@oldarchive
> 	--print-stats \
> 	-X exclude.txt \
> 	-v --dry-run
>
> Are the orders of args fine? Or should @@oldarchive be moved to the end after
> every arg has been passed as it was in your example?

You should put the @@oldarchive at the end.

> And from the 'tips' docs page it looks like my 'excludes.file' should look like
> this?:
>
> -----
> ._*
> .DS_Store
> *.png *.PNG Library

These might need to be on different lines, otherwise I think tarsnap will
exclude anything named "*.png *.PNG Library".

> Also: tarsnap-gui appends "Job_" as "Job_{job name}_{timestamp}" to every
> archive name. Can I choose not to add that "Job_" or decide on my own how to
> name the archive while still using the GUI?

I think the GUI lets you enter your own archive names when creating a single
archive, yes.

--
Colin Percival
FreeBSD Release Engineering Lead & EC2 platform maintainer
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid