Re: Copy/migrate archives/history with a new key?

[james young <pronoiac@gmail.com>]

Glancing at the tarsnapper page:
> tarsnapper --keyfile <key path> --target "$date" --deltas 1s 1d 7d 30d 365d - expire --dry-run

I wasn't familiar with tarsnapper; I think:
* change to "\$date" from "$date"
* change to "1h" from "1s"
* keyfile: I'd try something like "-o configfile tarsnap.conf". the
tarsnapper docs are ... handwavey about this.
* if you have multiple jobs backing up there, you should have multiple
tarsnapper runs and a different, specific --target for each of those

I personally use suffixes for the archives like "-daily", "-weekly",
"-monthly". Going in, I copied the archives to (effectively) rename
them, and then shell scripts for pruning them. It feels more
debuggable and less magic.

-James

On Sat, Aug 16, 2025 at 1:17 AM <creed-january-twig@duck.com> wrote:
>
> Hey Chris, thanks a lot for the details.
>
> This is what I ended up having:
>
> tarsnap:
>   keyfile: "/Users/user/tarsnap.key"
>   cachedir: "/Users/user/.cache/tarsnap"
>
> jobs:
>   prune:
>     target: "$date”     // (comment for this mail) this is the only one that didn’t throw a date-related error
>     dateformat: "%Y-%m-%d_%H-%M-%S"
>     deltas: 1h 1d 7d 30d 365d
>
> (^ I basically needed a target that was like <anything - multiple hostname><date as per dateformat>)
>
> I’ve been trying to run Tarsnapper for the past few hours, but I keep getting different errors. I’ve tried skipping the config file and passing everything in the command line, and I’ve tried every combination/order of commands and arguments.
>
> Even tried like this as the GitHub readme says:
>
> tarsnapper --keyfile <key path> --target "$date" --deltas 1s 1d 7d 30d 365d - expire --dry-run
>
> again - removing args, adding, changing orders, etc. (with and without the hyphen before expire and moving expire around).
>
> It hasn’t even reached a stage where it will finally show me the result of the dry-run.
>
> Some of these are here:
>
> - tarsnapper.config.ConfigError: %s: target must make use of the following placeholders: date
>
> > The best part are these two errors, tarsnap complains:
> - tarsnap: Keys must be provided via --keyfile option
>
> > and when I passed it, tarsnapper complained:
> - tarsnapper: error: unrecognized arguments: -c // same for --config, --config-file
>
> - tarsnapper: error: argument {make,expire,list}: invalid choice: '/Users/user/.config/tarsnapper/config.yaml' (choose from make, expire, list)
> - Skipping 'None', does not define deltas
>
> I’m not sure what I’m doing anything wrong (but obviously I am). I thought of reaching out to the GitHub issues, but the last issue was updated in 2022.
>
> I suppose I’ll give up. I should’ve thought before choosing Tarsnap. I lack the expertise, perseverance, and patience to use this tool I realise. Actually, I never considered Tarsnap before Tarsnap-GUI  was launched :) and when I started using the GUI, I didn’t check the maintenance features.
>
> > On 16 Aug 2025, at 3:01 AM, Chris Leyon <cleyon_at_gmail.com_creed-january-twig@duck.com> wrote:
> >
> > I haven't studied tarsnapper's code so I can't tell you exactly what it does or does not delete. But I can show you the resulting behavior from my deltas line (1d 7d 30d 90d 360d) and hopefully explai
> > DuckDuckGo did not detect any trackers. More Deactivate I haven't studied tarsnapper's code so I can't tell you exactly what it does or does not delete.  But I can show you the resulting behavior from my deltas line (1d 7d 30d 90d 360d) and hopefully explain the results:
> > $ # current date is 2025-08-15
> > $ tarsnap --list-archives -v | sort -n -k2,3 | column -t
> > host/job-20240723-045000  2024-07-23  04:50:00
> > host/job-20241021-045000  2024-10-21  04:50:00
> > host/job-20250119-045000  2025-01-19  04:50:00
> > host/job-20250419-045000  2025-04-19  04:50:00
> > host/job-20250519-045000  2025-05-19  04:50:00
> > host/job-20250618-045000  2025-06-18  04:50:00
> > host/job-20250718-045000  2025-07-18  04:50:00
> > host/job-20250725-045000  2025-07-25  04:50:00
> > host/job-20250801-045000  2025-08-01  04:50:00
> > host/job-20250808-045000  2025-08-08  04:50:00
> > host/job-20250809-045000  2025-08-09  04:50:00
> > host/job-20250810-045000  2025-08-10  04:50:00
> > host/job-20250811-045000  2025-08-11  04:50:00
> > host/job-20250812-045000  2025-08-12  04:50:00
> > host/job-20250813-045000  2025-08-13  04:50:00
> > host/job-20250814-045000  2025-08-14  04:50:00
> > host/job-20250815-045000  2025-08-15  04:50:00
> >
> > So starting with "1d", I get daily backups ever day until I reach 7 days on the 8th.  Then I get weekly (7d) backups until I reach a month (30 days) on July 18.  Then I get monthly (30 day) backups going back a year.  (My records stop here.)  tarsnap runs every day at 04:50 and tarsnapper's algorithm retains only these archives according to the schedule.  I hope this helps.  I would also add that tarsnapper seems to support a "--dry-run" option that you might explore.
> >
> > On Fri, Aug 15, 2025 at 2:04 PM <creed-january-twig@duck.com> wrote:
> > Thought of trying that. But I am not sure what it does if it doesn’t find any archive on a certain day where a certain delta will lead it to? What I am trying to say is, since it is not a very widely used or has layperson-proof documentation, I am not really sure how it behaves.
> >
> > Anyway, so if I have to achieve something like "10h 7d 6w 12m 5y” retention and delete the rest, shall I have to set it as
> >
> > ‘ deltas: 1d 2d 3d 4d 5d 6d 7d 8d …. '
> >
> > 1d: the GitHub says - this means 24 retentions, right?
> > 2d 3d 4d 5d 6d 7d 8d: for 7 days?
> > and beyond this basically find 6 day-numbers so that it will cover 6 weeks and then 12 day numbers that will cover 12 monthly retentions, right?
> >
> > (With the assumption that it will not only try to keep the exact archive on that exact day/date but one closest such match.)
> >
> > I have kept ‘ target: "{date}” ‘ because I have 4-5 different prefixes but they all backed up the same data set (it’s just because of a change in job name). So my assumption was that should work and it will only look at the {date} part of the archive names which is what I wanted.
> >
> >
> >> On 15 Aug 2025, at 11:17 PM, Chris Leyon <cleyon_at_gmail.com_creed-january-twig@duck.com> wrote:
> >>
> >> I use tarsnapper as well, but only run it once per day so I can't speak to a delta of "10h".  I just use the number of days corresponding to the duration.  My deltas line is
> >>     deltas: 1d 7d 30d 90d 360d
> >> Tweak as needed.  Also my target includes the `name' variable:
> >>     target: <HOSTNAME>/$name-$date
> >>
> >> On Fri, Aug 15, 2025 at 1:14 PM <creed-january-twig@duck.com> wrote:
> >> Thanks Colin, I’ll reach out once it's done.
> >>
> >> I have ~2000 archives. I likely skipped pruning planning when setting up tarsnap-gui (haven’t reinstalled it on my reset Mac, so not sure if it even supported pruning). I don’t want to keep this many archives — and even re-encryption won’t be fast with so many, even if each archive is small.
> >>
> >> I saw '--fsck-prune' but it’s not what I thought.
> >>
> >> Is there a simple way to delete all except something like — hourly:10 daily:7 weekly:6 monthly:12  yearly:5?
> >>
> >> I have 'tarsnap --list-archives | sort' saved (can add '-v' if needed), and since all archive names end in '%Y-%m-%d_%H-%M-%S', I can script something or use an LLM to pick which to keep.
> >>
> >> I found https://mail.tarsnap.com/tarsnap-users/msg01678.html and had seen the helper scripts section already. So Tarsnapper is available on homebrew luckily, because couldn’t make prunef ‘make install’ happen.
> >>
> >> But then Tarsnapper uses some retention scheme (i.e deltas) in such a way that I guess I can only use days. So it doesn't work:
> >>
> >> [tarsnapper’s config.yml]:
> >>
> >> jobs:
> >>   prune:
> >>     target: "{date}"
> >>     dateformat: "%Y-%m-%d_%H-%M-%S"
> >>     deltas: 10h 7d 6w 12m 5y
> >>
> >> [and then (I will remove dry-run after checking once)]:
> >> tarsnapper -c ~/.config/tarsnapper/config.yaml expire --dry-run
> >>
> >> results in:
> >>
> >> > tarsnapper.config.ConfigError: Not a valid delta: 12m
> >>
> >>
> >> PS. Any easy way to search across https://mail.tarsnap.com/tarsnap-users, other than opening every link one by one?
> >>
> >> > On 15 Aug 2025, at 3:07 AM, Colin Percival <cperciva_at_tarsnap.com_creed-january-twig@duck.com> wrote:
> >> >
> >> > On 8/14/25 11:04, creed-january-twig@duck.com wrote:
> >> >> I *have to* change my tarsnap key (or rather, stop using the old key).
> >> >> I see this https://www.tarsnap.com/tips.html#copy-archive as well but I don’t really understand what it is and what it does - but I don’t see a key mentioned in the command so I guess not like “restic copy”.
> >> >
> >> > Right, that's for copying one archive, using the same keys and within the
> >> > same archival space.
> >> >
> >> >> This https://www.tarsnap.com/man-tarsnap-recrypt.1.html seems to be the only way, right?
> >> >
> >> > Yes.  That creates a new archival space, copies everything across, and then
> >> > deletes the old copy.
> >> >
> >> >> Also, the original/existing key was not *passworded*, can I generate the new key as ‘--passphrased’ and then proceed with the recrypt? I am asking because I believe to re-encrypt, ‘tarsnap-keyregen’ has to be used and the key is derived from the old key.
> >> >
> >> > Correct.  To be more precise, the chunking parameters are kept from the old
> >> > key but everything else is generated anew.  (The chunking parameters need to
> >> > be kept so that new data will deduplicate against the copied data.)
> >> >> This also raised the question - does it render the old key useless after the re-encryption is done, or both keys have access now?
> >> >
> >> > Both keys will work but they'll access different archival spaces (and the
> >> > old keys will point to an archival space with no archives after recrypt
> >> > deletes everything using the old keys).
> >> >
> >> > If this is a "keys were stolen" scenario then let me know and I can disable
> >> > the old keys.
> >> >
> >> > --
> >> > Colin Percival
> >> > FreeBSD Release Engineering Lead & EC2 platform maintainer
> >> > Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
> >> >
> >>
> >>
> >
>
>