Re: Copy/migrate archives/history with a new key?

[creed-january-twig@duck.com]

Hey Chris, thanks a lot for the details.

This is what I ended up having:

tarsnap:
  keyfile: "/Users/user/tarsnap.key"
  cachedir: "/Users/user/.cache/tarsnap"

jobs:
  prune:
    target: "$date”	// (comment for this mail) this is the only one that didn’t throw a date-related error
    dateformat: "%Y-%m-%d_%H-%M-%S"
    deltas: 1h 1d 7d 30d 365d

(^ I basically needed a target that was like <anything - multiple hostname><date as per dateformat>)

I’ve been trying to run Tarsnapper for the past few hours, but I keep getting different errors. I’ve tried skipping the config file and passing everything in the command line, and I’ve tried every combination/order of commands and arguments.

Even tried like this as the GitHub readme says:

tarsnapper --keyfile <key path> --target "$date" --deltas 1s 1d 7d 30d 365d - expire --dry-run

again - removing args, adding, changing orders, etc. (with and without the hyphen before expire and moving expire around).

It hasn’t even reached a stage where it will finally show me the result of the dry-run. 

Some of these are here:

- tarsnapper.config.ConfigError: %s: target must make use of the following placeholders: date

> The best part are these two errors, tarsnap complains:
- tarsnap: Keys must be provided via --keyfile option

> and when I passed it, tarsnapper complained:
- tarsnapper: error: unrecognized arguments: -c // same for --config, --config-file

- tarsnapper: error: argument {make,expire,list}: invalid choice: '/Users/user/.config/tarsnapper/config.yaml' (choose from make, expire, list)
- Skipping 'None', does not define deltas

I’m not sure what I’m doing anything wrong (but obviously I am). I thought of reaching out to the GitHub issues, but the last issue was updated in 2022.

I suppose I’ll give up. I should’ve thought before choosing Tarsnap. I lack the expertise, perseverance, and patience to use this tool I realise. Actually, I never considered Tarsnap before Tarsnap-GUI  was launched :) and when I started using the GUI, I didn’t check the maintenance features.

> On 16 Aug 2025, at 3:01 AM, Chris Leyon <cleyon_at_gmail.com_creed-january-twig@duck.com> wrote:
> 
> I haven't studied tarsnapper's code so I can't tell you exactly what it does or does not delete. But I can show you the resulting behavior from my deltas line (1d 7d 30d 90d 360d) and hopefully explai 
> DuckDuckGo did not detect any trackers. More Deactivate I haven't studied tarsnapper's code so I can't tell you exactly what it does or does not delete.  But I can show you the resulting behavior from my deltas line (1d 7d 30d 90d 360d) and hopefully explain the results:
> $ # current date is 2025-08-15
> $ tarsnap --list-archives -v | sort -n -k2,3 | column -t
> host/job-20240723-045000  2024-07-23  04:50:00
> host/job-20241021-045000  2024-10-21  04:50:00
> host/job-20250119-045000  2025-01-19  04:50:00
> host/job-20250419-045000  2025-04-19  04:50:00
> host/job-20250519-045000  2025-05-19  04:50:00
> host/job-20250618-045000  2025-06-18  04:50:00
> host/job-20250718-045000  2025-07-18  04:50:00
> host/job-20250725-045000  2025-07-25  04:50:00
> host/job-20250801-045000  2025-08-01  04:50:00
> host/job-20250808-045000  2025-08-08  04:50:00
> host/job-20250809-045000  2025-08-09  04:50:00
> host/job-20250810-045000  2025-08-10  04:50:00
> host/job-20250811-045000  2025-08-11  04:50:00
> host/job-20250812-045000  2025-08-12  04:50:00
> host/job-20250813-045000  2025-08-13  04:50:00
> host/job-20250814-045000  2025-08-14  04:50:00
> host/job-20250815-045000  2025-08-15  04:50:00
> 
> So starting with "1d", I get daily backups ever day until I reach 7 days on the 8th.  Then I get weekly (7d) backups until I reach a month (30 days) on July 18.  Then I get monthly (30 day) backups going back a year.  (My records stop here.)  tarsnap runs every day at 04:50 and tarsnapper's algorithm retains only these archives according to the schedule.  I hope this helps.  I would also add that tarsnapper seems to support a "--dry-run" option that you might explore.
> 
> On Fri, Aug 15, 2025 at 2:04 PM <creed-january-twig@duck.com> wrote:
> Thought of trying that. But I am not sure what it does if it doesn’t find any archive on a certain day where a certain delta will lead it to? What I am trying to say is, since it is not a very widely used or has layperson-proof documentation, I am not really sure how it behaves.
> 
> Anyway, so if I have to achieve something like "10h 7d 6w 12m 5y” retention and delete the rest, shall I have to set it as 
> 
> ‘ deltas: 1d 2d 3d 4d 5d 6d 7d 8d …. '
> 
> 1d: the GitHub says - this means 24 retentions, right?
> 2d 3d 4d 5d 6d 7d 8d: for 7 days?
> and beyond this basically find 6 day-numbers so that it will cover 6 weeks and then 12 day numbers that will cover 12 monthly retentions, right? 
> 
> (With the assumption that it will not only try to keep the exact archive on that exact day/date but one closest such match.)
> 
> I have kept ‘ target: "{date}” ‘ because I have 4-5 different prefixes but they all backed up the same data set (it’s just because of a change in job name). So my assumption was that should work and it will only look at the {date} part of the archive names which is what I wanted.
> 
> 
>> On 15 Aug 2025, at 11:17 PM, Chris Leyon <cleyon_at_gmail.com_creed-january-twig@duck.com> wrote:
>> 
>> I use tarsnapper as well, but only run it once per day so I can't speak to a delta of "10h".  I just use the number of days corresponding to the duration.  My deltas line is
>>     deltas: 1d 7d 30d 90d 360d
>> Tweak as needed.  Also my target includes the `name' variable:
>>     target: <HOSTNAME>/$name-$date
>> 
>> On Fri, Aug 15, 2025 at 1:14 PM <creed-january-twig@duck.com> wrote:
>> Thanks Colin, I’ll reach out once it's done.
>> 
>> I have ~2000 archives. I likely skipped pruning planning when setting up tarsnap-gui (haven’t reinstalled it on my reset Mac, so not sure if it even supported pruning). I don’t want to keep this many archives — and even re-encryption won’t be fast with so many, even if each archive is small.
>> 
>> I saw '--fsck-prune' but it’s not what I thought.
>> 
>> Is there a simple way to delete all except something like — hourly:10 daily:7 weekly:6 monthly:12  yearly:5? 
>> 
>> I have 'tarsnap --list-archives | sort' saved (can add '-v' if needed), and since all archive names end in '%Y-%m-%d_%H-%M-%S', I can script something or use an LLM to pick which to keep.
>> 
>> I found https://mail.tarsnap.com/tarsnap-users/msg01678.html and had seen the helper scripts section already. So Tarsnapper is available on homebrew luckily, because couldn’t make prunef ‘make install’ happen.
>> 
>> But then Tarsnapper uses some retention scheme (i.e deltas) in such a way that I guess I can only use days. So it doesn't work:
>> 
>> [tarsnapper’s config.yml]:
>> 
>> jobs:
>>   prune:
>>     target: "{date}"
>>     dateformat: "%Y-%m-%d_%H-%M-%S"
>>     deltas: 10h 7d 6w 12m 5y
>> 
>> [and then (I will remove dry-run after checking once)]:
>> tarsnapper -c ~/.config/tarsnapper/config.yaml expire --dry-run
>> 
>> results in:
>> 
>> > tarsnapper.config.ConfigError: Not a valid delta: 12m
>> 
>> 
>> PS. Any easy way to search across https://mail.tarsnap.com/tarsnap-users, other than opening every link one by one?
>> 
>> > On 15 Aug 2025, at 3:07 AM, Colin Percival <cperciva_at_tarsnap.com_creed-january-twig@duck.com> wrote:
>> > 
>> > On 8/14/25 11:04, creed-january-twig@duck.com wrote:
>> >> I *have to* change my tarsnap key (or rather, stop using the old key).
>> >> I see this https://www.tarsnap.com/tips.html#copy-archive as well but I don’t really understand what it is and what it does - but I don’t see a key mentioned in the command so I guess not like “restic copy”.
>> > 
>> > Right, that's for copying one archive, using the same keys and within the
>> > same archival space.
>> > 
>> >> This https://www.tarsnap.com/man-tarsnap-recrypt.1.html seems to be the only way, right?
>> > 
>> > Yes.  That creates a new archival space, copies everything across, and then
>> > deletes the old copy.
>> > 
>> >> Also, the original/existing key was not *passworded*, can I generate the new key as ‘--passphrased’ and then proceed with the recrypt? I am asking because I believe to re-encrypt, ‘tarsnap-keyregen’ has to be used and the key is derived from the old key.
>> > 
>> > Correct.  To be more precise, the chunking parameters are kept from the old
>> > key but everything else is generated anew.  (The chunking parameters need to
>> > be kept so that new data will deduplicate against the copied data.)
>> >> This also raised the question - does it render the old key useless after the re-encryption is done, or both keys have access now?
>> > 
>> > Both keys will work but they'll access different archival spaces (and the
>> > old keys will point to an archival space with no archives after recrypt
>> > deletes everything using the old keys).
>> > 
>> > If this is a "keys were stolen" scenario then let me know and I can disable
>> > the old keys.
>> > 
>> > -- 
>> > Colin Percival
>> > FreeBSD Release Engineering Lead & EC2 platform maintainer
>> > Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
>> > 
>> 
>> 
>