On 13 Aug 2025, at 14:12, Michael Sierchio <kudzu@tenebras.com> wrote:
This gets worse the further away (network latency ping) you are from AWS EAST-1 West Viginia (If I recall correctly). There are no CloudFront to assist etc. as everything is run on the AWS server in AWS EAST1.
Yes, there is a bandwidth-delay product penalty with S3.
How would CloudFront help at all?
quicker TCP/IP session establishments to a local endpoint … yeah, that does help , but that is part of what I was trying to convey based on the design/setup of tarsnap Because of the nature of the fetches there is no value in caching, there's no prefetch possible for objects, either – so hiding the simple-but-slow RESTful nature of S3 isn't possible. There are many things that militate against efficient bulk restores of entire filesystems – check out the metadata and deduplication design components.
The most recent archive is the result of incremental changes, but a restore from the archive restores everything. It contains more than it contains, if you know what I mean.
Oh I agree, just, that in modern setups, I’ve found that single threads just can’t cope with the amount of data being shoved around. You need some multi threading and lots of prefetches etc. etc. Those are the random thoughts of an under-caffeinated person. Cheers.
I feel the pains.. getting caffeinated now… rather.. the day's caffeine already used up before I got to the real work :D |