This gets worse the further away (network latency ping) you are from AWS EAST-1 West Viginia (If I recall correctly). There are no CloudFront to assist etc. as everything is run on the AWS server in AWS EAST1.
Yes, there is a bandwidth-delay product penalty with S3.
How would CloudFront help at all? Because of the nature of the fetches there is no value in caching, there's no prefetch possible for objects, either – so hiding the simple-but-slow RESTful nature of S3 isn't possible. There are many things that militate against efficient bulk restores of entire filesystems – check out the metadata and deduplication design components.
The most recent archive is the result of incremental changes, but a restore from the archive restores everything. It contains more than it contains, if you know what I mean.
Those are the random thoughts of an under-caffeinated person. Cheers.