Thread

1. krn1p4n1c · 1791031193 · focus · HN ↗

   tar cf - . | tar xf - -C <dest>

   1. Rygian · 1791031542 · focus · HN ↗

      with a sandwiched `| pv |` for fun stats

      1. dspillett · 1791032973 · focus · HN ↗

         And `| ssh <target> ` for a remote copy.

         (or ssh <target> prepended rather than sandwiched, to copy from remote)

         1. Rygian · 1791035361 · focus · HN ↗

            At some point it starts very much looking like zfs send | pv | ssh <target> zfs receive

            1. krn1p4n1c · 1791039133 · focus · HN ↗

               Seriously. I do wish ZFS was everywhere.
         2. krn1p4n1c · 1791039096 · focus · HN ↗

            I usually sandwich zstd and base64 encode/decode in with the ssh pipe. Used when I need root permissions to pull the remote dirs.

            1. dspillett · 1791046192 · focus · HN ↗

               You shouldn't need to B64 encode for transfer - piping over SSH is binary safe. Verifying with 100Mbyte of random data:

                   user@host:/tmp$ cat rnd.file | sha256sum
                   71d3392be59210c517ab889fca715a46dee54f6326a9d901db0addd351607cb6  -
                   user@host:/tmp$ cat rnd.file | ssh user@otherhost sha256sum
                   71d3392be59210c517ab889fca715a46dee54f6326a9d901db0addd351607cb6  -
                   user@host:/tmp$ cat rnd.file | zstd -z | ssh user@otherhost "zstd -d | sha256sum"
                   71d3392be59210c517ab889fca715a46dee54f6326a9d901db0addd351607cb6  -
               

               When using zstd (or anything else) make sure you don't have something in your .sss/config that would make ssh use a compression as you'll waste a bit of CPU to send a little more data as a less efficient compression method is applied on top. Though TBH I think that unless I'm on a very slow link, if I'm sending something big enough to need compression it is probably in a pre-compressed or otherwise uncompressable format anyway (video, photos, …) so zstd does little, the one exception I can think of being if I'm throwing an unencrypted VM image from place to place.

Unofficial Hacker News client; not affiliated with Y Combinator.