2

on a remote machine i want to launch a command that slowly produces a big stream (stream compression) and in the same time start downloading that stream. i don't want to first save the stream in a file on remote as it takes time to produce whole content and later it takes time to download it. i want to compress and download in parallel to save time. i can ssh/scp to the remote machine

i saw: How do I scp the (huge) output of a command directly to a remote machine?

but i want to pull, not push

Community
  • 1
piotrek
  • 203

2 Answers2

6

You can do that similar way:

ssh -T user@example.com 'producing_command' > big.txt

Just run the producing command on the server and write to the local file.

Jakuje
  • 10,827
2

While I agree with the answer of piping output through STDOUT and through SSH, I'd like to offer a variation of perspective:

You can use SSHfs (a FUSE filesystem) on the remote machine to mount a directory from your local machine and then write to it. If necessary this can be tunneled back through a connection from your local machine to the remote machine.

Example A:

LOCAL$  ssh user@remotemachine.tld
REMOTE$ sshfs user@localmachine.tld:/directory/on/localmachine /directory/on/remotemachine
REMOTE$ bigstreamcommand >/directory/on/remotemachine/bigstreamfile

As a result, the output will be slowly written to /directory/on/localmachine/bigstreamfile

If you can't reach the local machine from the remote machine by direct SSH (NAT without port forward, CGNAT, firewalls, ...) then follow
Example B:

LOCAL$  ssh user@remotemachine.tld -R54223:localhost:22
REMOTE$ sshfs -p54223 user@localhost:/directory/on/localmachine /directory/on/remotemachine
REMOTE$ bigstreamcommand >/directory/on/remotemachine/bigstreamfile

The only requirement for example B to work is, that you are running a SSH server on your local machine. You connect sshfs to localhost:54223, as that is a reverse tunnel from the remote machine, redirecting all connections back through your initial SSH connection and to the SSH server on your local machine.

It's maybe not as straight forward, but it's far more flexible, in the sense, that you can execute arbitrary file operations from the remote machine, directly on your local filesystem. I don't recommend to run commands that need to read a lot of data through this, as that will then of course need to be sent from your local machine to the remote machine, which, on residential connections, tends to be slow. Writing large files to your local machine should be much faster, just as download bandwidth is larger for residential connections.

For more details, please reread and then consult man sshfs.

TBR
  • 313