0

I want to move all files in local directory to remote directory.

And I am using this code to achieve it

SOURCE_FILE=/var/www/oneserver/*
TARGET_DIR=/var/www/anotherServer
ARCHIEVEFILE=/var/www/archieveServer

/usr/bin/expect<<EOD
spawn /usr/bin/sftp -o Port=$PORT $USER@$HOST
expect "password:"
send "$PASSWORD\r"
expect "sftp>"
send "put $SOURCE_FILE $TARGET_DIR\r"
expect "sftp>"
send "bye\r"
EOD

It works fine, but sometime it stops and only send some files.

I also want to move the already sent file to ARCHIEVEFILE by using mv command. But don't get idea how to move it. I can't use SCP because the remote server not allowed using basic port, and the only way is using SFTP.

  • the $ARCHIEVEFILE is still the local server, just to backup/move so those files are not sent anymore with next cronjob

  • the TARGET_DIR is the remote server.

Can anybody help, please?

Martin Prikryl
  • 24,007
Al Kasih
  • 109

3 Answers3

2

There's no command in OpenSSH sftp to move files to remote directory.

What you can do is to:

  • use sftp put to upload the files (as you are doing already), and then

  • use shell rm command to delete the files after sftp is done (i.e. after EOD):

    rm $SOURCE_FILE

    Or use ! to escape to shell from sftp script, after you sent put:

    send "!rm $SOURCE_FILE\r"
expect "sftp>"

Of course, this is not an atomic solution. If a file is added between put and rm, it will be lost. For an atomic solution, you have to iterate files in a local directory and upload and delete them one by one. Also for a robust solution, you need to check if an upload was successful.

Martin Prikryl
  • 24,007
1

The following approach may simplify things a lot:

  1. Use sshfs to mount the remote share(s) as local path(s).
  2. Work with cp and/or mv as if all operations were local.

Compare this answer of mine.

You still need some logic to detect if cp to the remote location succeeded, only then mv to the local archive; otherwise retry or something. But now all SFTP-related work should be transparently handled by sshfs.

Additionally shell globbing and quotes you use makes your code prone to errors related to spaces in filenames etc.

After you mount the remote share e.g. in /mnt/a b/remote, it will be so much easier to handle this.

(Note: it is a good practice to use lowercase variable names).

  • To mount:

    sshfs -p $port $user@$host:"/path/on/the/remote/host/" "/mnt/a b/remote/"

    Use key based authentication or read Username and password in command line with sshfs. Read about security concerns.

  • A code stub to copy files:

    # I deliberately use paths with spaces to show how to handle them
for filename in "/source/location with spaces"/*; do
   cp "$filename" "/mnt/a b/remote/" &&
   mv "$filename" "/archive/location with spaces/"
done

    This && ensures mv will run only if cp succeeds.

  • To unmount:

    fusermount -u "/mnt/a b/remote/"
Kamil Maciorowski
  • 81,893
0

You could zip the source directory you wish to transfer 

zip -r -9 /path/to/your/zip_file.zip var/www/oneserver/*

(-r will recurse into subdirs , -9 will apply maximum compression for quicker transfer)

then scp the file to your remote server

scp /path/to/your/zip_file.zip user@host:/var/www/anotherServer

and then unzip the file on the remote server

ssh user@host 'unzip /var/www/anotherServer/zip_file.zip'

(there are many flags you can use with unzip you can use to overwrite or never overwrite exsiting files on your remote server, check man unzip for them , especially -n -o and -u )

hope it helps you

Sean Davey
  • 504