0

I have a very simple utility script using ftplib on python 2.7 to upload web files to my remote host. However, storlines hangs and just sits there for a couple of minutes. Doesn't have the decency to throw any kind of exception.

  • I've verified that the connection works from the REPL. storlines hangs both from the REPL and from the commandline script.

  • my test file is 22 bytes (not KB, bytes), so it seems very unlikely that things are just timing out. Also, at the REPL, after I kill the hung upload via cntrl-c, I can get output from ftp.dir(), so I know I'm not losing the connection at some point.

  • the aforementioned output from ftp.dir() is much larger than my test file, and arrives within a few seconds, so I know the problem isn't network latency.

  • based on this prior SO I've tried adding ftp.set_pasv(False) before the upload function definition, but that does nada.

Does anyone else have any ideas about what could be going wrong here? Here's the code:

import sys
from ftplib import FTP
ftp = FTP([MYSERVER], [MYUSER], [MYPASSWORD])
full = sys.argv[1]
path = 'public_html/' + full[:full.rfind('/') + 1]
filename = full[full.rfind('/') + 1:]
ftp.cwd(path)

def upload(method, mode, filename):
    with open(filename, mode) as fileobject:
        method('STOR' +filename, fileobject)

if filename.endswith(('.htm','.html','.py','.cgi','.js','.css','.txt','.md', '.svg')):
    upload(ftp.storlines, 'r', filename)
else:
    upload(ftp.storbinary, 'rb', filename)
ftp.quit()
print 'Done!'

and, seriously, this is the full file I'm trying to upload, with the filename test.md:

this is a test. 
#test
Community
  • 1
  • 1
Paul Gowder
  • 2,409
  • 1
  • 21
  • 36
  • Have you tried with another FTP client? FTP is a protocol which behaves really bad with NAT or firewalls and not always PASV helps. Thus chances are high that this is not related to python but a general FTP related connectivity problem between your client and the server. And it is not sufficient that connection to port 21 only works since FTP uses dynamic ports for the transmitting the files. – Steffen Ullrich Feb 06 '16 at 20:36
  • hmm. Part of the problem is that I know very little about networking. :-( I've only tried with GUI apps otherwise (fetch on OSX works fine), and I'm on a not-fancy home dsl connection, so it's not like there's some kind of corporate firewall on my end to pass. but yeah, who knows what's going on serverside, or, for that matter, what nightmares my isp has created this side. ugh. – Paul Gowder Feb 06 '16 at 20:39
  • Typical nightmares on DSL connection might involve [DS-Lite](https://en.wikipedia.org/wiki/IPv6_transition_mechanism#Dual-Stack_Lite_.28DS-Lite.29) where you don't get a public IPv4 address. In this case the ISP needs to employ some helper applications to deal with FTP traffic, which often just add some layer of chaotic behavior. If possible move away from FTP for transferring data, its just a nightmare in all but simple network setups. – Steffen Ullrich Feb 06 '16 at 21:06
  • Thanks. That seems like an eminently plausible explanation for what's going on here. Ugh. – Paul Gowder Feb 06 '16 at 21:23

2 Answers2

0

So this is ugly, but I eventually figured out a hack-tastic solution, which just lets the nice existing ftp application that was written by someone who actually knows networking do the work: 

import sys
import os
full = sys.argv[1]
path = 'public_html/' + full[:full.rfind('/') + 1]
filename = full[full.rfind('/') + 1:]

def picktype(filename):
    if filename.endswith(('.htm','.html','.py','.cgi','.js','.css','.txt','.md', '.svg')):
        return 'ascii'
    return 'binary'

ftpstring = """
open [SERVER]
user [USER] [PASSWORD]
cd {0}
{1}
put {2}
bye
""".format(path, picktype(filename), filename)

print ftpstring

with open("tempfile.txt", 'w') as tempfile:
    tempfile.write(ftpstring)

os.system('ftp -n < tempfile.txt')

print 'Done!'

I know these days you're supposed to use the subprocess module for this kind of junk, but I can't for the life of me figure out how to make a redirection work.

Paul Gowder
  • 2,409
  • 1
  • 21
  • 36
0

I had encountered the same problem likes your's. my python ftp script hangs without any output when i run it in ms cmd console. I even rewrite my script with multithread, multiprocess but it doesn't work. But i found my script works normally in linux, or in pycharm ide on windows.

so i think it maybe some problem with ms cmd environment. and then i found those issue python program stops in command line, command freezing.

I try to disable "QuickEdit Mode" option of cmd. it works!

funway
  • 113
  • 1
  • 7