4

I'm sending the file file.txt from my Rails controller using send_file, and then delete the folder containing it.

send_file("#{Rails.root}/public/folder/file.txt")
FileUtils.remove_dir "#{Rails.root}/public/folder", true

When I did this, file.txt was sent and deleted. However, folder was not deleted. But if I remove the send_file line, then folder will be deleted.

How do I make it delete folder?

EDIT: Interestingly, I found that inside folder there is a hidden file called .__afs2B0C, probably preventing the deletion. I have no idea how this file is created! The file stays for only around 15 minutes before disappearing.

EDIT2: I've tried inspecting the content of the temp file with vi, but it's unreadable gibberish. When I removed only the send_file line, the folder was correctly deleted. When I removed only the FileUtils.remove_dir line, the folder contains no temp file.

JJ Beck
  • 5,193
  • 7
  • 32
  • 36
  • If you do not have a real reason to use this setup (e.g. you want the folder to remain on error), you might consider using `TempFile` instead: those files are cleaned automatically after the request. – nathanvda Nov 16 '12 at 09:25
  • @nathanvda Thanks for your answer, but I don't quite get it. Where specifically in my code should I be using `Tempfile`? – JJ Beck Nov 16 '12 at 15:23
  • I am assuming you are creating files, since you delete them. Maybe that was a wrong assumption. – nathanvda Nov 16 '12 at 15:53
  • @nathanvda I see. I download some files using `wget` and then zip them, so it's probably too complicated to use `Tempfile`. – JJ Beck Nov 16 '12 at 16:03
  • Imho this is a perfect fit to use `TempFile`, but compressing a complete folder is maybe easier. – nathanvda Nov 16 '12 at 18:25
  • @nathanvda `TempFile` would **not** help, the `TempFile` object may still be garbage collected (and an attempt to remove it from disk be made) before the file has been sent, an explicit `remove_dir` is just as good. – vladr Nov 23 '12 at 02:58
  • @vladr imho no ruby object can be garbage collected while it is in use, so why would the `TempFile` be garbage collected prematurely? It is still in use while doing the send-file, the request will take as long as the send-file takes. – nathanvda Nov 23 '12 at 09:18
  • @nathanvda how would ruby know the object is still in use? The way this works is that `send_file` issues a response header `X-SendFile` with the path to the file, and sails through *immediately* completing the request as far as it is concerned (hello, garbage collector!) The reverse proxy intercepts the `X-SendFile` header and serves the file instead, while ruby treats the next request (this is the entire point of `X-SendFile`, offload file serving to the leaner-and-meaner nginx/mod_xsendfile/etc.) So mixing `X-SendFile` and temporary files are a sure way of shooting oneself in the foot. – vladr Nov 23 '12 at 09:28
  • @nathanvda [`X-SendFile` and temporary files: **BAD IDEA**](http://stackoverflow.com/questions/6043313/rails-x-sendfile-temporary-files) – vladr Nov 23 '12 at 09:35

5 Answers5

7

Are you sure the send_file is not still sending the file when you are removing the dir, it may be asynchronous if it uses X-SendFile? That would cause an error when trying to remove the dir.

So you should probably be queuing this delete action, or doing it with a sweeper later, rather than trying to do it straight after sending the file to streaming.

I'm not completely clear on which file you are sending, so it would be useful to include an actual example of file path, and file type, and how it is created in your question.

Possible help with debugging:

Log in and monitor the folder while you perform the following actions:

  • Write out a very large file (> 60MB say), and check there is no invisible file created during your file creation process - I'm not clear on which file you are actually sending
  • Set up a large file transfer on a slow connection, and watch for the creation and possibly growing of this file (it might be related to compressing the file served on the fly for example).

Given that sendfile may still be sending (for large files) via the web server (x-send-file is now default) when you try to delete, I'd try looking into delayed solutions.

Possible solutions:

  • Use send_data rather than send_file (if files are small)
  • Schedule the deletion of the folder for later with something like delayed_job
  • Set up a sweeper which removes the folders at the end of each day
Kenny Grant
  • 9,360
  • 2
  • 33
  • 47
  • The file was correctly sent and deleted, so I think `send_file` is **not** still sending the file. Even as I put in `sleep(10)` after `send_file`, then same result is observed. – JJ Beck Nov 12 '12 at 22:31
  • hmm, not sure then. Perhaps try taking the true out and see if an error is raised by remove_dir. – Kenny Grant Nov 13 '12 at 00:03
  • If I take the `true` out, it raises the error `Directory not empty` on the directory `public/folder`. What does that indicate? – JJ Beck Nov 13 '12 at 00:08
  • Perhaps try doing an ls -la on the directory straight after the send_file – Kenny Grant Nov 13 '12 at 07:41
  • Actually, I just found that there is indeed a hidden file called `.__afs2B0C`. I have no idea how this file is created! – JJ Beck Nov 14 '12 at 01:30
  • Often upload operations (e.g. rsync ) or zip operations will create a temp file during upload, before relinking the destination, to ensure atomic operations, so it's probably from the upload/file creation. Looks like that is the source of your problem anyway. You could try inspecting the file for clues as to its origin. – Kenny Grant Nov 14 '12 at 08:50
  • Why does the file prevent the deletion, and what can I do to delete it? – JJ Beck Nov 14 '12 at 15:15
  • See above - you can't delete a folder with remove_dir when it has files in it. You should remove the files first (or ideally find out what makes them in the first place). – Kenny Grant Nov 14 '12 at 16:37
  • Hi Kenny, thanks so much for your help so far. I've tried inspecting more -- please see my EDIT2 -- but I still cannot solve the problem. – JJ Beck Nov 16 '12 at 05:11
  • "That would cause an error when trying to remove the dir." On Windows and Mac perhaps, but not on your run-of-the-mill Linux though (the directory/files would be promptly deleted.) @JJBeck, are you on a Mac? – vladr Nov 23 '12 at 02:39
  • @KennyGrant incorrect; `remove_dir` "[r]emoves a directory dir and its contents recursively" (from the manual.) And the source code concurs. The most likely reason for the failure is a race condition, the creation of the `.__afs...` file after `remove_dir` traverses and builds the list of files to delete, but before the `folder` is actually deleted. – vladr Nov 23 '12 at 09:35
  • @vladr, thanks for the correction, because of the error raised I thought remove_dir wouldn't delete recursively, but in the docs it says it does remove recursively. – Kenny Grant Nov 24 '12 at 15:34
  • I found after senf_file delete the send file,in rails 3.2.3 it's work correct but work incorrect in rails 3.2.9,you should delete file after the send_file action done in rails3.2.9. – comme Jan 08 '13 at 06:46
1

Not sure why that hidden file is there, it could be an offshoot of X-send-file or even of wget (partial progress or something).

Ideally, you should use Tempfile to do things like this. The code is based of you're comment about what you are doing. Also, I am using two gems one for downloading and another for zipping. This way, you don't need to make a folder at all, just a zip file directly. All the content files of the zip will be deleted on their own. After downloading the zip just delete it. Here also I should mention that you could run into a glitch somewhere, since the send_file will hand over the transfer to the webserver, and as such you don't the rails process to delete the file while it is still being served. So even with this, and it working well on localhost, I would strongly advise using a custom scheduled background garbage collector in production.

require 'open-uri'
require 'zip/zip'
zip_path = "#{Rails.root}/public/test.zip"
urls_to_fetch = ['abc.com', 'xyz.com']

Zip::ZipFile.open(zip_path, Zip::ZipFile::CREATE) do |zipfile|
    urls_to_fetch.each_with_index do |url, index|
        # intialize new temp file
        file = Tempfile.new(index.to_s)
        # fetch the file using open-uri or wget and save it as a tmpfile
        open(url, 'rb') do |read_file|
            file.write(read_file.read)
        end
    end
    # add the temp file to the list of files to zip
    zipfile.add(File.basename(file), file.path)
end
# send the zipfile for download
send_file zip_path
# delete the zipfile
FileUtils.rm zip_path

However, this should not be mandatory. If you are doing things without Tempfiles, please check the rights that the rails runner has on the target directory.

The FileUtils documentation has details regarding local security vulnerabilities when trying to delete files / folders.

vvohra87
  • 5,594
  • 4
  • 22
  • 34
  • `TempFile` would not help, the `TempFile` object may still be garbage collected (and an attempt to remove it from disk be made) before the file has been sent, an explicit `remove_dir` is just as good. – vladr Nov 23 '12 at 03:07
  • The `TempFile` will be garbage collected only once that method is done executing. Not a problem since the TempFile becomes useless once it's added to the zipfile. `unlink` and `close` are not being called explicitly anywhere in the code. – vvohra87 Nov 24 '12 at 09:15
1

See here... works for me

 file = File.open(Rails.root.join('public', 'uploads', filename), "rb")
contents = file.read
file.close

File.delete(filepath) if File.exist?(filepath)

send_data(contents, :filename => filename)
Community
  • 1
  • 1
Sebastian
  • 11
  • 1
0

maybe you could try this solution: http://info.michael-simons.eu/2008/01/21/using-rubyzip-to-create-zip-files-on-the-fly/

Guilherme
  • 1,126
  • 9
  • 17
-1

it is so simple but dangerous. Use shell command to achieve it . Put it after send_file in Controller 

system ("rm -rf public/folder")
Paritosh Piplewar
  • 7,982
  • 5
  • 26
  • 41
  • 1
    Your "solution" is not only dangerous, but it's actually more complicated than `FileUtils.remove_dir "#{Rails.root}/public/folder", true`. Incidentally, the OP already tried the `rm -rf` approach and that failed as well -- see http://stackoverflow.com/questions/13336209/deleting-folder-containing-file – vladr Nov 23 '12 at 02:30