Django: How to allow a Suspicious File Operation / copy a file

Question

I want to do a SuspiciousFileOperation which django disallows by default.

I am writing a command (to run via manage.py importfiles) to import a given directory structure on the real file system in my self written filestorage in Django.

I think, this is my relevant code:

def _handle_directory(self, directory_path, directory):
    for root, subFolders, files in os.walk(directory_path):
        for filename in files:
            self.cnt_files += 1
            new_file = File(directory=directory, filename=filename, file=os.path.join(root, filename),
                 uploader=self.uploader)
            new_file.save()

The backtrace is:

Traceback (most recent call last):
  File ".\manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\Python27\lib\site-packages\django\core\management\base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "C:\Python27\lib\site-packages\django\core\management\base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "D:\Development\github\Palco\engine\filestorage\management\commands\importfiles.py", line 53, in handle
    self._handle_directory(args[0], root)
  File "D:\Development\github\Palco\engine\filestorage\management\commands\importfiles.py", line 63, in _handle_directory
    new_file.save()
  File "D:\Development\github\Palco\engine\filestorage\models.py", line 157, in save
    self.sha512 = hashlib.sha512(self.file.read()).hexdigest()
  File "C:\Python27\lib\site-packages\django\core\files\utils.py", line 16, in <lambda>
    read = property(lambda self: self.file.read)
  File "C:\Python27\lib\site-packages\django\db\models\fields\files.py", line 46, in _get_file
    self._file = self.storage.open(self.name, 'rb')
  File "C:\Python27\lib\site-packages\django\core\files\storage.py", line 33, in open
    return self._open(name, mode)
  File "C:\Python27\lib\site-packages\django\core\files\storage.py", line 160, in _open
    return File(open(self.path(name), mode))
  File "C:\Python27\lib\site-packages\django\core\files\storage.py", line 261, in path
    raise SuspiciousFileOperation("Attempted access to '%s' denied." % name)
django.core.exceptions.SuspiciousFileOperation: Attempted access to 'D:\Temp\importme\readme.html' denied.

The full model can be found at GitHub. The full command is currently on gist.github.com available.

If you do not want to check the model: the attribute file of my File class is a FileField.

I assume, this problem happens, because I am just "linking" to the file found. But I need to copy it, huh? How can I copy the file into the file?

What's the relevant place in Django that causes problem? You've only included part in your code and I am pretty sure you don't raise SuspiciousOperation by yourself :) ? — pajton, Feb 25 '14 at 16:12
I created a follow-up question: http://stackoverflow.com/questions/22038762/django-sqlite-encoding-of-filenames — tjati, Feb 26 '14 at 10:42

score 8 · Answer 1 · answered Aug 08 '18 at 10:08

In Django, SuspiciousFileOperation can be avoid by read the file from external dir and make a tmp file within the project media then save in the appropriate file filed as below

import tempfile

file_name="file_name.pdf"
EXT_FILE_PATH = "/home/somepath/"
file_path = EXT_FILE_PATH + file_name
if exists(file_path):
    #create a named temporary file within the project base , here in media

    lf = tempfile.NamedTemporaryFile(dir='media')
    f = open(file_path, 'rb')
    lf.write(f.read())
    #doc object with file FileField.

    doc.file.save(file_name, File(lf), save=True)
    lf.close()

score 7 · Answer 2 · answered Aug 29 '19 at 13:12

I haven't faced similar problem but related issue. I have recently upgraded Django 1.8 to 1.11.

Now I am getting the following error if try to save a file in a model having FileField field:

SuspiciousFileOperation at /api/send_report/ The joined path (/vagrant/tmp/test_file.pdf) is located outside of the base path component (/vagrant/media)

My model where I want to save the file:

class Report(BaseModel):
    file = models.FileField(max_length=200, upload_to=os.path.join(settings.REPORTS_URL, '%Y/week_%W/'))
    type = models.CharField(max_length=20, verbose_name='Type', blank=False, default='', db_index=True)

I am trying following codes to save the file from tmp folder which is not located in MEDIA_ROOT:

from django.core.files import File

filepath = "/vagrant/tmp/test_file.pdf"
file = File(open(filepath, "rb"))
report_type = "My_report_type"
report = Report.objects.create(
    file=file,
    type=report_type,
)

What I have done to solve the issue:

import os
from django.core.files import File

filepath = "/vagrant/tmp/test_file.pdf"
file = File(open(filepath, "rb"))
file_name = os.path.basename(file.name)
report_type = "My_report_type"
report = Report.objects.create(
    type=report_type,
)
report.file.save(file_name, file, save=True)

Hope it will help someone.

score 5 · Accepted Answer · edited Apr 18 '19 at 18:46

5

Analyzing this part of stacktrace:

File "C:\Python27\lib\site-packages\django\core\files\storage.py", line 261, in path
    raise SuspiciousFileOperation("Attempted access to '%s' denied." % name)

leads to the standard Django FileSystemStorage. It expects files to be within your MEDIA_ROOT. Your files can be anywhere in the file system, therefore this problem occurs.

You should pass file-like object instead of a path to your File model. The easiest way to achieve that would be to use Django File class, which is a wrapper around python file-like objects. See File object documentation for more details.

Update:

Ok, I am suggesting here a route taken from the docs:

from django.core.files import File as FileWrapper

def _handle_directory(self, directory_path, directory):
    for root, subFolders, files in os.walk(directory_path):
        for filename in files:
            self.cnt_files += 1
            new_file = File(
                 directory=directory, filename=filename,
                 file=os.path.join(root, filename),
                 uploader=self.uploader)
            with open(os.path.join(root, filename), 'r') as f:
                file_wrapper = FileWrapper(f)
                new_file = File(
                    directory=directory, filename=filename,
                    file=file_wrapper,
                    uploader=self.uploader)
                new_file.save()

If it works it should copy the file to the location provided by your secure_storage callable.

edited Apr 18 '19 at 18:46

hobs

18,473
10
83
106

answered Feb 25 '14 at 16:34

pajton

15,828
8
54
65

Okay, I understand the first paragraph. But I don't understand how to call the `File` model of Django and how the copy-operation works. – tjati Feb 25 '14 at 16:43
1

Updated to include more details on that part. Let me know if this helped. – pajton Feb 25 '14 at 17:01
Well, thank you very much! It helps and copying the files does work! Nevertheless, I have a new problem :( It's this exception: `django.db.utils.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str ). It is highly recommended that you instead just switch your application to Unicode strings.` - I updated my question and would appriciate it, if you could look into it. – tjati Feb 26 '14 at 08:28
Glad it worked! This is very separate issue and you should really post new question about it. Can you provide the SQL that fails? – pajton Feb 26 '14 at 09:27
Thank you. You're right, this is a new question. After posting I thought the same. So I revert my last update at this question and mark your answer as correct, as it is. I reposted my new question: http://stackoverflow.com/questions/22038762/django-sqlite-encoding-of-filenames - maybe you will look in it. – tjati Feb 26 '14 at 10:37
I didn't understand, why two times assigning new_file in your code? – user1012513 Aug 29 '19 at 11:32

score -1 · Answer 4 · answered Feb 25 '14 at 16:24

-1

Check if there is slash before your filepath

file_item = models.FileField(upload_to=content_file_name)


def content_file_name(username, filename):
    return '/'.join(['content', username, filename])

Note here "content" not "/content". That was the problem for me.

answered Feb 25 '14 at 16:24

Laxmikant Ratnaparkhi

4,745
5
33
49

Nope, this is not the solution :( I have problems to *read* the file I want to import. – tjati Feb 25 '14 at 16:31
@tjati - I had similar error when there was a '/' before the file path and I fixed it by removing that (`/`), thought it would help you... – Laxmikant Ratnaparkhi Oct 09 '17 at 09:29

Django: How to allow a Suspicious File Operation / copy a file

4 Answers4

Linked