Mocking builtin function open inside an imported module

Question

I want to test that the ftp.storbinary() function is called with the right arguments. This however is part of a different module from the one were the test live.

In the io.py module I have this method (which is part of a class):

def to_ftp(self, output_path, file_name, host, username, password):
    """Upload file to FTP server."""

    ftp = ftplib.FTP(host)
    ftp.login(username, password)
    full_path = os.path.join(output_path, file_name)
    with open(full_path, "r") as ftp_file:
        ftp.storbinary(" ".join(["STOR", file_name]), ftp_file.read)

I've create a test_io.py module where I have a series of unittests. I was thinking to patch both open and ftplib.FTP, because I'm passing to ftp.storbinary() ftp_file.read().

@patch("ppc_model.io.ftplib.FTP")
def test_store_call(self, mock_ftp):
    """The *storbinary* call must use the right arguments."""

    with patch("ppc_model.io.open", mock_open(read_data=None)) as m:
        self.writer.to_ftp(output_path="./output", file_name="output.zip",
                       host="myftp", username="username", password="password")
    mock_ftp.return_value.storbinary.assert_called_once_with(
    "STOR output.zip", m.read())

This however returns an AttributeError because the module doesn't have an attribute open. How can I make sure Mock understand I'm trying to mock the builtin function?

Is there also a better way to test I'm passing the right arguments to ftp.storbinary? I'm kinda new to mocking.

EDIT: I've made some progress. I think the issue was that I was trying to patch the wrong object. With open I think I have to patch builtins.open. A bit counter-intuitive.

@patch("ppc_model.io.ftplib.FTP")
def test_store_call(self, mock_ftp):
    """The *storbinary* call must use the right arguments."""

    with patch("builtins.open", mock_open(read_data=None), create=True) as m:
        self.writer.to_ftp(output_path="./output", file_name="output.zip",
                       host="myftp", username="username", password="password")
    mock_ftp.return_value.storbinary.assert_called_once_with(
    "STOR output.zip", m.read())

Unfortunately now the compiler is complaining about the fact that the Mock object doesn't have a read method.

AttributeError: Mock object has no attribute 'read'

EDIT 2: Following RedCraig's advice I've patched the open method in the local namespace and passed to ftp.storbinary ftp_file instead of ftp_file.read()). So the current unit test is:

@patch("ppc_model.io.ftplib.FTP")
def test_store_call(self, mock_ftp):
    """The *storbinary* call must use the right arguments."""

with patch("{}.open".format(__name__), mock_open(read_data=None),
           create=True) as m:
    self.writer.to_ftp(output_path="./output", file_name="output.zip",
                   host="myftp", username="username", password="password")
mock_ftp.return_value.storbinary.assert_called_once_with(
"STOR output.zip", m)

And the code I'm trying to test:

def to_ftp(self, output_path, file_name, host, username, password):
    """Upload file to FTP server."""

    ftp = ftplib.FTP(host)
    ftp.login(username, password)
    full_path = os.path.join(output_path, file_name)
    with open(full_path, "r") as ftp_file:
        ftp.storbinary(" ".join(["STOR", file_name]), ftp_file)

The current error I get is:

AssertionError: Expected call: storbinary('STOR output.zip', <MagicMock name='open' spec='builtin_function_or_method' id='140210704581632'>)
Actual call: storbinary('STOR output.zip', <_io.TextIOWrapper name='./output/output.zip' mode='r' encoding='UTF-8'>)

If `open` is the python builtin, then does: `with patch("open", ...` work? The way you're testing the args looks fine. — RedCraig, Jun 10 '16 at 16:53
Yes `open` is the python builtin, however `with patch("open", ...` doesn't work. — Gianluca, Jun 15 '16 at 15:10

score 1 · Answer 1 · edited May 23 '17 at 12:32

AssertionError: Expected call: storbinary('STOR output.zip', )

Actual call: storbinary('STOR output.zip', <_io.TextIOWrapper name='./output/output.zip' mode='r' encoding='UTF-8'>)

For the actual call storbinary was given <_io.TextIOWrapper... rather than the mocked object. Seems like the mocking of open isn't working, perhaps your module name is incorrect.

Here are the unittest.mock examples, this page has an example of mocking open:

>>> mock = MagicMock(return_value=sentinel.file_handle)
>>> with patch('builtins.open', mock):
...     handle = open('filename', 'r')
...
>>> mock.assert_called_with('filename', 'r')
>>> assert handle == sentinel.file_handle, "incorrect file handle returned"

I tried mocking open in another module and it works, there's a catch when using it with assert_called_once_with(). When patching open mock uses a MagicMock object, every call to open() returns a new object. So the test case which patches open using a with as m style statement will have a different value for m than the open call from to_ftp ad assert_called_once_with() will fail.

While I'm sure there's a better way with Mock and assert_called_once_with, I resorted to using a StringIO object when mocking open so that each call returns the same StringIO instance:

import ftplib
import os

from io import StringIO
from unittest import TestCase
from unittest.mock import patch


def to_ftp(output_path, file_name, host, username, password):
    """Upload file to FTP server."""
    ftp = ftplib.FTP(host)
    ftp.login(username, password)
    full_path = os.path.join(output_path, file_name)
    with open(full_path, "r") as ftp_file:
        ftp.storbinary(" ".join(["STOR", file_name]), ftp_file)


class TestOpenMock(TestCase):
    @patch("ftplib.FTP")
    def test_store_call(self, mock_ftp):
        """The *storbinary* call must use the right arguments."""
        open_file = StringIO('test_data')
        with patch('%s.open' % __name__, return_value=open_file):
            to_ftp(output_path="./output", file_name="output.zip",
                   host="myftp", username="username", password="password")

        mock_ftp.return_value.storbinary.assert_called_once_with(
            "STOR output.zip", open_file)

This passes. Obviously you'll have to change your '%s.open' % __name__ to your module containing to_ftp.

Original Answer (@any so mod: should I just delete this?):

1) This looks like the answer you're looking for: How do I mock an open used in a with statement (using the Mock framework in Python)? It mocks open using <local module name>.open.

2)

AttributeError: Mock object has no attribute 'read'

It's complaining about the read because you're passing the read method to storbinary:

ftp.storbinary(" ".join(["STOR", file_name]), ftp_file.read)

and your mocked instance of open ftp_file doesn't have that method.

3) You're checking that storbinary was called with m.read() but you're passing in m.read. m.read() invokes the read method whereas m.read is a reference to the method.

mock_ftp.return_value.storbinary.assert_called_once_with("STOR output.zip", m.read())

4) Looking at the docs for ftp.storbinary, it expects the file object itself, not a reference to the file objects read method. It should be:

ftp.storbinary(" ".join(["STOR", file_name]), ftp_file)

Passing in the (mocked) file object will address point 3) above.

I've edited the question to reflect your suggestions. Thanks! — Gianluca, Jun 16 '16 at 15:46

Mocking builtin function open inside an imported module

1 Answers1

Original Answer (@any so mod: should I just delete this?):