mock_s3 decorating pytest fixture

Question

I wonder why mock_s3 decorator doesn't work when used as a decorator for pytest fixture. test_with_fixture fails while it provides the same code as the test_without fixture. Well, "the same" as it is decorated explicitly.

test_with_fixture raises AccessDenied error, but the type of S3 error it not relevant in this case. The problem is that, client.list_objects is not mocked in the test which uses fixture.

pytest - 3.1.2
moto - 1.0.1
boto3 - 1.0.4

import pytest
import boto3

from moto import mock_s3

BUCKET = 'Foo'


@pytest.fixture()
@mock_s3
def moto_boto():
    res = boto3.resource('s3')
    res.create_bucket(Bucket=BUCKET)


def test_with_fixture(moto_boto):
    client = boto3.client('s3')
    client.list_objects(Bucket=BUCKET)


@mock_s3
def test_without_fixture():     
    res = boto3.resource('s3')
    res.create_bucket(Bucket=BUCKET)

    client = boto3.client('s3')
    client.list_objects(Bucket=BUCKET)

score 16 · Answer 1 · edited Dec 15 '19 at 19:27

An alternative is to use an 'autouse' test fixture in which you start and stop the moto server and create your test bucket.

This is based on mikegrima's comment on https://github.com/spulec/moto/issues/620.

import pytest
import boto3

from moto import mock_s3

BUCKET = 'Foo'


@pytest.fixture(autouse=True)
def moto_boto():
    # setup: start moto server and create the bucket
    mocks3 = mock_s3()
    mocks3.start()
    res = boto3.resource('s3')
    res.create_bucket(Bucket=BUCKET)
    yield
    # teardown: stop moto server
    mocks3.stop()


def test_with_fixture():
    client = boto3.client('s3')
    client.list_objects(Bucket=BUCKET)

score 9 · Answer 2 · answered Apr 14 '20 at 13:39

Using a context manager:

import pytest
import boto3

from moto import mock_s3

BUCKET = 'Foo'


@pytest.fixture()
def moto_boto():
    with mock_s3():
        res = boto3.resource('s3')
        res.create_bucket(Bucket=BUCKET)
        yield


def test_with_fixture(moto_boto):
    client = boto3.client('s3')
    client.list_objects(Bucket=BUCKET)

Using the context manager, start and stop are invoked under the hood.

score 8 · Answer 3 · answered Jun 24 '17 at 22:20

The problem of your fixture is that you are not using it later although it is in the signature of your test test_with_fixture(moto_boto). I suggest you to create a fixture that returns a function that can be instantiated within your test to create the mocked objects that your test requires (the s3 bucket). An example of such an implementation could be as follows:

import pytest
import boto3

from moto import mock_s3

BUCKET = 'Foo'

@pytest.fixture()
def moto_boto():
    @mock_s3
    def boto_resource():
        res = boto3.resource('s3')
        res.create_bucket(Bucket=BUCKET)
        return res
    return boto_resource

@mock_s3
def test_with_fixture(moto_boto):
        moto_boto()
        client = boto3.client('s3')
        client.list_objects(Bucket=BUCKET)

In this case I am using the moto library through a decorator in both the fixture and the test but the context manager could be similarly used as explained in the moto README

Then what is the point of even using a fixture? You could just import the moto_boto function and execute it first thing in your test. Fixtures have the advantage of having a setup and a teardown. yield is the right way to go in this case, because your fixture sets up the mock, returns to the test context, and only tears down once your test is over. That is what we wanted here, and you get there with fixture + start() + yield + stop(), or even better just fixture + context manager + yield — Diego Mora Cespedes, Apr 14 '20 at 13:32

mock_s3 decorating pytest fixture

3 Answers3