60

For an application I'm testing I'd like to create an autouse=True fixture which monkeypatches smtplib.SMTP.connect to fail tests if they try to send an email unexpectedly.

However, in cases where I do expect tests to send emails, I want to use a different fixture logging those emails instead (most likely by using the smtpserver fixture from pytest-localserver and monkeypatching the connect method to use the host/port returned by that fixture)

Of course that can only work if the autouse fixture is executed before the other fixture (loaded as funcarg). Is there any specific order in which fixtures are executed and/or is there a way to guarantee the execution order?

ThiefMaster
  • 310,957
  • 84
  • 592
  • 636
  • 4
    A very valid question, i've seen fixtures been abused many time, and one of the most problematic thing is which fixture run before which one – Fruch Dec 10 '14 at 18:41
  • 1
    An alternative trick that is useful to know is that fixtures can inspect test functions for pytest marks. This means you can `@mark` the tests that need to do something special, and then, using the request object, inspect if the test function has the mark. If it does, perform a different action in your fixture. – user2859458 Mar 24 '17 at 22:55
  • 2
    It would also be nice to know the teardown order – 4xy Dec 09 '21 at 16:31

5 Answers5

58

The easiest way to control the order in which fixtures are executed, is to just request the previous fixture in the later fixture. So to make sure b runs before a:

@pytest.fixture(autouse=True, scope="function")
def b():
    pass

@pytest.fixture(scope="function")
def a(b):
    pass

For details on the general fixture resolution order, see Maxim's excellent answer below or have a look at the documentation.

Chronial
  • 66,706
  • 14
  • 93
  • 99
  • Doesn't seem to work on pytest 3.0.2 (no autouse). Trying to use capfd with another fixture that creates a class which saves sys.stdout to a variable: avoiding the fixture captures the output, trying to force capfd to setup first like this does not. – Sam Brightman Sep 17 '16 at 23:23
  • What do you mean about pytest 3.0.2 and "no autouse"? This code works fine for me in pytest 3.0.2 and 3.0.3 (the latest). – Barry Nov 07 '16 at 20:24
  • Most of the time that I see this order failing is because one of my fixtures does not complete due to an error, look closely for setup errors – Manel Clos May 21 '20 at 16:22
24

TL;DR

There are 3 aspects being considered together when building fixture evaluation order, aspects themselves are placed in order of priority:

  • Fixture dependencies - fixtures are evaluated from rootest required back to one required in test function.
  • Fixture scope - fixtures are evaluated from session through module to function scoped. On the same scope autouse fixtures are evaluated before non-autouse ones.
  • Fixture position in test function arguments - fixtures are evaluated in order of presence in test function arguments, from leftest to rightest.

Official explanation with code example by link below

https://docs.pytest.org/en/stable/fixture.html#fixture-instantiation-order

UPDATE

Current documentation mention 3 factors, as only considered, while discouraging from relying on another factors (like order in test function arguments):

  1. scope
  2. dependencies
  3. autouse

Link above is outdated. See updated link

Maxim
  • 502
  • 1
  • 4
  • 13
  • 4
    The third point "order of presence . . . from leftest to rightest" is not corroborated by the pytest docs. From https://docs.pytest.org/en/stable/fixture.html#fixture-order: ". . . the order fixtures are requested in have no bearing on execution order beyond coincidence" – convoliution Jan 27 '21 at 18:42
  • 3
    [The link mentions only 3 cases](https://docs.pytest.org/en/stable/reference/fixtures.html#fixture-instantiation-order): 1- scope 2- dependencies 3- autouse Note: "Fixture position in test function arguments" is absent from the list. Do not rely on it. – jfs May 08 '22 at 03:59
  • 1
    Thank you guys for pointing out, I've updated the answer. – Maxim May 30 '22 at 08:51
  • I'm trying to determine if this positional argument left to right order can always be equivalently represented with fixture dependencies (the docs section fixtures-of-the-same-order-execute-based-on-dependencies) I'm trying to think if there's ever a situation where a test case needs the order one way while another test case needs the order different from the first. We can always define fixture dependencies however we wish, even if the fixture simply goes unused in the parent fixture's body and there for order. Can we imagine a scenario where 2 test cases conflict in their ordering needs? – jxramos Jun 09 '23 at 18:54
  • Actually maybe my claim was too broad, we can define fixture dependencies any which way except maybe circularly? I never actually tried that before. – jxramos Jun 09 '23 at 18:55
12

I was just having this problem with two function-scoped autouse fixtures. I wanted fixture b to run before fixture a, but every time, a ran first. I figured maybe it was alphabetical order, so I renamed a to c, and now b runs first. Pytest doesn't seem to have this documented. It was just a lucky guess. :-)

That's for autouse fixtures. Considering broader scopes (eg. module, session), a fixture is executed when pytest encounters a test that needs it. So if there are two tests, and the first test uses a session-scoped fixture named sb and not the one named sa, then sb will get executed first. When the next test runs, it will kick off sa, assuming it requires sa.

ebeezer
  • 430
  • 6
  • 12
  • 1
    Can confirm for py.test version 2.6.4. – Nitrodist Apr 23 '15 at 18:13
  • I was using 2.6.4 as well. – ebeezer Apr 24 '15 at 19:15
  • This answer is best deleted IMO, as it makes people "save time" by using the "feature" of alphabetical ordering. This is REALLY bad – Gulzar Feb 01 '22 at 14:35
  • 3
    This answer helped me to understand that I cannot properly enforce some particular order of fixtures. And I don't want using alphabetical ordering, so I just tried to rewrote my fixtures in a way they don't depend on ordering. TL;DR still find this helpful :) – ololobus Apr 19 '22 at 10:16
  • Pytest docs says that such behaviour "have no bearing on execution order beyond coincidence. While pytest will try to make sure coincidences like these stay consistent from run to run, it’s not something that should be depended on." – Maxim May 30 '22 at 08:55
2

IIRC you can rely on higher scoped fixtures to be executed first. So if you created a session scoped autouse fixture to monkeypatch smtplib.SMTP.connect then you could create a function-scoped fixture which undoes this monkeypatching for one test, restoring it afterwards. I assume the easiest way to do this is create your own smtpserver fixture which depends on both the disallow_smtp fixture as well as the smtpserver fixture from pytest-localserver and then handles all setup and teardown required to make these two work together.

This is vaguely how pytest-django handles it's database access btw, you could try and look at the code there but it is far from a simple example and has many of it's own weird things.

flub
  • 5,953
  • 27
  • 24
-2

With the help of below code we can easily set execution order of fixtures / functions

e.g:-

execution order first

@pytest.mark.order(1)

execution order second

@pytest.mark.order(2)
sandeep shewale
  • 125
  • 1
  • 5
  • 8
    This solution probably refers to an external pytest plugin called pytest-ordering, not pytest itself. – gbonetti Sep 06 '19 at 14:10
  • I've seen this employed for test cases / test functions, is the plugin also able to govern plugins as well? – jxramos Jun 09 '23 at 18:18