6

Imagine that I have test/unit/... which are safe to run in parallel and test/functional/... which cannot be run in parallel yet.

Is there an easy way to convince pytest to run the functional ones sequentially? Consider that we are talking about a big number of tests so altering each test function/method would be very noisy.

At this moment we run tests with marker filters so mainly we run them separated. Still, I am looking for a solution for removing the need to run them separated.

sorin
  • 161,544
  • 178
  • 535
  • 806

1 Answers1

8

You can implement your own scheduler that acts like load or loadscope, depending on what module the test is defined in. Example:

from xdist.scheduler.loadscope import LoadScopeScheduling


class MyScheduler(LoadScopeScheduling):
    def _split_scope(self, nodeid):
        if 'test/functional' in nodeid:
            return 'functional-tests'
        return nodeid


def pytest_xdist_make_scheduler(config, log):
    return MyScheduler(config, log)

All tests under test/functional will be grouped under the same node functional-tests (and thus run in the same worker), the rest will be run in parallel as usual.

hoefling
  • 59,418
  • 12
  • 147
  • 194
  • I'm getting an unresolved import error on the LoadScopeScheduling import. Do I need to be importing this in a certain location / is this possible to do when relying on pip or do I need to fork xdist? – Rustang Nov 12 '20 at 23:23
  • @Rustang most probably you haven't install the package correctly. Try running `python -c "import xdist"` first to verify the package is importable at all from your current environment. – hoefling Nov 12 '20 at 23:33
  • Hmm interesting, `python -c "import xdist" Traceback (most recent call last): File "", line 1, in ImportError: No module named xdist`, regardless of pytest-xdist definitely being in my pip list. OK well that's a start - thanks for your help! – Rustang Nov 13 '20 at 00:12
  • 1
    @Rustang the next guess would be that the `pip` command points to the wrong executable. List installed packages via `python -m pip list`, install via `python -m pip install` etc to match. – hoefling Nov 13 '20 at 07:03
  • 1
    This scheduler code goes in a `conftest.py` file in the main test directory, for anyone else who's confused about how to get pytest to use this scheduler. Quite a helpful and extensible answer! – ClimbsRocks Feb 23 '21 at 18:56