How do I write Python unit tests for scripts in my bin directory

Question

The Python unittest module seems to assume a directory structure for a project in which there's a project root level directory with the source code and tests under that directory.

I would like, however, to write Python scripts in my ~/bin directory and tests for it in another directory (say, ~/dev/tests). Is there a way for me to run the unit tests using the command line interface without setting my PYTHONPATH environment variable and creating __init__.py files and whatnot?

Here's a simple example demonstrating what I want:

~/bin/candy:

#!/usr/bin/env python

def candy():
    return "candy"

if __name__ == '__main__':
    print candy()

~/dev/tests/test_candy.py:

#!/usr/bin/env python

import unittest
import candy

class CandyTestCase(unittest.TestCase):

    def testCandy(self):
        candyOutput = candy.candy()

        assert candyOutput == "candy"

I notice that everything can be done conveniently if:

• The two files are named with py extensions (candy.py and test_candy.py)
• The two files are in the same directory
• The test is run with the following in the directory of the tests: $ python -m unittest test_candy

Can I run python with the unittest module to do the following without setting anything in my environment explicitly:

• My file under test does not have the py extension (just ~/candy).
• I don't care if test_candy has py as an extension or not.
• I want candy and test_candy.py to not share a common root (other than my home directory).

If that's not possible with a simple invocation of python -m unittest, what is the most simple way to accomplish this?

Answer 1

This is candy executable (no change):

➜ cat ~/bin/candy

#!/usr/bin/env python    
def candy():
  return "candy"

if __name__ == '__main__':
  print candy()

and this is ~/dev/tests/test_candy.py (changed):

➜ cat ~/dev/tests/test_candy.py

#!/usr/bin/env python

import imp
import unittest

from os.path import expanduser, join

# use expanduser to locate its home dir and join bin and candy module paths
candy_module_path =  join(expanduser("~"), "bin", "candy")

# load the module without .py extension
candy = imp.load_source("candy", candy_module_path)


class CandyTestCase(unittest.TestCase):

    def testCandy(self):
        candyOutput = candy.candy()

        assert candyOutput == "candy"

What changed?

• We added imp.load_source to import ~/bin/candy (a module without *.py extension)

• We added provision to locate home directory mention i.e. ~ using expanduser

• We are using os.path.join to join the paths for ~/bin/candy

Now you can run the tests with discover option of unittest module.

Check python -m unittest --help for more details.

Excerpts below

-s directory Directory to start discovery ('.' default)

-p pattern Pattern to match test files ('test*.py' default)

➜ python -m unittest discover -s ~/bin/ -p 'test*' -v ~/dev/tests
testCandy (test_candy.CandyTestCase) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Answer 2

Since Python 3.3 the imp package has been deprecated and the importlib package replaces it. This answer gives details of how to import a single file.

For your unit test, this would be:

from importlib.machinery import ModuleSpec, SourceFileLoader
from importlib.util import spec_from_loader, module_from_spec
import os.path
import types
import unittest


def import_from_source( name : str, file_path : str ) -> types.ModuleType:
    loader : SourceFileLoader = SourceFileLoader(name, file_path)
    spec : ModuleSpec = spec_from_loader(loader.name, loader)
    module : types.ModuleType = module_from_spec(spec)
    loader.exec_module(module)
    return module

script_path : str = os.path.abspath(
    os.path.join(
        os.path.dirname(os.path.abspath(__file__)), "..", "..", "bin", "candy",
    )
)

candy : types.ModuleType = import_from_source("candy", script_path)


class CandyTestCase(unittest.TestCase):
    def testCandy(self : "CandyTestCase" ) -> None:
        self.assertEqual( candy.candy(), "candy" )


if __name__ == '__main__':
    unittest.main()

Assuming that the file structure is:

base_directory/bin/candy
base_directory/dev/tests/test_candy.py

(Note: this unit test assumes a fixed relative path from the test rather than a fixed absolute path so that you can move the package to another directory and the test will not break so long as the files within the package do not change relative positions.)

Answer 3

I have not tried this with unittest, but my quick fix for problems like these are to just change my working directory inside the script using the os module. This SHOULD work for you though.

#!/usr/bin/env python

import unittest
import os
os.chdir("/usr/bin/candy")
import candy

class CandyTestCase(unittest.TestCase):

    def testCandy(self):
        candyOutput = candy.candy()

        assert candyOutput == "candy"

Answer 4

Short answer, no. The problem is that you need to import the modules you're trying to test, so at least you have to change your PYTHONPATH. My advice is that you change it temporarily as you execute the tests, like this:

import sys
sys.path.extend(['~/dir1','~/dir2','~/anotherdir'])

The solution of @Paritosh_Singh is algo good.

The long run is to install a test runner like tox and configure it so it "sees" your modules. But I think you don't want to do that.