Connecting to local dockerized Perforce server in Python unittests

Question

I maintain a Python tool that runs automation against a Perforce server. For obvious reasons, parts of my test suite (which are unittest.TestCase classes run with Pytest) require a live server. Until now I've been using a remote testing server, but I'd like to move that into my local environment, and make server initialization part of my pre-test setup.

I'm experimenting with dockerization as a solution, but I get strange connection errors when trying to run Perforce commands against the server in my test code. Here's my test server code (using a custom docker image, Singleton metaclass based on https://stackoverflow.com/a/6798042, and with the P4Python library installed):

class P4TestServer(metaclass=Singleton):
    def __init__(self, conf_file='conf/p4testserver.conf'):
        self.docker_client = docker.from_env()
        self.config = P4TestServerConfig.load_config(conf_file)

        self.server_container = None

        try:
            self.server_container = self.docker_client.containers.get('perforce')
        except docker.errors.NotFound:
            self.server_container = self.docker_client.containers.run(
                'perforce-server',
                detach=True,
                environment={
                    'P4USER': self.config.p4superuser,
                    'P4PORT': self.config.p4port,
                    'P4PASSWD': self.config.p4superpasswd,
                    'NAME': self.config.p4name
                },
                name='perforce',
                ports={
                    '1667/tcp': 1667
                },
                remove=True
            )

        self.p4 = P4()
        self.p4.port = self.config.p4port
        self.p4.user = self.config.p4superuser
        self.p4.password = self.config.p4superpasswd

And here's my test code:

class TestSystemP4TestServer(unittest.TestCase):
    def test_server_connection(self):
        testserver = P4TestServer()
        with testserver.p4.connect():
            info = testserver.p4.run_info()
        self.assertIsNotNone(info)

So this is the part that's getting to me: the first time I run that test (i.e. when it has to start the container), it fails with the following error:

E           P4.P4Exception: [P4#run] Errors during command execution( "p4 info" )
E           
E               [Error]: 'TCP receive failed.\nread: socket: Connection reset by peer'

But on subsequent runs, when the container is already running, it passes. What's frustrating is that I can't otherwise reproduce this error. If I run that test code in any other context, including:

In a Python interpreter
In a debugger stopped just before the testserver.p4.run_info() invokation

The code completes as expected regardless of whether the container was already running.

All I can think at this point is that there's something unique about the pytest environment that's tripping me up, but I'm at a loss for even how to begin diagnosing. Any thoughts?

score 1 · Accepted Answer · answered May 12 '20 at 21:05

1

I had a similar issue recently where I would start postgres container and then immediately run a python script to setup database as per my app requirement.

I had to introduce a sleep command in between the two steps and that resolved the issue.

Ideally you should check if the start sequence of the docker container is done before trying to use it. But for my local development use case, sleep 5 seconds was good enough workaround.

answered May 12 '20 at 21:05

Phani Kandula

387
2
3

I'm astonished, but that worked. So that presumably means container creation doesn't block on completion of the `CMD` command in the Dockerfile, right? Annoying, but okay – Jason Baker May 13 '20 at 15:28
Correct. CMD will run inside the context of container. But the host doesn't see the docker command block on it. It's always the responsibility of the user of the container to ensure the container is in a "ready" state and have corresponding error handling (like sleep and retry later for example). – Phani Kandula May 13 '20 at 20:22

Connecting to local dockerized Perforce server in Python unittests

1 Answers1