Flask and selenium-hub are not communicating when dockerised

Question

I am having issues with getting data back from a docker-selenium container, via a Flask application (also dockerized).

When I have the Flask application running in one container, I get the following error on http://localhost:5000, which goes to the selenium driver using a Remote driver that is running on http://localhost:4444/wd/hub

The error that is generated is: urllib.error.URLError: <urlopen error [Errno 99] Cannot assign requested address>

I have created a github repo with my code to test, see here.

My docker-compose file below seems ok:

version: '3.5'
services:
  web:
    volumes:
      - ./app:/app
    ports:
      - "5000:80"
    environment:
      - FLASK_APP=main.py
      - FLASK_DEBUG=1
      - 'RUN=flask run --host=0.0.0.0 --port=80'
    command: flask run --host=0.0.0.0 --port=80
    # Infinite loop, to keep it alive, for debugging
    # command: bash -c "while true; do echo 'sleeping...' && sleep 10; done"

  selenium-hub:
    image: selenium/hub:3.141
    container_name: selenium-hub
    ports:
      - 4444:4444

  chrome:
    shm_size: 2g
    volumes:
      - /dev/shm:/dev/shm
    image: selenium/node-chrome:3.141
#    image: selenium/standalone-chrome:3.141.59-copernicium
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444

What is strange is that when I run the Flask application in Pycharm, and the selenium grid is up in docker, I am able to get the data back through http://localhost:5000. The issue is only happening when the Flask app is running inside docker.

Thanks for the help in advance, let me know if you require further information.

Edit

So I amended my docker-compose.yml file to include a network (updated the code in github. As I've had the Flask app code running in debug and in a volume, any update to the code results in a refresh of the debugger.

I ran docker network inspect on the created network, and found the local docker IP address of selenium-hub. I updated the app/utils.py code, in get_driver() to use the IP address in command_executor rather than localhost. Saving, and re-running from my browser results in a successful return of data.

But I don't understand why http://localhost:4444/wd/hub would not work, the docker containers should see each other in the network as localhost, right?

score 2 · Accepted Answer · answered Dec 10 '18 at 20:09

the docker containers should see each other in the network as localhost, right?

No, this is only true when they use the host networking and expose ports through the host.

When you have services interacting with each other in docker-compose (or stack) the services should refer to each other by the service name. E.g. you would reach the hub container at http://selenium-hub:4444/wd/hub. Your Flask application could be reached by another container on the same network at http://web

You may be confused if your default when running docker normally is to use host networking because on the host network selenium-hub is also exposed on the same port 4444. So, if you started a container with host networking it could use http://localhost:4444 just fine there as well.

I changed the `webdriver.Remote{}` `command_executor` to use `http://selenium-hub:4444/wd/hub` and that works perfectly! No need to use the IP address. Thank you very much! — soapmctavish, Dec 11 '18 at 10:22

score 0 · Answer 2 · answered Dec 10 '18 at 17:51

0

Could potentially be a port in use issue related to the execution?

See: Python urllib2: Cannot assign requested address

answered Dec 10 '18 at 17:51

pypalms

461
4
12

I had look to see if port 5000 was being used, but it's not (unfortunately). I'm beginning to think this is to do with container-container communication, i.e. the Flask container can't connect to the selenium-hub using localhost from the webdriver - see "app/utils.py" in my github repo – soapmctavish Dec 10 '18 at 18:00

Flask and selenium-hub are not communicating when dockerised

Edit

2 Answers2