0

I know many people has requested similar questions, but none of the solutions I found, could be applied on my case. I am sure the solution should be simple, but I cannot really find it!

Basically I need to access a specific service running on my host machine from a docker container (dmz). I have a pool of docker containers and an " orchestrator" service that runs on the real machine (host). This guy is a python REST service running at port 5001 of the host and it is responsible, for example, to save the execution logs of the running docker machines, among others. I need one specific machine to be able to access the host (the dmz), but not the others! If they need to talk to the host they need to pass through the dmz. 

 ======================================================================
 |1-  HOST  -> 5002:5002   dmz(dockerContainer):5002/service1         |
 |2-  dmz(container)     <--> 9200:9200 elasticsearch(container):9200 |
 |3-  HOST:5002/service2  <- 5001:5001   dmz(dockerContainer)         |     
 ======================================================================

The connections 1 and 2 work. They are rest services and I get even the answer of the rest request. The 3 is what I need to do now, and I don' t mange to! I don' t know what happened, but I swear to god, three weeks ago from the containers I could access the host by its real IP address, and now I cannot anymore! I have no idea what has changed but just before I could and now I cannot anymore.

  • What I have tried so far:
    1. Access the host via its real IP, that worked in the past (swear to god, it did). I get a TimeoutError: [Errno 110] Connection timed out
    2. Access the docker0 172.17.0.1 (https://nickjanetakis.com/blog/docker-tip-65-get-your-docker-hosts-ip-address-from-in-a-container). The same TimeoutError: [Errno 110] Connection timed out. And that is even good, because it would mean that the other containers would also be able to do that, and I don't want that anyone could access the server like that!!
    3. Add the host in the extra-hosts (https://forums.docker.com/t/accessing-host-machine-from-within-docker-container/14248/5) - Does not allow both predefined and user-defined networks to be used at the same time... and I need it.
    4. Port forwarding ( maybe the direction, but I don't see how to do it on the container-> host direction) Add a port for the host machine to the docker. It works on the host-> container direction, adding 5002:5002 on the port of the composer file, but the oposit it is not possible. When I start the network, I cannot put my service up anymore (it is reasonable, since the port is not available). How to use that? I mean on the docker to host sense? How could I point the port forwarding to a service that is up and running?

Any Ideas of how to do that?!?!? 

OS Version/build : Ubuntu 16.04
Docker version : 19.03.2, build 6a30dfc
Docker-compose version : 1.24.0, build 0aa59064

Composer file:

version: '3.5'

services:   
  dmz:
    container_name: testbed_dmz
    # 1 - DMZ image name  
    image: testbed_dmz:latest
    ports:
        # 2 - DMZ ports 
      - "5002:5002"
    networks:
      data_network:
        # 3 - DMZ ip address  
        ipv4_address: 192.168.7.2   
  elasticsearch:
    container_name: data_server
    # 4 - Elasticsearch image name  
    image: docker.elastic.co/elasticsearch/elasticsearch:7.0.0
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - discovery.type=single-node
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - "9200:9200"
    networks:
      data_network:
        # 5 elasticsearch ip address
        ipv4_address: 192.168.7.3  
  kibana:
    container_name: testbed_kibana
    # 6 kibana image name
    image: docker.elastic.co/kibana/kibana:7.0.0
    environment:
        # 7 again the elasticsearch ip address
      - SERVER_NAME=192.168.7.3
    ports:
      - "5601:5601" networks:
      data_network:
        # 8 kibana ip address
        ipv4_address: 192.168.7.4   
networks:
    data_network:
        # 9 the name of the network
        name: DMZ_DATA_NET
        driver: bridge
        ipam:
            driver: default
            config:
                # 10 the network address
                - subnet: 192.168.7.0/29
DanielCamara
  • 51
  • 5
  • This is a simple networing problem. At the host, figure out what network your app is listening for connections on: `netstat -tulpn | grep LISTEN`. Then use the same network in the container. – John Hanley Nov 04 '19 at 15:22
  • Hi John, well.. not that simple I am afraid. The app is a python program, I have no access to netstat. In any case I know the networks by calling a _docker container inspect containerId_ And there the only network I see is the DMZ_DATA_NET and the port 5002 on the bridge! – DanielCamara Nov 04 '19 at 16:50
  • If you want to connect from your app to a program running on your host, you need to know the address and port of that program. First, figure that out. Then figure out the network required to access that program on that network. – John Hanley Nov 04 '19 at 17:05
  • Hi John, the port on the host is the 5001, that much I know. What I don't understand is that on the container, the rest receives the requests through the 192.168.7.1, the default gateway, and it answers without a problem. BUT if I try to go for the 192.168.7.3:5001, I get a timeout. The thing that I don't understand is how to explain to the composer that I have to access the port 5001 outside, in an existing service! If I declare the port, obviously it is in use if I have a service there, so the network does not goes UP! – DanielCamara Nov 05 '19 at 08:05
  • You are mixing terms so I do not understand exactly what you are solving. In Docker, a "service" does not run on the host. Do you mean a Linux service/program on the host OS or a Docker Swarm service? If a Docker Swarm service, use its name or alias. If a Linux program or service use its TCP listening IP address and port. – John Hanley Nov 05 '19 at 14:59
  • OK sorry for not being clear enough! :( What I mean by service in the host is really a program on my **real machine**, the one which hosts a series of containers. I have a program that listens a specific port on the host, and I need it to be accessed by one specific of those docker containers. However, this container MUST also access other containers at the same time. I would need this container to be part of a bridge network (for other containers) AND a host network, to access my service, for instance http://host:5001/service. Neither the host's IP, nor 172.17.0.1, nor 192.168.7.1 work! :( – DanielCamara Nov 05 '19 at 15:36
  • 172.17.0.1 and 192.168.7.1 are gateway addresses, not the address of your service (program) runnin on your host. As I mentioned use `netstat` on your host to see what TCP address and port your program is listening on. Then use those values in your program. – John Hanley Nov 05 '19 at 15:42
  • `netstat -tulpn | grep LISTEN tcp 0 0 0.0.0.0:5001 0.0.0.0:* LISTEN 25597/python3.5 tcp6 0 0 :::5601 :::* LISTEN - tcp6 0 0 :::5002 :::* LISTEN - ` – DanielCamara Nov 05 '19 at 16:28
  • I put the server on 0.0.0.0, by default.  However, I also tried on 172.17.0.1! And it works (it appears on the ifconfig), on the host I manage to access the http://172.17.0.1:5001, but not from the docker container. ( OK sorry, NOT trying to complexity, but...) I have also a service that runs on the docker container but, on port 5002! EVEN if in the python, on the container, I print the source of the REST requests, when coming from the host, the source is 172.17.0.1, even cooler the host, manage to receive the results of the requests rest to the container listening on port 5002! – DanielCamara Nov 05 '19 at 16:38

1 Answers1

0

To connect to the process running on the host try adding network_mode: host to the service definintion:

services:   dmz:
    container_name: testbed_dmz
    network_mode: host

But then dmz would need to connect to python process using localhost:[port_num], assuming it runs on port_num port of the host

rok
  • 9,403
  • 17
  • 70
  • 126
  • Hi rok, thanks for your answer. Good point, I tried that following the indications at [link](https://stackoverflow.com/questions/35960452/docker-compose-running-containers-in-nethost) but it seems you need to chose. Either user defined or alias on the network definition [link](https://stackoverflow.com/questions/47303141/how-to-use-the-host-network-and-any-other-user-defined-network-together-in-dock). – DanielCamara Nov 05 '19 at 07:51
  • I get the following error : > **docker-compose -f main_backbone_compose.yml up** `ERROR: for testbed_dmz network-scoped alias is supported only for containers in user defined networks ERROR: for dmz network-scoped alias is supported only for containers in user defined networks ERROR: Encountered errors while bringing up the project.` – DanielCamara Nov 05 '19 at 07:52