28

I am running a simple web server on https://0.0.0.0:4000 (accessible also as https://local.phx-cd.shoepping.at:4000 with mapping to 127.0.0.1 in Ubuntu hosts file) on my WSL2 Ubuntu. I can connect to it from both Ubuntu and Windows host - so far so good. But additionally, in my Docker for Win with WSL2 integration, I run a selenium chrome container which is connecting and testing stuff on that web server (using bridge), but it can't connect to it!

I connected to the container and tried to curl to the web server - connection refused. Since I have dual boot on my computer, I tried to switch to my Linux distro, run web server there and selenium in Linux Docker and connection to the local web server worked. So I think it has something to do with the WSL2.

My docker-compose.yaml (I left out my selenium hub config)

selenium-chrome-local:
      image: selenium/node-chrome-debug:3.141.59
      restart: always
      ports:
        - 5901-5902:5900
      volumes:
        - /dev/shm:/dev/shm
        - ../../temp:/home/seluser/Downloads
      depends_on:
        - selenium-hub-local
      environment:
        - SCREEN_WIDTH=1920
        - SCREEN_HEIGHT=1080
      extra_hosts:
        - "local.phx-cd.shoepping.at:10.99.99.1"
      networks:
        - selgrid
        - dockerhost

 networks:
    selgrid:
    dockerhost:
      driver: bridge
      ipam:
        config:
          - subnet: 10.99.99.0/24

Let me know if you need more config. Thanks.

Dan Macak
  • 16,109
  • 3
  • 26
  • 43
  • I have a few questions: 1) do you have the server in the same compose file? 2) if so, do the containers share the same network? 3) are you sure you didn't make a mistake using https over port 4000? 4) can you connect to the server if you change server IP in hosts of the selenium container to one of Ubuntu? 5) are there any errors in selenium container starting logs? – anemyte Dec 19 '20 at 16:09
  • @anemyte 1) No the webserver runs in nodejs process on Ubuntu. 2) The webserver doesn't share the same network with the selenium container per se, but they are supposed to be "connected" via the bridge, which btw works outside WSL. 3) Absolutely sure. 4) I am already doing it via the extra_hosts in docker-compose.yaml, but I also tried to set the IP to Ubuntu's ifconfig IP and it didn't help. 5) Absolutely no errors. The container works fine and I can test non local servers just fine. – Dan Macak Dec 19 '20 at 18:26
  • I suggest you either use `host` network mode (see 1) https://docs.docker.com/network/host/ 2) https://github.com/compose-spec/compose-spec/blob/master/spec.md#network_mode ) as a stupid but easy workaround or dive into iptables rules and see how container traffic goes. There might be something that drops packets from the container. There is also a third option to use `macvlan` network and put the container on static IP in your network. That'll give you something like a host mode without actually using it. – anemyte Dec 19 '20 at 19:17
  • @anemyte I tried the host network, and while I could connect to other containers (binding to docker host) via 127.0.0.1 (unlike before), I still couldn't connect to my local servers, be it nodejs, python and what not. If I however take the WSL's eth0 IP, I can connect to them alright. Putting that ip into WSL's /etc/hosts and giving it a name local.phx-cd.shoepping.at doesn't allow me however to connect to that server via this name, only the ip works. I didn't find anything unusual in the ip tables, though I could only use netstat, iptables required kernel upgrade in the container. – Dan Macak Dec 23 '20 at 10:42
  • When I installed matrix synapse on ubuntu on WSL2 I refused to give network permissions to the WSL2 ubuntu distro which made me not accessible the port that is running on ubuntu. After changing the permissions it got resolved and I can use the port. try this one. – Sarath Chandra Dec 24 '20 at 16:38
  • @sarathchandra I already tried with firewall off both in public and private networks and it didn't help. – Dan Macak Dec 26 '20 at 06:25

2 Answers2

23

Are you sure that the Ubuntu WSL2 instance is running bridged? By default, WSL2 instances run NAT'd (whereas WSL1 instances ran bridged). So, while yes, the Docker network is bridged, it still can't access the NAT'd WSL2 VM without some extra work.

I'm fairly sure that you are running into the root problem described in WSL issue #4150. If so, here are some things to try ...

Option #1 - Port forwarding to the WSL2 instance

There are several workarounds suggested in that GitHub issue, but the basics that would work for your case boil down to forwarding port 4000 from the Windows host interface to the WSL2 instance's private IP address. In PowerShell:

netsh interface portproxy delete v4tov4 listenport="4000" # Delete any existing port 4000 forwarding
$wslIp=(wsl -d Ubuntu -e sh -c "ip addr show eth0 | grep 'inet\b' | awk '{print $2}' | cut -d/ -f1") # Get the private IP of the WSL2 instance
netsh interface portproxy add v4tov4 listenport="4000" connectaddress="$wslIp" connectport="4000"

Note that you'll need to do this after each reboot, or else set up a script that runs at logon as described in the GitHub issue (see this comment).

Option #2 - WSL1

I would also propose that assuming it fits your workflow and if your web app runs on it, you can simply use WSL1 instead of WSL2. You can try this out by:

  1. Backing up your existing distro (from PowerShell or cmd, use wsl --export <DistroName> <FileName>
  2. Import the backup into a new WSL1 instance with wsl --import <NewDistroName> <InstallLocation> <FileNameOfBackup> --version 1

It's possible to simply change versions in place, but I tend to like to have a backup anyway before doing it, and as long as you are backing up, you may as well leave the original in place.

Possible Option #3 - socat forwarding or tunnel

While I haven't tested your particular use case directly, I have played around with socat in WSL2 with success. From the looks of it socat could be used for port forwarding from WSL2 to (at the least) the Windows host (which would be accessible to the Docker container). See this comment an example on GitHub about a similar use-case as yours.

Possible Option #4 - WSL2 in bridge mode

The GitHub thread referenced above also has some details on how to enable bridge-mode on the WSL2 interface using Hyper-V. I believe this requires Windows 10 Professional or Enterprise. It also has to be done after each reboot, as with Option 1. Again, probably overkill for this case, if port forwarding or WSL1 can accomplish what you need.

Matt Lachman
  • 4,541
  • 3
  • 30
  • 40
NotTheDr01ds
  • 15,620
  • 5
  • 44
  • 70
  • Thank you for your answer, the linked issue sounds pretty similar to mine. However, the posted solution 1) didn't work. I tried variations as v4tov6 but without success. From the github thread, I can see that my WSL is not running in bridge mode, but I don't understand how the suggested workaround might solve my issue, since it is about port forwarding between VM and its host, not VM and the docker container. 2) is not an option 3) I might try it, but I'd prefer not to set up too much. I didn't try 4) yet, and I'll keep it as the last resort for now. – Dan Macak Dec 23 '20 at 14:37
  • Re: port forwarding - Fair enough, but could you just port forward to the Windows host and then point the Selenium Chrome container at the Windows IP? – NotTheDr01ds Dec 24 '20 at 20:34
  • Sounds like it could work, but that IP is going to change since I travel with my laptop, so I would have to retrieve it and then set it before my containers start, or am I missing something? – Dan Macak Dec 25 '20 at 13:27
  • 4
    Would `host.docker.internal` as described in this [answer](https://stackoverflow.com/a/45002996/11810933) be a way around that? – NotTheDr01ds Dec 25 '20 at 20:31
  • 2
    Ok that actually works together with the port forwarding - when I curl https://host.docker.internal:4000, I get response from my local server. However, since the domain name local.phx-cd.shoepping.at is connected to a trusted certificate, I would like to be able to use it instead of the docker internal. I see `host.docker.internal` is generated entry in /etc/hosts, so one way to get this working would be to run a script on Ubuntu's boot to retrieve docker internal IP and enter my server's domain name with this IP there. Or is there a better way? – Dan Macak Dec 25 '20 at 21:23
  • The Docker docs mention that `/etc/hosts` shouldn't be modified manually (although it will probably work for now), but Docker will handle the hosts file itself if you use the `--add-host` parameter to your `docker run` (see [here](https://docs.docker.com/engine/reference/commandline/run/#add-entries-to-container-hosts-file---add-host)). Of course, that still goes back to needing to retrieve the IP before the containers start. – NotTheDr01ds Dec 26 '20 at 07:29
  • Latest thought - Tell me if this will work. Since you already have the port forwarding working from the Windows host to WSL, and you already have your server name mapped to 127.0.0.1 per your original question, then can't you just switch the network to host mode instead of bridge? The container will see 127.0.0.1:4000 (and local.phx-cd.shoepping.at) as the host, which is port forwarding 4000 to WSL. So the end-to-end should be there at that point, right? – NotTheDr01ds Dec 27 '20 at 05:54
  • Noticing that your `docker-compose.yaml` has `local.phx-cd.shoepping.at:10.99.99.1` in `extra_hosts`, but you mention in the body that it is mapped to 127.0.0.1. Probably best to map it to 127.0.0.1 there in the compose file. – NotTheDr01ds Dec 27 '20 at 05:58
  • I had it mapped to 10.99.99.1 since this is the gateway to the docker host's network - Ubuntu's. If I had it mapped to 127.0.0.1, it would resolve to container's local network, which is not aware of the docker host's local server in the bridge mode. I also tried the port forwarding with extra_hosts resolving to 127.0.0.1 to no avail :( - still can't curl my web server. I guess best solution will be to run my web server in a container which is in the same network as selenium or play with the `/etc/hosts` as mentioned above. Thank you for your help anyway! – Dan Macak Dec 27 '20 at 07:29
  • I stuck with this issue for `wsl2` with `Debain`, however, my coworker said they use `Ubuntu` and don't have this issue. Anyway, `solution #1` works for me, and if it doesn't check you're using the right os `distribution` or `port` in the script – cwhsu Jun 13 '22 at 05:04
6

Run this command on PowerShell as Administrator:

Replace {#requiredWindowsPort} with the port that will be used in the browser

Replace {#requiredWSL2Port} with the port running in WSL2 you want to connect to.

netsh interface portproxy add v4tov4 listenport={#requiredWindowsPort}
listenaddress=0.0.0.0 connectport={#requiredWSL2Port}
connectaddress=$($(wsl hostname -I).Trim());
chrki
  • 6,143
  • 6
  • 35
  • 55
tlejmi
  • 526
  • 8
  • 13