Why does Proxmox firewall block traffic from allowed IP range?

Question

I use Netbird (a Wireguard-based VPN) to secure our network, including a Proxmox PVE 8/Debian 12 server with a variety of containers on it. I have it setup so that with Proxmox's firewall, SSH is only available on containers if the originating IP is within Netbird's private range (100.64.0.0/10) using a "datacenter" level alias in the PVE web administration tool (dc/netbird).

This works fine with containers, where I've specified that port 22 is allowed if the origin is dc/netbird. Initially, this also worked fine also with the actual Proxmox node, allowing me to use the Netbird private IP of the server to access ports 22 and 8006. However, a failed attempt to add the node to a Proxmox cluster cleared my firewall rules and since then, I've been unable to allow SSH and Proxmox web administration (port 8006) through the firewall to the node. I have those ports in the node level firewall open to dc/netbird just as I do on the container level, but unlike the container level, as soon as I turn on pve-firewall, I find myself shut out of those services.

Here's the /etc/pve/nodes/[hostname]/host.fw:

[OPTIONS]
enable: 1
[RULES]
IN ACCEPT -p tcp -dport 8006 -log nolog # Proxmox GUI
IN ACCEPT -source +dc/management -p tcp -dport 10000 -log nolog
IN SSH(ACCEPT) -source dc/netbird -log nolog
IN DNS(DROP) -log nolog
|GROUP gluster -i bond0
|IN HTTPS(ACCEPT) -log nolog
|IN HTTP(ACCEPT) -log nolog

And here's the DC level config:

[OPTIONS]
enable: 1
[ALIASES]
netbird 100.64.0.0/10
[IPSET management]
dc/netbird
[RULES]
OUT ACCEPT -source dc/netbird -log nolog
IN ACCEPT -source dc/netbird -log nolog
[group hosting]
IN HTTP(ACCEPT) -log nolog
IN SSH(ACCEPT) -source +dc/management -log nolog
IN ACCEPT -p tcp -dport 20000 -log nolog # Usermin
IN ACCEPT -p tcp -dport 10000 -log nolog # Webmin
IN POP3(ACCEPT) -log nolog
IN IMAPS(ACCEPT) -log nolog
IN IMAP(ACCEPT) -log nolog
IN Mail(ACCEPT) -log nolog
IN HTTPS(ACCEPT) -log nolog
IN FTP(ACCEPT) -log nolog
IN DNS(ACCEPT) -log nolog

I've tried everything I can think of to resolve this, but can't seem to find the problem. As soon as I use IPMI to get into my server and turn off pve-firewall, everything works again, but obviously that's not a good idea!

score 1 · Answer 1 · edited Sep 04 '24 at 04:27

The problem turned out not to be Proxmox, but a quirk in how Netbird functions on different platforms. Netbird assigns a subdomain (by default hostname.netbird.selfhosted) to each client so that one can access the network without using IP addresses.

On my Linux systems, Netbird's DNS resolver seems to take priority over public DNS, but on macOS, the system was favoring public DNS over Netbird's internal resolver.

I had given Netbird a real domain (thus hostname.myrealdomain) to do internal resolution off of so I could obtain a wildcard SSL certificate. That works fine generally, but I accidentally included the Proxmox system's subdomain in the public name server's record for myrealdomain. This caused macOS to resolve to the public IP rather than using the VPN, thus failing to trigger the Proxmox firewall rule that was only for internal VPN traffic.

Why does Proxmox firewall block traffic from allowed IP range?

1 Answers1