1

We are having an issue with Alma Linux 9.4 that we do not have with any other version of linux we use (most of them) including redhat and CentOS. When we put multiple NIC cards in the box and set a network up on each one we have the default route set out the primary NIC and local subnets on the other NICs (one each). We set it up with NMTUI and and it all looks good. We set it up the same way we have on dozens of other servers (different IPs here, of course):

10.1.12.157/24 on NIC #1 (primary) with default gateway set to 10.1.12.1
192.168.10.99/32 on NIC #2
10.1.13.19/32 on NIC #3

When we ping from the CLI outward (or telnet to test all 7 layers of the OSI model) it works and routes out the appropriate NIC. We can ping 10.1.12.1 on NIC1, we can ping 192.168.10.129 on NIC2 and we can ping 10.1.13.1 on NIC3...We can also telnet.

Of course we have restarted the NetworkManager, and down/up each interface, etc.

The problem is, if we go to a box outside of this box on the internal network we can only:

  • ping 10.1.12.157 from anywhere
  • ping 10.1.13.19 from a box on
  • 10.1.13.0/24 ping 192.168.10.99 from a box on 192.168.10.0/24

In NMTUI we have, in the past set a gateway on each NIC to be the default gateway (and added a route /24 on each NIC routing that' NIC's IP range to it's own interface, e.g., 192.168.10.0/24 -> 192.168.10.99) and it has worked.

But under alma linux it tries to add 3 "default gateway routes", one one each NIC and it breaks routing and the box becomes unreachable. We we have not added default routes on each NIC because doing so on Alma Linux adds those to the routing table as default routes. So here is what our routing table looks like now:

# ip route show
default via 10.1.12.1 dev ens224 proto static metric 105 
10.1.12.0/24 dev ens224 proto kernel scope link src 10.1.12.157 metric 105 
10.1.13.0/24 via 10.1.13.19 dev ens256 proto static metric 103 
10.1.13.19 dev ens256 proto static scope link metric 103 
192.168.10.0/24 via 192.168.10.99 dev ens192 proto static metric 102 
192.168.10.99 dev ens192 proto static scope link metric 102

Each network is a VLAN and all of these are plugged into the same CISCO core switch.

I have even tried turning on ip4 ip forwarding in the kernel and it has not helped.

/etc/sysctl.d/99-ipforward.conf contains "net.ipv4.ip_forward = 1" and run "sysctl -p" to activate.

I have also set the default route to 10.1.12.1 in /etc/sysconfigs/network just to be sure.

I am clearly missing something simple, but I just can't figure it out. If anybody knows, please let me know.

Thank you! David

TekOps
  • 93
  • 6

1 Answers1

1

Some distributions come with "reverse path filtering" enabled by default. In strict mode (the rp_filter sysctl set to 1) it discards all inbound packets whose sources aren't reachable through the same interface, which is not suitable for multi-homed systems or any asymmetric routing situation.

Setting the sysctl to 2 switches it to "loose" mode which allows packets if their source is reachable through any interface. This is equally useless, but in the opposite way, as it will allow literally everything as long as the host has a default route. Setting the sysctl to 0 disables rp_filter completely.

According to the RedHat article, the IPReversePathFilter counter in nstat or netstat -s can be used to detect packets being dropped by rp_filter.

You really don't need a default gateway for each interface (unless you are using ip rule policy routing) as only the one with lowest metric will be used for everything.

grawity
  • 501,077