Symlink to iptables command defaulting to xtables-legacy-multi when in audo mode.

[wpbryan]

Using RKE2 calico version v3.27.2
OS is RHEL8.9

when using auto mode, we noticed that some of our nodes reported using
iptables -V
iptables v1.8.6 (legacy)

while others reported
iptables v1.8.6 (nf_tables)

Despite being on RHEL8.

We set FELIX_IPTABLESBACKEND=NFT on the nodes and redeployed.
Now ALL of the node are reporting
iptables v1.8.6 (legacy)

We looked at the soft-linkedin /usr/sbin. and noticed that iptables was pointing at:
iptables -> xtables-legacy-multi
However in the logs , we can see that while calico sees both iptables-save and iptables-nft-save, it says it is using iptables-nft-save. (we think)
--LOG entry---
Looked up iptables command backendMode="nft" candidates=[]string{"iptables-nft-restore", "iptables-restore"} command="iptables-nft-restore" ipVersion=0x4 saveOrRestore="restore"
--LOG entry---

We were wondering if when the FELIX_IPTABLESBACKEND to NFT,
it changes how calico is accessing commands,
so using iptables -V isn't a valid way to confirm which version of iptables ( legacy or nftables) it using.

Expected Behavior

when the FELIX_IPTABLESBACKEND is manually set to NFT, the iptables command is soft-linked to xtables-nft-multi
and/or we get confirmation that when the flag it set, calico ignores the iptables and uses the correct binaries ( i.e. xtables-nft-multi).

Current Behavior

when the FELIX_IPTABLESBACKEND is manually set to NFT, the iptables command is soft-linked to xtables-legacy-multi

Your Environment

• Calico version: v3.27.2 rancher/hardened-calico:v3.27.2-build20240308 rke2-canal.
• Orchestrator version: RKE2 1.27.12 ( but also seen on 1.26 and 1.25, which have calico 3.25.0).
  OS and Version: RHEL 8 kernel 4.18.0-513.24.1.el8_9.x86_64

[caseydavenport]

Calico doesn't use the iptables binaries on the host - it packages its own tools into the node container, so symlinks on the host won't impact which version Calico is using.

[wpbryan]

they symlink I was discussing , is in the host container.
is there a way to determine which tools calico is using, when the flag is set to Auto vs NFT?

[caseydavenport]

This code might be relevant: https://github.com/caseydavenport/calico/blob/7c23a599dd5d70ff6d3ed633c84ad2a8a6eb33da/felix/environment/feature_detect_linux.go#L376-L384

Looks like we emit a debug log when detecting which backend to use (so won't be visible at default log levels).