Multicast issue with Kathará External feature

[awaum]

I'm new to Kathara, I read the documentation. However, please apologize if I missed something there.

I have a small lab of two routers, connected through collision domain "B".

From my "lab.conf":

r1[0]="A"
r1[1]="B"
r1[image]="kathara/quagga"

r2[0]="C"
r2[1]="B"
r2[image]="kathara/bird"

Collision domains "A" and "C" connect to a virtual PC each, which are not interesting here.

My "lab.ext":
B end0

end0 connect to a physical shared network which has a hw router running OSPF lets call it "r3".

Network plugin "kathara/katharanp_vde" is used.

r1 and r2 also are running OSPF. They become neighbors.

r3 sees r1's and r2's OSPF-hellos, but gets stuck in "init" state, because its hellos never reach r1 and r2.

This changes when I start a tcpdump on "end0", my physical interface on my host. But when I stop the dump, no more multicast packets are forwarded to collision domain "B" any more.

My questions:
Is this an expected behavior?
How can I fix this, so my virtual and and physical routers can communicate through multicast?

Thank you for help!

[tcaiazzi]

Hi @awaum,

Thanks for opening the issue!

From your explanation, it could be a firewall related issue.
Probably, when you start dumping the host interface, tcpdump made some changes on the interface, so packets start flooding.

Do you have the same problem for all the packets, or only for multicast ones?

Do you have any firewall active on your host (e.g., firewalld, ufw)?

If it is not a firewall, maybe the issue is directly caused by iptables rules. In case, can you please send the output of iptables -nvL?

Thanks for the collaboration.

[awaum]

Ciao @tcaiazzi ,

thank you for your prompt reply!

Well, about the firewall:
Actually docker (or "moby-engine" in my case, as I'm on fedora 39 on a RP4) sets firewall-rules automatically.
I disabled this by adding --iptables=false to /etc/sysconfig/docker.

Now my Kathara-host's iptables looks like this:

# iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

When I capture on r1 or r2 I can see OSPF-Hellos of r1 and r2 (they are adjacent!), but never any Hello from r3.
As stated, r3 however sees r1 and r2's Hellos, but is stuck in "Init" state.

Now it came to my mind... why would my host interface (end0) be interested in a packet
that is for a GDA (there's no OSPF running on that interface...)?

So I told the interface to become member of that OSPF-group:
sudo ip addr add 224.0.0.5/32 dev end0 autojoin
and also
sudo ip addr add 224.0.0.6/32 dev end0 autojoin

Now we got one step further (r1 and r2 see r3's Hellos), DR and BDR get elected, and state goes on to "ExStart".

In the OSPF process r1 and r2 now want to unicast to r3 (which became the DR).
Therefor they are sending an ARP request, which is replied by r3....
...but never hits r1, for example:
? (192.168.88.1) at <incomplete> on eth1

capturing on r1:

listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
02:09:57.314646 9e:25:f7:ab:30:64 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.88.1 tell 192.168.88.101, length 28
02:09:57.570972 ca:7f:10:b7:b1:07 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.88.1 tell 192.168.88.102, length 46
02:09:59.520952 ca:7f:10:b7:b1:07 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.88.1 tell 192.168.88.102, length 46

and so on...

I was mistaken in the beginning. It's not just Multicast which is not bridged into my containers.
It is also Broadcast and (at least because ARP does not work) also Unicast. The full "BUM" ;-)

This will only change from the moment when I bring my host interface into promiscuous mode:
sudo ip link set dev end0 promisc on

From this moment on, the state on r3 changes to "Full" and we are good.

So, is it a requirement to bring the host NIC into promiscuous mode?

Grazie mille!

[tcaiazzi]

Hi @awaum,

Thanks for all your effort for debugging this issue.

So, is it a requirement to bring the host NIC into promiscuous mode?

Normally, it should not be required. But maybe is something related to Fedora configuration, since we already experimented some different behaviors on Fedora with respect to Debian.

Unfortunately, I do not have at my disposal a proper testbed for reproducing the issue easily :(

If you have a Debian based host at your disposal (and enough time :) ), it would be great if you try to see if there promisc on is required (otherwise will find a way to make the test).

If yes, probably is something related to multicast/broadcast packets that we did not consider during our tests (we never tested OSPF with external collision domains, so we could have missed something).

Otherwise, the issue should be related to some Fedora default configurations which differ from Debian ones.

Finally, you could try to switch to the old network plugin (based on Linux Bridges) and repeat the test.
Just to consider all the possibilities and speed up debugging.

To do so, open a terminal and type:

kathara settings

Then select the option Choose Docker Network Plugin version, and then select kathara/katharanp.
Now you can re-run the test. The test will use the old plugin, which manages differently the external feature. In case you perform this test, please let me know if you see any difference.

Many thanks for all your help and your clear explanation, that would be surely useful for other Fedora users in the community.
As we clarify the issue, we will update the documentation to clarify this point.

Ciao!

[awaum]

Ciao @tcaiazzi,

sorry, it took me some time to get my debian based lab running ;-)

My conclusion is,
that kathara/katharanp is working for BUM-traffic originating from an external interface.

Multicast traffic from my physical router hit the virtual ones, ARP replies from the physical reached my virtual routers.
This all worked without bringing the physical interface into promiscuous mode.

kathara/katharanp_vde is not bridging external traffic inbound. Outbound is working properly.
Only when the physical host interface is brought into promiscuous mode, BUM-traffic is bridged into the virtual collision domain.

I could see no difference in this behavior if the base system is fedora 39 or debian 12.5.

[tcaiazzi]

Ciao @awaum,

First of all, thanks for all your effort and your time! I truly appreciate it ❤️

Then, thanks for noticing the difference. We had some difficulties implementing external collision domains with the VDE plugin, and probably we do not test BUM packets properly.

I will do some further testing, and then I will update the documentation to explain the different behavior.

Many thanks for all your time and your help! 🚀 😍