EgressIP support on user defined networks

[pperiyasamy]

This commit adds an enhancement proposal to support EgressIP for the user defined segmentation networks which serves as a primary network for the pods.

[coveralls]

coverage: 52.644% (-0.09%) from 52.729%
when pulling dad57d3 on pperiyasamy:okep-multi-network-egressip
into 17dce5c on ovn-org:master.

[martinkennelly]

reviewed to ln 110

[martinkennelly]

Partially reviewed - as discussed via slack - we need to rework this for non-IC where pkt marks wont survive when pkts cross nodes.

[martinkennelly]

can you do me a favour? Feel free to say no cause it maybe a pain in the head for you.. but itll be easier for everyone to inspect the new arch. If theres no changes for any of the following deployment mods + network, fine to combine them in the same file, but name the file appropriately.

I think we need diagrams for the following combinations for each EIP "config" (defined later):

primaryeip-cdn-ic

primaryeip-cdn-non-ic

primaryeip-udn-l3-ic

primaryeip-udn-l3-non-ic

primaryeip-udn-l2

secondaryeip-cdn-ic

secondaryeip-cdn-non-ic

secondaryeip-udn-l3-ic

secondaryeip-udn-l3-non-ic

secondaryeip-udn-l2

primary means EIP is assigned to a pkt egressing primary interface and secondary is egress-ing a host secondary interface.

EIP "configs":

two pods, EIP with one IP - one on a node and another on the egress node
two pods, EIP with two IPs - one on a node and another on one of the egress nodes
Name the files so its clear which deployment model + network(s) we are in.

Peri will update the info to incorporate: https://docs.google.com/document/d/1mlhI9TJgy7DxJT-JX7Qz9_-rTgivZwlO4Pfyob616cs/edit

[tssurya]

I have not looked at anything here in depth, but I want to say I'm loving what's happening here in this PR 👏👏👏👏👏

[pperiyasamy]

Thank you @martinkennelly and @tssurya for your great feedback and bringing up non-IC scenario.
@martinkennelly As we discussed, have incorporated EIP configs from your document (it was a very nice one, thanks).

[martinkennelly]

reviewed to ln198

[coveralls]

coverage: 52.745% (-0.002%) from 52.747%
when pulling 9fb2ab6 on pperiyasamy:okep-multi-network-egressip
into afd362f on ovn-org:master.

[coveralls]

coverage: 52.742% (-0.005%) from 52.747%
when pulling 0ddb750 on pperiyasamy:okep-multi-network-egressip
into afd362f on ovn-org:master.

[martinkennelly]

note to self: check to see upgrade order - if node controllers updates before CM, it could be blocking reconfiguring EIP during upgrade until CM is upgraded.
Edit: its ovnkube-node first on ocp, therefore, during upgrade, waiting for the mark will block eip config.

[martinkennelly]

Initially, we thought we could derive the mark from the EIP CR name or UID but thats a bad idea for a number of reasons (name can change, string needs to map to int within a range and it may clash).
So, we thought CM seems a good place to allocate it but during upgrade, EIP config would be blocked until CM upgrades. Need input.

[coveralls]

coverage: 52.745% (-0.002%) from 52.747%
when pulling 40f831e on pperiyasamy:okep-multi-network-egressip
into afd362f on ovn-org:master.

[martinkennelly]

Options:

• Not support EIPs assigned to secondary interfaces for non-ic
• Use ovn_cluster_router to redirect to egress nodes GW and then also add a LRP to that egress nodes GW LRPs that redirect it to its mgnt port and pkt mark