Set bgp local-address implicitly for internal peers only

[rkojedzinszky]

Is your feature request related to a problem? Please describe.
A feature request.

#778 was about that internal peers could not establish neighborship, and #777 fixed it by setting local address to nodeIP for all peers, including external ones. This caused issues with existing setups (#1371). Additional fix has been implemented (#1392), howewer that needs the configuration be even more complex.

Describe the solution you'd like
I suggest relaxing #777 by setting localaddress for internal peers only. That would bring the old behavior back, while still providing the fix for #778. It would make configuration much simpler in many cases. In other cases, one can still set localaddress explicitly with #1392.

[aauren]

This isn't the direction we want to go here. Binding out of any available interface can be confusing for a number of reasons and cause issues for other users. It is less secure and also less reproducible.

I think getting more specific in the annotations was a better way to go and produces a better configuration. The only real problem was that a specific user wasn't originally using annotations and kube-router doesn't currently allow you to mix and match configurations between command-line flags and annotations.

In the future this can be shored up a couple of ways:

• Configurations for peers should be easier to specify - Rather than annotations and parameter flags that specify lists of comma separated values whose position and ordering is important while also being confusing, we should change to use a data structure. (Restructure node peer annotations #1393)
• In the future we should probably not make command-line configuration and node annotations mutually exclusive. This doesn't seem all that logical to me and defeats the principle of least surprise for users as comments in Mismatched local address when peering with external router on difference network #1371 point out

[rkojedzinszky]

@aauren please note that many hardware/software routers works this way, binding to wildcard address by default, and allowing stricter/more secure configurations explicitly. I mean, that all configurations made by a user should be entirely made by the user. The internal peer configurations are implicit, while external ones are needed by the user. This is advanced topic, and a user should know the operation details and consequences.

Also, it would still be helpful to toggle this behavior with a command line flag, to preserve compatibility.