Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(NLB): Introduce Service annotation to allow ICMP for Path MTU Discovery #3939

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

chriswachira
Copy link
Contributor

@chriswachira chriswachira commented Nov 12, 2024

Issue

#3897

Description

It might be necessary for some environments to allow Path MTU discovery for negotiation of MTU between two hosts. If a receiving host has a smaller MTU than the sending host, the receiving host sends an ICMP message to instruct the sending host to split the payload into multiple smaller packets and retransmit them:

  • Destination Unreachable: Fragmentation Needed and Don't Fragment was Set (Type 3, Code 4) for IPv4 networks.
  • ICMPv6 Packet Too Big (PTB) (Type 2) for IPv6 networks.

This work introduces a Service annotation (below) that when configured, will automatically add a security group rule to the managed security group, depending on the IP address type.

service.beta.kubernetes.io/aws-load-balancer-enable-icmp-for-path-mtu-discovery: "on"

 

For IPv4 VPCs, an explicit rule is added to the managed security group to allow ICMPv4 Type 3 Code 4.

Screenshot 2024-11-12 at 10 13 27
 

For dual stack VPCs, an explicit rule is added to the managed security group to allow both ICMPv4 Type 3 Code 4 and ICMPv6 Type 2 Code 0.

Screenshot 2024-11-12 at 00 40 16  

If the service.beta.kubernetes.io/load-balancer-source-ranges annotation is also present, it will also create an explicit rule for each source range.

Screenshot 2024-11-12 at 00 49 22

Checklist

  • Added tests that cover your change (if possible)
  • Added/modified documentation as required (such as the README.md, or the docs directory)
  • Manually tested
  • Made sure the title of the PR is a good description that can go into the release notes

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

  • Backfilled missing tests for code in same general area 🎉
  • Refactored something and made the world a better place 🌟

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 12, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @chriswachira. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Nov 12, 2024
@chriswachira chriswachira force-pushed the introduce-annotation-to-enable-icmp-rule-for-path-mtu-discovery branch from 47d0bc4 to 45aa281 Compare November 12, 2024 07:59

!!!example
```
service.beta.kubernetes.io/aws-load-balancer-enable-icmp-for-path-mtu-discovery: on
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the sample is incorrect, we need to surround the boolean value with quotes:

        service.beta.kubernetes.io/aws-load-balancer-enable-icmp-for-path-mtu-discovery: "on"

otherwise we get this error:

error: unable to decode "/Users/nixozach/echoserver/svc.yaml": json: cannot unmarshal bool into Go struct field ObjectMeta.metadata.annotations of type string

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made this fix, kindly test.

@@ -84,6 +97,18 @@ func (t *defaultModelBuildTask) buildManagedSecurityGroupIngressPermissions(ctx
},
},
})
if icmpForPathMtuConfigured {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that once the rules are added, they can't be removed using:

service.beta.kubernetes.io/aws-load-balancer-enable-icmp-for-path-mtu-discovery: "off"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a check to make sure it only adds the security group rules if the annotation value is "on", kindly test.

@chriswachira chriswachira force-pushed the introduce-annotation-to-enable-icmp-rule-for-path-mtu-discovery branch from 45aa281 to 6ed4baf Compare November 18, 2024 20:44
@chriswachira chriswachira force-pushed the introduce-annotation-to-enable-icmp-rule-for-path-mtu-discovery branch from 6ed4baf to 0ccf48d Compare November 21, 2024 07:08
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: chriswachira, zac-nixon
Once this PR has been reviewed and has the lgtm label, please assign oliviassss for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@shraddhabang
Copy link
Collaborator

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Dec 4, 2024
@chriswachira chriswachira force-pushed the introduce-annotation-to-enable-icmp-rule-for-path-mtu-discovery branch from 0ccf48d to 33e1738 Compare December 7, 2024 21:09
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 7, 2024
@chriswachira
Copy link
Contributor Author

/retest

@chriswachira chriswachira force-pushed the introduce-annotation-to-enable-icmp-rule-for-path-mtu-discovery branch from 33e1738 to 64e498f Compare December 8, 2024 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants