Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for '.' in Bucket Name #84

Merged
merged 3 commits into from
Jul 17, 2024

Conversation

avinas-kumar
Copy link
Collaborator

@avinas-kumar avinas-kumar commented Jul 2, 2024

Dear DIL maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

  • My PR addresses the following JIRA issues and references them in the PR title.

Description

  • Here are some details about my PR, including screenshots (if applicable): Currently for the separation of bucket name and endpoint for s3 uploads we rely on '.' present in the host url. The left side of '.' is considered as bucket name and right side of it is considered an endpoint.
    So added a new parameter in ms.source.s3.parameters for bucket name as "bucket_name" which would help to navigate through the edge cases.

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason: Added UTs for it.

Commits

  • My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

protected String getEndpoint(JsonObject parameters, String host) {
if (parameters.has(KEY_BUCKET_NAME)) {
String bucketName = parameters.get(KEY_BUCKET_NAME).getAsString().toLowerCase();
host = host.toLowerCase(Locale.ROOT).replace(bucketName, "");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor concern butt I don't see the tolowercase used in the original getEndpointFromHost method. Is there an existing use case that has issues with cases?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are replacing bucket name from the host, if some client enters bucket name in different case as in the source uri. It won't be replaced properly.

*
* @param parameters JsonObject containing ms.source.s3.parameters
* @param host hostname with bucket name in the beginning
* @return the endpoint name without the bucket name
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: in the comment maybe add some details to explain its difference from getEndpointFromHost. That will help with debugging in the future

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I will add

new Gson().fromJson("{\"region\" : \"us-east-2\", \"bucket_name\" : \"colleCt-us-west-2.tealium.com\"}",
JsonObject.class);
String host = "collect-us-west-2.tealium.com.s3.amazonaws.com";
String endpoint = s3SourceV2.getEndpoint(parameteres, host);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: parameteres -> parameters

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure with fix the typo.

@dihu-linkedin dihu-linkedin merged commit baf26e3 into linkedin:master Jul 17, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants