Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating incremental merge WHERE condition to handle nullable fields … #7598

Closed
wants to merge 1 commit into from

Conversation

amardatar
Copy link

…with an appropriate null comparison.

resolves dbt-labs/dbt-adapters#159

Description

This is a proposed resolution to dbt-labs/dbt-adapters#159 which updates the check in the WHERE condition of the DELETE statement of an incremental merge, to use an additional {{ target }}.{{ key }} is null and {{ source }}.{{ key }} is null check to handle nulls.

Checklist

@dbeatty10
Copy link
Contributor

Thank you for opening this @amardatar !

Do you think it's ready for review? Let us know if you need any assistance getting this across the line.

@amardatar amardatar marked this pull request as ready for review June 15, 2023 14:59
@amardatar amardatar requested review from a team as code owners June 15, 2023 14:59
@amardatar
Copy link
Author

Yep I think it's ready! Let me know if there's anything more that is required - but is functioning as expected (and seemed fine on the existing tests I ran).

@dbeatty10 dbeatty10 added Team:Adapters Issues designated for the adapter area of the code ready_for_review Externally contributed PR has functional approval, ready for code review from Core engineering labels Jun 15, 2023
@jtcohen6
Copy link
Contributor

@cla-bot check

@cla-bot cla-bot bot added the cla:yes label Jun 25, 2023
@cla-bot
Copy link

cla-bot bot commented Jun 25, 2023

The cla-bot has been summoned, and re-checked this pull request!

@amardatar
Copy link
Author

Hey @dbeatty10 / @jtcohen6 - just wanted to quickly follow-up on this in case it got lost! If it's sitting on a backlog already then don't mind me, just wanted to make sure it's going somewhere eventually :)

@dbeatty10
Copy link
Contributor

Thanks for checking in @amardatar! Indeed it's on our backlog.

Copy link
Contributor

This PR has been marked as Stale because it has been open with no activity as of late. If you would like the PR to remain open, please comment on the PR or else it will be closed in 7 days.

@github-actions github-actions bot added the stale Issues that have gone stale label Feb 19, 2024
@nikita-sheremet-flocktory
Copy link

nikita-sheremet-flocktory commented Feb 24, 2024

@dbeatty10
Any news for releasing this bug? May I help? Can I do something to help or speed up this PR to release?

Thanks in advance!

@dbeatty10
Copy link
Contributor

@dbeatty10 Any news for releasing this bug? May I help? Can I do something to help or speed up this PR to release?

Thanks in advance!

@nikita-sheremet-flocktory

Since this PR was originally opened, we split the code within dbt-core into several repos:

  • dbt-core
  • dbt-common
  • dbt-postgres
  • dbt-adapters

So there's two things we'll need to do:

  1. Open a new PR in https://github.com/dbt-labs/dbt-adapters that has the same contents as this one
  2. Add an applicable test case

Both would be helpful, but adding the test case would be the most helpful.

@nikita-sheremet-flocktory

@dbeatty10

Add an applicable test case

Could you please share a link to application test that can be used as example? For now I do not have ideas how it should be implemented.

Thanks in advance!

@amardatar
Copy link
Author

Hey @dbeatty10 @nikita-sheremet-flocktory I created dbt-labs/dbt-adapters#110 for this.

@dbeatty10 as far as the test case goes, and from what I could see, it looked like the existing tests should work for this just with a change in the fixtures (rather than the tests themselves). I've made those edits instead, but I couldn't really work out how tests are currently meant to run (if they're even meant to run from here, or if they should be run from the adapters themselves).

@dbeatty10 dbeatty10 added community This PR is from a community member dbt-adapters Needs migration to the dbt-adapters repo bug Something isn't working labels Apr 9, 2024
@dbeatty10
Copy link
Contributor

Thanks for taking the time to open this PR @amardatar! Since opening, we've decoupled dbt Adapters from dbt Core, and this code now lives in a separate repo: dbt-adapters.

A consequence of the decoupling is that PR can't be merged anymore as is, so we're closing it. For more context see #9171.

The linked issue has already been transferred. Please don't hesitate to re-open your proposed code changes within this PR in the dbt-adapters repo.

@dbeatty10 dbeatty10 closed this Apr 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cla:yes community This PR is from a community member dbt-adapters Needs migration to the dbt-adapters repo ready_for_review Externally contributed PR has functional approval, ready for code review from Core engineering stale Issues that have gone stale Team:Adapters Issues designated for the adapter area of the code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-2563] [Bug] Incremental updates using unique_key result in duplicates if fields in the unique_key are null
4 participants