-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Source DuckDB: β¨ Add MotherDuck support π¦π¦ #29428
Source DuckDB: β¨ Add MotherDuck support π¦π¦ #29428
Conversation
Before Merging a Connector Pull RequestWow! What a great pull request you have here! π To merge this PR, ensure the following has been done/considered for each connector added or updated:
If the checklist is complete, but the CI check is failing,
|
Tests are passing locally. I've opened this up for review. |
destination-duckdb test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
Code format checks | β |
Connector package install | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
destination-duckdb test report (commit
|
Step | Result |
---|---|
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
Code format checks | β |
Connector package install | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
Co-authored-by: Augustin <augustin@airbyte.io>
destination-duckdb test report (commit
|
Step | Result |
---|---|
Connector package install | β |
Build destination-duckdb docker image for platform linux/x86_64 | β |
Unit tests | β |
Integration tests | β |
Acceptance tests | β |
Code format checks | β |
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
|
||
This destination is meant to be used on a local workstation and won't work on Kubernetes | ||
This destination is meant to be used on a local workstation or with the MotherDuck service as the destination. Local file-based DBs will not work on Kubernetes or in Airbyte Cloud. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This connector is not enabled in cloud - if I understand correctly that should be part of the PR; if that's the case the enabled
flag in the metadata.yaml has to be adjusted
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for calling this out. And yes, we definitely want to enable for Cloud as part of this PR. π
airbyte-integrations/connectors/destination-duckdb/integration_tests/spec.json
Show resolved
Hide resolved
airbyte-integrations/connectors/destination-duckdb/destination_duckdb/destination.py
Show resolved
Hide resolved
β¦feat-motherduck-support
destination-duckdb test report (commit
|
Step | Result |
---|---|
Connector package install | β |
Build destination-duckdb docker image for platform linux/x86_64 | β |
Unit tests | β |
Integration tests | β |
Acceptance tests | β |
Code format checks | β |
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
destination-duckdb test report (commit
|
Step | Result |
---|---|
Connector package install | β |
Build destination-duckdb docker image for platform linux/x86_64 | β |
Unit tests | β |
Integration tests | β |
Acceptance tests | β |
Code format checks | β |
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
destination-duckdb test report (commit
|
Step | Result |
---|
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
destination-duckdb test report (commit
|
Step | Result |
---|---|
Connector package install | β |
Build destination-duckdb docker image for platform linux/x86_64 | β |
Unit tests | β |
Integration tests | β |
Acceptance tests | β |
Code format checks | β |
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
destination-duckdb test report (commit
|
Step | Result |
---|---|
Connector package install | β |
Build destination-duckdb docker image for platform linux/x86_64 | β |
Unit tests | β |
Integration tests | β |
Acceptance tests | β |
Code format checks | β |
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
destination-duckdb test report (commit
|
Step | Result |
---|---|
Connector package install | β |
Build destination-duckdb docker image for platform linux/x86_64 | β |
Unit tests | β |
Integration tests | β |
Acceptance tests | β |
Code format checks | β |
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
β¦feat-motherduck-support
destination-duckdb test report (commit
|
Step | Result |
---|---|
Connector package install | β |
Build destination-duckdb docker image for platform linux/x86_64 | β |
Unit tests | β |
Integration tests | β |
Acceptance tests | β |
Code format checks | β |
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
β¦feat-motherduck-support
destination-duckdb test report (commit
|
Step | Result |
---|---|
Connector package install | β |
Build destination-duckdb docker image for platform linux/x86_64 | β |
Unit tests | β |
Integration tests | β |
Acceptance tests | β |
Code format checks | β |
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
destination-duckdb test report (commit
|
Step | Result |
---|---|
Connector package install | β |
Build destination-duckdb docker image for platform linux/x86_64 | β |
Unit tests | β |
Integration tests | β |
Acceptance tests | β |
Code format checks | β |
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
destination-duckdb test report (commit
|
Step | Result |
---|---|
Connector package install | β |
Build destination-duckdb docker image for platform linux/x86_64 | β |
Unit tests | β |
Integration tests | β |
Acceptance tests | β |
Code format checks | β |
Validate airbyte-integrations/connectors/destination-duckdb/metadata.yaml | β |
Connector version semver check | β |
Connector version increment check | β |
QA checks | β |
π View the logs here
βοΈ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-duckdb test
βοΈ Click here to dev/test this in the browser.
What
md:
) database paths.How
/local/
paths, allowingmd:
andmotherduck:
paths.motherduck_api_key
config options which allows users to securely pass their API token from the motherduck service.π¨ User Impact π¨
Are there any breaking changes? What is the end result perceived by the user?
For connector PRs, use this section to explain which type of semantic versioning bump occurs as a result of the changes. Refer to our Semantic Versioning for Connectors guidelines for more information. Breaking changes to connectors must be documented by an Airbyte engineer (PR author, or reviewer for community PRs) by using the Breaking Change Release Playbook.
If there are breaking changes, please merge this PR with the π¨π¨ emoji so changelog authors can further highlight this if needed.
Pre-merge Actions
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.0.0.1
Dockerfile
has version0.0.1
README.md
bootstrap.md
. See description and examplesdocs/integrations/<source or destination>/<name>.md
including changelog with an entry for the initial version. See changelog exampledocs/integrations/README.md
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
Updating a connector
Community member or Airbyter
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
Connector Generator
-scaffold
in their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates
then checking in your changes