-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Source-MySQL] Enable Enhanced Standard sync with PK -> Cursor switch #30556
Conversation
Before Merging a Connector Pull RequestWow! What a great pull request you have here! 🎉 To merge this PR, ensure the following has been done/considered for each connector added or updated:
If the checklist is complete, but the CI check is failing,
|
source-mysql test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ❌ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
source-mysql-strict-encrypt test report (commit
|
Step | Result |
---|---|
Build connector tar | ❌ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
source-mysql test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
source-mysql-strict-encrypt test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ❌ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
source-mysql test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
source-mysql-strict-encrypt test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ❌ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
source-mysql-strict-encrypt test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ❌ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
source-mysql test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
6713874
to
eb897b9
Compare
source-mysql-strict-encrypt test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ❌ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
source-mysql test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
source-mysql-strict-encrypt test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
source-mysql test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
c4b0e1d
to
f6f0a27
Compare
source-mysql test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
source-mysql-strict-encrypt test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ❌ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
.put(JdbcUtils.DATABASE_KEY, Strings.addRandomSuffix("db", "_", 10)) | ||
.put(JdbcUtils.USERNAME_KEY, TEST_USER) | ||
.put(JdbcUtils.PASSWORD_KEY, TEST_PASSWORD.call()) | ||
.put("standard_via_pk", true) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is still required as we are adding this as the default?
@@ -92,6 +120,7 @@ public void setup() throws Exception { | |||
.put(JdbcUtils.DATABASE_KEY, Strings.addRandomSuffix("db", "_", 10)) | |||
.put(JdbcUtils.USERNAME_KEY, container.getUsername()) | |||
.put(JdbcUtils.PASSWORD_KEY, container.getPassword()) | |||
.put("standard_via_pk", true) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this still required?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh you right. I don't need that anymore.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good in general - approving to unblock.
I assume that MySqlPkStrictEncryptJdbcSourceAcceptanceTest.java was renamed to MySqlStrictEncryptJdbcSourceAcceptanceTest.java and the old test deleted (similar with the integration test)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can just delete this file for now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good. I'll delete the file
Renamed and removed pk sync only jdbc tests bumped version in metadata and dockerfile update mysql change logs
…f strict encrypt Enable per stream state for the last suite of strict encrypt tests Final per stream state enable Add stream capable state to MysqlDatatype Enable it for mysqlssl
f6f0a27
to
ac7269d
Compare
source-mysql-strict-encrypt test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
source-mysql test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
source-mysql-strict-encrypt test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
source-mysql test report (commit
|
Step | Result |
---|---|
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Unit Tests | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
…#30556) Enable enhanced standard sync
What
Enable PK for initial sync with cursor switch over
PR with change gated behind flag