Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix attachment of run to lineage job #2953

Merged
merged 3 commits into from
Oct 25, 2024
Merged

Conversation

phixMe
Copy link
Member

@phixMe phixMe commented Oct 25, 2024

Problem

Avoid matching jobs to runs using name and namespace and instead use the current_run_uuid recently added to retrieve the current run... This also has the advantage of avoiding looping and being simpler in aggregate.

Checklist

  • You've signed-off your work
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • You've included a one-line summary of your change for the CHANGELOG.md (Depending on the change, this may not be necessary).
  • You've versioned your .sql database schema migration according to Flyway's naming convention (if relevant)
  • You've included a header in any source code files (if relevant)

@boring-cyborg boring-cyborg bot added the api API layer changes label Oct 25, 2024
Copy link

netlify bot commented Oct 25, 2024

Deploy Preview for peppy-sprite-186812 canceled.

Name Link
🔨 Latest commit d54c180
🔍 Latest deploy log https://app.netlify.com/sites/peppy-sprite-186812/deploys/671c2478b38dbb0008cda039

@phixMe phixMe changed the title Fix/lineage run attachment Fix attachment of run to lineage job Oct 25, 2024
Copy link

codecov bot commented Oct 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.19%. Comparing base (46bbfa7) to head (d54c180).
Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2953      +/-   ##
============================================
+ Coverage     81.16%   81.19%   +0.02%     
+ Complexity     1506     1505       -1     
============================================
  Files           268      268              
  Lines          7363     7358       -5     
  Branches        329      325       -4     
============================================
- Hits           5976     5974       -2     
  Misses         1226     1226              
+ Partials        161      158       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@wslulciuc wslulciuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

@wslulciuc wslulciuc enabled auto-merge (squash) October 25, 2024 23:06
@wslulciuc wslulciuc merged commit 9e2e9b4 into main Oct 25, 2024
15 checks passed
@wslulciuc wslulciuc deleted the fix/lineage-run-attachment branch October 25, 2024 23:16
lmassaoy pushed a commit to nubank/NuMarquez that referenced this pull request Oct 28, 2024
* Fixing data quality display. (MarquezProject#2937)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Dataset Version call simplification (MarquezProject#2938)

* Fixing data quality display.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Fixing dataset version calls.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* feat: allow db-migrate without version (MarquezProject#2936)

Signed-off-by: David Goss <david@davidgoss.co>

* Display full `runID` and check icon when copied (MarquezProject#2940)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Deferred copy revert. (MarquezProject#2941)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Long text handling (MarquezProject#2942)

* Deferred copy revert.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Long text handling.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Adding search back in.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Use project root for docker volume prefix (MarquezProject#2943)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* fix: Correct SQL query pagination for DatasetVersion findAll method (MarquezProject#2945)

Signed-off-by: Alper İnan <kurumsal.alper@gmail.com>
Signed-off-by: Alper <kurumsal.alper@gmail.com>

* Update changelog for `0.50.0`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Replace `redoc-cli` with `redocly`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Prepare for release 0.50.0

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Prepare next development version 0.51.0-SNAPSHOT

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Templatize event time in `metadata.json` (MarquezProject#2946)

* Templatize event time in `metadata.json`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Use `metadata.template.json`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

---------

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update CHANGELOG.md

* Update `web/docs/demo.gif` (MarquezProject#2948)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* fix(deps): update dependency io.openlineage:openlineage-java to v1.23.0 (MarquezProject#2907)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency org.assertj:assertj-core to v3.26.3 (MarquezProject#2909)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* fix(deps): update dependency org.postgresql:postgresql to v42.7.4 (MarquezProject#2912)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency org.opensearch.client:opensearch-rest-client to v2.17.1 (MarquezProject#2911)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* fix(deps): update dependency org.apache.commons:commons-lang3 to v3.17.0 (MarquezProject#2908)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Ignore `**/stats/**` (MarquezProject#2952)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update compatibility for `0.50.0`

* fix(deps): update dependency org.opensearch.client:opensearch-java to v2.16.0 (MarquezProject#2910)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* `Dataset.currentVersionUuid` `->` `DatasetVersion.uuid` (MarquezProject#2954)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update Events Page (MarquezProject#2955)

* Tuning the events page for longer events.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Adding events file.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Refetch jobs button.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Refetch jobs button.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Lint

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* Lineage run attachment issue. (MarquezProject#2953)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* feature: Better handling of missing environment variables in setupProxy.js file. (MarquezProject#2956)

Signed-off-by: Artur Owczarek <owczarek.artur@gmail.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Signed-off-by: David Goss <david@davidgoss.co>
Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>
Signed-off-by: Alper İnan <kurumsal.alper@gmail.com>
Signed-off-by: Alper <kurumsal.alper@gmail.com>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: Artur Owczarek <owczarek.artur@gmail.com>
Co-authored-by: Peter Hicks <phixMe@users.noreply.github.com>
Co-authored-by: davidjgoss <david@davidgoss.co>
Co-authored-by: Willy Lulciuc <willy@datakin.com>
Co-authored-by: Alper İnan <65857758+inanalper@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy.lulciuc@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Artur Owczarek <owczarek.artur@gmail.com>
jonathanpmoraes pushed a commit to nubank/NuMarquez that referenced this pull request Nov 12, 2024
* Fixing data quality display. (MarquezProject#2937)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Dataset Version call simplification (MarquezProject#2938)

* Fixing data quality display.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Fixing dataset version calls.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* feat: allow db-migrate without version (MarquezProject#2936)

Signed-off-by: David Goss <david@davidgoss.co>

* Display full `runID` and check icon when copied (MarquezProject#2940)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Deferred copy revert. (MarquezProject#2941)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Long text handling (MarquezProject#2942)

* Deferred copy revert.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Long text handling.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Adding search back in.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Use project root for docker volume prefix (MarquezProject#2943)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* fix: Correct SQL query pagination for DatasetVersion findAll method (MarquezProject#2945)

Signed-off-by: Alper İnan <kurumsal.alper@gmail.com>
Signed-off-by: Alper <kurumsal.alper@gmail.com>

* Update changelog for `0.50.0`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Replace `redoc-cli` with `redocly`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Prepare for release 0.50.0

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Prepare next development version 0.51.0-SNAPSHOT

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Templatize event time in `metadata.json` (MarquezProject#2946)

* Templatize event time in `metadata.json`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Use `metadata.template.json`

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

---------

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update CHANGELOG.md

* Update `web/docs/demo.gif` (MarquezProject#2948)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* fix(deps): update dependency io.openlineage:openlineage-java to v1.23.0 (MarquezProject#2907)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency org.assertj:assertj-core to v3.26.3 (MarquezProject#2909)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* fix(deps): update dependency org.postgresql:postgresql to v42.7.4 (MarquezProject#2912)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(deps): update dependency org.opensearch.client:opensearch-rest-client to v2.17.1 (MarquezProject#2911)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* fix(deps): update dependency org.apache.commons:commons-lang3 to v3.17.0 (MarquezProject#2908)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Ignore `**/stats/**` (MarquezProject#2952)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update compatibility for `0.50.0`

* fix(deps): update dependency org.opensearch.client:opensearch-java to v2.16.0 (MarquezProject#2910)

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* `Dataset.currentVersionUuid` `->` `DatasetVersion.uuid` (MarquezProject#2954)

Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>

* Update Events Page (MarquezProject#2955)

* Tuning the events page for longer events.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Adding events file.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Refetch jobs button.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Refetch jobs button.

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

* Lint

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* Lineage run attachment issue. (MarquezProject#2953)

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>

* feature: Better handling of missing environment variables in setupProxy.js file. (MarquezProject#2956)

Signed-off-by: Artur Owczarek <owczarek.artur@gmail.com>

---------

Signed-off-by: phixMe <peter.hicks@pdtechsolutions.com>
Signed-off-by: David Goss <david@davidgoss.co>
Signed-off-by: Willy Lulciuc <willy.lulciuc@gmail.com>
Signed-off-by: Alper İnan <kurumsal.alper@gmail.com>
Signed-off-by: Alper <kurumsal.alper@gmail.com>
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: Artur Owczarek <owczarek.artur@gmail.com>
Co-authored-by: Peter Hicks <phixMe@users.noreply.github.com>
Co-authored-by: davidjgoss <david@davidgoss.co>
Co-authored-by: Willy Lulciuc <willy@datakin.com>
Co-authored-by: Alper İnan <65857758+inanalper@users.noreply.github.com>
Co-authored-by: Willy Lulciuc <willy.lulciuc@gmail.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Artur Owczarek <owczarek.artur@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api API layer changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants