-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/235 get flow partitionings #267
Conversation
Release notes:
|
JaCoCo model module code coverage report - scala 2.13.11
|
JaCoCo agent module code coverage report - scala 2.13.11
|
JaCoCo reader module code coverage report - scala 2.13.11
|
JaCoCo server module code coverage report - scala 2.13.11
|
# Conflicts: # server/src/main/scala/za/co/absa/atum/server/api/repository/PartitioningRepositoryImpl.scala # server/src/test/scala/za/co/absa/atum/server/api/controller/PartitioningControllerUnitTests.scala # server/src/test/scala/za/co/absa/atum/server/api/repository/PartitioningRepositoryUnitTests.scala
WITH limited_partitionings AS ( | ||
SELECT P.id_partitioning | ||
FROM flows.partitioning_to_flow PF | ||
JOIN runs.partitionings P ON PF.fk_partitioning = P.id_partitioning | ||
WHERE PF.fk_flow = i_flow_id | ||
ORDER BY P.created_at DESC, P.id_partitioning | ||
LIMIT i_limit OFFSET i_offset | ||
) | ||
SELECT | ||
11 AS status, | ||
'OK' AS status_text, | ||
P.id_partitioning AS id, | ||
P.partitioning, | ||
P.created_by AS author, | ||
_has_more AS has_more | ||
FROM | ||
runs.partitionings P | ||
WHERE | ||
P.id_partitioning IN (SELECT LP.id_partitioning FROM limited_partitionings LP) | ||
ORDER BY | ||
P.created_at DESC, | ||
P.id_partitioning; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
WITH limited_partitionings AS ( | |
SELECT P.id_partitioning | |
FROM flows.partitioning_to_flow PF | |
JOIN runs.partitionings P ON PF.fk_partitioning = P.id_partitioning | |
WHERE PF.fk_flow = i_flow_id | |
ORDER BY P.created_at DESC, P.id_partitioning | |
LIMIT i_limit OFFSET i_offset | |
) | |
SELECT | |
11 AS status, | |
'OK' AS status_text, | |
P.id_partitioning AS id, | |
P.partitioning, | |
P.created_by AS author, | |
_has_more AS has_more | |
FROM | |
runs.partitionings P | |
WHERE | |
P.id_partitioning IN (SELECT LP.id_partitioning FROM limited_partitionings LP) | |
ORDER BY | |
P.created_at DESC, | |
P.id_partitioning; | |
SELECT | |
11 AS status, | |
'OK' AS status_text, | |
P.id_partitioning, | |
P.partitioning, | |
P.created_by, | |
has_more | |
FROM | |
runs.partitionings P INNER JOIN | |
flows.partitioning_to_flow PF ON PF.fk_partitioning = P.id_partitioning | |
WHERE | |
PF.fk_flow = i_flow | |
ORDER BY | |
P.id_partitioning, | |
P.created_at DESC | |
LIMIT i_limit OFFSET i_offset; |
Several things:
- You don't need the
WITH
here. Generally the order of approaches to use:- use
JOIN
- use subquery - there are cases where this can be better then JOIN, but we don't have that much data yet to run a meaningful comparison
- use
WITH
- use temporary tables
- use
- in the
RETURN QUERY
statement it's not required the output column names to be the same as the OUT ones. Only the position matters. - I switched the order of
ORDER BY
columns, reasons described here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok changed
P.created_at DESC, | ||
P.id_partitioning; | ||
|
||
IF NOT FOUND THEN |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment as here. Do we return an error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok changed
OUT id BIGINT, | ||
OUT partitioning JSONB, | ||
OUT author TEXT, | ||
OUT has_more BOOLEAN |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we also want to care whether a given partitioning was primary
for the given flow?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Little value now, and adds a table to the JOIN.
IMHO not worth it.
Btw, we could add the main flow id to partitioning table (for easier search and one less index).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review finished, apart from 1 q from me and 1 note from David it looks good
IF NOT FOUND THEN | ||
status := 12; | ||
status_text := 'OK with no partitionings found'; | ||
RETURN NEXT; | ||
END IF; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IF NOT FOUND THEN | |
status := 12; | |
status_text := 'OK with no partitionings found'; | |
RETURN NEXT; | |
END IF; |
database/src/main/postgres/flows/V1.9.8__get_flow_partitionings.sql
Outdated
Show resolved
Hide resolved
…s.sql Co-authored-by: David Benedeki <14905969+benedeki@users.noreply.github.com>
# Conflicts: # server/src/main/scala/za/co/absa/atum/server/api/repository/PartitioningRepository.scala # server/src/main/scala/za/co/absa/atum/server/api/service/PartitioningServiceImpl.scala
Introduces GET endpoint for retrieval of partitionings of a given flow
Closes #235
Release notes: