Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/v4backend #115

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open

Feature/v4backend #115

wants to merge 25 commits into from

Conversation

consolethinks
Copy link
Collaborator

scicat-cli changes to support v4 be (v3 support is dropped by this). Closes #114

Copy link
Collaborator

@minottic minottic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A general comment. Since the Bearer addition is used in multiple places, would it make sense to make it a function for reusability/later changes?

datasetIngestor/checkMetadata.go Show resolved Hide resolved
datasetUtils/getUserInfoFromToken.go Outdated Show resolved Hide resolved
datasetIngestor/ingestDataset_test.go Outdated Show resolved Hide resolved
cmd/commands/commonConstants.go Show resolved Hide resolved
datasetIngestor/checkMetadata.go Show resolved Hide resolved
datasetUtils/createJob.go Outdated Show resolved Hide resolved
cmd/cliutils/transferType.go Show resolved Hide resolved
cmd/commands/datasetRetriever.go Show resolved Hide resolved
color.Unset()
break
}
datasetHasIdAndOwnerGroup := func(dataset Dataset) bool {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do I understand right that this, in addition to the filter here, checks if the ownerGroup is empty or it's equal to dataset.ownerGroup? If so, couldn't one achieve this by modifying the filter? Then (and also if not changing the filter), to find the missing datasets, one could transform datasetDetails into a map and loop over datasetList[i:end]. The ones which are not in the map are the ones missing. This will be O(n) (map creation) + O(m) (datasetList[i:end] loop), while the current implementation is O(~n (indexFunction) *m (loop)) (given the linear search, if I am not mistaken)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's also a good idea to add a test for this

Copy link
Collaborator Author

@consolethinks consolethinks Oct 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I actually wanted to optimize it but I ended up just simplifying the original code. I'll try to apply your suggestion. (TODO)

datasetIngestor/checkMetadata.go Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

scicat-cli v4 backend support
2 participants