Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: prevent hanging when calling get_groups with prefix and small page_size #326

Merged
merged 6 commits into from
Nov 8, 2024

Conversation

toph-allen
Copy link
Collaborator

Intent

get_groups(client, prefix = "prefix") would fail when the page_size required pagination — it would hang forever.

Fixes #319

Approach

  • This hang occur because the v1/groups API only ever returns the first page of results when using prefix.
  • Whereas v1/users returns no results when a subsequent page of a prefix request is called, v1/groups always returns the first page, which causes connectapi to just page eternally.
  • So, we just don't call the pagination function when we're using a prefix!
  • Updated documentation to clarify behavior of different parameters.

Checklist

  • Does this change update NEWS.md (referencing the connected issue if necessary)?
  • Does this change need documentation? Have you run `document()?

@tdstein
Copy link

tdstein commented Nov 8, 2024

This hang occur because the v1/groups API only ever returns the first page of results when using prefix.

Is this the rumored bug in Connect?

Copy link

@tdstein tdstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any unit tests that already verify this functionality?

NEWS.md Outdated Show resolved Hide resolved
R/get.R Show resolved Hide resolved
R/get.R Show resolved Hide resolved
R/get.R Outdated Show resolved Hide resolved
R/get.R Show resolved Hide resolved
Co-authored-by: Neal Richardson <neal.p.richardson@gmail.com>
@toph-allen
Copy link
Collaborator Author

toph-allen commented Nov 8, 2024

This hang occur because the v1/groups API only ever returns the first page of results when using prefix.

Is this the rumored bug in Connect?

@tdstein I mean, it's an odd behavior, but the v1/groups endpoint works exactly as described in its documentation:

  • For a prefix search, results are sorted based on similarity to the prefix. A prefix search ignores asc_order. The first page of results will always be returned.

So when calling v1/groups?prefix=c&page_size=2&page_number=2, the results returned are the same as for page 1. This behavior is kinda weird.

Aside: the v1/users endpoint docs say that it also behaves this way, but it does not. When calling it with a prefix with page_number > 1, no results are returned (which I feel like is the most sensible behavior.

I found Cole's original issue about this: https://github.com/rstudio/connect/issues/17543. Two notes on that: it (1) requests that the API return an error with prefix and page_number > 1, and (2) points to our docs on offset pagination, which I think point to a sensible answer.

The Cookbook docs on offset pagination are here. They recommend paging through results until you reach an empty page, which suggests that the behavior of the v1/users endpoint is the better approach and that the behavior of v1/groups endpoint is actually wrong.

Another option for this PR is to have the page_offset function check to see if the current result is zero or the same as the previous result, which would make it robust to current and future variations of Connect's API behavior.


[edit] another subtle bug with v1/groups is that the docs say that order is always based on similarity, but specifying page size does not just truncate the list, it seems to include the first list and then some number of results from the end. that's getting a connect bug too.

@toph-allen
Copy link
Collaborator Author

@toph-allen toph-allen merged commit 5a44722 into main Nov 8, 2024
19 checks passed
@toph-allen toph-allen deleted the toph/get-groups-updates branch November 8, 2024 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate error with pagination
4 participants