Fix race condition with forked monorepos #9723
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
TheSven73
force-pushed
the
concurrent-forked-repos-fix
branch
from
cc45daf
to
2af7993
Compare
|
Not sure how to add reviewers / watchers. @radoering @gustavgransbo @seeker25 |
radoering
requested changes
Oct 13, 2024
When installing packages from different directories of a forked
monorepo, a race condition may occur, where multiple git clients
are interacting in parallel with the same git repository.
Add a failing test to detect the race condition. Mark as xfail.
This test succeeds due to a problem with the poetry test framework.
Example pyproject.toml which triggers the problem:
```toml
[tool.poetry]
name = "some_other_repo"
version = "0.1.0"
description = ""
authors = ["Jonathan Rayner <jonathan.j.rayner@gmail.com>"]
packages = [{ include = "some_other_repo"}]
[tool.poetry.dependencies]
python = "^3.10 <3.13"
pkg_1 = {git = "git@github.com:JonathanRayner/some_monorepo.git", subdirectory = "pkg_1" }
pkg_2 = {git = "git@github.com:gustavgransbo/some_monorepo.git", subdirectory = "pkg_2", tag="v2"}
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
```
Source: @gustavgransbo
https://github.com/orgs/python-poetry/discussions/9718#discussioncomment-10785589
TheSven73
force-pushed
the
concurrent-forked-repos-fix
branch
from
2af7993
to
5297739
Compare
TheSven73
changed the title
radoering
reviewed
Oct 13, 2024
When git clones a repo in a base directory, it will place the clone in a subdirectory of the base. That subdirectory is named after the repo url name, which is the "testing" right before the .git in https://github.com/test/testing.git This behaviour may introduce race conditions to Poetry. Let the mock repo mimic it, so that the (absence of) race conditions can be verified.
When installing packages from different directories of a forked monorepo, a race condition may occur, where multiple git clients are interacting in parallel with the same git repository. Fix by serializing git operations that interact with the same git repository. This makes the test succeed. Remove xfail. Links: python-poetry#9658 (comment) https://github.com/orgs/python-poetry/discussions/9718#discussioncomment-10785589
TheSven73
force-pushed
the
concurrent-forked-repos-fix
branch
from
5297739
to
d34faea
Compare
radoering
approved these changes
Oct 14, 2024
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There is a race condition when installing packages from multiple forked monorepos. Fix it.
See discussion