Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

commitIdFromGitRepo: fix stackoverflow if many branches are used. #93337

Merged
merged 1 commit into from
Jul 17, 2020

Conversation

Mic92
Copy link
Member

@Mic92 Mic92 commented Jul 17, 2020

If many branches are created than builtins.match stack overflows because
of a bug in libstdc++: see NixOS/nix#2147

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

If many branches are created than builtins.match stack overflows because
of a bug in libstdc++: see NixOS/nix#2147
@Mic92 Mic92 requested review from edolstra, infinisil and nbp as code owners July 17, 2020 09:44
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Jul 17, 2020
@lheckemann
Copy link
Member

It's with a heavy heart that I'm ❤️ ing this, since it's terrible and shouldn't be necessary. But at the same time I'm happy for a workaround.

(fixes #42379)

@edolstra
Copy link
Member

IMHO a function like commitIdFromGitRepo shouldn't exist in the first place. Nix is not a general purpose language and we shouldn't be processing .git internal files from Nix expressions. It's better to use builtin functions like fetchGit or fetchTree that give you the revision without any magic.

@cole-h
Copy link
Member

cole-h commented Jul 17, 2020

@edolstra Do you have an example of how one would use fetchGit or fetchTree to get the revision? I'd gladly start using those two over commitIdFromGitRepo if there is a way to communicate this information in Nix.

@lheckemann
Copy link
Member

fetchGit will also import the whole thing into the store AFAIU, so I don't think that's desirable — particularly since this is used to generate NixOS version info.

@Mic92
Copy link
Member Author

Mic92 commented Jul 17, 2020

IMHO a function like commitIdFromGitRepo shouldn't exist in the first place. Nix is not a general purpose language and we shouldn't be processing .git internal files from Nix expressions. It's better to use builtin functions like fetchGit or fetchTree that give you the revision without any magic.

fetchGit would be a lot slower though since it copies everything to the nix store.

@Mic92
Copy link
Member Author

Mic92 commented Jul 17, 2020

I would leave fetchGit to another pr.

@Mic92 Mic92 merged commit a5c9a7c into NixOS:master Jul 17, 2020
@Mic92 Mic92 deleted the fix-stackoverflow branch July 17, 2020 18:20
@lheckemann
Copy link
Member

Backported to 20.03: e7c435c

@lheckemann
Copy link
Member

I guess an @edolstra -approved solution might be making builtins.fetchGit lazier, so that it only imports the path into the store when the value of outPath is used, allowing access to rev and revCount without importing it into the store?

@Mic92
Copy link
Member Author

Mic92 commented Jul 17, 2020

Well now with flakes nixpkgs the git is copied to nix store now every time anyway.

@edolstra
Copy link
Member

There is an issue about that: NixOS/nix#3121

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants