Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Git Remote Helper: Concurrency #29

Merged
merged 2 commits into from
Jan 23, 2019
Merged

Git Remote Helper: Concurrency #29

merged 2 commits into from
Jan 23, 2019

Conversation

kim
Copy link
Collaborator

@kim kim commented Jan 22, 2019

Closes #23

@kim kim changed the base branch from verify-lobs to master January 22, 2019 14:52
@kim kim changed the title [WIP] Concurrency Git Remote Helper: Concurrency Jan 22, 2019
@kim
Copy link
Collaborator Author

kim commented Jan 22, 2019

Notes:

  • The number of OS threads is bounded by the -maxN RTS option. Performance seems to deteriorate above 4.
  • We do not currently limit the number of Haskell threads. This should be fine for most repos, although an initial push of the linux kernel might hog some memory 😎

Copy link
Contributor

@adinapoli-mndc adinapoli-mndc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I have added a bunch of comments possibly worth dwelling upon.

(renderProcessError (errError e))
(renderSourceLoc (errCallStack e))

instance Exception ConcurrentException
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't like the fact of implementing Show in a non-principled way, I guess you could try to implement displayException for ConcurrentException and see if that works out?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only recently realised the existence of displayException, and am confuse. Show is a superclass of Exception, so I still have to derive it for all error types - but that is something I try to avoid: one should always use the appropriate render* function.

Am I asking too much?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I am with you on this one. I guess the "best" thing you could do is to derive Show using the standing deriving mechanism but write your "pretty printed" version as part of displayException. I do agree that the superclass constraint is a nuisance.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4b113d1 destroyed my hopes

git-remote-ipfs/git-remote-ipfs.cabal Show resolved Hide resolved
git-remote-ipfs/src/Network/IPFS/Git/RemoteHelper.hs Outdated Show resolved Hide resolved
kim added 2 commits January 23, 2019 16:11
This is rather naive for now, as we're essentially running every IPFS API call
in its own Haskell thread. Observed speedups are significant nevertheless.

Upper bounds are placed on the number of concurrent connections to the API, as
well as the number of OS threads (via the `-maxN` RTS option).
@kim kim merged commit 8a3444b into master Jan 23, 2019
@kim kim deleted the conc branch January 23, 2019 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants