Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7311 add get headers from peer task #7781

Open
wants to merge 237 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 226 commits
Commits
Show all changes
237 commits
Select commit Hold shift + click to select a range
4b80016
7311: Add PeerTask system for use in future PRs
Matilda-Clerke Sep 17, 2024
a8d5a9f
7311: Clean up some warnings
Matilda-Clerke Sep 17, 2024
4c64dbe
7311: Add feature toggle for enabling use of the peertask system wher…
Matilda-Clerke Sep 17, 2024
7a94fe2
7311: Remove log used for testing, apply spotless
Matilda-Clerke Sep 17, 2024
ace5dd1
7311: Add private constructor to PeerTaskFeatureToggle to prevent ins…
Matilda-Clerke Sep 17, 2024
52d440a
7311: Switch to logging a warning instead of throwing an exception wh…
Matilda-Clerke Sep 17, 2024
f392a0f
7311: Update javadoc to match previous commit
Matilda-Clerke Sep 17, 2024
2fb2690
7311: spotless
Matilda-Clerke Sep 17, 2024
f14aaeb
7311: Fix broken BesuCommandTest
Matilda-Clerke Sep 18, 2024
06553be
Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…
Matilda-Clerke Sep 18, 2024
1ba5184
7311: add class
Matilda-Clerke Sep 18, 2024
f2500dd
7311: Move PeerTaskFeatureToggle to more appropriate location
Matilda-Clerke Sep 18, 2024
0992c35
Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…
Matilda-Clerke Sep 18, 2024
33b810b
7311: add X prefix to peertask-system-enabled
Matilda-Clerke Sep 18, 2024
a378d22
Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…
Matilda-Clerke Sep 18, 2024
15b6bdf
7311: Move --Xpeertask-system-enabled out of BesuCommand and make hidden
Matilda-Clerke Sep 18, 2024
a9f6714
Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…
Matilda-Clerke Sep 18, 2024
e3fbc6c
7311: spotless
Matilda-Clerke Sep 18, 2024
c03bffb
Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…
Matilda-Clerke Sep 18, 2024
5e8b750
Merge branch 'main' into 7311-add-cli-feature-toggle-for-peertask-system
macfarla Sep 18, 2024
e4be5c0
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Sep 18, 2024
bc11e0c
Merge branch 'main' into 7311-add-cli-feature-toggle-for-peertask-system
Matilda-Clerke Sep 18, 2024
b2a45c2
7311: Add GetReceiptsFromPeerTask
Matilda-Clerke Sep 19, 2024
513b74f
7311: Move isPeerTaskSystemEnabled to SynchronizerOptions
Matilda-Clerke Sep 19, 2024
6b86919
Merge remote-tracking branch 'origin/7311-add-cli-feature-toggle-for-…
Matilda-Clerke Sep 19, 2024
645e0e3
Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…
Matilda-Clerke Sep 19, 2024
4522414
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Sep 19, 2024
28fc2cd
Merge remote-tracking branch 'origin/7311-add-GetReceiptsFromPeerTask…
Matilda-Clerke Sep 19, 2024
2364ed5
7311: Fix javadoc issue
Matilda-Clerke Sep 19, 2024
ced19cd
Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…
Matilda-Clerke Sep 19, 2024
03f6495
7311: Fix javadoc issue
Matilda-Clerke Sep 19, 2024
5859444
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Sep 19, 2024
9a4f3dd
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Sep 19, 2024
97e5918
7311: Move PeerTaskFeatureToggleTestHelper to TestUtil and fix Runner…
Matilda-Clerke Sep 20, 2024
e0f736d
7311: spotless
Matilda-Clerke Sep 20, 2024
df7f62d
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Sep 20, 2024
c335cbe
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Sep 20, 2024
98aefcd
Merge branch 'main' into 7311-add-cli-feature-toggle-for-peertask-system
Matilda-Clerke Sep 20, 2024
6e734f9
7311: Remove PeerTaskFeatureToggle in favor of including isPeerTaskSy…
Matilda-Clerke Sep 20, 2024
42ca85b
Merge branch 'main' into 7311-add-cli-feature-toggle-for-peertask-system
Matilda-Clerke Sep 20, 2024
76724ed
Merge branch 'refs/heads/7311-add-cli-feature-toggle-for-peertask-sys…
Matilda-Clerke Sep 20, 2024
fc9b3f2
7311: Adjust to the removal of PeerTaskFeatureToggle and use Synchron…
Matilda-Clerke Sep 20, 2024
08c66fd
7311: Reduce timeout in PeerTaskRequestSender to 5s
Matilda-Clerke Sep 20, 2024
049cae2
7311: Refactor PeerManager to be an interface
Matilda-Clerke Sep 20, 2024
b6ec075
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Sep 20, 2024
5afba63
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Sep 20, 2024
3c1178c
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Sep 20, 2024
5c3a61a
7311: Fix up compile errors after merge
Matilda-Clerke Sep 20, 2024
8448898
7311: Fix MetricsAcceptanceTest
Matilda-Clerke Sep 20, 2024
ab21100
7311: Fix MetricsAcceptanceTest
Matilda-Clerke Sep 20, 2024
2ac52f0
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Sep 20, 2024
f2ac53e
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Sep 23, 2024
3de578d
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Sep 23, 2024
e901fdf
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Sep 24, 2024
24e73a8
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Sep 24, 2024
f077206
7311: Fix DownloadReceiptsStep when using peer task system
Matilda-Clerke Sep 25, 2024
fa12495
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Sep 25, 2024
6e349e1
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Sep 25, 2024
ad86ae6
7311: Rename PeerManager to PeerSelector
Matilda-Clerke Sep 25, 2024
38f04ab
7311: Reword PeerSelector javadoc to avoid implementation details
Matilda-Clerke Sep 25, 2024
6de3fb3
7311: Use ConcurrentHashMap in DefaultPeerSelector
Matilda-Clerke Sep 25, 2024
da9cd43
7311: Reword trace log in DefaultPeerSelector
Matilda-Clerke Sep 25, 2024
ce7d245
7311: Remove unused imports
Matilda-Clerke Sep 25, 2024
c9eb22e
7311: Use a 1 second delay between retries in PeerTaskExecutor to mat…
Matilda-Clerke Sep 25, 2024
e2fda73
7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest
Matilda-Clerke Sep 25, 2024
608fece
7311: Add testGetPeerButNoPeerMatchesFilter to DefaultPeerSelectorTest
Matilda-Clerke Sep 25, 2024
0e76000
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Sep 25, 2024
2d07800
7311: spotless
Matilda-Clerke Sep 25, 2024
b910b4d
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Sep 25, 2024
ad26297
7311: Fix MetricsAcceptanceTest
Matilda-Clerke Sep 20, 2024
96c8030
7311: Fix MetricsAcceptanceTest
Matilda-Clerke Sep 20, 2024
b0f2ed0
7311: Modify PeerTaskExecutor metric to include response time from peer
Matilda-Clerke Sep 26, 2024
598b519
7311: Use SubProtocol instead of subprotocol name string in PeerTask
Matilda-Clerke Sep 26, 2024
bc25b16
7311: rename timing context to ignored to prevent intellij warnings
Matilda-Clerke Sep 26, 2024
e31bb70
7311: Use constants for number of retries
Matilda-Clerke Sep 26, 2024
41923d3
7311: Convert PeerTaskExecutorResult to a record
Matilda-Clerke Sep 26, 2024
720f94e
7311: Rename PeerTaskBehavior to PeerTaskRetryBehavior
Matilda-Clerke Sep 29, 2024
7d845b3
7311: Move peer selection logic to PeerSelector
Matilda-Clerke Sep 30, 2024
50c26f1
7311: spotless
Matilda-Clerke Sep 30, 2024
b7c0c95
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Sep 30, 2024
a81855d
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Sep 30, 2024
64adedc
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Sep 30, 2024
2c1446e
7311: Fix up everything broken after merge
Matilda-Clerke Oct 1, 2024
3c0c47b
7311: Attempt to improve performance of peer task system in pipeline
Matilda-Clerke Oct 2, 2024
d0bd5ed
7311: fix compile check
Matilda-Clerke Oct 2, 2024
1c25ac5
7311: Fix broken workflow
Matilda-Clerke Oct 2, 2024
2e6dfd9
7311: Reduce logging in JsonRpcExecutor to trace level
Matilda-Clerke Oct 2, 2024
aca8058
7311: More changes in DownloadReceiptsStep
Matilda-Clerke Oct 2, 2024
4d59b10
7311: Rework DownloadReceiptsStep
Matilda-Clerke Oct 3, 2024
8718102
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 3, 2024
e63f473
7311: Make changes as discussed in walkthrough meeting
Matilda-Clerke Oct 3, 2024
fae39a8
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 3, 2024
d1847f2
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 4, 2024
07852dc
7311: Update after merge and make discussed changes from walkthrough …
Matilda-Clerke Oct 4, 2024
c477d70
7311: Change to regular HashMap
Matilda-Clerke Oct 4, 2024
6c57a7c
7311: Remove runtime exception again
Matilda-Clerke Oct 4, 2024
6d2cb95
7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior
Matilda-Clerke Oct 6, 2024
d84520a
7311: Rename getPeerTaskBehavior to getPeerTaskRetryBehavior
Matilda-Clerke Oct 6, 2024
77ed748
Merge remote-tracking branch 'origin/7311-add-peertask-foundation-cod…
Matilda-Clerke Oct 6, 2024
0896e31
7311: Rework PeerTaskExecutor retry system to be 0-based
Matilda-Clerke Oct 6, 2024
5f924c4
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 6, 2024
b13ac92
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 6, 2024
5006b39
7311: Fix up compile errors after merge
Matilda-Clerke Oct 6, 2024
07f3a7e
7311: Fix broken DownloadReceiptsStepTest test
Matilda-Clerke Oct 6, 2024
493ac91
7311: Move GetReceipts to services worker for parallelism
Matilda-Clerke Oct 7, 2024
1a30174
7311: Refactor peer task system usage in DownloadReceiptsStep to bett…
Matilda-Clerke Oct 7, 2024
84af9f9
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Oct 7, 2024
2865625
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 7, 2024
2f86ed9
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Oct 7, 2024
82cedb0
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 7, 2024
bdd96ba
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 8, 2024
c047f42
7311: Remove unused async methods in PeerTaskExecutor
Matilda-Clerke Oct 8, 2024
5aa6b0b
7311: Return Optional<EthPeer> in PeerSelector.getPeer and utilise ex…
Matilda-Clerke Oct 8, 2024
4fd4724
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 8, 2024
d6120b0
7311: Update after merge
Matilda-Clerke Oct 8, 2024
8becdb3
7311: Redo getPeer again to include hasAvailableRequestCapacity check
Matilda-Clerke Oct 8, 2024
4ad85e8
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 8, 2024
86a1f0b
7311: Add protocol spec supplier to GetReceiptsFromPeerTask
Matilda-Clerke Oct 8, 2024
8186a77
7311: Rework getPeer again to use LEAST_TO_MOST_BUSY comparator
Matilda-Clerke Oct 8, 2024
37b0ec2
7311: Import PeerNotConnected class instead of using fully qualified …
Matilda-Clerke Oct 8, 2024
545fd5c
7311: Change to specifying retry counts in PeerTask instead of behavi…
Matilda-Clerke Oct 9, 2024
7bd048b
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 9, 2024
e9d08f3
7311: clean up after merge
Matilda-Clerke Oct 9, 2024
20478d3
7311: clean up after merge
Matilda-Clerke Oct 9, 2024
4f544f4
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 9, 2024
66a9de2
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 9, 2024
3ddfe71
7311: Fix up javadoc
Matilda-Clerke Oct 10, 2024
1c268b7
7311: Add additional metrics to PeerTaskExecutor
Matilda-Clerke Oct 10, 2024
b06f38b
7311: Add Predicate to PeerTask to check for partial success
Matilda-Clerke Oct 10, 2024
09ee1c8
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 10, 2024
3c12d3d
7311: Fix incorrect name on isPartialSuccessTest
Matilda-Clerke Oct 10, 2024
b1c47ae
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 10, 2024
4664db9
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 10, 2024
3b8b7d5
7311: Implement isPartialSuccess and add unit tests
Matilda-Clerke Oct 10, 2024
d66dd3a
7311: Add partialSuccessCounter and inflightRequestGauge in PeerTaskE…
Matilda-Clerke Oct 11, 2024
fa22e93
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 11, 2024
cff0099
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 11, 2024
a3f5d4a
7311: Also filter by whether a peer is fully validated
Matilda-Clerke Oct 11, 2024
382f7a5
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 11, 2024
714db0a
7311: Remove unneeded throws in RunnerTest
Matilda-Clerke Oct 11, 2024
3a68980
7311: Fix up inflight requests gauge in PeerTaskExecutor
Matilda-Clerke Oct 11, 2024
3ce476d
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 11, 2024
c422bc5
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 11, 2024
74aa7a0
7311: Update plugin api hash
Matilda-Clerke Oct 11, 2024
56c1f9d
7311: Update plugin api hash
Matilda-Clerke Oct 11, 2024
fe50d95
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 11, 2024
e733452
7311: Add javadoc to LabelledGauge.isLabelsObserved
Matilda-Clerke Oct 13, 2024
b3a252b
7311: Update plugin-api hash
Matilda-Clerke Oct 13, 2024
4f9cf52
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 13, 2024
af93824
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 13, 2024
3c96eba
7311: Update changelog
Matilda-Clerke Oct 13, 2024
fe27973
7311: Implement GetHeadersFromPeerTask and use in DetermineCommonAnce…
Matilda-Clerke Oct 15, 2024
e664a51
7311: Handle headers with no receipts as a special case in DownloadRe…
Matilda-Clerke Oct 15, 2024
7daf30f
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 15, 2024
c4e685d
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 15, 2024
6800cdd
7311: Complete merge
Matilda-Clerke Oct 15, 2024
302f2a8
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 15, 2024
00f51f3
7311: Get DetermineCommonAncestorTask working with peer task system
Matilda-Clerke Oct 16, 2024
44fd3a8
7311: Use taskName instead of className for labelNames
Matilda-Clerke Oct 16, 2024
ac1c4ed
7311: Use snake_case for metric names
Matilda-Clerke Oct 16, 2024
7503535
7311: Use _total metric name suffix
Matilda-Clerke Oct 16, 2024
a59bd30
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 16, 2024
2989f2a
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 16, 2024
ed25941
7311: rework partial success handling
Matilda-Clerke Oct 17, 2024
4f4b091
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 17, 2024
c396fb5
7311: Update GetReceiptsFromPeerTask with partialSuccess changes
Matilda-Clerke Oct 17, 2024
2e5d6f4
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 17, 2024
ffe1c76
7311: Update GetHeadersFromPeerTask with partialSuccess changes
Matilda-Clerke Oct 17, 2024
5a79636
7311: Add default implementation to LabelledGauge.isLabelsObserved
Matilda-Clerke Oct 17, 2024
0668083
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 17, 2024
3fa4f9c
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 17, 2024
8a34505
7311: Use Peer task systems GetHeadersFromPeerTask in GetBlockFromPee…
Matilda-Clerke Oct 17, 2024
b075a91
7311: Fix broken unit test
Matilda-Clerke Oct 17, 2024
74995bb
Merge branch 'main' into 7311-add-peertask-foundation-code
Matilda-Clerke Oct 17, 2024
17edf67
Merge branch 'refs/heads/7311-add-peertask-foundation-code' into 7311…
Matilda-Clerke Oct 17, 2024
2eaaf62
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 17, 2024
e2fea48
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Oct 18, 2024
e5b7d0e
7311: Remove unused constructor from AbstractPeerBlockValidator
Matilda-Clerke Oct 18, 2024
59d403a
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 18, 2024
2f2b295
Merge branch 'refs/heads/remove-unused-constructor-from-abstractpeerb…
Matilda-Clerke Oct 18, 2024
875375e
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Oct 20, 2024
1484f00
7311: Use GetHeadersFromPeerTask in AbstractPeerBlockValidator
Matilda-Clerke Oct 20, 2024
acf7b7c
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 20, 2024
1445f3f
7311: Use peer task executor in SyncTargetManager
Matilda-Clerke Oct 21, 2024
87847fa
7311: Fix javadoc on BesuControllerBuilder
Matilda-Clerke Oct 21, 2024
f11e695
7311: Remove logs used to confirm operation
Matilda-Clerke Oct 21, 2024
58be140
7311: Implement GetHeadersFromPeerTask in FastSyncActions and PivotBl…
Matilda-Clerke Oct 23, 2024
095e31d
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Oct 23, 2024
c2205c4
7311: Rename parseResponse to processResponse
Matilda-Clerke Oct 23, 2024
1b9922f
7311: Wrap peer task system usage in ethScheduler call to match other…
Matilda-Clerke Oct 24, 2024
d827e68
7311: apply spotless
Matilda-Clerke Oct 27, 2024
26476ae
7311: Move check for empty trie hash into GetReceiptsFromPeerTask and…
Matilda-Clerke Oct 27, 2024
a357ecf
7311: spotless
Matilda-Clerke Oct 27, 2024
e7a13a4
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Oct 27, 2024
e227e39
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 27, 2024
3d20f56
7311: Fix compile issue after merge
Matilda-Clerke Oct 27, 2024
b0f57f8
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 27, 2024
5187afc
7311: Fix compile issue after merge
Matilda-Clerke Oct 27, 2024
f1db42e
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Oct 28, 2024
6ee320b
7311: Remove BodyValidator and update code and test to match
Matilda-Clerke Oct 28, 2024
5419579
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 28, 2024
6b07948
7311: Implement GetHeadersForPeerTask usage in DownloadHeadersStep
Matilda-Clerke Oct 28, 2024
61f57aa
7311: spotless
Matilda-Clerke Oct 28, 2024
149e65a
7311: remove unneeded logs
Matilda-Clerke Oct 28, 2024
5fcad25
7311: spotless
Matilda-Clerke Oct 28, 2024
1923996
7311: Fix up pre-fill and add test to test failure scenario
Matilda-Clerke Oct 29, 2024
84b422c
7311: Use ProtocolSchedule.anyMatch to find if any ProtocolSpecs are …
Matilda-Clerke Oct 29, 2024
8e6e2b0
7311: Only attempt to remove headers on successful requests
Matilda-Clerke Oct 29, 2024
24d8f99
Merge branch 'main' into 7311-add-GetReceiptsFromPeerTask
Matilda-Clerke Oct 29, 2024
7606ae5
Merge branch 'refs/heads/7311-add-GetReceiptsFromPeerTask' into 7311-…
Matilda-Clerke Oct 29, 2024
1685e26
7311: clean up after merge
Matilda-Clerke Oct 29, 2024
cc6c017
7311: clean up after merge
Matilda-Clerke Oct 29, 2024
b75d453
7311: Use peer task system in RangeHeadersFetcher
Matilda-Clerke Oct 30, 2024
7a975c8
7311: Use peer task system in DownloadHeaderSequenceTask
Matilda-Clerke Oct 31, 2024
5b7fbf2
Merge branch 'main' into 7311-add-GetHeadersFromPeerTask
Matilda-Clerke Nov 1, 2024
6f8624c
7311: Fix GetHeadersFromPeerTask mocking in CheckPointSyncChainDownlo…
Matilda-Clerke Nov 4, 2024
3c597b0
Merge branch 'main' into 7311-add-GetHeadersFromPeerTask
Matilda-Clerke Nov 4, 2024
99e6d0e
7311: Extract peer task executor answer for getHeaders to separate cl…
Matilda-Clerke Nov 4, 2024
af53c9a
7311: spotless
Matilda-Clerke Nov 4, 2024
bbc1594
7311: Implement peer task system usage in BackwardSyncStep
Matilda-Clerke Nov 4, 2024
acc1191
7311: Implement peer task system usage in ChainHeadTracker
Matilda-Clerke Nov 6, 2024
5ce5fe9
7311: Implement peer task system usage in PivotSelectorFromSafeBlock …
Matilda-Clerke Nov 7, 2024
ad4fd13
7311: Implement unit test for GetHeadersFromPeerTask
Matilda-Clerke Nov 7, 2024
7ad664a
7311: Fix up merge compile error
Matilda-Clerke Nov 7, 2024
5a807b4
Merge branch 'main' into 7311-add-GetHeadersFromPeerTask
Matilda-Clerke Nov 7, 2024
48e40c0
Merge branch 'main' into 7311-add-GetHeadersFromPeerTask
Matilda-Clerke Nov 18, 2024
bedf3d0
Merge branch 'main' into 7311-add-GetHeadersFromPeerTask
Matilda-Clerke Nov 20, 2024
130bfac
7311: Ensure FastSyncActions and PivotSelectorFromSafeBlock retry get…
Matilda-Clerke Nov 25, 2024
c2515bf
Merge remote-tracking branch 'origin/7311-add-GetHeadersFromPeerTask'…
Matilda-Clerke Nov 25, 2024
30c2c39
7311: Change PeerTaskExecutorResult.ethPeer to an Optional
Matilda-Clerke Nov 25, 2024
c925e6b
7311: Use CancellationException instead of InterruptedException in Pi…
Matilda-Clerke Nov 25, 2024
b3181c6
7311: Use PivotBlockRetriever.MAX_QUERY_RETRIES_PER_PEER to set retri…
Matilda-Clerke Nov 25, 2024
adcf088
7311: spotless
Matilda-Clerke Nov 25, 2024
b54fc14
7311: Add PeerTask.shouldDisconnectPeer and ensure functionality matc…
Matilda-Clerke Nov 25, 2024
ddceb7c
7311: Remove old info logs
Matilda-Clerke Nov 25, 2024
f002ba3
7311: Fix broken test by correctly including peer in PeerTaskExecutor…
Matilda-Clerke Nov 26, 2024
75b091a
Merge branch 'main' into 7311-add-GetHeadersFromPeerTask
Matilda-Clerke Nov 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -690,7 +690,8 @@ public BesuController build() {
besuComponent.map(BesuComponent::getBlobCache).orElse(new BlobCache()),
miningConfiguration);

final List<PeerValidator> peerValidators = createPeerValidators(protocolSchedule);
final List<PeerValidator> peerValidators =
createPeerValidators(protocolSchedule, peerTaskExecutor);

final EthProtocolManager ethProtocolManager =
createEthProtocolManager(
Expand All @@ -708,7 +709,13 @@ public BesuController build() {

final PivotBlockSelector pivotBlockSelector =
createPivotSelector(
protocolSchedule, protocolContext, ethContext, syncState, metricsSystem, blockchain);
protocolSchedule,
protocolContext,
ethContext,
peerTaskExecutor,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the PeerTaskExecutor to the EthContext could be a nice home for this and save a lot of changes to wire through the PeerTaskExecutor through all the tasks.

syncState,
metricsSystem,
blockchain);

final DefaultSynchronizer synchronizer =
createSynchronizer(
Expand Down Expand Up @@ -743,6 +750,7 @@ public BesuController build() {
protocolContext,
transactionPool,
miningConfiguration,
peerTaskExecutor,
syncState,
ethProtocolManager);

Expand Down Expand Up @@ -881,6 +889,7 @@ private PivotBlockSelector createPivotSelector(
final ProtocolSchedule protocolSchedule,
final ProtocolContext protocolContext,
final EthContext ethContext,
final PeerTaskExecutor peerTaskExecutor,
final SyncState syncState,
final MetricsSystem metricsSystem,
final Blockchain blockchain) {
Expand Down Expand Up @@ -918,6 +927,8 @@ private PivotBlockSelector createPivotSelector(
ethContext,
metricsSystem,
genesisConfigOptions,
peerTaskExecutor,
syncConfig,
unverifiedForkchoiceSupplier,
unsubscribeForkchoiceListener);
} else {
Expand Down Expand Up @@ -982,6 +993,7 @@ protected SubProtocolConfiguration createSubProtocolConfiguration(
* @param protocolContext the protocol context
* @param transactionPool the transaction pool
* @param miningConfiguration the mining parameters
* @param peerTaskExecutor the peer task executor
* @param syncState the sync state
* @param ethProtocolManager the eth protocol manager
* @return the mining coordinator
Expand All @@ -991,6 +1003,7 @@ protected abstract MiningCoordinator createMiningCoordinator(
ProtocolContext protocolContext,
TransactionPool transactionPool,
MiningConfiguration miningConfiguration,
PeerTaskExecutor peerTaskExecutor,
SyncState syncState,
EthProtocolManager ethProtocolManager);

Expand Down Expand Up @@ -1162,29 +1175,42 @@ private ChainDataPruner createChainPruner(final BlockchainStorage blockchainStor
* Create peer validators list.
*
* @param protocolSchedule the protocol schedule
* @param peerTaskExecutor the peer task executor
* @return the list
*/
protected List<PeerValidator> createPeerValidators(final ProtocolSchedule protocolSchedule) {
protected List<PeerValidator> createPeerValidators(
final ProtocolSchedule protocolSchedule, final PeerTaskExecutor peerTaskExecutor) {
final List<PeerValidator> validators = new ArrayList<>();

final OptionalLong daoBlock = genesisConfigOptions.getDaoForkBlock();
if (daoBlock.isPresent()) {
// Setup dao validator
validators.add(
new DaoForkPeerValidator(protocolSchedule, metricsSystem, daoBlock.getAsLong()));
new DaoForkPeerValidator(
protocolSchedule, peerTaskExecutor, syncConfig, metricsSystem, daoBlock.getAsLong()));
}

final OptionalLong classicBlock = genesisConfigOptions.getClassicForkBlock();
// setup classic validator
if (classicBlock.isPresent()) {
validators.add(
new ClassicForkPeerValidator(protocolSchedule, metricsSystem, classicBlock.getAsLong()));
new ClassicForkPeerValidator(
protocolSchedule,
peerTaskExecutor,
syncConfig,
metricsSystem,
classicBlock.getAsLong()));
}

for (final Map.Entry<Long, Hash> requiredBlock : requiredBlocks.entrySet()) {
validators.add(
new RequiredBlocksPeerValidator(
protocolSchedule, metricsSystem, requiredBlock.getKey(), requiredBlock.getValue()));
protocolSchedule,
peerTaskExecutor,
syncConfig,
metricsSystem,
requiredBlock.getKey(),
requiredBlock.getValue()));
}

final CheckpointConfigOptions checkpointConfigOptions =
Expand All @@ -1193,6 +1219,8 @@ protected List<PeerValidator> createPeerValidators(final ProtocolSchedule protoc
validators.add(
new CheckpointBlocksPeerValidator(
protocolSchedule,
peerTaskExecutor,
syncConfig,
metricsSystem,
checkpointConfigOptions.getNumber().orElseThrow(),
checkpointConfigOptions.getHash().map(Hash::fromHexString).orElseThrow()));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
import org.hyperledger.besu.ethereum.core.MiningConfiguration;
import org.hyperledger.besu.ethereum.core.Util;
import org.hyperledger.besu.ethereum.eth.manager.EthProtocolManager;
import org.hyperledger.besu.ethereum.eth.manager.peertask.PeerTaskExecutor;
import org.hyperledger.besu.ethereum.eth.sync.state.SyncState;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
Expand Down Expand Up @@ -84,6 +85,7 @@ protected MiningCoordinator createMiningCoordinator(
final ProtocolContext protocolContext,
final TransactionPool transactionPool,
final MiningConfiguration miningConfiguration,
final PeerTaskExecutor peerTaskExecutor,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we need are adding unused dependencies on peerTaskExecutor via the createMiningCoordinator method inherited from the BesuControllerBuilderInterface. Is there a way around that? Perhaps overloading the method and providing a default implementation to be used by subclasses that don't care about it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, good idea

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, I can't really find a way around it. Overloading won't help because it's called from within BesuControllerBuilder, which can't make the distinction between which subclasses need the PeerTaskExecutor and which don't

final SyncState syncState,
final EthProtocolManager ethProtocolManager) {
final CliqueMinerExecutor miningExecutor =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
import org.hyperledger.besu.ethereum.eth.manager.EthProtocolManager;
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
import org.hyperledger.besu.ethereum.eth.manager.MergePeerFilter;
import org.hyperledger.besu.ethereum.eth.manager.peertask.PeerTaskExecutor;
import org.hyperledger.besu.ethereum.eth.manager.snap.SnapProtocolManager;
import org.hyperledger.besu.ethereum.eth.peervalidation.PeerValidator;
import org.hyperledger.besu.ethereum.eth.sync.SynchronizerConfiguration;
Expand Down Expand Up @@ -129,6 +130,7 @@ protected MiningCoordinator createMiningCoordinator(
final ProtocolContext protocolContext,
final TransactionPool transactionPool,
final MiningConfiguration miningConfiguration,
final PeerTaskExecutor peerTaskExecutor,
final SyncState syncState,
final EthProtocolManager ethProtocolManager) {

Expand All @@ -144,6 +146,7 @@ protected MiningCoordinator createMiningCoordinator(
protocolContext,
transactionPool,
miningConfiguration,
peerTaskExecutor,
syncState,
ethProtocolManager)))
.collect(Collectors.toList());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
import org.hyperledger.besu.ethereum.eth.EthProtocol;
import org.hyperledger.besu.ethereum.eth.SnapProtocol;
import org.hyperledger.besu.ethereum.eth.manager.EthProtocolManager;
import org.hyperledger.besu.ethereum.eth.manager.peertask.PeerTaskExecutor;
import org.hyperledger.besu.ethereum.eth.manager.snap.SnapProtocolManager;
import org.hyperledger.besu.ethereum.eth.sync.state.SyncState;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
Expand Down Expand Up @@ -142,6 +143,7 @@ protected MiningCoordinator createMiningCoordinator(
final ProtocolContext protocolContext,
final TransactionPool transactionPool,
final MiningConfiguration miningConfiguration,
final PeerTaskExecutor peerTaskExecutor,
final SyncState syncState,
final EthProtocolManager ethProtocolManager) {
final MutableBlockchain blockchain = protocolContext.getBlockchain();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import org.hyperledger.besu.ethereum.chain.Blockchain;
import org.hyperledger.besu.ethereum.core.MiningConfiguration;
import org.hyperledger.besu.ethereum.eth.manager.EthProtocolManager;
import org.hyperledger.besu.ethereum.eth.manager.peertask.PeerTaskExecutor;
import org.hyperledger.besu.ethereum.eth.sync.state.SyncState;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
import org.hyperledger.besu.ethereum.mainnet.EpochCalculator;
Expand All @@ -47,6 +48,7 @@ protected MiningCoordinator createMiningCoordinator(
final ProtocolContext protocolContext,
final TransactionPool transactionPool,
final MiningConfiguration miningConfiguration,
final PeerTaskExecutor peerTaskExecutor,
final SyncState syncState,
final EthProtocolManager ethProtocolManager) {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import org.hyperledger.besu.ethereum.eth.manager.EthProtocolManager;
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
import org.hyperledger.besu.ethereum.eth.manager.MergePeerFilter;
import org.hyperledger.besu.ethereum.eth.manager.peertask.PeerTaskExecutor;
import org.hyperledger.besu.ethereum.eth.peervalidation.PeerValidator;
import org.hyperledger.besu.ethereum.eth.peervalidation.RequiredBlocksPeerValidator;
import org.hyperledger.besu.ethereum.eth.sync.SynchronizerConfiguration;
Expand Down Expand Up @@ -68,6 +69,7 @@ protected MiningCoordinator createMiningCoordinator(
final ProtocolContext protocolContext,
final TransactionPool transactionPool,
final MiningConfiguration miningConfiguration,
final PeerTaskExecutor peerTaskExecutor,
final SyncState syncState,
final EthProtocolManager ethProtocolManager) {
return createTransitionMiningCoordinator(
Expand All @@ -79,6 +81,8 @@ protected MiningCoordinator createMiningCoordinator(
new BackwardSyncContext(
protocolContext,
protocolSchedule,
peerTaskExecutor,
syncConfig,
metricsSystem,
ethProtocolManager.ethContext(),
syncState,
Expand Down Expand Up @@ -235,14 +239,17 @@ protected PluginServiceFactory createAdditionalPluginServices(
}

@Override
protected List<PeerValidator> createPeerValidators(final ProtocolSchedule protocolSchedule) {
List<PeerValidator> retval = super.createPeerValidators(protocolSchedule);
protected List<PeerValidator> createPeerValidators(
final ProtocolSchedule protocolSchedule, final PeerTaskExecutor peerTaskExecutor) {
List<PeerValidator> retval = super.createPeerValidators(protocolSchedule, peerTaskExecutor);
final OptionalLong powTerminalBlockNumber = genesisConfigOptions.getTerminalBlockNumber();
final Optional<Hash> powTerminalBlockHash = genesisConfigOptions.getTerminalBlockHash();
if (powTerminalBlockHash.isPresent() && powTerminalBlockNumber.isPresent()) {
retval.add(
new RequiredBlocksPeerValidator(
protocolSchedule,
peerTaskExecutor,
syncConfig,
metricsSystem,
powTerminalBlockNumber.getAsLong(),
powTerminalBlockHash.get(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@
import org.hyperledger.besu.ethereum.eth.EthProtocol;
import org.hyperledger.besu.ethereum.eth.SnapProtocol;
import org.hyperledger.besu.ethereum.eth.manager.EthProtocolManager;
import org.hyperledger.besu.ethereum.eth.manager.peertask.PeerTaskExecutor;
import org.hyperledger.besu.ethereum.eth.manager.snap.SnapProtocolManager;
import org.hyperledger.besu.ethereum.eth.sync.state.SyncState;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
Expand Down Expand Up @@ -182,6 +183,7 @@ protected MiningCoordinator createMiningCoordinator(
final ProtocolContext protocolContext,
final TransactionPool transactionPool,
final MiningConfiguration miningConfiguration,
final PeerTaskExecutor peerTaskExecutor,
final SyncState syncState,
final EthProtocolManager ethProtocolManager) {
final MutableBlockchain blockchain = protocolContext.getBlockchain();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ protected MiningCoordinator createMiningCoordinator(
final ProtocolContext protocolContext,
final TransactionPool transactionPool,
final MiningConfiguration miningConfiguration,
final PeerTaskExecutor peerTaskExecutor,
final SyncState syncState,
final EthProtocolManager ethProtocolManager) {

Expand All @@ -120,6 +121,8 @@ protected MiningCoordinator createMiningCoordinator(
new TransitionBackwardSyncContext(
protocolContext,
transitionProtocolSchedule,
peerTaskExecutor,
syncConfig,
metricsSystem,
ethProtocolManager.ethContext(),
syncState,
Expand All @@ -138,6 +141,7 @@ protected MiningCoordinator createMiningCoordinator(
.isMiningEnabled(false)
.build())
.build(),
peerTaskExecutor,
syncState,
ethProtocolManager),
mergeBesuControllerBuilder.createTransitionMiningCoordinator(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import org.hyperledger.besu.ethereum.chain.MutableBlockchain;
import org.hyperledger.besu.ethereum.core.MiningConfiguration;
import org.hyperledger.besu.ethereum.eth.manager.EthProtocolManager;
import org.hyperledger.besu.ethereum.eth.manager.peertask.PeerTaskExecutor;
import org.hyperledger.besu.ethereum.eth.sync.state.SyncState;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
Expand Down Expand Up @@ -121,9 +122,11 @@ public void createsMigratingMiningCoordinator() {
final Map<Long, BesuControllerBuilder> consensusSchedule =
Map.of(0L, besuControllerBuilder1, 5L, besuControllerBuilder2);

when(besuControllerBuilder1.createMiningCoordinator(any(), any(), any(), any(), any(), any()))
when(besuControllerBuilder1.createMiningCoordinator(
any(), any(), any(), any(), any(), any(), any()))
.thenReturn(miningCoordinator1);
when(besuControllerBuilder2.createMiningCoordinator(any(), any(), any(), any(), any(), any()))
when(besuControllerBuilder2.createMiningCoordinator(
any(), any(), any(), any(), any(), any(), any()))
.thenReturn(miningCoordinator2);
final ProtocolContext mockProtocolContext = mock(ProtocolContext.class);
when(mockProtocolContext.getBlockchain()).thenReturn(mock(MutableBlockchain.class));
Expand All @@ -136,6 +139,7 @@ public void createsMigratingMiningCoordinator() {
mockProtocolContext,
mock(TransactionPool.class),
mock(MiningConfiguration.class),
mock(PeerTaskExecutor.class),
mock(SyncState.class),
mock(EthProtocolManager.class));

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
import org.hyperledger.besu.ethereum.core.InMemoryKeyValueStorageProvider;
import org.hyperledger.besu.ethereum.core.MiningConfiguration;
import org.hyperledger.besu.ethereum.eth.manager.EthProtocolManager;
import org.hyperledger.besu.ethereum.eth.manager.peertask.PeerTaskExecutor;
import org.hyperledger.besu.ethereum.eth.sync.state.SyncState;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
import org.hyperledger.besu.ethereum.mainnet.BlockHeaderValidator;
Expand Down Expand Up @@ -73,6 +74,7 @@ public class TransitionControllerBuilderTest {
@Mock ProtocolContext protocolContext;
@Mock MutableBlockchain mockBlockchain;
@Mock TransactionPool transactionPool;
@Mock PeerTaskExecutor peerTaskExecutor;
@Mock SyncState syncState;

@Mock(answer = Answers.RETURNS_DEEP_STUBS)
Expand Down Expand Up @@ -274,6 +276,7 @@ TransitionCoordinator buildTransitionCoordinator(
protocolContext,
transactionPool,
miningConfiguration,
peerTaskExecutor,
syncState,
ethProtocolManager);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
import org.hyperledger.besu.ethereum.ProtocolContext;
import org.hyperledger.besu.ethereum.core.Block;
import org.hyperledger.besu.ethereum.eth.manager.EthContext;
import org.hyperledger.besu.ethereum.eth.manager.peertask.PeerTaskExecutor;
import org.hyperledger.besu.ethereum.eth.sync.SynchronizerConfiguration;
import org.hyperledger.besu.ethereum.eth.sync.backwardsync.BackwardChain;
import org.hyperledger.besu.ethereum.eth.sync.backwardsync.BackwardSyncContext;
import org.hyperledger.besu.ethereum.eth.sync.state.SyncState;
Expand All @@ -43,13 +45,17 @@ public class TransitionBackwardSyncContext extends BackwardSyncContext {
public TransitionBackwardSyncContext(
final ProtocolContext protocolContext,
final TransitionProtocolSchedule transitionProtocolSchedule,
final PeerTaskExecutor peerTaskExecutor,
final SynchronizerConfiguration synchronizerConfiguration,
final MetricsSystem metricsSystem,
final EthContext ethContext,
final SyncState syncState,
final StorageProvider storageProvider) {
super(
protocolContext,
transitionProtocolSchedule,
peerTaskExecutor,
synchronizerConfiguration,
metricsSystem,
ethContext,
syncState,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ public InvalidPeerTaskResponseException() {
super();
}

public InvalidPeerTaskResponseException(final String message) {
super(message);
}

public InvalidPeerTaskResponseException(final Throwable cause) {
super(cause);
}
Expand Down
Loading
Loading