Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(batcher): impl default to ExecutionConfig for addition to Batch… #1277

Merged

Conversation

ayeletstarkware
Copy link
Contributor

@ayeletstarkware ayeletstarkware commented Oct 9, 2024

…erConfig


This change is Reviewable

Copy link

codecov bot commented Oct 9, 2024

Codecov Report

Attention: Patch coverage is 2.17391% with 45 lines in your changes missing coverage. Please review.

Project coverage is 55.13%. Comparing base (b0cfe82) to head (4bd5d2e).
Report is 375 commits behind head on main.

Files with missing lines Patch % Lines
crates/blockifier/src/bouncer.rs 0.00% 25 Missing ⚠️
crates/batcher/src/block_builder.rs 0.00% 12 Missing ⚠️
crates/blockifier/src/versioned_constants.rs 0.00% 7 Missing ⚠️
crates/batcher/src/batcher.rs 0.00% 1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (b0cfe82) and HEAD (4bd5d2e). Click for more details.

HEAD has 2 uploads less than BASE
Flag BASE (b0cfe82) HEAD (4bd5d2e)
3 1
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1277       +/-   ##
===========================================
- Coverage   74.18%   55.13%   -19.06%     
===========================================
  Files         359      148      -211     
  Lines       36240    17306    -18934     
  Branches    36240    17306    -18934     
===========================================
- Hits        26886     9541    -17345     
- Misses       7220     7343      +123     
+ Partials     2134      422     -1712     
Flag Coverage Δ
?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ayeletstarkware ayeletstarkware force-pushed the ayelet/batcher/add-default-to-execution-config branch from 8a69180 to 1475c9e Compare October 10, 2024 06:10
@ayeletstarkware ayeletstarkware changed the base branch from main to ayelet/blockifier/bouncer-config/replace-default-derive-by-create-for-testing October 10, 2024 06:10
@ayeletstarkware ayeletstarkware force-pushed the ayelet/batcher/add-default-to-execution-config branch from 1475c9e to 08fa221 Compare October 10, 2024 06:11
@ayeletstarkware ayeletstarkware force-pushed the ayelet/blockifier/bouncer-config/replace-default-derive-by-create-for-testing branch from 8988235 to 06611f3 Compare October 10, 2024 07:18
@ayeletstarkware ayeletstarkware force-pushed the ayelet/batcher/add-default-to-execution-config branch 2 times, most recently from d4d4bcf to 540cd31 Compare October 10, 2024 09:53
@ayeletstarkware ayeletstarkware changed the base branch from ayelet/blockifier/bouncer-config/replace-default-derive-by-create-for-testing to ayelet/blockifier/replace-default-derive-by-empty October 10, 2024 09:54
@ayeletstarkware ayeletstarkware force-pushed the ayelet/batcher/add-default-to-execution-config branch from 540cd31 to b0600ce Compare October 10, 2024 11:10
Copy link
Collaborator

@noaov1 noaov1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please give more context? :)
Are we planning to use the default values in the regular flow?
Can we add a config to allow an easy modification to the values? (eliminates the need to re-publish)

Reviewed 2 of 5 files at r1, all commit messages.
Reviewable status: 2 of 5 files reviewed, all discussions resolved (waiting on @Yael-Starkware)

Copy link
Contributor

@Yael-Starkware Yael-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 5 files at r1, all commit messages.
Reviewable status: 4 of 5 files reviewed, 6 unresolved discussions (waiting on @ayeletstarkware and @noaov1)


crates/blockifier/src/bouncer.rs line 136 at r1 (raw file):

        Self {
            gas: 4950000,
            n_steps: 20000000,

I see 40000000 in some starknet config files.
lets verify all numbers with @noaov1

Code quote:

20000000

crates/blockifier/src/bouncer.rs line 137 at r1 (raw file):

            gas: 4950000,
            n_steps: 20000000,
            message_segment_length: 3700,

I see the @Yoni-Starkware recently changed it from 3750 to 3700, did you check with him?

Code quote:

3700

crates/blockifier/src/bouncer.rs line 139 at r1 (raw file):

            message_segment_length: 3700,
            n_events: 5000,
            state_diff_size: 20000,

Suggestion:

//TODO: override this value when using kzg_da
state_diff_size: 20000,

crates/blockifier/src/bouncer.rs line 140 at r1 (raw file):

            n_events: 5000,
            state_diff_size: 20000,
            builtin_count: BuiltinCount::default(),

please implement this default as well.

Code quote:

builtin_count: BuiltinCount::default()

crates/blockifier/src/versioned_constants.rs line 868 at r1 (raw file):

            validate_max_n_steps: 1000000,
            max_recursion_depth: 50,
            invoke_tx_max_n_steps: 4000000,

I think it should be 10000000 according to starknet version 0_13_3
did you ask @noaov1 ?

Suggestion:

10000000

crates/blockifier/src/blockifier/config.rs line 32 at r1 (raw file):

impl Default for ConcurrencyConfig {
    fn default() -> Self {
        Self { enabled: true, n_workers: 28, chunk_size: 100 }

this was the number provided by @noaov1 ?

Code quote:

28

Copy link
Collaborator

@Yoni-Starkware Yoni-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 4 of 5 files reviewed, 6 unresolved discussions (waiting on @ayeletstarkware, @noaov1, @Yael-Starkware, and @Yoni-Starkware)


crates/blockifier/src/bouncer.rs line 137 at r1 (raw file):

Previously, Yael-Starkware (YaelD) wrote…

I see the @Yoni-Starkware recently changed it from 3750 to 3700, did you check with him?

3700 if fine (very high)


crates/blockifier/src/bouncer.rs line 139 at r1 (raw file):

            message_segment_length: 3700,
            n_events: 5000,
            state_diff_size: 20000,

Let it be 4000 in any case.


crates/blockifier/src/versioned_constants.rs line 868 at r1 (raw file):

Previously, Yael-Starkware (YaelD) wrote…

I think it should be 10000000 according to starknet version 0_13_3
did you ask @noaov1 ?

Why do you need to put concrete values here?
The override is for overriding the VC values, primarily for appchains or for bugfixes.
I.e., it should be passed as Option

Copy link
Contributor

@Yael-Starkware Yael-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes we plan to use these in the regular flow.
These are the default values and can be overridden by a config file.

Reviewable status: 4 of 5 files reviewed, 6 unresolved discussions (waiting on @ArniStarkware, @ayeletstarkware, and @noaov1)

Copy link
Contributor

@Yael-Starkware Yael-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 4 of 5 files reviewed, 6 unresolved discussions (waiting on @ArniStarkware, @ayeletstarkware, and @noaov1)


crates/blockifier/src/versioned_constants.rs line 868 at r1 (raw file):

Previously, Yoni-Starkware (Yoni) wrote…

Why do you need to put concrete values here?
The override is for overriding the VC values, primarily for appchains or for bugfixes.
I.e., it should be passed as Option

I fully agree, but this is the existing blockifier code.

Copy link
Contributor

@Yael-Starkware Yael-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 4 of 5 files reviewed, 6 unresolved discussions (waiting on @ArniStarkware, @ayeletstarkware, and @noaov1)


crates/blockifier/src/versioned_constants.rs line 868 at r1 (raw file):

Previously, Yael-Starkware (YaelD) wrote…

I fully agree, but this is the existing blockifier code.

we can do this as a separate PR.

Copy link
Contributor

@Yael-Starkware Yael-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 4 of 5 files reviewed, 7 unresolved discussions (waiting on @ArniStarkware, @ayeletstarkware, and @noaov1)


crates/batcher/src/block_builder.rs line 65 at r1 (raw file):

            chain_info: ChainInfo::default(),
            execute_config: TransactionExecutorConfig::default(),
            bouncer_config: BouncerConfig::empty(),

why empty?

Code quote:

bouncer_config: BouncerConfig::empty()

@ayeletstarkware ayeletstarkware force-pushed the ayelet/blockifier/replace-default-derive-by-empty branch 2 times, most recently from 0de31a9 to 78ae122 Compare October 13, 2024 13:18
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 4 of 5 files reviewed, 7 unresolved discussions (waiting on @ArniStarkware, @noaov1, and @Yael-Starkware)


crates/batcher/src/block_builder.rs line 65 at r1 (raw file):

Previously, Yael-Starkware (YaelD) wrote…

why empty?

A mistake. Done.


crates/blockifier/src/bouncer.rs line 136 at r1 (raw file):

Previously, Yael-Starkware (YaelD) wrote…

I see 40000000 in some starknet config files.
lets verify all numbers with @noaov1

used 20000000


crates/blockifier/src/bouncer.rs line 137 at r1 (raw file):

Previously, Yoni-Starkware (Yoni) wrote…

3700 if fine (very high)

Done.


crates/blockifier/src/bouncer.rs line 139 at r1 (raw file):

Previously, Yoni-Starkware (Yoni) wrote…

Let it be 4000 in any case.

Done.


crates/blockifier/src/bouncer.rs line 140 at r1 (raw file):

Previously, Yael-Starkware (YaelD) wrote…

please implement this default as well.

Done.


crates/blockifier/src/versioned_constants.rs line 868 at r1 (raw file):

Previously, Yael-Starkware (YaelD) wrote…

we can do this as a separate PR.

changed to 10000000. I will create a separate PR for passing VersionedConstantsOverrides as an option.


crates/blockifier/src/blockifier/config.rs line 32 at r1 (raw file):

Previously, Yael-Starkware (YaelD) wrote…

this was the number provided by @noaov1 ?

stale

@ayeletstarkware ayeletstarkware force-pushed the ayelet/batcher/add-default-to-execution-config branch from b0600ce to 460a98d Compare October 14, 2024 08:37
@ayeletstarkware ayeletstarkware changed the base branch from ayelet/blockifier/replace-default-derive-by-empty to main October 14, 2024 08:37
Copy link
Collaborator

@Yoni-Starkware Yoni-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 5 files reviewed, 8 unresolved discussions (waiting on @ArniStarkware, @ayeletstarkware, @noaov1, and @Yael-Starkware)


crates/batcher/src/block_builder.rs line 49 at r2 (raw file):

#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)]
pub struct ExecutionConfig {

I suggest a more specific name. Execution is used too much in our code

Suggestion:

BlockBuilderConfig

Copy link
Collaborator

@Yoni-Starkware Yoni-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 5 files reviewed, 10 unresolved discussions (waiting on @ArniStarkware, @ayeletstarkware, @noaov1, and @Yael-Starkware)


crates/blockifier/src/bouncer.rs line 135 at r2 (raw file):

    fn default() -> Self {
        Self {
            gas: 2500000,

Why?

Code quote:

gas: 2500000,

crates/blockifier/src/bouncer.rs line 303 at r2 (raw file):

            poseidon: 78125,
            range_check: 156250,
            range_check96: 0,

There should not be zeros here. Where did it come from?

Code quote:

            add_mod: 0,
            bitwise: 39062,
            ecdsa: 1220,
            ec_op: 2441,
            keccak: 0,
            mul_mod: 0,
            pedersen: 78125,
            poseidon: 78125,
            range_check: 156250,
            range_check96: 0,

@ArniStarkware
Copy link
Contributor

Non-blocking. If you have more comments for address in this PR, please fix the creation of execution_config in batcher.rs:

let execution_config = ExecutionConfig {

Should turn into:

let execution_config = ExecutionConfig::default();

Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 5 files reviewed, 11 unresolved discussions (waiting on @ArniStarkware, @noaov1, @Yael-Starkware, and @Yoni-Starkware)


crates/blockifier/src/bouncer.rs line 135 at r2 (raw file):

Previously, Yoni-Starkware (Yoni) wrote…

Why?

All values, except state_diff_size: 4000 (as you requested), were taken from starkware/starknet/services/batcher/config.yml as Lior suggested. These are placeholder values to get the program running and will be updated later.


crates/blockifier/src/bouncer.rs line 303 at r2 (raw file):

Previously, Yoni-Starkware (Yoni) wrote…

There should not be zeros here. Where did it come from?

The builtins were calculated as n_steps (2500000) / ratio from src/starkware/cairo/lang/instances_external.py. I set unlisted builtins to zero, which I think won’t stop the program from running, right?

Copy link
Collaborator

@Yoni-Starkware Yoni-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 5 files reviewed, 10 unresolved discussions (waiting on @ArniStarkware, @ayeletstarkware, @noaov1, and @Yael-Starkware)


crates/blockifier/src/bouncer.rs line 135 at r2 (raw file):

Previously, ayeletstarkware (Ayelet Zilber) wrote…

All values, except state_diff_size: 4000 (as you requested), were taken from starkware/starknet/services/batcher/config.yml as Lior suggested. These are placeholder values to get the program running and will be updated later.

Ok

Copy link
Collaborator

@Yoni-Starkware Yoni-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 5 files reviewed, 10 unresolved discussions (waiting on @ArniStarkware, @ayeletstarkware, @noaov1, and @Yael-Starkware)


crates/blockifier/src/bouncer.rs line 303 at r2 (raw file):

Previously, ayeletstarkware (Ayelet Zilber) wrote…

The builtins were calculated as n_steps (2500000) / ratio from src/starkware/cairo/lang/instances_external.py. I set unlisted builtins to zero, which I think won’t stop the program from running, right?

It would fail txs the use these builtins (even in tests it's not good)
use starknet_with_keccak_instance ratios and use the ratio of range_check for
"add_mod_builtin", "mul_mod_builtin", "range_check96_builtin"

@ayeletstarkware ayeletstarkware force-pushed the ayelet/batcher/add-default-to-execution-config branch from 460a98d to c573f0b Compare October 14, 2024 10:39
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 9 unresolved discussions (waiting on @ArniStarkware, @noaov1, @Yael-Starkware, and @Yoni-Starkware)


crates/blockifier/src/bouncer.rs line 303 at r2 (raw file):

Previously, Yoni-Starkware (Yoni) wrote…

It would fail txs the use these builtins (even in tests it's not good)
use starknet_with_keccak_instance ratios and use the ratio of range_check for
"add_mod_builtin", "mul_mod_builtin", "range_check96_builtin"

Done.

Copy link
Contributor

@ArniStarkware ArniStarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 2 files at r3.
Reviewable status: 1 of 6 files reviewed, 9 unresolved discussions (waiting on @noaov1, @Yael-Starkware, and @Yoni-Starkware)

Copy link
Contributor

@Yael-Starkware Yael-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 5 files at r1, 2 of 4 files at r2, 1 of 2 files at r3, all commit messages.
Reviewable status: 4 of 6 files reviewed, 2 unresolved discussions (waiting on @ayeletstarkware, @noaov1, and @Yoni-Starkware)


crates/blockifier/src/bouncer.rs line 137 at r1 (raw file):

Previously, ayeletstarkware (Ayelet Zilber) wrote…

Done.

I still see 3750

Copy link
Contributor

@Yael-Starkware Yael-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 4 of 6 files reviewed, 3 unresolved discussions (waiting on @ayeletstarkware, @noaov1, and @Yoni-Starkware)


crates/blockifier/src/bouncer.rs line 291 at r3 (raw file):

}

impl Default for BuiltinCount {

please add a todo for each struct that doesn't represent the real production values, so we won't forget to fix it in the future.

Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 4 of 6 files reviewed, 1 unresolved discussion (waiting on @noaov1 and @Yael-Starkware)


crates/blockifier/src/bouncer.rs line 137 at r1 (raw file):

Previously, Yael-Starkware (YaelD) wrote…

I still see 3750

Done


crates/blockifier/src/bouncer.rs line 291 at r3 (raw file):

Previously, Yael-Starkware (YaelD) wrote…

please add a todo for each struct that doesn't represent the real production values, so we won't forget to fix it in the future.

Done.

@ayeletstarkware ayeletstarkware force-pushed the ayelet/batcher/add-default-to-execution-config branch from c573f0b to 4bd5d2e Compare October 14, 2024 12:22
Copy link
Contributor

@Yael-Starkware Yael-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 5 of 5 files at r4, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @noaov1 and @Yoni-Starkware)

Copy link
Collaborator

@Yoni-Starkware Yoni-Starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 5 files at r4, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @noaov1)

Copy link
Contributor

@ArniStarkware ArniStarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @noaov1)

@ArniStarkware
Copy link
Contributor

-- commits line 2 at r4:
Extremely non-blocking. (you can fix the merge massage)

Suggestion:

impl default to BlockBuilderConfig towards implementation of BatcherConfig

@ayeletstarkware ayeletstarkware merged commit f15ac96 into main Oct 15, 2024
11 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Oct 16, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants