Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(indexer/server-v2): Listener integration in server/v2 #21917

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

cool-develope
Copy link
Contributor

@cool-develope cool-develope commented Sep 26, 2024

Description

Closes: #21312
ref: #21527


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title, you can find examples of the prefixes below:
  • confirmed ! in the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • reviewed "Files changed" and left comments if necessary
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • updated the relevant documentation or specification, including comments for documenting Go code
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic, API design and naming, documentation is accurate, tests and test coverage

Summary by CodeRabbit

  • New Features

    • Enhanced Consensus struct to support a listener for streaming various block-related data.
    • Introduced new method SetListener for setting the listener in the Consensus struct.
  • Improvements

    • Improved transaction and message index tracking in the DeliverBlock process.
  • Dependencies

    • Added a direct dependency on cosmossdk.io/schema v0.3.0.

Copy link
Contributor

coderabbitai bot commented Sep 26, 2024

📝 Walkthrough

Walkthrough

The pull request introduces enhancements to the Consensus struct by adding a listener field and a method for setting it. It establishes a direct dependency on the cosmossdk.io/schema package, facilitating the use of new event models. Modifications to the streamDeliverBlockChanges method enable streaming various block-related data to the listener. Additionally, changes in the STF structure improve tracking of transaction and message indices. Comments in the coreRouterImpl suggest future enhancements for event handling.

Changes

File Change Summary
server/v2/cometbft/abci.go Added listener *appdata.Listener field and SetListener method to Consensus struct.
server/v2/cometbft/go.mod Added direct dependency on cosmossdk.io/schema v0.3.0 and removed its indirect dependency.
server/v2/cometbft/streaming.go Enhanced streamDeliverBlockChanges to stream multiple data types to a listener if not nil.
server/v2/stf/stf.go Modified executionContext struct to include txIndex, msgIndex, and eventIndex fields.
server/v2/stf/stf_router.go Added comment in Invoke method regarding handling of ctx.eventIndex.

Assessment against linked issues

Objective Addressed Explanation
Consider using schema/appdata event model in core/app (#[21312])
Integrate appdata.Listener in STF (#[21527]) Integration location is not finalized yet.

Possibly related issues

Possibly related PRs

Suggested labels

C:server/v2, C:server/v2 cometbft, C:collections, C:Store, C:indexer/postgres

Suggested reviewers

  • facundomedica
  • sontrinh16
  • aaronc
  • julienrbrt
  • tac0turtle

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@cool-develope your pull request is missing a changelog!

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Outside diff range and nitpick comments (9)
server/v2/cometbft/abci.go (3)

43-43: New field added correctly, consider adding documentation.

The listener field is appropriately added to the Consensus struct. However, it would be beneficial to add a comment explaining the purpose and usage of this field.

Consider adding a comment above the listener field to explain its purpose, for example:

// listener is used to stream block-related data changes.
listener *appdata.Listener

108-111: New method added correctly, consider improvements.

The SetListener method is appropriately implemented. However, consider the following suggestions:

  1. Add a comment to explain the purpose of this method.
  2. Consider adding a nil check to handle cases where a nil listener is passed.

Here's a suggested improvement:

// SetListener sets the listener for the consensus module.
// It panics if a nil listener is provided.
func (c *Consensus[T]) SetListener(l *appdata.Listener) {
    if l == nil {
        panic("cannot set nil listener")
    }
    c.listener = l
}

This change adds documentation and ensures that a nil listener cannot be set, which could lead to runtime errors if not handled properly.


22-22: Overall, changes look good but documentation could be improved.

The additions of the appdata import, listener field, and SetListener method are well-implemented and integrate smoothly with the existing code. They follow Go conventions and best practices.

To enhance code maintainability and clarity, consider adding documentation comments for the new listener field and SetListener method. This will help other developers understand the purpose and usage of these new additions.

Also applies to: 43-43, 108-111

server/v2/stf/stf.go (4)

200-200: Offer assistance with handling msgIndex and eventIndex in transaction processing

There are TODO comments at lines 200 and 284 indicating uncertainty on how to handle msgIndex and eventIndex. I can help implement the logic to properly track and handle these indices during transaction execution.

Would you like me to provide a proposed implementation or open a GitHub issue to track this task?

Also applies to: 284-284


209-209: Offer assistance with integrating execCtx.txIndex with events

The TODO comment at line 209 suggests handling execCtx.txIndex with events. I can help implement the necessary changes to ensure that txIndex is correctly associated with events.

Would you like me to provide a proposed implementation or open a GitHub issue to track this task?


325-325: Offer assistance with handling execCtx.msgIndex in event handling

The TODO comment at line 325 indicates that execCtx.msgIndex should be handled with events. I can help implement the logic to associate msgIndex with events during message execution.

Would you like me to provide a proposed implementation or open a GitHub issue to track this task?


500-502: Offer assistance with passing index fields to handlers

The executionContext struct now includes txIndex, msgIndex, and eventIndex. There's a TODO comment at line 502 questioning how to pass eventIndex to the handlers. I can help design and implement a solution to pass these indices appropriately.

Would you like me to provide a proposed implementation or open a GitHub issue to track this task?

server/v2/cometbft/streaming.go (2)

63-64: Address TODOs: Provide HeaderBytes and HeaderJSON

The HeaderBytes and HeaderJSON fields in appdata.StartBlockData are currently set to nil with TODO comments. Providing the actual header bytes and JSON is important for the listener to receive complete block information.

Would you like assistance in implementing the serialization of the header data? I can help generate the necessary code or open a GitHub issue to track this task.


75-75: Address TODO: Provide transaction JSON representation

The JSON field in appdata.TxData is set to nil with a TODO comment. Supplying the JSON representation of transactions is crucial for the listener to process transaction data effectively.

Would you like assistance in implementing the JSON serialization of transactions? I can help generate the necessary code or open a GitHub issue to track this task.

📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 3856e77 and 6d8c45c.

📒 Files selected for processing (5)
  • server/v2/cometbft/abci.go (3 hunks)
  • server/v2/cometbft/go.mod (1 hunks)
  • server/v2/cometbft/streaming.go (2 hunks)
  • server/v2/stf/stf.go (8 hunks)
  • server/v2/stf/stf_router.go (1 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
server/v2/cometbft/abci.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

server/v2/cometbft/streaming.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

server/v2/stf/stf.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

server/v2/stf/stf_router.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

🔇 Additional comments not posted (2)
server/v2/cometbft/go.mod (1)

24-24: LGTM: Addition of direct dependency aligns with PR objectives.

The addition of cosmossdk.io/schema v0.3.0 as a direct dependency is appropriate and aligns with the PR objectives of utilizing cosmossdk.io/schema/appdata events. This change suggests that the codebase now explicitly uses features from this package.

To ensure this dependency is being used effectively in the codebase, please run the following command:

This will help verify that the newly added dependency is actually being imported and used in the code.

✅ Verification successful

Dependency Usage Verified and Approved

The added direct dependency on cosmossdk.io/schema v0.3.0 is utilized across multiple files, including core/event/event.go where cosmossdk.io/schema/appdata is imported. This verifies that the dependency is effectively used in the codebase.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for imports of cosmossdk.io/schema in the codebase
rg --type go 'import.*cosmossdk.io/schema'

Length of output: 505

server/v2/cometbft/abci.go (1)

22-22: Import statement looks good.

The new import for appdata is correctly placed and follows Go conventions for import grouping.

@@ -165,6 +165,6 @@ func (r coreRouterImpl) Invoke(ctx context.Context, req transaction.Msg) (res tr
if !exists {
return nil, fmt.Errorf("%w: %s", ErrNoHandler, typeName)
}

// TODO: should handle ctx.eventIndex, eventIndex should be embedded as a value in the context???
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider creating an issue for the suggested enhancement instead of using a TODO comment.

The TODO comment suggests an important enhancement for handling ctx.eventIndex by embedding it as a value in the context. This could potentially improve event tracking and processing throughout the system. However, inline TODO comments are generally discouraged in production code.

Instead of keeping this TODO comment:

  1. Create a GitHub issue to track this enhancement, providing more details about the proposed change and its implications.
  2. Remove the TODO comment from the code.
  3. If necessary, add a comment referencing the created issue for context.

This approach aligns better with best practices for code maintenance and ensures the proposed enhancement is properly tracked and addressed.

@@ -111,6 +111,7 @@ func (s STF[T]) DeliverBlock(

// reset events
exCtx.events = make([]event.Event, 0)
exCtx.txIndex = 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider consolidating exCtx.txIndex initialization

exCtx.txIndex is reset to 0 at lines 114, 127, and 145. If possible, consider initializing or resetting txIndex once to reduce code duplication and improve maintainability.

Also applies to: 127-127, 145-145

Comment on lines +95 to +102
if completionCallback, err := c.listener.Commit(appdata.CommitData{}); err != nil {
return err
} else if completionCallback != nil {
if err := completionCallback(); err != nil {
return err
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Refactor to simplify control flow and enhance readability

The else block after a return statement is unnecessary and can be refactored for clarity. According to the Uber Go Style Guide, it's preferable to avoid else statements when the if block ends with a return.

Apply this diff to refactor the code:

-		if completionCallback, err := c.listener.Commit(appdata.CommitData{}); err != nil {
-			return err
-		} else if completionCallback != nil {
-			if err := completionCallback(); err != nil {
-				return err
-			}
-		}
+		completionCallback, err := c.listener.Commit(appdata.CommitData{})
+		if err != nil {
+			return err
+		}
+		if completionCallback != nil {
+			if err := completionCallback(); err != nil {
+				return err
+			}
+		}

This refactoring removes the unnecessary else and enhances the readability of the control flow.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if completionCallback, err := c.listener.Commit(appdata.CommitData{}); err != nil {
return err
} else if completionCallback != nil {
if err := completionCallback(); err != nil {
return err
}
}
}
completionCallback, err := c.listener.Commit(appdata.CommitData{})
if err != nil {
return err
}
if completionCallback != nil {
if err := completionCallback(); err != nil {
return err
}
}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature]: Consider using schema/appdata event model in core/app
1 participant