Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add transaction type to DynamoDB item for a state store transaction held in S3 #3679

Open
patchwork01 opened this issue Nov 12, 2024 · 0 comments

Comments

@patchwork01
Copy link
Collaborator

patchwork01 commented Nov 12, 2024

Background

Split from:

Related to:

Description

We'd like to be able to decide to skip transactions in the log based on the transaction types we care about for a given query.

We'd like to ensure that we will never need to load a transaction from S3 to do this, and it can be included in a filter for DynamoDB.

Analysis

We can ensure that a transaction record in DynamoDB or S3 always corresponds to a single type of transaction, even if we combine multiple transactions together.

In the DynamoDB item we can separate the transaction type from the fact of whether it's held in the DynamoDB item or in S3.

This will need to be done in DynamoDBTransactionLogStore.

Note that if we combine transactions into one in an invocation of the state store committer lambda (in #2992), but we never combine transactions of different types, that means instead of one transaction added to the log per lambda invocation, we would have as many as there are different types of transaction. This should still allow us to get the throughput we need, and we could change our minds about it later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant