Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement BufferedData Bytes and Unsafe operations wrappers. #89

Closed
wants to merge 7 commits into from

Conversation

LLITCHEV
Copy link
Contributor

Implementation of BufferedData subclass to do fast operations on read-only Bytes data and Unsafe operations for Direct buffer.

Description:

Related issue(s):

Fixes #

Notes for reviewer:

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, etc.)

Implementation of BufferedData subclass to do fast operations on read-only Bytes data and Unsafe operations for Direct buffer.

Signed-off-by: Lubomir Litchev <lubomir@swirldslabs.com>
@LLITCHEV LLITCHEV self-assigned this Aug 23, 2023
@github-actions
Copy link

github-actions bot commented Aug 23, 2023

JUnit Test Report

     38 files       38 suites   44s ⏱️
   493 tests    492 ✔️ 1 💤 0
2 681 runs  2 672 ✔️ 9 💤 0

Results for commit b9c7e0c.

♻️ This comment has been updated with latest results.

@LLITCHEV LLITCHEV mentioned this pull request Aug 23, 2023
2 tasks
@github-actions
Copy link

github-actions bot commented Aug 23, 2023

Integration Test Report

86 721 tests   86 721 ✔️  3m 15s ⏱️
     209 suites           0 💤
     209 files             0

Results for commit b9c7e0c.

♻️ This comment has been updated with latest results.

This was linked to issues Aug 23, 2023
Without no changes
—————————-
Benchmark                         Mode  Cnt   Score   Error  Units
VarIntBench.dataBuffer            avgt    5   5.783 ± 0.025  ns/op
VarIntBench.dataBufferDirect      avgt    5   2.827 ± 0.009  ns/op
VarIntBench.dataBytes             avgt    5   3.189 ± 0.025  ns/op
VarIntBench.dataInputStream       avgt    5  18.746 ± 0.060  ns/op
VarIntBench.google                avgt    5   1.135 ± 0.006  ns/op
VarIntBench.googleDirect          avgt    5   0.980 ± 0.002  ns/op
VarIntBench.googleSlowPath        avgt    5   3.477 ± 0.027  ns/op
VarIntBench.googleSlowPathDirect  avgt    5   2.865 ± 0.023  ns/op
VarIntBench.richard               avgt    5   4.225 ± 0.021  ns/op

With these changes
———————
Benchmark                         Mode  Cnt   Score   Error  Units
VarIntBench.dataBuffer            avgt    5   5.736 ± 0.024  ns/op
VarIntBench.dataBufferDirect      avgt    5   1.540 ± 0.008  ns/op
VarIntBench.dataBytes             avgt    5   2.217 ± 0.025  ns/op
VarIntBench.dataInputStream       avgt    5  18.734 ± 0.060  ns/op
VarIntBench.google                avgt    5   1.135 ± 0.006  ns/op
VarIntBench.googleDirect          avgt    5   1.006 ± 0.005  ns/op
VarIntBench.googleSlowPath        avgt    5   3.479 ± 0.025  ns/op
VarIntBench.googleSlowPathDirect  avgt    5   2.867 ± 0.018  ns/op
VarIntBench.richard               avgt    5   4.104 ± 0.026  ns/op

Signed-off-by: Lubomir Litchev <lubomir@swirldslabs.com>
Lubomir Litchev added 3 commits September 5, 2023 08:01
Signed-off-by: Lubomir Litchev <lubomir@swirldslabs.com>
Change the test to allocate a direct buffer.

Signed-off-by: Lubomir Litchev <lubomir@swirldslabs.com>
Signed-off-by: Lubomir Litchev <lubomir@swirldslabs.com>
Lubomir Litchev added 2 commits September 15, 2023 13:57
Signed-off-by: Lubomir Litchev <lubomir@swirldslabs.com>
Signed-off-by: Lubomir Litchev <lubomir@swirldslabs.com>
start = pos;
}

private ReadWriteDirectUnsafeByteBuffer(@NonNull final ByteBuffer bufferIn, final long startIn, final long limitIn) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Codacy found a minor Code Style issue: Line is longer than 120 characters (found 125).

The code provided has a line that exceeds the maximum length of 120 characters as defined by the Checkstyle rule for maximum line length. This can make the code harder to read, especially on smaller screens or when using side-by-side diff views in version control systems.

To fix this issue, you can break the line into multiple lines. This often involves wrapping method parameters or chaining method calls to conform to the line length limit. Here's how you can refactor the constructor to keep each line under 120 characters:

Suggested change
private ReadWriteDirectUnsafeByteBuffer(@NonNull final ByteBuffer bufferIn, final long startIn, final long limitIn) {
private ReadWriteDirectUnsafeByteBuffer(@NonNull final ByteBuffer bufferIn,
final long startIn, final long limitIn) {
buffer = bufferIn;
address = UnsafeUtils.addressOffset(buffer);
limit = limitIn;
start = startIn;
pos = start;
}

By breaking the line after the first parameter, the code remains within the 120-character limit, making it compliant with the Checkstyle rule.


This comment was generated by an experimental AI tool.

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

import java.nio.ByteBuffer;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Codacy found a minor Code Style issue: Unused import - java.nio.ByteBuffer.

The code includes an import statement for java.nio.ByteBuffer but does not actually use the ByteBuffer class anywhere within the UnsafeUtilsTest class. Unused imports can clutter the codebase, make it harder to read, and can sometimes cause confusion about dependencies. It's a good practice to remove any import statements that are not used to keep the code clean and maintainable.

To fix the issue, you should remove the unused import statement. Here is the suggested change:

Suggested change
import java.nio.ByteBuffer;
// Removed the unused import statement
// import java.nio.ByteBuffer;

After removing the unused import, the code should look like this:

package com.hedera.pbj.runtime.io.buffer;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.*;

final class UnsafeUtilsTest {
    @Test
    @DisplayName("Supports Unsafe Operations")
    void supportsUnsafeOperations() {
        assertTrue(UnsafeUtils.hasUnsafeByteBufferOperations());
    }
}

This comment was generated by an experimental AI tool.

/** The unsafe address of the current read limit of the buffer. */
private long limit;
/** The unsafe address of the current read position of the buffer. */
private long pos;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Codacy found a minor Code Style issue: 'VARIABLE_DEF' should be separated from previous line.

The issue raised by Checkstyle is regarding the code style and formatting. According to the Checkstyle rules being used, there should be a blank line separating the declaration of the instance variable pos from the previous block of code or variable declaration.

Checkstyle is a tool used to enforce a coding standard or style guide. It ensures that Java code adheres to a set of predefined rules or patterns. In this case, the rule is that there should be a blank line separating variable definitions for readability and organization.

To fix this issue, you should insert a blank line before the declaration of the pos variable. Here's how the corrected code should look:

Suggested change
private long pos;
private long limit;
private long pos;

By adding a blank line before private long pos;, you adhere to the Checkstyle rule that requires a separation between variable definitions.


This comment was generated by an experimental AI tool.

JvmMemoryAccessor(sun.misc.Unsafe unsafe) {
super(unsafe);
}
@Override
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Codacy found a minor Code Style issue: 'METHOD_DEF' should be separated from previous line.

The issue reported by Checkstyle for the given code is that the method definition is not properly separated from the previous line. According to Java code style conventions, there should be a blank line before the start of a method definition to enhance readability and maintain a clean separation between methods and other code blocks.

To fix this issue, simply add a blank line before the @Override annotation to separate the method definition from the previous code block. Here is the corrected code snippet:

Suggested change
@Override
@Override
public boolean supportsUnsafeArrayOperations() {
if (!super.supportsUnsafeArrayOperations()) {
return false;
}
// ... rest of the code remains unchanged

Adding this blank line will ensure that the code complies with the style rule enforced by Checkstyle and improves the readability of the code.


This comment was generated by an experimental AI tool.

^ (~0L << 49)
^ (~0L << 56);
if (x < 0L) {
if (buffer[tempPos++] < 0L) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Codacy found a medium Error Prone issue: These nested if statements could be combined

The PMD linter is indicating that the nested if statements can be combined to make the code more concise and potentially easier to read. Combining nested if statements that do not have else clauses can reduce the nesting level and improve the clarity of the code.

In your code snippet, the nested if statement checks if x < 0L and then within that block checks if buffer[tempPos++] < 0L. These two conditions can be combined into a single if statement using the logical AND operator &&.

Here's the problematic part of your code with a suggestion on how to combine the if statements:

if (x < 0L) {
    if (buffer[tempPos++] < 0L) {
        return readVarIntLongSlow(zigZag);
    }
}

Here's how you can combine the two if statements:

Suggested change
if (buffer[tempPos++] < 0L) {
if (x < 0L && buffer[tempPos++] < 0L) {
return readVarIntLongSlow(zigZag);
}

By combining the conditions, you remove one level of nesting, which simplifies the control flow and makes the intention of the code clearer.


This comment was generated by an experimental AI tool.

/** BufferedData subclass that encapsulates immutable {@link Bytes} object */
static public final class ReadOnlyByteBuffer extends BufferedData {
final Bytes bytes;
private final byte[] buffer;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Codacy found a minor Code Style issue: 'VARIABLE_DEF' should be separated from previous line.

The Checkstyle issue here is about the code style, specifically regarding the placement of variable declarations. According to the coding standards enforced by the Checkstyle configuration being used, there should be a blank line separating a variable declaration from the previous block of code.

In Java, it is common to separate logical code blocks with blank lines for better readability. This includes separating variable definitions from previous blocks of code. The Checkstyle linter is flagging the line with private final byte[] buffer; because it expects a blank line before it to visually separate it from the preceding block.

To fix this issue, simply add a blank line before the variable declaration. Here's the corrected code snippet:

Suggested change
private final byte[] buffer;
private final byte[] buffer;

Adding this blank line will resolve the Checkstyle warning about the variable definition needing separation from the previous line.


This comment was generated by an experimental AI tool.

/** The unsafe address of the content of {@link #buffer}. */
private final long address;
/** The unsafe address of the current read limit of the buffer. */
private long limit;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Codacy found a minor Code Style issue: 'VARIABLE_DEF' should be separated from previous line.

The Checkstyle issue reported here is related to the formatting of the variable definitions in the Java code. Checkstyle enforces a style where each variable definition should be separated by a blank line to improve readability and maintainability. In the provided code snippet, the variable limit is not preceded by a blank line, which violates this rule.

To fix this issue, you should simply add a blank line before the limit variable definition. This will ensure that each variable definition is visually separated from the previous block of code, making it easier to read and understand the structure of the class.

Here's the corrected code snippet with the required blank line added:

Suggested change
private long limit;
private long limit;

This comment was generated by an experimental AI tool.

*/
@Override
public void writeInt(final int value, @NonNull final ByteOrder byteOrder) {
if (byteOrder == ByteOrder.BIG_ENDIAN) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Codacy found a medium Error Prone issue: Use equals() to compare object references.

The code provided is using the == operator to compare ByteOrder object references. In Java, the == operator checks if two references point to the exact same object, not if their values are equal. While ByteOrder is an enum and using == for enum comparisons is actually safe, PMD does not differentiate and suggests using .equals() for consistency and to avoid potential errors when dealing with non-enum objects.

In this specific case, since ByteOrder is an enum, using == is acceptable and will not cause any issues. Enums in Java are singletons, meaning that there is only one instance of each enum constant, so using == to compare these constants is perfectly fine.

However, if you want to follow PMD's recommendation for the sake of consistency or to prevent future errors if the code is modified to use non-enum types, you can replace == with .equals(). Here's how you can fix the code:

Suggested change
if (byteOrder == ByteOrder.BIG_ENDIAN) {
if (byteOrder.equals(ByteOrder.BIG_ENDIAN)) {
writeInt(value);
} else {

It's worth noting that for enums, using == is more performant than .equals(), and it's also the convention in Java to use == for enum comparisons. Therefore, in this specific case, you could consider suppressing the PMD warning instead of changing the code.


This comment was generated by an experimental AI tool.

^ (~0L << 49)
^ (~0L << 56);
if (x < 0L) {
if (UnsafeUtils.getByte(tempPos++) < 0L) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Codacy found a medium Error Prone issue: These nested if statements could be combined

The issue pointed out by PMD is that the nested if statements could be combined to make the code more concise and potentially easier to read. Nested if statements can sometimes make the code harder to follow, and combining them when the conditions are both necessary for the subsequent code to run can improve clarity.

In the provided code snippet, the nested if statement checks whether x is negative, and then within that block, it checks if the result of UnsafeUtils.getByte(tempPos++) is negative. These two conditions can be combined into a single if statement with a logical AND (&&) operator, which will ensure that the second condition is only evaluated if the first one is true.

Here's how you can combine the nested if statements:

Suggested change
if (UnsafeUtils.getByte(tempPos++) < 0L) {
if (x < 0L && UnsafeUtils.getByte(tempPos++) < 0L) {
return readVarIntLongSlow(zigZag);
}

By applying this change, you maintain the same logical flow but with less nesting, which can improve the readability of the code.


This comment was generated by an experimental AI tool.

if ((length() - (start + offset)) < Integer.BYTES) {
throw new BufferUnderflowException();
}
if (byteOrder == ByteOrder.LITTLE_ENDIAN) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Codacy found a medium Error Prone issue: Use equals() to compare object references.

The issue with the code is that it uses the == operator to compare ByteOrder object references. In Java, the == operator checks if two reference variables refer to the exact same object in memory. However, when comparing objects for equality in terms of their values, the equals() method should be used. This is because equals() can be overridden to compare the actual contents of objects, rather than their memory addresses.

In the case of enums like ByteOrder, using == is actually safe because enums have a guarantee that each constant is a singleton, meaning there is only one instance of each enum constant in the JVM. However, the linter is likely configured to flag all object reference comparisons that use == instead of equals(), without considering the special case for enums.

To adhere to the linter's expectations and to ensure that the code is robust in case the type of byteOrder is later changed from an enum to a class, you can replace the == with equals(). Here is how you can fix the code:

Suggested change
if (byteOrder == ByteOrder.LITTLE_ENDIAN) {
if (ByteOrder.LITTLE_ENDIAN.equals(byteOrder)) {

This change ensures that the code uses the equals() method to compare the byteOrder object's value with ByteOrder.LITTLE_ENDIAN, which is the recommended way to compare objects in Java for equality based on their values.


This comment was generated by an experimental AI tool.

*/
@Override
public int writeBytes(@NonNull final InputStream src, final int maxLength) {
// Check for a bad length or a null src
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Codacy found a minor Code Style issue: Comment has incorrect indentation level 0, expected is 12, indentation should be the same level as line 2,824.

The issue highlighted by Checkstyle is that the comment indentation does not match the code block it is commenting on. In Java, the convention is for the comments to have the same indentation as the code that follows them. This helps in maintaining readability and consistency in the codebase.

The comment in question is at the beginning of the block, so it should be indented to the same level as the code inside the block. Since the Objects.requireNonNull(src); statement is indented with 12 spaces, the comment should also be indented with 12 spaces to match the code style.

To fix this issue, you should simply add the proper indentation to the comment so that it aligns with the subsequent code block. Here's how you can adjust the comment indentation:

Suggested change
// Check for a bad length or a null src
// Check for a bad length or a null src

With this change, the comment is now properly indented, and it should satisfy the Checkstyle rule for comment indentation.


This comment was generated by an experimental AI tool.

@artemananiev
Copy link
Member

This PR is superseded by #120. Closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve BufferedData Performance Improve Bytes Performance
3 participants