Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle unknown fields part of a oneof #324

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

sigurdm
Copy link
Collaborator

@sigurdm sigurdm commented Nov 15, 2019

No description provided.

@sigurdm
Copy link
Collaborator Author

sigurdm commented Nov 15, 2019

Companion CL https://dart-review.googlesource.com/c/sdk/+/125345 has tests.

Copy link
Member

@szakarias szakarias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

protobuf/lib/src/protobuf/field_set.dart Show resolved Hide resolved
@@ -265,8 +265,9 @@ class _FieldSet {
}
}

// neither a regular field nor an extension.
// TODO(skybrian) throw?
if (_hasUnknownFields) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it intentional that _clearField(tagNumber) checks unknown field set, but e.g. _hasField(tagNumber) does not?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question!
I think we could change the _hasField semantics also.
Done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are now changing semantics to include unknowns, I agree that it is inconsistent if we don't also update _hasField. Good catch!

if (oneofIndex != null) {
_clearField(_oneofCases[oneofIndex]);
_oneofCases[oneofIndex] = tag;
_oneofCases[oneofIndex] = newTagnumber;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe only call it conditionally

final currentTag = _oneofCases[oneofIndex];
if (currentTag != null) _clearField(currentTag);

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - that's better

/// will be kept.
///
/// If the tagNumber is present as an unknown field, that field will be
/// removed.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GeneratedMessage.mergeFromMessage() to call _FieldSet._mergeFromMessage(), which looks like this:

  void _mergeFromMessage(_FieldSet other) {
    for (FieldInfo fi in other._infosSortedByTag) {
      var value = other._values[fi.index];
      if (value != null) _mergeField(fi, value, isExtension: false);
    }
    if (other._hasExtensions) {
      var others = other._extensions;
      for (int tagNumber in others._tagNumbers) {
        var extension = others._getInfoOrNull(tagNumber);
        var value = others._getFieldOrNull(extension);
        _mergeField(extension, value, isExtension: true);
      }
    }
  
    if (other._hasUnknownFields) {
      _ensureUnknownFields().mergeFromUnknownFieldSet(other._unknownFields);
    }
  }

This merging does not seem to account for oneofs. Does it need to be updated as well? Are there more places?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_FieldSet._mergeFromMessage() calls _mergeField that ends up calling _setNonExtensionFieldUnchecked that handles the oneofs

Copy link
Collaborator

@mkustermann mkustermann Nov 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe my comment was not clear, what I meant is the following:

We seem to have an invariant now

  • there is at most one of the oneof cases set (either a) it's set in field set b) it's set in unknown field set c) none are set)
  • whenever we add a new field value, we are guaranteed to clear the old one if one exists.

This invariant is violated because of this line:

    if (other._hasUnknownFields) {
      _ensureUnknownFields().mergeFromUnknownFieldSet(other._unknownFields);
    }

We might have already a oneof candidate in the fieldset (or the unknown fieldset). Yet this code blindly copies other._unknownFields, which possibly has a new oneof value, but we do not clear the old one.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah - now I get it - thanks! I updated the merging of unknown fields to go field by field, and also update the one-ofs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, Martin. Thanks.

Copy link
Collaborator Author

@sigurdm sigurdm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL

if (oneofIndex != null) {
_clearField(_oneofCases[oneofIndex]);
_oneofCases[oneofIndex] = tag;
_oneofCases[oneofIndex] = newTagnumber;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - that's better

/// will be kept.
///
/// If the tagNumber is present as an unknown field, that field will be
/// removed.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_FieldSet._mergeFromMessage() calls _mergeField that ends up calling _setNonExtensionFieldUnchecked that handles the oneofs

@@ -265,8 +265,9 @@ class _FieldSet {
}
}

// neither a regular field nor an extension.
// TODO(skybrian) throw?
if (_hasUnknownFields) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question!
I think we could change the _hasField semantics also.
Done

@sigurdm
Copy link
Collaborator Author

sigurdm commented Nov 22, 2019

I have also improved the _clearField methods handling of oneofs. It was doing the work twice, and only for known fields @szakarias Could you look over that?

Copy link
Collaborator

@mkustermann mkustermann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let Sarah do a careful review (she's probably more familiar with the code).

We should also consider having more unit tests for these brittle cases.

Copy link
Member

@szakarias szakarias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@osa1
Copy link
Member

osa1 commented May 19, 2022

@sigurdm it seems like the changes to _FieldSet._hasField are separate from the issues with oneof and can be merged separately, with the test you added. Should we do that?

@sigurdm
Copy link
Collaborator Author

sigurdm commented May 20, 2022

Yeah I think splitting is fine if it simplifies things

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants