comments should not change parser state

[jacob-wieland-gematik]

🤔 What's changed?

⚡️ What's your motivation?

🏷️ What kind of change is this?

• 📖 Documentation (improvements without changing code)
• 🏦 Refactoring/debt/DX (improvement to code design, tooling, etc. without changing behaviour)
• 🐛 Bug fix (non-breaking change which fixes a defect)
• ⚡ New feature (non-breaking change which adds new behaviour)
• 💥 Breaking change (incompatible changes to the API)

♻️ Anything particular you want feedback on?

📋 Checklist:

• I agree to respect and uphold the Cucumber Community Code of Conduct
• I've changed the behaviour of the code
  • I have added/updated tests to cover my changes.
• My change requires a change to the documentation.
  • I have updated the documentation accordingly.
• Users should know about my change
  • I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

---

This text was originally generated from a template, then edited by hand. You can modify the template here.

[mpkorstanje]

The parser is generated. To change the parser you'll have to change the grammar.

gherkin/gherkin.berp

Lines 35 to 38 in 00fcc43

	// we need to explicitly mention comment, to avoid merging it into the description line's #Other token
	// we also eat the leading empty lines, the tailing lines are not removed by the parser to avoid lookahead, this has to be done by the AST builder
	DescriptionHelper := #Empty* Description? #Comment*
	Description! := #Other+

To regenerate the parsers you can follow these instructions:

https://github.com/cucumber/gherkin/blob/main/CONTRIBUTING.md#generating-parsers

[jacob-wieland-gematik]

I have changed the grammar to allow any mixture of empty lines, comments and other-lines as the description.

Extracting only the other-lines from the Description-AST-Node (as before) should yield the description-text without the comments and empty lines.

Only difference is now that a collection of ony empty-lines and comments would yield an empty description text instead of a non-existing Description node.

[jacob-wieland-gematik]

Changed the grammar in a way that is more consistent with the former test cases (treating empty lines inside description blocks as '#Other' lines).