Direct Chevrotain Usage

[kevinkhill]

Hello, I'd like to start the discussion if possible, for directly using Chevrotain in this project.

What I mean is, I found Chevrotain first and have a partial working interpreter, so is it possible to plug my working project into Langium and skip over the "learning Langium" part?

I am leaning towards a no answer, but I thought I'd ask

[msujew]

Hi @kevinkhill,

so in Langium almost all functionality is build on the existence of your abstract grammar reprentation. Using this information, it is able build all the LSP features on top (looking for references, code completion, semantic highlighting, etc.) and perform AST construction. So you'll definitely have to replace your parser with the grammar representation provided by Langium.

Looking at your interpreter, as far as I can see, it works on the CST directly. While this approach is also feasible in Langium (we expose a CST in a similiar manner as Chevrotain does), computation/interpretation/code generation is usually performed on the AST.

So in total, reusing the code that builds on the parser implementation will probably not be possible given Langium's general architecture.

[kevinkhill]

Follow up question, I am so new to what I am doing (lexer/parser/interpreter) I don't know what you mean.

What is it I am doing vs what you explain.... Would I need to take out the computational stuff from my parser/interpreter so it can map over to Langium? Then I would be able to use the parser generated by Langium to compute stuff in my already made interpreter? So replace my hand made tokens/lexer/parser with Langium but keep my interpreter?

[msujew]

I see, sure I can go into more detail on this. Basically Langium only uses Chevrotain under the hood, without developers actually having to interact with it. Instead, developers deal with our grammar language, a language used to define the grammar of your own language in addition to some additional structural data (like assignments and cross references). Langium then takes this grammar to

1. translate it into a Chevrotain parser
2. generate an abstract syntax tree definition, where each rule maps to an interface definition. Perhaps you can read up on that topic here
3. use the structural information to provide default implementation for LSP (code editor) services

Given that Langium gives you an abstract syntax tree, instead of an concrete syntax tree like Chevrotain does, you would have to rewrite your interpreter as well.

Short question in between: What do you intend to use Langium for? If you just need a parser and interpreter, just continue using Chevrotain should be fine. If you want advanced editor support like Langium provides, you need to adhere to its constraints.

[kevinkhill]

Thank you for the info and links, very informative. It might help more to link to this conversation, which is how I found about this (langium) in the first place

[msujew]

Thanks for linking to the discussion. I'm a collaborator on Chevrotain (currently working on a reimplementation of Antlr4 style unbounded lookahead), but don't look too much into the discussions/issues of the repo 😅

Are you using Monaco as a standalone editor outside of VSCode, or are you actually looking to build a VSCode extension? While the LSP apis can be used in the web, it requires considerable overhead to just integrate it with Monaco. We plan to improve the web-support of Langium, but our first priority is language server (i.e. VSCode extension) and CLI support right now.

Seeing your requirements, Langium would provide out-of-the-box support for:

• Syntactic (using textmate grammars) and semantic highlighting (using AST information and the LSP)
• Validation/semantic checks/diagnostics: Aside from automatically displaying any parser errors, semantic checks have to be written manually of course, Langium can't infer the semantics of your language, only the AST.

Additionally, some nice other editing features:

• Code completion
• Hovering
• Following references/searching for references

We are constantly expanding the editing scope, e.g. planning to include code formatting as well in the near future.

[kevinkhill]

I'm really only using monaco because codemirror wasn't great and a textarea was even less great 😆 This whole project is sort of morphing and evolving... It's currently just for fun! Here is another little thing I made that could have benefit from an editor with said features.

In a shop, you wouldn't really write code by hand, but it would 100% benefit operators browsing a file. I have to open programs on a Windows XP box in wordpad, making edits dangerous. (SO MUCH FOOT GUNS)