Cannot figure out how to install language support

[hovinen]

With a fresh install of this plugin, when running Jekyll on a file containing a Rust codeblock, I get the following error:

  Conversion error: Jekyll::Converters::Markdown encountered an error while converting '_posts/<redacted>.md':
                    Error retrieving language configuration for scope 'source.rust': Language not found

The README.md file says:

...in order to syntax highlight a given language using this plugin, that language's Tree-sitter parser library must be installed to the correct directory on your machine.

How is the language parser installed? Neither running cargo install nor make install from the parser library's repository (as obtained from the tree-sitter documentation to which README.md links) help. Nor does building the parser and manually copying the library over to the correct directory.

Later, it says:

For most such parser libraries, installation simply amounts to downloading the repository into the configured Tree-sitter parsers directory.

I do not understand this. First off, it says "for most". For which parser libraries does this apply? How does one know?

Second, what does "downloading the repository" mean? Cloning the parser repository from GitHub? Or does one get it from somewhere else? I can't imagine that one just clones the repository into a directory, since the parser has to be compiled to work. So the most straightforward interpretation of this text can't be right.

I have tried every combination of intepretations of this text I can find and am unable to get Kramdown to recogize any languages. Could you please update the documentation to include clear, step-by-step instructions on how to get a working setup?

Thank you.

[andrewtbiehl]

Hi! Thanks for your inquiry. Sorry to hear you had some difficulty getting the plugin to work.

---

To answer your main question, "downloading the repository" / "installing the language parser" does indeed mean just downloading the parser repository (for example via a Git clone) from GitHub into a preconfigured directory and doing nothing else. When you run the Tree-sitter syntax highlighter and it finds the parser library, it will do the rest.

With respect to your specific use case, I expect based on the error message you received that the following steps should probably get things working locally for you:

1. Create the directory tree_sitter_parsers inside your home directory, for example by running the following command:

   mkdir ~/tree_sitter_parsers

   This is the default directory that this plugin tells Tree-sitter to look for when searching for parser libraries.

2. Download the tree-sitter-rust parser library repository as a subdirectory of the tree_sitter_parsers directory, for example by cloning it with Git via the following command:

   git -C ~/tree_sitter_parsers clone https://github.com/tree-sitter/tree-sitter-rust.git

It is probably a bit surprising, but no additional adjustments to the Tree-sitter Rust parser library are needed after this. Tree-sitter, and in turn this plugin, should now be able to successfully find, compile, and use the parser library when you build and serve your Jekyll site locally.

---

With respect to the question of the meaning of "for most", the situation with Tree-sitter language parser libraries is a bit inconsistent and therefore difficult to make general statements about. This is due to the design decision by Tree-sitter to uncouple the development of the core library from the development of individual language libraries. Unlike other popular syntax highlighting tools like Rouge and Pygments, which bundle all of their language parsing capabilities as a feature of the core tool, Tree-sitter only provides a parser generator and opens up the generation of language parser libraries to the community. So many Tree-sitter parser libraries that are publicly available are maintained by third parties.

One downside to this is that different maintainers make different decisions about how to structure their repositories, so I don't think it is possible to make a blanket statement about how to get every parser library working with Tree-sitter's syntax highlighter. Instead, for each parser library you want to use, you should ideally check its documentation to be certain about how to get it to work. Fortunately, as far as I can tell, the Tree-sitter syntax highlighter is able to make use of the vast majority of libraries out of the box, as long as their repositories are downloaded to the correct directory. Included in this majority as of now is the aforementioned Rust parser library.

Some exceptions (hence the use of the phrase "for most") that I have personally encountered include the C++ parser library, which has a dependency on the C parser library, and this Swift parser library, which does not provide a crucial file for parsing out of the box.

---

Hopefully this information clears some things up, but feel free to let me know if something is still confusing. This discussion may also be helpful. The tests for this plugin and my own personal website can also serve as working examples, although the latter is not a particularly "minimal" such example.

[hovinen]

Thanks for the detailed information!

I tried something like the method you described, but now see that I named the directory ~/tree-sitter-parsers rather than ~/tree_sitter_parsers. That explains why it didn't work.

I think that explains everything, so I'm closing this ticket. Thanks again!