v0.16.0

Breaking Changes

• Update to v0.23.0 of tree-sitter for CodeSplitter. There was a breaking change for language definitions, so this is also a breaking change for us, especially on the Python side, since we support passing the language in.
• Minimum Python version for the Python bindings is now 3.9 since 3.8 will be EOL next month.

Python

Make sure to upgrade to the latest version of your tree-sitter language package.

Rust

Make sure to upgrade to the latest version of your tree-sitter language package crate. These know have a LANGUAGE constant rather than a language() function.

// Before
tree_sitter_rust::language()
// After
tree_sitter_rust::LANGUAGE

What's New

• MarkdownSplitter can better parse the Commonmark HS extension for Definition Lists.

Full Changelog: v0.15.0...v0.16.0