docs: update supported tiktoken version

benbrandt · Oct 14, 2024 · 27fefce · 27fefce
1 parent fda16b5
commit 27fefce
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/README.md b/README.md
@@ -202,7 +202,7 @@ There are lots of methods of determining sentence breaks, all to varying degrees
 | Dependency Feature | Version Supported | Description                                                                                                                                                                    |
 | ------------------ | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
 | `rust_tokenizers`  | `^8.0.0`          | Enables `(Text/Markdown)Splitter::new` to take any of the provided tokenizers as an argument.                                                                                  |
-| `tiktoken-rs`      | `^0.5.0`          | Enables `(Text/Markdown)Splitter::new` to take `tiktoken_rs::CoreBPE` as an argument. This is useful for splitting text for `OpenAI` models.                                   |
+| `tiktoken-rs`      | `^0.6.0`          | Enables `(Text/Markdown)Splitter::new` to take `tiktoken_rs::CoreBPE` as an argument. This is useful for splitting text for `OpenAI` models.                                   |
 | `tokenizers`       | `^0.20.0`         | Enables `(Text/Markdown)Splitter::new` to take `tokenizers::Tokenizer` as an argument. This is useful for splitting text models that have a Hugging Face-compatible tokenizer. |
 
 ## Inspiration