Cloze Update: Adds Cloze hints, Anki-Like Clozes, Cloze Overlapping and Custom Cloze Patterns

[MotaOcimar]

Hello! This PR addresses the following issues: #581, #556, #451, #426, #387, #284, #93, introducing several new features:

• Cloze hints ([FEAT] Footnote syntax for cloze hints and Extra (back of card) fields #451, Anki Cloze syntax #93)
• New Cloze Types:
  • Numbered Clozes, just like Anki (Anki Cloze syntax #93)
  • Overlapped Clozes ([FEAT] Footnote syntax for cloze hints and Extra (back of card) fields #451, [FEAT] Cloze Overlapping Generalization #426, [FEAT] cloze overlapping #284)
• Custom Cloze Patterns ([FEAT] Custom Convert To Cloze Options #581, [FEAT] Custom cloze tokens #556, [FEAT] Adding support for strikethrough/custom cloze #387, Anki Cloze syntax #93)

The implementation leverages the Clozecraft package, a Cloze parser and Card formatter developed by me specifically for this purpose. Feel free to explore, ask questions, and contribute. Your feedback is highly appreciated.

I've opened this PR as a draft because there are still a few tasks pending:

• Fix failing tests
• Create tests for the other cloze types
• Decide which Cloze Patterns will be delivered by default with the plugin
• Play with the new features to ensure they work as expected. Fix any bugs and add tests for them if necessary
• Gather your feedback

Features

Cloze Types

Simplified Clozes

Simplified Clozes are the Cloze Type we have been using so far. They keep working as before, but now you can add hints to them. That means each card will occlude one deletion, showing its hint if available, while leaving the other deletions visible.

For instance, the following note:

This ==note== doesn't need to have ==number==^[hint] on clozes

Will generate two cards, with the following fronts:

1. This [...] doesn't need to have number on clozes
2. This note doesn't need to have [hint] on clozes

And both will have the same back:

This note doesn't need to have number on clozes

Classic Clozes

Classic Clozes are the same as Simplified Clozes, but they also have a sequence number used to sort and group deletions.

For instance, the following note:

This is a ==cloze==^[a hint][^1]. This is ==too, but without hint==[^2]. And this one is asked ==together with the first cloze==[^1].

Will generate two cards, with the following fronts:

1. This is a [a hint]. This is too, but without hint. And this one is asked [...].
2. This is a cloze. This [...]. And this one is asked together with the first cloze.

And both will have the same back:

This is a cloze. This is too, but without hint. And this one is asked together with the first cloze.

Generalized Cloze Overlapping

The Cloze Overlapping Generalization allows you to dictate the behavior of each deletion individually. Each deletion has its own set of actions that indicate how it will behave on each card:

• a for ask (hide the deletion answer in the front of the card while showing it in the back).
• h for hide (hide the deletion answer both in the front and back of the card).
• s for show (show the deletion answer both in the front and back of the card).

For instance, the following note:

==Some context for items 2 and 4, but that could spoil items 1 and 3. Note that this doesn't even need to be asked==[^hshs]
.
- ==Item 1==[^ashh]
- ==Item 2==[^hash]
- ==Item 3==[^hhas]
- ==Item 4==[^hhha]

In this example, 4 cards will be generated, where items 1 to 4 will be asked sequentially. When asking Item 1, all other text will be hidden. When asking Item 2, just the Item 1 and the context will be displayed. When asking Item 3, only the Item 2 will be displayed. Finally, when asking Item 4, both the Item 3 and the context text will be displayed.

Custom Cloze Patterns

You can now create your own Cloze Patterns. For instance, to emulate Anki-like clozes (e.g., {{1::text::hint}}), simply add the following pattern to your settings:

{{[123::]answer[::hint]}}

Brackets [] delineate where the sequence number and hint will be placed, along with any additional characters of your choice (e.g., the :: in the example above). Clozecraft automatically generates a regex pattern based on your custom pattern. Sequence numbers are identified by finding numbers between brackets, hints by finding the word hint in brackets, and the deleted text by finding the word answer.

Here are some examples of custom patterns:

Explanation	Pattern in the Settings	Simplified Cloze Usage	Numbered Cloze Usage	Generalized Cloze Overlapping Usage
Anki-Like Pattern	{{[123::]answer[::hint]}}	Brazilians speak {{Portuguese}} Or, with hint: Brazilians speak {{Portuguese::language}}	Brazilians speak {{1::Portuguese}} Or, with hint: Brazilians speak {{1::Portuguese::language}}	Brazilians speak {{a::Portuguese}} Or, with hint: Brazilians speak {{a::Portuguese::language}}
Highlighted pattern with hint and sequencer in footnotes. Note that you can use brackets as part of your pattern, but you need to escape them.	==answer==[^\[hint\]][\[^123\]]	Brazilians speak ==Portuguese== Or, with hint: Brazilians speak ==Portuguese==^[language]	Brazilians speak ==Portuguese==[^1] Or, with hint: Brazilians speak ==Portuguese==^[language][^1]	Brazilians speak ==Portuguese==[^a] Or, with hint: Brazilians speak ==Portuguese==^[language][^a]

In your settings, you can add multiple patterns at once, separating them with a new line:

{{[123::]answer[::hint]}}
==answer==[^\[hint\]][\[^123\]]

Then, you can use them in your notes:

This note has both ==highlighted clozes==[^1] and {{2::anki like clozes}}

And this note has both ==highlighted clozes==^[footnote hint][^1] and {{2::anki like clozes::anki like hint}}

Remember, deletions must be of the same Cloze Type. You can't mix, for instance, a Classic Cloze with a Simplified Cloze in the same note, as they employ different ways to sort and identify the cards.

[MotaOcimar]

Hello @st3v3nmw and @Stefanuk12,

I hope you're both doing well! First and foremost, I'd like to express my appreciation for the work you've both been doing on the project so far. It's inspiring to see your dedication to this incredible Obsidian plugin.

Yesterday, I had the opportunity to open this PR addressing several issues related to the use of Clozes. Upon submitting it, I noticed that @Stefanuk12 had already opened PR #769 addressing issue #93 five months ago, which unfortunately went unnoticed. Additionally, @st3v3nmw, I noticed that you updated the status of issue #93 to "In progress," which made me wonder if you were already working on it.

I'd like to sync our efforts to ensure that all contributions are considered in the best possible way. My goal is not only to offer my solution but also to recognize your previous work and ensure that the project progresses collaboratively and efficiently. Please let me know what you think about all of this, what stage you're at, your thoughts on the current PR, and if I can assist you in any way.

Thank you once again for your hard work. I'm looking forward to the evolution of Clozes in the project.

Best regards, Ocimar Santos (MotaOcimar)

[Stefanuk12]

Lovely PR btw, a lot better and expansive than mine! I have pretty much abandoned Obsidian so I won't be much help unfortunately. Really sad how my PR was overlooked :( , but I hope yours gets reviewed since this is a KEY thing this plugin is missing.

[MotaOcimar]

Hello again!

Firstly, @polyipseity, my apologies for the oversight in my previous message. I realized after sending it that I forgot to include you, who opened PR #607 addressing issues #556 and #387. I appreciate your work and would like to hear your thoughts on this PR too.

@Stefanuk12, thank you for your kind words regarding the PR. I'm sorry to hear that you've pretty much abandoned Obsidian; your opinion remains valuable to me, though. If any insights or suggestions arise, I'd be grateful to hear them.

[ronzulu]

Hi @MotaOcimar I've only had a chance for a brief look, but looks great!

I'm working on [FEAT] Refactor code to support diff methods of storing the scheduling info, and diff SR algorithms #878 , so after that I should have time to review in detail.

Great work!

Cheers
Ronny

[MotaOcimar]

Hey @ronzulu!

It's awesome to have you here! I just wanted to take a moment to express my gratitude not only for your willingness to review this PR but also for all the hard work you've been putting into the project. Your immense participation, both in code and in discussions, hasn't gone unnoticed. Rest assured that the time and effort you've invested here have made a significant impact. I'm particularly excited about feature #878 too. Keep up the fantastic work!

Best,
Ocimar Santos (MotaOcimar)

[SeLiEnisyst]

Hi,
@ronzulu pointed me to this place here.

I am very interested in using spaced repetition in combination with clozes for language and especially vocabulary training.
As I described here: #955
I would like to use your feature for:

• 1st side of the flashcard: exchange one or more word/s in a given sentence by the word/s of my mother language
• 2nd side of the flashcard: the original sentence

Would it be possible to integrate that feature?

Thank you very much in advance,
kind regards,

Silias

[MotaOcimar]

Hi Silias (@SeLiEnisyst)!

Let me see if I understood well.

You want to write something like this on the note:

;;;;picking a new painting, would be {{to succumb}};;;;{{unterliegen}} to the very impatience;;;;

Have this as card front/question:

picking a new painting, would be to succumb to the very impatience

And this as card back/answer:

unterliegen

If that's correct, although I don't think it's possible to create cards exactly like this yet, I think it will be possible to do something very similar after this PR is completed:

Assuming we are using the Anki-like pattern for Simplified Clozes (e.g., {{text::hint}}), you can write:

picking a new painting, would be {{unterliegen::to succumb}} to the very impatience

This will display the front of the card as:

picking a new painting, would be [to succumb] to the very impatience

And the back of the card as:

picking a new painting, would be unterliegen to the very impatience

I understand that the front and back formatting may not be exactly as desired. But I hope that it will still allow you to review and practice your vocabulary.

Additionally, I saw you expressed interest in having a delimiter for the beginning and end of the note (the ;;;; in your example). Currently, the plugin uses newlines to delimit notes. Adding a way to change this is probably a new feature that impacts other parts of the plugin, not just Clozes. I wonder if there's already an issue addressing this. Can you say @ronzulu?

Silias, let me know if I understood everything correctly and if, despite the limitations, this way can meet your request.

Best,
Ocimar

[ronzulu]

Hi @MotaOcimar sorry I haven't managed to look at your PR yet. I'm still working on #878. I'm hoping to put in my PR for it hopefully this week, then will have a chance. Would also be great if @st3v3nmw would look at this PR as well :-)

[MotaOcimar]

Additionally, I saw you expressed interest in having a delimiter for the beginning and end of the note (the ;;;; in your example). Currently, the plugin uses newlines to delimit notes. Adding a way to change this is probably a new feature that impacts other parts of the plugin, not just Clozes. I wonder if there's already an issue addressing this. Can you say @ ronzulu?

I found! It's issue #450 [BUG] Cloze Cards require blank lines before and after to create atomic card

[SeLiEnisyst]

Hi @MotaOcimar,

Thank you very much for your detailed answer!!! :)
I think I didn't express myself clearly enough, therefore I will try it another way:

Have this as the card front/question:

picking a new painting, would be to succumb to the very impatience

Additionally to the original sentence, there should be some kind of hint for which part of that sentence a translation is asked for. Therefore, I suggested indicating that part with the ==bold syntax==. My suggestion for the front card would be: picking a new painting, would be ==to succumb== to the very impatience.

The back of the card would just have the translation, in my case: "unterliegen".

...
The rest of my original feature request, you already answered sufficiently. To indicate the start and stop of the phrase for the front card (delimiter), I just would have to set new lines - that's absolutely fine! :)

[polyipseity]

MotaOcimar commented on Apr 26

Hello again!

Firstly, @polyipseity, my apologies for the oversight in my previous message. I realized after sending it that I forgot to include you, who opened PR #607 addressing issues #556 and #387. I appreciate your work and would like to hear your thoughts on this PR too.

Nice. It is definitely better than hardcoding the cloze token delimiters. This would supersede my PR.

[MotaOcimar]

Hi @SeLiEnisyst!

I wonder if using a basic card would achieve what you want.

Example:

Raw note:

picking a new painting, would be ==to succumb== to the very impatience::unterliegen

Card front and back:

Sorry for the delay in responding.
Very busy days here

[ronzulu]

Hi @MotaOcimar I've finally completed #878 and so should be able to look at your great PR over the next week or 2.

[ronzulu]

Hi @MotaOcimar thanks for your answers to #426 (comment)

I thought it might be best if we actually first work on the user documentation. This way we can agree on terminology et cetera, and it does need to be done in any case.

I've made a start on this, and was wondering if it's okay to push what I have done to your branch. I haven't done this before, but if permissions permit this, then I assume this should be possible.

I notice that "Maintainers are allowed to edit this pull request." is enabled. I am categorised as a "collaborator" rather than "maintainer", so I'm not sure.

[MotaOcimar]

Hi @ronzulu!

Thanks for your suggestion and sorry for the delayed response.

I agree that starting with user documentation is a good idea to establish consistent terminology. However, I believe it would be better handled in a separate PR, as they address different demands.

New terms introduced by this Clozes PR can still be included here.

What do you think?

[ronzulu]

Sounds good to me. I've been working on #1029 to improve the documentation and give it some structure.

After that I'm happy to separately work on the documentation for your work (unless of course you prefer doing it yourself)

[MotaOcimar]

Oh, great! Let me know if there's anything I can do to help.

I don't know if it will work either, but feel free to try adding here commits related to Cloze documentation.

Thanks for the support, Ronny!

[ronzulu]

Hi @MotaOcimar

I've made a start on the documentation, and decided to put it in its own branch for now.
https://github.com/ronzulu/obsidian-spaced-repetition/tree/pull-943-cloze-user-documentation

I've added in some notes and thoughts about terminology, so would be great if you can review.

Have you needed to build the documentation before? If you want to see how it all looks like rather than just reading the markdown files, see https://www.stephenmwangi.com/obsidian-spaced-repetition/contributing/#viewing-docs-locally

Cheers
Ronny

[nihil-admirari]

Hi @MotaOcimar.

I'm considering a switch from Anki to Obsidian. In Anki, I'm using https://github.com/nihil-admirari/modern-cloze-overlapper (a rewrite of https://github.com/michalrus/anki-simple-cloze-overlapper; original repo has a nice GIF with a demonstration of features). Is it possible to port the following features from Anki?

1. In Anki there is a single configuration field Before|After|OnlyContext|RevealAll|InactiveHints to control the overlapping clozes. In this implementation, [^ashh] etc. must be added to every cloze deletion. Can you please add support for a single configuration fields to save on typing, from which all [^ash] are computed given requested clozes-before, clozes-after etc?
2. Alternatively, all [^ash] can be generated by a separate Cloze Generator UI, which splits a string by a regex and adds the necessary suffixes given requested clozes-before, clozes-after etc.
3. Support for nested clozes. The are useful when memorising poetry, e.g.

   {{c1::To be, or not to be, that is the question:}}
   {{c1::Whether 'tis nobler in the mind to suffer}}
   {{c1::The slings and arrows of outrageous fortune,}}
   {{c1::{{c2::Or to take arms against a sea of troubles,}}}}
   {{c1::{{c2::And by opposing end them: to die, to sleep}}}}
   {{c2::No more; and by a sleep, to say we end}}
   {{c2::The heart-ache, and the thousand natural shocks}}
   {{c2::That Flesh is heir to? 'Tis a consummation}}
   

   generates two cards asking for 5 lines with the previous 3 lines as a hint (assuming context-before = 1). (Cloze nesting may not possible with arbitrary cloze patterns.)

[ronzulu]

Hi @nihil-admirari , thanks for the ideas.

I would suggest creating a new issue with them, and deleting the comment here.

@MotaOcimar has put in so much effort on this feature already, personally I think it's better if we get all the work done to date in a mainstream release before expanding the capabilities further.

Regards
Ronny

[MotaOcimar]

Thank you @nihil-admirari for the ideas!
And thank you @ronzulu for the reply!

I have to agree with Ronny, it will be easier to focus on one thing at a time. Create the issue and we can better understand your suggestion there

[st3v3nmw]

Hi @MotaOcimar.. First of all, apologies for taking forever to get to this pull request..

I've left two comments, mostly that the parser has evolved since this PR was made. I'm not sure how much effort it'll take to port those changes over.

Thanks for the effort you've put into this. Once we have this merged I'll update the documentation to reflect.

[st3v3nmw]

Just removing the existing settings would break existing instances of the plugin.. There's a upgradeSettings function in settings.ts that would help with gracefully migrating the settings to clozePatterns.

[st3v3nmw]

The parser was completely reworked in #1012 & #1081 to use Peggy. There's clozes_grammar in parser.ts/generateGrammar that we should update instead.