Support for multiline math formulas

[utensil]

For now, Verso supports inline math and display math but the latter can only use inline code block:

$`\sum_{i=0}^{10} i`

$$`\sum_{i=0}^{10} i`

It doesn't seem possible to use a multiline code block after $$ per my tests.

B.T.W. Previously I learned this syntax from Verso manual in the README but now it's 404, so I found it again in UsersGuide.Markup.

[david-christiansen]

It looks like I made a mistake in the CI script that releases the user's guide when I added the HTML rendering. Fixing that is tracked in #57 - in the meantime, running generate.sh in the repository root will get you both renderings. Sorry about that, and thanks for noticing!

Verso inherits the sharp distinction between inline and block elements from Markdown. Right now, both forms of math are inline elements, while multiline code blocks are block elements. Would you expect a multiline math formula to be inline or block?

That is, does

We can see this because:
$```
\begin{array}
...
\end{array}
```

mean the same thing as

We can see this because:

$```
\begin{array}
...
\end{array}
```

?

I would tend to think that it should not, and that multiline math should still be an inline element. In other words, this would extend the concrete syntax, but not the abstract syntax. Do you agree?

Another alternative would be to relax the parser for the existing math syntax to allow newlines within the backticks, so long as there's no completely blank lines. That is, I could write:

Here's some math: $`
\begin{cases}
...
\end{cases}
` and it's great.

but not
Here's some math: $`
\begin{cases}

...

\end{cases}

` and it's great.

Would that be sufficient for your needs?

[utensil]

Would you expect a multiline math formula to be inline or block?

IIUC, one can't make an inline math formula with mutiple lines in the source in Markdown, so this question doesn't really make sense to me. I tried it here:

$
\LaTeX
$

renders to:

$
\LaTeX
$

Does ... mean the same thing as ... ?

It took me a while to notice the empty line. If I recall and interpret correctly, most markdown flavors will consider the first case invalid (renders verbatim), and consider the second case a block, here I tried it again with Github's flavor:

> Case 1: Here is a multi-line formula, followed in an inline manner $$
\LaTeX
> $$

renders to:

Case 1: Here is a multi-line formula, followed in an inline manner $$
\LaTeX
$$

> Case 2: Here is a multi-line formula
>
> $$
\LaTeX
> $$

renders to

Case 2: Here is a multi-line formula

$$ \LaTeX $$

[utensil]

Would that be sufficient for your needs?

Probably, yes. I would like to add some context though.

These are what I would expect from writing math formulas in Markdown or Verso or a mark up in general:

1. I should be able to copy something from a LaTeX file, paste them into some syntax surrounding it, without modifying it;
2. The surrounding syntax should be intuitive for someone familiar with LaTeX.

Markdown itself breaks expectation 1, consider the following:

> The definition of the Lipschitz group $\\{ x \in \mathop{\mathcal{C}\ell} | x \text{ is invertible and } x v x^{-1} ∈ V \\}$

renders (correctly) to

The definition of the Lipschitz group $\{ x \in \mathop{\mathcal{C}\ell} | x \text{ is invertible and } x v x^{-1} ∈ V \}$

but it requires extra escaping for \, if I just copy it from the originally LaTeX, most markdown flavors won't work as it expects extra escaping rules for Markdown. This is also where doc-gen4 is broken for all non-trivial formulas as discussed on Zulip here, and I hope that Verso could come to rescue.

The syntax in Verso seems to break expectation 2:

I would expect similar things like a $, $$ pairs or \[, \] pairs, and also intuitively accepts a mixture of them and the code block syntax as in the Verso manual, but the mixture is expected to be: inline math mixing with inline code, multiline math mixing with multiline code block, like

$$```
\LaTeX
```

not a mixture that mix everything together, like

$`
\LaTeX
`

which I believe would cause much more trouble in the future, parser-wise or cognition-wise. And I would only use

Inline $`\LaTeX`

Multiline,

$$`
\LaTeX
`

so I wouldn't confuse myself if the final syntax is mixing $$, single back tick for multiline math formulas.

P.S. I wonder what would happen if someone writes:

$$`
\LaTeX blah blah ` blah blah
`

Does the backtick between blahs terminates the formular abruptly?

[david-christiansen]

Replies cut up by topic :-)

I should be able to copy something from a LaTeX file, paste them into some syntax surrounding it, without modifying it

This makes sense. Part of the reason for using ` around math in Verso is in order to not require extra escapes - this is consistent with how code literals work.

I wonder what would happen if someone writes: ... Does the backtick between blahs terminates the formular abruptly?

Yes. The way to work around this is to use more backticks to start and end the formula, just as with code literals. So $$`` \LaTeX blah blah ` blah `` won't be terminated early.

the mixture is expected to be: inline math mixing with inline code, multiline math mixing with multiline code block, like ...

The difficulty here is that Verso, like Markdown and HTML, has a distinction between "block-level" and "inline-level" content. Block-level content is things like paragraphs, lists, and headers; inline content is things like text, hyperlinks, and boldface. Paragraphs cannot contain blocks. Math is an inline element, whether it's in display mode or not (that is, whether it's $foo$ or \begin{displaymath}foo\end{displaymath}, to be pedantic and verbose about it). If multiline math is a block element, then it can't logically be part of a paragraph; if it's inline, then it can be. Hence the question about the blank line above. Does that cast some light on what I was going after?

In Verso and Markdown, multiline code blocks are blocks; they are logically distinct from the paragraphs that surround them. Should there be a block-level math element? Or should a math element standing alone be a paragraph that contains a single inline bit of math? I think the latter makes sense, probably, as it's more flexible.

Really, it's a problem that a high degree of Markdown compatibility has made it impossible for Verso to have a single logical paragraph that contains some text, then a bulleted list, then more text, then a formula, then some text. I've got a workaround planned for this, for when it really matters. Web typography typically hides this because paragraphs don't get indented, but it's not great for other output forms that we want to support.

[utensil]

Sorry that I didn't reply months ago, as I was quite confused by the block level discussion, then distracted by other things. I've used some other authoring tools in the last few months including Hugo, Quarto, Typst and Forester, with this issue in mind, and somehow rereading this issue makes sense to me now.

It seems very reasonable that Verso regards math as an inline element, regardless of whether it's multiple line math.

But there is a detail in one of your early questions I didn't notice before:

relax the parser for the existing math syntax to allow newlines within the backticks, so long as there's no completely blank lines

Months ago I would say this subtle behavior (not allowing blank lines in multiple line math) is unintuitive and error-prone, but taking how Verso has to process Lean and other syntax into account, now I would say this is also reasonable, as long as if a user makes such a mistake unintentionally, the error message is clear on how to fix it.

B.T.W. I wonder if there's any progress or some other thoughts on this topic lately. I'm aware that this might not be a high priority issue. Thanks!

[utensil]

Hi @david-christiansen , I just tried the latest Verso, it seems that something like

$$`
\f\relax{x} = \int_{-\infty}^\infty
    \f\hat\xi\,e^{2 \pi i \xi x}
    \,d\xi
`

is still not working?

(which allows multiline input for display math)

I can see @lecopivo has tried to work around this limitation in lecopivo/scientific-computing-lean 's Meta.Latex, but it's ideal to have built-in support for this.

[utensil]

Inspired by the workaround (which use less Verso-native plain text for the math block), my workaround mimics one variant of GFM math syntax:

import Verso
import VersoManual

open Verso Doc Elab in
@[code_block_expander math]
def Manual.math: CodeBlockExpander
  | _args, str => do
    return #[(← `(Doc.Block.para #[Doc.Inline.math Doc.MathMode.display s!"{$str}"]))]

so one can write

```math
f'(x) = \lim_{h → 0} \frac{f(x+h) - f(x)}{h}
```