Finding time as of late has been more difficult than I anticipated, and on top of that, just as I was thinking of writing last month’s post, I got distracted by an exciting patchset that has been in the works for over a year finally getting sorted out and landing. So, I hope that some of the fun developments in this post will make up the absense of the last one 🙂.
Since it’s been longer than I thought since the last standard post, we’ve got a fair few commits to catch up on — about 200. Most of these are miscellaneous minor improvements and bugfixes, but a few notable changes have arrived too.
The fabulous new folding engine (org-fold-core) should noticeably improve Org’s
performance with large files. It contains a number of key optimisations to
improve speed, namely:
- Deferring fontification of folded regions
- Using text properties (\(\mathcal{O}(n log n)\)) instead of overlays (\(\mathcal{O}(n^2)\)) for folded regions
- A collection of aggressive optimisations available under
org-fold-core--optimise-for-huge-buffers - Convert text properties to overlays for
isearch(which currently only supports overlays)
How noticeable is the overall performance impact? Well, I poked Ihor and he was kind enough to whip up some benchmarks.
Well this looks very promising[fn::Note the difference in scale, org-fold makes the most difference in the graph where the times are an order of magnitude more.]! Let’s see how much of an improvement this is overall.
| File size (Mb) | Headings (thousands) | Bugfix (no org-fold) | Main (with org-fold) | Improvement |
|---|---|---|---|---|
| 18 | 36 | 115.31 | 0.89 | 99% |
| 8.8 | 24 | 19.03 | 0.48 | 97% |
| 4.4 | 5 | 3.79 | 0.13 | 97% |
| 2.2 | 2 | 1.29 | 0.08 | 94% |
| 1.1 | 1 | 0.50 | 0.045 | 91% |
To be clear, even the smallest file in this data — a 1.1 Mb Org file with around a thousand headings, is fairly large. So, it’s unlikely you’ll notice much of a difference with small–medium files, but if you a few large+ files this should be a fantastic improvement. Once again, thanks Ihor!
| Headings (k) | File Size (Mb) | Bugfix (no org-fold) | Main (org-fold) |
|---|---|---|---|
| 217 | 150 | 173.28 | 10.73 |
| 36 | 18 | 1.97 | 0.92 |
| 24 | 8.8 | 0.81 | 0.46 |
| 5 | 4.4 | 0.09 | 0.21 |
| 2 | 2.2 | 0.03 | 0.07 |
| 1 | 1.1 | 0.02 | 0.04 |
| Headings (k) | File Size (Mb) | Bugfix (no org-fold) | Main (org-fold) |
|---|---|---|---|
| 217 | 150 | 8921.02 | 0.09 |
| 36 | 18 | 100.96 | 0.02 |
| 24 | 8.8 | 17.35 | 0.01 |
| 5 | 4.4 | 3.79 | 0.005 |
| 2 | 2.2 | 1.20 | 0.003 |
| 1 | 1.1 | 0.49 | 0.003 |
All too often exporting code to LaTeX has been a disappointment, with lovely
syntax highlighting from Emacs major modes replaced with a markedly inferior
attempt by pygments (setting org-latex-listings to minted) in a colour scheme I
don’t really like.
A bit over a year ago, a project called engrave-faces started with the aim of
making Emacs’ font-lock more exportable, like a generalised htmlize.el. This has
recently been used to provide a new option for inline and block source code
exports in LaTeX.
To use this, simply install the package and set org-latex-src-block-backend (a
rename of org-latex-listings to better reflect its usage) to engraved.
While this is sufficient to get started, this new backend also allows for some
new options. The theme used for engraving a source block can be set globally
with the new variable org-latex-engraved-theme, or per-file with the
#+latex_engraved_theme keyword. It takes either the name of a theme, or the
symbol t as a stand-in for the current theme.
The theme can also be set on a per-block level using the LaTeX attribute
:engraved-theme.
Here’s what using these new capabilities looks like in practice.
#+title: Engraving source blocks
#+latex_engraved_theme: modus-operandi
#+begin_src emacs-lisp
(message "look ma, some %s" 'code)
#+end_src
#+attr_latex: :engraved-theme modus-viviandi
#+begin_src shell
echo "This is shell code"
#+end_src
This may well be the best syntax-highlighting solution available for PDFs/LaTeX currently available, but I am a tad biased 😛.
Jonas Bernoulli has been using a custom TexInfo backend for Magit’s documentation for a while now, and over the past few months he’s worked the features he was missing into Org’s built-in TexInfo exporter.
Upstreaming like this always takes a fair bit of effort, so thank you Jonas for going through with this!
Previously, whenever a noweb reference appeared on a non-empty line, a multi-line replacement would duplicate the content before the noweb reference.
Clearly, this is not always desirable, and this behaviour can now be turned of
by setting the new header argument :noweb-prefix no.
#+begin_src emacs-lisp :noweb yes :noweb-prefix no
(setq example-data "<<example>>")
#+end_src
Will now expand to
#+begin_src emacs-lisp
(setq example-data "some
multi-line
content")
#+end_src
Instead of
#+begin_src emacs-lisp
(setq example-data "some
(setq example-data "multiline
(setq example-data "content")
#+end_src
I think we’ve all seen plenty of org-mode prettification packages before, so
what makes Minad’s org-modern special? It’s actually doing something similar to
Ihor’s org-fold improvements, switching out slower overlay-based approaches for
text properties. I can confirm that switching out org-superstar-mode for
org-modern has made a substantial improvement in my experience, halving the
first-load time of my config.org to around 20 seconds. If you’re a fan of Org
prettification and haven’t taken a look at this package, I highly recommend
giving it a shot.
- Clean up some magic numbers in
org-attachMarco Wahl - Allow any command form in
org-attach-commands(including keyboard macros) Marco Wahl - Allow
destinorg-list-send-itemto be a buffer position Sacha Chua - Improve CSL-JSON date handling in
oc-basicDavid Lukes - Add
TOMLanddesktoplanguage aliases TEC - Speed up cached bibliography retrieval in
oc-basicIhor Radchenko - Allow setting PlantUML jar arguments Ihor Radchenko
- Allow for customisation of property separators with
org-property-separatorsTyler Grinn - New
ox-latexmaintainer, Daniel Fleischer - More unit tests Kyle Keyer, Nick Dokos
- Documentation improvements Kyle Meyer, Juan Manuel Macias, Bastien, Karl Fogel, Cody Harris
- An Emacs <28 bug in
org-persistIhor Radchenko - Author extraction in
oc-basicNicolas Goaziou - Fix behaviour of
org-copy-visiblewith adjacent tex andbuffer-invisibility-specKyle Meyer - Parsing of inline footnotes with parentheses Nicolas Goaziou
- Honor
default-directoryinob-gnuplotIhor Radchenko - Heading fontification bug Anders Johansson
- Template expansion where one key is a substring of another Andrew Arensburger