What are doctags

[pwright]

Question

I'm not familiar with doctags, are there any references you could share that might help me understand this format?

[PeterStaar-IBM]

@pwright Doctags is a custom format (a but inspired by xml) that is intended to be optimal for LLM's. We try to represent the content and visual structure with special tags, that can be added to the tokenizer of the LLM.

Our aim is to train/finetune LLM's on doctags to make it intuitively understand document structure.