the code of Differential Transformers training

[mucunxie]

Model I am using (UniLM, MiniLM, LayoutLM ...):Differential Transformers
Thank you for your works! Can you provide you code of Differential Transformers training

[YTianZHU]

Hi @mucunxie , the basic training code for DIFF is similar to the code provided at https://aka.ms/yoco , you can make a few changes and merge DIFF code into it.

You can also use other open-source code frameworks and plug DIFF into it by changing a few lines.