Does or will ColossalAI support seq2seq model?

[drxmy]

I want to try 3d parallel to train t5 or bart. And i haven't find any example or doc concerning seq2seq issue. Do i miss something?

[kurisusnowdeng]

Hi @drxmy , we are mainly focusing on optimizing the Transformer backbone, but we will provide more pratical models and examples in future updates. We will also appreciate your contribution if you are interest in adding new models to our example projects. You are very welcome to reach out to us if there is still problem.