[Help]: Clarification on VALL-E and VALL-E V2 in Amphion Repository

[nngocson2002]

Upon reviewing the repository, I noticed it contains two versions: VALL-E and VALL-E V2. Could you kindly clarify if both implementations correspond to the same paper (paper link)? Or does VALL-E correspond to this paper (paper link) and VALL-E V2 to this one (paper link)?

Additionally, I am attempting to train VALL-E from scratch using the egs/tts/VALLE directory. Specifically, during the AR training step, I noticed that the train_stage and ar_model_ckpt_dir arguments mentioned in the egs/tts/VALLE/run.sh file are not reflected in the bins/tts/train.py script. Could you provide guidance on resolving this mismatch?

[lmxue]

Hi @nngocson2002 , thanks for your email and for bringing up this issue.
As mentioned in README of VALL-E and VALL-E v2, [VALL-E] is the implementation of the paper, while, VALLE_V2 includes improvements based on vanilla VALL-E.
Additionally, train_stage and are ar_model_ckpt_dir are defined in valle_train.py file.

[nngocson2002]

So, if I want to reproduce VALL-E, I just need to choose either VALL-E or VALL-E V2, right? I initially thought these repositories were based on different papers, so I assumed I would need to reproduce both.

[jiaqili3]

So, if I want to reproduce VALL-E, I just need to choose either or VALL-E V2, right? I initially thought these repositories were based on different papers, so I assumed I would need to reproduce both.

Yes, and if you want to reproduce VALL-E I suggest using VALL-E (v1) which uses the vanilla implementation.

[nngocson2002]

Thank you for your clarification!

One point I’d like to discuss: as I mentioned, when I attempt to run the training from scratch (as shown in the image below), it executes the bins/tts/train.py script. This results in the train_stage and ar_model_ckpt_dir arguments being unrecognized.

Additionally, could you provide the structure of the JSON file referenced in egs/tts/VALLE/exp_config.json? I noticed that the preprocessing script reads this JSON file, but I couldn't determine its structure.

I would greatly appreciate your guidance.