Singing, Beatboxing and Additional Sounds

[yukiarimo]

Hey there! I’m wondering if it’s possible to make the GPT-SoVITS-V2 sing. Specifically, I wonder if it would work if I simply input raw singing audio (without any reverb or other effects) in the same tempo and train a model on it. Could it potentially replicate the singing but with different lyrics?

Additionally, is it possible to incorporate tokens like , , or that can mimic non-verbal sounds?

[XXXXRT666]

You'd better look at RVC or Sovits-SVC. Most of SVC project needs F0.

[yukiarimo]

1. Yes, I know it, and I’m already doing it! But it is like making a cover of a song, but I want my TTS to sing by herself, not covering!
2. About the f0 I have a question from my Reddit post: The RVC (Retrieval-based Voice Conversion) is using something called RMVPE for the pitch extraction/guidance (also called f0). So, is it possible to make a custom f0 or extract it from audio and apply it as pitch guidance to another audio, for example, singing audio, without any voice conversion at all, just RMVPE?

[XXXXRT666]

1. If you want this sing by herself, you should look at some SVS Project such as Diff Singer.
2. Yes, Adobe Audition could just edited pitch

[yukiarimo]

1. What about something that runs on macOS or at least not diffusion?
2. How? And is it possible for Logic Pro, too?

[XXXXRT666]

1. ACE Studio, VOCALOID, I prefer OpenUtau as it's free
2. I don’t use Logic Pro often, but I believe Flex Pitch or Pitch Shifter could be what you’re looking for.

[yukiarimo]

1. I tried, but I need something AI. Cause, it is impossible to single with a custom voice there (all voice banks are Japanese and not customizable)
2. Okay, I'll try!

[XXXXRT666]

Diff Singer could train personalised voice, or you could just use SVS + SVC