An extension of AiCoverGen, which provides several new features and improvements, enabling users to generate song covers using RVC with ease. Ideal for people who want to incorporate singing functionality into their AI assistant/chatbot/vtuber, or for people who want to hear their favourite characters sing their favourite song.
Ultimate RVC is under constant development and testing, but you can try it out right now locally or on Google Colab!
- Easy and automated setup using launcher scripts for both windows and debian-based linux systems
- Caching system which saves intermediate audio files as needed, thereby reducing inference time as much as possible. For example, if song A has already been converted using model B and now you want to convert song A using model C, then vocal extraction can be skipped and inference time reduced drastically
- Ability to listen to intermediate audio files in the UI. This is useful for getting an idea of what is happening in each step of the song cover generation pipeline
- A "multi-step" song cover generation tab: here you can try out each step of the song cover generation pipeline in isolation. For example, if you already have extracted vocals available and only want to convert these using your voice model, then you can do that here. Besides, this tab is useful for experimenting with settings for each step of the song cover generation pipeline
- An overhaul of the song input component for the song cover generation pipeline. Now cached input songs can be selected from a dropdown, so that you don't have to supply the Youtube link of a song each time you want to convert it.
- A new "manage models" tab, which collects and revamps all existing functionality for managing voice models, as well as adds some new features, such as the ability to delete existing models
- A "manage audio", which allows you to to interact with all audio generated by the app. Currently, this tab supports deleting audio files.
- Lots of visual and performance improvements resulting from updating from Gradio 3 to Gradio 4 and from python 3.9 to python 3.11
While it is possible to run the Ultimate RVC web app on a CPU, it is highly recommended to use a GPU for faster processing. On an NVIDIA 3080 GPU, the AI cover generation process takes approximately 1.5 minutes, while on a CPU, it takes approximately 15 minutes. No testing has been done on AMD GPUs, so no guarantees are made for their performance.
For those without a powerful enough NVIDIA GPU, you may try Ultimate RVC out using Google Colab.
For those who want to run this locally, follow the setup guide below.
Follow the instructions here to install Git on your computer.
Open a terminal and run the following commands to clone this entire repository and open it locally.
git clone https://github.com/JackismyShephard/ultimate-rvc
cd ultimate-rvc
Run the following command to install the necessary dependencies on Windows:
./urvc.bat install
Note that this will install Miniconda in your user directory. The whole process may take upwards of 10 minutes, so grab a cup of coffee and wait.
Run the following command to install the necessary dependencies on Debian-based Linux distributions (e.g. Ubuntu):
./urvc.sh install
The command has been tested only on Ubuntu 22.04 and 24.04 so support for other distributions is not guaranteed. Also note that the command will install the CUDA 12.1 toolkit system-wide. In case you have problems, you may need to install the toolkit manually.
./urvc.bat run
./urvc.sh run
Once the following output message Running on local URL: http://127.0.0.1:7860 appears, you can click on the link to open a tab with the web app.
Navigate to the Download model subtab under the Manage models tab, and paste the download link to an RVC model and give it a unique name.
You may search the AI Hub Discord where already trained voice models are available for download.
The downloaded zip file should contain the .pth model file and an optional .index file.
Once the 2 input fields are filled in, simply click Download! Once the output message says [NAME] Model successfully downloaded!, you should be able to use it in the Generate song covers tab!
For people who have trained RVC v2 models locally and would like to use them for AI cover generations.
Navigate to the Upload model subtab under the Manage models tab, and follow the instructions.
Once the output message says Model with name [NAME] successfully uploaded!, you should be able to use it in the Generate song covers tab!
TBA
- From the Voice model dropdown menu, select the voice model to use.
- In the song input field, copy and paste the link to any song on YouTube, the full path to a local audio file, or select a cached input song.
- Pitch should be set to either -12, 0, or 12 depending on the original vocals and the RVC AI modal. This ensures the voice is not out of tune.
- Other advanced options for vocal conversion, audio mixing and etc. can be viewed by clicking the appropriate accordion arrow to expand.
Once all options are filled in, click Generate and the AI generated cover should appear in a less than a few minutes depending on your GPU.
TBA
Run the following command to pull latest changes from the repository and reinstall dependencies. Note that the process may take upwards of 5 minutes.
./urvc.bat update
./urvc.sh update
When developing new features or debugging, it is recommended to run the app in development mode. This enables hot reloading, which means that the app will automatically reload when changes are made to the code.
./urvc.bat dev
./urvc.sh dev
The use of the converted voice for the following purposes is prohibited.
-
Criticizing or attacking individuals.
-
Advocating for or opposing specific political positions, religions, or ideologies.
-
Publicly displaying strongly stimulating expressions without proper zoning.
-
Selling of voice models and generated voice clips.
-
Impersonation of the original owner of the voice with malicious intentions to harm/hurt others.
-
Fraudulent purposes that lead to identity theft or fraudulent phone calls.
I am not liable for any direct, indirect, consequential, incidental, or special damages arising out of or in any way connected with the use/misuse or inability to use this software.