This folder contains the implementations for subject-driven generation task.
conda create -n SdG python=3.10
conda activate SdG
pip install -r requirements.txt
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install .
accelerate config
- cat/ contains the training images. It is one folder from the DreamBooth dataset. You can also change cat/ with another folder in DreamBooth dataset.
- peft/ contains all the codes related to PEFT.
- train.sh is the training script.
- infer.py is the inference code.
The training script provides some PEFT methods, such as LoRA and LoRA-Dash, to fine-tune SDXL model for this task.
bash train.sh
instance_prompt
: the prompt for the input images.validation_prompt
: the prompt for validation.lora_use_dash
: whether to use LoRA-Dash.
After training, run the following commands to generate the images:
python infer.py
Considering that you may have different computing resources, we have tested that this task can be conducted on one RTX 3090 GPU.
If you encounter any issues, please refer to this link: huggingface/diffusers/examples. It covers the majority of problems you may encounter. Additionally, you are also welcome to contact us by submitting an issue or via email.
This directory is modified based on huggingface/diffusers/examples. We greatly appreciate their remarkable works.