Inpaint Anything performs stable diffusion inpainting on a browser UI using any mask selected from the output of Segment Anything.
Using Segment Anything enables users to specify masks by simply pointing to the desired areas, instead of manually filling them in. This can increase the efficiency and accuracy of the mask creation process, leading to potentially higher-quality inpainting results while saving time and effort.
Extension version for AUTOMATIC1111's Web UI
Please follow these steps to install the software:
- Create a new conda environment:
conda create -n inpaint python=3.10
conda activate inpaint
- Clone the software repository:
git clone https://github.com/Uminosachi/inpaint-anything.git
cd inpaint-anything
- For the CUDA environment, install the following packages:
pip install -r requirements.txt
- If you are using macOS, please install the package from the following file instead:
pip install -r requirements_mac.txt
python iasam_app.py
- Open http://127.0.0.1:7860/ in your browser.
- Note: If you have a privacy protection extension enabled in your web browser, such as DuckDuckGo, you may not be able to retrieve the mask from your sketch.
--save-seg
: Save the segmentation image generated by SAM.--offline
: Execute inpainting using an offline network.--sam-cpu
: Perform the Segment Anything operation on CPU.
- Launch this application.
- Click on the
Download model
button, located next to the Segment Anything Model ID. This includes the SAM 2, Segment Anything in High Quality Model ID, Fast Segment Anything, and Faster Segment Anything (MobileSAM).- Please note that the SAM is available in three sizes: Base, Large, and Huge. Remember, larger sizes consume more VRAM.
- Wait for the download to complete.
- The downloaded model file will be stored in the
models
directory of this application's repository.
- Drag and drop your image onto the input image area.
- Outpainting can be achieved by the
Padding options
, configuring the scale and balance, and then clicking on theRun Padding
button. - The
Anime Style
checkbox enhances segmentation mask detection, particularly in anime style images, at the expense of a slight reduction in mask quality.
- Outpainting can be achieved by the
- Click on the
Run Segment Anything
button. - Use sketching to point the area you want to inpaint. You can undo and adjust the pen size.
- Hover over either the SAM image or the mask image and press the
S
key for Fullscreen mode, or theR
key to Reset zoom.
- Hover over either the SAM image or the mask image and press the
- Click on the
Create mask
button. The mask will appear in the selected mask image area.
Expand mask region
button: Use this to slightly expand the area of the mask for broader coverage.Trim mask by sketch
button: Clicking this will exclude the sketched area from the mask.Add mask by sketch
button: Clicking this will add the sketched area to the mask.
- Enter your desired Prompt and Negative Prompt, then choose the Inpainting Model ID.
- Click on the
Run Inpainting
button (Please note that it may take some time to download the model for the first time).- In the Advanced options, you can adjust the Sampler, Sampling Steps, Guidance Scale, and Seed.
- If you enable the
Mask area Only
option, modifications will be confined to the designated mask area only.
- Adjust the iteration slider to perform inpainting multiple times with different seeds.
- The inpainting process is powered by diffusers.
- You can directly drag and drop the inpainted image into the input image field on the Web UI. (useful with Chrome and Edge browsers)
- The inpainting model, which is saved in HuggingFace's cache and includes
inpaint
(case-insensitive) in its repo_id, will also be added to the Inpainting Model ID dropdown list.- If there's a specific model you'd like to use, you can cache it in advance using the following Python commands:
python
from diffusers import StableDiffusionInpaintPipeline pipe = StableDiffusionInpaintPipeline.from_pretrained("Uminosachi/dreamshaper_5-inpainting") exit()
- The model diffusers downloaded is typically stored in your home directory. You can find it at
/home/username/.cache/huggingface/hub
for Linux and MacOS users, or atC:\Users\username\.cache\huggingface\hub
for Windows users.- When executing inpainting, if the following error is output to the console, try deleting the corresponding model from the cache folder mentioned above:
An error occurred while trying to fetch model name...
- Choose the Cleaner Model ID.
- Click on the
Run Cleaner
button (Please note that it may take some time to download the model for the first time). - Cleaner process is performed using Lama Cleaner.
- Gives ability to just save mask without any other processing, so it's then possible to use the mask in other graphic applications.
Get mask as alpha of image
button: Save the mask as RGBA image, with the mask put into the alpha channel of the input image.Get mask
button: Save the mask as RGB image.
- The inpainted image will be automatically saved in the folder that matches the current date within the
outputs
directory.
With the Inpaint Anything library, you can perform segmentation and create masks using sketches from other applications.
The source code is licensed under the Apache 2.0 license.
- Ravi, N., Gabeur, V., Hu, Y.-T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädel, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K. V., Carion, N., Wu, C.-Y., Girshick, R., Dollár, P., & Feichtenhofer, C. (2024). SAM 2: Segment Anything in Images and Videos. arXiv preprint.
- Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W-Y., Dollár, P., & Girshick, R. (2023). Segment Anything. arXiv:2304.02643.
- Ke, L., Ye, M., Danelljan, M., Liu, Y., Tai, Y-W., Tang, C-K., & Yu, F. (2023). Segment Anything in High Quality. arXiv:2306.01567.
- Zhao, X., Ding, W., An, Y., Du, Y., Yu, T., Li, M., Tang, M., & Wang, J. (2023). Fast Segment Anything. arXiv:2306.12156 [cs.CV].
- Zhang, C., Han, D., Qiao, Y., Kim, J. U., Bae, S-H., Lee, S., & Hong, C. S. (2023). Faster Segment Anything: Towards Lightweight SAM for Mobile Applications. arXiv:2306.14289.