Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations

Rectified flows for image inversion and editing. Our approach efficiently inverts reference style images in (a) and (b) without requiring text descriptions of the images and applies desired edits based on new prompts (e.g. “a girl” or “a dwarf”). For a reference content image (e.g. a cat in (c) or a face in (d)), it performs semantic image editing e.g. “ sleeping cat”) and stylization (e.g. “a photo of a cat in origmai style”) based on prompts, without leaking unwanted content from the reference image (input images have orange borders).

🔥 Updates

• [2024.10.15] Code reimplemented by open-source ComfyUI community, thanks logtd!
• [2024.10.14] Paper is published on arXiv!

🚀 Comfy User Interface

Try ComfyUI for better experience: ComfyUI Node🔥. Follow the guidelines below to setup locally.

Install ComfyUI to run flux

1. cd ComfyUI

2. python main.py

Install ComfyUI-Manager.

1. cd ComfyUI/custom_nodes

2. git clone https://github.com/ltdrdata/ComfyUI-Manager.git

3. cd ..

4. python main.py

Install RF-Inversion ComfyUI Node

1. Click on "Manager"
2. Install via Git URL: https://github.com/logtd/ComfyUI-Fluxtapoz
3. If you see error, change security level in ComfyUI/custom_nodes/ComfyUI-Manager/config.ini from "normal" to "weak"

4. cd ComfyUI

5. python main.py

6. Copy RF-Inversion workflow and paste on the ComfyUI window.
7. Install missing custom nodes in Manager
8. Click on "Queue Prompt" to see the result
9. Tune hyper-parameters (such as eta, start_step, stop_step) to get the desired outcome

Citation

@article{rout2024rfinversion,
  title={Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations},
  author={Litu Rout and Yujia Chen and Nataniel Ruiz and Constantine Caramanis and Sanjay Shakkottai and Wen-Sheng Chu},
  journal={arXiv preprint arXiv:2410.10792},
  year={2024}
}

Licenses

Copyright © 2024, Google LLC. All rights reserved.