README.md

Awesome Neural Radiance Field (NeRF) Editing

Welcome to awesome-nerf-editing🧙🏻‍♂️, your ultimate resource for exploring the exciting world of Neural Radiance Field (NeRF) editing! NeRF is revolutionizing computer vision and 3D graphics, and this curated collection of resources is your gateway to understanding and mastering NeRF-based editing techniques. Whether you're a researcher, developer, or simply curious about the cutting-edge applications of NeRF, you'll find a wealth of valuable information here.

🪬What is NeRF Editing?

NeRF Editing leverages the power of Neural Radiance Fields to manipulate and edit 3D scenes or objects in a highly realistic and immersive manner. This emerging field holds immense potential for a wide range of applications, including virtual reality, augmented reality, content creation, and more. However, neural radiance field implicitly encode shape and texture information of the scene or object, which implies natural difficulties in the NeRF editing task.

😶‍🌫️Why "awesome-nerf-editing"?

• Comprehensive Resources: We've curated an extensive list of papers and projects related to NeRF editing, making it easy for you to dive into this exciting field.
• Community Collaboration: Join our community of NeRF researchers, and developers. Share your insights, ask questions, and collaborate on groundbreaking projects.
• Stay Updated: We keep our resources up-to-date, ensuring that you're always at the forefront of NeRF research and development.

Recent Research [Latest Update: 2023/10/20☕]

🚨 Note: Please be aware that the acceptance status of some works might not be up-to-date. 📝💡

👋 Feel Welcome to Open an Issue or Pull Request! 🚀💬

• [2024]

• [2023]

  • [NeurIPS 2023] OBJect 3DIT: Language-guided 3D-aware Image Editing 丨 [Project Page]
  • [ICCV 2023 Oral] ENVIDR: Implicit Differentiable Renderer with Neural Environment Lighting 丨 [Project Page]
  • [ICCV 2023 Oral] Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions 丨 [Project Page]
  • [ICCV 2023] ICE-NeRF: Interactive Color Editing of NeRFs via Decomposition-Aware Weight Optimization
  • [ICCV 2023] ReNeRF: Relightable Neural Radiance Fields with Nearfield Lighting
  • [ICCV 2023] 3D-aware Blending with Generative NeRFs 丨 [Project Page]
  • [ICCV 2023] Vox-E: Text-guided Voxel Editing of 3D Objects 丨 [Project Page]
  • [ICCV 2023] 3D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capability 丨 [Project Page]
  • [ICCV 2023] Urban Radiance Field Representation with Deformable Neural Mesh Primitives 丨 [Project Page]
  • [ICCV 2023] DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields 丨 [Project Page]
  • [ICCV 2023] Learning Neural Implicit Surfaces with Object-Aware Radiance Fields
  • [ICCV 2023] X-Mesh:Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance 丨 [Project Page]
  • [ICCV 2023] Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Images
  • [ICCV 2023] DreamBooth3D: Subject-Driven Text-to-3D Generation 丨 [Project Page]
  • [ICCV 2023] Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction 丨 [Project Page]
  • [ICCV 2023] Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement 丨 [Project Page]
  • [ICCV 2023] Locally Stylized Neural Radiance Fields
  • [ICCV 2023] CC3D: Layout-Conditioned Generation of Compositional 3D Scenes 丨 [Project Page]
  • [ICCV 2023] IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis 丨 [Project Page]
  • [ICCV 2023] Dynamic Mesh-Aware Radiance Fields 丨 [Project Page]
  • [ICCV 2023] Blending-NeRF: Text-Driven Localized Editing in Neural Radiance Fields 丨 [Project Page]
  • [ICCV 2023] Text2Tex: Text-driven Texture Synthesis via Diffusion Models | [Project Page]
  • [ICCV 2023] Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models | [Project Page]
  • [ICCV 2023] Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis 丨 [Project Page]
  • [ICCV 2023] Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields 丨 [Project Page]
  • [ICCV 2023] FaceCLIPNeRF: Text-driven 3D Face Manipulation using Deformable Neural Radiance Fields 丨 [Project Page]
  • [ICCV 2023] Reference-guided Controllable Inpainting of Neural Radiance Fields 丨 [Project Page]
  • [ICCV 2023] Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior 丨 [Project Page]
  • [ICCV 2023] SKED: Sketch-guided Text-based 3D Editing 丨 [Project Page]
  • [ICCV 2023] MonoNeRF: Learning a Generalizable Dynamic Radiance Field from Monocular Videos | [code]
  • [ICCV 2023] ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field 丨 [Project Page]
  • [ICCVW 2023] LatentSwap3D: Semantic Edits on 3D Image GANs 丨 [Project Page]
  • [ICCVW 2023] S2RF: Semantically Stylized Radiance Fields
  • [ICCVW 2023] CoRF : Colorizing Radiance Fields using Knowledge Distillation
  • [ICCVW 2023] BluNF: Blueprint Neural Field 丨 [Project Page]
  • [ICCVW 2023] Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields 丨 [Project Page]
  • [TPAMI 2023] Interactive NeRF Geometry Editing With Shape Priors
  • [CVPR 2023 Highlight] Transforming Radiance Field with Lipschitz Network for Photorealistic 3D Scene Stylization
  • [CVPR 2023 Highlight] DiffRF: Rendering-Guided 3D Radiance Field Diffusion 丨 [Project Page]
  • [CVPRW 2023] FusedRF: Fusing Multiple Radiance Fields
  • [CVPRW 2023] SPIDR: SDF-based Neural Point Fields for Illumination and Deformation 丨 [Project Page]
  • [CVPR 2023] Interactive Segmentation of Radiance Fields 丨 Project Page
  • [CVPR 2023] NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Clouds 丨 [Project Page]
  • [CVPR 2023] PaletteNeRF: Palette-based Color Editing for NeRFs | [Project Page]
  • [CVPR 2023] SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields 丨 [Project Page]
  • [CVPR 2023] Removing Objects From Neural Radiance Fields | [Project Page]
  • [CVPR 2023] EditableNeRF: Editing Topologically Varying Neural Radiance Fields by Key Points | [Project Page]
  • [CVPR 2023] Local 3D Editing via 3D Distillation of CLIP Knowledge 丨 [Code]
  • [CVPR 2023] SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field 丨 [Project Page]
  • [CVPR 2023] PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision 丨 [Project Page]
  • [CVPR 2023] Magic3D: High-Resolution Text-to-3D Content Creation 丨 [Project Page]
  • [CVPR 2023] StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields 丨 [Project Page]
  • [CVPR 2023] Ref-NPR: Reference-Based Non-Photorealistic Radiance Fields for Controllable Scene Stylization 丨 [Project Page]
  • [CVPR 2023] Multi-Object Manipulation via Object-Centric Neural Scattering Functions 丨 [Project Page]
  • [CVPR 2023] UV Volumes for Real-time Rendering of Editable Free-view Human Performance 丨 [Project Page]
  • [CVPR 2023] Neural Fields meet Explicit Geometric Representations for Inverse Rendering of Urban Scenes 丨 [Project Page]
  • [CVPR 2023] TexIR:Multi-view Inverse Rendering for Large-scale Real-world Indoor Scenes 丨 [Project Page]
  • [CVPR 2023] TensoIR: Tensorial Inverse Rendering 丨 [Project Page]
  • [CVPR 2023] RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation 丨 [Project Page]
  • [CVPR 2023] I_{2}-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs 丨 [Project Page]
  • [CVPR 2023] Frequency-Modulated Point Cloud Rendering with Easy Editing 丨 [Code]
  • [CVPR 2023] Nerflets: Local Radiance Fields for Efficient Structure-Aware 3D Scene Representation from 2D Supervision 丨 [Project Page]
  • [CVPR 2023] HumanGen: Generating Human Radiance Fields with Explicit Priors
  • [CVPR 2023] NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animation
  • [ICLR 2023] General Neural Gauge Fields 丨 [Project Page]
  • [SIGGRAPH 2023] ClipFace: Text-guided Editing of Textured 3D Morphable Models | [Project Page]
  • [SIGGRAPH Asia 2023] DreamEditor: Text-Driven 3D Scene Editing with Neural Fields
  • [SIGGRAPH Asia 2023] SOL-NeRF: Sunlight Modeling for Outdoor Scene Decomposition and Relighting | [Project Page]
  • [AAAI 2023 Oral] Semantic 3D-aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Field 丨 [Code]
  • [AAAI 2023 Oral] One is All: Bridging the Gap Between Neural Radiance Fields Architectures with Progressive Volume Distillation | [Project Page]
  • [ACM MM 2023] Context-Aware Talking-Head Video Editing
  • [IJCAI 2023] RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models
  • [IJCAI 2023] TeSTNeRF: Text-Driven 3D Style Transfer via Cross-Modal Learning
  • [BMVC 2023] Breathing New Life into 3D Assets with Generative Repainting | [Project Page]
  • [WACV 2023] 3D GAN Inversion with Pose Optimization | [Project Page]
  • [WACV 2023] Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation | [Project Page]
  • [ICML 2023] RIP-NeRF: Learning Rotation-Invariant Point-based Neural

  ---

  • [arXiv] A Real-time Method for Inserting Virtual Objects into Neural Radiance Fields
  • [arXiv] Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints
  • [arXiv] MagicDrive: Street View Generation with Diverse 3D Geometry Control | [Project Page]
  • [arXiv] SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D | [Project Page]
  • [arXiv] Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models | [Project Page]
  • [arXiv] ED-NeRF: Efficient Text-Guided Editing of 3D Scene using Latent Space NeRF
  • [arXiv] ITEM3D: Illumination-Aware Directional Texture Editing for 3D Models
  • [arXiv] NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields | [Project Page]
  • [arXiv] MM-NeRF: Multimodal-Guided 3D Multi-Style Transfer of Neural Radiance Field
  • [arXiv] Controllable Dynamic Appearance for Neural 3D Portraits | [Project Page]
  • [arXiv] Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates
  • [arXiv] Compositional 3D Scene Generation using Locally Conditioned Diffusion | [Project Page]
  • [arXiv] Text-driven Editing of 3D Scenes without Retraining 丨 [Project Page]
  • [arXiv] Novel-view Synthesis and Pose Estimation for Hand-Object Interaction from Sparse Views 丨 [Project Page]
  • [arXiv] Watch Your Steps: Local Image and Scene Editing by Text Instructions 丨 [Project Page]
  • [arXiv] Focused Specific Objects NeRF
  • [arXiv] Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields 丨 [Project Page]
  • [arXiv] Car-Studio: Learning Car Radiance Fields from Single-View and Endless In-the-wild Images 丨 [Project Page]
  • [arXiv] Edit-DiffNeRF: Editing 3D Neural Radiance Fields using 2D Diffusion Model
  • [arXiv] Instruct-NeuralTalker: Editing Audio-Driven Talking Radiance Fields with Instructions
  • [arXiv] Edit-DiffNeRF: Editing 3D Neural Radiance Fields using 2D Diffusion Model
  • [arXiv] DORSal: Diffusion for Object-centric Representations of Scenes et al. 丨 [Project Page]
  • [arXiv] UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video 丨 [Project Page]
  • [arXiv] FDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Models 丨 [Code]
  • [arXiv] AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars 丨 [Project Page]
  • [arXiv] PlaNeRF: SVD Unsupervised 3D Plane Regularization for NeRF Large-Scale Scene Reconstruction
  • [arXiv] InpaintNeRF360: Text-Guided 3D Inpainting on Unbounded Neural Radiance Fields
  • [arXiv] OR-NeRF: Object Removing from 3D Scenes Guided by Multiview Segmentation with Neural Radiance Fields 丨 [Project Page]
  • [arXiv] Unsupervised Object-Centric Voxelization for Dynamic Scene Understanding
  • [arXiv] NaviNeRF: NeRF-based 3D Representation Disentanglement by Latent Semantic Navigation
  • [arXiv] UVA: Towards Unified Volumetric Avatar for View Synthesis, Pose rendering, Geometry and Texture Editing
  • [arXiv] PVD-AL: Progressive Volume Distillation with Active Learning for Efficient Conversion Between Different NeRF Architectures 丨 [Project Page]
  • [arXiv] VMesh: Hybrid Volume-Mesh Representation for Efficient View Synthesis 丨 [Project Page]
  • [arXiv] CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layout
  • [arXiv] TEGLO: High Fidelity Canonical Texture Mapping from Single-View Images
  • [arXiv] Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenes
  • [arXiv] Interactive Geometry Editing of Neural Radiance Fields
  • [arXiv] MovingParts: Motion-based 3D Part Discovery in Dynamic Radiance Field 丨 [Project Page]
  • [arXiv] IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF 丨 [Project Page]
  • [arXiv] LC-NeRF: Local Controllable Face Generation in Neural Randiance Field
  • [arXiv] In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition 丨 [Project Page]
  • [arXiv] RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes | [Project Page]

• [2022]

  • [NeurIPS 2022 Spotlight] Neural Shape Deformation Priors | [Project Page]
  • [NeurIPS 2022 Spotlight] TANGO: Text-driven Photorealistic and Robust 3D Stylization via Lighting Decomposition 丨 [Project Page]
  • [NeurIPS 2022] NeuPhysics: Editable Neural Geometry and Physics from Monocular Videos 丨 [Project Page]
  • [NeurIPS 2022] Compressible-composable NeRF via Rank-residual Decomposition 丨 [Project Page]
  • [NeurIPS 2022] CageNeRF: Cage-based Neural Radiance Fields for Genrenlized 3D Deformation and Animation 丨 [Project Page]
  • [NeurIPS 2022] Decomposing NeRF for Editing via Feature Field Distillation 丨 [Project Page]
  • [NeurIPS 2022] SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections 丨 [Project Page]
  • [NeurIPS 2022] Unsupervised Multi-View Object Segmentation Using Radiance Field Propagation 丨 [Project Page]
  • [NeurIPS 2022] D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Video
  • [ECCV 2022] MoFaNeRF: Morphable Facial Neural Radiance Field | [Code]
  • [ECCV 2022] LaTeRF: Label and Text Driven Object Radiance Fields
  • [ECCV 2022] NeRF for Outdoor Scene Relighting | [Project Page]
  • [ECCV 2022] Deforming Radiance Fields with Cages | [Project Page]
  • [ECCV 2022] Object-Compositional Neural Implicit Surfaces | [Project Page]
  • [ECCV 2022] ARF: Artistic Radiance Fields | [Project Page]
  • [ECCV 2022] Unified Implicit Neural Stylization | [Project Page]
  • [ECCV 2022] Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis 丨 [Project Page]
  • [CVPR 2022 Oral] Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields | [Project Page]
  • [CVPR 2022] NeRF-Editing: Geometry Editing of Neural Radiance Fields | [Code]
  • [CVPR 2022] CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields | [Project Page]
  • [CVPR 2022] CoNeRF: Controllable Neural Radiance Fields | [Project Page]
  • [CVPR 2022] Zero-Shot Text-Guided Object Generation with Dream Fields | [Project Page]
  • [CVPR 2022] StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learning | [Project Page]
  • [CVPR 2022] AutoRF: Learning 3D Object Radiance Fields from Single View Observations | [Project Page]
  • [CVPR 2022] Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation | [Project Page]
  • [CVPR 2022] RigNeRF: Fully Controllable Neural 3D Portraits | [Project Page]
  • [CVPR 2022] NeRFReN: Neural Radiance Fields with Reflections | [Project Page]
  • [CVPR 2022] FENeRF: Face Editing in Neural Radiance Fields | [Project Page]
  • [ICLR 2022] Unsupervised Discovery of Object Radiance Fields | [Project Page]
  • [SIGGRAPH 2022] AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars | [Project Page]
  • [SIGGRAPH Asia 2022] FDNeRF: Few-shot Dynamic Neural Radiance Fields for Face Reconstruction and Expression Editing
  • [SIGGRAPH Asia 2022] Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video | [Project Page]
  • [3DV 2022 Oral] Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations | Project Page
  • [3DV 2022] Controllable Radiance Fields for Dynamic Face Synthesis | [Project Page]
  • [PG & CGF 2022] Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects

  ---

  • [arXiv] Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion | [Project Page]
  • [arXiv] NeRFEditor: Differentiable Style Decomposition for Full 3D Scene Editing | [Project Page]
  • [arXiv] SSDNeRF: Semantic Soft Decomposition of Neural Radiance Fields | [Project Page]
  • [arXiv] ONeRF: Unsupervised 3D Object Segmentation from Multiple Views
  • [arXiv] FLNeRF: 3D Facial Landmarks Estimation in Neural Radiance Fields
  • [arXiv] NeRFFaceEditing: Disentangled Face Editing in Neural Radiance Fields
  • [arXiv] Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing
  • [arXiv] Estimating Neural Reflectance Field from Radiance Field using Tree Structures
  • [arXiv] PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereo
  • [arXiv] NeRF-In: Free-Form NeRF Inpainting with RGB-D Priors
  • [arXiv] ObPose: Leveraging Pose for Object-Centric Scene Inference and Generation in 3D
  • [arXiv] ERF: Explicit Radiance Field Reconstruction From Scratch
  • [arXiv] NeuVV: Neural Volumetric Videos with Immersive Rendering and Editing

• [2021]

  • [ICCV 2021] CodeNeRF: Disentangled Neural Radiance Fields for Object Categories | [Project Page]
  • [ICCV 2021] Editing Conditional Radiance Fields | [Project Page]
  • [ICCV 2021] Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering | [Project Page]
  • [ICCV 2021] Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video | [Project Page]
  • [ICCV 2021] AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis | [Project Page]
  • [SIGGRAPH Asia 2021] NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination | [Project Page]

  ---

  • [arXiv] DIVeR: Real-time and Accurate Neural Radiance Fields with Deterministic Integration for Volume Rendering
  • [arXiv] StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
  • [arXiv] Editable Free-viewpoint Video Using a Layered Neural Representation

• [2020]

  • [NeurIPS 2020 Spotlight] Neural Sparse Voxel Fields

TODO List

Currently we only provide a chronological grouping of papers, and we will categorize papers by topic by the end of 2023.

• [TODO#1] Language-guided
• [TODO#2] Color Editing
• [TODO#3] Deformation

Get Involved and be a contributor!

• Explore our curated list of resources.
• Contribute to our GitHub repository and help us expand this resource.
• Share your own NeRF editing projects and research with the community.

We welcome contributions to expand and improve this collection. If you have relevant resources to share or want to collaborate on this project, please follow these guidelines:

1. Fork this repository.
2. Add your resources to the appropriate sections in the README.
3. Commit your changes.
4. Push to the branch: git push origin master.
5. Submit a pull request.

Please ensure that the resources you contribute are relevant to NeRF-based editing techniques.

License

This project is licensed under the MIT License.

Citation

@misc{weize2023nerfeditingpaperslist,
title = {Awesome-NeRF-Editing},
author = {Weize Li, Tianshu Kuai},
journal = {GitHub repository},
url = {https://github.com/EricLee0224/awesome-nerf-editing},
year = {2023}
}