Codeformer messing somes faces and comparison with upscaled inswapper.

[a3nima]

Faces that are NOT the target image are still being altered. Checked with img2img and inpaint on latest branch A1111/faceswaplab.

The target face is being changed and this feature works as intended. But there are people in the background and around the scene, there images are not changed like the target image but some small changes happen to them. Its especially obvious in my case because theyre in a depth of field area, not sharp and after faceswap there are small parts around or on them that are being sharpened, though it's NOT a non-intended faceswap.

I tried to circumvent the problem by going to inpainting and masking the face that should be swapped only, but as I expected, this is overriden and the result is the same.

[glucauze]

Okay, either I didn't understand exactly what the problem was. Either the program doesn't behave as you'd expect (in which case it's a documentation or enhancement issue). Or there's a bug I can't reproduce.

Here's an example with inpainting.

Original image :

Mask (not grayscale, just the selected part :

Result :

I made sure to deactivate codeformer in postprocessing, which can cause artifacts. I used the sort_by_size option with face 0.

As you can see, the other faces have not been affected by the changes.

On the other hand, of course, if you use post-processing, the options will apply to the whole image and may therefore modify the image and other faces.

[a3nima]

hi. thanks for your answer. adetailer was the last process in the making of the image. but maybe its codeformer that has done that. the target image worked well but other faces have some sharpened parts on them. I'll try if it occurs without codeformer too.
I set inpainting with masked only still it happened. what inpainting settings have you used, and did you use the standard img2img inpainting (tab) ?

[glucauze]

Here are the details of the previous generation (I think). I did not used any prompt.

Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 3581328725, Size: 512x512, Model hash: bc7da767df, Model: consistent_inpainting.inpainting, Denoising strength: 0.75, Conditional mask weight: 1.0, Mask blur: 4, CFG Rescale phi: 0, Version: v1.5.1, similarity = {}, ref_similarity = {}

I did not use any postprocessing :

The idea of post-processing was to improve the poor results of insightface. I hardly use it anymore (except for the post-inpainting part, which also affects the whole image).

I prefer to use the upscaled inswapper which only affects the targetted face.

You really have to play with the parameters, because what works for one image doesn't always work for another.

As for codeformer, here is what i've got when it is enabled. The faces is the backgroud are hideous :

[a3nima]

yes, it's the codeformer causing the problem. just found out by testing, too..
if it could be prevented, would be nice, maybe in a future version. and thanks again!

[a3nima]

I prefer to use the upscaled inswapper which only affects the targetted face.

You really have to play with the parameters, because what works for one image doesn't always work for anothe

Ok I didnt see these options because they're under the settings tab. maybe it would be better to have them at the extension's tab/area ?

Thanks for your detailed answers incl. workflow/images!

[a3nima]

on the first try with a close up photo the result without inswapper and only codeformer in postprocessing (i think thats what roop does too), was more convincing. but I'll test further

[glucauze]

The upscaled inswapper is available in sd settings (the global settings > FaceSwapLab).

It should be better, but it depends on what you do :)

[a3nima]

I did not find it to work as well in my test (1. inswapper with your settings, 2. codeformer only).

I used SwinIR-4x as I've already had it installed (or comes with A1111?). Not sure if your SwinIR 4x is a different one?

[glucauze]

If you are using codeformer in inswapper, you should disable codeformer in postprocessing. Otherwise you will have 2 pass of codeformer which might be a bit too much. I think it is what happened here. You can also disable sharppening and color correction if you want.

Yes SwinIR 4x is worse than LDSR or the one i am using i think. You should try different ones to see what is the best for you.

[a3nima]

I will rerun my process, but as far as I remember, I did use your settings (with codeformer in the inswapper activated) and deactivated on the post-process tab.

without the sharpener and with ldsr it looks ok, but after all the codeformer in post is better for my taste, it also seems to blend in the edges of the mask better.

is there a chance the postprocessing face restoring could be limited to the swapped faces only in a future version?

[a3nima]

Currently using it like this and inpainting with low denoising at postprocessing to get it to fit more to the image's style. no postprocessing on the first page, if not inpainting. gonna try further and report back if I change it again ^^

[glucauze]

Version 1.2.0 will add pre-inpainting and post inpainting settings for upscaler for each faces unit. This will allow more customization per faces. The post-processing will become less useful i think