Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unexpected prompt behavior with xyPlot and list of inputs #127

Open
rjgoif opened this issue Jun 14, 2024 · 11 comments
Open

unexpected prompt behavior with xyPlot and list of inputs #127

rjgoif opened this issue Jun 14, 2024 · 11 comments

Comments

@rjgoif
Copy link

rjgoif commented Jun 14, 2024

Take the following workflow (which uses a custom node that is one of many that will iterate over a list to add to the queue)

The prompt output box suggests that the prompt is correctly changing per iteration from the loader, but the Ksampler only "sees" the first prompt.

I tried using pipe and non-pipe versions, no difference. I updated my nodes, also no difference. I also tried some similar "string list" nodes as inputs, still no effect.

Any ideas as to how to fix the problem? Some of my iterated prompt lists are huge and I can't input them any other way without manually (or with an OS level script) inputting them. Problem with this is that comfy starts to have input lag after like 50-60 prompts, and the internals don't get updated fast enough leading to shuffled prompt orders or duplicated/dropped prompts. It's a mess.

unexpected prompt decoupling

Note I edited the prompt list before submitting the image. That's why there are three output prompts but only two inputs. Ignore that.

@TinyTerra
Copy link
Owner

Interesting... I'll have to have a look into how the custom node works behind the scenes. If its purely queueing a new prompt with an enumerated text output, I'd expect the plot to output separate xy's for each positive (ie 3 lots of two images for the cfg change) not a group of 6...
If it 'queues' within the same execution then i can understand why its messing up.

But in any case something seems weird there.

What is your expected outcome? or what are you trying to achieve more specifically?
I'm assuming 3 sets of 2 image plots?

I don't have time right now, but I'll have a closer look and work on a fix tomorrow.

@rjgoif
Copy link
Author

rjgoif commented Jun 14, 2024

What is your expected outcome? or what are you trying to achieve more specifically? I'm assuming 3 sets of 2 image plots?

In my example I didn't hook up an xyplot preview, but yes. The first prompt in the prompt list should have fed in triggering two output images, then the second prompt triggering two new images with prompt 2. The really strange thing is that the loader node is correctly outputting a list of two prompt strings. But the strings that are leaving the loader node are not those that are used as inputs in the sampler node.

If I hook up a preview to the xyplot image output, I will get two grids, both with the first prompt.

Of note, if I don't use the xyplot option, everything works as expected: the sampler receives a list of two inputs and creates a list of two correct outputs.

I'd upload more examples, but I am at work and can't comfy from here (sadly).

@rjgoif
Copy link
Author

rjgoif commented Jun 14, 2024

Here we have an example working as expected. A list of prompts goes in, and a list of outputs matching the prompts comes out.

unexpected prompt decoupling 02

Next, another variation showing the bug. I even process the list of text prompts separately with their own clip encoder after verifying that the loader node took them as inputs correctly. Still doesn't work.
unexpected prompt decoupling 03

It appears that the bug is specific to how the xy node works, not the loader/sampler nodes.

@TinyTerra
Copy link
Owner

Okay, I see what the issue is. The xyplot wasn't designed with list inputs in mind for the actual plotting side.
(the execution uses core comfy elements which is why the correct 4 images are being output individually in temp).

This might be a little more involved than i initially thought, but i'll work on it.

@rjgoif
Copy link
Author

rjgoif commented Jun 16, 2024

I guess another option would be to add a Z axis which was a variable that could have a new XY plot under each value. That's what other XY plotters do (e.g. A1111, xyAny)

From there, I guess you could make an arbitrary number of dimensions, too. an XY plot for combo of values from Z1, Z2, Z3, etc.

@TinyTerra
Copy link
Owner

Sorry i haven't had as much time to work on this.

I have it "working" with lists to a point, except now it doubles up on the outputs.
I sort of expected the comfy list support to be within the base sampling functions - but somehow instead it re-executes the entire node. as of yet i haven't had much luck back tracing how that works in the base comfy code.

If i get the list support working properly, i probably wont add the Z axis - adding it properly with the list input as well will get too complex... But maybe on that note it would make more sense to use Z-axis instead of supporting lists..

@TinyTerra
Copy link
Owner

Okay, adding the Z-Axis was easier than tracing through comfys code..
I'll also add a advPlot String node (Similar to the range one) for node based string plotting instead of having to parse your longer lists into the xyPlot syntax.

Just want to test it a bit further before pushing

@rjgoif
Copy link
Author

rjgoif commented Jun 17, 2024

Does it output a series of XY images, one for each Z value? Or does it concatenate it all into one giant superimage?

@TinyTerra
Copy link
Owner

a series of XY plots

image

@TinyTerra
Copy link
Owner

I've pushed the changes if you want to update and have a play with it.

@rjgoif
Copy link
Author

rjgoif commented Jun 21, 2024

Hey this works really well! I want to try some creative ways to expand Z into combinatorial iterations, but I am crunched at work for the moment.

But great work getting the implementation up and running! I think anyone scared of using Comfy vs A1111 can now get basically full functionality from just your pack alone without having to learn the "scary" noodle constructions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants