Conjunction Issues about structured-diffusion-guidance HOT 1 OPEN

Thanks for the question. v_c[-1:] is a sequence where the tokens of "B" in prompt "A and B" is replaced by the output sequence from "B" prompt. If I understand correctly, v_c[-2:-1] should be the same as the output sequence from the full prompt, i.e., v_c[0]. Perhaps the single_align function in text2img_demo.py is less confusing.

When I did a preliminary experiment to compare using v_c[0] and v_c[-1], I found that the former helps with details, and the latter better mitigates the issue of the missing object. From my limited observations on like 100 images, I think the former is a better choice considering both color binding and missing objects. So I went with v_c[-1] on all "A and B" prompts.

More recently, I found that using v_c[-1] is potentially equivalent to assigning a very large weight to V_k in eq. 4. (For now, eq. 4 is an equally weighted average but can be expanded to a weighted sum by designing some dynamic weighting mechanism) I am still looking into this and seeing if there are any empirical explanations.

from structured-diffusion-guidance.