diffuser img2Img code doesn't work
#5
by
perk11
- opened
I tried the diffusers I2I code, and it just doesn't follow the prompt at all. It does some random edits, but nothing that was aksed.
Input
Output
Prompt was "Replace raspberry with mouse"
Did you use the diffuser image 2 image pipeline? I think that one is simpilly adding noise to original image and re-denoise it again. You should use the GlmImagePipeline
Using the code from README, only changed the prompt and the input file name
import torch
from diffusers.pipelines.glm_image import GlmImagePipeline
from PIL import Image
pipe = GlmImagePipeline.from_pretrained("zai-org/GLM-Image", torch_dtype=torch.bfloat16, device_map="cuda")
image_path = "cond.jpg"
prompt = "Replace the background of the snow forest with an underground station featuring an automatic escalator."
image = Image.open(image_path).convert("RGB")
image = pipe(
prompt=prompt,
image=[image], # can input multiple images for multi-image-to-image generation such as [image, image1]
height=33 * 32, # Must set height even it is same as input image
width=32 * 32, # Must set width even it is same as input image
num_inference_steps=50,
guidance_scale=1.5,
generator=torch.Generator(device="cuda").manual_seed(42),
).images[0]
image.save("output_i2i.png")