0xkojiAs you may know last month, people were passionate about generating QR codes with Stable Diffusion.
Actually, I tried to generate some QR codes with Automatic1111.
It was fun but at the same time, it was kind of hard since the success rate to generate a scannable QR code was almost 50% 🥲
Then I found an interesting modal for generating QR codes on Hugging Face.
https://huggingface.co/DionTimmer/controlnet_qrcode-control_v1p_sd15
In this post, I will introduce a way to generate a QR code with controlnet_qrcode-control_v1p_sd15 and v2_1.
This post will use Google Colab (free plan).
generate a QR code
First, we need to generate a QR code.
You can generate a QR code with python package(I use that when I need to create multiple QR codes).
But for this case, we won't need to create many QR codes so I recommend you to use this, https://keremerkan.net/qr-code-and-2d-code-generator/
install packages
!pip install diffusers transformers accelerate torch xformers
import packages
import torch
from PIL import Image
from diffusers import StableDiffusionControlNetImg2ImgPipeline, ControlNetModel, DDIMScheduler
from diffusers.utils import load_image
load pre_trainned model controlnet_qrcode-control and stable diffusion model
If you want to try v1.5, please comment out the bottom 2 line and remove the comment from the lines for v1.5.
# v1.5
# controlnet = ControlNetModel.from_pretrained("DionTimmer/controlnet_qrcodecode-control_v1p_sd15", torch_dtype=torch.float16)
# pipe = StableDiffusionControlNetImg2ImgPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", controlnet=controlnet, safety_checker=None, torch_dtype=torch.float16)
# v2.1
controlnet = ControlNetModel.from_pretrained("DionTimmer/controlnet_qrcode-control_v11p_sd21", torch_dtype=torch.float16)
pipe = StableDiffusionControlNetImg2ImgPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", controlnet=controlnet, safety_checker=None, torch_dtype=torch.float16)
pipe.enable_xformers_memory_efficient_attention()
pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config)
pipe.enable_model_cpu_offload()
generate a qr code
For this step, what you need to do is the followings.
- Upload the QR code you generated to your Google Colab folder
- Get the source image / the source image URL If you want an image that is uploaded somewhere, you can use the image URL directly The image must be square. The image must be square.
- Set prompt and negative prompt
- Run (it will take 1-2min)
def resize_image(input_image: Image, resolution: int):
input_image = input_image.convert("RGB")
W, H = input_image.size
k = float(resolution) / min(H, W)
H *= k
W *= k
H = int(round(H / 64.0)) * 64
W = int(round(W / 64.0)) * 64
img = input_image.resize((W, H), resample=Image.LANCZOS)
return img
orriginal_qr_code_image = load_image('/content/code_with_block.png')
# img_path = 'https://images.squarespace-cdn.com/content/v1/59413d96e6f2e1c6837c7ecd/1536503659130-R84NUPOY4QPQTEGCTSAI/15fe1e62172035.5a87280d713e4.png'
img_path = load_image('/content/test.png')
init_image = load_image(img_path)
condition_image = resize_image(orriginal_qr_code_image, 768)
init_image = resize_image(init_image, 768)
generator = torch.manual_seed(123121231)
image = pipe(prompt="8k, 3d, ((photo-realistic)), bonsai, Japanese style",
negative_prompt="ugly, disfigured, low quality, blurry, nsfw, worst quality, illustration, drawing",
image=init_image,
control_image=condition_image,
width=768,
height=768,
guidance_scale=20,
controlnet_conditioning_scale=2.5,
generator=generator,
strength=0.9,
num_inference_steps=150,
)
image.images[0]
Google Colab code
https://colab.research.google.com/drive/1D2V-5hebvOeHJOOK19l0Bujqh4MlcLTl?usp=sharing
Right now, I'm trying Gradio to add the UI for the above script.
