diffusers

--- license: openrail++ base_model: stabilityai/stable-diffusion-xl-base-1.0 tags: - stable-diffusion-xl - stable-diffusion-xl-diffusers - text-to-image - diffusers - inpainting inference: false ---

These are controlnet weights trained on stabilityai/stable-diffusion-xl-base-1.0 with canny conditioning. You can find some example images in the following. prompt: a couple watching a romantic sunset, 4k photo prompt: a woman, close up, detailed, beautiful, street photography, photorealistic, detailed, Kodak ektar 100, natural, candid shot prompt: Cinematic, neoclassical table in the living room, cinematic, contour, lighting, highly detailed, winter, golden hour prompt: a tornado hitting grass field, 1980's film grain. overcast, muted colors. To more details, check out the official documentation of `StableDiffusionXLControlNetPipeline`. Our training script was built on top of the official training script that we provide here. Training data This checkpoint was first trained for 20,000 steps on laion 6a resized to a max minimum dimension of 384. It was then further trained for 20,000 steps on laion 6a resized to a max minimum dimension of 1024 and then filtered to contain only minimum 1024 images. We found the further high resolution finetuning was necessary for image quality. Batch size Data parallel with a single gpu batch size of 8 for a total batch size of 64. Hyper Parameters Constant learning rate of 1e-4 scaled by batch size for total learning rate of 64e-4

These are ControlNet weights trained on stabilityai/stable-diffusion-xl-base-1.0 with zoe depth conditioning. Zoe-depth is an open-source SOTA depth estimation model which produces high-quality depth maps, which are better suited for conditioning. To more details, check out the official documentation of `StableDiffusionXLControlNetPipeline`. Our training script was built on top of the official training script that we provide here. Training data and Compute The model is trained on 3M image-text pairs from LAION-Aesthetics V2. The model is trained for 700 GPU hours on 80GB A100 GPUs. Batch size Data parallel with a single gpu batch size of 8 for a total batch size of 256.

Visual comparison of Flux-dev model outputs using BF16 and torchao float8weightonly quantization To use this quantized FLUX.1 [dev] checkpoint, you need to install the 🧨 diffusers and torchao library: After installing the required library, you can run the following script: This checkpoint was created with the following script using "black-forest-labs/FLUX.1-dev" checkpoint:

Visual comparison of Flux-dev model outputs using BF16 and torchao int4weightonly quantization To use this quantized FLUX.1 [dev] checkpoint, you need to install the 🧨 diffusers and torchao library: For now, we require this specific branch in diffusers library to fix an error when loading the model After installing the required library, you can run the following script: This checkpoint was created with the following script using "black-forest-labs/FLUX.1-dev" checkpoint: