A few days ago, I had a simple experience with sdxl, and I must say the effect was very stunning. Although the original model of sd is just average, the idea of xl is nothing short of opening up a new generation for AI drawing.
Optimization Refine#
Unlike sdxl and other ordinary models, this model has released two versions. One is the base model, which is commonly used for generating images, but there is also a refiner model. The refiner model does not participate in image generation; instead, it intervenes in the final stage of iteration to optimize and repair the images. In other words, sdxl actually requires running two models for one image. After the base model completes the basic image generation, it hands it over to the refiner for optimization. This approach indeed improves the image quality significantly. Many areas prone to errors in drawing have been resolved in sd.
In fact, when combined with lora and cn, it is possible to produce a very good image.
Drawbacks#
The drawbacks are also quite apparent. The sdxl model this time is very large, with the two models totaling over 14GB (perhaps due to the variety of styles). Additionally, sdxl has increased the requirements for computer configuration once again. The webui version requires 8GB of VRAM to run smoothly. For low-end machines, it is recommended to use comfyui for a better experience.
Image Generation Method#
For image generation, if you are using a version before 1.6.0, you can first use the base model to generate the image and then run the refiner model through the image generation tool to achieve similar results. If your sdwebui version is 1.6.0 or later, the webui already comes with the refiner plugin, so you just need to select the model and adjust the switching timing to generate the image directly.
Multi-Style#
Another feature of the sdxl model this time is multi-style, meaning it can draw in various styles (although the results may be average for realistic human drawings and below average for anime drawings). It is recommended to install a plugin called sdxl_styles to easily switch between styles, or else you will end up with a bunch of tags to remember.
Conclusion#
Although the original model of sdxl is still not great, it requires fine-tuning and secondary development. However, this new approach to image generation is nothing short of revolutionary for AI drawing. Looking forward to more great images!
This article is synchronized and updated by Mix Space to xLog.
The original link is https://blog.xiaohan-kaka.me/posts/ai/sdxl