Project Description
Generate a personalized avatar artwork images by fine-tuning stable diffusion on personal pictures
Goal for this Hackweek
Get a new fancy and unique avatar!
Resources
- https://huggingface.co/docs/diffusers/using-diffusers/sdxl
- https://huggingface.co/docs/diffusers/training/dreambooth
- https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_sdxl.md
- https://civitai.com/models/133005/juggernaut-xl?modelVersionId=198530
Looking for hackers with the skills:
This project is part of:
Hack Week 23
Activity
Comments
-
10 months ago by STorresi | Reply
These are generated after a bespoke LoRA training using DreamBooth over the JuggernautXL model, which in turn is based on SDXL 1.0.
As you can see, hands are still tricky (a known issue of diffusion models, apparently), but I didn't try inpainting and img2img fine-tuning, which are supposed to be the go-to way to solve small issues like that. I must say the overall experience was quite painful due to the hardware requirements of SDXL and the amount of memory leaks in pytorch. A high-end consumer grade GPU like an NVIDIA 4080 with 16GB of VRAM often wasn't enough and ran OOM.
Similar Projects
This project is one of its kind!