Project Description

Generate a personalized avatar artwork images by fine-tuning stable diffusion on personal pictures

Goal for this Hackweek

Get a new fancy and unique avatar!

Resources

  • https://huggingface.co/docs/diffusers/using-diffusers/sdxl
  • https://huggingface.co/docs/diffusers/training/dreambooth
  • https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_sdxl.md
  • https://civitai.com/models/133005/juggernaut-xl?modelVersionId=198530

Looking for hackers with the skills:

ai stable-diffusion imaging

This project is part of:

Hack Week 23

Activity

  • over 1 year ago: dfaggioli liked this project.
  • over 1 year ago: mgrossu joined this project.
  • over 1 year ago: jgoldschmidt liked this project.
  • over 1 year ago: STorresi added keyword "ai" to this project.
  • over 1 year ago: STorresi added keyword "stable-diffusion" to this project.
  • over 1 year ago: STorresi added keyword "imaging" to this project.
  • over 1 year ago: STorresi started this project.
  • over 1 year ago: STorresi originated this project.

  • Comments

    • STorresi
      over 1 year ago by STorresi | Reply

      ...and here are the results!

      In paris

      In New York City

    • STorresi
      over 1 year ago by STorresi | Reply

      These are generated after a bespoke LoRA training using DreamBooth over the JuggernautXL model, which in turn is based on SDXL 1.0.

      As you can see, hands are still tricky (a known issue of diffusion models, apparently), but I didn't try inpainting and img2img fine-tuning, which are supposed to be the go-to way to solve small issues like that. I must say the overall experience was quite painful due to the hardware requirements of SDXL and the amount of memory leaks in pytorch. A high-end consumer grade GPU like an NVIDIA 4080 with 16GB of VRAM often wasn't enough and ran OOM.

    Similar Projects

    This project is one of its kind!