youngmki / musinsaigo Goto Github PK

Hey,

Big thanks for posting this code. There aren't many examples of getting SDXL to run in SageMaker online. I've been making my way through the repo code, but it's tough to figure out the config.yaml file because I don't know what all the variables mean, so I'm making my way through aws docs at the same time...

1. Can you send me an example of how your original dataset was formatted before it gets captioned via CLIP?

EDIT
I think I see. sdxl_lora/trian.py is just expecting the traditional DataLoader format
dataset_dir/
train/
image1.jpg
image2.jpg
...
val/
image1.jpg
image2.jpg
...

I'm trying to run lora finetuning. I have a data set that has no text captions and I would like to add them. Is it formatted in the way a traditional DataLoader expects the data, like "data/image-1.png", ..., "data/image-n.png"? It seems like the model takes images with no captions, spins up an instance based on an image that uses CLIP to create captions for the images, then spins up an instance based on another image that does the training. It looks like the DataLoader is in sd_lora/train.py

2. Also, what would be the best way to load the baseline sdxl v1 from HF so I can finetune using lora using those pretrained weights?
   https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0

I'm really experienced with SM notebooks, but I haven't containerized anything and leverage training jobs yet via sagemaker sdk code yet. So I'm learning.

Best regards,
JB

Hi, I read your post from 'TensorFlow Korea' (or perhaps from 'PyTorch Korea,' not entirely sure), and tried to test the musinsago-1.0 model. By looking at the size of musinsago-1.0, which is 3.44 GB, I presume it is a fine-tuned DreamBooth model, not the LoRA model. Of course, when comparing the size to musinsago-2.0, which is 29.9MB, not in GB, I doubt the ver1.0 is surely not LoRA.

Are you sure you used 'train_dreambooth_lora.py' or 'train_text_to_image_lora.py' for training musinsago-1.0, instead of 'train_dreambooth.py'?

I would appreciate your reply soon :)"