confusion about dalle-pytorch HOT 3 OPEN

It seems @deepglugs has got somewhere with his own training
https://github.com/deepglugs/dalle / there's specific helpful training instructions (requires tags / image foo.txt foo.png ) tags file format. (should be a pull request) maybe @lucidrains this is helpful?

python3 dalle.py --source path/to/images/and/tags/ \
				 --vocab curated_512.vocab \
				 --vae vae.pt \
				 --train_dalle \
				 --dalle dalle.pt \
				 --batch_size=16 \
				 --samples_out samples/dalle/ \
				 --epochs=2

There's a ticket related to training sets -
#7

Looking at @josephcappadona sample 50mb file - there's the following format.
1166-004-E9B61A49.jpg

1166-004-E9B61A49.txt
The basic design of Hawaii's state seal has been in use since 1894, but the legend now reads “State of Hawaii” rather than “Republic of Hawaii.” The Hawaiian coat of arms is supported by Kamehameha I and the goddess of liberty, with a rising sun behind. The motto “Ua Mau ke Ea o ka Aina i ka Pono” (The Life of the Land Is Perpetuated in Righteousness) is along the bottom edge. Below the shield are various symbols: a phoenix rising from flames, taro leaves, banana foliage, and maidenhair ferns.

I'm thinking the tag format by @deepglugs may yield better results initially on smaller datasets??
1girl, white_swimsuit, red_hair
There seems to be some guidance around vocab tokens (.vocab file) ~ 512 tags which would help things converge, right?

I'd also consider we could use an google /aws image recognition software to find tags on a large image dataset.

from dalle-pytorch.

+1

Am also confused on how to use this. Even with existing data... How do I enter input and get a visual output?

from dalle-pytorch.

@lucidrains any help on this one? Would love to get started but confused just like this other guy :)

from dalle-pytorch.