Comments (1)
Thank you team for the sharing the project resources. I am trying to process the CelebA-HQ 256x256 dataset for the DDGAN model. The DDGAN repository recommends going over the dataset preparation methods in the NVAE repository (this repository).
The following commands will download tfrecord files from GLOW and convert them to store them in an LMDB dataset. Use the link by openai/glow for downloading the CelebA-HQ 256x256 dataset (4 Gb). To convert/store the CelebA-HQ 256x256 dataset to/as the lmdb dataset one needs to install module called "tfrecord". The missing module error can be rectified by simply executing the command
pip install tfrecord
.!mkdir -p $DATA_DIR/celeba
%cd $DATA_DIR/celeba
!wget https://openaipublic.azureedge.net/glow-demo/data/celeba-tfr.tar
!tar -xvf celeba-tfr.tar
%cd $CODE_DIR/scripts
!pip install tfrecord
!python convert_tfrecord_to_lmdb.py --dataset=celeba --tfr_path=$DATA_DIR/celeba/celeba-tfr --lmdb_path=$DATA_DIR/celeba/celeba-lmdb --split=train
The final command
!python convert_tfrecord_to_lmdb.py --dataset=celeba --tfr_path=$DATA_DIR/celeba/celeba-tfr --lmdb_path=$DATA_DIR/celeba/celeba-lmdb --split=train
gives the following output:. . . 26300 26400 26500 26600 26700 26800 26900 27000 added 27000 items to the LMDB dataset. Traceback (most recent call last): File "convert_tfrecord_to_lmdb.py", line 73, in <module> main(args.dataset, args.split, args.tfr_path, args.lmdb_path) File "convert_tfrecord_to_lmdb.py", line 58, in main print('added %d items to the LMDB dataset.' % count) lmdb.Error: mdb_txn_commit: Disk quota exceeded
I am not sure I have made the LMDB dataset properly, I request you to guide me.
@KomputerMaster64 hello, I also want to convert the CelebA-HQ 256x256 dataset to the lmdb dataset, however, there is an error message:"AttributeError: 'bytes' object has no attribute 'cpu'" when runs to code:"im = data['data'][0].cpu().numpy()", and if I remove 'cpu' to let the code be "im = data['data'][0].numpy()", it also throws an error:"AttributeError: 'bytes' object has no attribute 'numpy'", so I want to ask you how you resolved this problem? Thanks a lot !
from nvae.
Related Issues (20)
- FID score of CelebA-HQ 256x256 HOT 1
- NomalDecoder & num_bits
- TypeError: batch_norm_backward_elemt() missing 1 required positional arguments: "count" HOT 1
- How to run without using parallelization? HOT 1
- Can you provide pretrained models? HOT 1
- why is there self.prior_ftr0 in the decoder model?
- Why some of the generate images by the official checkpoint of CelebA64 are NaN-value? HOT 2
- Query: CelebA HQ 256
- Query: Dataset CelebA-HQ 256x256 issue
- Query: FFHQ Pre-Processing HOT 3
- FFHQ Training
- Possible typo in the log_p() function
- ImageNet Checkpoint
- Question regarding traversing the latent space
- Why output for 3-rd channel is unused in Logistic mixture? HOT 1
- how can i use the code on my own dataset. if it's necessary to modify the code carefully myself? HOT 1
- "arch_instance" argument
- Problem while converting tfrecord to lmdb data AttributeError: 'bytes' object has no attribute 'cpu' HOT 4
- Question about KL computation HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nvae.