guanyingc / ps-fcn Goto Github PK

Sorry to trouble you.
Thank you for you code!
Have you processed the feature before visualization, like normalization?
'cause i visualize the feature map after fusion and get 128 disorderly and unsystematic feature maps.

While running the command python eval/run_model.py --retrain data/models/PS-FCN_B_S_32.pth.tar --in_img_num 96 I get the below error:

Please help me how to remove this error here.

TEST Iter [1/10] Epoch [1/30]
[Time 64.936s] ToCPU: 57.957s| ToGPU: 0.105s| Forward: 4.040s|
Traceback (most recent call last):
File "eval/run_model.py", line 23, in
main(args)
File "eval/run_model.py", line 19, in main
test_utils.test(args, 'test', test_loader, model, log, 1, recorder)
File "D:\git_hub\PS-FCN\test_utils.py", line 33, in test
log.printItersSummary(opt)
File "D:\git_hub\PS-FCN\utils\logger.py", line 54, in printItersSummary
print(opt['recorder'].iterRecToString(opt['split'], epoch))
File "D:\git_hub\PS-FCN\utils\recorders.py", line 56, in iterRecToString
strs += '{}: {:.3f}| '.format(k, np.mean(self.iter_rec[split][k]))
File "C:\Users\Garima\Anaconda3\envs\PYTORCH\lib\site-packages\numpy\core\fromnumeric.py", line 3118, in mean
out=out, **kwargs)
File "C:\Users\Garima\Anaconda3\envs\PYTORCH\lib\site-packages\numpy\core_methods.py", line 85, in _mean
ret = ret.dtype.type(ret / rcount)
AttributeError: 'torch.dtype' object has no attribute 'type'

Hi Guanying, thank you for sharing the synthetic data. I wonder the light directions represented by a vector are in which coordinate sysytem for both the synthetic data and DiLiGenT. More specifically, what are the axes directions? Are they x pointing to right, y pointing to up and z pointing towards the camera?

Hey Guanying,

Thank you for sharing your wonderful code with us.
In your paper, I noticed that you provided the results when the input number is 16 in the testing phase. For each sample in DiliGenT, you firstly chose 16 random images from 96. Then, repeated 100 times. Finally, you calculated the average values. (Is this right?).

I guess you wrote a script to repeat 100 times and changed the "--seed" value in "base_opts.py" (or maybe the np.random.seed). Could you please share the script with me, or give me some suggestions?

Best,
Zewei

I found that you published a paper named :
Deep Photometric Stereo for Non-Lambertian Surfaces

and I found that you normalized the images before putting images into network and obtained better result.

Did you normalize all image which are participated in training step ? Or you just normalized test datasets?
I tried to normalize the images in dataset(blobby and sculpture) and found that normalization process cannnot output not that great result( like the image below )

Hello, thank you for sharing.
I have a few questions about the synthetic datasets.
How do you get the groundtruth of the training set Blobby and Sculpture? Thank you again！

Hello, I want to ask, can I reproduce the rendering scene described in the paper exactly the same?

Because I noticed that for the objects in the synthetic data set in the paper, the BRDFS in each view randomly selects 2 of the 100 BRDFs.And the finally obtained image is under 64 illumination directions, which are also randomly sampled.

So does this mean that the original rendering scenes of these two synthetic datasets are no longer reproducible?
Thank you for your reply!

I have some problems during the use, download with the pre-trained model you provided, showing a "corrupted". As shown in the figure, can you please tell me the solution? Thank you.

Hi,
Can you tell me how to let normal map become to depth map?
I tried to use the DGP method but it will let the depth map loss a lots detail features.
i really want to know,please help me.

It seems the link in scripts/download_synthetic_datasets http://www.visionlab.cs.hku.hk/data/PS-FCN/datasets points to a corrupted .tgz file. And these two dataset files are very small (196M and 49M). It shows that the files are updataed recently, so maybe something go wrong. Please check and fix it.
Thanks.

Hello, thanks a lot for your code!
When I am testing your pre-trained model on dataset DiLiGenT, about 14G RAM is taken up, which makes my machine crash down . I guess it is because that too many pictures are loaded at the same time. So I'm here to ask how can I modify something in the code such as batch size to decrease the number of pictures loaded at the same time to lower the RAM occupation and is it possible to lower the memory cost to about 10G while testing your pre-trained model?
Looking forward to your reply

Hi. Now I almost completed implementing your networks to the keras environment.

Now I want to know whether I made the network right, since my network outputs loss value larger than 0.18, when adjusting no scaling/random crop/noise aug. with sculpture dataset.

Can you tell me how much the loss of training should be ? for example, 0.0001 theoretically.

Hi Chen, thanks for your code.
I have a question about the acquirement of the light intensity in your dataset.
Is it just a scale or physical quantity.
If it just a scale. And how do you get the physical lgiht intensity and convert it to scale.
I appreciate for lots of answers that you gave me previously.

I'm interested in your research and tried to run the code, but encountered the following error, could you help me to fix it? Thanks!

How to reproduce

I wanted to test PS-FCN on DiLiGenT main dataset using all of the 96 image-light pairs by running
CUDA_VISIBLE_DEVICES=0 python eval/run_model.py --retrain data/models/PS-FCN_B_S_32.pth.tar --in_img_num 96 --cuda
with arg --cuda added since my computer has no GPU

Current Behavior

------------ Options -------------
        benchmark: DiLiGenT_main
        bm_dir: data/datasets/DiLiGenT/pmsData
        cuda: False
        epochs: 30
        fuse_type: max
        in_img_num: 96
        in_light: True
        item: calib
        log_dir: data/models\run_model
        model: PS_FCN_run
        normalize: False
        resume: None
        retrain: data/models/PS-FCN_B_S_32.pth.tar
        run_model: True
        save_root: data/Training/
        seed: 0
        start_epoch: 1
        test_batch: 1
        test_disp: 1
        test_intv: 1
        test_save: 1
        time_sync: False
        train_img_num: 32
        use_BN: False
        workers: 8
-------------- End ----------------

=> fetching img pairs in data/DiLiGenT_main
[test Data]     10 objs 96 lights. Root: data/datasets/DiLiGenT/pmsData
Files for intensity: light_intensities.txt
         Found Benchmark Data: 10 samples
         Test Batch 1
Creating Model PS_FCN_run
[Network Input] Color image as input
[Network Input] Adding Light direction as input
[Network Input] Input channel: 6
Conv pad = 1
Conv pad = 1
Conv pad = 1
Conv pad = 1
Conv pad = 1
Conv pad = 1
Conv pad = 1
Conv pad = 1
=> using pre-trained model data/models/PS-FCN_B_S_32.pth.tar
PS_FCN(
  (extractor): FeatExtractor(
    (conv1): Sequential(
      (0): Conv2d(6, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): LeakyReLU(negative_slope=0.1, inplace=True)
    )
    (conv2): Sequential(
      (0): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (1): LeakyReLU(negative_slope=0.1, inplace=True)
    )
    (conv3): Sequential(
      (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): LeakyReLU(negative_slope=0.1, inplace=True)
    )
    (conv4): Sequential(
      (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
      (1): LeakyReLU(negative_slope=0.1, inplace=True)
    )
    (conv5): Sequential(
      (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): LeakyReLU(negative_slope=0.1, inplace=True)
    )
    (conv6): Sequential(
      (0): ConvTranspose2d(256, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
      (1): LeakyReLU(negative_slope=0.1, inplace=True)
    )
    (conv7): Sequential(
      (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): LeakyReLU(negative_slope=0.1, inplace=True)
    )
  )
  (regressor): Regressor(
    (deconv1): Sequential(
      (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): LeakyReLU(negative_slope=0.1, inplace=True)
    )
    (deconv2): Sequential(
      (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): LeakyReLU(negative_slope=0.1, inplace=True)
    )
    (deconv3): Sequential(
      (0): ConvTranspose2d(128, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)
      (1): LeakyReLU(negative_slope=0.1, inplace=True)
    )
    (est_normal): Sequential(
      (0): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    )
  )
)
=> Model Parameters: 2210048
---- Start test Epoch 1: 10 batches ----
Traceback (most recent call last):
  File "eval/run_model.py", line 23, in <module>
    main(args)
  File "eval/run_model.py", line 19, in main
    test_utils.test(args, 'test', test_loader, model, log, 1, recorder)
  File ".\test_utils.py", line 21, in test
    for i, sample in enumerate(loader):
  File "C:\Users\Fisher\miniconda3\envs\cv\lib\site-packages\torch\utils\data\dataloader.py", line 435, in __next__
    data = self._next_data()
  File "C:\Users\Fisher\miniconda3\envs\cv\lib\site-packages\torch\utils\data\dataloader.py", line 1085, in _next_data
    return self._process_data(data)
  File "C:\Users\Fisher\miniconda3\envs\cv\lib\site-packages\torch\utils\data\dataloader.py", line 1111, in _process_data
    data.reraise()
  File "C:\Users\Fisher\miniconda3\envs\cv\lib\site-packages\torch\_utils.py", line 428, in reraise
    raise self.exc_type(msg)
TypeError: __init__() missing 1 required positional argument: 'dtype'

My working environment

• Win10
• no GPU or CUDA
• torch 1.7.1+cpu
• torchvision 0.8.2+cpu

BTW, I don't understand why the required PyTorch version is 1.10, since the latest version on pytorch.org is 1.8. Do you mean 0.1.10?

Hi. I want to make a dataset with my own 3d models, just following your methods.
Can I know the program/ method you used?
I appreciate for lots of answers that you gave me previously.

Hi, I kinda have problem understanding the whole parts of your codes so would you please tell me where can I find the max pooling layer in it? By the way, thanks for your research :)

Can you please provide us with the ready sphere and bunny dataset?

hello，
I create the new dataset with blender and I used the dateset to test,but got a poor result. I would like to ask if the light direction refers to the coordinates of the light in the coordinate system (after normalization).
Looking forward to your reply。

@guanyingc Hello, thanks you for sharing the synthetic data.
I have some questions about the dataset:

1. what type emitter did you use to generate the data? Is the point light far from the object to mimic the effect of parallel light?
2. what sensor do you used to mimic the orthogonal camera? In the document of Mitsuba2, I don't find the orthographic type sensor.
3. Mitsuba2 use xml file to create the scene. How do you use python to dynamically edit scene and render all result?
   Thanks again!

Hi there, thanks for your codes
I was trying to train a new PS-FCN model with the training datasets following the steps just like you said but since l use google colab l have disk limitation on GPU(not premium) mode.
I want to ask if is possible to use only one of the two datasets for training and if it yes how far it would change the final results?
Actually I suppose this may decrease the accuracy of the model so what if I just use both datasets but cut off some of the views of each object? Do you think this might help?

Thank you for your code.
I train the model with the code and the test it in DiLiGent dataset according to the device . The mean error range is from 8.6° to 8.7°.
Then I use the model you provided the result，and the result 8.39°。
It is same with the result in paper. I have no idea about this difference.
My GPU is 1080Ti.