yihangchen-ee / hac Goto Github PK

Firstly, thanks for your efforts in releasing your code. However, I found something strange in train.py. In line 669, the script tried to recover the latest iteration of GaussianModel, and evaluated its performance. Are the evaluated results consistent with what you reported in your Arxiv paper? It seems these results are based on the parameters without encoding and decoding.

Furthermore, in function generate_neural_gaussians, if the GaussianModel is in a decoded version, the neural gaussians will be directly generated by pc._anchor_feat, which are decoded from a bitstream encoded from pc._anchor_feat. In other words, in the decoded version, it seems the used features are not interpolated by grid_encoder but the original pc._anchor_feat instead.

你好，Nice work！
问题一：请问这里anchor的三个属性都是高维向量，横坐标的value具体指代的是什么？是把高维向量的量化后的每一维元素都看做一个符号，value指的就是这些符号吗？

问题二：这个损失函数是不是也可以理解为最大似然？

Hello! This is a great job!
I found such a problem during the replication process. How can I solve it?

Building wheel for gridencoder (setup.py) ... error

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for gridencoder Running setup.py clean for gridencoder Failed to build gridencoder ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (gridencoder)

您好，感谢您出色的工作。我在A6000上初步运行了您的代码，发现truck bitstream=0.004 30000次迭代之后point_cloud.ply的大小是153.36MB，和论文Table D中的9.26MB相差较大，请问是我弄错什么了吗？希望您不吝赐教！

Thanks for your great work!

Did you meet the problem that the testing accuracy is much lower than the training one?
My reproduced PSNR is 21.35 rather than 25.98 in the paper. The log is attached.

Looking for your reply.

 Starting evaluation...
2024-04-10 14:27:39,298 - INFO: model_paths: �[1;35moutputs/bungeenerf/rome_HAC/0.004�[0m
2024-04-10 14:27:39,307 - INFO:   SSIM : �[1;35m   0.7054738�[0m
2024-04-10 14:27:39,307 - INFO:   PSNR : �[1;35m  21.3558159�[0m
2024-04-10 14:27:39,307 - INFO:   LPIPS: �[1;35m   0.2179699�[0m
2024-04-10 14:27:39,331 - INFO: 
Evaluating complete.
[
[outputs.log](https://github.com/YihangChen-ee/HAC/files/14932153/outputs.log)
](url)

最终产物不应该是一个.ply文件吗？

Hey do you know how to deal with CUDA out of memory during rendering process?

encoding_param_num=1766272, size=0.2105560302734375MB. [08/06 19:57:10] Reading camera 251/251 [08/06 19:57:11] start fetching data from ply file [08/06 19:57:11] Loading Training Cameras [08/06 19:57:11] Loading Test Cameras [08/06 19:57:15] Initial voxel_size: 0.01 [08/06 19:57:16] Number of points at initialisation : 114293 [08/06 19:57:16] anchor_bound_updated [08/06 19:57:16] Training progress: 100%|███████████████████████████████████████████| 1000/1000 [01:00<00:00, 16.49it/s, Loss=0.0915876] 2024-06-08 19:58:17,128 - INFO: [ITER 1000] Saving Gaussians 2024-06-08 19:58:18,246 - INFO: Total Training time: 60.38757371902466 2024-06-08 19:58:18,351 - INFO: Training complete. 2024-06-08 19:58:18,351 - INFO: Starting Rendering~ hash_params: True 4 13 (18, 24, 33, 44, 59, 80, 108, 148, 201, 275, 376, 514) 15 (130, 258, 514, 1026) True False False [08/06 19:58:18] encoding_param_num=1766272, size=0.2105560302734375MB. [08/06 19:58:18] Loading trained model at iteration 1000 [08/06 19:58:18] Reading camera 251/251 [08/06 19:58:19] start fetching data from ply file [08/06 19:58:19] Loading Training Cameras [08/06 19:58:19] Loading Test Cameras [08/06 19:58:22] Rendering progress: 0%| | 0/32 [00:01<?, ?it/s] Traceback (most recent call last): File "train.py", line 669, in <module> visible_count = render_sets(args, lp.extract(args), -1, pp.extract(args), wandb=wandb, logger=logger, x_bound_min=x_bound_min, x_bound_max=x_bound_max) File "train.py", line 472, in render_sets t_test_list, visible_count = render_set(dataset.model_path, "test", scene.loaded_iter, scene.getTestCameras(), gaussians, pipeline, background) File "train.py", line 403, in render_set render_pkg = render(view, gaussians, pipeline, background, visible_mask=voxel_visible_mask) File "C:\Users\jay\Desktop\HAC\gaussian_renderer\__init__.py", line 225, in render cov3D_precomp = None) File "C:\Users\jay\AppData\Local\anaconda3\envs\HAC_env\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\jay\AppData\Local\anaconda3\envs\HAC_env\lib\site-packages\diff_gaussian_rasterization\__init__.py", line 222, in forward raster_settings, File "C:\Users\jay\AppData\Local\anaconda3\envs\HAC_env\lib\site-packages\diff_gaussian_rasterization\__init__.py", line 41, in rasterize_gaussians raster_settings, File "C:\Users\jay\AppData\Local\anaconda3\envs\HAC_env\lib\site-packages\diff_gaussian_rasterization\__init__.py", line 92, in forward num_rendered, color, radii, geomBuffer, binningBuffer, imgBuffer = _C.rasterize_gaussians(*args) RuntimeError: CUDA out of memory. Tried to allocate 8.54 GiB (GPU 0; 8.00 GiB total capacity; 147.41 MiB already allocated; 5.60 GiB free; 264.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Hi, I tried to train using the kitti360 dataset (>3000 images) and the results don't seem to be very good as shown below. I tried manually adjusting the voxel size and iteration, but not many changes. I feel the problem is that there are not enough Gaussians in the scene to fit the image. Is there any other parameter I can try to adjust? Thanks for your help!

How to view the trained model? Thanks!

I found the Scaffold-GS-viewer can't be used, the origin GS-viewer view like full black scene.

log.txt

AssertionError: While the loader support other types, the rest of the code assumes PINHOLE

import gridencoder as _backend
ModuleNotFoundError: No module named '_gridencoder'
您好，很出色的工作，请问这个问题如何解决呢？