nmoehrle / uavmvs Goto Github PK

UAV capture planning for MVS reconstructions

License: Other

C++ 74.25% Cuda 21.02% C 1.46% Lua 2.99% GLSL 0.28%

uavmvs's Introduction

UAVMVS - UAV capture planning for MVS reconstructions

The algorithm was published in Dec. 2018 at SIGGRAPH Asia. Please refer to our project website (https://vccimaging.org/Publications/Smith2018UAVPathPlanning/) for the paper and further information.

Dependencies

The code and the build system have the following prerequisites:

git
make
gcc (>= 5.4.0)
cuda (>= 8.0.0)
libglfw, libGL, libgomp, libpng, libjpg, libtiff

Furthermore the code depends on the following projects, some with adaptions so please clone and build the respective branches:

Premake https://premake.github.io/
- https://github.com/nmoehrle/premake-core/tree/toolset-nvcc
Multi-View Environment https://www.gcc.tu-darmstadt.de/home/proj/mve
- https://github.com/nmoehrle/mve/tree/master
MVS-Texturing https://www.gcc.tu-darmstadt.de/home/proj/texrecon
- https://github.com/nmoehrle/mvs-texturing/tree/develop
Eigen http://eigen.tuxfamily.org

Execution

Starting any application without parameters will print a description and an explanation of the parameters.

Compilation

git clone https://github.com/nmoehrle/uavmvs.git
cd uavmvs
premake5 gmake
cd build
make (or make -j for parallel compilation)

License and Citing

Our software is licensed under the BSD 3-Clause license, for more details see the LICENSE.txt file.

If you use our capture planning code for research purposes, please cite our paper:

@inproceedings{Smith2018Aerial,
  title={Aerial Path Planning for Urban Scene Reconstruction:
    a Continuous Optimization Method and Benchmark},
  author={Smith, Neil and Moehrle, Nils and Goesele, Michael and Heidrich, Wolfgang},
  year={2018},
  publisher={ACM}
}

Contact

If you have trouble compiling or using this software, if you found a bug or if you have an important feature request, please use the issue tracker of github: https://github.com/nmoehrle/uavmvs

uavmvs's People

Contributors

Stargazers

Watchers

Forkers

yilinliu77 peterzhousz cxfchxy peterzs dongyanchaotj yuancaimaiyi bluestupidyu zhanshenman xiaojiean ogsdave laurelkeys yangzhengshi gcx158632979 xiazhenyu555 cuijianzhu gukazma zebiao1998

uavmvs's Issues

optimize_trajectory not update camera views correctly. Any suggestion?

Hi, I am able to generate the nadir_trajectory through 'generate-trajectory', then I use the nadir trajectory to compute the optimal traj using 'optimize_trajectory'. Figure below shows nadir and opt traj of NY-1 case

Nadir Traj (151 cameras)

Opt Traj

We see that the updated cameras are not update correctly. And the optimization step in around 2 seconds with return:
0 3 0 2.22444
1 2 0 2.22444
What is the meaning of the return?

I also tried the function plan_trajectory, but fails to get any camera view. The parameters used here are same as in the makeTraj.sh.

Any comment is appreciated.

About the GPS information in reconstruct pipeline

Hi,in the paper you say that you use the GPS information in reconstruct pipeline,how to use it?

Proxy meshes of the synthetic scenes?

Hi,

Where can we find the proxy meshes of the synthetic scenes (e.g. NY-1, CA-1) discussed in the article? I could not find them either here or through the project page.

Any comment is really appreciated.

I can't find toolset-nvcc in premake-core , how can i get toolset-nvcc?

How to match the point in ue4?

Sorry,I'm not proficient.May I know haw to match the point position in ue4 of the NY-1 dateset?
Just like how can I get the exact coordinates of the four corners in ue4?
The corner like picture show.
And may I know all the coordinate of dataset like GOTH-1,CA-1,UK-1?

Cannot set the completeness range

I want to get a larger range of calculations with completeness. However, in the evaluation tool provided in Benchmark, evaluate reconstruction cannot pass in an appropriate range. After investigation, it was found that the option was set to compareness-range, but the i->opt->lopt was judged to be compareness-threshold, resulting in the inability to pass in
I wonder if it is possible to update a new executable.

The evaluation tool doesn't work

I found the Evaluation Tools in the website. After reading the README, I use the trajectory provided in NY-1 and generate the image successfully. Then I import the images and reconstruct a model using Reality Capture .

After that, I want to use the evaluation tools to evaluate the model. So I can evaluate my own trajectory. Here is my workflow of the reconstruct model:

    mesh=NY-1.ply
    alignMesh=alignMesh.ply
    gtMesh=NY-1.bin

    ./estimate_transform-mesh $mesh $gtMesh --transform=Talign.m
    ./convert-mesh $mesh $alignMesh --transform=Talign.m

    ./estimate_transform-mesh $alignMesh $gtMesh --transform=Talign.m
    ./convert-mesh $alignMesh $alignMesh --transform=Talign.m

    ./estimate_transform-mesh $alignMesh $gtMesh --transform=Talign.m
    ./convert-mesh $alignMesh $alignMesh --transform=Talign.m

    ./evaluate_reconstruction $alignMesh $gtMesh

After that I found the evaluation scores are very low, only 0.00267581 completeness when 0.0075. So i'm wondering if the convert is not work. Because no matter how many times I use the convert tools, it's average distance is the same.(0.09 for the gesox-mesh_orig.ply, and 0.12 for the model reconstructed by the images provided in the dataset).

Did I misuse the tools?

Limiting the Camera Orientation

Would it be possible to add an extra constraint for a mini-UAV that has 2 fixed cameras (front facing and down facing) instead of a gimbals camera that can look into all directions?

Any points what/where changes would be necessary?

About the trajectory about NY-1

I took the trajectory following makeTraj.sh successfully and get a trajectory file ny1_opti.traj. However, I was confused about the content.

In makeTraj.sh, the overlap is set to 80/80. Which generate 140+ cameras in nadir. And optimize_trajectory will optimize the 140+ cameras. But in the SyntheticProxyAndTrajectory.zip, the trajectory file for NY-1 contains 400+ cameras. Does the spline calculation or tsp calculation generate new cameras? But after I run interpolate-trajectory, I get 2000+ cameras which have keys 0 or 1. I was quiet confused about it.

Could you please provide the lines.* file in folder res/shaders/?

I would like to be able to use the visualizer to see the result of the path planning, but this cannot be done due to missing lines.* files. Could you please provide them?

Inquiry about Discrepancies in Image Results from Aerial Path Planning for Urban Scene Reconstruction

We have identified some disparities between the images I captured using the Unreal Engine 4 tool provided by author and the benchmark results their supplied. After a thorough analysis, we believe that these differences may be attributed to variations in the camera focal length. However, despite our diligent review of the documentation and settings provided with the tool, we encountered challenges in finding a solution to adjust the focal length.

something miss

Excuse me！There are some surface was disappear in the model ‘CA-1’ of mvsSim, like this:
image

I want to know if it'is the model or if I opened it the wrong way, may I get the full model in mvsSim?

How can I use this software?

I have some images taken from UAV . And can you tell me the wary to use uavmvs to generate trajectory?

segmentation fault

when I ran the generate_proxy_mesh to generate a mesh from a point cloud，I got this error

Received signal SIGSEGV (segmentation fault)
Obtained 5 stack frames: 0x5589f7dcfcb8 0x7f5f1fa360c0 0x5589f7d7942a 0x7f5f1fa170b3 0x5589f7d7a15e
generate_proxy_mesh(+0x6ccb8)[0x5589f7dcfcb8]
/lib/x86_64-linux-gnu/libc.so.6(+0x430c0)[0x7f5f1fa360c0]
generate_proxy_mesh(+0x1642a)[0x5589f7d7942a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f5f1fa170b3]
generate_proxy_mesh(+0x1715e)[0x5589f7d7a15e]
Segmentation fault

how can I solve this problem, thx

Showing "invalid value 'nvcc' for toolset" when run premake5 gmake

Test both in win10 and ubuntu18.04, with cuda 8.0

../premake-core/bin/release/premake5 gmake

➜  uavmvs git:(master) ../premake-core/bin/release/premake5 gmake
** Warning: the flags value C++11 has been deprecated and will be removed.
   Use `cppdialect "C++11"` instead
   @/home/whatseven/Desktop/uavmvs/premake5.lua(3)

Error: invalid value 'nvcc' for toolset

make error: util/endian.h: No such file or directory

All dependencies are installed, and I use ubuntu 16.04 and cuda-10.0. Any comment is appreciate.

Error message shows below:

==== Building fmt (debug) ====
==== Building sim (debug) ====
==== Building utp (debug) ====
==== Building eval (debug) ====
==== Building template (debug) ====
==== Building interop (debug) ====
==== Building generate_sphere_mesh (debug) ====
==== Building generate_aabb_mesh (debug) ====
==== Building convert-mesh (debug) ====
==== Building normalize-mesh (debug) ====
==== Building colorize-mesh (debug) ====
==== Building estimate_transform-mesh (debug) ====
==== Building convert-image (debug) ====
==== Building normalize-image (debug) ====
==== Building colorize-image (debug) ====
==== Building convert-scene (debug) ====
==== Building evaluate-scene (debug) ====
==== Building annotate-scene (debug) ====
==== Building estimate_transform-scene (debug) ====
==== Building reduce_randomly (debug) ====
==== Building deduce_bundle (debug) ====
==== Building match-scene (debug) ====
==== Building convert-cloud (debug) ====
==== Building convert-volume (debug) ====
==== Building normalize-volume (debug) ====
==== Building generate-trajectory (debug) ====
==== Building shorten-trajectory (debug) ====
==== Building interpolate-trajectory (debug) ====
==== Building evaluate-trajectory (debug) ====
==== Building determine_matching_mask (debug) ====
==== Building estimate_ground_plane (debug) ====
==== Building generate_texture (debug) ====
==== Building generate_gcp_bundle (debug) ====
==== Building generate_proxy_mesh (debug) ====
==== Building generate_proxy_cloud (debug) ====
==== Building generate_guidance_volume (debug) ====
==== Building generate_initial_trajectory (debug) ====
==== Building selector (debug) ====
==== Building simulator (debug) ====
==== Building visualizer (debug) ====
==== Building plan_trajectory (debug) ====
==== Building capture_trajectory (debug) ====
==== Building optimize_trajectory (debug) ====
==== Building evaluate_trajectory (debug) ====
==== Building evaluate_heuristic (debug) ====
evaluate_heuristic.cu
nvcc warning : Resource usage is not shown as the final resource allocation is not done.
In file included from ../apps/evaluate_heuristic/evaluate_heuristic.cu:16:0:
../libs/util/numpy_io.h:12:25: fatal error: util/endian.h: No such file or directory
compilation terminated.
evaluate_heuristic.make:158: recipe for target 'obj/debug/evaluate_heuristic/evaluate_heuristic.o' failed
make[1]: *** [obj/debug/evaluate_heuristic/evaluate_heuristic.o] Error 1
Makefile:436: recipe for target 'evaluate_heuristic' failed
make: *** [evaluate_heuristic] Error 2

Some documents?

Although I think I'm missing some required procedure to step forward, I tried to use this script ( https://vccimaging.org/Publications/Smith2018UAVPathPlanning/makeTraj.sh ) after building the program is succeeded.

Then I got this error:

generate_proxy_mesh (built on Feb  9 2019, 07:02:25)
	Could not load cloud: No such file or directory

Does this case happen becuase I didn't prepare any ply files before use this script?
When is the ply file exported then?

Could you please tell me more detail of using the program?
Thank you.

Can we get the ground truth of NY-1 and UK-1 for further experiements?

We would like to use the ground truth of the model in the benchmark for further experiments. Such as visualizing the error map of the model or find the place with the lowest reconstrucbility. We will be very appreciated if you could share the ground truth of the benchmark to us.

Build error inside "eval.make"

My test environment is Ubuntu 18.04 with cuda 9.1.85.

I've got this error at uavmvs make.

Error limit reached.
100 errors detected in the compilation of "/tmp/tmpxft_00005a74_00000000-6_bvh_tree.cpp1.ii".
Compilation terminated.
eval.make:160: recipe for target 'obj/debug/eval/bvh_tree.o' failed
make[1]: *** [obj/debug/eval/bvh_tree.o] Error 1
Makefile:190: recipe for target 'eval' failed
make: *** [eval] Error 2

Is there any suggestion to solve this?

build problems

i tried to build this project but this error keeps coming

`==== Building fmt (debug) ====
Creating debug
Creating obj/debug/fmt
format.cc
ostream.cc
posix.cc
Linking fmt
==== Building sim (debug) ====
Creating obj/debug/sim
model.cpp
window.cpp
In file included from ../libs/sim/model.cpp:18:0:
../libs/sim/shader_type.h:19:21: warning: ‘shader_names’ defined but not used [-Wunused-variable]
static const char * shader_names[] = {"vcolor", "surface", "texture", "lines"};
^~~~~~~~~~~~
Linking sim
==== Building utp (debug) ====
Creating obj/debug/utp
trajectory.cpp
trajectory_io.cpp
Linking utp
==== Building eval (debug) ====
Creating obj/debug/eval
bvh_tree.cu
kd_tree.cu
kernels.cu
nvcc warning : Resource usage is not shown as the final resource allocation is not done.
nvcc warning : Resource usage is not shown as the final resource allocation is not done.
nvcc warning : Resource usage is not shown as the final resource allocation is not done.
../libs/acc/primitives.h(60): error: namespace "std" has no member "vector"

../libs/acc/primitives.h(60): error: expected a ")"

../libs/acc/bvh_tree.h(66): error: qualified name is not allowed

../libs/acc/bvh_tree.h(66): error: this declaration has no storage class or type specifier

../libs/acc/bvh_tree.h(66): error: expected a ";"

../libs/acc/bvh_tree.h(67): error: qualified name is not allowed

../libs/acc/bvh_tree.h(67): error: this declaration has no storage class or type specifier

../libs/acc/bvh_tree.h(67): error: expected a ";"

../libs/acc/bvh_tree.h(70): error: qualified name is not allowed

../libs/acc/bvh_tree.h(70): error: this declaration has no storage class or type specifier

../libs/acc/bvh_tree.h(70): error: expected a ";"

../libs/acc/bvh_tree.h(84): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(84): error: expected a ")"

../libs/acc/bvh_tree.h(86): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(86): error: expected a ")"

../libs/acc/bvh_tree.h(88): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(88): error: expected a ")"

../libs/acc/bvh_tree.h(89): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(89): error: expected a ")"

../libs/acc/bvh_tree.h(98): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(98): error: expected a ")"

../libs/acc/bvh_tree.h(114): error: qualified name is not allowed

../libs/acc/bvh_tree.h(114): error: expected a ")"

../libs/acc/bvh_tree.h(115): error: qualified name is not allowed

../libs/acc/bvh_tree.h(115): error: expected a ";"

../libs/acc/bvh_tree.h(132): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(132): error: expected a ")"

../libs/acc/bvh_tree.h(161): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(161): error: expected a ")"

../libs/acc/bvh_tree.h(179): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(179): error: expected a ")"

../libs/acc/bvh_tree.h(277): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(277): error: expected a ")"

../libs/acc/bvh_tree.h(334): error: class template "acc::BVHTree<IdxType, Vec3fType>" has no member "BVHTree"

../libs/acc/bvh_tree.h(356): warning: parsing restarts here after previous syntax error

../libs/acc/bvh_tree.h(358): error: identifier "max_threads" is undefined

../libs/acc/bvh_tree.h(359): error: this declaration has no storage class or type specifier

../libs/acc/bvh_tree.h(359): error: expected a ")"

../libs/acc/bvh_tree.h(361): error: this declaration has no storage class or type specifier

../libs/acc/bvh_tree.h(361): error: expected a ";"

../libs/acc/bvh_tree.h(364): error: a class or namespace qualified name is required

../libs/acc/bvh_tree.h(364): error: the global scope has no "ID"

../libs/acc/bvh_tree.h(364): error: expected a ";"

../libs/acc/bvh_tree.h(365): error: a class or namespace qualified name is required

../libs/acc/bvh_tree.h(365): error: the global scope has no "ID"

../libs/acc/bvh_tree.h(365): error: expected a ";"

../libs/acc/bvh_tree.h(367): error: identifier "IdxType" is undefined

../libs/acc/bvh_tree.h(369): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(369): error: expected a ";"

../libs/acc/bvh_tree.h(370): error: this declaration has no storage class or type specifier

../libs/acc/bvh_tree.h(370): error: expected a ";"

../libs/acc/bvh_tree.h(371): error: namespace "std" has no member "vector"

../libs/acc/bvh_tree.h(371): error: expected a ";"

../libs/acc/bvh_tree.h(373): error: identifier "idx" is undefined

../libs/acc/bvh_tree.h(373): error: identifier "node_id" is undefined

../libs/acc/bvh_tree.h(374): error: expected a declaration

../libs/acc/bvh_tree.h(390): warning: parsing restarts here after previous syntax error

../libs/acc/bvh_tree.h(391): error: this declaration has no storage class or type specifier

../libs/acc/bvh_tree.h(391): error: overloaded function "std::swap" cannot be defined in the current scope

../libs/acc/bvh_tree.h(391): error: expected a ")"

../libs/acc/bvh_tree.h(393): error: this declaration has no storage class or type specifier

../libs/acc/bvh_tree.h(393): error: expected a ";"

../libs/acc/bvh_tree.h(394): error: expected a declaration

../libs/acc/bvh_tree.h(397): warning: parsing restarts here after previous syntax error

../libs/acc/bvh_tree.h(400): error: BVHTree is not a template

../libs/acc/bvh_tree.h(400): error: identifier "Ray" is undefined

../libs/acc/bvh_tree.h(400): error: a class or namespace qualified name is required

../libs/acc/bvh_tree.h(400): error: global-scope qualifier (leading "::") is not allowed

../libs/acc/bvh_tree.h(400): error: expected a ")"

../libs/acc/bvh_tree.h(400): error: a type qualifier is not allowed on a nonmember function

../libs/acc/bvh_tree.h(418): error: BVHTree is not a template

../libs/acc/bvh_tree.h(418): error: identifier "Ray" is undefined

../libs/acc/bvh_tree.h(418): error: identifier "Hit" is undefined

../libs/acc/bvh_tree.h(418): error: a type qualifier is not allowed on a nonmember function

../libs/acc/bvh_tree.h(469): error: BVHTree is not a template

../libs/acc/bvh_tree.h(470): error: a class or namespace qualified name is required

../libs/acc/bvh_tree.h(470): error: global-scope qualifier (leading "::") is not allowed

../libs/acc/bvh_tree.h(470): error: expected a ")"

../libs/acc/bvh_tree.h(470): error: a type qualifier is not allowed on a nonmember function

../libs/acc/bvh_tree.h(492): error: BVHTree is not a template

../libs/acc/bvh_tree.h(493): error: a type qualifier is not allowed on a nonmember function

../libs/acc/bvh_tree.h(554): error: BVHTree is not a template

../libs/acc/bvh_tree.h(560): error: expected a declaration

/usr/include/c++/6/bits/locale_classes.h(86): error: invalid specifier outside a class declaration

/usr/include/c++/6/bits/locale_classes.h(98): error: identifier "category" is undefined

/usr/include/c++/6/bits/locale_classes.h(99): error: identifier "category" is undefined

/usr/include/c++/6/bits/locale_classes.h(100): error: identifier "category" is undefined

/usr/include/c++/6/bits/locale_classes.h(101): error: identifier "category" is undefined

/usr/include/c++/6/bits/locale_classes.h(102): error: identifier "category" is undefined

/usr/include/c++/6/bits/locale_classes.h(102): error: declaration is incompatible with "time_t time(time_t *)"
/usr/include/time.h(75): here

/usr/include/c++/6/bits/locale_classes.h(103): error: identifier "category" is undefined

/usr/include/c++/6/bits/locale_classes.h(104): error: identifier "category" is undefined

/usr/include/c++/6/bits/locale_classes.h(105): error: identifier "category" is undefined

/usr/include/c++/6/bits/locale_classes.h(105): error: declaration is incompatible with "__nv_bool all(__nv_bool)"
/usr/include/device_atomic_functions.hpp(216): here

/usr/include/c++/6/bits/locale_classes.h(126): error: function "locale" is not a type name

/usr/include/c++/6/bits/locale_classes.h(136): error: "explicit" is not allowed

/usr/include/c++/6/bits/locale_classes.h(151): error: overloaded function "locale" is not a type name

/usr/include/c++/6/bits/locale_classes.h(151): error: identifier "category" is undefined

/usr/include/c++/6/bits/locale_classes.h(162): error: "explicit" is not allowed

/usr/include/c++/6/bits/locale_classes.h(163): error: expected a "{"

/usr/include/c++/6/bits/locale_classes.h(177): error: overloaded function "locale" is not a type name

/usr/include/c++/6/bits/locale_classes.h(177): error: identifier "category" is undefined

/usr/include/c++/6/bits/locale_classes.h(178): error: expected a "{"

Error limit reached.
100 errors detected in the compilation of "/tmp/tmpxft_000038af_00000000-6_bvh_tree.cpp1.ii".
Compilation terminated.
eval.make:160: recipe for target 'obj/debug/eval/bvh_tree.o' failed
make[1]: *** [obj/debug/eval/bvh_tree.o] Error 1
make[1]: *** Waiting for unfinished jobs....
../libs/acc/primitives.h(60): error: namespace "std" has no member "vector"