decayale / vlocnet Goto Github PK

This is a repository to implement the work of ``Deep Auxiliary Learning for Visual Localization and Odometry'' known as VLocNet.

Python 57.95% Jupyter Notebook 41.80% Shell 0.24%

vlocnet's Introduction

VLocNet

This repository is a PyTorch implementation of Deep Auxiliary Learning for Visual Localization and Odometry, known as VLocNet.

This is NOT an official version published by the authors, but a self-implemented project (with some codes referring to the project of MapNet). It should be mentioned that the current performance is still subpar compared with the reported by the paper, although our best have been tried, but close to the MapNet, on 7Scenes dataset.

We release this code mainly for two purpose:

Providing a bedrock for those who also endeavor to implement this work
Encouraging the guys interested in our work to find the bugs if existing and explore the tricks before the code released by the authors.

Feel free to create new issues about our work, and share your ideas and suggestions.

Setup

Install miniconda with Python 3.5+.
Create the mapnet Conda environment: conda env create -f environment.yml.
Activate the environment: conda activate mapnet_release.

Data

We only support the 7Scenes dataset for now. The datasets live in the datasets directory. We provide skeletons with symlinks to get you started. Let us call your 7Scenes download directory 7SCENES_DIR. You will need to make the following symlinks:

cd datasets && ln -s 7SCENES_DIR 7Scenes

Running the code

The python training script and test script are included in train.py and test.py separately. The configuration is straightforward and for simplicity some bool options are replaced by int ( 0 for False and 1 for True).
The training and testing can be carried out by the commands python train.py and python test.py. The specific configurations are at your discretion.

vlocnet's People

Contributors

Stargazers

Watchers

Forkers

stiphyjay hanahimi zlthinker cumtchenchang olibd buyiasd fddehnavi euwen zumbalamambo 123456789mojtaba muyuman flyinggh

vlocnet's Issues

A discussion on the performance and share your result

I opened this issue so that I hope people can share their best results and see if we can figure out how to reproduce VLocNet paper result together.

My trained heads scene result using the VLocNet-M4 setting in this repo:
train.sh.txt

Median error 0.40m, 36.018 degree
Mean error 0.469m, 39.855 degree

Paper reported result for heads:
Median error 0.046m, 6.64 degree
Mean error Unknown

However, if I test my model with feeding in the previous pose with the ground truth value (which is CHEATING), I got:
Median error 0.02m, 3.25 degree
Mean error 0.02m, 3.84 degree

I notice that the error of the predicted pose will accumulate and propagate at testing time if I use network predicted pose as previous pose. Does anyone know why this happened and how to solve it?

Cannot run train.py

Hello,

I try to run train.py and follow the setup, but I cannot create environment by the .yml of MapNet because packages are incompatible with each other. Therefore, I created an environment myself. My system is Win 10 and CUDA is 9.2. The below are the packages I installed (via pip):

python = 3.7
torch = 0.4.1
torchvision = 0.2.1
pillow = 6.1.0
tensorvoardX = 1.4
numpy-quaternion
transforms3d
numba
scipy

Then I run train.py and get the following error:

1 False
Epoch 0 checkpoint saved for VLocNet
Traceback (most recent call last):
  File "train.py", line 207, in <module>
	trainer.train_val()
  File "C:\Users\Tu\GitHub\VLocNet\model\trainer.py", line 396, in train_val
	for batch_idx, (data, target) in enumerate(self.train_loader):
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\site-packages\torch\utils\data\dataloader.py", line 501, in __iter__
	return _DataLoaderIter(self)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\site-packages\torch\utils\data\dataloader.py", line 289, in __init__
	w.start()
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\process.py", line 112, in start
	self._popen = self._Popen(self)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\context.py", line 223, in _Popen
	return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\context.py", line 322, in _Popen
	return Popen(process_obj)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
	reduction.dump(process_obj, to_child)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\reduction.py", line 60, in dump
	ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x0000024739E4E048>: attribute lookup <lambda> on __main__ failed

1 False
Epoch 0 checkpoint saved for VLocNet
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\spawn.py", line 105, in spawn_main
	exitcode = _main(fd)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\spawn.py", line 114, in _main
	prepare(preparation_data)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\spawn.py", line 225, in prepare
	_fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
	run_name="__mp_main__")
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\runpy.py", line 263, in run_path
	pkg_name=pkg_name, script_name=fname)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\runpy.py", line 96, in _run_module_code
	mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\runpy.py", line 85, in _run_code
	exec(code, run_globals)
  File "C:\Users\Tu\GitHub\VLocNet\train.py", line 207, in <module>
	trainer.train_val()
  File "C:\Users\Tu\GitHub\VLocNet\model\trainer.py", line 396, in train_val
	for batch_idx, (data, target) in enumerate(self.train_loader):
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\site-packages\torch\utils\data\dataloader.py", line 501, in __iter__
	return _DataLoaderIter(self)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\site-packages\torch\utils\data\dataloader.py", line 289, in __init__
	w.start()
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\process.py", line 112, in start
	self._popen = self._Popen(self)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\context.py", line 223, in _Popen
	return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\context.py", line 322, in _Popen
	return Popen(process_obj)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
	prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
	_check_not_importing_main()
  File "C:\Users\Tu\Miniconda3\envs\VLocNet\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
	is not going to be frozen to produce an executable.''')
RuntimeError:
		An attempt has been made to start a new process before the
		current process has finished its bootstrapping phase.

		This probably means that you are not using fork to start your
		child processes and you have forgotten to use the proper idiom
		in the main module:

			if __name__ == '__main__':
				freeze_support()
				...

		The "freeze_support()" line can be omitted if the program
		is not going to be frozen to produce an executable.

It looks like multiprocessing problem, but I can't solve it. And I think the problem maybe is version of the packages. Could you give me your packages' version?

Thanks.

train problem

it seems that the 'trainsplit.txt' was forgotten to push

mapnet checkpoint files can not be loaded by torch

Could not find the correct prefixes between odom_en1_head.conv1.weight and mapnet.feature_extractor.conv1.weight
Traceback (most recent call last):
File "test.py", line 156, in
test_dataset=test_set, config=args, resume_optim=False)
File "/home/adastec/vlocnet/model/evaluator.py", line 96, in init
load_state_dict(self.model, checkpoint['model_state_dict'])
File "/home/adastec/vlocnet/model/evaluator.py", line 47, in load_state_dict
raise KeyError
KeyError
is there any checkpoint file that trained on vlocnet model?

No opensource code?

Hello. I check the paper but no sharing the code anymore?

Accuracy about your code with original paper

Thanks for your opensource for VlocNet. Recently I'm also doing similar work. So there are some questions for the result:

I wonder what accuracy (median translation and rotation error) do you obtain? Can you update ReadMe.md for that part?
you seems only trained on 7scenes, how about cambridge dataset?
as training needs groundtruth previous pose as input and testing needs predicted counterpart, I found it extremely easy to be overfit between gt and predicted, have you noticed this phenomenon?

training problem

Hello, I trained this network using scene fire in 7Scenes dataset. I firstly trained for 10 epochs, but the result is not very good. Then I trained another 40 epochs(50 in total) and tested it using the test images. Results show that the network outputs same pose for every picture.I watched the log in tensorboard and found all parameter sx and sq kept dropping. I wonder if this is a normal state or there is someting wrong. I noticed in paper posenet2, initial sx and sq are 0.0 and -3.0. Howerver in your code they are both -3.0.
I don't know what error I made during training. I would be grateful if you have any suggestion on this problem, or you can provide more training detail.Thank you very much