Comments (13)
Hello! First of all, nice work!
What do you mean by "rich sky texturing"? Can you explain a bit and give an example?
Thank you!
Thanks for your comments! I can give you an example. In the figure below, the left one is with "rich sky textures" but the right one is not.
from skyar.
Hi, thanks for your feedback. I watched your video and I believe the problem lies in the sky background. As mentioned in my preprint paper, when there are no textures in the sky, the motion of the sky background cannot be accurately modeled. This is one of the limitations of the method.
I have updated the skyboxengine.py, where this time I limited the motion to translation + rotation, and raise the threshold of the number of effective matching points (3 ->10). You can try again and tell me whether it works this time. With any luck, this update will work on your case. But as I mentioned above, the most reliable solution is to choose a video with a rich sky texture for testing. Good luck.
from skyar.
Hello! First of all, nice work!
What do you mean by "rich sky texturing"? Can you explain a bit and give an example?
Thank you!
from skyar.
Thank you, I understand now! This problem occurs because of the dataset? I mean, less images with clear skies day/night?
from skyar.
Thank you, I understand now! This problem occurs because of the dataset? I mean, less images with clear skies day/night?
I believe the problem is in the motion estimation rather than the dataset training. If you set "save_jpgs" in the config file to "true" and check those sky region detection results, you will see those images with clear sky regions are nicely detected. However, when there are not enough feature points in the sky, the tracker may fail to find any effective matching points between adjacent frames, and that why we need a rich textured sky to make sure the motion of the virtual background is accurately modeled.
from skyar.
I understand! Thank you very much for the info! Once again, nice work :)
from skyar.
I understand! Thank you very much for the info! Once again, nice work :)
Cheers!
from skyar.
Hi, thanks for your feedback. I watched your video and I believe the problem lies in the sky background. As mentioned in my preprint paper, when there are no textures in the sky, the motion of the sky background cannot be accurately modeled. This is one of the limitations of the method.
I have updated the skyboxengine.py, where this time I limited the motion to translation + rotation, and raise the threshold of the number of effective matching points (3 ->10). You can try again and tell me whether it works this time. With any luck, this update will work on your case. But as I mentioned above, the most reliable solution is to choose a video with a rich sky texture for testing. Good luck.
Will try the fixes, thx a lot.
from skyar.
@jiupinjia I made a new test video with the settings. I think it´s a little bit better but still have the jumping effects.
Here is the new video:
https://www.youtube.com/watch?v=rzJWS3jqiy8
Maybe we work with pre-marker in the sky that can help to match the points for tracking?
from skyar.
@jiupinjia I made a new test video with the settings. I think it´s a little bit better but still have the jumping effects.
Here is the new video:
https://www.youtube.com/watch?v=rzJWS3jqiy8Maybe we work with pre-marker in the sky that can help to match the points for tracking?
Seems the problem is still there. Can you show me the input video?
from skyar.
@jiupinjia Sure, you can download it here, I remove the zip in 48 hours.
http://rebotnix.com/tmp/sky_videos.zip
Thanks for your help. Will test any changes.
from skyar.
@jiupinjia Sure, you can download it here, I remove the zip in 48 hours.
http://rebotnix.com/tmp/sky_videos.zipThanks for your help. Will test any changes.
Hi, I have carefully checked your input videos, and I believe the problem is in the sky background. As I mentioned in my preprint paper Sec 4.4, one of the limitations of the method is that it only works with input videos with rich sky textures. When there are no textures, the motion of the virtual camera cannot be accurately modeled. To better understand this, I can give you an example. In the figure below, the left one shows a frame with rich sky textures but the right one does not.
I have put SkyAR version 2.0 on my agenda, where I will be working on a more robust motion estimation (motion propagation from foreground to background) and hopefully, the SkyAR 2.0 will have no more requirements on the sky textures. Thanks again for your valuable feedback.
from skyar.
I fully understand it and i thank you that you take your time to check the videos. It´s a great project and I will follow it for sure. Good luck.
from skyar.
Related Issues (20)
- 请问一下NVIDIA显卡是不是必要条件呢? HOT 1
- Great library HOT 1
- Input Error HOT 5
- I can only generate the 40frame GIF previews in Colab HOT 3
- Skybox follows the camera HOT 3
- how to save the videos HOT 3
- I make a simply implementation, and have a small question. HOT 10
- How to make the picture or video of sky match the aspect ratio of video which being processed
- Awesome
- Question regarding CoordConv HOT 4
- import error HOT 2
- ImportError: cannot import name 'compare_ssim' from 'skimage.measure'. I need help as I cannot start train.py HOT 3
- Need to do on Images only HOT 1
- Cloudy Footage
- Noncommercial
- Awesome work HOT 1
- Can't install Requirements.txt HOT 13
- 请问能帮忙把预训练模型放一放在百度网盘吗,国内上不了google网盘 HOT 2
- tile_skybox_img in skyboxengine.py HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from skyar.