Color issue with pcl cloud about libfreenect2pclgrabber HOT 30 CLOSED

Hi,
you are calling getFullCloud() right? could you try also getCloud()? try also to run getFullCloud without openmp support

from libfreenect2pclgrabber.

I'm just running the example you included with the code, not using the library yet in my own code.

from libfreenect2pclgrabber.

So if the Kinect2Grabber executable calls getFullCloud(), then I am :-)

I also have openmp disabled, since it isn't easy to set up on OS X at the moment.

from libfreenect2pclgrabber.

Try to change it to getCloud()
Is protonect working correctly?

from libfreenect2pclgrabber.

Yeah protonect works correctly, I'm able to use their test app and get full data out. However there is a small caveat that I believe isn't relevant to this issue because it works with my integrated graphics and the "regular" version and with the opencl version, but not with the "regular" version and my discrete graphics on OS X.

So I should look for a call to either getCloud() or getFullCloud() and swap it to the other one? Can't try that till tonight but I will!

from libfreenect2pclgrabber.

I tried switching it to:

cloud = k2g.getCloud();

and it crashes immediately, and I've been able to confirm again that with getFullCloud() it has the color problem.

from libfreenect2pclgrabber.

I fixed the crash, that was my fault. Now it also looks the same with getCloud()

from libfreenect2pclgrabber.

I also created a pull request with my current code:
#4

from libfreenect2pclgrabber.

I modified getCloud as follows:

    typename pcl::PointCloud<PointT>::Ptr
    getCloud(const int size_x = 512, const int size_y = 424){

        typename pcl::PointCloud<PointT>::Ptr cloud_(new pcl::PointCloud<PointT>());
        cloud_->is_dense = false;

        frames_ =  *getRawFrames();
        rgb_ = frames_[libfreenect2::Frame::Color];
        depth_ = frames_[libfreenect2::Frame::Depth];
        tmp_depth_ = cv::Mat(depth_->height, depth_->width, CV_32FC1, depth_->data);
        remapDepth(tmp_depth_,depth_final_,cv::Size(size_x,size_y),CV_32FC1);

        tmp_rgb_ = cv::Mat(rgb_->height, rgb_->width, CV_8UC3, rgb_->data);
        //cv::remap(tmp_rgb_, rgb_final_, map_x_rgb_, map_y_rgb_, cv::INTER_LINEAR);
        rgb_final_ = tmp_rgb_;
        static int i = 0;
        cv::imwrite("test"+boost::lexical_cast<std::string>(i++)+".jpg",rgb_final_);
        // ...snip...
}

And got the following image:

Clearly there is a problem with the rgb image, rather than the mapping of the rgb image onto the point cloud. I'll see if I can check if the image before it is rgb is ok.

from libfreenect2pclgrabber.

Figured out one major issue, the image definition given to opencv should have be CV_8UC4 instead of CV_8UC3. Now the image looks normal when I display it or write it to a file. However, the striping of the point cloud is even more visible so there is still another bug to address.

Here is the updated cloud image, the colors seem to show up with a higher frequency:

from libfreenect2pclgrabber.

Hi!
I am on vacation until the 31August but then I will have a look. I worked a bit on the code lately so there might be some bugs. As soon as I get back I will check and fix everything.

from libfreenect2pclgrabber.

Awesome, thanks! I'd also appreciate if you could take my pull request 4 into consideration when you do that.

Also, here is the commit that fixes the color issue

from libfreenect2pclgrabber.

I change the code:

                //const cv::Vec3b tmp = color.at<cv::Vec3b>(rgb_img.y(), rgb_img.x());
                const cv::Vec4b tmp = color.at<cv::Vec4b>(rgb_img.y(), rgb_img.x());

And then, work great;

from libfreenect2pclgrabber.

nice, thanks; I will add it tomorrow if you don't want to make a pull request

from libfreenect2pclgrabber.

cool! I will have to try that out

from libfreenect2pclgrabber.

where was that line, or perhaps you could submit a pull request?

from libfreenect2pclgrabber.

I am sorry to late, i will make a pull request later.

from libfreenect2pclgrabber.

could you maybe just tell me the file/line? that'd help!

from libfreenect2pclgrabber.

I tried it and it doesn't work for me. I start to think that you have a different kinect v2 version
This is also true for you ahundt. Probably they changed the data which you get from the kinect.
I had also to undo the last commit since it was not building anymore on linux. If you want you could create a branch with your version for osx.

from libfreenect2pclgrabber.

@ahundt I'm facing the same problem display that you've shown above. And change CV_8UC3 to CV_8UC4 does not change the result. Any cue?

from libfreenect2pclgrabber.

Get the new version
Il 24/set/2015 15:25, "filipetrocadoferreira" [email protected] ha
scritto:

@ahundt https://github.com/ahundt I'm facing the same problem display
that you've shown above. And change CV_8UC3 to CV_8UC4 does not change the
result. Any cue?

—
Reply to this email directly or view it on GitHub
#5 (comment)
.

from libfreenect2pclgrabber.

Hi, thanks!

I'm trying to compile the new version but i get these errors:

:~/Documents/tests/libfreenect2pclgrabber/build$ make
[100%] Building CXX object CMakeFiles/Kinect2Grabber.dir/test.cpp.o
In file included from /home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h:1:0,
from /home/followinspiration/Documents/tests/libfreenect2pclgrabber/test.cpp:15:
/usr/local/include/libfreenect2/libfreenect2.hpp:86:11: error: ‘string’ in namespace ‘std’ does not name a type
virtual std::string getSerialNumber() = 0;
^
/usr/local/include/libfreenect2/libfreenect2.hpp:87:11: error: ‘string’ in namespace ‘std’ does not name a type
virtual std::string getFirmwareVersion() = 0;
^
/usr/local/include/libfreenect2/libfreenect2.hpp:119:3: error: ‘string’ in namespace ‘std’ does not name a type
std::string getDeviceSerialNumber(int idx);
^
/usr/local/include/libfreenect2/libfreenect2.hpp:120:3: error: ‘string’ in namespace ‘std’ does not name a type
std::string getDefaultDeviceSerialNumber();
^
/usr/local/include/libfreenect2/libfreenect2.hpp:124:37: error: ‘string’ in namespace ‘std’ does not name a type
Freenect2Device openDevice(const std::string &serial);
^
/usr/local/include/libfreenect2/libfreenect2.hpp:125:37: error: ‘string’ in namespace ‘std’ does not name a type
Freenect2Device *openDevice(const std::string &serial, const PacketPipeline *factory);
^
In file included from /home/followinspiration/Documents/tests/libfreenect2pclgrabber/test.cpp:15:0:
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h: In constructor ‘K2G::K2G(processor)’:
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h:43:24: error: ‘class libfreenect2::Freenect2’ has no member named ‘getDefaultDeviceSerialNumber’
serial = freenect2_.getDefaultDeviceSerialNumber();
^
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h:63:50: error: no matching function for call to ‘libfreenect2::Freenect2::openDevice(std::string&, libfreenect2::PacketPipeline_&)’
dev_ = freenect2_.openDevice(serial_, pipeline_);
^
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h:63:50: note: candidates are:
In file included from /home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h:1:0,
from /home/followinspiration/Documents/tests/libfreenect2pclgrabber/test.cpp:15:
/usr/local/include/libfreenect2/libfreenect2.hpp:122:20: note: libfreenect2::Freenect2Device* libfreenect2::Freenect2::openDevice(int)
Freenect2Device openDevice(int idx);
^
/usr/local/include/libfreenect2/libfreenect2.hpp:122:20: note: candidate expects 1 argument, 2 provided
/usr/local/include/libfreenect2/libfreenect2.hpp:123:20: note: libfreenect2::Freenect2Device libfreenect2::Freenect2::openDevice(int, const libfreenect2::PacketPipeline_)
Freenect2Device *openDevice(int idx, const PacketPipeline *factory);
^
/usr/local/include/libfreenect2/libfreenect2.hpp:123:20: note: no known conversion for argument 1 from ‘std::string {aka std::basic_string}’ to ‘int’
/usr/local/include/libfreenect2/libfreenect2.hpp:124:20: note: libfreenect2::Freenect2Device_ libfreenect2::Freenect2::openDevice(const int&)
Freenect2Device openDevice(const std::string &serial);
^
/usr/local/include/libfreenect2/libfreenect2.hpp:124:20: note: candidate expects 1 argument, 2 provided
/usr/local/include/libfreenect2/libfreenect2.hpp:125:20: note: libfreenect2::Freenect2Device libfreenect2::Freenect2::openDevice(const int&, const libfreenect2::PacketPipeline_)
Freenect2Device *openDevice(const std::string &serial, const PacketPipeline *factory);
^
/usr/local/include/libfreenect2/libfreenect2.hpp:125:20: note: no known conversion for argument 1 from ‘std::string {aka std::basic_string}’ to ‘const int&’
/usr/local/include/libfreenect2/libfreenect2.hpp:130:20: note: libfreenect2::Freenect2Device_ libfreenect2::Freenect2::openDevice(int, const libfreenect2::PacketPipeline_, bool)
Freenect2Device *openDevice(int idx, const PacketPipeline *factory, bool attempting_reset);
^
/usr/local/include/libfreenect2/libfreenect2.hpp:130:20: note: candidate expects 3 arguments, 2 provided
make[2]: *_* [CMakeFiles/Kinect2Grabber.dir/test.cpp.o] Error 1
make[1]: *** [CMakeFiles/Kinect2Grabber.dir/all] Error 2
make: *** [all] Error 2

from libfreenect2pclgrabber.

Thats a libfreenect2 bug.
Add the include for string in
/usr/local/include/libfreenect2/libfreenect2.hpp
Il 24/set/2015 16:01, "filipetrocadoferreira" [email protected] ha
scritto:

Hi, thanks!

I'm trying to compile the new version but i get these errors:

:~/Documents/tests/libfreenect2pclgrabber/build$ make
[100%] Building CXX object CMakeFiles/Kinect2Grabber.dir/test.cpp.o
In file included from
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h:1:0,
from
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/test.cpp:15:
/usr/local/include/libfreenect2/libfreenect2.hpp:86:11: error: ‘string’ in
namespace ‘std’ does not name a type
virtual std::string getSerialNumber() = 0;
^
/usr/local/include/libfreenect2/libfreenect2.hpp:87:11: error: ‘string’ in
namespace ‘std’ does not name a type
virtual std::string getFirmwareVersion() = 0;
^
/usr/local/include/libfreenect2/libfreenect2.hpp:119:3: error: ‘string’ in
namespace ‘std’ does not name a type
std::string getDeviceSerialNumber(int idx);
^
/usr/local/include/libfreenect2/libfreenect2.hpp:120:3: error: ‘string’ in
namespace ‘std’ does not name a type
std::string getDefaultDeviceSerialNumber();
^
/usr/local/include/libfreenect2/libfreenect2.hpp:124:37: error: ‘string’
in namespace ‘std’ does not name a type
Freenect2Device

openDevice(const std::string &serial); ^
/usr/local/include/libfreenect2/libfreenect2.hpp:125:37: error: ‘string’ in
namespace ‘std’ does not name a type Freenect2Device *openDevice(const
std::string &serial, const PacketPipeline *factory); ^ In file included
from
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/test.cpp:15:0:
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h: In
constructor ‘K2G::K2G(processor)’:
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h:43:24:
error: ‘class libfreenect2::Freenect2’ has no member named
‘getDefaultDeviceSerialNumber’ serial =
freenect2_.getDefaultDeviceSerialNumber(); ^
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h:63:50:
error: no matching function for call to
‘libfreenect2::Freenect2::openDevice(std::string&,
libfreenect2::PacketPipeline_&)’
dev_ = freenect2_.openDevice(serial_, pipeline_);
^
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h:63:50:
note: candidates are:
In file included from
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/k2g.h:1:0,
from
/home/followinspiration/Documents/tests/libfreenect2pclgrabber/test.cpp:15:
/usr/local/include/libfreenect2/libfreenect2.hpp:122:20: note:
libfreenect2::Freenect2Device* libfreenect2::Freenect2::openDevice(int)
Freenect2Device

openDevice(int idx); ^
/usr/local/include/libfreenect2/libfreenect2.hpp:122:20: note: candidate
expects 1 argument, 2 provided
/usr/local/include/libfreenect2/libfreenect2.hpp:123:20: note:
libfreenect2::Freenect2Device libfreenect2::Freenect2::openDevice(int,
const libfreenect2::PacketPipeline

) Freenect2Device *openDevice(int idx, const PacketPipeline *factory); ^
/usr/local/include/libfreenect2/libfreenect2.hpp:123:20: note: no known
conversion for argument 1 from ‘std::string {aka std::basic_string}’ to
‘int’ /usr/local/include/libfreenect2/libfreenect2.hpp:124:20: note:
libfreenect2::Freenect2Device libfreenect2::Freenect2::openDevice(const
int&)
Freenect2Device

openDevice(const std::string &serial); ^
/usr/local/include/libfreenect2/libfreenect2.hpp:124:20: note: candidate
expects 1 argument, 2 provided
/usr/local/include/libfreenect2/libfreenect2.hpp:125:20: note:
libfreenect2::Freenect2Device libfreenect2::Freenect2::openDevice(const
int&, const libfreenect2::PacketPipeline

) Freenect2Device *openDevice(const std::string &serial, const
PacketPipeline *factory); ^
/usr/local/include/libfreenect2/libfreenect2.hpp:125:20: note: no known
conversion for argument 1 from ‘std::string {aka std::basic_string}’ to
‘const int&’ /usr/local/include/libfreenect2/libfreenect2.hpp:130:20: note:
libfreenect2::Freenect2Device libfreenect2::Freenect2::openDevice(int,
const libfreenect2::PacketPipeline

, bool) Freenect2Device *openDevice(int idx, const PacketPipeline
*factory, bool attempting_reset); ^
/usr/local/include/libfreenect2/libfreenect2.hpp:130:20: note: candidate
expects 3 arguments, 2 provided make[2]: **
[CMakeFiles/Kinect2Grabber.dir/test.cpp.o] Error 1
make[1]: *** [CMakeFiles/Kinect2Grabber.dir/all] Error 2
make: *** [all] Error 2

—
Reply to this email directly or view it on GitHub
#5 (comment)
.

from libfreenect2pclgrabber.

Thanks, it's solved! :D

from libfreenect2pclgrabber.

He posted in my issue, not a new one! Mine isn't solved yet unless there is a commit I missed fixing the coloring. :-)

from libfreenect2pclgrabber.

Sorry,
did you try the new grabber version?

from libfreenect2pclgrabber.

should be able to in the next week or two, sorry it has taken a while I've been very busy with classes

from libfreenect2pclgrabber.

Did you try the new version?

from libfreenect2pclgrabber.

Should be trying it out very soon, sorry it took so long to come up in the todo queue but it is at the top now for next week!

from libfreenect2pclgrabber.

Works beautifully!

from libfreenect2pclgrabber.