samsung / kv2streamer Goto Github PK

Hi ,
I am trying to run the kv2streamer in x64 architecture in Visual studio 2012 of windows platform .I have modified all supporting dlls in to x64 .

I am getting following Linker error .

error LNK2019: unresolved external symbol gst_app_src_set_callbacks referenced in function "public: void __cdecl kv2s::GstAppSrcPipeline::Initialize(class std::basic_string<char,struct std::char_traits,class std::allocator >)" (?Initialize@GstAppSrcPipeline@kv2s@@QEAAXV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@@z)

error LNK2019: unresolved external symbol gst_app_sink_set_drop referenced in function "public: void __cdecl kv2s::GstAppSinkPipeline::Initialize(class std::basic_string<char,struct std::char_traits,class std::allocator >)"

error LNK2019: unresolved external symbol gst_app_sink_pull_preroll referenced in function "private: static enum GstFlowReturn __cdecl

Please suggest me which gstreamer library i have to modify in order to resolve the above error

Thanks

Hi,
In my scenario i connected two kinectv2sensors.one sensor at same PC called primary. another sensor from network PC called secondary.using the Kv2streamer application we are getting streams from network PC.Using those data we mapping (color and depth)pointcloud based on coordinatemapper from primary.when i access the primary sensor color buffer data it gives (1920 * 1080 * 4)=8294400 bytes .but the secondary color buffer throws Access violation error when i multiply (1920 * 1080 * 4) bytes per pixels.so i multiply (1920 * 1080 * 3)= 6220800 it gives the pointcloud output.Actually the primary sensor gives exact isolated human but the secondary sensor depth and color not mapped correctly it show shadow around the human cloud.please suggest me to get full resolution color and depth data from network PC for exactly mapping human depth and color.Here i have attached what the result i getting from primary and secondary sensor.

Thanks,
Kirubha

Hi Kevin.
This is kirubha again, i am facing an issue during display of point cloud data from secondary sensor.Here i attached two cloud images. First one is primary sensor that is connected in the same machine. Second image is secondary sensor data which is from server machine. When i project the point cloud data in the my client machine, the Human shows like double human. I dont know where is an issue. To understand clearly i have also added my logic below.

Code logic:
//Getting ColoredBuffer data from your code KV2ClientExample.cpp(secondarysensor)
void KV2ClientExample::AcquireAndProcessColorFrame()
{
IColorFramePtr colorFrame;
if (colorStreamer->AcquireLatestFrame(&colorFrame))
{
UINT bufferSize;
unsigned char* buffer;
colorFrame->AccessRawUnderlyingBuffer(&bufferSize, &buffer);
}
}
In our logic, we did like this
BYTE* bufferbyte = nullptr;
void AcquireAndProcessColorFrame() //In this function we are receiving the buffer data from secondary sensor
{
kv2s::IColorFramePtr colorFrame;
if (colorStreamer->AcquireLatestFrame(&colorFrame))
{
UINT bufferSize;
unsigned char* buffer;
colorFrame->AccessRawUnderlyingBuffer(&bufferSize, &buffer);
// converting unsigned char* to byte buffer
bufferbyte=reinterpret_cast<BYTE*>(buffer);
}
}

//This function gets bodyindex buffer
void AcquireAndProcessBodyIndexFrame()
{
IBodyIndexFramePtr bodyIndexFrame;
signed char* s2buffer;
if (bodyIndexStreamer->AcquireLatestFrame(&bodyIndexFrame))
{
UINT bufferSize;
bodyIndexFrame->AccessRawUnderlyingBuffer(&bufferSize, &s2buffer);
unsigned char* output = bodyIndexFrameRenderBuffer;
const signed char* bufferEnd = s2buffer + DEPTH_MULTICAST_WIDTH_DEPTH_MULTICAST_HEIGHT;
while(s2buffer < bufferEnd)
{
signed char index = *s2buffer;
output = color_mapping[3_(index+1)+0]; ++output;
output = color_mapping[3(index+1)+1]; ++output;
output = color_mapping[3(index+1)+2]; ++output;
++s2buffer;
}
indexSensor2buffer=reinterpret_cast<BYTE>(s2buffer);
bodyIndexFrame.reset();
}
}
//In this function, we are displaying the rgb point cloud data in our viewport
void PointCloudGL::PointCloudDisplay()
{
float fl_x = 1063.118f;
float fl_y = 1065.233f;
float pp_x = 962.473f;
float pp_y = 526.789f;

// Unit16* depthSensor2buffer - data comes from secondary sensor 
//Cameraspacepoint* pCSS2Points - data from depthsensor2buffer is copied to cameraspacepoint
pCoordinateMapper->MapColorFrameToCameraSpace(512*424, depthSensor2buffer,1920 * 1080, pCSS2Points);

//pDepthS2SpaceBuffer - depthspacepoint - here secondary sensor buffer is copied to depthspacepoint
pCoordinateMapper->MapColorFrameToDepthSpace(512 * 424, depthSensor2buffer, 1920 * 1080, pDepthS2SpaceBuffer);

glBegin(GL_POINTS); 
float xValue=0;
float yValue=0;
//Looping through rgb resolution 1920 * 1080
for (int i = 0; i < 1920; i++)
{   
    for (int j = 0; j < 1080; j++)
    {
        int colorIndex = i + (j * 1920);
        DepthSpacePoint p ;
        DepthSpacePoint p2 ;
        if(pDepthS2SpaceBuffer!= NULL)
        {
            p2=pDepthS2SpaceBuffer[colorIndex]; //we are retrieving the depth space buffer values as array and store it in depthspacepoint
            const CameraSpacePoint& rPt1 = pCSS2Points[colorIndex];
            if (p2.X != -std::numeric_limits<float>::infinity() && p2.Y != -std::numeric_limits<float>::infinity())
            {
                int depthX = static_cast<int>(p2.X+0.5);
                int depthY = static_cast<int>(p2.Y+0.5);
                if ((depthX >= 0 && depthX < 512) && (depthY >= 0 && depthY < 424))
                {
                    BYTE player = indexSensor2buffer[depthX + (depthY * 512)];
                        if (player != 0xff)
                    {
                        int Z = rPt1.Z;
                                float xx=0;
                                float yy=0;
                                float zz=0;
                                if (rPt1.Z > 0)
                                {
                                    xx =(i-pp_x)*rPt1.Z/fl_x;
                                    yy = (j-pp_y)*rPt1.Z/fl_y;
                                    zz=rPt1.Z;
                                    if(Sensor2Selection==0) //sensor 1 selected
                                        {
                                            if (bufferbyte !=nullptr)
                                            {

glColor4ub(bufferbyte[ 3 *colorIndex], bufferbyte[3 * colorIndex + 1], bufferbyte[3 * colorIndex + 2], bufferbyte[3 * colorIndex + 3]);
glVertex3f(xx/15, -yy/15, rPt1.Z/15);
}
}
}
}
}
}
}
// In the above logic the following code used to draw 3d point cloud
float fl_x = 1063.118f;
float fl_y = 1065.233f;
float pp_x = 962.473f;
float pp_y = 526.789f;

xx =(i-pp_x)*rPt1.Z/fl_x; // 3dpoint X
yy = (j-pp_y)*rPt1.Z/fl_y; // 3dpoint Y
zz=rPt1.Z; // 3dpoint Z

glColor4ub(bufferbyte[ 3 *colorIndex], bufferbyte[3 * colorIndex + 1], bufferbyte[3 * colorIndex + 2], bufferbyte[3 * colorIndex + 3]);
glVertex3f(xx, -yy, rPt1.Z);
if we used the above logic , we are getting following screenshot named Convertedtoours.jpg and if you see primarysensor.jpg there is no overlapping of human.

Actually in our existing logic we put the following code to display point cloud from primary sensor,

glColor4ub(pColorBuffer[4 * colorIndex], pColorBuffer[4 * colorIndex + 1], pColorBuffer[4 * colorIndex + 2], pColorBuffer[4 * colorIndex + 3]);
glVertex3f(xx, -yy, rPt.Z);

Note:-But if we put 4*colorIndex in secondary sensor, we are getting the following error
Unhandled exception: Access violation reading location

Please guide me how to proceed with our existing logic.

Thanks
Kiruba

Hi,

After a long time i got a doubt in kv2streamer application, multiple clients can connect to single server by using multicast ip.But is it possible to connect multiple servers to single client.(i.e To show multiple server kinect data output to single client machine.)

Please suggest me with some ideas.

Thanks in Advance
Kiruba

Hi Seo-Young and Anette,
Is the Server Side portion of this missing or did I overlook something. Thanks! I look forward to implementing this.

Hi,

what's the approximate network load when streaming the complete data? Or, put differently: would this also work for streaming data from multiple Kinect v2 (each with it's own PC running the server) to the same client?

Also, does it add a noticeable lag?

Thanks.

Hi,
Now i am trying to execute Kv2Streamer source in Windows platform. Now i compiled Kinectv2server
suceesfully.but when i configuring the KinectClient side API.I am facing some issues.Also i am beginner in C++ . the following errors are i facing now:

Error 30 error LNK2019: unresolved external symbol "public: __thiscall kv2s::KV2Client::~KV2Client(void)" (??1KV2Client@kv2s@@QAE@XZ) referenced in function "public: void * __thiscall kv2s::KV2Client::`scalar deleting destructor'(unsigned int)" (??_GKV2Client@kv2s@@QAEPAXI@Z) KinectClient
Error 59 error LNK2019: unresolved external symbol "public: __thiscall kv2s::GstAppSrcPipeline::GstAppSrcPipeline(class std::basic_string<char,struct std::char_traits,class std::allocator >,int)" (??0GstAppSrcPipeline@kv2s@@QAE@V?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@h@Z) referenced in function "public: __thiscall KV2ClientExample::KV2ClientExample(int,int,char const *)" (??0KV2ClientExample@@QAE@HHPBD@Z) KinectClient

Error 67 error LNK2019: unresolved external symbol __imp__wglShareLists@8 referenced in function __glfwCreateContext KinectClient
Error 66 error LNK2019: unresolved external symbol __imp__wglMakeCurrent@8 referenced in function __glfwPlatformMakeContextCurrent
..\KinectClient\KinectClient\glfw3.lib(wgl_context.obj) KinectClient

Hi,
In Kv2streamer,the Gstreamer pipeline (GstreamerPipelines.h) for encoding/decoding bytes over the network you are using following function.

// RGB Multicasting Pipeline
static std::string CreateAppSrc_RGB8_EncodedMulticastingPipeline(VideoMulticastingPipelineCreationParameters input)
{
std::stringstream pipelineString;

pipelineString

<< "rtpbin name=" << RTPBIN_NAME

<< NEW_LINKAGE

<< "appsrc name=" << APPSRC_NAME << " is-live=true block=true stream-type=0 format=3 do-timestamp=true min-latency=0" 
<< LINK << "video/x-raw, format=" << input.pixelFormat << ", width=" << input.inputWidth << ", height=" << input.inputHeight << ", framerate=30/1" << LINK
<< "videoconvert"
<< LINK
<< "videoscale"
<< LINK << "video/x-raw, width=" << input.multicastWidth << ", height=" << input.multicastHeight << LINK 
<< "queue"
<< LINK
<< "x264enc tune=zerolatency speed-preset=ultrafast"; if (input.useHighestQuality) pipelineString << " qp-min=0 qp-max=0 qp-step=0"; pipelineString
<< LINK << "video/x-h264, profile=baseline" <<  LINK
<< "rtph264pay"
<< LINK
<< RTPBIN_NAME << PAD << "send_rtp_sink_0"

<< NEW_LINKAGE

<< RTPBIN_NAME << PAD << "send_rtp_src_0"
<< LINK
<< "udpsink port=" << input.port << " host=" << input.multicastIP << " auto-multicast=true" 
;
return pipelineString.str();

}

Basically in my application both Kv2streamer server and client application executed in windows platform.I think this pipeline string unable to send full color bytes through network that is the reason in my output the depth and color is not mapped correctly .Can u help me to modify this pipeline string that work for windows environment to receive loss less data over the network? And also in which PipelineString is suitable for receiving full length(1920*1080) color bytes.

I have referred some gstreamer pipeline links from internet but not able to get which piplelinestring is suitable to receive (1920*1080) color bytes.
http://processors.wiki.ti.com/index.php/Example_GStreamer_Pipelines#DM355
https://gstreamer.ti.com/gf/project/gstreamer_ti/wiki/?pagename=NotesOnDM365Performance
http://lists.freedesktop.org/archives/gstreamer-devel/2013-June/041229.html
https://developer.ridgerun.com/wiki/index.php?title=Introduction_to_network_streaming_using_GStreamer

Please help me to solve this issue

Hi,
Now i am try to execute Kv2Streamer source in Windows platform. Now i compiled Kinectv2server suceesfully.but when i configuring the KinectClient side API.I am facing some issues.Also i'm a beginner in C++ . the following errors are i facing now:

Error 30 error LNK2019: unresolved external symbol "public: __thiscall kv2s::KV2Client::~KV2Client(void)" (??1KV2Client@kv2s@@QAE@XZ) referenced in function "public: void * __thiscall kv2s::KV2Client::`scalar deleting destructor'(unsigned int)" (??_GKV2Client@kv2s@@QAEPAXI@Z) KinectClient
Error 59 error LNK2019: unresolved external symbol "public: __thiscall kv2s::GstAppSrcPipeline::GstAppSrcPipeline(class std::basic_string<char,struct std::char_traits,class std::allocator >,int)" (??0GstAppSrcPipeline@kv2s@@QAE@V?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@h@Z) referenced in function "public: __thiscall KV2ClientExample::KV2ClientExample(int,int,char const *)" (??0KV2ClientExample@@QAE@HHPBD@Z) KinectClient

Error 67 error LNK2019: unresolved external symbol __imp__wglShareLists@8 referenced in function __glfwCreateContext KinectClient
Error 66 error LNK2019: unresolved external symbol __imp__wglMakeCurrent@8 referenced in function __glfwPlatformMakeContextCurrent
..\KinectClient\KinectClient\glfw3.lib(wgl_context.obj) KinectClient