Libfreenect wrong depth map

Question

I've been using OpenNI+PrimeSense+NiTE with OpenCV on my project to segment objects according to their distances. However I meant to deploy it in a NVIDIA Jetson TX1 board and it couldn't manage to compile OpenNI+PrimeSense+NiTE with OpenCV on it. I endded up with libfreenect. However the depth map provided by libfreenect is very, very wrong. I'll share some examples.

Here is the working depth map of OpenNI: OpenNI Depth Map

The libfreenect wrong depth map is here: Libfreenect Depth Map

I based my libfreenect code on the default C++ wrapper at OpenKinect website.

Can someone help me here? Thank you so much.

score 1 · Accepted Answer · answered Jun 13 '17 at 21:36

Well, for those who are working with libfreenect on ARM or AARCH64 architectures (mainly Jetson TX1) because OpenNI and SensorKinect are problematic to build, I made some adjustments on OpenNI and SensorKinect sources to run with Aarch64 and avoid having to use libfreenect.

Links: OpenNI for TX1 and SensorKinect for TX1

score 0 · Answer 2 · answered Jun 10 '17 at 09:21

It looks like different mappings of the depth data.

You can try to put the libfreenect data into a cv::Mat and scale that:

const float scaleFactor = 0.05f;
depth.convertTo(depthMat8UC1, CV_8UC1, scaleFactor);
imshow("depth gray",depthMat8UC1);

You can also checkout this article and on building OpenNI2 on a Jetson TK1. Once you have OpenNI setup and working, you should be able to compile OpenCV from source enabling WITH_OPENNI with cmake. After that you should be able to grab the depth data directly in OpenCV:

#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"

#include <iostream>

using namespace cv;
using namespace std;

const float scaleFactor = 0.05f;

int main(){
cout << "opening device(s)" << endl;

VideoCapture sensor;
sensor.open(CV_CAP_OPENNI);

if( !sensor.isOpened() ){
    cout << "Can not open capture object 1." << endl;
    return -1;
}

for(;;){
    Mat depth,depthScaled;

    if( !sensor.grab() ){
        cout << "Sensor1 can not grab images." << endl;
        return -1;
    }else if( sensor.retrieve( depth, CV_CAP_OPENNI_DEPTH_MAP ) ) {
        depth.convertTo(depthScaled, CV_8UC1, scaleFactor);
        imshow("depth",depth);
        imshow("depth scaled",depthScaled);
    }

    if( waitKey( 30 ) == 27 )   break;



   }
}

Libfreenect wrong depth map

2 Answers2