6

I am trying to convert a depth image (RGBD) into a 3d point cloud. The solution I am currently using is taken from this post where:

  • cx = image center height
  • cy = image center width
  • fx and fy = 250, chosen by iterating through a few options

The depth measurements have been taken from a pin hole camera and the point cloud is projecting away from the centre (example images below). Can anyone help me understand why and how I can solve this?

enter image description here enter image description here enter image description here

martineau
  • 119,623
  • 25
  • 170
  • 301
Lloyd Rayner
  • 1,009
  • 2
  • 7
  • 10

1 Answers1

18

You may easily solve this using open3d package. Install it using sudo pip install -U open3d-python (not just open3d -- that's another package).

Once installed:

import open3d as o3d

rgbd = o3d.geometry.RGBDImage.create_from_color_and_depth(color, depth, convert_rgb_to_intensity = False)
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(rgbd, pinhole_camera_intrinsic)

# flip the orientation, so it looks upright, not upside-down
pcd.transform([[1,0,0,0],[0,-1,0,0],[0,0,-1,0],[0,0,0,1]])

draw_geometries([pcd])    # visualize the point cloud

The above code assumes you have your color image in color and depth image in depth, check the samples coming with open3d for more information.

If you have your own camera intrinsic, you may replace pinhole_camera_intrinsic with those, but for the test run, the pinhole camera works more or less fine.

Andrei Miculiță
  • 321
  • 1
  • 11
lenik
  • 23,228
  • 4
  • 34
  • 43
  • Thank you for the suggestion @Lenik. The output of this gives me a sliced point cloud (https://ibb.co/yy4F94b) do you know why? – Lloyd Rayner Jan 04 '20 at 20:41
  • @LloydRayner if you find my answer useful, you may upvote and/or accept it, why your point cloud look "sliced" -- I don't know, it's unlikely to be a problem of `open3d` package -- I use that quite a lot. Probably taking a look at your depth data might help, especially something like plotting number of points versus the distance -- it might show you some "missing" depth regions or other irregularities. – lenik Jan 05 '20 at 03:17
  • I have a question about how to load Image and Depth data. In function "create_rgbd_image_from_color_and_depth", input arguments are "color" and "depth". Could you explain how to load image? Do you use OpenCV or PIL Image for this purpose? I really appreciate it if you could share the whole code. – Mohammad Sep 28 '20 at 23:21
  • @Mohammad You can load the images from file with `depth = o3d.io.read_image(filename)`. You can also convert numpy images to o3d as mentioned [here](https://github.com/intel-isl/Open3D/issues/957#issuecomment-491571979), but for me that sometimes doesn't work (for whatever reason, I don't know). – ketza Jan 25 '21 at 15:20