1

I'm trying to replicate a real-world camera within Three.js, where I have the camera's distortion specified as parameters for a "plumb bob" model. In particular I have P, K, R and D as specified here:

enter image description here

If I understand everything correctly, I want to set up Three to do the translation from "X" on the right, to the "input image" in the top left. One approach I'm considering is making a Three.js Camera of my own, setting the projectionMatrix to ... something involving K and D. Does this make sense? What exactly would I set it to, considering K and D are specified as 1 dimensional arrays, of length 9 and 5 respectively. I'm a bit lost how to combine all the numbers :(

I notice in this answer that there are complicated things necessary to render straight lines as curved, they way they would be with certain camera distortions (like a fish eye lens). I do not need that for my purposes if that is more complicated. Simply rendering each vertex is the correct spot is sufficient.

Eric Simonton
  • 5,702
  • 2
  • 37
  • 54
  • What's a plumb bob? – M - Aug 20 '20 at 22:44
  • In this case, it's the name of a "distortion model". There is some explanation of it here: https://calib.io/blogs/knowledge-base/camera-models – Eric Simonton Aug 21 '20 at 12:01
  • 1
    Yes, it makes a lot of sense to create your own Camera class, and it's not even a lot of work. One other thing you could consider is to do have the distortion you're talking about using a post-processing effect that performs only the radial distortion part, since it seems to only need image coordinates (as opposed to needing xyz coordinates) – Michele Lambertucci Aug 26 '20 at 16:19
  • @MicheleLambertucci Thank you. I am not familiar with post-processing effects, but I did notice them when looking for options. I was starting to think I may need to write some webgl code to achieve the radial distortion, which I think means it would be in such a post-processing effect? Do you know if that's true, or can that be represented in a projection matrix? Or is there some other option? – Eric Simonton Aug 26 '20 at 17:38
  • A post-processing effect is an additional shader pass applied over the rendered image. This is especially useful in this case, since the distortion you want to achieve works in "rendered space" (it doesn't need any information about the depth). Yes, I think you can use a simple GLSL shader to apply the radial distortion over the rendered image – Michele Lambertucci Aug 27 '20 at 13:05

1 Answers1

0

This document shows the step by step derivation of the camera matrix (Matlab reference).

See: http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/parameters.html

So, yes. You can calculate the matrix using this procedure and use that to map the real-world 3D point to 2D output image.

vvg
  • 1,010
  • 7
  • 25
  • Great! I have seen that reference, but I do not understand how use that to create a 4x4 projection matrix for a Three.js camera. Can you help answer that? I see the "camera matrix" that they define as KK, but that is 3x3 whereas Three.js uses a 4x4 matrix. Also, none of its values relate to the "distortion coefficients", so I'm not sure how it could be the only thing I need. – Eric Simonton Aug 27 '20 at 12:39