SceneKit unproject a point gives weird result

Question

First, I project a specific point in the scene.

SCNVector3 topLeft = SCNVector3Make(
    -50,
    -50,
    -UIScreen.mainScreen.bounds.size.width);
SCNVector3 projectedPoint = [self.sceneView projectPoint:topLeft];

When I print the value I get an expected location on screen.

(-50 -50 -667) --> (309 212 1)

But when I unproject this, basically looking for a reverse operation.

SCNVector3 point = SCNVector3Make(309, 212, 1);
SCNVector3 unprojectedPoint = [self.sceneView unprojectPoint:point];

I get something very different.

(309, 212, 1) --> (-7.544 -7.544 -100)

What am I missing/doing wrong?

Answer 1

Your SCNVector3 topLeft uses as z-coordinate the screen width, which is unrelated to the world coordinate system.

Coordinate Systems

SceneKit uses a right-handed coordinate system where (by default) the direction of view is along the negative z-axis, as illustrated below.

https://developer.apple.com/documentation/scenekit/organizing_a_scene_with_nodes

Project Point from World Coordinates to 2D View

projectPoint(_:)

Projects a point from the 3D world coordinate system of the scene to the 2D pixel coordinate system of the renderer.

https://developer.apple.com/documentation/scenekit/scnscenerenderer/1524089-projectpoint

Illustration

Here we see a cube with dimension 2. It is located at the origin of the world coordinate system (x: 0, y: 0, z: 0). The camera is at at x: 0, y: 0, z: 10.

Now to project the cube's position (0/0/0) to the 2D view, one would write this in Objective-C:

SCNVector3 projectedPoint = [self.sceneView projectPoint:SCNVector3Make(0, 0, 0)];

On a iPhone 8 Plus we know the screen dimensions are 414 x 736. So we would expect as result x: 207 y: 368.

If we log the output then with this code:

NSLog(@"projectedPoint: %f %f %f", projectedPoint.x, projectedPoint.y, projectedPoint.z);

one would get something like:

projectedPoint: 207.000000 368.000000 0.909091

So now using this projectedPoint and unproject it we should get the origin again:

SCNVector3 unprojectedPoint = [self.sceneView unprojectPoint:projectedPoint];
NSLog(@"unprojectedPoint: %f %f %f", unprojectedPoint.x, unprojectedPoint.y, unprojectedPoint.z);

And indeed the output is

unprojectedPoint: 0.000000 0.000000 0.000000

Complete Example

Here the complete code for the scenario described above:

@interface GameViewController ()

@property (nonatomic, weak) SCNView *scnView;
@property (nonatomic, strong) SCNNode *cameraNode;
@property (nonatomic, strong) SCNScene *scnScene;
@property (nonatomic, strong) SCNNode *boxNode;

@end

@implementation GameViewController

- (void)viewDidLoad {
    [super viewDidLoad];
    [self _setupView];
    [self _setupScene];
    [self _setupCamera];
    [self _setupTapping];
    [self _addCube];
}

-(void)_setupTapping {
    UITapGestureRecognizer *tapGestureRecognizer = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(_tapHandler:)];
    self.scnView.gestureRecognizers = @[tapGestureRecognizer];
}

-(void)_tapHandler:(UIGestureRecognizer *)gestureRecognizer {
    NSLog(@"--------------");
    SCNVector3 projectedPoint = [self.scnView projectPoint:SCNVector3Make(0, 0, 0)];
    NSLog(@"projectedPoint: %f %f %f", projectedPoint.x, projectedPoint.y, projectedPoint.z);

    SCNVector3 unprojectedPoint = [self.scnView unprojectPoint:projectedPoint];
    NSLog(@"unprojectedPoint: %f %f %f", unprojectedPoint.x, unprojectedPoint.y, unprojectedPoint.z);

    CGPoint point = [gestureRecognizer locationInView:self.scnView];
    SCNVector3 scenePoint = [self.scnView unprojectPoint:SCNVector3Make(point.x, point.y, projectedPoint.z)];
    NSLog(@"point %f %f -> scenePoint: %f %f %f", point.x, point.y, scenePoint.x, scenePoint.y, scenePoint.z);
}

-(void)_setupView {
    self.scnView = (SCNView *)self.view;
    self.scnView.autoenablesDefaultLighting = YES;
}

-(void)_setupScene {
    self.scnScene = [SCNScene new];
    self.scnView.scene = self.scnScene;
}

- (void)_setupCamera {
    self.cameraNode = [SCNNode new];
    self.cameraNode.camera = [SCNCamera camera];
    self.cameraNode.position = SCNVector3Make(0, 0, 10);
    [self.scnScene.rootNode addChildNode:self.cameraNode];
}

- (void)_addCube {
    SCNBox *box = [SCNBox boxWithWidth:2 height:2 length:2 chamferRadius:0];
    SCNMaterial *mat = [SCNMaterial new];
    mat.diffuse.contents = @"art.scnassets/crosshair.png";
    box.materials = @[mat];
    self.boxNode = [SCNNode new];
    self.boxNode.geometry = box;
    self.boxNode.position = SCNVector3Make(0, 0, 0);
    [self.scnScene.rootNode addChildNode:self.boxNode];
}

- (BOOL)prefersStatusBarHidden {
    return YES;
}

@end

Tap on upper left corner of Cube

The cube in the example has a crosshair texture. On a iPhone this would look like this:

So now you can tap somewhere on the view.

With the following code

CGPoint point = [gestureRecognizer locationInView:self.scnView];
SCNVector3 scenePoint = [self.scnView unprojectPoint:SCNVector3Make(point.x, point.y, projectedPoint.z)];
NSLog(@"point %f %f -> scenePoint: %f %f %f", point.x, point.y, scenePoint.x, scenePoint.y, scenePoint.z);

we output the unprojected point (world coordinate system) on the area shown as a grid in the illustration above.

Since the cube has the dimension 2 and is located at the origin the exact world coordinate point would be x: -1, y: -1, z: 0.

For example when I tap on the upper left corner of the cube in the simulator I get something like:

point 141.000000 299.000000 -> scenePoint: -1.035465 1.082531 0.000000