Getting the color value of a pixel on click of a mesh with three.js

Question

I'm working with the last version of three.js and I got a scene with a 2D mesh, with gradient color. The color depends of a value assigned to each vertex. I would like to get the value of every point of my gradient by clicking on it with the mouse (by getting the color, and make some math for my value)

I tried to use a system like this one, since it works here : http://jsbin.com/wepuca/2/edit?html,js,output

var canvas = document.getElementById("myCanvas");
var ctx = canvas.getContext("2d");

function echoColor(e){
    var imgData = ctx.getImageData(e.pageX, e.pageX, 1, 1);
    red = imgData.data[0];
    green = imgData.data[1];
    blue = imgData.data[2];
    alpha = imgData.data[3];
    console.log(red + " " + green + " " + blue + " " + alpha);  
}

My problem is that this technique is used to get data from the canvas, when an image is displayed, and not a mesh. When I tried to use it on my mesh, I have that error :

"TypeError: ctx is null" or "imageData is not a function" (depending on how I declared my canvas and his context).

Since I didn't find if this method can be used with mesh, I'm here asking if it's possible or not, more than how to do it.

Thanks in advance

Answer 1

First, you cannot get a 2D context from a canvas which is already using a WebGL context. This is why you're getting the ctx is null error.

To get the pixel color, you have two options.

1. Transfer the WebGL canvas image onto a canvas with a 2D context.

To do this, create a new canvas and size it to be the same size as your THREE.js canvas, and get a 2D context from it. You can then draw the rendered image from the THREE.js canvas onto your new canvas as an image, using the 2D context:

var img2D = new Image();
twoDeeCanvasImage.addEventListener("load", function () {
    ctx2D.clearRect(0, 0, canvas2D.width, canvas2D.height);
    ctx2D.drawImage(img2D, 0, 0);
    // from here, get your pixel data
});
img2D.src = renderer.domElement.toDataURL("img/png");

From there you can use getImageData, like you did before.

One caveat of this method is that you may see image degradation due to compression introduced when exporting the image via toDataURL. You can circumvent this in some cases:

if(ctx2D.imageSmoothingEnabled !== undefined){
    ctx2D.imageSmoothingEnabled  = false;
}
else if(ctx2D.mozImageSmoothingEnabled  !== undefined){
    ctx2D.mozImageSmoothingEnabled   = false;
}
else if(ctx2D.webkitImageSmoothingEnabled  !== undefined){
    ctx2D.webkitImageSmoothingEnabled   = false;
}
else{
    console.warn("Browser does not support disabling canvas antialiasing. Results image may be blurry.");
}

2. You can also get a pixel value directly from the renderer, but it will cost you an extra render pass.

See this THREE.js example: https://threejs.org/examples/?q=instanc#webgl_interactive_instances_gpu

This example uses readRenderTargetPixels in the THREE.WebGLRenderer. https://threejs.org/docs/#api/renderers/WebGLRenderer

I don't have any personal experience using this method, so hopefullysomeone else can fill in some of the blanks.