Context:
I'm using Andrei Kashcha's (anvaka) panzoom library: https://github.com/anvaka/panzoom to provide detailed color picking functionalities:
I've created a 1' movie to illustrate the use case: https://www.dropbox.com/s/5tv9rs18vvpv95d/tfs_cp.mov?dl=0
Issue:
- Part I [ Resolved, see below for Part II which remains unresolved ]
// 14-12-2020: Initial post
This all works fine on both Chrome/Safari on Desktop, but when switching to Mobile, I'm receiving the following error on touchend (_listener2):
Uncaught ReferenceError: x is not defined
at cp_vfinal:1229
at useCanvas (cp_vfinal:761)
at HTMLImageElement._listener2 (cp_vfinal:1221)
More specifically, the following part of the code which I've shared below:
var p = inst.canvas.getContext('2d')
.getImageData(x, y, 1, 1).data;
My assumption is that this error will also manifest on touchmove (_listener4), but I cannot test this properly with Chrome's built-in Mobile device-mode. I also have no access to XCode, which makes it also difficult to test in Safari.
// 29/12: Update
Meanwhile I figured out that following piece of code was causing the ReferenceError:
// chrome
if(e.offsetX) {
x = e.offsetX;
y = e.offsetY;
}
// firefox
else if(e.layerX) {
x = e.layerX;
y = e.layerY;
}
Replacing this by the following, resolved my issue.
x = e.changedTouches[0].clientX - e.changedTouches[0].target.offsetLeft
y = e.changedTouches[0].clientY - e.changedTouches[0].target.offsetTop;
- Part II [ Unresolved ]
// 14-12-2020: Initial post
When zooming/panning on Mobile and subsequently selecting a color on Mobile, it seems that the color selection made is based on a canvas that is not yet zoomed/panned.
I've tried quite some things already, but I simply cannot figure it out. Although other's have attempted the same:
- Implementing zoom on Canvas
- Zooming canvas on mobile
- Horrible Canvas GetImageData() / PutImageData() performance on mobile
- Canvas getImageData returning incorrect data on certain mobile devices
I couldn't use that input to resolve my issue either.
// 29-12-2020: Update
As of today, I still wasn't able to resolve the second part of the issue.
When comparing
console.log(inst.canvas);
console.log(inst.canvas.getContext('2d'))
for both mobile and desktop I don't see any difference, so it's really strange that on mobile I cannot retrieve the correct data through getImageData(). How can I ensure that getImageData() is actually checking for the zoomed/scaled/panned image (canvas) and not for the unzoomed/unscaled/unpanned image (canvas)? That would already help me debug further.
// 05-04-2021: Update
I still haven't figured out what causes the issue. Three key questions remain:
- Q1: Am I using drawImage() correctly?
- Q2: Am I using getImageData() correctly?
- Q3: Regardless of Q1/Q2, why is it working on Desktop then?
// 15-05-2021: Update
I still haven't figured out what causes the issue. The same key questions remain, but I've made some progress in terms of investigations:
- Q1: Am I using drawImage() correctly?
I've now tried to use void ctx.drawImage(image, sx, sy, sWidth, sHeight, dx, dy, dWidth, dHeight); instead of void ctx.drawImage(image, dx, dy, dWidth, dHeight); as this seemed more logical after reading the documentation on MDN: https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/drawImage Unfortunately this didn't change much. I've updated below visualization to reflect my adjustments.
- Q2: Am I using getImageData() correctly?
I have no idea what could be wrong here.
- Q3: Regardless of Q1/Q2, why is it working on Desktop then?
I've also tested my implementation in a smartphone simulator: https://www.webmobilefirst.com/en/ Works like a charm. Hence it's really the physical device that is causing the issues.
Visualization:
The testimage with intrinsic dimensions of 2075x3112 is loaded into a canvas with width = 375 and height = 562 - this is the situation on the left. The situation on the right depicts what happens after panning/zooming.
Snippets:
- drawImage(): called when panning/zooming
function useCanvas(el, image, callback){
console.log("useCanvas")
console.log(image);
//https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/drawImage
//console.log(el.width);
//console.log(el.height);
if(!checkCanvasSize(el, image)){
console.log("checkCanvasSize")
el.width = image.width; // img width
el.height = image.height; // img height
console.log(el.width);
console.log(el.height);
// draw image in canvas tag - void ctx.drawImage(image, dx, dy, dWidth, dHeight);
el.getContext('2d').drawImage(image, 0, 0, image.width, image.height);
console.log(el)
console.log(el.getContext('2d'))
} else {
console.log("else ...")
console.log(image.style.transform)
//if no transformation - do nothing
//if transformation - do "something"
if(image.style.transform !== "matrix(1, 0, 0, 1, 0, 0)"){
//- draw transformed image onto canvas
el.getContext('2d').drawImage(image, 0, 0, image.width, image.height);
}
}
return callback();
}
- getImageData()
_listener2 = function(e) { //touchend
//console.log(e.srcElement)
tlength = e.touches.length;
cpsig1 = cpsig1 + tlength.toString();
if(tlength === 0){
if( (tf.x < -1E-9 || tf.x > 1E-9) || (tf.y < -1E-9 || tf.y > 1E-9) || tf.scale > 1){ //pan or zoom is true
//activate CP again
activateCP = true;
//wait for the next user action to do something (e.g. click/tap)
if(cpsig1.slice(cpsig1.length - 2) === "00"){
registerC = true;
} else {
registerC = false;
}
} else {
//perform a check whether we came from a 'transformed' situation or not, in case yes, don't set the color.
if(transform === true){ //we came from a 'transformed' situation
registerC = false; //we wait for a user action
transform = false;
//tmpvar = true;
} else { //register the underlying color that was touched before going to TE=0 and move to next color (registerC should be true here)
//extra handle
if(cpsig1.slice(cpsig1.length - 2) === "20"){
registerC = false;
} else {
registerC = true;
}
}
}
}
// chrome
if(e.offsetX) {
x = e.offsetX;
y = e.offsetY;
}
// firefox
else if(e.layerX) {
x = e.layerX;
y = e.layerY;
}
useCanvas(inst.canvas, inst.img, function(){
// get image data
//console.log("Debug:")
//console.log(inst.img);
//console.log(inst.canvas);
//console.log(inst.canvas.getContext('2d'))
//https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Pixel_manipulation_with_canvas
var p = inst.canvas.getContext('2d')
.getImageData(x, y, 1, 1).data;
// show info
inst.result_hex = rgbToHex(p[0],p[1],p[2]);
inst.result_rgb_string = 'rgb('+p[0]+','+ p[1]+','+ p[2]+')';
// set value
console.log("skinSelection: ", skinSelection)
if(skinSelection === false && tlength === 0 && registerC === true){
document.getElementById("preview_skin").style.backgroundColor = rgbToHex(p[0],p[1],p[2]);
document.getElementById("preview_skin").style.backgroundSize = "42.5px 42.5px"
document.getElementById("preview_skin").style.backgroundImage = "url('https://static.wixstatic.com/media/3e929c_000c0389ad654985be5d7983694c7695~mv2.png')";
document.getElementById("preview_skin").style.backgroundRepeat = "no-repeat"
document.getElementById("preview_skin").style.backgroundPosition = "center 50%"
skinSelection = true;
document.getElementById('preview_skin').style.border = "5px solid #fff";
} else {
if(hairSelection === false && tlength === 0 && registerC === true){
document.getElementById("preview_hair").style.backgroundColor = rgbToHex(p[0],p[1],p[2]);
document.getElementById("preview_hair").style.backgroundSize = "42.5px 42.5px"
document.getElementById("preview_hair").style.backgroundImage = "url('https://static.wixstatic.com/media/3e929c_cad76816bc4441b6bfe34f5008b2741d~mv2.png')";
document.getElementById("preview_hair").style.backgroundRepeat = "no-repeat"
document.getElementById("preview_hair").style.backgroundPosition = "center 50%"
hairSelection = true;
document.getElementById('preview_hair').style.border = "5px solid #fff"
} else {
if(eyeSelection === false && tlength === 0 && registerC === true){
document.getElementById("preview_eye").style.backgroundColor = rgbToHex(p[0],p[1],p[2]);
document.getElementById("preview_eye").style.backgroundSize = "42.5px 42.5px"
document.getElementById("preview_eye").style.backgroundImage = "url('https://static.wixstatic.com/media/3e929c_f628dfa5fe0a412da6f06954a8657458~mv2.png')";
document.getElementById("preview_eye").style.backgroundRepeat = "no-repeat"
document.getElementById("preview_eye").style.backgroundPosition = "center 50%"
eyeSelection = true;
document.getElementById('preview_eye').style.border = "5px solid #fff"
}
}
}
console.log("R: " + p[0] + " - G: " + p[1] + " - B: " + p[2]);
});
}
'MWE':
As it is difficult to create a MWE for this, I will share the full working example which can be found here: https://yvervoort.github.io/cp-acc_web_v0.2.html The Javascript is available under the 'sources' tab of your Developer Tools.
Changelog:
- 14-12-2020: Initial question
- 29-12-2020: Updated question - Part I resolved | Part II remains unresolved
- 29-12-2020: Updated title to better reflect what remains to be resolved
- 05-04-2021: Updated explanation (included a visualization of the issue) and added new key questions I believe are related to the issue of Part II which remains unresolved
- 15-05-2021: Updated the progress I made, which is limited. I do know now that the implementation also works fine on a smartphone emulator and hence only seems to cause issues on the actual physical device.
