I am attempting to zoom in/out relative to mouse pointer position. What I am expecting is similar to what can be found here. Zooming should be centered at the pointer position using CSS transform-origin and scale. When using the mousewheel to zoom, it works as expected at first. However, the moment I zoom and move the mouse pointer and attempt to zoom again, the divs I am zooming into move all over the page. I seem to have the math wrong. Or am I not understanding how the transform-origin works? How would I fix? Working code can be found here but I have also included it below.
    <div id="viewport">
        <div class="layer" id="stage">
        </div>
    </div>
$(document).mousewheel(function(event) {
    oldScale = scale
    scaleBy = 1.01
    let delta = event.deltaY
    let newScale = delta < 0
                 ? oldScale / scaleBy
                 : oldScale * scaleBy;
    scale = newScale
    let offset = document.querySelector('#stage').getBoundingClientRect();
    var offsetX = event.clientX - $('#stage').offset().left;
    var offsetY = event.clientY - $('#stage').offset().top;
    $('.layer').css('transform-origin', Math.round(offsetX/oldScale)+'px '+Math.round(offsetY/oldScale)+'px')
    $('.layer').css('transform', 'scale('+newScale+')')
})