0

I'm using a MTKView to display some pixel art, but it shows up blurry.

Here is the really weird part: I took a screenshot to show you all what it looks like, but the screenshot is perfectly sharp! Yet, the contents of the MTKView is blurry. Here's the screenshot, and a simulation of what it looks like in the app:

Note the test pattern displayed in the app is 32 x 32 pixels.

enter image description here enter image description here

When switching from one app to this one, the view is briefly sharp, before instantly becoming blurry.

I suspect this has something to do with anti-aliasing, but I can't seem to find a way to turn it off. Here is my code:

import UIKit
import MetalKit

class ViewController: UIViewController, MTKViewDelegate {
    
    var metalView: MTKView!
    var image: CIImage!
    var commandQueue: MTLCommandQueue!
    var context: CIContext!

    override func viewDidLoad() {
        super.viewDidLoad()
        setup()
        layout()
    }
    
    func setup() {
        guard let image = loadTestPattern() else { return }
        self.image = image
        
        let metalView = MTKView(frame: CGRect(origin: CGPoint.zero, size: image.extent.size))
        
        metalView.device = MTLCreateSystemDefaultDevice()
        metalView.delegate = self
        metalView.framebufferOnly = false
        
        metalView.isPaused = true
        metalView.enableSetNeedsDisplay = true
        
        commandQueue = metalView.device?.makeCommandQueue()
        context = CIContext(mtlDevice: metalView.device!)
        
        self.metalView = metalView
        view.addSubview(metalView)
    }
    
    func layout() {
        let size = image.extent.size
        
        metalView.translatesAutoresizingMaskIntoConstraints = false
        NSLayoutConstraint.activate([
            metalView.centerXAnchor.constraint(equalTo: view.centerXAnchor),
            metalView.centerYAnchor.constraint(equalTo: view.centerYAnchor),
            metalView.widthAnchor.constraint(equalToConstant: size.width),
            metalView.heightAnchor.constraint(equalToConstant: size.height),
       ])
        
        let viewBounds = view.bounds.size
        let scale = min(viewBounds.width/size.width, viewBounds.height/size.height)
        
        metalView.layer.magnificationFilter = CALayerContentsFilter.nearest;
        metalView.transform = metalView.transform.scaledBy(x: floor(scale * 0.8), y: floor(scale * 0.8))
    }
    
    func loadTestPattern() -> CIImage? {
        guard let uiImage = UIImage(named: "TestPattern_32.png") else { return nil }
        guard let image = CIImage(image: uiImage) else { return nil }
        return image
    }

    func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {}
    
    func draw(in view: MTKView) {

        guard let image = self.image else { return }
        
        if let currentDrawable = view.currentDrawable,
           let commandBuffer = self.commandQueue.makeCommandBuffer() {
            
                let drawableSize = view.drawableSize
                let scaleX = drawableSize.width / image.extent.width
                let scaleY = drawableSize.height / image.extent.height
                let scale = min(scaleX, scaleY)
            
                let scaledImage = image.samplingNearest().transformed(by: CGAffineTransform(scaleX: scale, y: scale))
                
                let destination = CIRenderDestination(width: Int(drawableSize.width),
                                                     height: Int(drawableSize.height),
                                                pixelFormat: view.colorPixelFormat,
                                              commandBuffer: nil,
                                         mtlTextureProvider: { () -> MTLTexture in return currentDrawable.texture })
            
                try! self.context.startTask(toRender: scaledImage, to: destination)
                
                commandBuffer.present(currentDrawable)
                commandBuffer.commit()
        }
    }
}

Any ideas on what is going on?

Edit 01:

Some additional clues: I attached a pinch gesture recognizer to the MTKView, and printed how much it's being scaled by. Up to a scale factor of approximately 31-32, it appears to be using a linear filter, but beyond 31 or 32, nearest filtering takes over.

Clue #2: Problem disappears when MTKView is replaced with a standard UIImageView.

I'm not sure why that is.

RRR
  • 71
  • 7

1 Answers1

1

You can find how to turn on/off multisampling anti-aliasing How to use multisampling with an MTKView? Just have .sampleCount = 1. However, you problem doesn't look like MSAA-related.

My only idea. Here I'd check framebuffer sizes in Metal Debugger in XCode. Sometimes (depending on contentScale factor on your device) framebuffer can be stretched. E.g. your have a device with virtual resolution 100x100 and content scale factor 2. Physical resolution would be 200x200 in this case, and framebuffer 100x100 will be stretched by the system. This may happen with implicit linear filtering, instead of nearest one you set for main render pass. For screenshots it can use 1:1 resolution and system stretching doesn't happen.

rokuz
  • 386
  • 1
  • 7
  • 1
    Thank you Rokuz, for your suggestions. I've ruled out MSAA, and contenScale is indeed 2.0. But I suspect that isn't the issue either, because I attached a pinch gesture to the MTKView, and the blurring only happens up to a scaling factor of approx 31. Beyond that, nearest filtering takes over. I've set MTKView's magnificationFilter to also be nearest, as well as with any scaling going on in the draw function. – RRR Nov 26 '22 at 20:11
  • 1
    second clue, this only appears to be happening with MTKView. When I replace it with a standard UIImageView, the issue disappears. – RRR Nov 26 '22 at 20:33