How to specify a Double-sided material in RealityKit?

Question

I'm trying to load a model and texture in RealityKit (set up in an ARView instance), but I can't seem to figure out how to specify the material should be two-sided.

I have the model loaded up as a ModelEntity, the texture loaded up as a TextureResource. The model and texture are loading up, but are rending one-sided. Since the model is open (i.e., back faces are visible), there are gaps on how it is rendered.

So far, I have,

let entity: ModelEntity = try .loadModel(named: "model.obj")

var material = SimpleMaterial()
material.baseColor = try .texture(.load(named: "texture.png"))
entity.model?.materials = [material]

I was hoping to find a property such as

material.twoSided = true

but so far, I have not found the equivalent thing in RealityKit.

Anyone know how to set two-sided materials in RealityKit?

Answer 1

There doesn't seem to be any way to do this programmatically at the moment via the RealityKit APIs.

Can you change your model definition so it doesn't do back face culling? For example in a USDZ file I am importing it defines one part of the mesh as:

def Mesh "Plane_1"
  {
    uniform bool doubleSided = 1

You might be able to convert your obj file to a used file using usdzconvert first (https://developer.apple.com/download/more/?=USDPython) then edit the file manually, then import that in to your scene.

It might also be depending on how the model is setup that you can pass in more than one material to the materials array that are applied to different parts of the model, you can see how many materials the model expects:

entity.model?.mesh.expectedMaterialCount

Answer 2

As others already answered you can't do it programmatically. You can however do it manually for each model via the inspection panel. See the image below. Near the bottom, you have "Double Sided" checkbox.

Answer 3

It can be done, but it takes a change of perspective: instead of making a double sided material, you create a double sided mesh. This is accomplished by taking every part in every model and creating a double with the normals inverted (and the triangles reversed). Using the code below, the solution to the stated question becomes:

do {
   let entity: ModelEntity = try .loadModel(named: "model.obj")
   if let model = entity.model {
       try model.mesh.addInvertedNormals()
       // or alternatively, since this model isn't onscreen yet:
       // model.mesh = try model.mesh.addingInvertedNormals()
       var material = SimpleMaterial()
       material.baseColor = try .texture(.load(named: "texture.png"))
       model.materials = [material]
       entity.model = model
   }
} catch {}

And the entity will now display the material on both sides. Here is the code to do it:

import Foundation
import RealityKit

public extension MeshResource {
    // call this to create a 2-sided mesh that will then be displayed 
    func addingInvertedNormals() throws -> MeshResource {
        return try MeshResource.generate(from: contents.addingInvertedNormals())
    }
    
    // call this on a mesh that is already displayed to make it 2 sided
    func addInvertedNormals() throws {
        try replace(with: contents.addingInvertedNormals())
    }

    static func generateTwoSidedPlane(width: Float, depth: Float, cornerRadius: Float = 0) -> MeshResource {
        let plane = generatePlane(width: width, depth: depth, cornerRadius: cornerRadius)
        let twoSided = try? plane.addingInvertedNormals()
        return twoSided ?? plane
    }
}

public extension MeshResource.Contents {
    func addingInvertedNormals() -> MeshResource.Contents {
        var newContents = self

        newContents.models = .init(models.map { $0.addingInvertedNormals() })

        return newContents
    }
}

public extension MeshResource.Model {
    func partsWithNormalsInverted() -> [MeshResource.Part] {
        return parts.map { $0.normalsInverted() }.compactMap { $0 }
    }
    
    func addingParts(additionalParts: [MeshResource.Part]) -> MeshResource.Model {
        let newParts = parts.map { $0 } + additionalParts
        
        var newModel = self
        newModel.parts = .init(newParts)
        
        return newModel
    }
    
    func addingInvertedNormals() -> MeshResource.Model {
        return addingParts(additionalParts: partsWithNormalsInverted())
    }
}

public extension MeshResource.Part {
    func normalsInverted() -> MeshResource.Part? {
        if let normals, let triangleIndices {
            let newNormals = normals.map { $0 * -1.0 }
            var newPart = self
            newPart.normals = .init(newNormals)
            // ordering of points in the triangles must be reversed,
            // or the inversion of the normal has no effect
            newPart.triangleIndices = .init(triangleIndices.reversed())
            // id must be unique, or others with that id will be discarded
            newPart.id = id + " with inverted normals"
            return newPart
        } else {
            print("No normals to invert, returning nil")
            return nil
        }
    }
}

So, calling addingInvertedNormals() creates a mesh that will show the same material on both sides. I have used this to create a 2-sided grid plane.

With a little extra work (left as an exercise!), you could give the created parts different material indexes, and show different materials on each side.

Answer 4

I don't think there is a way to do this. RealityKit is still in early days. Material support in RealityKit is very limited right now. I think there are plans to change this in iOS 14 or beyond. There are comments in the documentation that describe features that don't yet exist such as Material protocol says "Describes a material (colors, shaders, textures, etc.) that define the look of a mesh part." There currently is no way to define custom shaders. If you look at the RealityKit framework bundle, there are shader graph definitions and new material features that are not yet exposed in the public API. I suspect there will be a shader graph editor, support for custom shaders, and double-sided materials coming.

Answer 5

Specifying two-sided material in RealityKit

You can achieve this natively in RealityKit by using PhysicallyBasedMaterial instead of SimpleMaterial for your ModelEntity, and changing the faceCulling property to .none

Your original code might be updated like this to have two-sided material:

let entity: ModelEntity = try! .loadModel(named: "model.obj")

var material = PhysicallyBasedMaterial()
material.baseColor.texture = // your texture here
material.faceCulling = .none
entity.model?.materials = [material]

You can read more about PhysicallyBasedMaterial and the faceCulling property in the official documentation.

Answer 6

Since RealityKit still doesn't have the isDoubleSided instance property that you can find in SceneKit, I offer two workarounds to bypass this limitation.

Autodesk Maya –> Solution 1

In Autodesk Maya, create two polygonal cubes, then slightly scale one of them down, then select it and apply Mesh Display > Reverse command from the Modeling main menu set. This command reverses faces' normals 180 degrees. Export both models (first, with default normals direction, and second, with reversed normals). Here's the Python code for Maya Script Editor:

import maya.cmds as cmds

cmds.polyCube(n='InnerCube', w=2.99, h=2.99, d=2.99)

# reversing normals of polyfaces
cmds.polyNormal(nm=0)

cmds.polyCube(n='OuterCube', w=3, h=3, d=3)
cmds.select(clear=True)

After that, import both models into the RealityKit.

RealityKit –> Solution 2

To do it programmatically in RealityKit, load original model and its duplicate into a scene, then for inner model, use faceCulling = .front to cull front-facing polygons. Scale down the internal model slightly.

Also, you can use culling for such effects as outline border.

import UIKit
import RealityKit

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        let boxScene = try! Experience.loadBox()
        
        // INNER
        var innerMaterial = PhysicallyBasedMaterial()
        innerMaterial.faceCulling = .front           // face culling
        innerMaterial.baseColor.tint = .green
        let box = boxScene.steelBox?.children[0] as! ModelEntity
        box.scale = [1,1,1] * 12
        box.model?.materials[0] = innerMaterial
        box.name = "Inner_Green"
        arView.scene.anchors.append(boxScene)
        
        // OUTER
        var outerMaterial = SimpleMaterial()
        let outerBox = box.clone(recursive: false)
        outerBox.model?.materials[0] = outerMaterial
        outerBox.scale = [1,1,1] * 12.001
        outerBox.name = "Outer_White"
        // boxScene.steelBox?.addChild(outerBox)     // Enable it
        
        print(boxScene)
    }
}

Answer 7

What you describe is called culling. Check MTLCullMode for example. From there you can jump to various points where you can set culling mode (you are interested in no culling).