I do it like this:
Public Class ApplicationDomainBridge
    Inherits MarshalByRefObject
    Public ReturnValue As Object
    Public ErrorObject As System.Exception
End Class
<Serializable>
Public Class AsmHelper
    ' Inherits MarshalByRefObject
    Private Class ApplicationDomainBridge
        Inherits MarshalByRefObject
        Public ErrorObject As System.Exception
        Public Eval As ReportTester.Parameters.AbstractEvaluator
    End Class
    Private AsmData As Byte()
    Private CallbackResult As ApplicationDomainBridge
    Sub New()
        Me.CallbackResult = New ApplicationDomainBridge
    End Sub
    Public Sub LoadAsmAndExecuteEval()
        Try
            ' System.Console.WriteLine("Executing in: " & AppDomain.CurrentDomain.FriendlyName)
            ' Throw New System.InvalidCastException("Test")
            Dim assembly As System.Reflection.Assembly = System.Reflection.Assembly.Load(AsmData)
            ' Here we do something
            ' The object you return must have 
            ' Inherits MarshalByRefObject
            Dim programType As System.Type = assembly.GetType("RsEval")
            Dim eval As ReportTester.Parameters.AbstractEvaluator = DirectCast(System.Activator.CreateInstance(programType), ReportTester.Parameters.AbstractEvaluator)
            Me.CallbackResult.Eval = eval
        Catch ex As Exception
            Me.CallbackResult.ErrorObject = ex
        End Try
    End Sub
    Public Shared Function ExecuteInAppTemporaryAppDomain(temporaryAppDomain As AppDomain, ByVal assemblyBytes As Byte()) As ReportTester.Parameters.AbstractEvaluator
        Dim loadExecute As AsmHelper = New AsmHelper() With {
        .AsmData = assemblyBytes
    }
        temporaryAppDomain.DoCallBack(New CrossAppDomainDelegate(AddressOf loadExecute.LoadAsmAndExecuteEval))
        loadExecute.AsmData = Nothing
        Dim retValue As ReportTester.Parameters.AbstractEvaluator = Nothing
        If loadExecute.CallbackResult.ErrorObject Is Nothing Then
            retValue = loadExecute.CallbackResult.Eval
            loadExecute.CallbackResult = Nothing
            loadExecute = Nothing
        End If
        If loadExecute IsNot Nothing AndAlso loadExecute.CallbackResult IsNot Nothing AndAlso loadExecute.CallbackResult.ErrorObject IsNot Nothing Then
            Throw loadExecute.CallbackResult.ErrorObject
        End If
        Return retValue
    End Function
End Class
Usage:
Dim assemblyBytes As Byte() = System.IO.File.ReadAllBytes(results.PathToAssembly)
Dim temporaryAppDomain As AppDomain = CreateAppDomain()
Dim evaluator As ReportTester.Parameters.AbstractEvaluator = AsmHelper.ExecuteInAppTemporaryAppDomain(temporaryAppDomain, assemblyBytes)
    evaluator.Domain = temporaryAppDomain
And it you thought loading an appdomain is hacky, wait until you must unload the appdomain in a disposable. 
I did that like this:
Protected Overridable Sub Dispose(disposing As Boolean)
    If Not disposedValue Then
        If disposing Then
            ' TODO: verwalteten Zustand (verwaltete Objekte) entsorgen.
            If Me.LoadContext IsNot Nothing Then
                Me.LoadContext.Unload()
                Me.LoadContext = Nothing
            End If
            If Me.Stream IsNot Nothing Then
                Me.Stream.Dispose()
                Me.Stream = Nothing
            End If
            'If Parameters.m_parameterValues IsNot Nothing Then
            '    Parameters.m_parameterValues.Clear()
            '    Parameters.m_parameterValues = Nothing
            'End If
            If Me.Domain IsNot Nothing Then
                ' https://stackoverflow.com/questions/7793074/unload-an-appdomain-while-using-idisposable
                Dim thread As System.Threading.Thread = New System.Threading.Thread(
                  Sub(obj As System.AppDomain)
                      Try
                          ' System.Threading.Thread.Sleep(1000)
                          System.AppDomain.Unload(obj)
                      Catch ex As System.Threading.ThreadAbortException
                          ' System.Console.WriteLine(ex.Message)
                          System.GC.Collect()
                          System.GC.WaitForPendingFinalizers()
                      End Try
                  End Sub
                )
                thread.IsBackground = True
                thread.Start(Me.Domain)
                Me.Domain = Nothing
            End If
            System.GC.Collect()
            System.GC.WaitForPendingFinalizers()
        End If
        ' TODO: nicht verwaltete Ressourcen (nicht verwaltete Objekte) freigeben und Finalize() weiter unten überschreiben.
        ' TODO: grosse Felder auf Null setzen.
    End If
    disposedValue = True
End Sub
Because if you do it with the generic idisposable method from SO, it will fail because the action is not serializable...
Note that the origin of the trouble is, that in the new appdomain, Assembly.Load code can't access the byte array from the old appdomain, and instead gets an empty byte-array, thus a class with attribute Serializable... Or respectively, if you want to return the assembly in ApplicationDomainBridge, you get a "missing assembly" exception (serialization issue?), although it loaded fine in the other domain.