No, it can not be done with regular expressions "directly". And here you can read why.
Anyway, for a solution using regular expressions (a lot of code, but depending of your data length it can be faster or not, you will need to try)
Dim dicEncode
    set dicEncode = WScript.CreateObject("Scripting.Dictionary")
Dim encodeRE
    Set encodeRE = New RegExp
    With encodeRE
        .Pattern = "\{[^{}]*\}"
        .Global = True
        .IgnoreCase = True
    End With
Dim decodeRE
    Set decodeRE = New RegExp
    With decodeRE
        .Pattern = "\x00(K[0-9]+)\x00"
        .Global = True
        .IgnoreCase = True
    End With
Function encodeFunction(matchString, position, fullString)
    Dim key
        key = "K" & CStr(dicEncode.Count)
    dicEncode.Add key , matchString
    encodeFunction = Chr(0) & key & Chr(0)
End Function 
Function decodeFunction(matchString, key, position, fullString)
    decodeFunction = dicEncode.Item(key)
End Function
Dim originalString    
    originalString = "{abc.def{gh.ijk}l.m}n.o.p{q.r{s{t{u.v}}}w}.x"
Dim encodedString, workBuffer
    encodedString = originalString
    Do
        workBuffer = encodedString
        encodedString = encodeRE.Replace(encodedString,GetRef("encodeFunction"))
    Loop While encodedString <> workBuffer
    encodedString = Replace(encodedString, ".", Chr(0))
    Do 
        workBuffer = encodedString
        encodedString = decodeRE.Replace(encodedString,GetRef("decodeFunction"))
    Loop While encodedString <> workBuffer
Dim aElements, element
    aElements = Split(encodedString, Chr(0))
    WScript.Echo originalString
    For Each element In aElements
        WScript.Echo element
    Next 
All this code just uses regular expressions to find the pairing curly brackets in the string, replacing them and its enclosed data with a key indicator that is stored in a dictionary. When all the "enclosed" data is removed from the string, the remaining dots (your split points) are replaced with a new character (that will later be used to split the string) and then the string is reconstructed. All the "enclosed" dots has been protected and the split can be done over the string using the new character ( Chr(0) in the code).
It is similar to the dictionary creation of a statistical compressor but without any statistics and no compression, of course. 
But only useful with long strings. If not, your original approach is way better.
EDITED to adapt to comments
For a better performing code, based on the OP original approach. No exotic regular expresions. Just reduced string concatenation and unnecesary checks removed.
Function mySplit(originalString)
Dim changedString, currentPoint, currentChar, stringEnd, level
    changedString = originalString
    stringEnd = Len(originalString)
    level = 0
    For currentPoint = 1 To stringEnd
        currentChar = Mid(originalString, currentPoint, 1)
        If currentChar = "{" Then 
            level = level + 1
        ElseIf currentChar = "}" Then
            If level > 0 Then 
                level = level - 1
            End If
        ElseIf level = 0 Then 
            If currentChar = "." Then 
                changedString = Left(changedString,currentPoint-1) & Chr(0) & Right(changedString,stringEnd-currentPoint)
            End If
        End If
    Next 
    mySplit = split( changedString, Chr(0) )
End Function