SQL Server 2012 added a Try_Convert function, which returns NULL if the value cannot be casted as the given type. http://technet.microsoft.com/en-us/library/hh230993.aspx
WHERE NOT (OldValue is Null AND 
    (NewValue is null OR try_convert(float, NewValue) = 0.0)
)
If using a version prior to 2012, check out Damien_The_Unbeliever's answer here: Check if a varchar is a number (TSQL) ...based on Aaron's comment this will not work in all cases.
Since you are using SQL 2008, then it appears a combination of isnumeric and a modified version of Damien's answer from the link above will work.  Your current solution in your question would have problems with values like '.', '-', currency symbols ($, etc.), and scientific notation like '1e4'.
Try this for SQL 2008 (here is SQLFiddle with test cases: http://sqlfiddle.com/#!3/fc838/3 ): Note: this solution will not convert text values to numeric if the text has commas (ex: 1,000) or accounting notation with parens (ex: using "(1)" to represent "-1"), because SQL Server will throw an error when trying to cast to decimal.
WHERE t.OldValue is null
AND 
(
    t.NewValue is null
    OR
    0.0 = 
    case 
        when isnumeric(t.NewValue) = 1 
            --Contains only characters that are numeric, negative sign, or decimal place. 
            --This is b/c isnumeric will return true for currency symbols, scientific notation, or '.'
            and not (t.NewValue like '%[^0-9.\-\+]%' escape '\')  
            --Not other single char values where isnumeric returns true.
            and t.NewValue not in ( '.', '-', '+')
        then cast(t.NewValue as decimal(18,4)) 
        else null --can't convert to a decimal type 
    end
)