You can combine AllowCurrencySymbol, AllowDecimalPoint and AllowThousands styles and use a culture that has $ as a CurrencySymbol like en-US
var s = "$232,680.00";
decimal d = decimal.Parse(s, NumberStyles.AllowCurrencySymbol |
                             NumberStyles.AllowDecimalPoint | 
                             NumberStyles.AllowThousands, 
                             CultureInfo.GetCultureInfo("en-US"));
or more simple use NumberStyles.Currency instead which contains those styles as well.
decimal d = decimal.Parse(s, NumberStyles.Currency, CultureInfo.GetCultureInfo("en-US"));
The problem in your code is InvariantCulture has ¤ (which is Currency Sign (U+00A4)) not $ as a CurrencySymbol. If you change your InvariantCulture to CultureInfo.GetCultureInfo("en-US"), your code will work as well.
en-US culture totally fits for your string since it has . as a decimal separator and , as a thousand separator.