Usually when working with CSV we double quotations:
a                 -> "a"
a"b               -> "a""b"
NewClient"Name    -> "NewClient""Name"
To cut such a quotation, i.e. 
"NewClient""Name" -> NewClient"Name
when "NewClient"Name" being a syntax error you can try
private static string CutQuotation(string value) {
  if (string.IsNullOrEmpty(value))
    return value;
  else if (!value.Contains('"'))
    return value;
  if (value.Length == 1)
    throw new FormatException("Incorrect quotation format: string can't be of length 1.");
  else if (value[0] != '"')
    throw new FormatException("Incorrect quotation format: string must start with \".");
  else if (value[value.Length - 1] != '"')
    throw new FormatException("Incorrect quotation format: string must end with \".");
  StringBuilder builder = new StringBuilder(value.Length);
  for (int i = 1; i < value.Length - 1; ++i) 
    if (value[i] == '"') 
      if (i == value.Length - 2)
        throw new FormatException("Incorrect quotation format. Dangling \".");
      else if (value[++i] == '"') 
        builder.Append(value[i]);
      else
        throw new FormatException("Incorrect quotation format. Dangling \".");
    else
      builder.Append(value[i]);
  return builder.ToString();
}
As you can see, it's not just single Replace() routine.
Tests:
 // abc - no quotation
 Console.WriteLine(CutQuotation("abc")); 
 // abc - simple quotation cut
 Console.WriteLine(CutQuotation("\"abc\"")); 
 // "abc" - double quotation
 Console.WriteLine(CutQuotation("\"\"abc\"\"")); 
 // a"bc - quotation in the middle
 Console.WriteLine(CutQuotation("\"a\"\"bc\""));