basic_string::replace doesn't do what you think it does.
basic_string::replace(it_a, it_e, ... ) replaces all of the characters between it_a and it_e with whatever you specify, not just those that match something.
There are a hundred ways to do what you're trying to do, but the simplest is probably to use the std::replace from <algorithm>, which does do what you want:
std::replace(mystring.begin(), mystring.end(), '_', ',');
Another method is to use std::transform in conjunction with a functor.  This has an advantage over std::replace in that you can perform multiple substitutions in a single pass.
Here is a C++03 functor that would do it:
struct ReplChars
{
    char operator()(char c) const
    {
        if( c == '_' )
            return ',';
        if( c == '*' )
            return '.';
        return c;
    }
};
...and the use of it:
std::transform(mystring.begin(), mystring.end(), mystring.begin(), ReplChars());
In C++11, this can be reduced by using a lambda instead of the functor:
std::transform(mystring.begin(), mystring.end(), mystring.begin(), [](char c)->char
{
    if( c == '_' )
        return ',';
    if( c == '*' )
        return '.';
    return c;
});