c++ strtok in function changes original string value as parameter

Question

when I use strtok to tokenize a c++ string, it happens a confusing problem, see the simple code below:

void a(string s){
    strtok((char*)s.c_str(), " ");
}
int main(){
    string s;
    s = "world hello";
    a(s);
    cout<<s<<endl;
    return 0;
}

the program outputs "world". Shouldn't it output "world hello"? Because I pass the string as a value parameter to function a, the strtok shouldn't modify the original s... Can anyone explain this trick. thank you.

Don't do that, you're asking for memory corruption issues. Yes, strtok does alter it's input. http://www.cplusplus.com/reference/clibrary/cstring/strtok/ — John Carter, Oct 01 '11 at 03:35

score 0 · Accepted Answer · answered Oct 01 '11 at 03:34

0

The problem is (char*)s.c_str(), you are casting the constness away and modified the string contents in a way that you are not supposed to. While the original s should not be modified, I pressume you may have been hit by a smart optimization that expects you to play by the rules. For instance, a COW implementation of string would happen to show that behavior.

answered Oct 01 '11 at 03:34

K-ballo

80,396
20
159
169

thx. Btw, do you know how to convert a c++ string to char* safely and efficiently, so i can use char* in the strtok without problems? – stackunderflow Oct 01 '11 at 04:20
@zwx: If you intend to modify the `char*` then you need to make a copy of it. – K-ballo Oct 01 '11 at 04:24

Chris Eberle · Answer 2 · 2011-10-01T03:43:23.900

0

c_str() returns a const pointer, which is a promise to the compiler that the thing being pointed at won't be modified. And then you're calling strtok which modifies it.

When you lie to the compiler, you will be punished.

edited Oct 01 '11 at 03:43

answered Oct 01 '11 at 03:35

Chris Eberle

47,994
12
82
119

An `std::string` has no problems with nulls. – K-ballo Oct 01 '11 at 03:39

score 0 · Answer 3 · answered Oct 01 '11 at 03:43

That's the way strtok() works. It use the first parameter as a buffer. By casting it to a char*, you allow it to modify the string. strtok() does not known about the original std::string. It also store the string pointer in a static variable, that's why you have to call it with a null pointer the next times to continue to parse the same string.

By the way, in c++, you should use std::istringstream instead. It does not use an internal static variable, which is not thread-safe. And you can extract the parameters directly into int, double, etc like we do with cin. std::ostringstring replace sprintf().

c++ strtok in function changes original string value as parameter

3 Answers3

Linked