Segmentation Fault when using strtok_r

Question

Can anyone explain why I am getting segmentation fault in the following example?

#include <stdio.h>
#include <string.h>

int main(void) {
  char *hello = "Hello World, Let me live.";
  char *tokens[50];
  strtok_r(hello, " ,", tokens);
  int i = 0;
  while(i < 5) {
    printf("%s\n", tokens[i++]);
  }
}

Answer 1

Try this:

#include <stdio.h>
#include <string.h>

int main(void) {
        char hello[] = "Hello World, Let me live."; // make this a char array not a pointer to literal.
        char *rest; // to point to the rest of the string after token extraction.
        char *token; // to point to the actual token returned.
        char *ptr = hello; // make q point to start of hello.

        // loop till strtok_r returns NULL.
        while(token = strtok_r(ptr, " ,", &rest)) {

                printf("%s\n", token); // print the token returned.
                ptr = rest; // rest contains the left over part..assign it to ptr...and start tokenizing again.    
        }
}
/*
Output:
Hello
World
Let
me
live.
*/

Answer 2

• You need to call strtok_r in a loop. The first time you give it the string to be tokenized, then you give it NULL as the first parameter.
• strtok_r takes a char ** as the third parameter. tokens is an array of 50 char * values. When you pass tokens to strtok_r(), what gets passed is a char ** value that points to the first element of that array. This is okay, but you are wasting 49 of the values that are not used at all. You should have char *last; and use &last as the third parameter to strtok_r().
• strtok_r() modifies its first argument, so you can't pass it something that can't be modified. String literals in C are read-only, so you need something that can be modified: char hello[] = "Hello World, Let me live."; for example.

Answer 3

A bunch of things wrong:

1. hello points to a string literal, which must be treated as immutable. (It could live in read-only memory.) Since strtok_r mutates its argument string, you can't use hello with it.

2. You call strtok_r only once and don't initialize your tokens array to point to anything.

Try this:

#include <stdio.h>
#include <string.h>

int main(void) {
  char hello[] = "Hello World, Let me live.";
  char *p = hello;
  char *tokens[50];
  int i = 0;

  while (i < 50) {
     tokens[i] = strtok_r(p, " ,", &p);
     if (tokens[i] == NULL) {
        break;
     }
     i++;
  }

  i = 0;
  while (i < 5) {
    printf("%s\n", tokens[i++]);
  }

  return 0;
}

Answer 4

strtok_r tries to write null characters into hello (which is illegal because it is a const string)

Answer 5

You have understood the usage of strtok_r incorrectly. Please check this example and documentation

And try & see this:

#include <stdio.h>
#include <string.h>    

int main(void)
{
    char hello[] = "Hello World, let me live.";

    char *tmp;
    char *token = NULL;
    for(token = strtok_r(hello, ", ", &tmp);
        token != NULL;
        token = strtok_r(NULL, ", ", &tmp))
    {
        printf("%s\n", token);
    }

    return 0;
}

Answer 6

I think it might be the char *tokens[50]; because you are declaring it a pointer when it is already a pointer. An array is already a pointer upon declaration. You mean to say char tokens[50];. That should do the trick.