issue with strtok to compare two words from a nested results of strtok function

Question

I'm looking for comparing words from an array with words in the dictionary from another array to look for the maximum number of words found

I used strtok since the words in both are delimited with spaces, but it's not working. I need your help please

 void chercherScoreMotDansDico(char msgBootforce [], int* 
 maxCorrepondance, char* mot, char* dicoActuel, char* 
 bonResultatBootforce) {
     int i = 0;
     char* motdico = NULL;
     char tmpMsgBootForce [3000] = {0};

     strcpy(tmpMsgBootForce, msgBootforce);

     mot = strtok (tmpMsgBootForce, " ");

     while (mot != NULL) {
          motdico = strtok (dicoActuel, " ");

          while (motdico != NULL) {
              if (strcmp(mot,motdico) == 0)   ++i;
              motdico = strtok (NULL, " ");
          }

          mot = strtok (NULL," ");  
    }

    if (i > *(maxCorrepondance)) {
        *(maxCorrepondance) = i;
        strcat(bonResultatBootforce, msgBootforce);
    }   
 }

Answer 1

You can't have two uses of strtok() on two different strings being done at the same time.; strtok() has an internal pointer where it stores the address of the current string being processed. If you call strtok() with a string and then call strtok() with a different string then when you do strtok(NULL, delim) it will continue with the last string that was specified.

See https://en.cppreference.com/w/c/string/byte/strtok

This function is destructive: it writes the '\0' characters in the elements of the string str. In particular, a string literal cannot be used as the first argument of strtok. Each call to strtok modifies a static variable: is not thread safe. Unlike most other tokenizers, the delimiters in strtok can be different for each subsequent token, and can even depend on the contents of the previous tokens. The strtok_s function differs from the POSIX strtok_r function by guarding against storing outside of the string being tokenized, and by checking runtime constraints.

There is a new version of the strtok() function strtok_s() which has an additional argument of an address for a pointer variable to use instead of the internal pointer variable that strtok() uses.

Answer 2

You can't use strtok with two different strings at the same time.

strtok(string, delim) stores its position in string internally for future calls to strtok (NULL, delim). It can only remember one at a time. strtok (tmpMsgBootForce, " ") says to look through tmpMsgBootForce and then motdico = strtok (dicoActuel, " ") overwrites that with dicoActuel.

---

What to use instead depends on your compiler. The C standard defines strtok_s, but that's from the 2011 standard and has proven to be controversial. POSIX defines strtok_r, most Unix compilers will understand that. Finally, Visual Studio has their own slightly different strtok_s.

They all work basically the same way. You manually store the position in each string you're iterating through.

Here it is using strtok_r. next_tmpMsgBootforce and next_dicoActuel hold the position for parsing tmpMsgBootForce and dicoActuel respectively.

 char *next_tmpMsgBootforce;
 char *next_dicoActuel;

 strcpy(tmpMsgBootForce, msgBootforce);

 mot = strtok_r(tmpMsgBootForce, " ", &next_tmpMsgBootforce);

 while (mot != NULL) {
      motdico = strtok_r(dicoActuel, " ", &next_dicoActuel);

      while (motdico != NULL) {
          if (strcmp(mot,motdico) == 0)   ++i;
          motdico = strtok_r(NULL, " ", &next_dicoActuel);
      }

      mot = strtok_r(NULL," ", &next_tmpMsgBootforce);  
}

Because this is all such a mess, I recommend using a library such as GLib to smooth out these incompatibilities and unsafe functions.

---

As a side note, the strcpy and strcat are not safe. If their destination does not have enough space it will try to write outside its memory bounds. As with strtok the situation to do this safely is a mess. There's the non-standard but ubiquitous strlcpy and strlcat. There's the standard but not ubiquitous strcpy_s and strcat_s. Thankfully for once Visual Studio follows the standard.

On POSIX systems you can use strdup to duplicate a string. It will handle the memory allocation for you.

char *tmpMsgBootForce = strdup(msgBootForce);

The caveat is you have to free this memory at the end of the function.

Doing a strcat safely gets complicated. Let's simplify this by splitting it into two functions. One to do the searching.

int theSearching(
    const char *msgBootforce,
    const char *dicoActuel
) {
     int i = 0;
     char *next_tmpMsgBootforce;
     char *next_dicoActuel;

     char *tmpMsgBootForce = strdup(msgBootforce);
     char *tmpDicoActuel = strdup(dicoActuel);

     char *mot = strtok_r(tmpMsgBootForce, " ", &next_tmpMsgBootforce);

     while (mot != NULL) {
          char *motdico = strtok_r(tmpDicoActuel, " ", &next_dicoActuel);

          while (motdico != NULL) {
              if (strcmp(mot,motdico) == 0) {
                  ++i;
              }
              motdico = strtok_r(NULL, " ", &next_dicoActuel);
          }

          mot = strtok_r(NULL," ", &next_tmpMsgBootforce);  
    }

    return i;
}

And one to do the appending. This function ensures there's enough space for the concatenation.

char *tryAppend( char *dest, const char *src, int *maxCorrepondance, const int numFound ) {
    char *new_dest = dest;

    if (numFound > *maxCorrepondance) {
        *(maxCorrepondance) = numFound;

        // Allocate enough memory for the concatenation.
        // Don't forget space for the null byte.
        new_dest = realloc( dest, strlen(dest) + strlen(src) + 1 );
        strcat( new_dest, src);
    }

    // Return a pointer to the reallocated memory,
    // or just the old one if no reallocation was necessary.   
    return new_dest;
}

Then use them together.

int numFound = theSearching(msgBootforce, dicoActuel);
bonResultatBootforce = tryAppend(bonResultatBootforce, msgBootforce, &maxCorrepondance, numFound);