1
#include <stdio.h>
#include <string.h> /* needed for strtok */
#include <unistd.h>
#include <stdlib.h>

int main(int argc, char **argv) {
        char text[10000];
    fgets(text, sizeof(text), stdin);
    char *t;
    int i;

    t = strtok(text, "\"\'| ");
    for (i=0; t != NULL; i++) {
        printf("token %d is \"%s\"\n", i, t);
        t = strtok(NULL, "\"\'| ");
    }
}

This is part of the code that im trying to make it is supposed to separate tokens

Let's say the input is 'abc' "de f'g" hij| k "lm | no"

The output should be

token 1: "abc"
token 2: "de f'g"
token 3: "hij"
token 4: "|"
token 5: "k"
token 6: "lm | no"

I get something different but close anyway I can change it to this format?

Wasif Hossain
  • 3,900
  • 1
  • 18
  • 20
lumpsidemash
  • 11
  • 2

2 Answers2

0

What you're trying to do is essentially a parser. strtok isn't a very good tool for this, and you may have better luck writing your own. strtok works on the presumption that whatever delimits your tokens is unimportant and so can be overwritten with '\0'. But you DO care what the delimiter is.

The only problem you'll have is that | syntax. The fact that you want to use it as a token delimiter and a token is likely to make your code more complicated (but not too much). Here, you have the issue that hij is followed immediately by |. If you terminate hij to get the token, you will have to overwrite the |. You either have to store the overwritten character and restore it, or copy the string out somewhere else.

You basically have three cases:

  • The bar | is a special delimiter that is also a token;
  • Quoted delimiters " and ' match everything until the next quote of the same kind;
  • Otherwise, tokens are delimited by whitespace.
paddy
  • 60,864
  • 6
  • 61
  • 103
  • What if I HAVE TO use strtok, then how do I solve this – lumpsidemash Feb 20 '14 at 02:52
  • @lumpsidemash Well then it's a bit like trying to chop an onion when you only have a spoon. You can do it of course, except for that pesky `|` token. Are you *absolutely certain* that it's a token as well as a delimiter? The idea is that you just change the delimiters provided to `strtok`. For example, if your current token starts with quotes, then you use only that character as the delimiter. Otherwise you use whitespace and the `|` only. – paddy Feb 20 '14 at 03:26
  • One could change the delimiters on every call to `strtok`, but I can't see a pattern that would help the OP. – Joseph Quinsey Feb 20 '14 at 06:56
  • @JosephQuinsey As I mentioned in my question, and then made clearer in the comment, there are three cases. The main search pattern is `" |"`. But if the first character is a single or double quote, then the pattern is just that character. You need to search at the next character of course. And the `|` is a special case. Beyond that, there's the memo issue, which is not easily dealt with when using `strtok`. Since you have to search the string anyway, it makes `strtok` redundant. It's easy to write something and post it, but that won't really help them if this is a homework exercise. – paddy Feb 20 '14 at 07:58
0
#include <stdio.h>
#include <string.h>

char *getToken(char **sp){
    static const char *sep = " \t\n";
    static char vb[] = "|", vbf;
    char *p, *s;
    if(vbf){
        vbf = 0;
        return vb;
    }
    if (sp == NULL || *sp == NULL || **sp == '\0') return(NULL);
    s = *sp;
    if(*s == '"')
        p = strchr(++s, '"');
    else if(*s == '\'')
        p = strchr(++s, '\'');
    else
        p = s + strcspn(s, "| \t\n");
    if(*p != '\0'){
        if(*p == '|'){
            *vb = vbf = '|';
        }
        *p++ = '\0';
        p += strspn(p, sep);
    }
    *sp = p;
    if(!*s){
        vbf = 0;
        return vb;
    }
    return s;
}

int main(int argc, char **argv) {
    char text[10000];
    fgets(text, sizeof(text), stdin);
    char *t, *p = text;
    int i;

    t = getToken(&p);
    for (i=1; t != NULL; i++) {
        printf("token %d is \"%s\"\n", i, t);
        t = getToken(&p);
    }
    return 0;
}
BLUEPIXY
  • 39,699
  • 7
  • 33
  • 70