Parsing a string into different tokens

Question

How does one parse a single string such as "ducks hollow23 !" into three different tokens. This string is read into a single string called input and needs to be checked if it is valid. Basically, the function is similar to this:

int inputString(string name, string keyWord, bool trueFalse){
     string input;
     cin >> input;

     // input = "ducks hollow23 !"

     // how to I put "ducks hollow23 !" into name, keyWord, trueFalse and then 
     // check if name is a valid name and if keyWord is valid or if not needed, and trueFalse
     // is valid

}

You have to use a lexer. You can either use your own or generated one: https://theantlrguy.atlassian.net/wiki/display/ANTLR3/Code+Generation+Targets — ibre5041, Sep 25 '14 at 18:05
I just want to parse "ducks" into name, "hollow23" into keyWord, ! into trueFalse — Bill Cunningham, Sep 25 '14 at 18:14
use strtok to tokenize, as in http://stackoverflow.com/questions/3889992/how-does-strtok-split-the-string-into-tokens-in-c — Dr. Debasish Jana, Sep 25 '14 at 18:23

score 0 · Answer 1 · answered Sep 25 '14 at 18:29

An example code snippet given below:

char str[] = input.c_str(); // e.g. "ducks hollow23 !"
char * pch;
pch = strtok (str," ");
string str[3];
int i = 0;
while (pch != NULL) {
  if (i > 2)
    break; // no more parsing, we are done
  str[i++] = pch;
  pch = strtok(NULL, " ");
}
// str[0] = "ducks", str[1] = "hollow23", str[2] = "!"

score 0 · Answer 2 · answered Sep 25 '14 at 18:34

your example is not fair - string will be just "ducks" in your setting, which you can interpret as name other >> will give you more tokens. Also you need references & to get values back

int inputString(string& name, string& keyWord, bool& trueFalse) {
    string flag;
    cin >> input >> keyWord >> flag;
    trueFalse = (flag == "!");
}

score 0 · Answer 3 · answered Sep 25 '14 at 18:37

The easiest way is probably to use istringstream.

I'm not sure what you consider valid input so the only error checking I've used is that the istringstream is in a good state.

I've modified inputString() to take the full input string, which you would get from cin.

#include <iostream>
#include <sstream> // for std::istringstream
using namespace std;

// Note call by reference for the three last parameters
// so you get the modified values
int inputString(string input, string &name, string &keyWord, bool &trueFalse){
    std::istringstream iss(input); // put input into stringstream

    // String for third token (the bool)
    string boolString;

    iss >> name; // first token

    // Check for error (iss evaluates to false)
    if (!iss) return -1;

    iss >> keyWord; // second token

    // Check for error (iss evaluates to false)
    if (!iss) return -1;

    iss >> boolString; // third token

    // Check for error (iss evaluates to false)
    if (!iss) return -1;

    if (boolString == "!") trueFalse = false;
    else trueFalse = true;

    return 0;
}

int main() {
    string input, name, keyWord;
    bool trueFalse;
    //cin << input;

    // For this example I'll just input the string
    // directly into the source
    input = "ducks hollow23 !";

    int result = inputString(input, name, keyWord, trueFalse);

    // Print results
    cout << "name = " << name << endl;
    cout << "keyWord = " << keyWord << endl;

    // Use std::boolalpha to print "true" or "false"
    // instead of "0" or "1"
    cout << "bool result = " << boolalpha << trueFalse << endl;

    return result;
}

The output is

name = ducks
keyWord = hollow23
bool result = false

Parsing a string into different tokens

3 Answers3