Tokenizing string and put tokens in variables of different data types?

Question

i want to tokenize a string e.g. "5 + 3 * 2 - 4 " everytime when a " " (space) appears. the operator signs shall be saved in different named char variables. the numbers in different named float variables. e.g.

float num_1 = 5;
float num_2 = 3;
float num_3 = 2;
float num_4 = 4;

char op_1 = '+';
char op_2 = '*';
char op_3 = '-';

this is my tokenizer so far. but this can only print the numbers and operatos in console. thats not what i want.

std::vector<std::string> strings;
    std::istringstream f("3 + 2 * 4 ");
    std::string s;
    while (std::getline(f, s, ' ')) {
        std::cout << s << std::endl;
        strings.push_back(s);
    }

i googled this stuff for an eternity, but i never found something helpful to me. im just a beginner and most of this code i googled was too hard for me to actually understand or it was just not what i need :/

Here are good information about tokenizing: http://stackoverflow.com/questions/53849/how-do-i-tokenize-a-string-in-c . I suggest thinking backwards from the result: what is it that you want to achieve with this? What data structure would be necessary for that? And then how can you compute that data from your input. — Gerriet, May 10 '17 at 13:37
The only way to store each character and integer in a different variable is declaring each variable by hand and assign a value to it. If the expression you are trying to tokenize grows bigger, you'll need more variables. Consider using `array` or `vector` to store your values instead. — alejandrogiron, May 10 '17 at 13:41

score 0 · Answer 1 · answered May 10 '17 at 13:50

If I guess right what you need is to have a variable which stores s in strings at each iteration. Your problem is that getline() takes the parameter delimitor which you set at ' 'and stops at it. So for example :

s = '3'; // First iteration

And then the while stops because f stores only one line. You should consider using strtok_r() or iterating char by char and storing them into a string until you encounter ' '

AlexG · Accepted Answer · 2017-05-10T14:36:15.133

It seems you have most of the job done. What you're missing is a way to know whether the substring s is an operator or a number. You can easily code helper functions for that. Here's a modified version of your code that should work just well. It's certainly not bulletproof as far as input goes, but it should get you going.

// Checks if the string is an operator
bool IsOperator(std::string const& str)
    {
    if (1 != str.size())
        return false;

    if (str[0] == '*' || str[0] == '-' || str[0] == '+' || str[0] == '/')
        return true;

    return false;
    }

// Converts the string into a float
float ParseValue(std::string const& str)
    {
    return stof(str);
    }

int main()
    {
    float* = n
    std::vector<float> values;
    std::vector<char> operators;

    std::istringstream f("3 + 2 * 4 ");
    std::string s;
    while (std::getline(f, s, ' '))
        {
        if (IsOperator(s))
            operators.push_back(s[0]);
        else
            values.push_back(ParseValue(s));
        }

    return 0;
    }

EDIT -- with arrays.

int main()
    {
    float values[100];
    char operators[100];

    size_t valuesIdx = 0;
    size_t operatorsIdx = 0;

   std::istringstream f("3 + 2 * 4 ");
    std::string s;
    while (std::getline(f, s, ' '))
        {
        if (IsOperator(s))
            {
            operators[operatorsIdx] = s[0];
            ++operatorsIdx;
            }
        else
            {
            values[valuesIdx] = ParseValue(s);
            ++valuesIdx;
            }
        }

    return 0;
    }

thx for this answer. i tried the code and printed the content of the vector to proof if its working. and yes it does *thumbs up but what if i want to use an array instead. is this also possible? — , May 10 '17 at 14:16
@neo1t7 it sure is. You could just keep indexes to know where you're at inside your array (i've added an edit in my example to show you). A vector is more convenient because it resizes itself automatically if it doesn't have enough space.. — AlexG, May 10 '17 at 14:32

Tokenizing string and put tokens in variables of different data types?

2 Answers2