How can i parse whitespace, equal and quotes to vector in C++?

Question

Is there any I could parse space and quotes to vector? like for example:

#include <iostream>
#include <vector>
#include <string>
using namespace std;
void ParseFunc(string str, vector<string>& output)
{
    //How can I do it?
}
int main()
{
    string command = "hello world=\"i love coding\" abc=123 end";
    vector<string> str;
    ParseFunc(command, str);
    for(int i = 0; i != str.size(); i++)
    {
        cout << str[i] << endl;
    }
}

and i want the output be:

hello
world
=
i love coding
abc
=
123

Please help me!

Does this answer your question? [Parse (split) a string in C++ using string delimiter (standard C++)](https://stackoverflow.com/questions/14265581/parse-split-a-string-in-c-using-string-delimiter-standard-c) — polfosol ఠ_ఠ, Aug 24 '21 at 09:35
The way I'd do it would be to define the grammar associated with your command. Then build a parser for it. See Boost Spirit (www.boost.org). The learning curve for that is gigantic however. Allow anything up to a year if you're a beginner. — Bathsheba, Aug 24 '21 at 10:04
Can you explain the tokenization rules (since that's what you're doing) in English? — Botje, Aug 24 '21 at 10:09
@polfosolఠ_ఠ sorry it doesnt answer, because it only parse space, but i want to parse quotes too! — Atomic, Aug 24 '21 at 10:34
hello @Botje, i already explained the result i want in question. I want word in quotes add in vector's 1 array and not 3 array(bcuz there's 2 space in the quote, so if I just parse space it will turn into 3 array, but I want it in 1 array since its quoted.) — Atomic, Aug 24 '21 at 10:35
Okay, do you want to support escaped quotes? eg `"foo \" bar"` ? The answer will determine the difficulty of writing a tokenizer. Also, what about `"foo=bar"`? Is that a single string or `"foo`, `=`, `bar"`? And what about `foo"thing with spaces"`? These details matter, which is why I asked you to write a formal specification instead of a handwavy explanation. — Botje, Aug 24 '21 at 10:48
Alternatively, from your *current specification*: `output = { "Hello", "world", "=", "i love coding", "abc", "=", "123" };` — Caleth, Aug 24 '21 at 11:06
yea @Caleth thats what i want, i want all quote work the same way and u specified. Double quote, single quote — Atomic, Aug 24 '21 at 11:07
I mean [literally that](https://coliru.stacked-crooked.com/a/1e91921228d7bdde) in place of `//How can I do it?` — Caleth, Aug 24 '21 at 11:07

Botje · Accepted Answer · 2021-08-24T11:33:28.980

This is trivial with std::regex_iterator:

string command = "hello world=\"i love coding\" abc=123 end";
std::regex words_regex(R"([^ ="]+|"[^"]+"|=)");
for (auto it = std::sregex_iterator(command.begin(), command.end(), words_regex);
        it != std::sregex_iterator();
        it++) {
    std::string token = (*it).str();
    if (token == "end") {
        break;
    }
    if (token[0] == '"')
        token = token.substr(1, token.length()-2);
    cout << "Token: " << token << endl;
}

Output:

Token: hello
Token: world
Token: =
Token: i love coding
Token: abc
Token: =
Token: 123

score 0 · Answer 2 · answered Aug 24 '21 at 12:19

Even more simple is:

#include <iostream>
#include <string>
#include <regex>
#include <vector>
#include <algorithm>
#include <iterator>

const std::string com = "hello world=\"i love coding\" abc=123 end";
const std::regex re(R"(\"(.*?)\"|(\w+)|([=]))");

int main() {

    // Here we will store the results of the splitting
    std::vector<std::string> result{};

    // Split
    std::transform(std::sregex_token_iterator(com.begin(), com.end(), re), {}, std::back_inserter(result),
        [](std::string s) { return std::regex_replace(s, std::regex("\""), ""); });

    // Output
    std::copy(result.begin(), result.end(), std::ostream_iterator<std::string>(std::cout, "\n"));

    return 0;
}

A typical one statement with Lambda approach ...

How can i parse whitespace, equal and quotes to vector in C++?

2 Answers2