Efficient parsing of std::stringstream data into std::vector>

Question

I am parsing a data stream of pairs of double values into std::vector>. I use boost as I think it is more efficient. My code is as follow.

                std::stringstream tmp_stream;                
                typedef double data_type;            
                typedef ::std::vector < data_type > V_d;
                // type below describes type of the container of all data
                typedef ::std::vector < V_d > V_v_d;
                // list container 
                //typedef ::std::list < V_d > V_v_d;

                V_v_d data;

                ::data_parser::Data_parser < V_v_d > data_parser;
                data_parser ( tmp_stream, data );

My input text file is formatted as { (132.181,0.683431), (136.886,0.988517), (137.316,0.504297), (133.653,0.602269), (150.86,0.236839) } The pairs are not parsed correctly and I received empty pair. What could be the problem? Thanks

Not sure it relates to the question, but if you will always be working with pairs of doubles, it would probably be much more efficient to use `std::vector >` for the storage. — icabod, Feb 17 '14 at 09:24
Not sure what code is doing the string -> float conversion, but I recommend using the Intel Numeric String Conversion library, if you're on Intel: http://software.intel.com/en-us/node/485147 — Avi Tevet, Feb 17 '14 at 09:24
Are you skipping all the other stuff like `(,{})`? If you use an `istringstream` and try to extract those characters into a `double`, weird stuff may happen. Also, do as the previous commenters suggested and use a `std::pair` to store the pairs. — JorenHeit, Feb 17 '14 at 09:40

sehe · Accepted Answer · 2014-02-17T11:25:39.527

Using a bit of Boost Spirit, you can use a single line:

if (tmp_stream >> std::noskipws >> 
       qi::phrase_match((+qi::double_) % qi::eol, qi::blank, data))
{

and another single line to display the results as a bonus:

    std::cout << karma::format((karma::double_ % ", ") % karma::eol, data) << "\n";
}

Note It handles inf, -inf, nan :)

See it Live on Coliru:

#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/karma.hpp>
#include <boost/spirit/include/qi_match.hpp>

namespace qi = boost::spirit::qi;
namespace karma = boost::spirit::karma;

int main()
{
    std::vector<std::vector<double>> data;

    std::stringstream tmp_stream(
            "123 45 inf -inf nan .7e-99\n"
            "42\n"
            "-1");

    if (tmp_stream >> std::noskipws >> qi::phrase_match((+qi::double_) % qi::eol, qi::blank, data))
    {
        std::cout << karma::format((karma::double_ % ", ") % karma::eol, data) << "\n";
    }
}

UPDATE Slightly adapted to your input grammar, I'd suggest parsing into a vector of pairs instead:

#include <boost/fusion/adapted/std_pair.hpp> // handle std::pair
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/karma.hpp>
#include <boost/spirit/include/qi_match.hpp>

namespace qi = boost::spirit::qi;
namespace karma = boost::spirit::karma;

int main()
{
    std::vector<std::pair<double, double>> data;

    std::stringstream tmp_stream("{ (132.181,0.683431), (136.886,0.988517), (137.316,0.504297), (133.653,0.602269), (150.86,0.236839) }");

    if (tmp_stream >> std::noskipws >> qi::phrase_match(
                   '{' 
                >> 
                     ( '(' >> qi::double_ >> "," >> qi::double_ >> ')' )
                     % ','
                >> '}',
                qi::space, data))
    {
        std::cout << karma::format(karma::delimit(" ") [karma::auto_] % karma::eol, data) << "\n";
    }
}

Which prints: (see Live On Coliru

132.181 0.683 
136.886 0.989 
137.316 0.504 
133.653 0.602 
150.86 0.237

Note To make things quicker, consider not parsing form a stream, but using

streambuf_iterator
the underlying byte data if you have it in memory anyways

About the performance of the above: [How to parse space-separated floats in C++ quickly?](http://stackoverflow.com/questions/17465061/how-to-parse-space-separated-floats-in-c-quickly/17479702#17479702) contains benchmarks — sehe, Feb 17 '14 at 10:20

Efficient parsing of std::stringstream data into std::vector>

1 Answers1