Boost Spirit X3 cannot compile repeat directive with variable factor

Question

I am trying to use the Boost Spirit X3 directive repeat with a repetition factor that is variable. The basic idea is that of a header + payload, where the header specifies the size of the payload. A simple example “3 1 2 3” is interpreted as header = 3, data= {1, 2, 3} (3 integers).

I could only find examples from the spirit qi documentation. It uses boost phoenix reference to wrap the variable factor: http://www.boost.org/doc/libs/1_50_0/libs/spirit/doc/html/spirit/qi/reference/directive/repeat.html

std::string str;
int n;
test_parser_attr("\x0bHello World",
    char_[phx::ref(n) = _1] >> repeat(phx::ref(n))[char_], str);
std::cout << n << ',' << str << std::endl;  // will print "11,Hello World"

I wrote the following simple example for spirit x3 without luck:

#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <string>
#include <iostream>

namespace x3 = boost::spirit::x3;
using x3::uint_;
using x3::int_;
using x3::phrase_parse;
using x3::repeat;
using x3::space;
using std::string;
using std::cout;
using std::endl;

int main( int argc, char **argv )
{
  string data("3 1 2 3");
  string::iterator begin = data.begin();
  string::iterator end = data.end();

  unsigned int n = 0;

  auto f = [&n]( auto &ctx ) { n = x3::_attr(ctx); };
  bool r = phrase_parse( begin, end, uint_[f] >> repeat(boost::phoenix::ref(n))[int_], space );
  if ( r && begin == end  )
    cout << "Parse success!" << endl; 
  else
    cout << "Parse failed, remaining: " << string(begin,end) << endl;

  return 0;
}

Compiling the code above with boost 1.59.0 and clang++ (flags: -std=c++14) gives the following:

boost_1_59_0/boost/spirit/home/x3/directive/repeat.hpp:72:47: error: no matching constructor for

      initialization of 'proto_child0' (aka 'boost::reference_wrapper<unsigned int>')

            typename RepeatCountLimit::type i{};

If I hardcode repeat(3) instead of repeat(boost::phoenix::ref(n)) it works properly, but it is not a possible solution since it should support a variable repetition factor.

Compilation with repeat(n) completes successfully, but it fails parsing with the following output: “Parse failed, remaining: 1 2 3"

Looking at the source code for boost/spirit/home/x3/directive/repeat.hpp:72 it calls the empty constructor for template type RepeatCountLimit::type variable i and then assign during the for loop, iterating over min and max. However since the type is a reference it should be initialized in the constructor, so compilation fails. Looking at the equivalent source code from the previous library version boost/spirit/home/qi/directive/repeat.hpp:162 it is assigned directly:

        typename LoopIter::type i = iter.start();

I am not sure what I am doing wrong here, or if x3 currently does not support variable repetition factors. I would appreciate some help solving this issue. Thank you.

Answer 1

From what I gather, reading the source and the mailing list, Phoenix is not integrated into X3 at all: the reason being that c++14 makes most of it obsolete.

I agree that this leaves a few spots where Qi used to have elegant solutions, e.g. eps(DEFERRED_CONDITION), lazy(*RULE_PTR) (the Nabialek trick), and indeed, this case.

Spirit X3 is still in development, so we might see this added¹

For now, Spirit X3 has one generalized facility for stateful context. This essentially replaces locals<>, in some cases inherited arguments, and can be /made to/ validate the number of elements in this particular case as well:

• x3::with²

Here's how you could use it:

with<_n>(std::ref(n)) 
    [ omit[uint_[number] ] >> 
    *(eps [more] >> int_) >> eps [done] ]

Here, _n is a tag type that identifies the context element for retrieval with get<_n>(cxtx).

Note, currently we have to use a reference-wrapper to an lvalue n because with<_n>(0u) would result in constant element inside the context. I suppose this, too, is a QoI that may be lifted as X# matures

Now, for the semantic actions:

unsigned n;
struct _n{};

auto number = [](auto &ctx) { get<_n>(ctx).get() = _attr(ctx); };

This stores the parsed unsigned number into the context. (In fact, due to the ref(n) binding it's not actually part of the context for now, as mentioned)

auto more   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) >  _val(ctx).size(); };

Here we check that we're actually not "full" - i.e. more integers are allowed

auto done   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) == _val(ctx).size(); };

Here we check that we're "full" - i.e. no more integers are allowed.

Putting it all together:

Live On Coliru

#include <string>
#include <iostream>
#include <iomanip>

#include <boost/spirit/home/x3.hpp>

int main() {
    for (std::string const input : { 
            "3 1 2 3", // correct
            "4 1 2 3", // too few
            "2 1 2 3", // too many
            // 
            "   3 1 2 3   ",
        })
    {
        std::cout << "\nParsing " << std::left << std::setw(20) << ("'" + input + "':");

        std::vector<int> v;

        bool ok;
        {
            using namespace boost::spirit::x3;

            unsigned n;
            struct _n{};

            auto number = [](auto &ctx) { get<_n>(ctx).get() = _attr(ctx); };
            auto more   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) >  _val(ctx).size(); };
            auto done   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) == _val(ctx).size(); };

            auto r = rule<struct _r, std::vector<int> > {} 
                  %= with<_n>(std::ref(n)) 
                        [ omit[uint_[number] ] >> *(eps [more] >> int_) >> eps [done] ];

            ok = phrase_parse(input.begin(), input.end(), r >> eoi, space, v);
        }

        if (ok) {
            std::copy(v.begin(), v.end(), std::ostream_iterator<int>(std::cout << v.size() << " elements: ", " "));
        } else {
            std::cout << "Parse failed";
        }
    }
}

Which prints:

Parsing '3 1 2 3':          3 elements: 1 2 3 
Parsing '4 1 2 3':          Parse failed
Parsing '2 1 2 3':          Parse failed
Parsing '   3 1 2 3   ':    3 elements: 1 2 3 

---

¹ lend your support/voice at the [spirit-general] mailing list :)

² can't find a suitable documentation link, but it's used in some of the samples