Splitting Boost.Spirit.X3 parsers into several TUs

Question

I'm struggling with Boost.Spirit.X3 again.

I have several logical groups of parsers (statements, expressions, etc.), each of which is represented by several files:

• group.hpp - contains typedefs, BOOST_SPIRIT_DECLARE and extern variable declaration for those parsers that are used "outside"
• group_def.hpp - includes the previous one and contains actual definitions of parsers, BOOST_SPIRIT_DEFINE, etc.
• group.cpp - includes the previous one and contains explicit template instantiations (via BOOST_SPIRIT_INSTANTIATE)

Basically, it more or less follows the structure proposed in the official tutorial. The only difference is that my grammar is much more complicated so I'm trying to split it into several translation units. All these TUs are then compiled into one library which, in turn, is linked then to the main executable.

I've tried to make a "minimal" example here (live on Wandbox) as it would be inconvenient to list all the files here. It doesn't work though because of some unresolved externals, so, most probably, I instantiate something incorrectly, but I've already spent about a week for this, so I'm quite desperate and don't feel like I'm able to handle this on my own.

A few questions and answers:

Why do I prefer using three files per "group"?

First, because I tried to make it in such a way that I don't want to recompile everything on any small change (not sure if I succeeded in this), so the idea is that somegroup.hpp is just a "lightweight" header with declarations, somegroup_def.hpp is a header with definitions, and somegroup.cpp is just used in order to create a translation unit.

Second, I split _def.hpp and .cpp because I also include these _def.hpp-files directly to tests where I cover not only extern parsers but also "internal" auxiliary ones.

Why am I using extern variables?

I tried it also with functions that return parsers instead (similar to how it is done in the tutorial). Basically, this is the way how it is implemented and does work now. I don't like it because, for instance, given a parser lang::parser::import, I have to either give a function another name (lang::parser::import_) or place it within another namespace (i.e. lang::import). Also, I like the way to use parsers directly how it is done in the Spirit itself (i.e. without parentheses: import vs import_()).

My actual questions are the following:

• How to properly organize the structure if I want to spread my parsers over several translation units?
• And what exactly am I missing in the example of code above, so that it doesn't link?

I would appreciate any help.

Answer 1

Why do I prefer using three files per "group"?

I am, myself, find separation into grammar_def.hpp and grammar.cpp useless, but that is just an opinion.

Why am I using extern variables?

I tried it also with functions that return parsers instead

Do not do this. It will lead to static initialization order fiasco. Rule placeholders are lightweight enough to instantiate them in every translation unit.

How to properly organize the structure if I want to spread my parsers over several translation units?

That is a question about tastes. All you need is to instantiate a rule in one of the TU and have a x3::rule<...>+BOOST_SPIRIT_DECLARE in every other that uses it.

The best way to achieve that is to split x3::rule<...> off .cpp/_def.hpp into separate header (place it into your "lightweight" .hpp), and include it in every TU that needs those rules.

See https://github.com/mapnik/mapnik/pull/4072/files and https://github.com/boostorg/spirit/pull/493/files

And what exactly am I missing in the example of code above, so that it doesn't link?

1. You are mixing std::string::const_iterator and std::string::iterator iterators.
2. You are using some of your skippers as parsers (f.ex. document_def = eols >> +megarule >> eols) but do not instantiate them with a proper context. Either simply do not make them rules or add BOOST_SPIRIT_INSTANTIATE with a context that you see in the error message.