Create a compile time key-to-type map which is filled by calls to a variadic function

Question

Is it possible to create a key->type map at compile time, with each key-value being added when an instance of a variadic function is called?

template <typename T, typename ... Args>
void writeToQueue(Args... args) {
    //Do something with args.
    // Insert to map. something akin to:
    // CODEMAP[T] = Args...
    // T -> Args... mapped when foo<T,Args...> is called.
}

Or

template <int code, typename ... Args>
void writeToQueue(Args... args) {
    //Do something with args.
    // Insert to map. something akin to:
    // CODEMAP[code] = Args...
    // code -> Args... mapped when foo<code,Args...> is called.
}

In the above, the requirement is have a CODEMAP that maps either type->type or int->type (whichever is feasible) but the map is populated when the function foo is called, so that it is not a requirement beforehand to know code and args.

Is this at all possible? Either through boost/preprocessor/template programming?

Edit: CODEMAP is as stated, a map to store code -> type information. During runtime, a reader block (say R) would read the messages stored/processed by foo() and parse based on the code at the beginning of a message. the code is always fixed size (4 chars or 1 int).

It's the same translation unit.

Edit: So here's the deal:

Producer: writes data to FIFO queue (critical code hot path) -> Consumer thread reads and process the info from the queue.

A pseudo code is below:

Producer:

void Producer::run() {
    // This guy shouldn't worry about the type of data being written.
    // So, encapsulating data into structs and passing it to queue is
    // out of question.
    writeToQueue<Code1>(1,2,"123123",'a', 3.1416);
    writeToQueue<Code2>(4,1,'b');

    template <int Code, typename ...Args>
    void writeToQueue(Args... args) {
     queue.insert(args...);
     // Need code to args... mapping. So, decided to create static
     // instantiation of a formatspecifier class.
     static formatspecifier<Code, args...> f{};
    }

    // To encode the type information to be used in run time.
    template <int Code, typename ... Args>
    class formatspecifier{
       formatspecifier() {
          global::codemap[Code] = encodeTypeInfo<Args...>();
       }
    };
}

Consumer:

void Consumer::readfromQueue() {
    while(true) {
      if (queue.dataAvailable()) {
        const auto code = queue.getCode();
        // get encoded type info format from global::codemap map.
        const auto fmt = global::codemap[code];
        for (int i=0; i < fmt.len; i++) {
           // I am unsure how this part should look.
           process<fmt[0]::type>(queue.getData<fmt[0]::type>());
        } 
      }  
    }
 }

What's your goal for CODEMAP? Please provide a usage example. Also how would you deal with multiple translation units (do you want one single codemap for the entire project, or is one for each TU enough)? — Dutow, Jul 25 '16 at 09:43
So you are using a multimap and you want to insert an arbitrary number of values at once for the key `code`. Is that what you want? — Rerito, Jul 25 '16 at 09:46
If the "previous" CODEMAP is passed into every call to `foo` and a new CODEMAP returned, then the problem reduces to implementing a map data structure in a functional programming language. But it looks as if you're trying to implement a mutable global in a programming language (namely TMP) that only has immutable values... — Steve Jessop, Jul 25 '16 at 09:47
@Rerito No. I don't want a multimap. I don't specifically want the values of `args...` but rather the typepack `Args...`. — themagicalyang, Jul 25 '16 at 09:52
Oh I see, you want the value of the map to be types. Well may we know why? That sounds like an XY problem — Rerito, Jul 25 '16 at 09:55
@Rerito The number of instances foo() would have is numerous. It's tedious to declare a struct/tuple manually for each instance of foo(). If I have some code->typepack information for all the types of code for which foo() was called, I can use this information in the reader block R which uses code information to parse out data from a buffer. — themagicalyang, Jul 25 '16 at 10:04
So in other words, you want to deserialize data from buffers into tuples/struct — Rerito, Jul 25 '16 at 10:45
Without explicitly creating tuples/struct. Since I am already instantiating foo for every type of message, shouldn't that information suffice for providing run time with information of the types of messages involved? — themagicalyang, Jul 25 '16 at 10:47
Assuming the given type is constructible with `Args... args` you could simply forward them to that constructor and do the job with the created object. What do you want to do with objects that your `foo` template will create? — Rerito, Jul 25 '16 at 10:58
@themagicalyang Could you provide us with a complete example of the chain (with a message type, etc...) so that we can see better? — Rerito, Jul 25 '16 at 13:04
@SteveJessop I don't want Producer to worry about the types of data it is writing to queue, however the consumer thread should be aware of the type of data written based on the code. So, I am now trying with encoding the type info at run time. — themagicalyang, Jul 26 '16 at 06:02
See this [answer](http://stackoverflow.com/a/28580813/4182275), I guess the problem is similar. In you case we store the data in the queue as a pointer to raw values, store the function in some map where key defines what function to call and what arguments it should have and when message arrives we parse it (get the key and then get the values because we know what arguments function with this key has) and do the invocation. Have I understood your problem right? — Nikolay K, Jul 26 '16 at 06:27
Seems quite delicate: you are at the frontier between static and dynamic polymorphism. You can't have all the good sides of both at the same time! — Rerito, Jul 26 '16 at 08:43

Rerito · Answer 1 · 2016-07-27T07:07:28.007

Instead of having a map, you can template a struct over code like the following:

enum Codes {
    BEGIN_CODE = 0,
    Code1,
    Code2,
    NB_CODES
};

template <typename ... Args>
struct param_pack {
    // Alternatively you could also use std::tuple?!
};

template <Code code>
struct code_info;

// You still have to manually define this at some point...
// For all your different messages...
template <>
struct code_info<Code1> {
    typedef param_pack<int, double, double> args_t;
};

template <>
struct code_info<Code2> {
    typedef param_pack<int, float, float, long> args_t;
}

First step checked, we have the type information somewhere for our different message codes. Now, how do we process them using that information? It's time for some template magic:

namespace details {
template <typename ArgPack>
struct pack_processor;

template <typename T, typename ... Args>
struct pack_processor<param_pack<T, Args...>> {
    static void process_pack(YourQueue& queue) {
        process<T>(queue.getData<T>());
        pack_processor<param_pack<Args...>>::process_pack(queue);
    }
};

template <typename T>
struct pack_processor<param_pack<T>> {
    static void process_pack(YourQueue& queue) {
        process<T>(queue.getData<T>());
    }
};
} // namespace details

template <Code code>
process_message(YourQueue& queue) {
    details::pack_processor<typename code_info<code>::args_t>::process_pack(queue);
}

Then you can add another templates to find the relevant processing steps to apply on your queue depending on the message's code... To do so we must "cheat" a little bit: since we can only have the desired code at runtime, we can't branch the processing immediately, we need to use the "template switch" trick. This is illustrated below:

namespace details {
// This is not static:
// you can't have static polymorphism when you get information from runtime...
template <Code ref_code>
void process_data_with_code(Code code, YourQueue& queue) {
    // We found the good code:
    if (code == ref_code) {
        // We retrieve the static information
        // and run the appropriate process_pack specialization -> this is static
        process_message<ref_code>(queue);
    } else {
        process_data_with_code<static_cast<Code>(ref_code-1)>(code, queue);
    }
}

template <>
void process_data_for_code<BEGIN_CODE>(Code code, YourQueue& queue) {
    std::cout << "Code not found..." << std::endl;
}

} // namespace details

void process_data(Code code, YourQueue& queue) {
    process_data_for_code<static_cast<Code>(NB_CODE-1)>(code, queue);
}

You can find a running example on Coliru with a dummy YourQueue and process() implementation.

This solves the consumer part. You can solve the producer part similarly by adding the relevant methods inside pack_processor specializations and a generic writeToQueue method which will use the same kind of Template Switch trick we just saw.

It's good. Solves most except I didn't want to manually define the different messages. I have done a different approach using polymorphism, a bit finnicky but it works. — themagicalyang, Jul 27 '16 at 08:05
Well, if you use static polymorphism (IOW template tricks), you need to have everything constant and available to your producer and consumer. This requires a manual definition of the messages types at some point. — Rerito, Jul 27 '16 at 08:08

themagicalyang · Accepted Answer · 2016-07-29T12:20:21.117

0

So, I tried using polymorphism. It seems to work by storing the derived formatted message to the queue. While de-queuing, the vptr should point to correct implementation of process().

class Message {
   virtual void process() = 0;
}

template <typename... Args>
class FormattedMessage : public Message {
   std::tuple<Args...> data;
   //Specialize process function for each formatted message.
   void process() {
      //As now I have the tuple, I can easily iterate/process it.
   }
}

Producer:

template <typename ...Args>
void Producer::writeToQueue(Args... args) {
    using fmttype = FormattedMessage<Args...>;
    this->queue.push<fmttype>(args...);
}

Consumer:

void Consumer::readfromQueue() {
    while(true) {
      if (queue.dataAvailable()) {
          this->queue.template front<Message>().process();
          this->queue.pop(this->queue.template front<Message>().size());
      }  
    }
 }

edited Jul 29 '16 at 12:20

answered Jul 27 '16 at 08:13

themagicalyang

2,493
14
21

*`formatted message`* - this new detail maybe should better be added to the question. In your answer I don't see a *`derived`* class, did you forget something? – Wolf Jul 27 '16 at 11:00
Edited. `FormatteMessage` is just a holder class so that the overriding process function specific to argpack `Args...` can have a unique definition. Virtual pointer solves my problem of mapping `code` and `Args...`, i.e, a unque process function (which handles the a specific argpack `Args...` ) – themagicalyang Jul 29 '16 at 12:24

Create a compile time key-to-type map which is filled by calls to a variadic function

2 Answers2

Linked