QtConcurrent blockingMappedReduced error

Question

This is my first attempt at using QtConcurrent::blockingMappedReduced, and I can't get it to build in MSVC 2010 Express (with QT 4.7.1 source code).

I have created a small example that is similar to my actual code, and it has the same problem building:

// Here's the general outline:
// 1. create a list of random numbers
// 2. pass the list to blockingMappedReduced
// 3. the map function calculates the sine of the given random number
// 4. the reduce function finds the random number with the maximum sine value

// Here's the implementation:

#include "stdafx.h"
#include <qlist.h>
#include <qtconcurrentmap.h>

// My class for the map/reduce functions
class myClass
{
private:

    // Nested class to hold the intermediate results from the map function
    // I think I need this because the reduce function needs more from the map function than a single return value
    class Temp
    {
    public:

        // For example, let's pass these two member variables from the map function to the reduce function
        int randomInput;
        double resultingOutput;

        // The Temp constructor
        Temp::Temp(double randomInput, double resultingOutput)
        {
            this->randomInput = randomInput;
            this->resultingOutput = resultingOutput;
        }
    };

public:

    // For example, these myClass members will hold the final result from the reduce function
    double maximumOutput;
    double maximumInput;

    // The myClass constructor
    myClass::myClass()
    {
        this->maximumOutput = -1;
    }

    // The map function
    const Temp mapFunction(const double& randomInput)
    {
        // For example, let's calculate the sine of the given random number
        double resultingOutput = sin(randomInput);

        // Construct the temporary structure to pass multiple values to the reduce function
        const Temp temp(randomInput, resultingOutput);
        return(temp);
    }

    // The reduce function
    void reduceFunction(double& maxInput, const Temp& temp)
    {
        // For example, let's find the maximum computed sine value
        if (temp.resultingOutput > this->maximumOutput)
        {
            this->maximumOutput = temp.resultingOutput;
            this->maximumInput = temp.randomInput;
        }

        maxInput = this->maximumInput;
    }
};

// Main function
void main(int argc, _TCHAR* argv[])
{
    // Build a list of random numbers
    QList<int> aList;
    for (int count = 8; count > 0; --count)
    {
        aList.append(rand());
    }

    // Invoke the parallel map/reduce function
    myClass myClassInstance;
    double theAnswer = QtConcurrent::blockingMappedReduced(aList, &myClass::mapFunction, &myClass::reduceFunction);
}

The compiler complains about the last line, where blockingMappedReduced is invoked.

c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2780: 'D QtConcurrent::blockingMappedReduced(Iterator,Iterator,T (__thiscall C::* )(void) const,U (__thiscall D::* )(V),QtConcurrent::ReduceOptions)' : expects 5 arguments - 3 provided
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(659) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2780: 'C QtConcurrent::blockingMappedReduced(Iterator,Iterator,T (__cdecl *)(U),V (__thiscall C::* )(W),QtConcurrent::ReduceOptions)' : expects 5 arguments - 3 provided
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(643) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2780: 'V QtConcurrent::blockingMappedReduced(Iterator,Iterator,T (__thiscall C::* )(void) const,U (__cdecl *)(V &,W),QtConcurrent::ReduceOptions)' : expects 5 arguments - 3 provided
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(627) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2780: 'W QtConcurrent::blockingMappedReduced(Iterator,Iterator,T (__cdecl *)(U),V (__cdecl *)(W &,X),QtConcurrent::ReduceOptions)' : expects 5 arguments - 3 provided
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(611) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2780: 'ResultType QtConcurrent::blockingMappedReduced(Iterator,Iterator,T (__thiscall C::* )(void) const,ReduceFunctor,QtConcurrent::ReduceOptions)' : expects 5 arguments - 3 provided
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(595) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2780: 'ResultType QtConcurrent::blockingMappedReduced(Iterator,Iterator,T (__cdecl *)(U),ReduceFunctor,QtConcurrent::ReduceOptions)' : expects 5 arguments - 3 provided
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(579) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2780: 'C QtConcurrent::blockingMappedReduced(Iterator,Iterator,MapFunctor,T (__thiscall C::* )(U),QtConcurrent::ReduceOptions)' : expects 5 arguments - 3 provided
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(563) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2780: 'U QtConcurrent::blockingMappedReduced(Iterator,Iterator,MapFunctor,T (__cdecl *)(U &,V),QtConcurrent::ReduceOptions)' : expects 5 arguments - 3 provided
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(547) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2780: 'ResultType QtConcurrent::blockingMappedReduced(Iterator,Iterator,MapFunctor,ReduceFunctor,QtConcurrent::ReduceOptions)' : expects 5 arguments - 3 provided
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(536) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2784: 'D QtConcurrent::blockingMappedReduced(const Sequence &,T (__thiscall C::* )(void) const,U (__thiscall D::* )(V),QtConcurrent::ReduceOptions)' : could not deduce template argument for 'T (__thiscall C::* )(void) const' from 'const myClass::Temp (__thiscall myClass::* )(const double &)'
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(522) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2784: 'C QtConcurrent::blockingMappedReduced(const Sequence &,T (__cdecl *)(U),V (__thiscall C::* )(W),QtConcurrent::ReduceOptions)' : could not deduce template argument for 'T (__cdecl *)(U)' from 'const myClass::Temp (__thiscall myClass::* )(const double &)'
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(508) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2784: 'V QtConcurrent::blockingMappedReduced(const Sequence &,T (__thiscall C::* )(void) const,U (__cdecl *)(V &,W),QtConcurrent::ReduceOptions)' : could not deduce template argument for 'T (__thiscall C::* )(void) const' from 'const myClass::Temp (__thiscall myClass::* )(const double &)'
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(494) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2784: 'W QtConcurrent::blockingMappedReduced(const Sequence &,T (__cdecl *)(U),V (__cdecl *)(W &,X),QtConcurrent::ReduceOptions)' : could not deduce template argument for 'T (__cdecl *)(U)' from 'const myClass::Temp (__thiscall myClass::* )(const double &)'
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(480) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2784: 'ResultType QtConcurrent::blockingMappedReduced(const Sequence &,T (__thiscall C::* )(void) const,ReduceFunctor,QtConcurrent::ReduceOptions)' : could not deduce template argument for 'T (__thiscall C::* )(void) const' from 'const myClass::Temp (__thiscall myClass::* )(const double &)'
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(466) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2784: 'ResultType QtConcurrent::blockingMappedReduced(const Sequence &,T (__cdecl *)(U),ReduceFunctor,QtConcurrent::ReduceOptions)' : could not deduce template argument for 'T (__cdecl *)(U)' from 'const myClass::Temp (__thiscall myClass::* )(const double &)'
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(452) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2784: 'C QtConcurrent::blockingMappedReduced(const Sequence &,MapFunctor,T (__thiscall C::* )(U),QtConcurrent::ReduceOptions)' : could not deduce template argument for 'T (__thiscall C::* )(U)' from 'void (__thiscall myClass::* )(double &,const myClass::Temp &)'
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(438) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2784: 'U QtConcurrent::blockingMappedReduced(const Sequence &,MapFunctor,T (__cdecl *)(U &,V),QtConcurrent::ReduceOptions)' : could not deduce template argument for 'T (__cdecl *)(U &,V)' from 'void (__thiscall myClass::* )(double &,const myClass::Temp &)'
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(424) : see declaration of 'QtConcurrent::blockingMappedReduced'
c:\gwa\tmp\stackoverflow\stackoverflow\stackoverflow.cpp(77): error C2783: 'ResultType QtConcurrent::blockingMappedReduced(const Sequence &,MapFunctor,ReduceFunctor,QtConcurrent::ReduceOptions)' : could not deduce template argument for 'ResultType'
          c:\gwa\third_party\qt\qt-4.7.1\src\corelib\concurrent\qtconcurrentmap.h(414) : see declaration of 'QtConcurrent::blockingMappedReduced'

From the online QT documentation (http://doc.qt.nokia.com/4.7-snapshot/qtconcurrentmap.html#blockingMappedReduced), here's the prototype:

T  blockingMappedReduced ( const Sequence & sequence, MapFunction mapFunction, ReduceFunction reduceFunction, QtConcurrent::ReduceOptions reduceOptions = UnorderedReduce | SequentialReduce )

I'm afraid I don't have the expertise to debug this. Any help would be greatly appreciated. Again, since my C++ skill is not expert, the help would need to be explicit in order for me to understand it (i.e. actual code snippets, not something that presumes more knowledge than I have, like "your argument must be a constant reference").

Thanks in advance for your help.

score 2 · Answer 1 · edited May 23 '17 at 12:23

Your code has a few general issues to patch up that aren't QtConcurrent related. Firstly, you shouldn't qualify the constructors, so instead of:

    // The Temp constructor
    Temp::Temp(double randomInput, double resultingOutput)

You should write simply:

    // The Temp constructor
    Temp(double randomInput, double resultingOutput)

Same for myClass. Also, your function signature for main should always return an int although in C99 and C++ you can omit the return statement if you want to return 0. (I, however, always explicitly put return 0 as a matter of personal preference.)

What should main() return in C and C++?

As for your QtConcurrent issues, there are a couple. First your QList is of integers, and the documentation points out that the type in the sequence should match the first parameter of the map function (where you have a double). So the QList needs to change to double -or- your map function needs to take an int.

Another problem you have is the fact that you are trying to pass pointers to member functions into slots that expect just plain-ol functions:

    // Invoke the parallel map/reduce function
    myClass myClassInstance;
    double theAnswer = QtConcurrent::blockingMappedReduced(aList,
        &myClass::mapFunction, &myClass::reduceFunction);

Note that there is no mention of myClassInstance besides the declaration. In the general case...working around this is not as easy as changing the call to use &myClassInstance::mapFunction, &myClassInstance::reduceFunction. As you look around at C++ you will find there's no generally workable way to make objects that can fit in any slot a function call can, even if they're designed to appear to be callable like functions:

Help with boost bind/functions

Luckily, QtConcurrent doesn't paint you into a corner on this. It anticipated the need for function objects:

http://doc.qt.io/archives/qt-4.7/qtconcurrentmap.html#using-function-objects

But rather than digging into how to make that work, we should consider why you would want these as member functions in the first place. The Map/Reduce model lets you define custom accumulator and result types, and that can get you what you want without that trouble. Here's a simpler version that hopefully will compile and get you closer:

#include <qlist.h>
#include <qtconcurrentmap.h>

#include <cmath> // for sin()
#include <algorithm> // for min, max, etc.
#include <limits> // for numeric_limits

struct MapResult // same as class, but "public" by default
{
    double input;
    double output;

    // must be "default constructible" due to guts of QtConcurrent :-/    
    // implementation artifact, don't worry about it affecting you...
    MapResult () {}

    MapResult (double input, double output) :
        // initializing members like this is cleaner than this->x = x...
        // and for certain members you HAVE to do it this way
        input (input), output (output)
    {
    }
};

struct ReduceAccumulator // same as class, but "public" by default
{
    MapResult largest;

    ReduceAccumulator () :
        // a better choice than 0 for an "uninitialized max" since
        // any other double will be larger...but you really should
        // probably accomodate a "no entries" case
        largest (MapResult (0, std::numeric_limits<double>::min())
    {
    }
};

// pattern of return type should be "U", not "const U" according to docs
MapResult mapFunction(const double& input) 
{
    // no need to construct a named local var if you don't want to
    // also no parentheses needed on return(x); ... return x; is fine 
    return MapResult (input, sin(input));
}

void reduceFunction(ReduceAccumulator& acc, const MapResult& oneResult)
{
    if (oneResult.output > acc.largest.output) {
        acc.largest = oneResult
    }
}

int main(int argc, char* argv[])
{
    QList<double> aList;
    for (int count = 8; count > 0; --count)
    {
        aList.append(rand());
    }

    ReduceAccumulator answer = QtConcurrent::blockingMappedReduced(
        aList, &mapFunction, &reduceFunction);

    return 0;
}

Included some general C++ notes, hopefully helpful...

UPDATE: Another route they offer does use member functions, but it makes a couple of different assumptions from the style you were trying to write. Firstly, it assumes that your sequence input type is a class (double is a "POD" or "plain-old-data" type and does not have method dispatch). Then the parameter is implicit.

But note that if you wanted to use this technique it would stylize the code in a particular way. Here are the changes:

struct ReduceAccumulator
{
    MapResult largest;

    ReduceAccumulator () :
        largest (MapResult(0, std::numeric_limits<double>::min())
    {
    }

    // runs on non-const object, and drops first parameter (now implicit)
    void reduceFunction(const MapResult& oneResult)
    {
        if (oneResult.output > largest.output) {
            largest = oneResult
        }
    }
};

struct MyDouble {
    double d;

    MyDouble (double d) : d (d) {}

    // as member it takes no arguments, and needs to run on const objects
    MapResult mapFunction() const 
    {
        return MapResult (d, sin(d));
    }
};

int main(int argc, char* argv[])
{
    // to use a member as mapFunction, must be a member of input sequence type
    QList<MyDouble> aList;
    for (int count = 8; count > 0; --count)
    {
        aList.append(MyDouble (rand()));
    }

    ReduceAccumulator theAnswer = QtConcurrent::blockingMappedReduced(
        aList, &MyDouble::mapFunction, &ReduceAccumulator::reduceFunction);
}

You can pick and choose...in this case it's probably better to do the reduceFunction as a member. But kind of ugly to be changing the input sequence type, and that's not the right place for parameters to the process--you want a function object.

(Also bear in mind that a writable state that the map function can get to undermines the parallelism of MapReduce. The map operations are supposed to really only generate output via their result so they stay independent...)

HostileFork: Thanks for your solution. Aside from the general C++ clean up, it seems the following were the issues related to blockingMappedReduced: 1) the map and reduce functions could not be member functions, but had to be global functions, 2) my intermediate class to pass values from the map function to the reduce function needed a default constructor, 3) the type of the map function is not const. Follow-on question: the QT doc gives an example of using member functions with map and reduce for QtConcurrent::mappedReduced. But this does not work for blockingMappedReduced? — user1020872, Nov 01 '11 at 02:40
To answer my own follow-on question, when using member functions, apparently they must be static. — user1020872, Nov 01 '11 at 02:44
Can I ask you some questions regarding your C++ clean up of my code? 1) why is it preferrable to use struct rather than class for the intermediate objects? 2) why is it preferrable to not have a nested class for the intermediate objects? My motivation is just that myClass in my actual code is a class that encapsulates many other things, not just the simple things that I showed in the example. — user1020872, Nov 01 '11 at 02:53
It's no different for blockingMappedReduced. But I updated the answer to point out that using the member functions forces certain changes to the types from how you were expecting it to work, and you probably don't want to do that for the input. — HostileFork says dont trust SE, Nov 01 '11 at 03:18
RE struct: It's not so much preferable other than it let me shorten the sample code and I wanted to explain it since you were new. Plus if you *actually* have a type that you don't use the typical object oriented accessor pattern, then labeling it a struct makes it more like "I meant to do that". RE nesting: If it fits your case, use it...but when doing Q&A on code examples it's best to remove anything superfluous: http://sscce.org/ — HostileFork says dont trust SE, Nov 01 '11 at 03:22
Thanks for explaining the restructuring. Regarding the coupling between threads: the threads are still independent, it's just they need to return more than a single value. The reduce function is selecting the "best" element, and I need both the element and the value it produced. I imagine this is not an uncommon scenario. Actually the thing I worried about more is that a single object is shared across multiple reduce calls. The guarantee that reduce is called one at a time makes it okay, but is there a better design pattern that should be used? — user1020872, Nov 01 '11 at 03:57
Nothing wrong with returning an object from your map function to let the reduce function know what input produced what map result. It does make things a little less "generically applicable"...so your map and reduce functions are probably not going to be the sorts of things you'd use somewhere else in the program that isn't MapReduce-oriented. As for sharing an object across reduce calls, that's the way MapReduce works...no problem. — HostileFork says dont trust SE, Nov 01 '11 at 04:17

user1020872 · Answer 2 · 2011-11-01T16:32:15.863

HostileFork: thanks again for all your help. You got me back on track to making my real code work.

Just for completeness, here's my original example with the minimal changes needed to make blockingMappedReduced work (but still with my messy C++ style issues). This is how I worked through your solution to come to an understanding of what's what. It might help someone else sort through which are the blockingMappedReduced problems and which are the C++ problems:

#include "stdafx.h"
#include <qlist.h>
#include <qtconcurrentmap.h>


// My class for the map/reduce functions
class myClass
{
    // Nested class to hold the intermediate results from the map function
    // I think I need this because the reduce function needs more from the map function than a single return value
    class Temp
    {
    public:

        // For example, let's pass these two member variables from the map function to the reduce function
        int randomInput;
        double resultingOutput;

        // default Temp constructor
        Temp()
        {
        };

        // The Temp constructor
        Temp(double randomInput, double resultingOutput)
        {
            this->randomInput = randomInput;
            this->resultingOutput = resultingOutput;
        }
    };

public:

    // For example, these myClass members will hold the final result from the reduce function
    double maximumOutput;
    double maximumInput;

    // The myClass constructor
    myClass()
    {
        this->maximumOutput = -1;
    }

    // The map function
    static Temp myClass::mapFunction(const int& randomInput)
    {
        // For example, let's calculate the sine of the given random number
        double resultingOutput = sin((double)randomInput);

        // Construct the temporary structure to pass multiple values to the reduce function
        Temp temp(randomInput, resultingOutput);
        return(temp);
    }

    // The reduce function
    static void myClass::reduceFunction(myClass& accumulator, const Temp& temp)
    {
        // For example, let's find the maximum computed sine value
        if (temp.resultingOutput > accumulator.maximumOutput)
        {
            accumulator.maximumOutput = temp.resultingOutput;
            accumulator.maximumInput = temp.randomInput;
        }
    }
};


// Main function
int main(int argc, _TCHAR* argv[])
{
    // Build a list of random numbers
    QList<int> aList;
    for (int count = 8; count > 0; --count)
    {
        aList.append(rand());
    }

    // Invoke the parallel map/reduce function
    myClass theAnswer = QtConcurrent::blockingMappedReduced(aList, &myClass::mapFunction, &myClass::reduceFunction);

    return(0);
}

Ah, at first I hadn't realized that you wanted the maximum result input (not finding the maximum input and output). While you could rename variables to try and make that more obvious, do note that if you have a MapResult type already declared then you can re-use that type for this...I changed my code to do it that way. Also note changes to the constructor (`Temp::Temp` => `Temp`) and return type of main are not superfluous. Whether MSVC 2010 is willing to compile that for you or not, it's incorrect C++ and will not compile on standards-compliant compilers. — HostileFork says dont trust SE, Nov 01 '11 at 15:00
Thanks again for the advice. In my actual code, I don't have everything in one file, the constructor prototypes are in a header file and the implementations are in a cpp file, and the prototypes are as you advise. And my actual main function does indeed return an int status. But my example could confuse someone else reading it, so I will edit it. — user1020872, Nov 01 '11 at 16:37

QtConcurrent blockingMappedReduced error

2 Answers2