Received unexpected output in MRUnit

Question

I'm getting the following MRUnit error:

ERROR mrunit.TestDriver: Received unexpected output (60, mrdp.MyCustomClass@73207f36)

ERROR mrunit.TestDriver: Missing expected output (60, mrdp.MyCustomClass@6f73cf45) at position 0

I've created a MyCustomClass which implements Writable, and has 4 int attributes. This is the output Value of my Mapper.

The following is the code of the mapper's MRUnit test:

@Test
public void testMapper() throws IOException {
    MyCustomClass result = new MyCustomClass();
    result.setAttr1(1);
    result.setAttr2(0);
    result.setAttr3(0);
    result.setAttr4(0);

    mapDriver.withInput(new LongWritable(1), new Text("60,5596,1,256"));
    mapDriver.addOutput(new Text("60"), result);
    mapDriver.runTest();
}

My Mapper should call its setter setAttr1(1), when locating the "1" in the new Text("60,5596,1,256") here above.

How can I test for this result with a custom class (with several attributes)? The job execution is successful, I just don't know how to make the MRUnit test work.

$ hadoop fs -cat patterns/minmaxcount/outuserprefs/part*
23  mrdp.MyCustomClass@4cf15f6c
60  mrdp.MyCustomClass@4cf15f6c

Answer 1

You need to override equals() and hascode() for your custom classes if you want to test for equality. If you don't, there's no way to test for "semantic equality". The default Object methods will be used. That is what you are facing. For further discussion, see Why do I need to override the equals and hashCode methods in Java?

Below a simple JUnit test using a custom class CustomClass. I commented out the the equals and hashcode. If you run the test, it will fail, with a similar message to what you are receiving. If you remove the comments and run it, it will pass.

import static org.junit.Assert.*;
import org.junit.Test;

public class CustomClass {

    String firstName;
    String lastName;

    public void setFirstName(String firstName) { this.firstName = firstName; }
    public void setLastName(String lastName) { this.lastName = lastName; }

    @Test
    public void testEqaulity() {
        CustomClass clazz1 = new CustomClass();
        clazz1.setFirstName("Stack");
        clazz1.setLastName("Overflow");

        CustomClass clazz2 = new CustomClass();
        clazz2.setFirstName("Stack");
        clazz2.setLastName("Overflow");

        assertEquals(clazz1, clazz2);
    }

    /*
    @Override
    public int hashCode() {
        final int prime = 31;
        int result = 1;
        result = prime * result
                + ((firstName == null) ? 0 : firstName.hashCode());
        result = prime * result
                + ((lastName == null) ? 0 : lastName.hashCode());
        return result;
    }
    @Override
    public boolean equals(Object obj) {
        if (this == obj)
            return true;
        if (obj == null)
            return false;
        if (getClass() != obj.getClass())
            return false;
        CustomClass other = (CustomClass) obj;
        if (firstName == null) {
            if (other.firstName != null)
                return false;
        } else if (!firstName.equals(other.firstName))
            return false;
        if (lastName == null) {
            if (other.lastName != null)
                return false;
        } else if (!lastName.equals(other.lastName))
            return false;
        return true;
    }
    */
}

If you don't have experience or knowledge in implementing these methods, most IDE have the option to create them for you.

• Eclipse: right click on class -> Source -> Generate equals and hashcode
• Netbeans: right click on source editor -> Insert Code -> equals() hashcode()

In both cases, you need to select the properties you want to include (check) in the equals and hashcode. These are the only two IDEs I use :-)