UPDATE
I've had a little play and reckon I've come up with something that solves this issue for the original provided code.
Essentially I have assigned the same :after CSS to a "dummy" class and then create and destroy an element with the dummy class on the fly. Between the create and destroy we are able to get the necessary dimensions (width, height, positioning). Finally we can compare these values to the click co-ordinates...
DEMO: http://jsfiddle.net/gvee/gNDCV/6/
CSS
#main {
    background-color: red;
    width: 100%;
    height: 200px;
    position: relative;
}
#main:after, .mainafter {
    position: absolute;
    bottom: -100px;
    right: 50%;
    background-color: blue;
    width: 40%;
    height: 20px;
    content:" ";
}
JQuery
$('#main').click(function (e) {
    var pseudoElement = $('<div></div>').addClass('mainafter');
    $(this).append(pseudoElement);
    var w = pseudoElement.width();
    var h = pseudoElement.height();
    var x = pseudoElement.position().left;
    var y = pseudoElement.position().top;
    pseudoElement.remove();
    
    if (e.pageY - $(this).position().top  > y     &&
        e.pageY - $(this).position().top  < y + h &&
        e.pageX - $(this).position().left > x     &&
        e.pageX - $(this).position().left < x + w
       ) {
        alert('Not red');
    }
});
If statement illustrated:

PageX is the horizontal co-ordinate of the cursor.
X is the co-ordinate of the left-hand edge of the blue box.
W is the width of the blue box.
Therefore we can work out the right-hand edge of the blue box by simple addition: X+W.
The same logic can be applied on the vertical co-ordinates (top=Y, bottom=Y+H).
The if statement in our JQuery above checks if PageX falls between the left and right-hand edges of the blue box and that PageY is between the top and bottom.
i.e. is the cursor in the blue box!
There is a [kludgy] workaround to your provided code (where the :after content is positioned below it's container)...
Work out the co-ordinate of the mouse click and see if that exceeds the height of the #main div...
DEMO: http://jsfiddle.net/gvee/gNDCV/3/
CSS
#main {
    background-color: red;
    width: 100%;
    height: 200px;
    position: relative;
    margin-bottom: 20px; /* to clear content:after */
}
#main:after {
    position: absolute;
    bottom: -20px; /* -(this.height) */
    background-color: blue;
    width: 20px;
    height: 20px;
    content:" ";
}
JQuery
$('#main').click(function (e) {
    if (e.pageY - $(this).offset().top > $(this).height()) {
        alert('Not red');
    }
});