Hiya there are few other issues like:
Working demo Smooth: http://jsfiddle.net/wWxVL/2/
- The left arrow keys don't work.
- The stuff is bit quirky when user type. 
- Tab/select all issue (known)
An awesome post here: How to allow only numeric (0-9) in HTML  inputbox using jQuery?
Hope it helps, cheers
code
$(document).ready(function() {
    $(".numbersOnly").keydown(function(event) {
        // Allow: backspace, delete, tab, escape, and enter
        if (event.keyCode == 46 || event.keyCode == 8 || event.keyCode == 9 || event.keyCode == 27 || event.keyCode == 13 ||
        // Allow: Ctrl+A
        (event.keyCode == 65 && event.ctrlKey === true) ||
        // Allow: home, end, left, right
        (event.keyCode >= 35 && event.keyCode <= 39)) {
            // let it happen, don't do anything
            return;
        }
        else {
            // Ensure that it is a number and stop the keypress
            if (event.shiftKey || (event.keyCode < 48 || event.keyCode > 57) && (event.keyCode < 96 || event.keyCode > 105)) {
                event.preventDefault();
            }
        }
    });
});