(simplistic) explanation:
- A virtual function works because each class maintains a table of addresses of the virtual functions for that class.  
- A template function (or method) is not code, it's instructions on how to generate code. 
so this:
template <std::size_t N>
void handle(const std::array<char, N>& msg, std::vector<boost::asio::const_buffer>& buffers)
{
   // stuff that looks like code
}
does not define a method. It tells the compiler how to write a different method (function) for each and any value of N.
in which case:
template <>
void handle(const std::array<char, 10>& msg, std::vector<boost::asio::const_buffer>& buffers)
and 
template <>
void handle(const std::array<char, 20>& msg, std::vector<boost::asio::const_buffer>& buffers)
are actually different functions (that happen to behave similarly). Because they are different functions, they will occupy different addresses. 
Therefore in order for them to be virtual, there must be 2 entries in the virtual function table.
This, for example, would work:
class Handler:
    {
    public:
        virtual void handle(const std::array<char, 10>& msg, std::vector<boost::asio::const_buffer>& buffers) = 0;
        virtual void handle(const std::array<char, 20>& msg, std::vector<boost::asio::const_buffer>& buffers) = 0;
        // and so on for every value on N you need.
    };