std::free is inherited from the C standard library. C does not have overloading, so a const overload could not have been inherited.
While the C++ standard library has extended the inherited C library with some useful overloads, a const overload has not been added for free.
Either such an overload has never been considered, or it has not been considered necessary enough to warrant its addition to the standard. I have not come across any openly available proposal for such an addition -- although I haven't read all proposals there have ever been, nor have I attended any committee meetings or workshops, so I cannot deny the possibility of the existence of such a document.
To work around the lack of such an overload, you can indeed use const_cast. In this context, it is completely safe.