I would use std::all_of in combination with a predicate (better if a short, terse lambda) that uses std::isdigit():
bool ok = std::all_of(begin(str), end(str), [] (char c) { return std::isdigit(c); });
This allows you to test any sub-range you wish, and the iteration cycle comes for free.
You have to include the <cctype> header (for std::isdigit()) and the <algorithm> header (for std::all_of).
EDIT:
If you do not want to use a lambda, you can even pass in the std::digit function directly, if you provide a proper cast (see this Q&A for an explanation):
bool ok = std::all_of(begin(str), end(str), (int (*)(int))std::isdigit);
The cast is necessary because in the std namespace, there are overloads of isdigit which is causing problem in overload resolution. The other topic discusses a similar case in a bit detail.
Or if that looks cumbersome, you can do this instead:
int (*predicate)(int) = std::isdigit;
bool ok = std::all_of(begin(str), end(str), predicate);
That should work fine!