I can have a definition like this in a header file? 
Yes
Is there any problem in having this in a header file that would be included to several cpp files?
No
A constexpr variable (int, double, etc) in do not occupy memory, thus it does not have memory address and compiler handles it like #define, it replaces variables with value. This is not true for objects though, that is completely different. Read this:
To elaborate on comments made. To avoid overhead, in most cases constexpr is replaced with its value, but in cases where you have to get an address of constexpr compiler does allocate memory each time. So if you have ab.h which contains:
constexpr double PI = 3.14;
and you have a.cpp which contains:
std::cout << PI << "\n";
PI would be replaced no memory would be allocated.
Where as if you have b.cpp:
double *MY_PI = &PI;
memory would be allocated specifically for that instance (or maybe for entire b.cpp file).
EDIT:
Thanks to @HolyBlackCat and his code he had in comments bellow it seems that memory is allocated per file. 
EDIT 2:
it is file based. So I have constExpr.h containing follwoing:
#ifndef CONSTEXPR_H
#define CONSTEXPR_H
#include <iostream>
constexpr int a = 5;
void bb ();
void cc ();
#endif
a.cpp containing follwing:
#include <iostream>
#include "constExpr.h"
void aa () {
    std::cout << &a << "\n";
}
int main () {
    aa ();
    bb ();
    cc ();
    return 0;                                                                                                                 
}
and b.cpp containing following:
#include "constExpr.h"
void bb () {
    std::cout << &a << "\n";
}
void cc () {                                                                                                  
    std::cout << &a << "\n";
}
output is:
0x400930
0x400928
0x400928
CONCLUSION
But, honestly I would never do something like I did in my examples. This was a great challenge for me and my brain. constexpr is added mostly to replace #define. As we know #define is hard to debug due to the fact that compiler cannot check #define statements for error. So unless you do something like above it is just like #define except it is handled at compile time not by precomiler.