The difference is due to two separate facts, both of which are "quirks" of C which can be confusing at first.
The first quirk is that when you have a pointer variable initialization, it's different from an assignment. As you know, when you say
int *p = 123;
this means, "p is a pointer to an int, with an initial value of 123". But what if we defined the variable p on one line, and gave it a value on a second line using an ordinary assignment statement? It's easy to imagine that would look like this:
int *p;
*p = 123; /* WRONG */
But in fact it would look like this:
int *p;
p = 123;
In a pointer declaration/definition with initialization, the * means you're declaring a pointer variable, but it doesn't mean you're initializing the contents of what the pointer points to. In a regular assignment expression, on the other hand, the * is an operator that means, "I am talking about the memory location pointed to by this pointer". So when you say
*p = 123;
that means, "Pointer p points to some memory location, and I want to store the value 123 into that memory location." On the other hand, when you say
p = 123;
that means, "Make pointer p point to some memory location 123." In that case, it might be more clear to say
p = 0x0000007b;
But, of course, unless we're doing some kind of specialized embedded programming, we rarely pick exact numeric values for the memory locations we want to access.
But let's get back to the question you asked. To avoid confusion, let's take your two declarations-with-initializers
int *p = 123;
char *s = "hi";
and change them into separate declarations:
int *p;
char *s;
and assignments:
p = 123;
s = "hi";
As we just saw, the integer pointer assignment p = 123 doesn't do what we want, and it probably doesn't even do anything useful at all. So why does the character pointer assignment
s = "hi";
work?
That has to do with the second quirk of C, which is in two parts. The first part is that, as you probably know, strings in C are represented as arrays of characters. (That's fine, it's hardly even a "quirk".) But then the second part is, whenever you mention an array in an expression and try to take its value, the value you get is a pointer to the array's first element. But what does that mean?
First of all, when you mentioned that string constant "hi" in your program, the compiler automatically created a little array for you. The effect was just as if you had said
char __string_constant = { 'h', 'i', '\0' };
(perhaps that's a third quirk).
And then, when you said
s = "hi";
it was just exactly as if you had written
s = &__string_constant[0];
This has been kind of a long answer, but if you're still with me, the bottom line is that when you're working with pointers, you have to be clear in your mind about the distinction between the pointer and what it points to. When you said
char *s = "hi";
you were taking care of both things: You were making the pointer p point somewhere, and you were arranging that, at the location that it pointed to, there was a string "hi". But when you said
int *p = 123;
you were making the pointer p point to location 123, but you weren't doing anything to affect the value at memory location 123 — and you were certainly not making p point to the integer value 123.
With all of that said, you might still be wondering, is there a one-step way to set an integer pointer to point to an array of integers, like there is for character arrays with s = "hi"`?
Traditionally, there wasn't. But C99 introduced something called an array initializer that's just what you'd want. It looks like this:
int *p = (int[]){ 123 };
That sets up the same sort of situation as with the string literal. The compiler creates an array and sets the pointer to point to it, just as if you had said
int __array initializer[] = { 123 };
int *p = &array initializer[0];
So, in summary, you can say
int *p = (int[]){ 123 };
char *s = "hi";
and, once you've done it this way, there's not so much difference between int pointers and char pointers, after all.
Addendum: It didn't really come up in your question or this answer, but there's one more closely-related C quirk that it's worth mentioning while we're at it. I said, "whenever you mention an array in an expression and try to take its value, the value you get is a pointer to the array's first element." And I said that when you wrote
s = "hi";
it was just as if you had written
char string_constant = { 'h', 'i', '\0' };
s = &string_constant[0];
But it would also work if you wrote
s = string_constant;
That looks wrong at first — how can you take an array like string_constant, and assign it to a pointer like s? Well, it's because, as I said, when you mention an array in an expression and try to take its value, what you get is a pointer to the array's first element. Or, in other words,
s = &string_constant[0];
