0

I am trying to learn the basic of how numbers are stored in computer memory. Hence I've picked up C++ recently. I wrote this following function, to calculate the square of any int value.

 // since integer can go upto 65K value, hence the return
 // value is double to accomodate the square of highest int value.
 double square(int val) {
     return val*val;
 }

Why is it giving me a wrong answer for 65535, which as per my understanding is the largest number that can be stored in the variable of type int? I've also discovered that there are things like:

std::numeric_limits<int>::max()

which gives me 2147483647. This in turn confused me even more, but I am seeking an explanation for why my square function is failing.

Edit

For 65535 my g++ compiler is giving me: -131071.

Rajat Saxena
  • 3,834
  • 5
  • 45
  • 63
  • The largest value stored in an int would really depend on your compiler and the number of bytes an int is, if you have an unsigned int, that is 2 bytes(16 bits) then 65535 would be the largest value. In this case an int appears to be signed and 4 bytes. – AresCaelum Feb 21 '18 at 17:40
  • 1
    A 16 bit `int` cannot hold 65535 as the largest value. The largest would be 32767 since `int` is signed. You would want `unsigned int` for 65535 max value if your word is 16 bits. Or is your `int` 32 bit? It's unclear from your question. Also, when you say, *wrong answers*, exactly what answers are you seeing that you believe are wrong? This post is overall very unclear. – lurker Feb 21 '18 at 17:41
  • You are making an incorrect assumption about the maximum size of `int`. It can vary between platforms/compilers. All the C++ standard gives you is a guaranteed *minimum* size. – Jesper Juhl Feb 21 '18 at 17:42
  • See [cppreference.com - Integer types](http://en.cppreference.com/w/cpp/language/types#Integer_types) for info on minimum integer sizes. – Olaf Dietsche Feb 21 '18 at 17:42
  • 4
    Unless you cast to `double` *before* the multiplication, you're still performing signed integer multiplication, whose behavior is undefined in the case of overflow. You're just casting the final result to `double`, which is not helpful. – Silvio Mayolo Feb 21 '18 at 17:44
  • Look up the terminology called truncation. – Francis Cugler Feb 21 '18 at 18:16

1 Answers1

5

You have signed integer overflow here which is undefined behavior. You say std::numeric_limits<int>::max() gives 2147483647 and since 65535 is less then that you feel you should be okay. The problem is you are not. When you multiply to int's together the result is stored in a int (even if that is not what you are assigning it to). Since 65535 * 65535 is 4294836225 it overflows the result.

The only way to get around this is to use a wider/bigger type that can hold the result. For instance if you multiply two long long's you can do 65535 * 65535 and be okay.

As long at least one of the variables' type in the multiplication is big enough to store the result then you are okay. So in this case both

val * static_cast<double>(val)

and 

val * static_cast<long long>(val)

Will give you the correct result. You could also just change val to be a double and that will also solve the problem.

NathanOliver
  • 171,901
  • 28
  • 288
  • 402